Array jobs with SLURM and Singularity

Singularity containers can be used to package workflows, software, data and libraries. Some of the software packages at vital-it and unibe are housed in singularity images. These are loaded with a different syntax then the pre-exisitng modules.

#!/bin/bash

#SBATCH –mail-user=email@unibe.ch
#SBATCH –mail-type=END,FAIL

#SBATCH –job-name=”bax2bam”

# do 50 jobs labeled 01-50 and run at most 20 at a time
#SBATCH –array=01-50%20

# Runtime and memory
#SBATCH –time=08:00:00 #
#SBATCH –mem-per-cpu=8G # n
#SBATCH –cpus-per-task=1

# Partition
#SBATCH –partition=all

# use the full path to the working directory
#SBATCH –workdir=pacbio_raw

#run the instance of the software

singularity run /software/singularity/containers/PacBioSuite-6.0.0.47841-1.centos7.simg

# use a formatted print statement to pad the numbers to all have the same number of digits. “%02g” pads each number with 0’s to 2 positions

NUM=$(printf “%02g” ${SLURM_ARRAY_TASK_ID})

# the value has to be exported to the singularity instance
export NUM

#execute the command

singularity exec /software/singularity/containers/PacBioSuite-7.0.0.63985-1.centos7.simg bax2bam data/00$NUM.1.bax.h5 data/00$NUM.2.bax.h5 data/00$NUM.3.bax.h5 -o 00$NUM –subread