Package: simple-slurm
Simple SlurmA simple Python wrapper for Slurm with flexibility in mindimportdatetimefromsimple_slurmimportSlurmslurm=Slurm(array=range(3,12),cpus_per_task=15,dependency=dict(after=65541,afterok=34987),gres=['gpu:kepler:2','gpu:tesla:2','mps:400'],ignore_pbs=True,job_name='name',output=f'{Slurm.JOB_ARRAY_MASTER_ID}_{Slurm.JOB_ARRAY_ID}.out',time=datetime.timedelta(days=1,hours=2,minutes=3,seconds=4),)slurm.add_cmd('module load python')slurm.sbatch('python demo.py',Slurm.SLURM_ARRAY_TASK_ID)The above snippet is equivalent to running the following command:sbatch<< EOF#!/bin/sh#SBATCH --array               3-11#SBATCH --cpus-per-task       15#SBATCH --dependency          after:65541,afterok:34987#SBATCH --gres                gpu:kepler:2,gpu:tesla:2,mps:400#SBATCH --ignore-pbs#SBATCH --job-name            name#SBATCH --output              %A_%a.out#SBATCH --time                1-02:03:04module load pythonpython demo.py \$SLURM_ARRAY_TASK_IDEOFGet it using either one of :pipinstallsimple_slurm
condainstall-cconda-forgesimple_slurmContentsIntroductionMany syntaxes availableUsing configuration filesUsing the command lineJob dependenciesAdditional featuresFilename PatternsOutput Environment VariablesIntroductionThesbatchandsruncommands inSlurmallow submitting parallel jobs into a Linux cluster in the form of batch scripts that follow a certain structure.The goal of this library is to provide a simple wrapper for these functions (sbatchandsrun) so that Python code can be used for constructing and launching the aforementioned batch script.Indeed, the generated batch script can be shown by printing theSlurmobject:fromsimple_slurmimportSlurmslurm=Slurm(array=range(3,12),job_name='name')print(slurm)>>#!/bin/sh>>>>#SBATCH --array               3-11>>#SBATCH --job-name            nameThen, the job can be launched with either command:slurm.srun('echo hello!')slurm.sbatch('echo hello!')>>Submittedbatchjob34987While both commands are quite similar,srunwill wait for the job completion, whilesbatchwill launch and disconnect from the jobs.More information can be found inSlurm's Quick Start Guideand inhere.Moreover, multi-line commands can be added usingadd_cmdand reset withreset_cmd.
Thesbatchdirective will calladd_cmdbefore launching the job.slurm.add_cmd('echo hello for the first time!')slurm.add_cmd('echo hello for the second time!')slurm.sbatch('echo hello for the last time!')slurm.reset_cmd()slurm.sbatch('echo hello again!')This results in two outputshello for the first time!
hello for the second time!
hello for the last time!hello again!Many syntaxes availableslurm=Slurm('-a','3-11')slurm=Slurm('--array','3-11')slurm=Slurm('array','3-11')slurm=Slurm(array='3-11')slurm=Slurm(array=range(3,12))slurm.add_arguments(array=range(3,12))slurm.set_array(range(3,12))All these arguments are equivalent!
It's up to you to choose the one(s) that best suits you needs."With great flexibility comes great responsability"You can either keep a command-line-like syntax or a more Python-like oneslurm=Slurm()slurm.set_dependency('after:65541,afterok:34987')slurm.set_dependency(['after:65541','afterok:34987'])slurm.set_dependency(dict(after=65541,afterok=34987))All the possible arguments have their own setter methods
(ex.set_array,set_dependency,set_job_name).Please note that hyphenated arguments, such as--job-name, need to be underscored
(so to comply with Python syntax and be coherent).slurm=Slurm('--job_name','name')slurm=Slurm(job_name='name')# slurm = Slurm('--job-name', 'name')  # NOT VALID# slurm = Slurm(job-name='name')       # NOT VALIDMoreover, boolean arguments such as--contiguous,--ignore_pbsor--overcommitcan be activated withTrueor an empty string.slurm=Slurm('--contiguous',True)slurm.add_arguments(ignore_pbs='')slurm.set_wait(False)print(slurm)#!/bin/sh#SBATCH --contiguous#SBATCH --ignore-pbsUsing configuration filesLet's define thestaticcomponents of a job definition in a YAML filedefault.slurmcpus_per_task:15job_name:'name'output:'%A_%a.out'Including these options with the using theyamlpackage is verysimpleimportyamlfromsimple_slurmimportSlurmslurm=Slurm(**yaml.load(open('default.slurm')))...slurm.set_array(range(NUMBER_OF_SIMULATIONS))The job can be updated according to thedynamicproject needs (ex.NUMBER_OF_SIMULATIONS).Using the command lineFor simpler dispatch jobs, a comand line entry point is also made available.simple_slurm[OPTIONS]"COMMAND_TO_RUN_WITH_SBATCH"As such, both of thesepythonandbashcalls are equivalent.slurm=Slurm(partition='compute.p',output='slurm.log',ignore_pbs=True)slurm.sbatch('echo \$HOSTNAME')simple_slurm--partition=compute.p--outputslurm.log--ignore_pbs"echo \$HOSTNAME"Job dependenciesThesbatchcall prints a message if successful and returns the correspondingjob_idjob_id=slurm.sbatch('python demo.py '+Slurm.SLURM_ARRAY_TAKSK_ID)If the job submission was successful, it prints:Submitted batch job 34987And returns the variablejob_id = 34987, which can be used for setting dependencies on subsequent jobsslurm_after=Slurm(dependency=dict(afterok=job_id)))Additional featuresFor convenience, Filename Patterns and Output Environment Variables are available as attributes of the Simple Slurm object.Seehttps://slurm.schedmd.com/sbatch.htmlfor details on the commands.fromslurmimportSlurmslurm=Slurm(output=('{}_{}.out'.format(Slurm.JOB_ARRAY_MASTER_ID,Slurm.JOB_ARRAY_ID))slurm.sbatch('python demo.py '+slurm.SLURM_ARRAY_JOB_ID)This example would result in output files of the form65541_15.out.
Here the job submission ID is65541, and this output file corresponds to the submission number15in the job array. Moreover, this index is passed to the Python codedemo.pyas an argument.Note that they can be accessed either asSlurm.<name>orslurm.<name>, hereslurmis an instance of theSlurmclass.Filename Patternssbatchallows for a filename pattern to contain one or more replacement symbols.They can be accessed withSlurm.<name>namevaluedescriptionJOB_ARRAY_MASTER_ID%Ajob array's master job allocation numberJOB_ARRAY_ID%ajob array id (index) numberJOB_ID_STEP_ID%Jjobid.stepid of the running job. (e.g. "128.0")JOB_ID%jjobid of the running jobHOSTNAME%Nshort hostname. this will create a separate io file per nodeNODE_IDENTIFIER%nnode identifier relative to current job (e.g. "0" is the first node of the running job) this will create a separate io file per nodeSTEP_ID%sstepid of the running jobTASK_IDENTIFIER%ttask identifier (rank) relative to current job. this will create a separate io file per taskUSER_NAME%uuser nameJOB_NAME%xjob namePERCENTAGE%%the character "%"DO_NOT_PROCESS\\do not process any of the replacement symbolsOutput Environment VariablesThe Slurm controller will set the following variables in the environment of the batch script.They can be accessed withSlurm.<name>.namedescriptionSLURM_ARRAY_TASK_COUNTtotal number of tasks in a job arraySLURM_ARRAY_TASK_IDjob array id (index) numberSLURM_ARRAY_TASK_MAXjob array's maximum id (index) numberSLURM_ARRAY_TASK_MINjob array's minimum id (index) numberSLURM_ARRAY_TASK_STEPjob array's index step sizeSLURM_ARRAY_JOB_IDjob array's master job id number......Seehttps://slurm.schedmd.com/sbatch.htmlfor a complete list.