Package: sball
sballsballfacilitates submitting multiple scripts in Slurm job arrays using Yale's dSQ module. "sball" is short for "sbatch all".UsageAfter installing usingpip, you can runsballas a command. It takes a few arguments, which are the following. Any additional arguments not provided here are passed along tosbatchwhen the job array is submitted. Arguments must be specified askey=valuepairs, with the exception of thepatternargument, which should always be passed as the final argument. Additional arguments should be specified in the usual way forsbatch; i.e.,--$argument_name $argument_value.If you need to include spaces in arguments, make sure to escape them by using single quotes. If you do not escapepatternusing single quotes, make sure to escape any*in the glob expression with a backslash, since otherwise they will be expanded by the shell before sball gets them.name: (required) the name to give the job array.regex: (optional) only scripts with filenames matchingregexwill be included in any arrays.log_dir: (optional) the directory where log files for the job array should be stored. Default isjoblogs.(pattern): (required, not named) a glob expression that matches scripts to include in job arrays. Only files ending in.shthat match the glob will be included. If a glob expression is insufficient to filter to just the scripts you want, you should use theregexargument.Additional arguments are passed to the underlying calls tosbatch. This allows you to, e.g., set up job dependencies. They should be insertedbeforethe final, pattern argument.Descriptionsballfinds all.shscripts matchingpattern(andregex, if provided). If scripts are found, it sorts them into bins with unique sets of SBATCH options (since job arrays must all be run with the same SBATCH options). For each bin, the scripts are added to a job file, where the contents of each script takes up a single line. These files are named$name.txt, and are saved in the deepest directory common to all scripts in a bin. In case there is more than one bin, the file is suffixed with their bin number. Then,dsqcalled to create a job script from this file with the SBATCH options for that bin. Finally, the created job script is submitted to the queue withsbatch, and the (now unnecessary) job script is removed. The$name.txtfiles must be present when the job actually procs in the queue, so they are not removed (and should not be removed until after the jobs finish).In case a bin contains only one script, that script is just submitted in the usual way withsbatch, as a fallback.ExamplesSample directory structure:main-project-directory/
├── ...
│
└── scripts/
	├── script1.sh
	├── script2.sh
	├── script3.sh
	└── ...To submit all the scripts inmain-project-directory/scriptsin a job array namedmy-job-array, frommain-project-directoryyou would run:moduleloaddSQ# if not already loadedsballname=my-job-arrayscripts/\*Note that the*in the glob expression inpatternis escaped. After running this, your job array will be created, and you will have a new file inmain-project-directory/scriptsthat Slurm references to find the jobs.main-project-directory/
├── ...
│
└── scripts/
	├── script1.sh
	├── script2.sh
	├── script3.sh
	├── ...
	└── my-job-array.txtAfter your job array finishes running, you can safely removemy-job-array.txt. You can make further refinements of which scripts to run by changing the glob expression, or by using theregex=argument.