Package: tessti
TESSTI - The Easiest SLURM Scheduler There IsTessti has been made to be the simplest SLURM scheduler possible.It does not offer much but it works, and is extremely easy to use.Getting StartedSuppose you run locally a functionfin a script namedscript.pyin a folder/some/path/with some arguments.The end of thescript.pymay look like this:if__name__=='__main__':f(arg1=value1,arg2=value2)You want to schedule many executions of this script with different values for argumentsarg1andarg2.You will be able to run something like:$ tessti schedule --file some/path/script.py --function f --args arg1=value1,arg2=value2 --schedule arg1=[value3, value4],arg2=[value5, value6]The--argsflag defines the default parameters, whereas the--scheduleflag defines all the runs to be requested to SLURM.Comprehensive overviewtesstirelies on (toml) config files. It will creates one config file per job to run, plus two special config files, one acting as a template to generate the job configs, and one for the schedule itself.Each config is composed of two parts:the SLURM config, where ressources are requested, and general environment is set up.the script config, required to execute a single job.The SLURM config can be set globally, so that it doesn't need to be specified for every schedule.Hence, tessti exposes three commands:tessti set-config: allows to update the global SLURM configuration.tessti get-config: displays the current SLURM configuration.tessti schedule: requests SLURM jobs based on the SLURM and script configs. SLURM config can also be modified here for this schedule only.(Technically, there is a fourth commandrun. This is the one called by SLURM jobs. You are not really supposed to use it but of course you can if you want to).For each of these three commands, you can runtessti command --helpto see the parameters.
For the sake of completeness, they are written bellow as well.The SLURM configurationjobs_dir: str: relative or absolute path of the directory in which the SLURM inputs and outputs files will be placed.Defaults to 'jobs'.partition: str: HPC partition to use.Defaults to 'publigpu'.account: str: HPC account to use.Defaults to 'miv'.node: int: number of node(s).Defaults to 1.task: int: number of task(s) per node.cpu: int: CPU cores per task.Defaults to 1.gpu: int: GPU per node.Defaults to 1.ram: int: RAM memory in Go.Defaults to 16.constraint: str: Any constraint for the nodes requested.Defaults to'gpua100|gpurtx6000|gpurtx5000|gpuv100'.modules: list[str]: Modules that must the loaded before starting a job.Defaults to['python/Anaconda3-2019', 'cuda/cuda-11.8', 'gcc/gcc-11'].commands: list[str]: Any command that must be executed before starting a job.Defaults to['source /usr/local/Anaconda/Anaconda3-2019.07/etc/profile.d/conda.sh', 'conda deactivate', 'conda activate torch2cu118'].WARNING: when providing list of strings in argument,do not use spaces between list elements.The schedule parametersJob parameters:file: str: relative or absolute path to the file each job must execute. The '.py' extension is not required.function: str: name of the function withinfile.pythat each job must execute.args: dict[Any: Any]: Base args of the function withinfile.pythat each job must execute. For each job, exactly one of these args will be overwritten.Scheduler parameters:name: str: base name for the whole schedule. Created jobs will be named alikename_arg=value.job.schedule: dict[str: Sequence[Any]]: Sequence of values for each arg that must be set in a separated job.WARNING: when providing list of strings in argument,do not use spaces between list elements.Some examplesThe relevant code can be found in theexamplefolder.The filescript.pycontains a functionaddwith the following signature:add(a:int,b:int)->int:First, let's set up the global SLURM config, which will be the base for all subsequent schedules.$ tessti set-config --account some_account_name --cpu 2 --gpu 4 --commands ['command1','command2']WARNING: when providing list of strings in argument,do not use spaces between list elements.We can then check the current configuration with$ tessti get-configNow let's schedule some jobs. We also change some SLURM configuration parameters on the fly for this schedule only.$ tessti schedule --file example/script.py --function add --args a=0,b=0 --schedule a=[1,2],b=[3,4] --cpu 4WARNING: when providing list of parameters with--argsand--schedule,do not use spaces between list elements.This schedule sets the defaults values for theadd()function both to 0 through theargskeyword argument.It schedules four jobs:- one witha=1, b=0(first scheduled value fora, default value forb).- one witha=2, b=0(second scheduled value fora, default value forb).- one witha=0, b=4(default value fora, first scheduled values forb).- one witha=0, b=5(default value fora, second scheduled valueb).You can see the generated files in a folder within the directory containing the file to be executed, named after the--jobs_dirflag.You can check in the generatedschedule.tomlfile that the SLURM config was indeed updated and that the global config (as returned bytessti get-config) was **not updated.Using the schedule functionIf you prefer not to use the tessti CLI, you are free to create a python script that directly uses tessti's schedule function.Please refer to the example inexample/schedule.py.