Package: jug-schedule
jug_scheduleis ajugsubcommand that provides automatic deployment to queueing systems.Currently supportsSGE/GE(Grid Engine),LSF(IBM LSF), andSLURM.This project is currently experimental so bug reports are welcome.RequirementsThis project depends ondrmaaand obviouslyjug.InstallationInstalljug_schedulewith:pip install jug-scheduleor from Anaconda with:conda install -c unode jug-scheduleand then simply add the following to your~/.config/jug/jug_user_commands.py:try:
    from jug_schedule.schedule import ScheduleCommand
    schedule = ScheduleCommand()
except Exception as e:
    import sys
    sys.stderr.write("Couldn't import schedule, error was {0}\n".format(e))If you are running directly from git you can instead use:import sys

sys.path.insert(0, "/path/to/clone/of/jug_schedule/")

try:
    from jug_schedule.schedule import ScheduleCommand
    schedule = ScheduleCommand()
except Exception as e:
    sys.stderr.write("Couldn't import schedule, error was {0}\n".format(e))NoteIf you have never setupDRMAAon your environment, check theConfigurationsection below.UsageIf installed properly, runningjugshould now include aschedulesubcommand.Running it will try to detect a queueing system and submit jobs to it.jug schedulewill only produce warning and errors. Use--verbosedebugfor detailed messages.jug statuswill behave as usual and is the recommended way to check progress of execution.ScriptIf your system requires additional setup for jug to run on remote servers you will need to use the–scriptoption.
The script should calljugwith all given arguments. As an example:#!/usr/bin/env bash

# Enabling modules (http://modules.sourceforge.net/) in the current shell
if [ -f /etc/profile.d/modules.sh ]; then
    source /etc/profile.d/modules.sh
fi

# Loading (ana)conda
module add conda
# and an environment called py3 where jug was installed
source activate py3

# Then calling jug with all given arguments. Make sure to keep the quotes on $@
jug "$@"Assuming the above content is saved in a filescript.helperyou can then call:jug schedule --script script.helperif instead you give it the same name as your jugfile it will be automatically loaded:$ ls
myjugfile.helper myjugfile.py
$ jug schedule myjugfile.py  # <- will use myjugfile.helperConfigurationjug_schedulerelies onDRMAAfor interaction with the queueing system.DRMAAsupport is limited and its quality varies considerably across platforms.Currently supported platforms includeLSF,SGEandSLURM.In order to usejug_scheduleyour environment needs to defineDRMAA_LIBRARY_PATH.
If runningenv | grep DRMAA_LIBRARY_PATHreturns no match, ask your system administrators for the location of this library.Then use:export DRMAA_LIBRARY_PATH=/path/to/libdrmaa.soYou only need to set this option on the environment that runsjug schedule.NoteYou can also usecontrib/find_libdrmaato locatelibdrmaa.soon your system.
Check theREADMEinside thecontrib/folder for more information.ResourcesAn additional feature ofjug_scheduleis the ability to define job resources.If you already know jug’sTaskGeneratordecorator you can simply replace it with the following where applicable:from jug_schedule.resources import ResourcesTaskGenerator

@ResourcesTaskGenerator(cpu=10, mem=100, queue="default")
def func(...):
    ...Supported arguments include:cpu,mem(in MB),queueandcustomfor arbitrary options.NoteWhen usingcustom, be aware that providing invalid or misformatted options may cause crashes.
SLURM is a known offender as reportedon this issue.Command-line optionsThe following options are available:--script            - command used to run jug on the cluster. Point this to a shell script if you need to setup jug's environment prior to execution
--max-jobs          - how big is the pool of jug jobs (max number of simultaneous jobs)
--max-array         - when submitting jobs to the queue system, limit the maximum number of jobs per submission
--logs              - where to write job logs. Defaults to a directory 'jug_logs' in the current directory.
--cycle-time        - how many seconds to wait between every interaction with the queue system. Defaults to 60
--stop-on-error     - jug_schedule will continue until all jobs fail. Default is to continue queueing jobs when a job fails.
--recycle           - when a job fails, instead of removing one job from the pool, recycle it and keep the pool size constant.