Slurm Workload Manager

Slurm Workload Manager
The Slurm Workload Manager, formerly known as Simple Linux Utility for Resource Management (SLURM), or simply Slurm, is a free and open-source job scheduler for Linux and Unix-like kernels, used by many of the world's supercomputers and computer clusters.
It provides three key functions:
allocating exclusive and/or non-exclusive access to resources (computer nodes) to users for some duration of time so they can perform work,
providing a framework for starting, executing, and monitoring work, typically a parallel job such as Message Passing Interface (MPI) on a set of allocated nodes, and
arbitrating contention for resources by managing a queue of pending jobs.
Slurm is the workload manager on about 60% of the TOP500 supercomputers.
Slurm uses a best fit algorithm based on Hilbert curve scheduling or fat tree network topology in order to optimize locality of task assignments on parallel computers.
History
Slurm began development as a collaborative effort primarily by Lawrence Livermore National Laboratory, SchedMD, Linux NetworX, Hewlett-Packard, and Groupe Bull as a Free Software resource manager. It was inspired by the closed source Quadrics RMS and shares a similar syntax. The name is a reference to the soda in Futurama. Over 100 people around the world have contributed to the project. It has since evolved into a sophisticated batch scheduler capable of satisfying the requirements of many large computer centers.
, TOP500 list of most powerful computers in the world indicates that Slurm is the workload manager on more than half of the top ten systems.
Structure
Slurm's design is very modular with about 100 optional plugins. In its simplest configuration, it can be installed and configured in a couple of minutes. More sophisticated configurations provide database integration for accounting, management of resource limits and workload prioritization.
Features
Slurm features include:
No single point of failure, backup daemons, fault-tolerant job options
Highly scalable (schedules up to 100,000 independent jobs on the 100,000 sockets of IBM Sequoia)
High performance (up to 1000 job submissions per second and 600 job executions per second)
Free and open-source software (GNU General Public License)
Highly configurable with about 100 plugins
Fair-share scheduling with hierarchical bank accounts
Preemptive and gang scheduling (time-slicing of parallel jobs)
Integrated with database for accounting and configuration
Resource allocations optimized for network topology and on-node topology (sockets, cores and hyperthreads)
Advanced reservation
Idle nodes can be powered down
Different operating systems can be booted for each job
Scheduling for generic resources (e.g. Graphics processing unit)
Real-time accounting down to the task level (identify specific tasks with high CPU or memory usage)
Resource limits by user or bank account
Accounting for power consumption by job
Support of IBM Parallel Environment (PE/POE)
Support for job arrays
Job profiling (periodic sampling of each task's CPU use, memory use, power consumption, network and file system use)
Sophisticated multifactor job prioritization algorithms
Support for MapReduce+
Support for burst buffer that accelerates scientific data movement
The following features are announced for version 14.11 of Slurm, was released in November 2014:
Improved job array data structure and scalability
Support for heterogeneous generic resources
Add user options to set the CPU governor
Automatic job requeue policy based on exit value
Report API use by user, type, count and time consumed
Communication gateway nodes improve scalability
Supported platforms
Slurm is primarily developed to work alongside Linux distributions, although there is also support for a few other POSIX-based operating systems, including BSDs (FreeBSD, NetBSD and OpenBSD). Slurm also supports several unique computer architectures, including:
IBM BlueGene/Q models, including the 20 petaflop IBM Sequoia
Cray XT, XE and Cascade
Tianhe-2 a 33.9 petaflop system with 32,000 Intel Ivy Bridge chips and 48,000 Intel Xeon Phi chips with a total of 3.1 million cores
IBM Parallel Environment
Anton
License
Slurm is available under the GNU General Public License v2.
Commercial support
In 2010, the developers of Slurm founded SchedMD, which maintains the canonical source, provides development, level 3 commercial support and training services. Commercial support is also available from Bull, Cray, and Science + Computing.
See also
Job Scheduler and Batch Queuing for Clusters
Beowulf cluster
Maui Cluster Scheduler
Open Source Cluster Application Resources (OSCAR)
TORQUE
Univa Grid Engine
Platform LSF
References
Further reading
External links
Slurm Documentation
SchedMD
Slurm Workload Manager Architecture Configuration and Use
Job scheduling
Parallel computing
Grid computing
Cluster computing
Free software