Realtime process management

Related articles
cgroups
Performance
This article provides information on prioritizing process threads in real time, as opposed to at startup only. It shows how you can control CPU, memory, and other resource utilization of individual processes, or all processes run by a particular group.
While many recent processors are powerful enough to play a dozen video or audio streams simultaneously, it is still possible that another thread hijacks the processor for half a second to complete another task. This results in short interrupts in audio or video streams. It is also possible that video/audio streams get out of sync. While this is annoying for a casual music listener; for a content producer, composer or video editor this issue is much more serious as it interrupts their workflow.
The solution is to run time-sensitive processes in realtime. In linux, this means changing the process to a realtime scheduler, like
SCHED_RR
or
SCHED_FIFO
. See
sched(7)
for descriptions of these schedulers.
Configuration
On Arch Linux, system, group and user-wide configuration can be achieved using
PAM
and
systemd
.
The
realtime
package group provides additional tools to modify the realtime scheduling policies of
IRQs
and processes.
Note
When running a custom kernel, ensure it is
PREEMPT
enabled to make use of the methods mentioned above.
Configuring PAM
The
/etc/security/limits.conf
file provides configuration for the
pam_limits
PAM module, which sets limits on system resources (see
limits.conf(5)
).
Tip
It is advised to externalize the configuration of
pam_limits
to separate files below
/etc/security/limits.d
as those take precedence over the main configuration file.
There are two types of resource limits that
pam_limits
provides:
hard limits
and
soft limits
. Hard limits are set by
root
and enforced by the kernel, while soft limits may be configured by the user within the range allowed by the hard limits.
Installing the package
realtime-privileges
and adding the user to the
realtime
group, provides reasonable default values (e.g. relevant for
Professional audio
).
Configuring systemd services
Processes spawned by systemd system services need to specifically set up equivalents to
limits.conf
. Further information can be found in the sections
systemd.exec(5) § CREDENTIALS
and
systemd.exec(5) § PROCESS PROPERTIES
in
systemd.exec(5)
.
Usage
To run in realtime, a process running in the
realtime
group must set
LIMIT_RTPRIO
to any value above 0, then change to a realtime scheduler, like
SCHED_FIFO
or
SCHED_RR
. Here is an example:
#include <assert.h>
#include <sys/resource.h>
#include <sched.h>

void realtime()
{
    struct rlimit rl;
    assert(getrlimit(RLIMIT_RTPRIO, &rl) == 0);
    assert(rl.rlim_max != 0);
    rl.rlim_cur = rl.rlim_max;
    assert(setrlimit(RLIMIT_RTPRIO, &rl) == 0);

    assert(sched_setscheduler(0, SCHED_FIFO, &(struct sched_param){
            .sched_priority = rl.rlim_cur
        }) != -1);
}
Again,
LIMIT_RTPRIO
must be above 0 for the process to be able to change its scheduler. This is why
realtime-privileges
raises the
rtprio
limit to 98 for all processes in the
realtime
group. See
sched(7)
for more.
Using chrt
To do this automatically, or in case you are running somebody else's software, you can use
chrt
. Here is an example of starting
ls -al
in realtime under the
SCHED_FIFO
scheduler:
$ chrt --fifo 98 ls -al
This shouldn't normally be required. Applications that need to run in realtime, like
jack2
, normally set their schedulers all on their own.
Hard and soft realtime
Realtime is a synonym for a process which has the capability to run
in time without being interrupted by any other process. However, cycles can occasionally be dropped despite this. Low power supply or a process with higher priority could be a potential cause. To solve this problem, there is a scaling of realtime quality. This article deals with
soft
realtime.  Hard realtime is usually not so much desired as it is
needed
. An example could be made for car's ABS (anti-lock braking system). This can not be "rendered" and there is no second chance.
Tips and tricks
PAM-enabled login
Warning
Before making any changes in
/etc/pam.d
, read
PAM
to avoid breaking your system's authentication or creating security issues. Default installations should work without changes to this configuration.
PAM
is installed and configured on default Arch Linux installations. Nearly all
display managers
are pam-enabled, too. You can check which modules use
pam_limits.so
with the following command:
$ grep pam_limits.so /etc/pam.d/*
On a default installation, only two files use the module directly:
/etc/pam.d/system-auth:session    required                    pam_limits.so
/etc/pam.d/system-services:session   required    pam_limits.so
/etc/pam.d/system-auth
is included by other
pam-aware
applications. You can further search
/etc/pam.d
:
$ grep system-auth /etc/pam.d/*
In the default configuration this shows which applications are using
system-auth
and thus
pam_limits.so
. For example
...
/etc/pam.d/su:password        include         system-auth
/etc/pam.d/su-l:password        include         system-auth
/etc/pam.d/system-login:auth       include    system-auth
/etc/pam.d/system-login:account    include    system-auth
/etc/pam.d/system-login:password   include    system-auth
/etc/pam.d/system-login:session    include    system-auth
/etc/pam.d/system-services:account   include     system-auth
This covers both graphical and console login.
Checking Limits
In
Bash
, use
ulimit
(see
bash(1) § ulimit
) to check limits, for example the
rtprio
limit can be checked as follows:
$ ulimit -r
98
This is the value configured by
realtime-privileges
.
See also
getrlimit(2)
https://docs.kernel.org/scheduler/sched-design-CFS.html
https://docs.kernel.org/block/ioprio.html
IO Benchmarking: How, why and with what
SYSSTAT utilities home page