Change:
Describe the bug Systemd has implemented an Out-Of-Memory Killer in 247, and I would for sure love to have this and use it default. Fedora has been taking measures to introduce this in their next release over recently using earlyoom and as you can see from their recent commits in their systemd build expressions they're actively tweaking the defaults.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
If you want to try oomd right now:
What does everyone think about turning oomd on by default? At least my system livelocks so badly (for dozens of minutes) without oomd that it is frankly unconscionable to be shipped to end users that way.
I agree, I also always enable earlyoom (so oomd should also be fine)
Looking into this, my understanding is that systemd-oomd only became fully supported with systemd 248 and was still experiencing notable changes after the release of 247, but those changes are being backported to stable, so we should be good to enable it by default even though 21.05 will be released with systemd 247.
Maybe the service should give a warning if both and are enabled?
Is there any progress? :thinking: I would love to migrated from earlyoom to systemd-oomd
Note that turning this on by default is a horrible idea, as of: In practice it means that it will kill whole user session if you are using a window manager. I have managed to to start every app in a separate cgroup using , but requiring every user to do this is bogus.
Whoops sorry wrong button. I would love to have a “thing” that captures orphaned processes into a systemd-run context, but idk if you can change cgroups after the fact. Getting these cgroups right “by construction” will require a lot of patching and policy and setup to existing tools. (Maybe a around execve? But hard to get correct in general without breaking everything horribly.)
But yes, for a desktop system, where your use-case is “don’t hang my desktop indefinitely when the load goes over 1 per CPU” I’ve had much success with just enabling . It will even kill single hungry browser tabs, since modern browsers spawn one process per tab. Maybe both can be productively used together. Given that works with cgroups, it sounds like it’s best for a server environment where everything is managed by systemd services.
You can, just not using systemd-run, you would directly use dbus, or can even change them by hands using sysfs Tbh i'm not even sure how they do it by construction, when i used dbus interface, they only had option to add existing pid, probably should look into it again. Also not sure orphanated is the right word, but yes you probably can do something to move processes to their own cgroups, but doing it the "right way" would require some thinking process. I will try to look into it, i think some daemon that moves processes from a window manager scope to different scopes should be easily implementable, albeit sounds like a stupid hack. Well at least GNOME and KDE Plasma have their management thing, like the one i made for xmonad, so oomd can safely enabled for them. Actually my hack was directly inspired by their session management. EDIT: Also oomd wont kill your apps if you haven't made it do so by setting , it will just report that your system is in low memory condition. I think the best is to enable it in the systemd derivation as a feature, and add nixos module that will configure and run it.
One more reason oomd should be opt-in is that, unless I misunderstand, it seems to require swap. But swap is less and less necessary as RAM gets cheaper. I'm not running any swap at all these days, just enough RAM that swap isn't needed. I like the concept though, and it would be nice to be able to run it regardless of swap config.
Based on my reading, they encourage the use of swap, but do not require it. systemd-oomd has two criteria it can react to, one being swap usage, and the other being "memory pressure". That might mean oomd needs to be configured differently for swap-enabled systems than for swap-less systems. Personally, I generally do not swap to disk, but I do enable zramSwap.
Any news on this? I would really like to have systemd-oomd on nixos.
Counterintuitively, swap is useful even under no/low memory contention:
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
I think that oomd is not yet ready for desktop systems. In my experience earlyoom is by far more reliable than oomd, oomd kill even my DE process.

Discussion:
</listitem> <listitem> <para> <ins> systemd-oomd is enabled by default. Depending on which systemd units have <literal>ManagedOOMSwap=kill</literal> or <literal>ManagedOOMMemoryPressure=kill</literal>, systemd-oomd will SIGKILL all the processes under the appropriate descendant cgroups when the configured limits are exceeded. NixOS does currently not configure cgroups with oomd by default, this can be enabled using <link xlink:href="options.html#opt-systemd.oomd.enableRootSlice">systemd.oomd.enableRootSlice</link>, <link xlink:href="options.html#opt-systemd.oomd.enableSystemSlice">systemd.oomd.enableSystemSlice</link>, and <link xlink:href="options.html#opt-systemd.oomd.enableUserServices">systemd.oomd.enableUserServices</link>. </para> </listitem> <listitem> <para> </ins> The <literal>pass-secret-service</literal> package now includes systemd units from upstream, so adding it to the NixOS <literal>services.dbus.packages</literal> option will

- Add udev rules for the Teensy family of microcontrollers. <ins> - systemd-oomd is enabled by default. Depending on which systemd units have `ManagedOOMSwap=kill` or `ManagedOOMMemoryPressure=kill`, systemd-oomd will SIGKILL all the processes under the appropriate descendant cgroups when the configured limits are exceeded. NixOS does currently not configure cgroups with oomd by default, this can be enabled using [systemd.oomd.enableRootSlice](options.html#opt-systemd.oomd.enableRootSlice), [systemd.oomd.enableSystemSlice](options.html#opt-systemd.oomd.enableSystemSlice), and [systemd.oomd.enableUserServices](options.html#opt-systemd.oomd.enableUserServices). </ins> - The `pass-secret-service` package now includes systemd units from upstream, so adding it to the NixOS `services.dbus.packages` option will make it start automatically as a systemd user service when an application tries to talk to the libsecret D-Bus API. - There is a new module for AMD SEV CPU functionality, which grants access to the hardware.

./system/boot/systemd/journald.nix ./system/boot/systemd/logind.nix ./system/boot/systemd/nspawn.nix <ins> ./system/boot/systemd/oomd.nix </ins> ./system/boot/systemd/shutdown.nix ./system/boot/systemd/tmpfiles.nix ./system/boot/systemd/user.nix

<ins> { config, lib, ... }: let cfg = config.systemd.oomd; in { options.systemd.oomd = { enable = lib.mkEnableOption "the systemd-oomd OOM killer" // { default = true; }; # Fedora enables the first and third option by default. See the 10-oomd-* files here: # https://src.fedoraproject.org/rpms/systemd/tree/acb90c49c42276b06375a66c73673ac351025597 enableRootSlice = lib.mkEnableOption "oomd on the root slice (-.slice)"; enableSystemSlice = lib.mkEnableOption "oomd on the system slice (system.slice)"; enableUserServices = lib.mkEnableOption "oomd on all user services (user@.service)"; extraConfig = lib.mkOption { type = with lib.types; attrsOf (oneOf [ str int bool ]); default = {}; example = lib.literalExpression ''{ DefaultMemoryPressureDurationSec = "20s"; }''; description = '' Extra config options for systemd-oomd. See man oomd.conf for available options. ''; }; }; config = lib.mkIf cfg.enable { systemd.additionalUpstreamSystemUnits = [ "systemd-oomd.service" "systemd-oomd.socket" ]; systemd.services.systemd-oomd.wantedBy = [ "multi-user.target" ]; environment.etc."systemd/oomd.conf".text = lib.generators.toINI {} { OOM = cfg.extraConfig; }; systemd.oomd.extraConfig.DefaultMemoryPressureDurationSec = lib.mkDefault "20s"; # Fedora default users.users.systemd-oom = { description = "systemd-oomd service user"; group = "systemd-oom"; isSystemUser = true; }; users.groups.systemd-oom = { }; systemd.slices."-".sliceConfig = lib.mkIf cfg.enableRootSlice { ManagedOOMSwap = "kill"; }; systemd.slices."system".sliceConfig = lib.mkIf cfg.enableSystemSlice { ManagedOOMSwap = "kill"; }; systemd.services."user@".serviceConfig = lib.mkIf cfg.enableUserServices { ManagedOOMMemoryPressure = "kill"; ManagedOOMMemoryPressureLimit = "50%"; }; }; } </ins>

systemd-networkd-ipv6-prefix-delegation = handleTest ./systemd-networkd-ipv6-prefix-delegation.nix {}; systemd-networkd-vrf = handleTest ./systemd-networkd-vrf.nix {}; systemd-nspawn = handleTest ./systemd-nspawn.nix {}; <ins> systemd-oomd = handleTest ./systemd-oomd.nix {}; </ins> systemd-shutdown = handleTest ./systemd-shutdown.nix {}; systemd-timesyncd = handleTest ./systemd-timesyncd.nix {}; systemd-misc = handleTest ./systemd-misc.nix {};

<ins> import ./make-test-python.nix ({ pkgs, ... }: { name = "systemd-oomd"; nodes.machine = { pkgs, ... }: { systemd.oomd.extraConfig.DefaultMemoryPressureDurationSec = "1s"; # makes the test faster # Kill cgroups when more than 1% pressure is encountered systemd.slices."-".sliceConfig = { ManagedOOMMemoryPressure = "kill"; ManagedOOMMemoryPressureLimit = "1%"; }; # A service to bring the system under memory pressure systemd.services.testservice = { serviceConfig.ExecStart = "${pkgs.coreutils}/bin/tail /dev/zero"; }; # Do not kill the backdoor systemd.services.backdoor.serviceConfig.ManagedOOMMemoryPressure = "auto"; virtualisation.memorySize = 1024; }; testScript = '' # Start the system machine.wait_for_unit("multi-user.target") machine.succeed("oomctl") # Bring the system into memory pressure machine.succeed("echo 0 > /proc/sys/vm/panic_on_oom")  # NixOS tests kill the VM when the OOM killer is invoked - override this machine.succeed("systemctl start testservice") # Wait for oomd to kill something # Matches these lines: # systemd-oomd[508]: Killed /system.slice/systemd-udevd.service due to memory pressure for / being 3.26% > 1.00% for > 1s with reclaim activity machine.wait_until_succeeds("journalctl -b | grep -q 'due to memory pressure for'") ''; }) </ins>