Change:
In NixOS VM tests, service sometimes fails to start. This breaks e.g. GlusterFS tests. Steps to reproduce the behavior: Building — GlusterFS service fails due to dependency on RPC Bind service. See also Some result related to GlusterFS. Have not yet had time to bisect and determine what's going on.
Hey, I started down this path looking for an updated Glusterfs package, then found the paused PR, which linked here. Fortunately, I think I know the root cause. As you noted, the failure is due to a dependency failure on - the error message in journald is: This was sort of confusing, but I found that led me to . The NixOS config includes: L /var/run - - - - ../run So my suspicion is that starts before and can't check for the lockfile. Since there's no unit, which was my first guess, I've instead been adding and tied to , which makes the tests pass. The diff that makes this work is: Obviously we should probably make this fix in NixOS proper, but I want to double-check with you that such a wider-impact change is safe to make. The proposed global_ fix for NixOS would be: thoughts/feedback? I can PR the fix if this all sounds right.
The real problem is that I am competent about neither rpcbind nor systemd-tmpfiles (otherwise I would try to write a more sensible complaint). Maybe PR the change (if it indeed fixes rpcbind judging from logs + fixes GlusterFS tests + doesn't break any NixOS tests that passed right before) and see if more competent people show up? It does feel to me that ideally tmpfiles should not depend on rpcbind and if they do (dunno, because we have /tmp on GlusterFS???) we are doomed anyway…
You and I are in the same boat, friend :joy: I'm only rpcbind-aware as it applies to GlusterFS, so I don't count myself as an expert, either. The only reason I feel confident enough to PR a fix is that all the pieces line up here and the passing tests make me think that this does indeed strike at the root cause. If NixOS is shipping with that tmpfiles configuration to bootstrap , then I think it makes sense for the downstream NixOS module to accomodate how NixOS is handling . So my initial instinct of patching upstream probably doesn't apply since distros are probably handling differently. I'll PR the change and we can see about additional feedback for us.
I'm feeling pretty confident that fixed the rpcbind problem - are there any nixosTests that should be re-enabled now that rpcbind reliably runs? I looked around for disabled tests but am not sure where those might live, if any are still disabled. Otherwise, I think we can close this :crossed_fingers:
(looked at the tests, doesn't look like there is a ton of disabled ones)

Discussion:
systemd.services.rpcbind = { wantedBy = [ "multi-user.target" ]; <ins> # rpcbind performs a check for /var/run/rpcbind.lock at startup # and will crash if /var/run isn't present. In the stock NixOS # var.conf tmpfiles configuration file, /var/run is symlinked to # /run, so rpcbind can enter a race condition in which /var/run # isn't symlinked yet but tries to interact with the path, so # controlling the order explicitly here ensures that rpcbind can # start successfully. The `wants` instead of `requires` should # avoid creating a strict/brittle dependency. wants = [ "systemd-tmpfiles-setup.service" ]; after = [ "systemd-tmpfiles-setup.service" ]; </ins> }; users.users.rpc = {