NixOS/nix — pull request #213 by shlevy (2014-02-18 19:15:12+00:00)
Title: Recursive nix

This PR makes it possible to run nix from within a nix build. This is useful for things like using nix as a low level build tool, and is usable in places where import-from-derivation isn't (see #13 for more details).

This is done by introducing a new remote mode for nix. If `NIX_REMOTE` is `recursive`, then the remote client (e.g. `RemoteStore` or `guix`) should do the following:

1. Read `NIX_REMOTE_RECURSIVE_PROTOCOL_VERSION` as the ASCII representation of an unsigned int. This is the version of the remote protocol in use by the parent nix process (currently `0x101`)
2. Verify that the major version (using the same definition for major and minor as the daemon protocol) of the protocol that the parent process uses matches a major version the client understand
3. Read `NIX_REMOTE_SOCKET_FD` as the ASCII representation of a file descriptor. This is a writable unix domain socket (note that this can't be used as the daemon socket directly as multiple processes may use the same fd).
4. Write a message containing the version of the protocol the client will use as a little-endian unsigned integer in the main data and a connected full-duplex unix domain socket (e.g. one of a pair created by `socketpair`) in the ancillary data to the fd found in `NIX_REMOTE_RECURSIVE_SOCKET_FD`. The socket that the sent socket is connected to can then be treated as a freshly-connected socket for the daemon protocol.
5. Read `NIX_REMOTE_RECURSIVE_PATHS_FD` as the ASCII representation of a file descriptor. This file descriptor should be used to tell the parent nix process of new inputs that it should scan for when looking for references in this build's outputs (which is needed since recursive nix can make store paths that aren't inputs to the build accessible to it, which is really the whole point). Any paths so reported are also guaranteed not to be garbage collected until after the reference scan. Before appending (the fd is opened with `O_APPEND`), you must take an `fcntl` write lock on the file. The file format is just a sequential series of null-terminated store paths. The `RemoteStore` implementation collects every newly-discovered path in `PathSet` throughout its lifetime, then in the destructor reports them all at once to avoid holding the lock for a long time, but that isn't necessary.

And then the daemon protocol is followed as usual. This required several changes on the part of the parent nix process:

* Instead of per-process temproots files, we have per-process temproots directories. This allows a single nix process to keep the reported paths of several recursive nix invocations alive.
* When started normally, in addition to listening on the traditional socket `nix-daemon` also creates a pair of unix sockets to receive recursive nix connections on. It passes the write end of the pair to all children `LocalStore` instances.
* When starting a build, `LocalStore` sets `NIX_STORE_DIR` and friends to the values appropriate for this store instance, `pack`s the `settings` into `_NIX_OPTIONS` (see `Settings::pack` and `Settings::unpack`), and sets `NIX_REMOTE` to `recursive` and puts the protocol version in `NIX_REMOTE_RECURSIVE_PROTOCOL_VERSION`. It also creates a new temproot (named after the derivation file for guaranteed uniqueness) and puts that into `NIX_REMOTE_RECURSIVE_PATHS_FD`. If this is the first build, it then checks if it was passed a socket upon construction. If it was, that means this is a child of a `nix-daemon` process and it passes that socket in `NIX_REMOTE_RECURSIVE_SOCKET_FD`. If it wasn't, then it creates a socket pair and a pipe, puts both ends of the socket and the read end of the pipe into `NIX_RECURSIVE_FDS` (space separated) and starts `nix-daemon`. When `NIX_RECURSIVE_FDS` is set, `nix-daemon` does not listen on its traditional socket and instead uses the passed fds to listen for recursive nix connections. The read end of the pipe is used by `nix-daemon` to know when the parent process has died so it knows to exit. The parent `LocalStore` then closes the read end of the pipe and socket pair, purposefully leaks the write end of the pipe (though it is `O_CLOEXEC`), and uses the write end of the socket pair as `NIX_REMOTE_RECURSIVE_SOCKET_FD` for all builds.
* After a build, `LocalStore` reads the temproot for that build, adds the closure of each path written there (including outputs if any are derivations) to its temproots and to the `PathSet` used when scanning for references, and then deletes the temproot.

cc @civodul for `guix`

Fixes #13