Fix deadlock in SubstitutionGoal

We were relying on SubstitutionGoal's destructor releasing the lock,
but if a goal is a top-level goal, the destructor won't run in a
timely manner since its reference count won't drop to zero.  So
release it explicitly.

Fixes #178.
Correctly detect infinite recursion in function application

If we're evaluating some application ‘v = f x’, we can't store ‘f’
temporarily in ‘v’, because if ‘f x’ refers to ‘v’, it will get ‘f’
rather than an infinite recursion error.

Unfortunately, this breaks the tail call optimisation introduced in
c897bac54954373f63511702731fe2cb23c0c98e.

Fixes #217.
Client-side handling of recursive nix

If NIX_REMOTE=recursive, the client checks
NIX_REMOTE_RECURSIVE_PROTOCOL_VERSION for an integer version. If it is
set and its major version matches the client major version (current
protocol version 0x101), it then reads a connected unix domain socket
file descriptor from NIX_REMOTE_RECURSIVE_FD. It sends a message
containing the client protocol version in the message data and a
connected unix domain socket in the ancillary data, and the other end of
that socket is then used for the normal daemon protocol.
Recursive nix client side: Report new paths

With recursive nix, it is possible for a build to legally gain access to
paths that weren't specified as its inputs (indeed, that is the whole
point). In order for the nix process that started the build to be able
to scan for dependencies properly, it must be told when these new paths
are (potentially) accessed.

This amends version 0x101 of the recursive protocol to put the socket fd
into NIX_REMOTE_RECURSIVE_SOCKET_FD and to put another fd into
NIX_REMOTE_RECURSIVE_PATHS_FD. This new fd is assumed to point to a new
kind of temporary gc root, where the root is kept from being considered
invalid by the parent nix process staying alive and thus only a write
lock is needed to avoid a race between the gc reading the root file and
a process adding to it. Upcoming commits should clarify this if it's
confusing.
nix-daemon handling of recursive nix

nix-daemon now supports listening on an already-connected socket. In the
normal case, nix-daemon will listen on its traditional socket (passed
through systemd or not) and makes a new socketpair for listening for the
new kind of connection. If, however, nix-daemon was started with
NIX_RECURSIVE_FDS set, it uses the two fds listed in that env var as the
two ends of the connected socket and doesn't listen on the traditional
socket at all. In a future commit, LocalStores that were *not* launched
from a nix-daemon child process will launch nix-daemon with that env var
properly set if they need to perform a build.

In either case, nix-daemon now has a connected socket to read from and
possibly an unconnected socket it's listening to to connect to. When the
connected socket is readable, it reads the client recursive version from
the message data and the client domain socket from the ancillary data to
form a new connection. If the listening socket is connectable, a
connection is formed as usual.

Finally, when a child process is launched, its LocalStore gets passed
the other side of the connected socket so it can use the existing
nix-daemon process for recursive nix instead of having to spawn a new
nix-daemon process every time (to be implemented in a future commit)
Replace per-pid temproots file with per-pid temproots dir

This will allow one process to keep several temproots files alive, as is
required for recursive nix.

The semantics are as follows:

* When creating its first temproot file, a process takes the gc lock,
  creates /nix/var/nix/temproots/{pid} directory (deleting it if it
  exits), takes a read lock on /nix/var/nix/temproots/{pid}/lock, and
  releases the gc lock
* Before adding a new temproot file (including the first one), the
  process upgrades its read lock on /nix/var/nix/temproots/{pid}/lock to
  a write lock. It can then make as many temproot files in
  /nix/var/nix/temproots/{pid}/lock as it likes, so long as they are not
  named "lock", and then it downgrades the write lock to a read lock
* Before adding a new root to a temproot file, the process takes a write
  lock on the file. Afterwards, it releases the lock. The locking
  process *never* holds a read lock on individual temproot files
* For each directory in /nix/var/nix/temproots, the gc first tries to
  get a write lock on /nix/var/nix/temproots/{dirname}/lock. If it
  succeeds, or if the lock file does not exist, the directory must be
  stale so it deletes it and moves on. If it fails, it waits until it
  can get a read lock on /nix/var/nix/temproots/{dirname}/lock (after
  which the owning process can't create more temproot files until
  released) and then iterates through every file not named "lock" in
  /nix/var/nix/tmproots/{dirname}, taking a read lock on the file then
  reading roots out of it as before. The gc does not release any of the
  read locks it takes here until collection is done.

For now, local-store uses /nix/var/nix/temproots/{pid}/main for its
temproots file, but future commits will also put recursive nix temproots
there.
fix openLockFile when create is false and the file doesn't exist
Start derivations with recursive nix set up

This involves:

* Setting up the NIX_STORE_PATH etc. env vars to match the current
  process
* Packing settings into _NIX_OPTIONS
* Setting NIX_REMOTE=recursive and NIX_REMOTE_RECURSIVE_PROTOCOL_VERSION
  to the protocol version
* If this process is not itself a daemon worker and it has not started a
  recursive nix daemon instance yet, making a new unidirectional
  socketpair and a pipe, sending both ends of the socketpair and the
  read end of the pipe to a new nix-daemon process in NIX_RECURSIVE_FDS,
  keeping the write end of the pipe open (so that nix-daemon can know
  if/when we die), and using the write end of the new socketpair as the
  store's recursive nix daemon socket
* Setting NIX_REMOTE_RECURSIVE_SOCKET_FD to the store's recursive nix
  daemon socket (and keeping it alive past closeMostFDs)
* Creating a new temproots file with the same name as this drv's path
* Setting NIX_REMOTE_RECURSIVE_PATHS to the new temproots file (and
  keeping it alive past closeMostFDs)
* After the build (if not using the build hook, which for now must use
  nix-store --import if it wants to report references not in the inputs
  closure), read through the new temproots file, add the closure of each
  temproot (include outputs of derivations) to allPaths, and add each
  path as a temproot so we can delete the new temproots file

This required adding functions to gc.cc to create and delete arbitrary
temproots files (which addTempRoot uses to creat the main temproots
file). nix-daemon was updated to add the third fd in NIX_RECURSIVE_FDS
(the read side of the close notification pipe) to its select set so that
it can exit once its parent dies (closes the write side of the pipe).

Now recursive nix should work! Next up: Test case.
Add test for recursive nix
remote-store.cc: Send the version little-endian
Can't do --add-root on a nix store path
The recursive paths fd passed to the build can't be close-on-exec
Make recursive test test a bit more
Clean up NIX_* variables passed to derivations
Open temproots files in O_APPEND mode so parallel remote nix invocations don't step on each other
Report recursive paths immediately and wait for confirmation

This allows the parent nix process to add any needed new paths to the
chroot. The reporting protocol is changed to have a cap of 4096 bytes
per write and to require reading a single confirmation byte after each
write.

Temp roots handling is reverted back to the pre-recursive-nix way, since
we no longer create a new temp roots file for each build.
Report results of building/ensuring paths

Also fix a dumb bug
Remove now-unused argument
Sending fds is shockingly non-portable
Mount recursive paths in the child's namespace, not the parents