delete_ref: avoid stale-lock race while deleting loose ref

When we take a lock on a ref, we do not remember the actual
filename of the ref, but rather only its lockfile, which
will be `$GIT_DIR/$ref.lock`. To delete the loose ref
itself, we munge the lockfile's buffer to drop the ".lock"
suffix, delete the loose ref file, then restore the original
buffer.

If we happen to receive a signal while the filename is
munged, then our automatic cleanup for the lockfile will
kick in, but it will delete the wrong file (i.e., the actual
loose ref file). This isn't that bad, as the point of the
code is to delete that file anyway, so nothing of value is
lost.  However, we will fail to delete our real lockfile,
leaving a stale lock in place.
add API for cleaning up temporary files on exit

Many parts of the code introduce temporary files, and then
clean them up with a custom signal/atexit handler. We can
reduce code duplication and make it simpler to add tempfile
handling in new places by abstracting this into a simple
API.

The API is based largely on the lockfile cleanup code, which
is the most comprehensive. In particular, it has a few
features that other tempfile code does not:

  1. It keeps an "owner" field with the pid of the process
     that took the lock. That means that a sub-process
     (e.g., start_async on a machine without pthreads) will
     not accidentally clean up the tempfiles.

  2. It keeps the file descriptor of the opened tempfile.
     Ideally the tempfile code would not need to care about
     the descriptor at all, and only unlink the filename;
     however, on some systems one must close the file before
     unlinking (e.g., Windows).

In addition, we add one new feature not present even in the
lockfile code: an "active" flag. The lockfile code decides
that a non-empty filename field in the lock means that the
entry is valid.  Because we may reuse a "struct lock_file",
meaning it is already on the list of items to be cleaned,
and because we write the lockfile name into the "struct
lock_file" before we open it, there is a race that can
happen like:

  1. We write "foo.lock" into a "struct lock_file" that has
     previously been used.

  2. We try to acquire the lock but fail because it is
     already locked.

  3. Before we get a chance to zero out lk->filename[0], we
     get a signal.

  4. The signal runs remove_lock_file_on_signal, which
     thinks that the lockfile is valid and removes it.

The new tmpfile code solves this by keeping an active flag,
and never turning it on until we are sure the file is
"ours" (and turning it off before dropping control of
the file via rename or removal).
lockfile: use tmpfile API to handle file cleanup

Now that the tmpfile API implements all of our features
(plus the "active" flag described in the previous commit),
we can drop the duplicated code, and just put a "struct
tmpfile" into our "struct lock_file".

Most of the changes are mechanical, as we now need to refer
to "lk->tmp.filename" instead of "lk->filename". Since we
don't need any additional fields beyond that "struct
tmpfile" provides, we could instead just convert all "struct
lock_file" into "struct tmpfile" (or even hide it behind a
typedef).  However, that would be similarly disruptive (we
would not have to change member access, but we would have to
change many function signatures), and it would not leave any
room for additional lock-specific fields to be added later.
diff: check error code of close() when writing tempfiles

When we write a tempfile to disk for textconv or an external
diff, we carefully check that we are able to write the full
content to disk before invoking the sub-program. However, we
did not check the return value of close(), meaning that the
file might not have been fully written (e.g., on a networked
filesystem), meaning that the sub-program may have seen an
incomplete version of the file.
diff: use tmpfile API for temporary file cleanup

This reduces code duplication and gives us extra safety
features (like owner pid checks and removal in the proper
order for signal-race safety).

Furthermore, since we no longer need to maintain a global
list of files to clean up (tmpfile handles this itself), we
can do away with the ugly claim_tempfile system, and just
use a static tmpfile in each function which needs one.
sha1_file: use struct tmpfile to create loose object files

We do not currently clean half-written objects when we exit.
However, by refactoring the code to use "struct tmpfile", it
will be easy to make that configurable in the future.

Note that this patch is a pure refactoring; we use "struct
tmpfile" but we never call tmpfile_register. The behavior
should be identical before and after.
tmpfile-ize packfile writing
tmpfile-ize index-pack

This needs some work; we cannot replace move_temp_to_file
with tmpfile_rename. Also, the naming is a quite hairy. We
probably need to copy the existing pack name we are given
into output.filename.
add core.cleantempobjects