read-cache: connect to file watcher

This patch establishes a connection between a new file watcher daemon
and git. Each index file may have at most one file watcher attached to
it. The file watcher maintains a UNIX socket at
$GIT_DIR/index.watcher. Any process that has write access to $GIT_DIR
can talk to the file watcher.

A validation is performed after git connects to the file watcher to
make sure both sides have the same view. This is done by exchanging
the index signature (*) The file watcher keeps a copy of the signature
locally while git computes the signature from the index. If the
signatures do not match, something has gone wrong so both sides
reinitialize wrt. to file watching: the file watcher clears all
watches while git clears CE_WATCHED flags.

If the signatures match, we can trust the file watcher and git can
start asking questions that are not important to this patch.

This file watching thing is all about speed. So if the daemon is not
responding within 20ms (or even hanging), git moves on without it.

A note about per-repo vs global (or per-user) daemon approach. While I
implement per-repo daemon, this is actually implementation
details. Nothing can stop you from writing a global daemon that opens
unix sockets to many repos, e.g. to avoid hitting inotify's 128 user
instances limit.

If env variable GIT_NO_FILE_WATCHER is set, the file watcher is
ignored. 'WATC' extension is kept, but if the index is updated
(likely), it'll become invalid at the next connection.

(*) for current index versions, the signature is the index SHA-1
trailer. But it could be something else (e.g. v5 does not have SHA-1
trailer)
read-cache: ask file watcher to watch files

We want to watch files that are never changed because lstat() on those
files is a wasted effort. So we sort unwatched files by date and start
adding them to the file watcher until it barfs (e.g. hits inotify
limit). Recently updated entries are also excluded from watch list.
CE_VALID is used in combination with CE_WATCHED. Those entries that
have CE_VALID already set will never be watched.

We send as many paths as possible in one packet in pkt-line
format. For small projects like git, all entries can be packed in one
packet. For large projects like webkit (182k entries) it takes two
packets. We may do prefix compression as well to send more in fewer
packets..

The file watcher replies how many entries it can watch (because at
least inotify has system limits).

Note that we still do lstat() on these new watched files because they
could have changed before the file watcher could watch them. Watched
files may only skip lstat() at the next git run.
read-cache: put some limits on file watching

watch_entries() is a lot of computation and could trigger a lot more
lookups in file-watcher. Normally after the first set of watches are
in place, we do not need to update often. Moreover if the number of
entries is small, the overhead of file watcher may actually slow git
down.

This patch only allows to update watches if the number of watchable
files is over a limit (and there are new files added if this is not
the first time). Measurements on Core i5-2520M and Linux 3.7.6, about
920 lstat() take 1ms. Somewhere between 2^16 and 2^17 lstat calls that
it starts to take longer than 100ms. 2^16 is chosen at the minimum
limit to start using file watcher.

Recently updated files are not considered watchable because they are
likely to be updated again soon, not worth the ping-pong game with
file watcher. The default limit 30min is just a random value.
read-cache: get modified file list from file watcher

A new command is added to file watcher to send back the list of
updated files to git. These entries will have CE_WATCHED removed. The
remaining CE_WATCHED entries will have CE_VALID set (i.e. no changes
and no lstat either).

The file watcher does not cache stat info and send back to git. Its
main purpose is to reduce lstat on most untouched files, not to
completely eliminate lstat.

The file watcher keeps reporting the same "updated" list until it
receives "forget" commands, which should only be issued after the
updated index is written down. This ensures that if git crashes half
way before it could update the index (or multiple processes is reading
the same index), "updated" info is not lost.

After the index is updated (e.g. in this case because of toggling
CE_WATCHED bits), git sends the new index signature to the file
watcher.
read-cache: add config to start file watcher automatically
file-watcher: add --shutdown option
file-watcher: automatically quit

If $GIT_DIR/index.watcher is gone, exit. We could watch this path too,
but we'll waste precious resources (at least with inotify). And with
inotify, it seems to miss the case when $GIT_DIR is moved. Just check
if the socket path still exists every minute.

As the last resort, if we do not receive any commands in the last 6
hours, exit.
file-watcher: support inotify

"git diff" on webkit:

        no file watcher  1st run   subsequent runs
real        0m1.361s    0m1.445s      0m0.691s
user        0m0.889s    0m0.940s      0m0.649s
sys         0m0.469s    0m0.495s      0m0.040s
pkt-line.c: increase buffer size to 8192

test "very long name in the index handled sanely" in t0000 stretches
it a bit with a pathname of 4096 char length. Because we pass paths to
the file watcher using pkt-line protocol, it dies with "protocol
error: impossibly long line". Increase the buffer size a bit.
t1301: shutdown file watcher in shared repo test

$GIT_DIR/index.watcher violates the rules with perm srwxrwxr-x
1