pkt-line.c: add packet_read_timeout()
Add git-file-watcher and basic connection handling logic

git-file-watcher is a daemon (*) that watches file changes and tells
git about them. The intent is to avoid lstat() calls at index refresh
time, which could be a lot on big working directory.

The actual monitoring needs support from OS (inotify, FSEvents,
FindFirstChangeNotification or kqueue) and is not part of this patch
or the next few yet. This patch only provides a UNIX socket server.

(*) it will be a a daemon in this end, but in this patch it runs in
the foreground.
file-watcher: remove socket on exit
file-watcher: add --detach

In daemon mode, stdout and stderr are saved in $WATCHER/log.
read-cache: save trailing sha-1

This will be used as signature to know if the index has changed.
read-cache: new flag CE_WATCHED to mark what file is watched

This bit is basically "dynamic CE_VALID". It marks entries that are
being watched by the incoming file-watcher. When an index is loaded,
file watcher is contacted and the list of updated paths is retrieved.

These paths will have CE_WATCHED cleared and lstat() will be called on
them. Those that have CE_WATCHED and not in the list will have
CE_VALID turn on to skip lstat(). The setting is temporarily, CE_VALID
is not saved to disk if CE_WATCHED is also set.

We keep the CE_WATCHED in a new extension, separated from the entries
to save some space because extended ce_flags adds 2 bytes per entry
and this flag would be present in the majority of entries. When stored
as bitmap, this extension could compress very well with ewah algorithm.
Clear CE_WATCHED when set CE_VALID alone

If CE_WATCHED is set, CE_VALID is controlled by CE_WATCHED and will be
cleared bfore writing to disk. Users of --assume-unchanged therefore
need to clear CE_WATCHED to avoid this side effect.
read-cache: basic file-watcher hand shaking

read_cache() connects to the file watcher, specified by
filewatcher.path config, and performs basic hand shaking. CE_WATCHED
is cleared if git and file watcher have different views on the index
state.

All send/receive calls must be complete within a limited time to avoid
a buggy file-watcher hang "git status" forever. And the whole point of
doing this is speed. If file watcher can't respond fast enough, for
whatever reason, then it should not be used.

TODO: support ~user in filewatcher.path. People might want to set it
to ~/.config/git/watcher or something
read-cache: ask file watcher to watch files

We want to watch files that are never changed because lstat() on those
files is a wasted effort. So we sort unwatched files by date and start
adding them to the file watcher until it barfs (e.g. hits inotify
limit).

Note that we still do lstat() on these new watched files because they
could have changed before the file watcher could watch them. Watched
files may only skip lstat() at the next run.

In the future, file watcher should figure out what paths are
watchable, what not (e.g. network filesystems) and reject them. For
now it's the user resposibility to set (or unset) filewatcher.path
properly.
read-cache: put some limits on file watching

watch_entries() is a lot of computation and could trigger a lot more
lookups in file-watcher. Normally after the first set of watches are
in place, we do not need to update often. Moreover if the number of
entries is small, the overhead of file watcher may actually slow git
down.

This patch only allows to update watches if the number of watchable
files is over a limit (and there are new files added if this is not
the first time). Measurements on Core i5-2520M and Linux 3.7.6, about
920 lstat() take 1ms. Somewhere between 2^16 and 2^17 lstat calls that
it starts to take longer than 100ms. 2^16 is chosen at the minimum
limit to start using file watcher.

Recently updated files are not considered watchable because they are
likely to be updated again soon, not worth the ping-pong game with
file watcher. Recent limit is ignored if there are no watched files
(e.g. fresh clone). The default limit 10min is just a random value.
read-cache: get changed file list from file watcher

When some paths are watched, they are added to the "watched" list in
file watcher. When a path in this list is updated, the path is moved
to "changed" list and no longer watched.

After loading the index, we could ask for this "changed"
list. CE_WATCHED entries that are not in the list are not changed and
will have CE_VALID set. CE_WATCHED in the list will have CE_WATCHED
removed.

The "changed" list remains until told to delete something and that
only happens after the index is written down (i.e. new CE_WATCHED set
saved).
git-compat-util.h: add inotify stubs on non-Linux platforms

This is to avoid spreading #ifdef HAVE_INOTIFY around and keep most
code compiled even if it's never active.
file-watcher: inotify support, watching part

"struct dir" manages inotify file descriptor and forms a tree. "struct
file" manages a file. When a file is watched, all dirs up to the file
is watched. Any changes on a directory impacts all subdirs and files.

The way data structure is made might be inotify-specific. I haven't
thought of how other file notification mechanisms may be
implemented. So there may be some refactoring later when a new OS is
supported.
file-watcher: inotify support, notification part
file-watcher: tests for the daemon

test-file-watcher is a simplied chat program to talk to file
watcher. Typically you would write something like this

  cat >expect <<EOF
  # send "hello". Oh and this is a comment!
  <hello
  # Wait for reply and print to stdout.
  # test-file-watcher does not care about anything after '>'
  >hello
  <index foo bar
  >ok
  EOF
  test-file-watcher . <expect >actual

and test-file-watcher will execute the commands and get responses. If
all go well, "actual" should be the same as "expect". '<' and '>'
denote send and receive packets respectively. '<<' and '>>' assume the
packets are in form: "something SP PKT-LINE PKT-LINE..." and it will
encode or decode the inner PKT-LINE for you.

$GIT_TEST_WATCHER enables a few more commands for testing purposes.
The most important one is 'test-mode' where system inotify is taken
out and inotify events could be injected via test-file-watcher.

There are two debug commands in file-watcher that's not used by the
test suite, but would help debugging: setenv and log. They can be used
to turn on GIT_TRACE_PACKET then any "log" command will show, which
functions as barrier between events file watcher.

GIT_TRACE_WATCHER can also be enabled (dynamically or at startup) to
track inotify events.
file-watcher: debugging support on the client side

This is to force running the test suite with file-watcher with

git-file-watcher /tmp/

then open another terminal and run

GIT_TEST_WATCHER=2 GIT_TEST_WATCHER_PATH=/tmp/ make test

TIME_WAIT set set to unlimited by GIT_TEST_WATCHER=2 so the test suite
could hang up indefinitely due to a file-watcher bug..