name-hash.c: fix endless loop with core.ignorecase=true

With core.ignorecase=true, name-hash.c builds a case insensitive index of
all tracked directories. Currently, the existing cache entry structures are
added multiple times to the same hashtable (with different name lengths and
hash codes). However, there's only one dir_next pointer, which gets
completely messed up in case of hash collisions. In the worst case, this
causes an endless loop if ce == ce->dir_next:

    # "V/", "V/XQANY/" and "WURZAUP/" all have the same hash_name
    mkdir V
    mkdir V/XQANY
    mkdir WURZAUP
    touch V/XQANY/test
    git init
    git config core.ignorecase true
    git add .
    git status

Use a separate hashtable and separate structures for the directory index
so that each directory entry has its own next pointer. Use reference
counting to track which directory entry contains files.

There are only slight changes to the name-hash.c API:
- new free_name_hash() used by read_cache.c::discard_index()
- remove_name_hash() takes an additional index_state parameter
- index_name_exists() for a directory (trailing '/') may return a cache
  entry that has been removed (CE_UNHASHED). This is not a problem as the
  return value is only used to check if the directory exists (dir.c) or to
  normalize casing of directory names (read-cache.c).

Getting rid of cache_entry.dir_next reduces memory consumption, especially
with core.ignorecase=false (which doesn't use that member at all).

With core.ignorecase=true, building the directory index is slightly faster
as we add / check the parent directory first (instead of going through all
directory levels for each file in the index). E.g. with WebKit (~200k
files, ~7k dirs), time taken in lazy_init_name_hash is reduced from 176ms
to 130ms.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'kb/name-hash' into jch

* kb/name-hash:
  name-hash.c: fix endless loop with core.ignorecase=true
Merge branch 'jn/shell-disable-interactive' into pu

Expecting a reroll.
$gmane/216229

* jn/shell-disable-interactive:
  shell: pay attention to exit status from 'help' command
  shell doc: emphasize purpose and security model
Merge branch 'nd/branch-show-rebase-bisect-state' into pu

Expecting a reroll.
$gmane/215771

* nd/branch-show-rebase-bisect-state:
  branch: show rebase/bisect info when possible instead of "(no branch)"
Merge branch 'mb/gitweb-highlight-link-target' into pu

Expecting a reroll.
$gmane/211935

* mb/gitweb-highlight-link-target:
  Highlight the link target line in Gitweb using CSS
Merge branch 'mb/remote-default-nn-origin' into pu

When the user does not specify what remote to interact with, we
often attempt to use 'origin'.  This can now be customized via a
configuration variable.

Expecting a reroll.
$gmane/210151

"The first remote becomes the default" bit is better done as a
separate step.

* mb/remote-default-nn-origin:
  Teach get_default_remote to respect remote.default.
  Test that plain "git fetch" uses remote.default when on a detached HEAD.
  Teach clone to set remote.default.
  Teach "git remote" about remote.default.
  Teach remote.c about the remote.default configuration setting.
  Rename remote.c's default_remote_name static variables.

Conflicts:
	builtin/remote.c
	t/t5505-remote.sh
Merge branch 'gp/forbid-describe-all-match' into pu

Not sure if we should forbid the combination or explain the
semantics better, perhaps by giving a better synonym to "--all",
which meant "disable the usual limit that only gives the annotated
tags".

* gp/forbid-describe-all-match:
  describe: make --all and --match=PATTERN mutually incompatible
Merge branch 'mg/unsigned-time-t' into pu

A few workarounds for systems with unsigned time_t.

* mg/unsigned-time-t:
  Fix time offset calculation in case of unsigned time_t
  date.c: fix unsigned time_t comparison