Add git-list-files, a user friendly version of ls-files and more

This is more user friendly version of ls-files:

* it's automatically colored and columnized
* it refreshes the index like all porcelain commands
* it defaults to non-recursive behavior like ls
* :(glob) is on by default so '*.c' means a.c but not a/b.c, use
  '**/*.c' for that.
* auto pager

The name 'ls' is not taken. It is left for the user to make an alias
with better default options.
list-files: -u does not imply showing stages

Showing full index entry information is something for ls-files
only. The users of "git list-files" may just want to know what entries
are not unmerged.
list-files: add -R/--recursive short for --max-depth=-1
list-files: add -1 short for --no-column
list-files: add -t back

Tag "H" (cached) is not shown though because it's usually the majority
and becomes noise. Not showing it makes the other tags stand out. -t
is on by default if more than one file category is selected.
list-files: sort output and remove duplicates

When you mix different file types, with ls-files you may get separate
listing. For example, "ls-files -cm" will show file "abc" twice: one
as part of cached list, one of modified list. With "ls" (and this
patch) they will be in a single sorted list (easier for the eye).

Duplicate entries are also removed. Note that display content is
compared, so if you have "-t" on, or you color file types differently,
you will get duplicate textual entries. This is good imo.
list-files: do not show duplicate cached entries

With the current show_files() "list-files -tcm" will show

  foo.c
M foo.c

The first item is redundant. If "foo.c" is modified, we know it's in
the cache. Introduce show_files_compact to do that because ls-files is
plumbing and scripts may already depend on current display behavior.

Another difference in show_files_compact() is it does not show
skip-worktree (aka outside sparse checkout) entries anymore, which
makes sense in porcelain context.
list-files: show directories as well as files

The index does not store directories explicitly (except submodules) so
we have to figure them out from file list. The function
show_directories() deliberately generates duplicate directories and
expects the previous patch to remove duplicates.
list-files: add -F/--classify

This appends an indicator after the file name if it's executable, a
directory and so on, like in GNU ls. In fact append_indicator() is a
rewrite from get_type_indicator() in coreutils.git commit
7326d1f1a67edf21947ae98194f98c38b6e9e527.
list-files -F: show submodules with the new indicator '&'
list-files: -M aka diff-cached
column: support "denser" mode

Sometimes a few long entries in the listing would stretch out columns,
wasting space. If these entries are cut out, the columns could become
smaller and more columns can be displayed.

This mode checks the shortest and longest entries of a range of
items. If the difference is over a limit, the range is cut out for
displaying. This makes sure a group or long file names won't break
make any columns too wide.

Try "git ls -R t" with --column=denser and --column=dense to see the
difference.