submodule: Explicit local branch creation in module_clone

The previous code only checked out branches in cmd_add.  This commit
moves the branch-checkout logic into module_clone, where it can be
shared by cmd_add and cmd_update.  I also update the initial checkout
command to use 'reset' to preserve branches setup during module_clone.

With this change, folks cloning submodules for the first time via:

  $ git submodule update ...

will get a local branch instead of a detached HEAD, unless they are
using the default checkout-mode updates.  This is a change from the
previous situation where cmd_update always used checkout-mode logic
(regardless of the requested update mode) for updates that triggered
an initial clone, which always resulted in a detached HEAD.

This commit does not change the logic for updates after the initial
clone, which will continue to create detached HEADs for checkout-mode
updates, and integrate remote work with the local HEAD (detached or
not) in other modes.

The motivation for the change is that developers doing local work
inside the submodule are likely to select a non-checkout-mode for
updates so their local work is integrated with upstream work.
Developers who are not doing local submodule work stick with
checkout-mode updates so any apparently local work is blown away
during updates.  For example, if upstream rolls back the remote branch
or gitlinked commit to an earlier version, the checkout-mode developer
wants their old submodule checkout to be rolled back as well, instead
of getting a no-op merge/rebase with the rolled-back reference.

By using the update mode to distinguish submodule developers from
black-box submodule consumers, we can setup local branches for the
developers who will want local branches, and stick with detached HEADs
for the developers that don't care.

Testing
=======

In t7406, just-cloned checkouts now update to the gitlinked hash with
'reset', to preserve the local branch for situations where we're not
on a detached HEAD.

I also added explicit tests to t7406 for HEAD attachement after
cloning updates, showing that it depends on their update mode:

* Checkout-mode updates get detached HEADs
* Everyone else gets a local branch, matching the configured
  submodule.<name>.branch and defaulting to master.

The 'initial-setup' tag makes it easy to reset the superproject to a
known state, as several earlier tests commit to submodules and commit
the changed gitlinks to the superproject, but don't push the new
submodule commits to the upstream subprojects.  This makes it
impossible to checkout the current super master, because it references
submodule commits that don't exist in the upstream subprojects.  For a
specific example, see the tests that currently generate the
'two_new_submodule_commits' commits.

Documentation
=============

I updated the docs to describe the 'submodule update' modes in detail.
The old documentation did not distinguish between cloning and
non-cloning updates and lacked clarity on which operations would lead
to detached HEADs, and which would not.  The new documentation
addresses these issues while updating the docs to reflect the changes
introduced by this commit's explicit local branch creation in
module_clone.

I also add '--checkout' to the usage summary and group the update-mode
options into a single set.

Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation: Describe 'submodule update --remote' use case

Make it clear that there is no implicit floating going on; --remote
lets you explicitly integrate the upstream branch in your current
HEAD (just like running 'git pull' in the submodule).  The only
distinction with the current 'git pull' is the config location and
setting used for the upstream branch, which is hopefully clear now.

Signed-off-by: W. Trevor King <wking@tremily.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3030-merge-recursive: Test known breakage with empty work tree

Sometimes when working with a large repository it can be useful to try
out a merge and only check out conflicting files to disk (for example as
a speed optimization on a server).  Until v1.7.7-rc1~28^2~20
(merge-recursive: When we detect we can skip an update, actually skip
it, 2011-08-11), it was possible to do so with the following idiom:

	# Prepare a temporary index and empty work tree.
	GIT_INDEX_FILE="$PWD/tmp-$$-index" &&
	export GIT_INDEX_FILE &&
	GIT_WORK_TREE="$PWD/tmp-$$-work" &&
	export GIT_WORK_TREE &&
	mkdir "$GIT_WORK_TREE" &&

	# Convince the index that our side is on disk.
	git read-tree -i -m $ours &&
	git update-index --ignore-missing --refresh &&

	# Merge their side into our side.
	bases=$(git merge-base --all $ours $theirs) &&
	git merge-recursive $bases -- $ours $theirs &&
	tree=$(git write-tree)

Nowadays, that still works and the exit status is the same, but
merge-recursive produces a diagnostic if "our" side renamed a file:

	error: addinfo_cache failed for path 'dst'

Add a test to document this regression.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read-cache.c: Refactor --ignore-missing implementation

Move lstat ENOENT handling from refresh_index to refresh_cache_ent and
activate it with a new CE_MATCH_IGNORE_MISSING option.  This will allow
other call paths into refresh_cache_ent to use the feature.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read-cache.c: Extend make_cache_entry refresh flag with options

Convert the make_cache_entry boolean 'refresh' argument to a more
general 'refresh_options' argument.  Pass the value through to the
underlying refresh_cache_ent call.  Add option CE_MATCH_REFRESH to
enable stat refresh.  Update call sites to use the new signature.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
merge-recursive.c: Tolerate missing files while refreshing index

Teach add_cacheinfo to tell make_cache_entry to skip refreshing stat
information when a file is missing from the work tree.  We do not want
the index to be stat-dirty after the merge but also do not want to fail
when a file happens to be missing.

This fixes the 'merge-recursive w/ empty work tree - ours has rename'
case in t3030-merge-recursive.

Suggested-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move diffcore_skip_stat_unmatch core logic out for reuse later

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff: do not quit early on stat-dirty files

When QUICK is set (i.e. with --quiet) we try to do as little work as
possible, stopping after seeing the first change. stat-dirty is
considered a "change" but it may turn out not, if no actual content is
changed. The actual content test is performed too late in the process
and the shortcut may be taken prematurely, leading to incorrect return
code.

Assume we do "git diff --quiet". If we have a stat-dirty file "a" and
a really dirty file "b". We break the loop in run_diff_files() and
stop after "a" because we have got a "change". Later in
diffcore_skip_stat_unmatch() we find out "a" is actually not
changed. But there's nothing else in the diff queue, we incorrectly
declare "no change", ignoring the fact that "b" is changed.

This also happens to "git diff --quiet HEAD" when it hits
diff_can_quit_early() in oneway_diff().

This patch does the content test earlier in order to keep going if "a"
is unchanged. The test result is cached so that when
diffcore_skip_stat_unmatch() is done in the end, we spend no cycles on
re-testing "a".

Reported-by: IWAMOTO Toshihiro <iwamoto@valinux.co.jp>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff: turn off skip_stat_unmatch on "diff --cached"

skip_stat_unmatch flag is added in fb13227 (git-diff: squelch "empty"
diffs - 2007-08-03) to ignore empty diffs caused by stat-only
dirtiness. In "diff --cached" case, stat is not invovled at all. While
the code is written in a way that no expensive I/O is done, we still
need to move all file pairs from the old queue to the new queue in
diffcore_skip_stat_unmatch(). Avoid it.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'kb/fast-hashmap' (early part) into jch

* 'kb/fast-hashmap' (early part):
  name-hash: retire unused index_name_exists()
  hashmap.h: Use 'unsigned int' for hash-codes everywhere
  Drop unnecessary #includes from test-hashmap
  Add test-hashmap to .gitignore
  read-cache.c: fix memory leaks caused by removed cache entries
  builtin/update-index.c: cleanup update_one
  fix 'git update-index --verbose --again' output
  remove old hash.[ch] implementation
  name-hash.c: remove cache entries instead of marking them CE_UNHASHED
  name-hash.c: use new hash map implementation for cache entries
  name-hash.c: remove unreferenced directory entries
  name-hash.c: use new hash map implementation for directories
  diffcore-rename.c: use new hash map implementation
  diffcore-rename.c: simplify finding exact renames
  diffcore-rename.c: move code around to prepare for the next patch
  buitin/describe.c: use new hash map implementation
  add a hashtable implementation that supports O(1) removal
  submodule: don't access the .gitmodules cache entry after removing it
Merge branch 'nv/commit-gpgsign-config' into jch

Introduce commit.gpgsign configuration variable to force every
commit to be GPG signed.  The variable cannot be overriden from the
command line of some of the commands that create commits except for
"git commit" and "git commit-tree", but I am not convinced that it
is a good idea to sprinkle support for --no-gpg-sign everywhere.

* nv/commit-gpgsign-config:
  test the commit.gpgsign config option
  commit-tree: add and document --no-gpg-sign
  Add the commit.gpgsign option to sign all commits
Merge branch 'jk/pack-bitmap' into jch

Borrows the bitmap index into packfiles from JGit to speed up
enumeration of objects involved in a commit range without having to
fully traverse the history.

* jk/pack-bitmap: (25 commits)
  ewah: support platforms that require aligned reads
  read-cache: use get_be32 instead of hand-rolled ntoh_l
  block-sha1: factor out get_be and put_be wrappers
  do not discard revindex when re-preparing packfiles
  pack-bitmap: implement optional name_hash cache
  t/perf: add tests for pack bitmaps
  t: add basic bitmap functionality tests
  count-objects: recognize .bitmap in garbage-checking
  repack: consider bitmaps when performing repacks
  repack: handle optional files created by pack-objects
  repack: turn exts array into array-of-struct
  repack: stop using magic number for ARRAY_SIZE(exts)
  pack-objects: implement bitmap writing
  rev-list: add bitmap mode to speed up object lists
  pack-objects: use bitmaps when packing objects
  pack-objects: split add_object_entry
  pack-bitmap: add support for bitmap indexes
  documentation: add documentation for the bitmap format
  ewah: compressed bitmap implementation
  compat: add endianness helpers
  ...

# Conflicts:
# 	builtin/pack-objects.c
Merge branch 'da/pull-ff-configuration' into jch

"git pull" learned to pay attention to pull.ff configuration
variable.

* da/pull-ff-configuration:
  pull: add --ff-only to the help text
  pull: add pull.ff configuration
### 2.0
Merge branch 'jc/push-2.0-default-to-simple' into jch

* jc/push-2.0-default-to-simple:
  push: switch default from "matching" to "simple"

# Conflicts:
# 	Documentation/config.txt
# 	Documentation/git-push.txt
# 	builtin/push.c
Merge branch 'jn/add-2.0-u-A-sans-pathspec' into jch

* jn/add-2.0-u-A-sans-pathspec:
  git add: -u/-A now affects the entire working tree

# Conflicts:
# 	builtin/add.c
Merge branch 'jc/add-2.0-ignore-removal' into jch

Updated endgame for "git add <pathspec>" that defaults to "--all"
aka "--no-ignore-removal".

* jc/add-2.0-ignore-removal:
  git add <pathspec>... defaults to "-A"

# Conflicts:
# 	builtin/add.c
Merge branch 'jc/core-checkstat-2.0' into jch

* jc/core-checkstat-2.0:
  core.statinfo: remove as promised in Git 2.0
Merge branch 'gj/push-more-verbose-advice' into jch

Explain 'simple' and 'matching' in "git push" advice message; the
topmost patch is a rebase of jc/push-2.0-default-to-simple on top
of it.

* gj/push-more-verbose-advice:
  push: switch default from "matching" to "simple"
Merge branch 'jc/hold-diff-remove-q-synonym-for-no-deletion' into jch

* jc/hold-diff-remove-q-synonym-for-no-deletion:
  diff: remove "diff-files -q" in a version of Git in a distant future

# Conflicts:
# 	diff-no-index.c
# 	diff.c