git stash: Avoid data loss when saving a stash

I have been recently again bitten by git stash versus an uncommitted
file-to-directory change that happily threw away few gigabytes of my
data. This has been recently discussed in the past, e.g.

	http://thread.gmane.org/gmane.comp.version-control.git/202332/

and I implemented Junio's suggestion in this patch - if ls-files --killed
produced any output, the stash save is stopped and the user is required
to pass --force to really have the data removed.

A test case is included. I think that the (currently failing) tests
'stash file to directory' and 'stash directory to file' are related to
this, but I consider their expectation to be bogus - I would personally
be surprised by `git stash` suddenly importing the complete
never-meant-to-be-tracked contents of my directory to Git during a stash
save, and I think the approach of aborting in this situation is more
reasonable.

Other parts of Git also behave in a troublesome way in case of tracked
file being replaced by an untracked directory - I wouldn't expect `git
reset --hard` alone to remove the directory (i.e. touch untracked files)
either. However, this matter is much more complicated since `git reset
--hard` is assumed to "never fail in ordinary circumstances" (see e.g.
git-stash code ;-) and I'm unable to devote sufficient effort to seeing
such a change through.

Signed-off-by: Petr Baudis <pasky@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
SQUASH??? minimum fixes
SQUASH??? no need to limit tests on symlink capable platforms
tests: work around "stash save --force" fallout

Until "ls-files --killed" is taught to recognize submodules,
stashing with a repository with submodules is broken with the
earlier patch.  This just works around to see how bad the damage
is, until "ls -files --killed" is updated.
treat_directory(): do not declare submodules in index to be untracked

When the working tree walker encounters a directory, it asks this
function if it should descend into it, show it as an untracked
directory, or do something else.  When the directory is the top of
the submodule working tree, we used to say "That is an untracked
directory", which was quite bogus.  It is an entity that is tracked
in the index of the repository we are looking at, and that is not to
be descended into it.  Return path_none.

The existing case that path_untracked is returned for a newly
discovered submodule that is not tracked in the index (this only
happens when DIR_NO_GITLINKS option is not used) is unchanged and
returns path_untracked, but that is exactly because the submodule is
not tracked in the index.

The changes we made temporarily to the tests to work around
breakages in the previous patch can now be reverted.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add --depth to submodule update/add

When a submodule is clone, clone it width the --depth flag. This is useful
when the submodule(s) are huge and you're not really interested in anything
but the latest commit.

Tests are added and to make --depth work the path for test "setup a submodule
tree" had to be modified. Also did some indent adjustments to conform to the
rest of the testfile on "submodule update can handle symbolic links in pwd".

Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jk/commit-info-slab' into jch

Allow adding custom information to commit objects in order to
represent unbound number of flag bits etc.

* jk/commit-info-slab:
  commit-slab: introduce a macro to define a slab for new type
  commit-slab: avoid large realloc
  commit: allow associating auxiliary info on-demand
Merge branch 'jc/topo-author-date-sort' into jch

"git log" learned the "--author-date-order" option, with which the
output is topologically sorted and commits in parallel histories
are shown intermixed together based on the author timestamp.

* jc/topo-author-date-sort:
  t6003: add --author-date-order test
  topology tests: teach a helper to set author dates as well
  t6003: add --date-order test
  topology tests: teach a helper to take abbreviated timestamps
  t/lib-t6000: style fixes
  log: --author-date-order
  sort-in-topological-order: use prio-queue
  prio-queue: priority queue of pointers to structs
  toposort: rename "lifo" field

# Conflicts:
# 	.gitignore
# 	Makefile
# 	revision.c
Merge branch 'jk/submodule-subdirectory-ok' into jch

Allow various subcommands of "git submodule" to be run not from the
top of the working tree of the superproject.

* jk/submodule-subdirectory-ok:
  submodule: drop the top-level requirement
  rev-parse: add --prefix option
  submodule: show full path in error message
  t7403: add missing && chaining
  t7403: modernize style
  t7401: make indentation consistent
Merge branch 'nk/name-rev-abbreviated-refs' into jch

"git name-rev --refs=tags/v*" were forbidden, which was a bit
inconvenient (you had to give a pattern to match refs fully, like
--refs=refs/tags/v*).

* nk/name-rev-abbreviated-refs:
  name-rev: allow to specify a subpath for --refs option
Merge branch 'mh/loose-refs-race-with-pack-ref' into jch

We read loose and packed rerferences in two steps, but after
deciding to read a loose ref but before actually opening it to read
it, another process racing with us can unlink it, which would cause
us to barf. Update the codepath to retry when such a race is
detected.

* mh/loose-refs-race-with-pack-ref:
  resolve_ref_unsafe(): close race condition reading loose refs
  resolve_ref_unsafe(): handle the case of an SHA-1 within loop
  resolve_ref_unsafe(): extract function handle_missing_loose_ref()
Merge branch 'ap/diff-ignore-blank-lines' into jch

"git diff" learned a mode that ignores hunks whose change consists
only of additions and removals of blank lines, which is the same as
"diff -B" (ignore blank lines) of GNU diff.

* ap/diff-ignore-blank-lines:
  diff: add --ignore-blank-lines option
Merge branch 'mh/ref-races' into jch

"git pack-refs" that races with new ref creation or deletion have
been susceptible to lossage of refs under right conditions, which
has been tightened up.

* mh/ref-races:
  for_each_ref: load all loose refs before packed refs
  get_packed_ref_cache: reload packed-refs file when it changes
  add a stat_validity struct
  Extract a struct stat_data from cache_entry
  packed_ref_cache: increment refcount when locked
  do_for_each_entry(): increment the packed refs cache refcount
  refs: manage lifetime of packed refs cache via reference counting
  refs: implement simple transactions for the packed-refs file
  refs: wrap the packed refs cache in a level of indirection
  pack_refs(): split creation of packed refs and entry writing
  repack_without_ref(): split list curation and entry writing
Merge branch 'wk/doc-in-linux-3.x-era' into jch

Update documentation to match more recent realities.

* wk/doc-in-linux-3.x-era:
  Documentation: Update 'linux-2.6.git' -> 'linux.git'
  Documentation: Update the NFS remote examples to use the staging repo
  doc/clone: Pick more compelling paths for the --reference example
  doc/clone: Remove the '--bare -l -s' example
Merge branch 'tr/test-v-and-v-subtest-only' into jch

Allows N instances of tests run in parallel, each running 1/N parts
of the test suite under Valgrind, to speed things up.

The tip two may be useful in practice but is a tad ugly ;-)
Rerolled to fix breakage in non-prove tests.

* tr/test-v-and-v-subtest-only:
  test-lib: support running tests under valgrind in parallel
  test-lib: allow prefixing a custom string before "ok N" etc.
  test-lib: valgrind for only tests matching a pattern
  test-lib: verbose mode for only tests matching a pattern
  test-lib: self-test that --verbose works
  test-lib: rearrange start/end of test_expect_* and test_skip
  test-lib: refactor $GIT_SKIP_TESTS matching
  test-lib: enable MALLOC_* for the actual tests
Merge branch 'rr/remote-branch-config-refresh' into jch

The original way to specify remote repository using .git/branches/
used to have a nifty feature.  The code to support the feature was
still in a function but the caller was changed not to call it 5
years ago, breaking that feature and leaving the supporting code
unreachable.

* rr/remote-branch-config-refresh:
  t/t5505-remote: test multiple push/pull in remotes-file
  ls-remote doc: don't encourage use of branches-file
  ls-remote doc: rewrite <repository> paragraph
  ls-remote doc: fix example invocation on git.git
  t/t5505-remote: test url-with-# in branches-file
  remote: remove dead code in read_branches_file()
  t/t5505-remote: use test_path_is_missing
  t/t5505-remote: test push-refspec in branches-file
  t/t5505-remote: modernize style
Merge branch 'jc/triangle-push-fixup' into jch

Earlier remote.pushdefault (and per-branch branch.*.pushremote)
were introduced as an additional mechanism to choose what
repository to push into when "git push" did not say it from the
command line, to help people who push to a repository that is
different from where they fetch from.  This attempts to finish that
topic by teaching the default mechanism to choose branch in the
remote repository to be updated by such a push.

The 'current', 'matching' and 'nothing' modes (specified by the
push.default configuration variable) extend to such a "triangular"
workflow naturally, but 'upstream' and 'simple' have to be updated.

. 'upstream' is about pushing back to update the branch in the
remote repository that the current branch fetches from and
integrates with, it errors out in a triangular workflow.

. 'simple' is meant to help new people by avoiding mistakes, and
will be the safe default in Git 2.0.  In a non-triangular
workflow, it will continue to act as a cross between 'upstream'
and 'current' in that it pushes to the current branch's
@{upstream} only when it is set to the same name as the current
branch (e.g. your 'master' forks from the 'master' from the
central repository).  In a triangular workflow, this series
tentatively defines it as the same as 'current', but we may have
to tighten it to avoid surprises in some way.

* jc/triangle-push-fixup:
  t/t5528-push-default: test pushdefault workflows
  t/t5528-push-default: generalize test_push_*
  push: change `simple` to accommodate triangular workflows
  config doc: rewrite push.default section
  t/t5528-push-default: remove redundant test_config lines
Merge branch 'jx/clean-interactive' (early part) into jch

* 'jx/clean-interactive' (early part):
  git-clean: refactor git-clean into two phases
  write_name{_quoted_relative,}(): remove redundant parameters
  quote_path_relative(): remove redundant parameter
  quote.c: substitute path_relative with relative_path
  path.c: refactor relative_path(), not only strip prefix
  test: add test cases for relative_path
Merge branch 'rr/rebase-checkout-reflog' (early part) into jch

* 'rr/rebase-checkout-reflog' (early part):
  status: do not depend on rebase reflog messages
  t/t2021-checkout-last: "checkout -" should work after a rebase finishes
  wt-status: remove unused field in grab_1st_switch_cbdata
  t7512: test "detached from" as well
Merge branch 'sg/bash-prompt' into jch

* sg/bash-prompt:
  bash prompt: mention that PROMPT_COMMAND mode is faster
  bash prompt: avoid command substitution when finalizing gitstring
  bash prompt: avoid command substitution when checking for untracked files
  bash prompt: use bash builtins to check stash state
  bash prompt: use bash builtins to check for unborn branch for dirty state
  bash prompt: combine 'git rev-parse' for detached head
  bash prompt: combine 'git rev-parse' executions in the main code path
  bash prompt: use bash builtins to find out current branch
  bash prompt: use bash builtins to find out rebase state
  bash prompt: run 'git rev-parse --git-dir' directly instead of __gitdir()
  bash prompt: return early from __git_ps1() when not in a git repository
  bash prompt: print unique detached HEAD abbreviated object name
  bash prompt: add a test for symbolic link symbolic refs
  completion, bash prompt: move __gitdir() tests to completion test suite
  bash prompt: use 'write_script' helper in interactive rebase test
  bash prompt: fix redirection coding style in tests