safe_create_leading_directories(): fix format of "if" chaining

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): reduce scope of local variable

This makes it more obvious that values of "st" don't persist across
loop iterations.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): add explicit "slash" pointer

Keep track of the position of the slash character independently of
"pos", thereby making the purpose of each variable clearer and
working towards other upcoming changes.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): rename local variable

Rename "pos" to "next_component", because now it always points at the
next component of the path name that has to be processed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): split on first of multiple slashes

If the input path has multiple slashes between path components (e.g.,
"foo//bar"), then the old code was breaking the path at the last
slash, not the first one.  So in the above example, the second slash
was overwritten with NUL, resulting in the parent directory being
sought as "foo/".

When stat() is called on "foo/", it fails with ENOTDIR if "foo" exists
but is not a directory.  This caused the wrong path to be taken in the
subsequent logic.

So instead, split path components at the first intercomponent slash
rather than the last one.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): always restore slash at end of loop

Always restore the slash that we scribbled over at the end of the
loop, rather than also fixing it up at each premature exit from the
loop.  This makes it harder to forget to do the cleanup as new paths
are added to the code.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): introduce enum for return values

Instead of returning magic integer values (which a couple of callers
go to the trouble of distinguishing), return values from an enum.  Add
a docstring.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cmd_init_db(): when creating directories, handle errors conservatively

safe_create_leading_directories_const() returns a non-zero value on
error.  The old code at this calling site recognized a couple of
particular error values, and treated all other return values as
success.  Instead, be more conservative: recognize the errors we are
interested in, but treat any other nonzero values as failures.  This
is more robust in case somebody adds another possible return value
without telling us.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): add new error value SCLD_VANISHED

Add a new possible error result that can be returned by
safe_create_leading_directories() and
safe_create_leading_directories_const(): SCLD_VANISHED.  This value
indicates that a file or directory on the path existed at one point
(either it already existed or the function created it), but then it
disappeared.  This probably indicates that another process deleted the
directory while we were working.  If SCLD_VANISHED is returned, the
caller might want to retry the function call, as there is a chance
that a new attempt will succeed.

Why doesn't safe_create_leading_directories() do the retrying
internally?  Because an empty directory isn't really ever safe until
it holds a file.  So even if safe_create_leading_directories() were
absolutely sure that the directory existed before it returned, there
would be no guarantee that the directory still existed when the caller
tried to write something in it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
lock_ref_sha1_basic(): on SCLD_VANISHED, retry

If safe_create_leading_directories() fails because a file along the
path unexpectedly vanished, try again (up to 3 times).

This can occur if another process is deleting directories at the same
time as we are trying to make them.  For example, "git pack-refs
--all" tries to delete the loose refs and any empty directories that
are left behind.  If a pack-refs process is running, then it might
delete a directory that we need to put a new loose reference in.

If safe_create_leading_directories() thinks this might have happened,
then take its advice and try again (maximum three attempts).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
lock_ref_sha1_basic(): if locking fails with ENOENT, retry

If hold_lock_file_for_update() fails with errno==ENOENT, it might be
because somebody else (for example, a pack-refs process) has just
deleted one of the lockfile's ancestor directories.  So if this
condition is detected, try again (up to 3 times).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
remove_dir_recurse(): tighten condition for removing unreadable dir

If opendir() fails on the top-level directory, it makes sense to try
to delete it anyway--but only if the failure was due to EACCES.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
remove_dir_recurse(): handle disappearing files and directories

If a file or directory that we are trying to remove disappears (e.g.,
because another process has pruned it), do not consider it an error.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rename_ref(): extract function rename_tmp_log()

It's about to become a bit more complex.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rename_tmp_log(): handle a possible mkdir/rmdir race

If a directory vanishes while renaming the temporary reflog file,
retry (up to 3 times).  This could happen if another process deletes
the directory created by safe_create_leading_directories() just before
we rename the file into the directory.

As far as I can tell, this race could not occur internal to git.  The
only time that a directory under $GIT_DIR/logs is deleted is if room
has to be made for a log file for a reference with the same name;
for example, in the following sequence:

    git branch foo/bar    # Creates file .git/logs/refs/heads/foo/bar
    git branch -d foo/bar # Deletes file but leaves .git/logs/refs/heads/foo/
    git branch foo        # Deletes .git/logs/refs/heads/foo/

But the only reason the last command deletes the directory is because
it wants to create a file with the same name.  So if another process
(e.g.,

    git branch foo/baz

) wants to create that directory, one of the two is doomed to failure
anyway because of a D/F conflict.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rename_tmp_log(): limit the number of remote_empty_directories() attempts

This doesn't seem to be a likely error, but we've got the counter
anyway, so we might as well use it for an added bit of safety.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rename_tmp_log(): on SCLD_VANISHED, retry

If safe_create_leading_directories() fails because a file along the
path unexpectedly vanished, try again from the beginning.  Try at most
3 times.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
safe_create_leading_directories(): on Windows, \ can separate path components

When cloning to a directory "C:\foo\bar" from Windows' cmd.exe where
"foo" does not exist yet, Git would throw an error like

    fatal: could not create work tree dir 'c:\foo\bar'.: No such file or directory

Fix this by not hard-coding a platform specific directory separator
into safe_create_leading_directories().

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'kb/fast-hashmap' (early part) into jch

* 'kb/fast-hashmap' (early part):
  name-hash: retire unused index_name_exists()
  hashmap.h: Use 'unsigned int' for hash-codes everywhere
  Drop unnecessary #includes from test-hashmap
  Add test-hashmap to .gitignore
  read-cache.c: fix memory leaks caused by removed cache entries
  builtin/update-index.c: cleanup update_one
  fix 'git update-index --verbose --again' output
  remove old hash.[ch] implementation
  name-hash.c: remove cache entries instead of marking them CE_UNHASHED
  name-hash.c: use new hash map implementation for cache entries
  name-hash.c: remove unreferenced directory entries
  name-hash.c: use new hash map implementation for directories
  diffcore-rename.c: use new hash map implementation
  diffcore-rename.c: simplify finding exact renames
  diffcore-rename.c: move code around to prepare for the next patch
  buitin/describe.c: use new hash map implementation
  add a hashtable implementation that supports O(1) removal
  submodule: don't access the .gitmodules cache entry after removing it
Merge branch 'nd/negative-pathspec' into jch

Introduce "negative pathspec" magic, to allow "git log -- . ':!dir'" to
tell us "I am interested in everything but 'dir' directory".

* nd/negative-pathspec:
  pathspec.c: support adding prefix magic to a pathspec with mnemonic magic
  Support pathspec magic :(exclude) and its short form :!
  glossary-content.txt: rephrase magic signature part