mv: Fix spurious warning when moving a file in presence of submodules

In commit 0656781fa "git mv" learned to update the submodule path in the
.gitmodules file when moving a submodule in the work tree. But since that
commit update_path_in_gitmodules() gets called no matter if we moved a
submodule or a regular file, which is wrong and leads to a bogus warning
when moving a regular file in a repo containing a .gitmodules file:

    warning: Could not find section in .gitmodules where path=<filename>

Fix that by only calling update_path_in_gitmodules() when moving a
submodule. To achieve that, we introduce the special SUBMODULE_WITH_GITDIR
define to distinguish the cases where we also have to connect work tree
and git directory from those where we only need to update the .gitmodules
setting.

A test for submodules using a .git directory together with a .gitmodules
file has been added to t7001. Even though newer git versions will always
use a gitfile when cloning submodules, repositories cloned with older git
versions will still use this layout.

Reported-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
remote-hg: unquote C-style paths when exporting

git-fast-import documentation says that paths can be C-style quoted.
Unfortunately, the current remote-hg helper doesn't unquote quoted
path and pass them as-is to Mercurial when the commit is created.

This results in the following situation:

 - clone a mercurial repository with git
 - add a file with space in a directory: `>dir/foo\ bar`
 - commit that new file, and push the change to mercurial
 - the mercurial repository now has a new directory named '"dir',
   which contains a file named 'foo bar"'

Use Python str.decode('string-escape') to unquote the string if it
starts and ends with ".  It has been tested with quotes, spaces, and
utf-8 encoded file-names.

Signed-off-by: Antoine Pelisse <apelisse@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
add-interactive: handle unborn branch in patch mode

The list_modified function already knows how to handle an
unborn branch by diffing against the empty tree. However,
the diff we perform to get the actual hunks does not. Let's
use the same logic for both diffs.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
reset: pass real rev name to add--interactive

The add--interactive --patch mode adjusts the UI based on
whether we are pulling changes from HEAD or elsewhere (in
the former case it asks to unstage the reverse hunk, rather
than apply the forward hunk).

Commit 166ec2e taught reset to work on an unborn branch, but
in doing so, switched to always providing add--interactive
with the sha1 rather than the symbolic name. This meant we
always used the "apply" interface, even for "git reset -p
HEAD".

We can fix this by passing the symbolic name to
add--interactive.  Since it understands unborn branches
these days, we do not even have to cover this special case
ourselves; we can simply pass HEAD.

The tests in t7105 now check that the right interface is
used in each circumstance (and notice the regression from
166ec2e we are fixing). The test in t7106 checks that we
get this right for the unborn case, too (not a regression,
since it didn't work at all before, but a nice improvement).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
howto: add article on recovering a corrupted object

This is an asciidoc-ified version of a corruption post-mortem sent to
the git list. It complements the existing howto article, since it covers
a case where the object couldn't be easily recreated or copied from
elsewhere.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
refs: remove unused function invalidate_ref_cache

The function 'invalidate_ref_cache' was introduced in 79c7ca5 (2011-10-17,
invalidate_ref_cache(): rename function from invalidate_cached_refs())
by a rename and elevated to be publicly usable in 8be8bde (2011-10-17,
invalidate_ref_cache(): expose this function in the refs API)

However it is not used anymore, as 8bf90dc (2011-10-17, write_ref_sha1():
only invalidate the loose ref cache) and (much) later 506a760 (2013-04-22,
refs: change how packed refs are deleted) removed any calls to this
function. So it seems as if we don't need that function any more,
good bye!

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cache: remove unused function 'have_git_dir'

This function was added in d2b0708 (2008-09-27, add have_git_dir()
function) as a preparation for adbc0b6 (2008-09-30, cygwin: Use native
Win32 API for stat).

However the second referenced commit was reverted in f66450a (2013-06-22,
cygwin: Remove the Win32 l/stat() implementation), so we don't need to
expose this wrapper function any more as a public API.

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5300-pack-object: do not compare binary data using test_cmp

Users may set test_cmp to a comparison tool of their liking. The intent is
that the tool performs comparison of line-oriented texts. However, t5300
uses it also to compare binary data. Change those tests to use 'cmp'.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Windows: a test_cmp that is agnostic to random LF <> CRLF conversions

In a number of tests, output that was produced by a shell script is
compared to expected output using test_cmp. Unfortunately, the MSYS bash--
when invoked via git, such as in hooks--converts LF to CRLF on output
(as produced by echo and printf), which leads to many false positives.

Implements a diff tool that undoes the converted CRLF. To avoid that
sub-processes are spawned (which is very slow on Windows), the tool is
implemented as a shell function. Diff is invoked as usual only when a
difference is detected by the shell code.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: undo special treatment of CRLF for Windows

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change sed i\ usage to something Solaris' sed can handle

Solaris' sed was choking on the i\ commands used in
t4015-diff-whitespace as it couldn't parse the program properly.
Modify two uses of sed that worked in GNU sed but not Solaris'
(/usr/bin or /usr/xpg4/bin) to an equivalent form that is handled
properly by both.

Signed-off-by: Ben Walton <bdwalton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pathspec: stop --*-pathspecs impact on internal parse_pathspec() uses

Normally parse_pathspec() is used on command line arguments where it
can do fancy thing like parsing magic on each argument or adding magic
for all pathspecs based on --*-pathspecs options.

There's another use of parse_pathspec(), where pathspec is needed, but
the input is known to be pure paths. In this case we usually don't
want --*-pathspecs to interfere. And we definitely do not want to
parse magic in these paths, regardless of --literal-pathspecs.

Add new flag PATHSPEC_LITERAL_PATH for this purpose. When it's set,
--*-pathspecs are ignored, no magic is parsed. And if the caller
allows PATHSPEC_LITERAL (i.e. the next calls can take literal magic),
then PATHSPEC_LITERAL will be set.

This fixes cases where git chokes when GIT_*_PATHSPECS are set because
parse_pathspec() indicates it won't take any magic. But
GIT_*_PATHSPECS add them anyway. These are

   export GIT_LITERAL_PATHSPECS=1
   git blame -- something
   git log --follow something
   git log --merge

"git ls-files --with-tree=path" (aka parse_pathspec() in
overlay_tree_on_cache()) is safe because the input is empty, and
producing one pathspec due to PATHSPEC_PREFER_CWD does not take any
magic into account.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t/README: tests can use perl even with NO_PERL

The git build system supports a NO_PERL switch to avoid installing
perl bindings or other features (like "git add --patch") that rely on
perl on runtime, but even with NO_PERL it has not been possible for a
long time to run tests without perl.  Helpers such as

	nul_to_q () {
		"$PERL_PATH" -pe 'y/\000/Q/'
	}

use perl as a better tr or sed and are regularly used in tests without
worrying to add a PERL prerequisite.

Perl is portable enough that it seems fine to keep relying on it for
this kind of thing in tests (and more readable than the alternative of
trying to find POSIXy equivalents).  Update the test documentation to
clarify this.

Reported-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
doc/howto: warn about (dumb)http server document being too old

Describe when it is still applicable, and tell people where to go
for most normal cases.

Signed-off-by: Sitaram Chamarty <sitaram@atc.tcs.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t: provide a perl() function which uses $PERL_PATH

Once upon a time, we assumed that calling a bare "perl" in
the test scripts was OK, because we would find the perl from
the user's PATH, and we were only asking that perl to do
basic operations that work even on old versions of perl.

Later, we found that some systems really prefer to use
$PERL_PATH even for these basic cases, because the system
perl misbehaves in some way (e.g., by handling line endings
differently). We then switched "perl" invocations to
"$PERL_PATH" to respect the user's choice.

Having to use "$PERL_PATH" is ugly and cumbersome, though.
Instead, let's provide a perl() shell function that tests
can use, which will transparently do the right thing.

Unfortunately, test writers still have to use $PERL_PATH in
certain situations, so we still need to keep the advice in
the README.

Note that this may fix test failures in t5004, t5503, t6002,
t6003, t6300, t8001, and t8002, depending on your system's
perl setup. All of these can be detected by running:

  ln -s /bin/false bin-wrappers/perl
  make test

which fails before this patch, and passes after.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t: use perl instead of "$PERL_PATH" where applicable

As of the last commit, we can use "perl" instead of
"$PERL_PATH" when running tests, as the former is now a
function which uses the latter. As the shorter "perl" is
easier on the eyes, let's switch to using it everywhere.

This is not quite a mechanical s/$PERL_PATH/perl/
replacement, though. There are some places where we invoke
perl from a script we generate on the fly, and those scripts
do not have access to our internal shell functions. The
result can be double-checked by running:

  ln -s /bin/false bin-wrappers/perl
  make test

which continues to pass even after this patch.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cvsserver: Determinize output to combat Perl 5.18 hash randomization

Perl 5.18 randomizes the seed used by its hash function, so iterating
through hashes results in different orders from run to run:
  http://perldoc.perl.org/perl5180delta.html#Hash-overhaul

This usually broke t9400 (gitcvs.dbname, gitcvs.ext.dbname, when
running cmp on two .sqlite files) and t9402 (check [cvswork3] diff,
when running test_cmp on two diffs).

To fix this, hide the internal order of hashes with sort when sending
output or running database queries.

(An alternative workaround is PERL_HASH_SEED=0, but this seems nicer.)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
for-each-ref: avoid loading objects to print %(objectname)

If you ask for-each-ref to print each ref and its object,
like:

  git for-each-ref --format='%(objectname) %(refname)'

this should involve little more work than looking at the ref
files (and packed-refs) themselves. However, for-each-ref
will actually load each object from disk just to print its
sha1. For most repositories, this isn't a big deal, but it
can be noticeable if you have a large number of refs to
print. Here are best-of-five timings for the command above
on a repo with ~10K refs:

  [before]
  real    0m0.112s
  user    0m0.092s
  sys     0m0.016s

  [after]
  real    0m0.014s
  user    0m0.012s
  sys     0m0.000s

This patch checks for %(objectname) and %(objectname:short)
before we actually parse the object (and the rest of the
code is smart enough to avoid parsing if we have filled all
of our placeholders).

Note that we can't simply move the objectname parsing code
into the early loop. If the "deref" form %(*objectname) is
used, then we do need to parse the object in order to peel
the tag. So instead of moving the code, we factor it out
into a separate function that can be called for both cases.

While we're at it, we add some basic tests for the
dereferenced placeholders, which were not tested at all
before. This helps ensure we didn't regress that case.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Avoid difference in tr semantics between System V and BSD

Solaris' tr (both /usr/bin/ and /usr/xpg4/bin) uses the System V
semantics for tr whereby string1's length is truncated to the length
of string2 if string2 is shorter. The BSD semantics, as used by GNU tr
see string2 padded to the length of string1 using the final character
in string2. POSIX explicitly doesn't specify the correct behavior
here, making both equally valid.

This difference means that Solaris' native tr implementations produce
different results for tr ":\t\n" "\0" than GNU tr. This breaks a few
tests in t0008-ignores.sh.

Possible fixes for this are to make string2 be "\0\0\0" or "[\0*]".

Instead, use perl to perform these transliterations which means we
don't need to worry about the difference at all. Since we're replacing
tr with perl, we also use perl to replace the sed invocations used to
transform the files.

Replace four identical transforms with a function named
broken_c_unquote. Replace the other two identical transforms with a
fuction named broken_c_unquote_verbose.

Signed-off-by: Ben Walton <bdwalton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
subtree: add makefile target for html docs

The Makefile currently builds the roff manpage, but not the
html form. As some people may prefer the latter, let's make
it an option to build that, too. We also wire it into "make
doc" so that it is built by default.

This patch does not build or install it as part of
"install-doc"; that would require extra infrastructure to
handle installing the html as we do in git's regular
Documentation/ tree. That can come later if somebody is
interested.

Tested-by: Sebastian Schuberth <sschuberth@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>