config: allow inaccessible configuration under $HOME

The changes v1.7.12.1~2^2~4 (config: warn on inaccessible files,
2012-08-21) and v1.8.1.1~22^2~2 (config: treat user and xdg config
permission problems as errors, 2012-10-13) were intended to prevent
important configuration (think "[transfer] fsckobjects") from being
ignored when the configuration is unintentionally unreadable (for
example with EIO on a flaky filesystem, or with ENOMEM due to a DoS
attack).  Usually ~/.gitconfig and ~/.config/git are readable by the
current user, and if they aren't then it would be easy to fix those
permissions, so the damage from adding this check should have been
minimal.

Unfortunately the access() check often trips when git is being run as
a server.  A daemon (such as inetd or git-daemon) starts as "root",
creates a listening socket, and then drops privileges, meaning that
when git commands are invoked they cannot access $HOME and die with

 fatal: unable to access '/root/.config/git/config': Permission denied

Any patch to fix this would have one of three problems:

  1. We annoy sysadmins who need to take an extra step to handle HOME
     when dropping privileges (the current behavior, or any other
     proposal that they have to opt into).

  2. We annoy sysadmins who want to set HOME when dropping privileges,
     either by making what they want to do impossible, or making them
     set an extra variable or option to accomplish what used to work
     (e.g., a patch to git-daemon to set HOME when --user is passed).

  3. We loosen the check, so some cases which might be noteworthy are
     not caught.

This patch is of type (3).

Treat user and xdg configuration that are inaccessible due to
permissions (EACCES) as though no user configuration was provided at
all.

An alternative method would be to check if $HOME is readable, but that
would not help in cases where the user who dropped privileges had a
globally readable HOME with only .config or .gitconfig being private.

This does not change the behavior when /etc/gitconfig or .git/config
is unreadable (since those are more serious configuration errors),
nor when ~/.gitconfig or ~/.config/git is unreadable due to problems
other than permissions.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t2024: Add tests verifying current DWIM behavior of 'git checkout <branch>'

The DWIM mode of checkout allows you to run "git checkout foo" when there is
no existing local ref or path called "foo", and there is exactly one remote
with a remote-tracking branch called "foo". Git will then automatically
create a new local branch called "foo" using the remote-tracking "foo" as
its starting point and configured upstream.

Improved-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t2024: Show failure to use refspec when DWIMming remote branch names

When using "git checkout foo" to DWIM the creation of local "foo" from some
existing upstream "foo", we assume conventional refspecs as created by "git
clone" or "git remote add", and fail to work correctly if the current
refspecs do not follow the conventional "refs/remotes/$remote/*" pattern.

Improved-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
checkout: Use remote refspecs when DWIMming tracking branches

The DWIM mode of checkout allows you to run "git checkout foo" when there
is no existing local ref or path called "foo", and there is exactly _one_
remote with a remote-tracking branch called "foo". Git will automatically
create a new local branch called "foo" using the remote-tracking "foo" as
its starting point and configured upstream.

For example, consider the following unconventional (but perfectly valid)
remote setup:

	[remote "origin"]
		fetch = refs/heads/*:refs/remotes/origin/*
	[remote "frotz"]
		fetch = refs/heads/*:refs/remotes/frotz/nitfol/*

Case 1: Assume both "origin" and "frotz" have remote-tracking branches called
"foo", at "refs/remotes/origin/foo" and "refs/remotes/frotz/nitfol/foo"
respectively. In this case "git checkout foo" should fail, because there is
more than one remote with a "foo" branch.

Case 2: Assume only "frotz" have a remote-tracking branch called "foo". In
this case "git checkout foo" should succeed, and create a local branch "foo"
from "refs/remotes/frotz/nitfol/foo", using remote branch "foo" from "frotz"
as its upstream.

The current code hardcodes the assumption that all remote-tracking branches
must match the "refs/remotes/$remote/*" pattern (which is true for remotes
with "conventional" refspecs, but not true for the "frotz" remote above).
When running "git checkout foo", the current code looks for exactly one ref
matching "refs/remotes/*/foo", hence in the above example, it fails to find
"refs/remotes/frotz/nitfol/foo", which causes it to fail both case #1 and #2.

The better way to handle the above example is to actually study the fetch
refspecs to deduce the candidate remote-tracking branches for "foo"; i.e.
assume "foo" is a remote branch being fetched, and then map "refs/heads/foo"
through the refspecs in order to get the corresponding remote-tracking
branches "refs/remotes/origin/foo" and "refs/remotes/frotz/nitfol/foo".
Finally we check which of these happens to exist in the local repo, and
if there is exactly one, we have an unambiguous match for "git checkout foo",
and may proceed.

This fixes most of the failing tests introduced in the previous patch.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3200.39: tracking setup should fail if there is no matching refspec.

We are formalizing a requirement that any remote-tracking branch to be used
as an upstream (i.e. as an argument to --track), _must_ "belong" to a
configured remote by being matched by the "dst" side of a fetch refspec.

This patch encodes the new expected behavior of this test, and marks the
test with "test_expect_failure" in anticipation of a following patch to
introduce the new behavior.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7201.24: Add refspec to keep --track working

We are formalizing a requirement that any remote-tracking branch to be used
as an upstream (i.e. as an argument to --track), _must_ "belong" to a
configured remote by being matched by the "dst" side of a fetch refspec.

Without this patch, this test would start failing when the new behavior is
introduced.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t9114.2: Don't use --track option against "svn-remote"-tracking branches

We are formalizing a requirement that any remote-tracking branch to be used
as an upstream (i.e. as an argument to --track), _must_ "belong" to a
configured remote by being matched by the "dst" side of a fetch refspec.

This test uses --track against a "remotes/trunk" ref which does not belong
to any configured (git) remotes, but is instead created by "git svn fetch"
operating on an svn-remote. It does not make sense to use an svn-remote as
an upstream for a local branch, as a regular "git pull" from (or "git push"
to) it would obviously fail (instead you would need to use "git svn" to
communicate with this remote). Furthermore, the usage of --track in this
case is unnecessary, since the upstreaming config that would be created is
never used.

Simply removing --track fixes the issue without changing the expected
behavior of the test.

Acked-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
branch.c: Validate tracking branches with refspecs instead of refs/remotes/*

The current code for validating tracking branches (e.g. the argument to
the -t/--track option) hardcodes refs/heads/* and refs/remotes/* as the
potential locations for tracking branches. This works with the refspecs
created by "git clone" or "git remote add", but is suboptimal in other
cases:

 - If "refs/remotes/foo/bar" exists without any association to a remote
   (i.e. there is no remote named "foo", or no remote with a refspec
   that matches "refs/remotes/foo/bar"), then it is impossible to set up
   a valid upstream config that tracks it. Currently, the code defaults
   to using "refs/remotes/foo/bar" from repo "." as the upstream, which
   works, but is probably not what the user had in mind when running
   "git branch baz --track foo/bar".

 - If the user has tweaked the fetch refspec for a remote to put its
   remote-tracking branches outside of refs/remotes/*, e.g. by running
       git config remote.foo.fetch "+refs/heads/*:refs/foo_stuff/*"
   then the current code will refuse to use its remote-tracking branches
   as --track arguments, since they do not match refs/remotes/*.

This patch removes the "refs/remotes/*" requirement for upstream branches,
and replaces it with explicit checking of the refspecs for each remote to
determine whether a given --track argument is a valid remote-tracking
branch. This solves both of the above problems, since the matching refspec
guarantees that there is a both a remote name and a remote branch name
that can be used for the upstream config.

However, this means that refs located within refs/remotes/* without a
corresponding remote/refspec will no longer be usable as upstreams.
The few existing tests which depended on this behavioral quirk has
already been fixed in the preceding patches.

This patch fixes the last remaining test failure in t2024-checkout-dwim.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
glossary: Update and rephrase the definition of a remote-tracking branch

The definition of a remote-tracking branch in the glossary have been
out-of-date for a while (by e.g. referring to "Pull:" from old-style
$GIT_DIR/remotes files).

Also, the preceding patches have formalized that a remote-tracking branch
must match a configured refspec in order to be usable as an upstream.

This patch rewrites the paragraph on remote-tracking branches accordingly.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: ignore 'shallow' lines with unknown obj-ids

When the client sends a 'shallow' line for an object that the server does
not have, the server currently dies with the error: "did not find object
for shallow <obj-id>".  The client may have truncated the history at
the commit by fetching shallowly from a different server, or the commit
may have been garbage collected by the server. In either case, this
unknown commit is not relevant for calculating the pack that is to be
sent and can be safely ignored, and it is not used when recomputing where
the updated history of the client is cauterised.

The documentation in technical/pack-protocol.txt has been updated to
remove the restriction that "Clients MUST NOT mention an obj-id which it
does not know exists on the server". This requirement is not realistic
because clients cannot know whether an object has been garbage collected
by the server.

Signed-off-by: Michael Heemskerk <mheemskerk@atlassian.com>
Reviewed-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
contrib/subtree: don't delete remote branches if split fails

When using "git subtree push" to split out a subtree and push it to a
remote repository, we do not detect if the split command fails which
causes the LHS of the refspec to be empty, deleting the remote branch.

Fix this by pulling the result of the split command into a variable so
that we can die if the command fails.

Reported-by: Steffen Jaeckel <steffen.jaeckel@stzedn.de>
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5500: add test for fetching with an unknown 'shallow'

When the client sends a 'shallow' line for an object that the server does
not have, the server should just ignore it and let the client keep that
unknown shallow boundary.

Signed-off-by: Michael Heemskerk <mheemskerk@atlassian.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
deprecate core.statinfo at Git 2.0 boundary

c08e4d5b5cfa (Enable minimal stat checking, 2013-01-22) advertised
the configuration variable core.checkstat in the documentation and
its log message, but the code expected core.statinfo instead.

For now, add core.checkstat, and warn people who have core.statinfo
in their configuration file that we will remove it in Git 2.0.

Noticed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
help: add help_unknown_ref()

When the user gives an unknown string to a command that expects to
get a ref, we could be more helpful than just saying "that's not a
ref" and die.

Add helper function help_unknown_ref() to take care of displaying an
error message along with a list of suggested refs the user might
have meant.  An interaction with "git merge" might go like this:

	$ git merge foo
	merge: foo - not something we can merge

	Did you mean one of these?
	    origin/foo
	    upstream/foo

Signed-off-by: Vikrant Varma <vikrant.varma94@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
merge: use help_unknown_ref()

Use help.c:help_unknown_ref() instead of die() to provide a
friendlier error message before exiting, when one of the refs
specified in a merge is unknown.

Signed-off-by: Vikrant Varma <vikrant.varma94@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5004: avoid using tar for checking emptiness of archive

Test 2 of t5004 checks if a supposedly empty tar archive really
contains no files.  24676f02 (t5004: fix issue with empty archive test
and bsdtar) removed our commit hash to make it work with bsdtar, but
the test still fails on NetBSD and OpenBSD, which use their own tar
that considers a tar file containing only NULs as broken.

Here's what the different archivers do when asked to create a tar
file without entries:

	$ uname -v
	NetBSD 6.0.1 (GENERIC)
	$ gtar --version | head -1
	tar (GNU tar) 1.26
	$ bsdtar --version
	bsdtar 2.8.4 - libarchive 2.8.4

	$ : >zero.tar
	$ perl -e 'print "\0" x 10240' >tenk.tar
	$ sha1 zero.tar tenk.tar
	SHA1 (zero.tar) = da39a3ee5e6b4b0d3255bfef95601890afd80709
	SHA1 (tenk.tar) = 34e163be8e43c5631d8b92e9c43ab0bf0fa62b9c

	$ : | tar cf - -T - | sha1
	da39a3ee5e6b4b0d3255bfef95601890afd80709
	$ : | gtar cf - -T - | sha1
	34e163be8e43c5631d8b92e9c43ab0bf0fa62b9c
	$ : | bsdtar cf - -T - | sha1
	34e163be8e43c5631d8b92e9c43ab0bf0fa62b9c

So NetBSD's native tar creates an empty file, while GNU tar and bsdtar
both give us 10KB of NULs -- just like git archive with an empty tree.
Now let's see how the archivers handle these two kinds of empty tar
files:

	$ tar tf zero.tar; echo $?
	tar: Unexpected EOF on archive file
	1
	$ gtar tf zero.tar; echo $?
	gtar: This does not look like a tar archive
	gtar: Exiting with failure status due to previous errors
	2
	$ bsdtar tf zero.tar; echo $?
	0

	$ tar tf tenk.tar; echo $?
	tar: Cannot identify format. Searching...
	tar: End of archive volume 1 reached
	tar: Sorry, unable to determine archive format.
	$ gtar tf tenk.tar; echo $?
	0
	$ bsdtar tf tenk.tar; echo $?
	0

NetBSD's tar complains about both, bsdtar happily accepts any of them
and GNU tar doesn't like zero-length archive files.  So the safest
course of action is to stay with our block-of-NULs format which is
compatible with GNU tar and bsdtar, as we can't make NetBSD's native
tar happy anyway.

We can simplify our test, however, by taking tar out of the picture.
Instead of extracting the archive and checking for the non-presence of
files, check if the file has a size of 10KB and contains only NULs.
This makes t5004 pass on NetBSD and OpenBSD.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5004: resurrect original empty tar archive test

Add a test to verify the emptiness of an archive by extracting its
contents.  Don't run this test if the version of tar doesn't support
archives containing only a comment header, though.

The existing check 'tar archive of empty tree is empty' used to work
like that (minus the tar capability check) but was changed to depend
on the exact representation of empty tar files created by git archive
instead of on the behaviour of tar in order to avoid issues with
different tar versions.

The different approaches test different things: The existing one is
for empty trees, for which we know the exact expected output and thus
we can simply check it without extracting; the new one is for commits
with empty trees, whose archives include stamps and so the more
"natural" check by extraction is a better fit because it focuses on
the interesting aspect, namely the absence of any archive entries.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
clone: let the user know when check_everything_connected is run

check_everything_connected could take a long time, especially in the
clone case where the whole DAG is traversed. The user deserves to know
what's going on.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Makefile: fix default regex settings on Darwin

t0070-fundamental.sh fails on Mac OS X 10.8:

	$ uname -a
	Darwin lustrous 12.2.0 Darwin Kernel Version 12.2.0:
	Sat Aug 25 00:48:52 PDT 2012;
	root:xnu-2050.18.24~1/RELEASE_X86_64 x86_64

	$ ./t0070-fundamental.sh -v
	fatal: regex bug confirmed: re-build git with NO_REGEX=1

Fix it by using Git's regex library.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5551: do not use unportable sed '\+'

The set-up step to prepare a repository with 50000 tags used a
non-porable '\+' to match one-or-more.

The error was not caught because the next test that uses that
repository did not even bother to check if these expected tags were
actually cloned to the resulting repository.

Fix the sed construct to use BRE and update the "clone" test that
wanted to test cloning from such a repository with many refs to
check the resulting repository.

Signed-off-by: Junio C Hamano <gitster@pobox.com>