upload/receive-pack: allow hiding ref hierarchies

Teach upload-pack and receive-pack to omit some refs from their
initial advertisements by paying attention to the transfer.hiderefs
multi-valued configuration variable.  Any ref that is under the
hierarchies listed on the value of this variable is excluded from
responses to requests made by "ls-remote", "fetch", "clone", "push",
etc.

A typical use case may be

	[transfer]
		hiderefs = refs/pull

to hide the refs that are internally used by the hosting site and
should not be exposed over the network.

Because these hidden refs do not count as OUR_REF, an attempt to
fetch objects at the tip of them will be rejected, and because these
refs do not get advertised, "git push :" will not see local branches
that have the same name as them as "matching" ones to be sent.

An attempt to update/delete these hidden refs with an explicit
refspec, e.g. "git push origin :refs/pull/11/head", is rejected.

This is not a new restriction.  To the pusher, it would appear that
there is no such ref, so its push request will conclude with "Now
that I sent you all the data, it is time for you to update the refs.
I saw that the ref did not exist when I started pushing, and I want
the result to point at this commit".  The receiving end will apply
the compare-and-swap rule to this request and rejects the push with
"Well, your update request conflicts with somebody else; I see there
is such a ref.", which is the right thing to do. Otherwise a push to
a hidden ref will always be "the last one wins", which is not a good
default.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse_fetch_refspec(): clarify the codeflow a bit

Most parts of the cascaded if/else if/... checked an allowable
condition but some checked forbidden conditions.  This makes adding
new allowable conditions unnecessarily inconvenient.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use struct ref to represent refs to be fetched

Even though "git fetch" has full infrastructure to parse refspecs to
be fetched and match them against the list of refs to come up with
the final list of refs to be fetched, the list of refs that are
requested to be fetched were internally converted to a plain list of
strings at the transport layer and then passed to the underlying
fetch-pack driver.

Stop this conversion and instead pass around an array of refs.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: optionally allow fetching from the tips of hidden refs

With uploadpack.allowtipsha1inwant configuration option set, future
versions of "git fetch" that allow an exact object name (likely to
have been obtained out of band) on the LHS of the fetch refspec can
make a request with a "want" line that names an object that may not
have been advertised due to transfer.hiderefs configuration.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: fetch objects by their exact SHA-1 object names

Teach "git fetch" to accept an exact SHA-1 object name the user may
obtain out of band on the LHS of a pathspec, and send it on a "want"
message when the server side advertises the allow-tip-sha1-in-want
capability.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
WIP: receive.allowupdatestohidden

This does not work yet, and for a good reason.  As the side that
pushes to a hidden ref never sees that the ref already exists, a
request to update such a ref will come in the form of "please
_create_ this ref and point it at this object", which will not pass
the compare-and-swap based anti-race safety at the receiving end.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jc/hidden-refs' into jch

Allow the server side to unclutter the refs/ namespace it shows to
the client.  Optionally allow requests for histories leading to the
tips of hidden refs by updated clients (which are not written yet).

* jc/hidden-refs:
  WIP: receive.allowupdatestohidden
  fetch: fetch objects by their exact SHA-1 object names
  upload-pack: optionally allow fetching from the tips of hidden refs
  fetch: use struct ref to represent refs to be fetched
  parse_fetch_refspec(): clarify the codeflow a bit
  upload/receive-pack: allow hiding ref hierarchies
  upload-pack: simplify request validation
  upload-pack: share more code

Conflicts:
	builtin/receive-pack.c
	cache.h
Merge branch 'dg/subtree-fixes' into pu

contrib/subtree updates; there are a few more from T. Zheng that
were posted separately, with an overlap.

Expecting a reroll.

* dg/subtree-fixes:
  contrib/subtree: mkdir the manual directory if needed
  contrib/subtree: honor $(DESTDIR)
  contrib/subtree: fix synopsis and command help
  contrib/subtree: better error handling for "add"
  contrib/subtree: add --unannotate option
  contrib/subtree: use %B for split Subject/Body
  t7900: remove test number comments
Merge branch 'mb/gitweb-highlight-link-target' into pu

Expecting a reroll.
$gmane/211935

* mb/gitweb-highlight-link-target:
  Highlight the link target line in Gitweb using CSS
Merge branch 'mp/diff-algo-config' into pu

Add diff.algorithm configuration so that the user does not type
"diff --histogram".

Looking better; may want tests to protect it from future breakages,
but otherwise it looks ready for 'next'.

Expecting a follow-up to add tests.

* mp/diff-algo-config:
  diff: Introduce --diff-algorithm command line option
  config: Introduce diff.algorithm variable
  git-completion.bash: Autocomplete --minimal and --histogram for git-diff

Conflicts:
	Documentation/diff-config.txt
Merge branch 'mb/remote-default-nn-origin' into pu

When the user does not specify what remote to interact with, we
often attempt to use 'origin'.  This can now be customized via a
configuration variable.

Expecting a reroll.
$gmane/210151

"The first remote becomes the default" bit is better done as a
separate step.

* mb/remote-default-nn-origin:
  Teach get_default_remote to respect remote.default.
  Test that plain "git fetch" uses remote.default when on a detached HEAD.
  Teach clone to set remote.default.
  Teach "git remote" about remote.default.
  Teach remote.c about the remote.default configuration setting.
  Rename remote.c's default_remote_name static variables.

Conflicts:
	builtin/remote.c
	t/t5505-remote.sh
Merge branch 'rc/maint-complete-git-p4' into pu

Comment from Pete will need to be addressed ($gmane/206172).

* rc/maint-complete-git-p4:
  Teach git-completion about git p4
Merge branch 'jc/xprm-generation' into pu

A toy to analyze how bad the clock skews are in histories of real
world projects.

Stalled mostly due to lack of responses.

* jc/xprm-generation:
  test-generation: compute generation numbers and clock skews
Merge branch 'jc/maint-name-rev' into pu

"git name-rev" names the given revision based on a ref that can be
reached in the smallest number of steps from the rev, but that is
not useful when the caller wants to know which tag is the oldest one
that contains the rev.  This teaches a new mode to the command that
uses the oldest ref among those which contain the rev.

I am not sure if this is worth it; for one thing, even with the help
from notes-cache, it seems to make the "describe --contains" even
slower. Also the command will be unusably slow for a user who does
not have a write access (hence unable to create or update the
notes-cache).

Stalled mostly due to lack of responses.

* jc/maint-name-rev:
  describe --contains: use "name-rev --algorithm=weight"
  name-rev --algorithm=weight: tests and documentation
  name-rev --algorithm=weight: cache the computed weight in notes
  name-rev --algorithm=weight: trivial optimization
  name-rev: --algorithm option
  name_rev: clarify the logic to assign a new tip-name to a commit
  name-rev: lose unnecessary typedef

Conflicts:
	builtin/name-rev.c
Merge branch 'jk/lua-hackery' into pu

Interesting exercise. When we do this for real, we probably would want
to wrap a commit to make it more like an "object" with methods like
"parents", etc.

* jk/lua-hackery:
  pretty: fix up one-off format_commit_message calls
  Minimum compilation fixup
  Makefile: make "lua" a bit more configurable
  add a "lua" pretty format
  add basic lua infrastructure
  pretty: make some commit-parsing helpers more public
Merge branch 'jl/submodule-deinit' into pu

There was no Porcelain way to say "I no longer am interested in
this submodule", once you express your interest in a submodule with
"submodule init".  "submodule deinit" is the way to do so.

Expecting a reroll.
$gmane/212884

* jl/submodule-deinit:
  submodule: add 'deinit' command

Conflicts:
	Documentation/git-submodule.txt
	git-submodule.sh
Merge branch 'jc/add-delete-default' into pu

"git add dir/" updated modified files and added new files, but does
not notice removed files, which may be "Huh?" to some users.  They
can of course use "git add -A dir/", but why should they?

Resurrected from graveyard, as I thought it was a worthwhile thing
to do in the longer term.

Stalled mostly due to lack of responses.

* jc/add-delete-default:
  git add: notice removal of tracked paths by default