revision: introduce --exclude=<glob> to tame wildcards

People often find "git log --branches" etc. that includes _all_
branches is cumbersome to use when they want to grab most but except
some.  The same applies to --tags, --all and --glob.

Teach the revision machinery to remember patterns, and then upon the
next such a globbing option, exclude those that match the pattern.

With this, I can view only my integration branches (e.g. maint,
master, etc.) without topic branches, which are named after two
letters from primary authors' names, slash and topic name.

    git rev-list --no-walk --exclude=??/* --branches |
    git name-rev --refs refs/heads/* --stdin

This one shows things reachable from local and remote branches that
have not been merged to the integration branches.

    git log --remotes --branches --not --exclude=??/* --branches

It may be a bit rough around the edges, in that the pattern to give
the exclude option depends on what globbing option follows.  In
these examples, the pattern "??/*" is used, not "refs/heads/??/*",
because the globbing option that follows the -"-exclude=<pattern>"
is "--branches".  As each use of globbing option resets previously
set "--exclude", this may not be such a bad thing, though.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5510: use the correct tag name in test

Fix an apparent copy-paste error: A few lines earlier, a tag
"refs/tags/sometag" is created.  Check for the (non-)existence of that
tag, not "somebranch", which is otherwise never mentioned in the
script.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5510: prepare test refs more straightforwardly

"git fetch" was being used with contrived refspecs to create tags and
remote-tracking branches in test repositories in preparation for the
actual tests.  This is obscure and also makes one wonder whether this
is indeed just preparation or whether some side-effect of "git fetch"
is being tested.

So use the more straightforward commands "git tag" / "git update-ref"
when preparing branches in test repositories.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5510: check that "git fetch --prune --tags" does not prune branches

"git fetch --prune --tags" is currently interpreted as follows:

* "--tags" is equivalent to specifying a refspec
  "refs/tags/*:refs/tags/*", and supersedes any default refspecs
  configured via remote.$REMOTE.fetch.

* "--prune" only operates on the refspecs being fetched.

Therefore, "git fetch --prune --tags" prunes tags in refs/tags/* but
does not fetch or prune other references.  The fact that this command
does not prune references outside of refs/tags/* was previously
untested.  So add a test that verifies the status quo.

However, the status quo is surprising, so it will be changed later in
this patch series.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
api-remote.txt: correct section "struct refspec"

* Replace reference to function parse_ref_spec() with references to
  functions parse_fetch_refspec() and parse_push_refspec().

* Correct description of src and dst: they *do* include the '*'
  characters.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_ref_map(): rename local variables

Rename "refs" -> "refspecs" and "ref_count" -> "refspec_count" to
reduce confusion, because they describe an array of "struct refspec",
as opposed to the "struct ref" objects that are also used in this
function.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
builtin/fetch.c: reorder function definitions

Reorder function definitions to avoid the need for a forward
declaration of function find_non_local_tags().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_expanded_map(): add docstring

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_expanded_map(): avoid memory leak

The old code could leak *expn_name if match_name_with_pattern()
succeeded but ignore_symref_update() returned true.  So make sure that
*expn_name is freed in any case.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: only opportunistically update references based on command line

The old code processed (tags == TAGS_SET) before adding the entries
used to opportunistically update references mentioned on the command
line.  The result was that all tags were also considered candidates
for opportunistic updating.

This is harmless for two reasons: (a) because it would only add
entries if there is a configured refspec that covers tags *and* both
--tags and another refspec appear on the command-line; (b) because any
extra entries would be deleted later by the call to
ref_remove_duplicates() anyway.

But, to avoid extra work and extra memory usage, and to make the
implementation better match the intention, change the algorithm
slightly: compute the opportunistic refspecs based only on the
command-line arguments, storing the results into a separate temporary
list.  Then add the tags (which have to come earlier in the list so
that they are not de-duped in favor of an opportunistic entry).  Then
concatenate the temporary list onto the main list.

This change will also make later changes easier.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch --tags: fetch tags *in addition to* other stuff

Previously, fetch's "--tags" option was considered equivalent to
specifying the refspec "refs/tags/*:refs/tags/*" on the command line;
in particular, it caused the remote.<name>.refspec configuration to be
ignored.

But it is not very useful to fetch tags without also fetching other
references, whereas it *is* quite useful to be able to fetch tags *in
addition to* other references.  So change the semantics of this option
to do the latter.

If a user wants to fetch *only* tags, then it is still possible to
specifying an explicit refspec:

    git fetch <remote> 'refs/tags/*:refs/tags/*'

Please note that the documentation prior to 1.8.0.3 was ambiguous
about this aspect of "fetch --tags" behavior.  Commit

    f0cb2f137c 2012-12-14 fetch --tags: clarify documentation

made the documentation match the old behavior.  This commit changes
the documentation to match the new behavior.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch --prune: prune only based on explicit refspecs

The old behavior of "fetch --prune" was to prune whatever was being
fetched.  In particular, "fetch --prune --tags" caused tags not only
to be fetched, but also to be pruned.  This is inappropriate because
there is only one tags namespace that is shared among the local
repository and all remotes.  Therefore, if the user defines a local
tag and then runs "git fetch --prune --tags", then the local tag is
deleted.  Moreover, "--prune" and "--tags" can also be configured via
fetch.prune / remote.<name>.prune and remote.<name>.tagopt, making it
even less obvious that an invocation of "git fetch" could result in
tag lossage.

Since the command "git remote update" invokes "git fetch", it had the
same problem.

The command "git remote prune", on the other hand, disregarded the
setting of remote.<name>.tagopt, and so its behavior was inconsistent
with that of the other commands.

So the old behavior made it too easy to lose tags.  To fix this
problem, change "fetch --prune" to prune references based only on
refspecs specified explicitly by the user, either on the command line
or via remote.<name>.fetch.  Thus, tags are no longer made subject to
pruning by the --tags option or the remote.<name>.tagopt setting.

However, tags *are* still subject to pruning if they are fetched as
part of a refspec, and that is good.  For example:

* On the command line,

      git fetch --prune 'refs/tags/*:refs/tags/*'

  causes tags, and only tags, to be fetched and pruned, and is
  therefore a simple way for the user to get the equivalent of the old
  behavior of "--prune --tag".

* For a remote that was configured with the "--mirror" option, the
  configuration is set to include

      [remote "name"]
              fetch = +refs/*:refs/*

  , which causes tags to be subject to pruning along with all other
  references.  This is the behavior that will typically be desired for
  a mirror.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
query_refspecs(): move some constants out of the loop

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
builtin/remote.c: reorder function definitions

Reorder function definitions to remove the need for forward
declarations.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
builtin/remote.c:update(): use struct argv_array

Use struct argv_array for calling the "git fetch" subprocesses.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch, remote: properly convey --no-prune options to subprocesses

If --no-prune is passed to one of the following commands:

    git fetch --all
    git fetch --multiple
    git fetch --recurse-submodules
    git remote update

then it must also be passed to the "fetch" subprocesses that those
commands use to do their work.  Otherwise there might be a fetch.prune
or remote.<name>.prune configuration setting that causes pruning to
occur, contrary to the user's express wish.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch-options.txt: simplify ifdef/ifndef/endif usage

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-fetch.txt: improve description of tag auto-following

Make it clearer that tags are fetched independent of which branches
were fetched from the remote in any particular fetch.  (Tags are even
fetched if they point at objects that are in the current repository
but not reachable, which is probably a bug.)

Put less emphasis on the mechanism and more on the effect of tag
auto-following.  Also mention the options and configuration settings
that can change the tag-fetching behavior.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
ref_remove_duplicates(): avoid redundant bisection

The old code called string_list_lookup(), and if that failed called
string_list_insert(), thus doing the bisection search through the
string list twice in the latter code path.

Instead, just call string_list_insert() right away.  If an entry for
that peer reference name already existed, then its util pointer is
always non-NULL.

Of course this doesn't change the fact that the repeated
string_list_insert() calls make the function scale like O(N^2) if the
input reference list is not already approximately sorted.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5536: new test of refspec conflicts when fetching

Add some tests that "git fetch" handles refspec conflicts (i.e., when
the same local reference should be updated from two different remote
references) correctly.

There is a small bug when updating references opportunistically,
namely that an explicit user wish like

    git fetch origin \
        refs/heads/branch1:refs/remotes/origin/branch2 \
        refs/heads/branch2:refs/remotes/origin/branch1

should override a configured refspec like

    +refs/heads/*:refs/remotes/origin/*

The current code incorrectly treats this as a fatal error.

In a few commits we will improve the error messages for refspec
conflicts in general and also turn this buggy fatal error into a
warning.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>