revision: introduce --exclude=<glob> to tame wildcards

People often find "git log --branches" etc. that includes _all_
branches is cumbersome to use when they want to grab most but except
some.  The same applies to --tags, --all and --glob.

Teach the revision machinery to remember patterns, and then upon the
next such a globbing option, exclude those that match the pattern.

With this, I can view only my integration branches (e.g. maint,
master, etc.) without topic branches, which are named after two
letters from primary authors' names, slash and topic name.

    git rev-list --no-walk --exclude=??/* --branches |
    git name-rev --refs refs/heads/* --stdin

This one shows things reachable from local and remote branches that
have not been merged to the integration branches.

    git log --remotes --branches --not --exclude=??/* --branches

It may be a bit rough around the edges, in that the pattern to give
the exclude option depends on what globbing option follows.  In
these examples, the pattern "??/*" is used, not "refs/heads/??/*",
because the globbing option that follows the -"-exclude=<pattern>"
is "--branches".  As each use of globbing option resets previously
set "--exclude", this may not be such a bad thing, though.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule foreach: skip eval for more than one argument

'eval "$@"' creates an extra layer of shell interpretation, which is
probably not expected by a user who passes multiple arguments to git
submodule foreach:

 $ git grep "'"
 [searches for single quotes]
 $ git submodule foreach git grep "'"
 Entering '[submodule]'
 /usr/lib/git-core/git-submodule: 1: eval: Syntax error: Unterminated quoted string
 Stopping at '[submodule]'; script returned non-zero status.

To fix this, if the user passes more than one argument, execute "$@"
directly instead of passing it to eval.

Examples:

 * Typical usage when adding an extra level of quoting is to pass a
   single argument representing the entire command to be passed to the
   shell.  This doesn't change that.

 * One can imagine someone feeding untrusted input as an argument:

 	git submodule foreach git grep "$variable"

   That currently results in a nonobvious shell code injection
   vulnerability.  Executing the command named by the arguments
   directly, as in this patch, fixes it.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Acked-by: Johan Herland <johan@herland.net>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
merge-base: use OPT_CMDMODE and clarify the command line parsing

The --octopus, --independent and --is-ancestor are mutually
exclusive command modes (in addition to not giving any of these
options), so represent them as such using the recent OPT_CMDMODE
facility available since 11588263 (parse-options: add OPT_CMDMODE(),
2013-07-30), which is in v1.8.4-82-g366b80b.  --all is compatible
only with plain vanilla mode and --octopus mode, and the minimum
number of arguments the command takes depends on the command modes,
so these are now separately checked in each command mode.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
log_tree_diff: die when we fail to parse a commit

We currently call parse_commit and then assume we can
dereference the resulting "tree" struct field. If parsing
failed, however, that field is NULL and we end up
segfaulting.

Instead of a segfault, let's print an error message and die
a little more gracefully.

Note that this should never happen in practice, but may
happen in a corrupt repository.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
assume parse_commit checks commit->object.parsed

The parse_commit function will check the "parsed" flag of
the object and do nothing if it is set. There is no need
for callers to check the flag themselves, and doing so only
clutters the code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
assume parse_commit checks for NULL commit

The parse_commit function will check whether it was passed a
NULL commit pointer, and if so, return an error. There is no
need for callers to check this separately.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
use parse_commit_or_die instead of segfaulting

Some unchecked calls to parse_commit should obviously die on
error, because their next step is to start looking at the
parsed fields, which will cause a segfault. These are
obvious candidates for parse_commit_or_die, which will be a
strict improvement in behavior.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
use parse_commit_or_die instead of custom message

Many calls to parse_commit detect errors and die. In some
cases, the custom error messages are more useful than what
parse_commit_or_die could produce, because they give some
context, like which ref the commit came from. Some, however,
just say "invalid commit". Let's convert the latter to use
parse_commit_or_die; its message is slightly more informative,
and it makes the error more consistent throughout git.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
checkout: do not die when leaving broken detached HEAD

If we move away from a detached HEAD that has broken or
corrupted commits, we might die in two places:

  1. Printing the "old HEAD was..." message.

  2. Printing the list of orphaned commits.

In both cases, we ignore the return value of parse_commit
and feed the resulting commit to the pretty-print machinery,
which will die() upon failing to read the commit object
itself.

Since both cases are ancillary to the real operation being
performed, let's be more robust and keep going. This lets
users more easily checkout away from broken history.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1_file.c:create_tmpfile(): Fix race when creating loose object dirs

There are cases (e.g. when running concurrent fetches in a repo) where
multiple Git processes concurrently attempt to create loose objects
within the same objects/XX/ dir. The creation of the loose object files
is (AFAICS) safe from races, but the creation of the objects/XX/ dir in
which the loose objects reside is unsafe, for example:

Two concurrent fetches - A and B. As part of its fetch, A needs to store
12aaaaa as a loose object. B, on the other hand, needs to store 12bbbbb
as a loose object. The objects/12 directory does not already exist.
Concurrently, both A and B determine that they need to create the
objects/12 directory (because their first call to git_mkstemp_mode()
within create_tmpfile() fails witn ENOENT). One of them - let's say A -
executes the following mkdir() call before the other. This first call
returns success, and A moves on. When B gets around to calling mkdir(),
it fails with EEXIST, because A won the race. The mkdir() error causes B
to return -1 from create_tmpfile(), which propagates all the way,
resulting in the fetch failing with:

  error: unable to create temporary file: File exists
  fatal: failed to write object
  fatal: unpack-objects failed

Although it's hard to add a testcase reproducing this issue, it's easy
to provoke if we insert a sleep after the

  if (mkdir(buffer, 0777) || adjust_shared_perm(buffer))
      return -1;

block, and then run two concurrent "git fetch"es against the same repo.

The fix is to simply handle mkdir() failing with EEXIST as a success.
If EEXIST is somehow returned for the wrong reasons (because the relevant
objects/XX is not a directory, or is otherwise unsuitable for object
storage), the following call to adjust_shared_perm(), or ultimately the
retried call to git_mkstemp_mode() will fail, and we end up returning
error from create_tmpfile() in any case.

Note that there are still cases where two users with unsuitable umasks
in a shared repo can end up in two races where one user first wins the
mkdir() race to create an objects/XX/ directory, and then the other user
wins the adjust_shared_perms() race to chmod() that directory, but fails
because it is (transiently, until the first users completes its chmod())
unwriteable to the other user. However, (an equivalent of) this race also
exists before this patch, and is made no worse by this patch.

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
use @@PERL@@ in built scripts

Several of the built shell commands invoke a bare "perl" to
perform some one-liners. This will use the first perl in the
PATH rather than the one specified by the user's SHELL_PATH.
We are not asking these perl invocations to do anything
exotic, so typically any old system perl will do; however,
in some cases the system perl may have unexpected behavior
(e.g., by handling line endings differently). We should err
on the side of using the perl the user pointed us to.

The downside of this is that on systems with a sane perl
setup, we no longer find the perl at runtime, but instead
point to a static perl (like /usr/bin/perl). That means we
will not handle somebody moving perl without rebuilding git,
whereas before we tracked it just fine. This is probably not
a big deal, though, as the built perl scripts already
suffered from this.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
merge-base: teach "--fork-point" mode

The "git pull --rebase" command computes the fork point of the
branch being rebased using the reflog entries of the "base" branch
(typically a remote-tracking branch) the branch's work was based on,
in order to cope with the case in which the "base" branch has been
rewound and rebuilt.  For example, if the history looked like this:

                     o---B1
                    /
    ---o---o---B2--o---o---o---Base
            \
             B3
              \
               Derived

where the current tip of the "base" branch is at Base, but earlier
fetch observed that its tip used to be B3 and then B2 and then B1
before getting to the current commit, and the branch being rebased
on top of the latest "base" is based on commit B3, it tries to find
B3 by going through the output of "git rev-list --reflog base" (i.e.
Base, B1, B2, B3) until it finds a commit that is an ancestor of the
current tip "Derived".

Internally, we have get_merge_bases_many() that can compute this
with one-go.  We would want a merge-base between Derived and a
fictitious merge commit that would result by merging all the
historical tips of "base".  When such a commit exist, we should get
a single result, which exactly match one of the reflog entries of
"base".

Teach "git merge-base" a new mode, "--fork-point", to compute
exactly that.

Helped-by: Martin von Zweigbergk <martinvonz@gmail.com>
Helped-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
http: return curl's AUTHAVAIL via slot_results

Callers of the http code may want to know which auth types
were available for the previous request. But after finishing
with the curl slot, they are not supposed to look at the
curl handle again. We already handle returning other
information via the slot_results struct; let's add a flag to
check the available auth.

Note that older versions of curl did not support this, so we
simply return 0 (something like "-1" would be worse, as the
value is a bitflag and we might accidentally set a flag).
This is sufficient for the callers planned in this series,
who only trigger some optional behavior if particular bits
are set, and can live with a fake "no bits" answer.

Signed-off-by: Jeff King <peff@peff.net>
remote-curl: pass curl slot_results back through run_slot

Some callers may want to know more than just the integer
error code we return. Let them optionally pass a
slot_results struct to fill in (or NULL if they do not
care). In either case we continue to return the integer
code.

We can also give probe_rpc the same treatment (since it
builds directly on run_slot).

Signed-off-by: Jeff King <peff@peff.net>
remote-curl: fix large pushes with GSSAPI

Due to an interaction between the way libcurl handles GSSAPI
authentication over HTTP and the way git uses libcurl, large
pushes (those over http.postBuffer bytes) would fail due to
an authentication failure requiring a rewind of the curl
buffer.  Such a rewind was not possible because the data did
not fit into the entire buffer.

Enable the use of the Expect: 100-continue header for large
requests where the server offers GSSAPI authentication to
avoid this issue, since the request would otherwise fail.
This allows git to get the authentication data right before
sending the pack contents.  Existing cases where pushes
would succeed, including small requests using GSSAPI, still
disable the use of 100 Continue, as it causes problems for
some remote HTTP implementations (servers and proxies).

Signed-off-by: Brian M. Carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
Use the word 'stuck' instead of 'sticked'

The past participle of 'stick' is 'stuck'.

Signed-off-by: Nicolas Vigier <boklm@mars-attacks.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rev-parse --parseopt: add the --stuck-long mode

Add the --stuck-long option to output the options in their long form
if available, and with their arguments stuck.

Contrary to the default form (non stuck arguments and short options),
this can be parsed unambiguously when using options with optional
arguments :

 - in the non stuck form, when an option is taking an optional argument
   you cannot know if the next argument is its optional argument, or the
   next option.

 - the long options form allows to differentiate between an empty argument
   '--option=' and an unset argument '--option', which is not possible
   with short options.

Signed-off-by: Nicolas Vigier <boklm@mars-attacks.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
document --exclude option

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rev-list --exclude: tests

Add tests for the --exclude=<glob> feature.

A few tests are added for cases where use of globbing and
"--exclude" results in no positive revisions:

 * "--exclude=<glob>" before "--all" etc. resulted in no results;

 * "--stdin" is used but no input was given;

 * "--all" etc. is used but no matching refs are found.

Currently, we fail such a request with the same error message we
would give to a command line that does not specify any positive
revision (e.g. "git rev-list<ENTER>").

We may want to treat these cases differently and not error out, but
the logic to detect that would be common to all of them, so I'd
leave it outside this topic for now, and stop at adding these tests
as food-for-thought.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
rev-list --exclude: export add/clear-ref-exclusion and ref-excluded API

... while updating their function signature.  To be squashed into
the initial patch to rev-list.

Signed-off-by: Junio C Hamano <gitster@pobox.com>