signed push: fortify against replay attacks

In order to prevent a valid push certificate for pushing into an
repository from getting replayed in a different push operation, send
a nonce string from the receive-pack process and have the signer
include it in the push certificate.  The receiving end uses an HMAC
hash of the path to the repository it serves and the current time
stamp, hashed with a secret seed (the secret seed does not have to
be per-repository but can be defined in /etc/gitconfig) to generate
the nonce, in order to ensure that a random third party cannot forge
a nonce that looks like it originated from it.

The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks
to examine and match against the value on the "nonce" header in the
certificate to notice a replay, but returned "nonce" header in the
push certificate is examined by receive-pack and the result is
exported as GIT_PUSH_CERT_NONCE_STATUS, whose value would be "OK"
if the nonce recorded in the certificate matches what we expect, so
that the hooks can more easily check.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: teach smart-HTTP to pass "git push --signed" around

The "--signed" option received by "git push" is first passed to the
transport layer, which the native transport directly uses to notice
that a push certificate needs to be sent.  When the transport-helper
is involved, however, the option needs to be told to the helper with
set_helper_option(), and the helper needs to take necessary action.
For the smart-HTTP helper, the "necessary action" involves spawning
the "git send-pack" subprocess with the "--signed" option.

Once the above all gets wired in, the smart-HTTP transport now can
use the push certificate mechanism to authenticate its pushes.

Add a test that is modeled after tests for the native transport in
t5534-push-signed.sh to t5541-http-push-smart.sh.  Update the test
Apache configuration to pass GNUPGHOME environment variable through.
As PassEnv would trigger warnings for an environment variable that
is not set, export it from test-lib.sh set to a harmless value when
GnuPG is not being used in the tests.

Note that the added test is deliberately loose and does not check
the nonce in this step.  This is because the stateless RPC mode is
inevitably flaky and a nonce that comes back in the actual push
processing is one issued by a different process; if the two
interactions with the server crossed a second boundary, the nonces
will not match and such a check will fail.  A later patch in the
series will work around this shortcoming.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: allow stale nonce in stateless mode

When operating with the stateless RPC mode, we will receive a nonce
issued by another instance of us that advertised our capability and
refs some time ago.  Update the logic to check received nonce to
detect this case, compute how much time has passed since the nonce
was issued and report the status with a new environment variable
GIT_PUSH_CERT_NONCE_SLOP to the hooks.

GIT_PUSH_CERT_NONCE_STATUS will report "SLOP" in such a case.  The
hooks are free to decide how large a slop it is willing to accept.

Strictly speaking, the "nonce" is not really a "nonce" anymore in
the stateless RPC mode, as it will happily take any "nonce" issued
by it (which is protected by HMAC and its secret key) as long as it
is fresh enough.  The degree of this security degradation, relative
to the native protocol, is about the same as the "we make sure that
the 'git push' decided to update our refs with new objects based on
the freshest observation of our refs by making sure the values they
claim the original value of the refs they ask us to update exactly
match the current state" security is loosened to accomodate the
stateless RPC mode in the existing code without this series, so
there is no need for those who are already using smart HTTP to push
to their repositories to be alarmed any more than they already are.

In addition, the server operator can set receive.certnonceslop
configuration variable to specify how stale a nonce can be (in
seconds).  When this variable is set, and if the nonce received in
the certificate that passes the HMAC check was less than that many
seconds old, hooks are given "OK" in GIT_PUSH_CERT_NONCE_STATUS
(instead of "SLOP") and the received nonce value is given in
GIT_PUSH_CERT_NONCE, which makes it easier for a simple-minded
hook to check if the certificate we received is recent enough.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
refs: make rev-parse --quiet actually quiet

When a reflog is deleted, e.g. when "git stash" clears its stashes,
"git rev-parse --verify --quiet" dies:

	fatal: Log for refs/stash is empty.

The reason is that the get_sha1() code path does not allow us
to suppress this message.

Pass the flags bitfield through get_sha1_with_context() so that
read_ref_at() can suppress the message.

Use get_sha1_with_context1() instead of get_sha1() in rev-parse
so that the --quiet flag is honored.

[jc: squashed in a fixup from David, $gmane/257211]

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
stash: prefer --quiet over shell redirection of the standard error stream

Use `git rev-parse --verify --quiet` instead of redirecting
stderr to /dev/null.

Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
add macro REALLOC_ARRAY

The macro ALLOC_GROW manages several aspects of dynamic memory
allocations for arrays: It performs overprovisioning in order to avoid
reallocations in future calls, updates the allocation size variable,
multiplies the item size and thus allows users to simply specify the
item count, performs the reallocation and updates the array pointer.

Sometimes this is too much.  Add the macro REALLOC_ARRAY, which only
takes care of the latter three points and allows users to specfiy the
number of items the array can store.  It can increase and also decrease
the size.  Using the macro avoid duplicating the variable name and
takes care of the item sizes automatically.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
use REALLOC_ARRAY for changing the allocation size of arrays

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
branch: clean up commit flags after merge-filter walk

When we run `branch --merged`, we use prepare_revision_walk
with the merge-filter marked as UNINTERESTING. Any branch
tips that are marked UNINTERESTING after it returns must be
ancestors of that commit. As we iterate through the list of
refs to show, we check item->commit->object.flags to see
whether it was marked.

This interacts badly with --verbose, which will do a
separate walk to find the ahead/behind information for each
branch. There are two bad things that can happen:

  1. The ahead/behind walk may get the wrong results,
     because it can see a bogus UNINTERESTING flag leftover
     from the merge-filter walk.

  2. We may omit some branches if their tips are involved in
     the ahead/behind traversal of a branch shown earlier.
     The ahead/behind walk carefully cleans up its commit
     flags, meaning it may also erase the UNINTERESTING
     flag that we expect to check later.

We can solve this by moving the merge-filter state for each
ref into its "struct ref_item" as soon as we finish the
merge-filter walk. That fixes (2). Then we are free to clear
the commit flags we used in the walk, fixing (1).

Note that we actually do away with the matches_merge_filter
helper entirely here, and inline it between the revision
walk and the flag-clearing. This ensures that nobody
accidentally calls it at the wrong time (it is only safe to
check in that instant between the setting and clearing of
the global flag).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
help: fix the size passed to qsort

We actually want to have the size of one 'name' and not the size
of the pointer.

Signed-off-by: Stefan Beller <stefanbeller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
unblock and unignore SIGPIPE

Blocked and ignored signals -- but not caught signals -- are inherited
across exec.  Some callers with sloppy signal-handling behavior can call
git with SIGPIPE blocked or ignored, even non-deterministically.  When
SIGPIPE is blocked or ignored, several git commands can run indefinitely,
ignoring EPIPE returns from write() calls, even when the process that
called them has gone away.  Our specific case involved a pipe of git
diff-tree output to a script that reads a limited amount of diff data.

In an ideal world, git would never be called with SIGPIPE blocked or
ignored.  But in the real world, several real potential callers, including
Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE
ignored.  It is easier and more productive to harden git against this
mistake than to clean it up in every potential parent process.

Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jk/fsck-exit-code-fix' into jch

"git fsck" failed to report that it found corrupt objects via its
exit status in some cases.

* jk/fsck-exit-code-fix:
  fsck: return non-zero status on missing ref tips
  fsck: exit with non-zero status upon error from fsck_obj()
Merge branch 'ir/makefile-typofix' into jch

* ir/makefile-typofix:
  Makefile: fix some typos in the preamble
Merge branch 'so/rebase-doc' into jch

* so/rebase-doc:
  Documentation/git-rebase.txt: <upstream> must be given to specify <branch>
Merge branch 'wk/pre-push-sample-hook' into jch

* wk/pre-push-sample-hook:
  pre-push.sample: Write error message to stderr
### match next
Merge branch 'jk/write-packed-refs-via-stdio' into jch

Optimize the code path to write out the packed-refs file, which
especially matters in a repository with a large number of refs.

* jk/write-packed-refs-via-stdio:
  refs: write packed_refs file using stdio
Merge branch 'js/fsck-tag-validation' into jch

Teach "git fsck" to inspect the contents of annotated tag objects.

* js/fsck-tag-validation:
  Make sure that index-pack --strict checks tag objects
  Add regression tests for stricter tag fsck'ing
  fsck: check tag objects' headers
  Make sure fsck_commit_buffer() does not run out of the buffer
  fsck_object(): allow passing object data separately from the object itself
  Refactor type_from_string() to allow continuing after detecting an error
Merge branch 'jc/hash-object' into jch

"hash-object" learned a new "--literally" option to hash any random
garbage into a loose object, to allow us to create a test data for
mechanisms to catch corrupt objects.

* jc/hash-object:
  hash-object: add --literally option
  hash-object: pass 'write_object' as a flag
  hash-object: reduce file-scope statics
Merge branch 'jc/hash-object-fsck-tag' into jch

Using "hash-object --literally", test one of the new breakages
js/fsck-tag-validation topic teaches "fsck" to catch is caught.

* jc/hash-object-fsck-tag:
  t1450: make sure fsck detects a malformed tagger line
Merge branch 'jk/faster-name-conflicts' into jch

Optimize the check to see if a ref $F can be created by making sure
no existing ref has $F/ as its prefix, which especially matters in
a repository with a large number of existing refs.

* jk/faster-name-conflicts:
  refs: speed up is_refname_available

# Conflicts:
# 	t/t3210-pack-refs.sh