push: the beginning of "git push --signed"

While signed tags and commits assert that the objects thusly signed
came from you, who signed these objects, there is not a good way to
assert that you wanted to have a particular object at the tip of a
particular branch.  My signing v2.0.1 tag only means I want to call
the version v2.0.1, and it does not mean I want to push it out to my
'master' branch---it is likely that I only want it in 'maint', so
the signature on the object alone is insufficient.

The only assurance to you that 'maint' points at what I wanted to
place there comes from your trust on the hosting site and my
authentication with it, which cannot easily audited later.

Introduce a mechanism that allows you to sign a "push certificate"
(for the lack of better name) every time you push, asserting that
what object you are pushing to update which ref that used to point
at what other object.  Think of it as a cryptographic protection for
ref updates, similar to signed tags/commits but working on an
orthogonal axis.

The basic flow based on this mechanism goes like this:

 1. You push out your work with "git push --signed".

 2. The sending side learns where the remote refs are as usual,
    together with what protocol extension the receiving end
    supports.  If the receiving end does not advertise the protocol
    extension "push-cert", an attempt to "git push --signed" fails.

    Otherwise, a text file, that looks like the following, is
    prepared in core:

	certificate version 0.1
	pusher Junio C Hamano <gitster@pobox.com> 1315427886 -0700

	7339ca65... 21580ecb... refs/heads/master
	3793ac56... 12850bec... refs/heads/next

    The file begins with a few header lines, which may grow as we
    gain more experience.  The 'pusher' header records the name of
    the signer (the value of user.signingkey configuration variable,
    falling back to GIT_COMMITTER_{NAME|EMAIL}) and the time of the
    certificate generation.  After the header, a blank line follows,
    followed by a copy of the protocol message lines.

    Each line shows the old and the new object name at the tip of
    the ref this push tries to update, in the way identical to how
    the underlying "git push" protocol exchange tells the ref
    updates to the receiving end (by recording the "old" object
    name, the push certificate also protects against replaying).  It
    is expected that new command packet types other than the
    old-new-refname kind will be included in push certificate in the
    same way as would appear in the plain vanilla command packets in
    unsigned pushes.

    The user then is asked to sign this push certificate using GPG,
    formatted in a way similar to how signed tag objects are signed,
    and the result is sent to the other side (i.e. receive-pack).

    In the protocol exchange, this step comes immediately before the
    sender tells what the result of the push should be, which in
    turn comes before it sends the pack data.

 3. When the receiving end sees a push certificate, the certificate
    is written out as a blob.  The pre-receive hook can learn about
    the certificate by checking GIT_PUSH_CERT environment variable,
    which, if present, tells the object name of this blob, and make
    the decision to allow or reject this push.  Additionally, the
    post-receive hook can also look at the certificate, which may be
    a good place to log all the received certificates for later
    audits.

Because a push certificate carry the same information as the usual
command packets in the protocol exchange, we can omit the latter
when a push certificate is in use and reduce the protocol overhead.
This however is not included in this patch to make it easier to
review (in other words, the series at this step should never be
released without the remainder of the series, as it implements an
interim protocol that will be incompatible with the final one,
merely for reviewing purposes).  As such, the documentation update
for the protocol is left out of this step.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
receive-pack: GPG-validate push certificates

Reusing the GPG signature check helpers we already have, verify
the signature in receive-pack and give the results to the hooks
via GIT_PUSH_CERT_{SIGNER,KEY,STATUS} environment variables.

Policy decisions, such as accepting or rejecting a good signature by
a key that is not fully trusted, is left to the hook and kept
outside of the core.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
send-pack: send feature request on push-cert packet

We would want to update the interim protocol so that we do not send
the usual update commands when the push certificate feature is in
use, as the same information is in the certificate.  Once that
happens, the push-cert packet may become the only protocol command,
but then there is no packet to put the feature request behind, like
we always did.

As we have prepared the receiving end that understands the push-cert
feature to accept the feature request on the first protocol packet
(other than "shallow ", which was an unfortunate historical mistake
that has to come before everything else), we can give the feature
request on the push-cert packet instead of the first update protocol
packet, in preparation for the next step to actually update to the
final protocol.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: remove duplicated protocol info

With the interim protocol, we used to send the update commands even
though we already send a signed copy of the same information when
push certificate is in use.  Update the send-pack/receive-pack pair
not to do so.

The notable thing on the receive-pack side is that it makes sure
that there is no command sent over the traditional protocol packet
outside the push certificate.  Otherwise a pusher can claim to be
pushing one set of ref updates in the signed certificate while
issuing commands to update unrelated refs, and such an update will
evade later audits.

Finally, start documenting the protocol.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: add "pushee" header to push certificate

Record the URL of the intended recipient for a push (after
anonymizing it if it has authentication material) on a new "pushee
URL" header.  Because the networking configuration (SSH-tunnels,
proxies, etc.) on the pushing user's side varies, the receiving
repository may not know the single canonical URL all the pushing
users would refer it as (besides, many sites allow pushing over
ssh://host/path and https://host/path protocols to the same
repository but with different local part of the path).  So this
value may not be reliably used for replay-attack prevention
purposes, but this will still serve as a human readable hint to
identify the repository the certificate refers to.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: fortify against replay attacks

In order to prevent a valid push certificate for pushing into an
repository from getting replayed to push to an unrelated one, send a
nonce string from the receive-pack process and have the signer
include it in the push certificate.  The receiving end uses an HMAC
hash of the path to the repository it serves and the current time,
hashed with a secret key (this does not have to be per-repository
but can be defined in /etc/gitconfig) to ensure that a random third
party cannot forge a nonce that looks like it originated from it.

The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks
to examine and match against the value on the "nonce" header in the
certificate to notice a replay.  Returned "nonce" header is examined
and the validation state is exported as GIT_PUSH_CERT_NONCE_STATUS
to the hooks.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
signed push: allow stale nonce in stateless mode

When operating with the stateless RPC mode, we will receive a nonce
issued by another instance of us that advertised our capability and
refs some time ago.  Update the logic to check received nonce to
detect this case, compute how much time has passed since the nonce
was issued and report the status with a new environment variable
GIT_PUSH_CERT_NONCE_SLOP to the hooks.

GIT_PUSH_CERT_NONCE_STATUS will report "SLOP" in such a case.  The
hooks are free to decide how large a slop it is willing to accept.

Strictly speaking, the "nonce" is not really a "nonce" anymore in
the stateless RPC mode, as it will happily take any "nonce" issued
by it (which is protected by HMAC and its secret key) as long as it
is fresh enough.  The degree of this security degradation, relative
to the native protocol, is about the same as the "we make sure that
the 'git push' decided to update our refs with new objects based on
the freshest observation of our refs by making sure the values they
claim the original value of the refs they ask us to update exactly
match the current state" security is loosened to accomodate the
stateless RPC mode in the existing code without this series, so
there is no need for those who are already using smart HTTP to push
to their repositories to be alarmed any more than they already are.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jc/push-cert' into jch

Allow "git push" request to be signed, so that it can be verified
audited, using the GPG signature of the person who pushed, that the
tips of branches at a public repository really point the commits
the pusher wanted to, without having to "trust" the server.

The tip one feels still a bit slushy, but otherwise should be
reasonably solid to build upon.

* jc/push-cert: (22 commits)
  signed push: allow stale nonce in stateless mode
  signed push: fortify against replay attacks
  signed push: add "pushee" header to push certificate
  signed push: remove duplicated protocol info
  send-pack: send feature request on push-cert packet
  receive-pack: GPG-validate push certificates
  push: the beginning of "git push --signed"
  pack-protocol doc: typofix for PKT-LINE
  gpg-interface: move parse_signature() to where it should be
  gpg-interface: move parse_gpg_output() to where it should be
  send-pack: clarify that cmds_sent is a boolean
  send-pack: refactor inspecting and resetting status and sending commands
  send-pack: rename "new_refs" to "need_pack_data"
  receive-pack: factor out capability string generation
  send-pack: factor out capability string generation
  send-pack: always send capabilities
  send-pack: refactor decision to send update per ref
  send-pack: move REF_STATUS_REJECT_NODELETE logic a bit higher
  receive-pack: factor out queueing of command
  receive-pack: do not reuse old_sha1[] for other things
  ...
Merge branch 'hv/submodule-config' into pu

Kicked back to 'pu'.

* hv/submodule-config:
  do not die on error of parsing fetchrecursesubmodules option
  use new config API for worktree configurations of submodules
  extract functions for submodule config set and lookup
  implement submodule config cache for lookup of submodule names
###
Merge branch 'rr/mergetool-temporary-filename-tweak' into pu

* rr/mergetool-temporary-filename-tweak:
  Allow the user to change the temporary file name for mergetool
Merge branch 'rh/prompt-tests' into pu

to be broken with older zsh that are still in the field ($gmane/251231).

* rh/prompt-tests:
  t9904: new __git_ps1 tests for Zsh
  test-lib: make it possible to override how test code is eval'd
  lib-prompt-tests.sh: add variable for string that encodes percent in PS1
  lib-prompt-tests.sh: put all tests inside a function
  t9903: move prompt tests to a new lib-prompt-tests.sh file
  t9903: move PS1 color code variable definitions to lib-bash.sh
  t9903: include "Bash" in test names via new $shellname var
  t9903: run pc mode tests again with PS1 expansion disabled
  t9903: move test name prefix to a separate variable
  t9903: put the Bash pc mode prompt test cases in a function
  t9903: remove Zsh test from the suite of Bash prompt tests
Merge branch 'bg/rebase-off-of-previous-branch' into pu

Teach "git rebase -" to report the concrete name of the branch
(i.e. the previous one).

But it stops short and does not do the same for "git rebase @{-1}".

* bg/rebase-off-of-previous-branch:
  git-rebase: print name of rev when using shorthand
Merge branch 'rb/merge-prepare-commit-msg-hook' into pu

Expose more merge states (e.g. $GIT_DIR/MERGE_MODE) to hooks that
run during "git merge".  The log message stresses too much on one
hook, prepare-commit-msg, but it would equally apply to other hooks
like post-merge, I think.

* rb/merge-prepare-commit-msg-hook:
  merge: drop unused arg from abort_commit method signature
  merge: make prepare_to_commit responsible for write_merge_state
  t7505: ensure cleanup after hook blocks merge
  t7505: add missing &&

# Conflicts:
# 	builtin/merge.c
# 	t/t7505-prepare-commit-msg-hook.sh
Merge branch 'jc/show-branch' into pu

* jc/show-branch:
  show-branch: use commit slab to represent bitflags of arbitrary width
  show-branch.c: remove "all_mask"
  show-branch.c: abstract out "flags" operation
  show-branch.c: lift all_mask/all_revs to a global static
  show-branch.c: update comment style
Merge branch 'tr/merge-recursive-index-only' into pu

* tr/merge-recursive-index-only:
  merge-recursive: -Xindex-only to leave worktree unchanged
  merge-recursive: internal flag to avoid touching the worktree
  merge-recursive: remove dead conditional in update_stages()
Merge branch 'tg/perf-lib-test-perf-cleanup' into pu

Add test_perf_cleanup shell function to the perf suite, that allows
the script writers to define a test with a clean-up action.

Will hold.

* tg/perf-lib-test-perf-cleanup:
  perf-lib: add test_perf_cleanup target
  perf-lib: split starting the test from the execution
Merge branch 'ab/add-interactive-show-diff-func-name' into pu

* ab/add-interactive-show-diff-func-name:
  SQUASH??? git-add--interactive: Preserve diff heading when splitting hunks
  git-add--interactive: Preserve diff heading when splitting hunks
Merge branch 'ss/userdiff-update-csharp-java' into pu

Reviews sent; waiting for a response.

* ss/userdiff-update-csharp-java:
  userdiff: support Java try keyword
  userdiff: support C# async methods and correct C# keywords
Merge branch 'jn/gitweb-utf8-in-links' into pu

$gmane/250758?

* jn/gitweb-utf8-in-links:
  gitweb: Harden UTF-8 handling in generated links