pack-objects: do not reuse packfiles without --delta-base-offset

When we are sending a packfile to a remote, we currently try
to reuse a whole chunk of packfile without bothering to look
at the individual objects. This can make things like initial
clones much lighter on the server, as we can just dump the
packfile bytes.

However, it's possible that the other side cannot read our
packfile verbatim. For example, we may have objects stored
as OFS_DELTA, but the client is an antique version of git
that only understands REF_DELTA. We negotiate this
capability over the fetch protocol. A normal pack-objects
run will convert OFS_DELTA into REF_DELTA on the fly, but
the "reuse pack" code path never even looks at the objects.

This patch disables packfile reuse if the other side is
missing any capabilities that we might have used in the
on-disk pack. Right now the only one is OFS_DELTA, but we
may need to expand in the future (e.g., if packv4 introduces
new object types).

We could be more thorough and only disable reuse in this
case when we actually have an OFS_DELTA to send, but:

  1. We almost always will have one, since we prefer
     OFS_DELTA to REF_DELTA when possible. So this case
     would almost never come up.

  2. Looking through the objects defeats the purpose of the
     optimization, which is to do as little work as possible
     to get the bytes to the remote.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'ks/tree-diff-nway' (early part) into jch

* 'ks/tree-diff-nway' (early part):
  tree-diff: reuse base str(buf) memory on sub-tree recursion
  tree-diff: no need to call "full" diff_tree_sha1 from show_path()
  tree-diff: rework diff_tree interface to be sha1 based
  tree-diff: diff_tree() should now be static
  tree-diff: remove special-case diff-emitting code for empty-tree cases
  tree-diff: simplify tree_entry_pathcmp
  tree-diff: show_path prototype is not needed anymore
  tree-diff: rename compare_tree_entry -> tree_entry_pathcmp
  tree-diff: move all action-taking code out of compare_tree_entry()
  tree-diff: don't assume compare_tree_entry() returns -1,0,1
  tree-diff: consolidate code for emitting diffs and recursion in one place
  tree-diff: show_tree() is not needed
  tree-diff: no need to pass match to skip_uninteresting()
  tree-diff: no need to manually verify that there is no mode change for a path
  combine-diff: move changed-paths scanning logic into its own function
  combine-diff: move show_log_first logic/action out of paths scanning
Merge branch 'jc/rev-parse-argh-dashed-multi-words' into jch

Make sure that the help text given to describe the "<param>" part
of the "git cmd --option=<param>" does not contain SP or _,
e.g. "--gpg-sign=<key-id>" option for "git commit" is not spelled
as "--gpg-sign=<key id>".

* jc/rev-parse-argh-dashed-multi-words:
  parse-options: make sure argh string does not have SP or _
  update-index: teach --cacheinfo a new syntax "mode,sha1,path"
  parse-options: multi-word argh should use dash to separate words
Merge branch 'mr/msvc-link-with-invalidcontinue' into jch

* mr/msvc-link-with-invalidcontinue:
  MSVC: link in invalidcontinue.obj for better POSIX compatibility
Merge branch 'ib/rev-parse-parseopt-argh' into jch

Finishing touch to a new topic scheduled for 2.0

* ib/rev-parse-parseopt-argh:
  rev-parse: fix typo in example on manpage
Merge branch 'as/grep-fullname-config' into jch

Add a configuration variable to force --full-name to be default for
"git grep".

This may cause regressions on scripted users that do not expect
this new behaviour.

* as/grep-fullname-config:
  grep: add grep.fullName config variable
Merge branch 'mr/opt-set-ptr' into jch

OPT_SET_PTR() implementation was broken on IL32P64 platforms;
it turns out that the macro is not used by any real user.

* mr/opt-set-ptr:
  parse-options: remove unused OPT_SET_PTR
  parse-options: add cast to correct pointer type to OPT_SET_PTR
  MSVC: fix t0040-parse-options crash
Merge branch 'jc/apply-ignore-whitespace' into jch

"--ignore-space-change" option of "git apply" ignored the
spaces at the beginning of line too aggressively, which is
inconsistent with the option of the same name "diff" and "git diff"
have.

* jc/apply-ignore-whitespace:
  apply --ignore-space-change: lines with and without leading whitespaces do not match
Merge branch 'jk/commit-dates-parsing-fix' into jch

* jk/commit-dates-parsing-fix:
  t4212: loosen far-in-future test for AIX
  date: recognize bogus FreeBSD gmtime output

# Conflicts:
# 	Makefile
Merge branch 'mh/update-ref-batch-create-fix' into jch

* mh/update-ref-batch-create-fix:
  update-ref: fail create operation over stdin if ref already exists
Merge branch 'jl/nor-or-nand-and' into jch

Eradicate mistaken use of "nor" (that is, essentially "nor" used
not in "neither A nor B" ;-)) from in-code comments, command output
strings, and documentations.

* jl/nor-or-nand-and:
  code and test: fix misuses of "nor"
  comments: fix misuses of "nor"
  contrib: fix misuses of "nor"
  Documentation: fix misuses of "nor"
Merge branch 'jk/pack-bitmap' into jch

* jk/pack-bitmap:
  pack-objects: do not reuse packfiles without --delta-base-offset
  add `ignore_missing_links` mode to revwalk
### match next
status/commit: show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for the diff family,
status and commit. For status and commit this is really confusing, as it
even when the user chooses to record a new commit for an ignored submodule
by adding it manually this change won't show up under the to-be-committed
changes. To add insult to injury, a later "git commit" will error out with
"nothing to commit" when only ignored submodules are staged.

Fix that by making wt_status always print staged submodule changes, no
matter what ignore settings are configured. The only exception is when the
user explicitly uses the "--ignore-submodules=all" command line option, in
that case the submodule output is still suppressed. This also makes "git
commit" work again when only modifications of ignored submodules are
staged, as that command uses the "commitable" member of the wt_status
struct to determine if staged changes are present. But this only happens
when the commit command uses the wt_status* functions to produce status
output for human consumption (when forking an editor or with --dry-run),
in all other cases (e.g. when run in a script with '-m') another code path
is taken which uses index_differs_from() to determine if any changes are
staged which still ignores submodules according to their configuration.
This will be fixed in a follow-up commit.

Change t7508 to reflect this new behavior and add three new tests to show
that a single staged submodule configured to be ignored will be committed
when the status output is generated and won't be if not. Also update the
documentation of the ignore config options accordingly.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit -m: commit staged submodules regardless of ignore config

The previous commit fixed the problem that the staged but that ignored
submodules did not show up in the status output of the commit command and
weren't committed afterwards either. But when commit doesn't generate the
status output (e.g. when used in a script with '-m') the ignored submodule
will still not be committed. This is because in that case a different code
path is taken which calls index_differs_from() instead of calling the
wt_status functions.

Fix that by calling index_differs_from() from builtin/commit.c with a
diff_options argument value that tells it not ignore any submodule changes
unless the '--ignore-submodules' option is used. Even though this option
isn't yet implemented for cmd_commit() but only for cmd_status() this
prepares cmd_commit() to correctly handle the '--ignore-submodules' option
later. As status and commit share the same ignore_submodule_arg variable
this makes the code more robust against accidental breakage and documents
how to correctly call index_differs_from().

Change the expected result of the test documenting this problem from
failure to success.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-multimail: update to version 1.0.0

This commit contains the squashed changes from the upstream
git-multimail repository since the last code drop.  Highlights:

* Fix encoding of non-ASCII email addresses in email headers.

* Fix backwards-compatibility bugs for older Python 2.x versions.

* Fix a backwards-compatibility bug for Git 1.7.1.

* Add an option commitDiffOpts to customize logs for revisions.

* Pass "-oi" to sendmail by default to prevent premature
  termination
  on a line containing only ".".

* Stagger email "Date:" values in an attempt to help mail clients
  thread the emails in the right order.

* If a mailing list setting is missing, just skip sending the
  corresponding email (with a warning) instead of failing.

* Add a X-Git-Host header that can be used for email filtering.

* Allow the sender's fully-qualified domain name to be configured.

* Minor documentation improvements.

* Add a CHANGES file.

Contributions-by: Raphaël Hertzog <hertzog@debian.org>
Contributions-by: Eric Berberich <eric.berberich@gmail.com>
Contributions-by: Michiel Holtkamp <git@elfstone.nl>
Contributions-by: Malte Swart <mswart@devtation.de>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: fix name and expected result of one test

The test

    stdin -z create ref fails with zero new value

actually passes an empty new value, not a zero new value.  So rename
the test s/zero/empty/, and change the expected error from

    fatal: create $c given zero new value

to

    fatal: create $c missing <newvalue>

Of course, this makes the test fail now, because although "git
update-ref" tries to distinguish between these two errors, it does not
succeed in this situation.  Fixing it is more than a one-liner, so
mark the test test_expect_failure for now.  The failure will be fixed
later in this patch series.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: provide more usual input to the command

The old version was passing (among other things)

    update SP refs/heads/c NUL NUL 0{40} NUL

to "git update-ref -z --stdin" to test whether the old-value check for
c is working.  But the <newvalue> is empty, which is a bit off the
beaten track.

So, to be sure that we are testing what we want to test, provide an
actual <newvalue> on the "update" line.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse_arg(): really test that argument is properly terminated

The old parse_arg(), when fed an argument

    "refs/heads/a"master

parsed 'refs/heads/a' off of the front of the argument and considered
itself successful.  It was only when parse_next_arg() tried to parse
the *next* argument that a problem was noticed.  But in fact, the
definition of the input format requires arguments to be terminated by
SP or NUL, so *this* argument is already erroneous and parse_arg()
should diagnose the problem.

So teach parse_arg() to verify that C-quoted arguments are terminated
correctly.  If not, emit a more specific error message.

There is no corresponding error case of a non-C-quoted argument that
is not terminated correctly, because the end of a non-quoted argument
is *by definition* a space or NUL, so there is no way to insert other
junk between the "end" of the argument and the argument terminator.

Adjust the tests to expect the new error message.  Add a docstring to
the function, incorporating the comments that were formerly within the
function plus some added information.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: add some more tests involving quoted arguments

Previously there were no good tests of C-quoted arguments.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>