pack-objects: do not reuse packfiles without --delta-base-offset

When we are sending a packfile to a remote, we currently try
to reuse a whole chunk of packfile without bothering to look
at the individual objects. This can make things like initial
clones much lighter on the server, as we can just dump the
packfile bytes.

However, it's possible that the other side cannot read our
packfile verbatim. For example, we may have objects stored
as OFS_DELTA, but the client is an antique version of git
that only understands REF_DELTA. We negotiate this
capability over the fetch protocol. A normal pack-objects
run will convert OFS_DELTA into REF_DELTA on the fly, but
the "reuse pack" code path never even looks at the objects.

This patch disables packfile reuse if the other side is
missing any capabilities that we might have used in the
on-disk pack. Right now the only one is OFS_DELTA, but we
may need to expand in the future (e.g., if packv4 introduces
new object types).

We could be more thorough and only disable reuse in this
case when we actually have an OFS_DELTA to send, but:

  1. We almost always will have one, since we prefer
     OFS_DELTA to REF_DELTA when possible. So this case
     would almost never come up.

  2. Looking through the objects defeats the purpose of the
     optimization, which is to do as little work as possible
     to get the bytes to the remote.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
status/commit: show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for the diff family,
status and commit. For status and commit this is really confusing, as it
even when the user chooses to record a new commit for an ignored submodule
by adding it manually this change won't show up under the to-be-committed
changes. To add insult to injury, a later "git commit" will error out with
"nothing to commit" when only ignored submodules are staged.

Fix that by making wt_status always print staged submodule changes, no
matter what ignore settings are configured. The only exception is when the
user explicitly uses the "--ignore-submodules=all" command line option, in
that case the submodule output is still suppressed. This also makes "git
commit" work again when only modifications of ignored submodules are
staged, as that command uses the "commitable" member of the wt_status
struct to determine if staged changes are present. But this only happens
when the commit command uses the wt_status* functions to produce status
output for human consumption (when forking an editor or with --dry-run),
in all other cases (e.g. when run in a script with '-m') another code path
is taken which uses index_differs_from() to determine if any changes are
staged which still ignores submodules according to their configuration.
This will be fixed in a follow-up commit.

Change t7508 to reflect this new behavior and add three new tests to show
that a single staged submodule configured to be ignored will be committed
when the status output is generated and won't be if not. Also update the
documentation of the ignore config options accordingly.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit -m: commit staged submodules regardless of ignore config

The previous commit fixed the problem that the staged but that ignored
submodules did not show up in the status output of the commit command and
weren't committed afterwards either. But when commit doesn't generate the
status output (e.g. when used in a script with '-m') the ignored submodule
will still not be committed. This is because in that case a different code
path is taken which calls index_differs_from() instead of calling the
wt_status functions.

Fix that by calling index_differs_from() from builtin/commit.c with a
diff_options argument value that tells it not ignore any submodule changes
unless the '--ignore-submodules' option is used. Even though this option
isn't yet implemented for cmd_commit() but only for cmd_status() this
prepares cmd_commit() to correctly handle the '--ignore-submodules' option
later. As status and commit share the same ignore_submodule_arg variable
this makes the code more robust against accidental breakage and documents
how to correctly call index_differs_from().

Change the expected result of the test documenting this problem from
failure to success.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-multimail: update to version 1.0.0

This commit contains the squashed changes from the upstream
git-multimail repository since the last code drop.  Highlights:

* Fix encoding of non-ASCII email addresses in email headers.

* Fix backwards-compatibility bugs for older Python 2.x versions.

* Fix a backwards-compatibility bug for Git 1.7.1.

* Add an option commitDiffOpts to customize logs for revisions.

* Pass "-oi" to sendmail by default to prevent premature
  termination
  on a line containing only ".".

* Stagger email "Date:" values in an attempt to help mail clients
  thread the emails in the right order.

* If a mailing list setting is missing, just skip sending the
  corresponding email (with a warning) instead of failing.

* Add a X-Git-Host header that can be used for email filtering.

* Allow the sender's fully-qualified domain name to be configured.

* Minor documentation improvements.

* Add a CHANGES file.

Contributions-by: Raphaël Hertzog <hertzog@debian.org>
Contributions-by: Eric Berberich <eric.berberich@gmail.com>
Contributions-by: Michiel Holtkamp <git@elfstone.nl>
Contributions-by: Malte Swart <mswart@devtation.de>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: fix name and expected result of one test

The test

    stdin -z create ref fails with zero new value

actually passes an empty new value, not a zero new value.  So rename
the test s/zero/empty/, and change the expected error from

    fatal: create $c given zero new value

to

    fatal: create $c missing <newvalue>

Of course, this makes the test fail now, because although "git
update-ref" tries to distinguish between these two errors, it does not
succeed in this situation.  Fixing it is more than a one-liner, so
mark the test test_expect_failure for now.  The failure will be fixed
later in this patch series.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: provide more usual input to the command

The old version was passing (among other things)

    update SP refs/heads/c NUL NUL 0{40} NUL

to "git update-ref -z --stdin" to test whether the old-value check for
c is working.  But the <newvalue> is empty, which is a bit off the
beaten track.

So, to be sure that we are testing what we want to test, provide an
actual <newvalue> on the "update" line.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse_arg(): really test that argument is properly terminated

The old parse_arg(), when fed an argument

    "refs/heads/a"master

parsed 'refs/heads/a' off of the front of the argument and considered
itself successful.  It was only when parse_next_arg() tried to parse
the *next* argument that a problem was noticed.  But in fact, the
definition of the input format requires arguments to be terminated by
SP or NUL, so *this* argument is already erroneous and parse_arg()
should diagnose the problem.

So teach parse_arg() to verify that C-quoted arguments are terminated
correctly.  If not, emit a more specific error message.

There is no corresponding error case of a non-C-quoted argument that
is not terminated correctly, because the end of a non-quoted argument
is *by definition* a space or NUL, so there is no way to insert other
junk between the "end" of the argument and the argument terminator.

Adjust the tests to expect the new error message.  Add a docstring to
the function, incorporating the comments that were formerly within the
function plus some added information.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: add some more tests involving quoted arguments

Previously there were no good tests of C-quoted arguments.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
refs.h: rename the action_on_err constants

Given that these constants are only being used when updating
references, it is inappropriate to give them such generic names as
"DIE_ON_ERR".  So prefix their names with "UPDATE_REFS_".

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update_refs(): fix constness

The old signature of update_refs() required a
(const struct ref_update **) for its updates_orig argument.  The
"const" is presumably there to promise that the function will not
modify the contents of the structures.

But this declaration does not permit the function to be called with a
(struct ref_update **), which is perfectly legitimate.  C's type
system is not powerful enough to express what we'd like.  So remove
the first "const" from the declaration.

On the other hand, the function *can* promise not to modify the
pointers within the array that is passed to it without inconveniencing
its callers.  So add a "const" that has that effect, making the final
declaration
(struct ref_update * const *).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref --stdin: read the whole input at once

Read the whole input into a strbuf at once, and then parse it from
there.  This might also be a tad faster, but that is not the point.
The point is to decouple the parsing code from the input source (the
old parsing code had to read new data even in the middle of commands).
Add docstrings for the parsing functions.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse_cmd_verify(): copy old_sha1 instead of evaluating <oldvalue> twice

Aside from avoiding a tiny bit of work, this makes it transparently
obvious that old_sha1 and new_sha1 are identical.  It is arguably a
bit silly to have to set new_sha1 in order to verify old_sha1, but
that is a problem for another day.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref.c: extract a new function, parse_refname()

There is no reason to obscure the fact that parse_first_arg() always
parses refnames.  Form the new function by combining parse_first_arg()
and update_store_ref_name().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref --stdin: improve error messages for invalid values

If an invalid value is passed to "update-ref --stdin" as <oldvalue> or
<newvalue>, include the command and the name of the reference at the
beginning of the error message.  Update the tests accordingly.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref --stdin: make error messages more consistent

The old error messages emitted for invalid input sometimes said
"<oldvalue>"/"<newvalue>" and sometimes said "old value"/"new value".
Convert them all to the former.  Update the tests accordingly.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref --stdin: simplify error messages for missing oldvalues

Instead of, for example,

    fatal: update refs/heads/master missing [<oldvalue>] NUL

emit

    fatal: update refs/heads/master missing <oldvalue>

Update the tests accordingly.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: test that stdin -z update treats empty <newvalue> as zeros

This is the (slightly inconsistent) status quo; make sure it doesn't
change by accident.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref.c: extract a new function, parse_next_sha1()

Replace three functions, update_store_new_sha1(),
update_store_old_sha1(), and parse_next_arg(), with a single function,
parse_next_sha1().  The new function takes care of a whole argument,
including checking whether it is there, converting it to an SHA-1, and
emitting errors on EOF or for invalid values.  The return value
indicates whether the argument was present or absent, which requires
a bit of intelligence because absent values are represented
differently depending on whether "-z" was used.

The new interface means that the calling functions, parse_cmd_*(),
don't have to interpret the result differently based on the
line_termination mode that is in effect.  It also means that
parse_cmd_create() can distinguish unambiguously between an empty new
value and a zeros new value, which fixes a failure in t1400.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
update-ref --stdin -z: deprecate interpreting the empty string as zeros

In the original version of this command, for the single case of the
"update" command's <newvalue>, the empty string was interpreted as
being equivalent to 40 "0"s.  This shorthand is unnecessary (binary
input will usually be generated programmatically anyway), and it
complicates the parser and the documentation.

So gently deprecate this usage: remove its description from the
documentation and emit a warning if it is found.  But for reasons of
backwards compatibility, continue to accept it.

Helped-by: Brad King <brad.king@kitware.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t1400: test one mistake at a time

This case wants to test passing a bad refname to the "update" command.
But it also passes too few arguments to "update", which muddles the
situation: which error should be diagnosed?  So split this test into
two:

* One that passes too few arguments to update

* One that passes all three arguments to "update", but with a bad
  refname.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>