pack-objects: shrink struct object_entry

Turn some boolean fields into bitfields and use uint32_t for name
hash.  This shrinks the size of the structure from 128 bytes to 120
bytes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4030: demonstrate behavior of show with textconv

"git show <commit>" honors the --textconv option while "git show <blob>"
does not. Demonstrate this in the test.

Since the current behavior is supposed to stay as is, we expect the
default for "git show <blob>" to remain --no-textconv.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff_opt: track whether flags have been set explicitly

The diff_opt infrastructure sets flags based on defaults and command
line options.  It is impossible to tell whether a flag has been set
as a default or on explicit request.  Update the structure so that
this detection is possible:

 * Add an extra "opt->touched_flags" that keeps track of all the
   fields that have been touched by DIFF_OPT_SET and DIFF_OPT_CLR.

 * You may continue setting the default values to the flags, like
   commands in the "log" family do in cmd_log_init_defaults(), but
   after you finished setting the defaults, you clear the
   touched_flags field;

 * And then you let the usual callchain call diff_opt_parse(),
   allowing the opt->flags be set or unset, while keeping track of
   which bits the user touched;

 * There is an optional callback "opt->set_default" that is called
   at the very beginning to let you inspect touched_flags and update
   opt->flags appropriately, before the remainder of the diffcore
   machinery is set up, taking the opt->flags value into account.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
show: honor --textconv for blobs

Currently, "diff" and "cat-file" for blobs honor "--textconv" options
(with the former defaulting to "--textconv" and the latter to
"--no-textconv") whereas "show" does not honor this option, even though
it takes diff options.

Make "show" on blobs honor "--textconv" when it is asked.  The default
is not to apply textconv, which is in line with what "cat-file" does.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cat-file: do not die on --textconv without textconv filters

When a command is supposed to use textconv filters (by default or with
"--textconv") and none are configured then the blob is output without
conversion; the only exception to this rule is "cat-file --textconv".

Make it behave like the rest of textconv aware commands.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7008: demonstrate behavior of grep with textconv

Currently, "git grep" does not honor any textconv filters, with nor
without --textconv. Demonstrate this in the tests.

The default is expected to remain unchanged.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
grep: allow to use textconv filters

Recently and not so recently, we made sure that log/grep type operations
use textconv filters when a userfacing diff would do the same:

ef90ab6 (pickaxe: use textconv for -S counting, 2012-10-28)
b1c2f57 (diff_grep: use textconv buffers for add/deleted files, 2012-10-28)
0508fe5 (combine-diff: respect textconv attributes, 2011-05-23)

"git grep" currently does not use textconv filters at all, that is
neither for displaying the match and context nor for the actual grepping,
even when requested by --textconv.

Introduce an option "--textconv" which makes git grep use any configured
textconv filters for grepping and output purposes. It is off by default.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
grep: honor --textconv for the case rev:path

Make "grep" honor the "--textconv" option also for the object case, i.e.
when used with an argument "rev:path".

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
clear parsed flag when we free tree buffers

Many code paths will free a tree object's buffer and set it
to NULL after finishing with it in order to keep memory
usage down during a traversal. However, out of 8 sites that
do this, only one actually unsets the "parsed" flag back.
Those sites that don't are setting a trap for later users of
the tree object; even after calling parse_tree, the buffer
will remain NULL, causing potential segfaults.

It is not known whether this is triggerable in the current
code. Most commands do not do an in-memory traversal
followed by actually using the objects again. However, it
does not hurt to be safe for future callers.

In most cases, we can abstract this out to a
"free_tree_buffer" helper. However, there are two
exceptions:

  1. The fsck code relies on the parsed flag to know that we
     were able to parse the object at one point. We can
     switch this to using a flag in the "flags" field.

  2. The index-pack code sets the buffer to NULL but does
     not free it (it is freed by a caller). We should still
     unset the parsed flag here, but we cannot use our
     helper, as we do not want to free the buffer.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
setup_reflog_action: document the rules for using GIT_REFLOG_ACTION

The set_reflog_action helper (in git-sh-setup) is designed to be
used once at the very top of a program, like this in "git am", for
example:

	set_reflog_action am

The helper function sets the given string to GIT_REFLOG_ACTION only
when GIT_REFLOG_ACTION is not yet set.  Thanks to this, "git am",
when run as the top-level program, will use "am" in GIT_REFLOG_ACTION
and the reflog entries made by whatever it does will record the
updates of refs done by "am".

Because of the conditional assignment, when "git am" is run as a
subprogram (i.e. an implementation detail) of "git rebase" that
already sets GIT_REFLOG_ACTION to its own name, the call in "git am"
to the helper function at the beginning will *not* have any effect.

So "git rebase" can do this:

	set_reflog_action rebase
	... do its own preparation, like checking out "onto" commit
        ... decide to do "format-patch" to "am" pipeline
        	git format-patch --stdout >mbox
		git am mbox

and the reflog entries made inside "git am" invocation will say
"rebase", not "am".

Calls to "git" commands that update refs would use GIT_REFLOG_ACTION
to record who did that update.  Most such calls in scripted Porcelains
do not define custom reflog message and rely on GIT_REFLOG_ACTION to
contain its (or its caller's, when it is called as a subprogram) name.

If a scripted Porcelain wants to record a custom reflog message for
a single invocation of "git" command (e.g. when "git rebase" uses
"git checkout" to detach HEAD at the commit a series is to be
replayed on), it needs to set GIT_REFLOG_ACTION to the custom
message and export it while calling the "git" command, but such an
assignment must be restricted to that single "git" invocation and
should not be left behind to affect later codepath.

Document the rules to avoid future confusion.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: sanitize P4CHARSET

In the tests, p4d is started without using "internationalized
mode".  Make sure this environment variable is unset, otherwise
a mis-matched user setting would break the tests.  The error
message would be "Unicode clients require a unicode enabled server."

[pw: use unset, add commit text]

Signed-off-by: Kazuki Saitoh <ksaitoh560@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
move setup_alternate_shallow and write_shallow_commits to shallow.c

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --no-index: clarify operation when not inside a repository

Clarify documentation for "diff --no-index".  State that when not
inside a repository, --no-index is implied and two arguments are
mandatory.

Clarify error message from diff-no-index to inform user that CWD is
not inside a repository and thus two arguments are mandatory.

Signed-off-by: Dale Worley <worley@ariadne.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-sha1: add a binary output mode

The test-sha1 helper program will run our internal sha1
routines over its input and output the 40-byte hex sha1.
Sometimes, however, it is useful to have the binary 20-byte
sha1 (and it's a pain to convert back in the shell). Let's
add a "-b" option to output the binary version.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit: search author pattern against mailmap

"git commit --author=$name" sets the author to one whose name
matches the given string from existing commits, when $name is not in
the "Name <e-mail>" format. However, it does not honor the mailmap
to use the canonical name for the author found this way.

Fix it by telling the logic to find a matching existing author to
honor the mailmap, and use the name and email after applying the
mailmap.

Signed-off-by: Antoine Pelisse <apelisse@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1-lookup: handle duplicate keys with GIT_USE_LOOKUP

The sha1_entry_pos function tries to be smart about
selecting the middle of a range for its binary search by
looking at the value differences between the "lo" and "hi"
constraints. However, it is unable to cope with entries with
duplicate keys in the sorted list.

We may hit a point in the search where both our "lo" and
"hi" point to the same key. In this case, the range of
values between our endpoints is 0, and trying to scale the
difference between our key and the endpoints over that range
is undefined (i.e., divide by zero). The current code
catches this with an "assert(lov < hiv)".

Moreover, after seeing that the first 20 byte of the key are
the same, we will try to establish a value from the 21st
byte. Which is nonsensical.

Instead, we can detect the case that we are in a run of
duplicates, and simply do a final comparison against any one
of them (since they are all the same, it does not matter
which). If the keys match, we have found our entry (or one
of them, anyway).  If not, then we know that we do not need
to look further, as we must be in a run of the duplicate
key.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
add tests for indexing packs with delta cycles

If we receive a broken or malicious pack from a remote, we
will feed it to index-pack. As index-pack processes the
objects as a stream, reconstructing and hashing each object
to get its name, it is not very susceptible to doing the
wrong with bad data (it simply notices that the data is
bogus and aborts).

However, one question raised on the list is whether it could
be susceptible to problems during the delta-resolution
phase. In particular, can a cycle in the packfile deltas
cause us to go into an infinite loop or cause any other
problem?

The answer is no.

We cannot have a cycle of delta-base offsets, because they
go only in one direction (the OFS_DELTA object mentions its
base by an offset towards the beginning of the file, and we
explicitly reject negative offsets).

We can have a cycle of REF_DELTA objects, which refer to
base objects by sha1 name. However, index-pack does not know
these sha1 names ahead of time; it has to reconstruct the
objects to get their names, and it cannot do so if there is
a delta cycle (in other words, it does not even realize
there is a cycle, but only that there are items that cannot
be resolved).

Even though we can reason out that index-pack should handle
this fine, let's add a few tests to make sure it behaves
correctly.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test index-pack on packs with recoverable delta cycles

The previous commit added tests to show that index-pack
correctly bails in unrecoverable situations. There are some
situations where the data could be recovered, but it is not
currently:

  1. If we can break the cycle using an object from another
     pack via --fix-thin.

  2. If we can break the cycle using a duplicate of one of
     the objects found in the same pack.

Note that neither of these is particularly high priority; a
delta cycle within a pack should never occur, and we have no
record of even a buggy git implementation creating such a
pack.

However, it's worth adding these tests for two reasons. One,
to document that we do not currently handle the situation,
even though it is possible. And two, to exercise the code
that runs in this situation; even though it fails, by
running it we can confirm that index-pack detects the
situation and aborts, and does not misbehave (e.g., by
following the cycle in an infinite loop).

In both cases, we hit an assert that aborts index-pack.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make test "using invalid commit with -C" more strict

In the test 'using invalid commit with -C' git-commit would have failed
even if the -C option had been given the correct commit, as there was
nothing to commit. Pass --allow-empty to make sure it would make a commit,
were there no issues with the argument given to the -C option.

Signed-off-by: Kacper Kornet <draenog@pld-linux.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
add -i test: use skip_all instead of repeated PERL prerequisite

It is too easy to forget to add the PERL prerequisite for new
"add -i" tests, especially given that many people do not test with
NO_PERL so the missing prereq is not always noticed quickly.

The test had used the skip_all mechanism since 1b19ccd2 (2009-04-03)
but switched to explicit PERL prereqs in f0459319 (2010-10-13) in hope
of helping people see how many tests were skipped, perhaps to motivate
them to tweak their platform or tests to improve test coverage.  That
didn't pan out much in practice, so let's move back to the simpler
skip_all method.

Reported-by: Kacper Kornet <draenog@pld-linux.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>