Merge branch 'jc/reflog-reverse-walk' into nd/branch-show-rebase-bisect-state

* jc/reflog-reverse-walk:
  reflog: add for_each_reflog_ent_reverse() API
  for_each_recent_reflog_ent(): simplify opening of a reflog file
  for_each_reflog_ent(): extract a helper to process a single entry
Allow combined diff to ignore white-spaces

The combined diff --cc output does not honor options to ignore
whitespace changes (-b, -w, and --ignore-space-at-eol).

Correct this by passing diff flags to diff engine, so that combined
diff behaves as normal diff does with spaces, and by coalescing
lines that are removed from both (or more) parents, honoring the
same rule to ignore whitespace changes.

With this change, a conflict-less merge done using a ignore-*
strategy option will not show any conflict if shown in combined-diff
using the same option.

Signed-off-by: Antoine Pelisse <apelisse@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git status: document trade-offs in choosing parameters to the -u option

In some repostories users experience that "git status" command takes
long time.  The command spends some time searching the file system
for untracked files.

Explain the trade-off struck by the default choice of `normal` to
help users make an appropriate choice better, before talking about
the configuration variable.

Inspired by Torsten Bögershausen.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
status: advise to consider use of -u when read_directory takes too long

Introduce advice.statusUoption to suggest considering use of -u to
strike different trade-off when it took more than 2 seconds to
enumerate untracked/ignored files.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
archive-zip: use deflateInit2() to ask for raw compressed data

We use the function git_deflate_init() -- which wraps the zlib function
deflateInit() -- to initialize compression of ZIP file entries.  This
results in compressed data prefixed with a two-bytes long header and
followed by a four-bytes trailer.  ZIP file entries consist of ZIP
headers and raw compressed data instead, so we remove the zlib wrapper
before writing the result.

We can ask zlib for the the raw compressed data without the unwanted
parts in the first place by using deflateInit2() and specifying a
negative number of bits to size the window.  For that purpose, factor
out the function do_git_deflate_init() and add git_deflate_init_raw(),
which wraps it.  Then use the latter in archive-zip.c and get rid of
the code that stripped the zlib header and trailer.

Also rename the helper function zlib_deflate() to zlib_deflate_raw()
to reflect the change.

Thus we avoid generating data that we throw away anyway, the code
becomes shorter and some magic constants are removed.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
index-pack: fix buffer overflow caused by translations

The translation of "completed with %d local objects" is put in a
48-byte buffer, which may be enough for English but not true for any
translations. Convert it to use strbuf (i.e. no hard limit on
translation length).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
wt-status: move strbuf into read_and_strip_branch()

The strbufs are placed outside read_and_strip_branch as a premature
optimization: when it reads "refs/heads/foo" to strbuf and wants to
return just "foo", it could do so without memory movement. In return
the caller must not use the returned pointer after releasing strbufs,
which own the buffers that contain the returned strings. It's a clumsy
design.

By moving strbufs into read_and_strip_branch(), the returned pointer
always points to a malloc'd buffer or NULL. The pointer can be passed
around and freed after use.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
wt-status: split wt_status_state parsing function out

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
wt-status: move wt_status_get_state() out to wt_status_print()

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
status: show more info than "currently not on any branch"

When a remote ref or a tag is checked out, HEAD is automatically
detached. There is no user-friendly way to find out what ref is
checked out in this case. This patch digs in reflog for this
information and shows "HEAD detached from origin/master" or "HEAD
detached at v1.8.0" instead of "currently not on any branch".

When it cannot figure out the original ref, it shows an abbreviated
SHA-1. "Currently not on any branch" would never display (unless
reflog is pruned to near empty that the last checkout entry is lost).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
branch: show more information when HEAD is detached

This prints more helpful info when HEAD is detached: is it detached
because of bisect or rebase? What is the original branch name in those
cases? Is it detached because the user checks out a remote ref or a
tag (and which one)?

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: drop lookup-before-parse optimization

When we receive a "have" line from the client, we want to
load the object pointed to by the sha1. However, we are
careful to do:

  o = lookup_object(sha1);
  if (!o || !o->parsed)
	  o = parse_object(sha1);

to avoid loading the object from disk if we have already
seen it.  However, since ccdc603 (parse_object: try internal
cache before reading object db), parse_object already does
this optimization internally. We can just call parse_object
directly.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: make sure "want" objects are parsed

When upload-pack receives a "want" line from the client, it
adds it to an object array. We call lookup_object to find
the actual object, which will only check for objects already
in memory. This works because we are expecting to find
objects that we already loaded during the ref advertisement.

We use the resulting object structs for a variety of
purposes. Some of them care only about the object flags, but
others care about the type of the object (e.g.,
ok_to_give_up), or even feed them to the revision parser
(when --depth is used), which assumes that objects it
receives are fully parsed.

Once upon a time, this was OK; any object we loaded into
memory would also have been parsed. But since 435c833
(upload-pack: use peel_ref for ref advertisements,
2012-10-04), we try to avoid parsing objects during the ref
advertisement. This means that lookup_object may return an
object with a type of OBJ_NONE. The resulting mess depends
on the exact set of objects, but can include the revision
parser barfing, or the shallow code sending the wrong set of
objects.

This patch teaches upload-pack to parse each "want" object
as we receive it. We do not replace the lookup_object call
with parse_object, as the current code is careful not to let
just any object appear on a "want" line, but rather only one
we have previously advertised (whereas parse_object would
actually load any arbitrary object from disk).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
upload-pack: load non-tip "want" objects from disk

It is a long-time security feature that upload-pack will not
serve any "want" lines that do not correspond to the tip of
one of our refs. Traditionally, this was enforced by
checking the objects in the in-memory hash; they should have
been loaded and received the OUR_REF flag during the
advertisement.

The stateless-rpc mode, however, has a race condition here:
one process advertises, and another receives the want lines,
so the refs may have changed in the interim.  To address
this, commit 051e400 added a new verification mode; if the
object is not OUR_REF, we set a "has_non_tip" flag, and then
later verify that the requested objects are reachable from
our current tips.

However, we still die immediately when the object is not in
our in-memory hash, and at this point we should only have
loaded our tip objects. So the check_non_tip code path does
not ever actually trigger, as any non-tip objects would
have already caused us to die.

We can fix that by using parse_object instead of
lookup_object, which will load the object from disk if it
has not already been loaded.

We still need to check that parse_object does not return
NULL, though, as it is possible we do not have the object
at all. A more appropriate error message would be "no such
object" rather than "not our ref"; however, we do not want
to leak information about what objects are or are not in
the object database, so we continue to use the same "not
our ref" message that would be produced by an unreachable
object.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Preallocate hash tables when the number of inserts are known in advance

This avoids unnecessary re-allocations and reinsertions. On webkit.git
(i.e. about 182k inserts to the name hash table), this reduces about
100ms out of 3s user time.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pull: Apply -q and -v options to rebase mode as well

git pull passed -q and -v only to git merge, but they can be useful for
git rebase as well, so pass them there, too.

In particular, using -q shuts up the "Already up-to-date." message.
Especially, a new test script runs the same "pull --rebase" twice to
make sure both cases are quiet, when it has something to fetch and
when it is already up to date.

Signed-off-by: Peter Eisentraut <peter@eisentraut.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
avoid segfaults on parse_object failure

Many call-sites of parse_object assume that they will get a
non-NULL return value; this is not the case if we encounter
an error while parsing the object.

This patch adds a wrapper function around parse_object that
handles dying automatically, and uses it anywhere we
immediately try to access the return value as a non-NULL
pointer (i.e., anywhere that we would currently segfault).

This wrapper may also be useful in other places. The most
obvious one is code like:

  o = parse_object(sha1);
  if (!o)
	  die(...);

However, these should not be mechanically converted to
parse_object_or_die, as the die message is sometimes
customized. Later patches can address these sites on a
case-by-case basis.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
use parse_object_or_die instead of die("bad object")

Some call-sites do:

  o = parse_object(sha1);
  if (!o)
	  die("bad object %s", some_name);

We can now handle that as a one-liner, and get more
consistent output.

In the third case of this patch, it looks like we are losing
information, as the existing message also outputs the sha1
hex; however, parse_object will already have written a more
specific complaint about the sha1, so there is no point in
repeating it here.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pack-refs: write peeled entry for non-tags

When we pack an annotated tag ref, we write not only the
sha1 of the tag object along with the ref, but also the sha1
obtained by peeling the tag. This lets readers of the
pack-refs file know the peeled value without having to
actually load the object, speeding up upload-pack's ref
advertisement.

The writer marks a packed-refs file with peeled refs using
the "peeled" trait at the top of the file. When the reader
sees this trait, it knows that each ref is either followed
by its peeled value, or it is not an annotated tag.

However, there is a mismatch between the assumptions of the
reader and writer. The writer will only peel refs under
refs/tags, but the reader does not know this; it will assume
a ref without a peeled value must not be a tag object. Thus
an annotated tag object placed outside of the refs/tags
hierarchy will not have its peeled value printed by
upload-pack.

The simplest way to fix this is to start writing peel values
for all refs. This matches what the reader expects for both
new and old versions of git.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-export: rename handle_object function

The handle_object function is rather vaguely named; it only
operates on blobs, and its purpose is to export the blob to
the output stream. Let's call it "export_blob" to make it
more clear what it does.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>