packfile: use varint functions

The OFS_DELTA codepaths can use encode/decode_varint() to read and write
variable-length integers in the pack stream.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
new representation types in the packstream

In addition to the four basic types (commit, tree, blob and tag), the pack
stream can encode a few other "representation" types, such as REF_DELTA
and OFS_DELTA.  As we allocate 3 bits in the first byte for this purpose,
we do not have much room to add new representation types in place, but we
do have one value reserved for future expansion.

When encoding a new representation type, the early part of the in-pack
object header is encoded as if its type is OBJ_EXT (= 5).  The lower 4-bit
of the first byte is used for the lowest 4-bit of the size information,
the next 3-bit has the type information, and the MSB says if the
subsequent bytes need to be read to decode higher bits for the size
information.

An in-pack object header that records OBJ_EXT as its type is followed by
an integer in the same variable-length encoding as OFS_DELTA offset is
encoded in.  This value is the real type of the representation minus 8 (as
we do not need to use OBJ_EXT to encode types smaller than 8).  Because we
do not foresee very many representation types, in practice we would have a
single byte with its MSB clear, to represent types 8-135.

The code does not add nor use type=8 and upwards for anything yet.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
bulk-checkin: allow the same data to be multiply hashed

This updates stream_to_pack() machinery to feed the data it is writing out
to multiple hash contexts at the same time. Right now we only use a single
git_SHA_CTX, so there is no change in functionality.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
bulk-checkin: support chunked-object encoding

Instead of recording a huge object as a single entry in the packfile,
record it as a concatenation of smaller blob objects. This is primarily to
make it possible to repack repositories with huge objects and transfer
them out of such repositories.

This is the first step of a long and painful journey. We would still need
to teach many codepaths about the new encoding:

 * The streaming checkout codepath must learn how to read these from the
   object store and write a concatenation of component blob contents to
   the working tree. Note that after a repack, a component blob object
   could be represented as a delta to another blob object.

 * The in-core object reading codepath must learn to notice and at least
   reject reading such objects entirely in-core. It is expected that
   nobody is interested in producing a patch out of these huge objects, at
   least initially.

 * The object connectivity machinery must learn that component blob
   objects are reachable from an object that uses them, so that "gc"
   will not lose them, and "fsck" will not complain about them.

 * The pack-object machinery must learn to copy an object that is encoded
   in chunked-object encoding as-is to the destination, while perhaps
   validating the object name, without having to first come up with a huge
   concatenated blob and then split it again into smaller pieces. It also
   must learn that the component blobs need to be packed when it does so.

 * The index-pack and verify-pack machineries need to be told about the
   objects in the chunked encoding.

The split-chunk logic used here is kept deliberately useless in order to
avoid distracting the reviewers.  We will want to replace it with a better
heuristics---the rest of the system is designed not to care, so that the
heuristics can be improved without any compatibility issues.

One possibility is to use the heuristics "bup" uses, which is based on a
self-synchronizing rolling checksum logic; if it works as advertised, we
should be able to obtain a good reuse of component blobs when only small
parts of a large blob in chunked encoding are updated.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
chunked-object: fallback checkout codepaths

This prepares the default codepaths based on the traditional "slurping
everything in-core" model around read_sha1_file() API for objects that use
chunked encoding. Needless to say, these codepaths are unsuitable for the
kind of objects that use chunked encoding and are intended to only serve
as the fallback where specialized "large object API" support is still
lacking.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
chunked-object: streaming checkout

Define a new streaming decoder to handle blobs stored in chunked encoding,
so that we do not have to slurp the data for the whole object to check it
out to the working tree.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
push: start warning upcoming default change for push.default

In preparation for flipping the default to the "simple" mode from
the "matching" mode that is the historical default, start warning
users when they rely on unconfigured "git push" to default to the
"matching" mode.

Also, advertise for 'simple' where 'current' and 'upstream' are advised.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5541: warning message is given even with --quiet

Explicitly set push.default to matching to avoid the warning message;
the test wants to verify there is no post-push report when --quiet is
given, and this message interferes with it.

This change is *iffy*, not in the sense that it breaks the test it
touches, but because it is unclear if the output should be given when
the user explicitly asked --quiet.

On one hand, --quiet is a request to be "quiet", but on the other hand,
this warning is meant to be shown in the face in large flashing red
letters no matter what the user says, so...

Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch/push: allow refs/*:refs/*

There are a handful of places where we call check_refname_format() on a
substring after "refs/" of a refname we are going to use, and filter out a
valid match with "refs/stash" with such a pathspec.  Not sending a stash
may arguably be a feature (as stash is inherently a local workflow
element), but the code in the transport layer is oblivious to this
filtering performed by the lower layer of the code, and complains that the
other side did not send all the objects that needs to complete refs/stash
at the end, even though the code will not write refs/stash out anyway, and
making the whole command fail.

This is an attempt to "fix" it by using check_refname_format() on the
whole "refs/....." string and allowing refs/stash to be also copied.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_fetch_map(): tighten checks on dest refs

The code to check the refname we store the fetched result locally did not
bother checking the first 5 bytes of it, presumably assuming that it
always begin with "refs/".  For a fetch refspec (or the result of applying
wildcard on one), we always want the RHS to map to something inside
"refs/" hierarchy, so let's spell that rule out in a more explicit way.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: move code to run tests under bash into a helper library

The following patch will add tests for the bash prompt functions as a
new test script, which also has to be run under bash.

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: add tests for the bash prompt functions in the completion script

The tests cover the discovery of the '.git' directory in the
__gitdir() function in different scenarios, and the prompt itself,
i.e. branch name, detached heads, operations (rebase, merge,
cherry-pick, bisect), and status indicators (dirty, stash, untracked
files; but not the upstream status).

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
completion: use __gitdir() in _git_log()

The standard way to find out the path to the repository in the
completion script is the __gitdir() helper function, because that
handles the repository path given on the command line (i.e. git
--git-dir=/path/to/repo log --<TAB>).  However, there is one
exception: the completion function for 'git log' still uses 'git
rev-parse --git-dir' directly, and could offer (or not) the '--merge'
option erroneously when the repository is specified on the command
line.

Use __gitdir() there, too.

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
completion: respect $GIT_DIR

The __gitdir() helper function finds out the path of the git
repository by running 'git rev-parse --git-dir'.  However, it has a
shortcut first to avoid the overhead of running a git command in a
subshell when the current directory is at the top of the work tree,
i.e. when it contains a '.git' subdirectory.

If the 'GIT_DIR' environment variable is set then it specifies the
path to the git repository, and the autodetection of the '.git'
directory is not necessary.  However, $GIT_DIR is only taken into
acocunt by 'git rev-parse --git-dir', and the check for the '.git'
subdirectory is performed first, so it wins over the path given in
$GIT_DIR.

There are several completion (helper) functions that depend on
__gitdir(), and when the above case triggers the completion script
will do weird things, like offering refs, aliases, or stashes from a
different repository, or displaying wrong or broken prompt, etc.

So check first whether $GIT_DIR is set, and only proceed with checking
the '.git' directory in the current directory if it isn't.  'git
rev-parse' would also check whether the path in $GIT_DIR is a proper
'.git' directory, i.e. 'HEAD', 'refs/', and 'objects/' are present and
accessible, but we don't have to be that thorough for the bash prompt.
And we've lived with an equally permissive check for '.git' in the
current working directory for years anyway.

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: clear_image() clears things a bit more

The clear_image() function did not clear the line table in the image
structure; this does not matter for the current callers, as the function
is only called from the codepaths that deal with binary patches where the
line table is never populated, and the codepaths that do populate the line
table free it themselves.

But it will start to matter when we introduce a codepath to retry a failed
patch, so make sure it clears and frees everything.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: refactor read_file_or_gitlink()

Reading a blob out of the object store does not have to require that the
caller has a cache entry for it.

Create a read_blob_object() helper function that takes the object name and
mode, and use it to reimplement the original function as a thin wrapper to
it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: split load_preimage() helper function out

The function apply_data() gets a patch for a single path, reads the
preimage in core, and applies the change represented in the patch.

Separate out the first part that reads the preimage into a separate
helper function load_preimage().

Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: accept -3/--3way command line option

This is the beginning of teaching the three-way merge fallback logic "git
am -3" uses to the underlying "git apply".  It only implements the command
line parsing part, and does not do anything interesting yet, other than
making sure that "--reject" and "--3way" are not given together.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
apply: fall back on three-way merge

Grab the preimage blob the patch claims to be based on out of the object
store, apply the patch, and then call three-way-merge function.  This step
still does not plug the actual three-way merge logic yet, but we are
getting there.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
http: Avoid limit of retrying request only twice

Current code, after receiving HTTP_REAUTH, only retried
once, so couldn't do step 3 of the following sequence:

  1. We make a request; proxy returns 407, because we didn't give it a
     password. We ask for the password and return HTTP_REAUTH.

  2. We make another request; the proxy passes it to the actual server,
     who returns 401, because we didn't give an http password. We ask
     for the password and return HTTP_REAUTH.

  3. We make a third request, but this time everybody is happy.

Now we retry as long as we keep receiving HTTP_REAUTH, so the previous
sequence correctly completes.

Patch by Jeff King <peff@peff.net>

Signed-off-by: Nelson Benitez Leon <nbenitezl@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>