hooks/post-receive-email: use plumbing instead of git log/show

This way the hook doesn't have to keep being tweaked as porcelain
learns new features like color and pagination.

While at it, replace the "git rev-list | git shortlog" idiom with
plain "git shortlog" for simplicity.

Except for depending less on the value of settings like '[log]
abbrevCommit', no change in output intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hooks/post-receive-email: force log messages in UTF-8

Git commands write commit messages in UTF-8 by default, but that
default can be overridden by the [i18n] commitEncoding and
logOutputEncoding settings.  With such a setting, the emails written
by the post-receive-email hook use a mixture of encodings:

 1. Log messages use the configured log output encoding, which is
    meant to be whatever encoding works best with local terminals
    (and does not have much to do with what encoding should be used
    for email)

 2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw
    port (which uses Unicode filesystem APIs), always UTF-8

 3. The "This is an automated email" preface uses a project description
    from .git/description, which is typically in UTF-8 to support
    gitweb.

So (1) is configurable, and (2) and (3) are unconfigurable and
typically UTF-8.  Override the log output encoding to always use UTF-8
when writing the email to get the best chance of a comprehensible
single-encoding email.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hooks/post-receive-email: set declared encoding to utf-8

Some email clients (e.g., claws-mail) display the message body
incorrectly when the charset is not defined explicitly in a
Content-Type header.  "git log" generates logs in UTF-8 encoding by
default, so add a Content-Type header declaring that encoding to
the emails the post-receive-email example hook sends.

[jn: also setting the Content-Transfer-Encoding so MTAs know what
 kind of mangling might be needed when sending to a non 8-bit clean
 SMTP host]

Requested-by: Alexander Gerasiov <gq@debian.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5802: add test for connect helper

This is an attempt to reproduce a problem reported for a third-party
custom "connect" remote helper.  The conjecture is that sometimes
"git fetch" wants to make two connections (one for the primary
transfer with 'follow-tags' option set, and then after noticing that
some tags are not packed because the primary transfer did not have
to send any commit that is pointed by them, another to explicitly
ask for the missing tags), and their "connect" helper is not called
in the second request, breaking the "fetch" as a whole.

Unfortunately this test script does not trigger the alleged failure
and happily passes when talking to upload-pack from git-core (see
patch 5/5 for details).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: rename file-scope global "transport" to "gtransport"

Although many functions in this file take a "struct transport" as a
parameter, "fetch_one()" assigns to the global singleton instance
which is a file-scope static, in order to allow a parameterless
signal handler unlock_pack() to access it.

Rename the variable to gtransport to make sure these uses stand out.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that prepares a transport

Make a helper function prepare_transport() that returns a transport
to talk to a given remote.

The set_option() helper that used to always affect the file-scope
global "gtransport" now takes a transport as its parameter.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that fetches leftover tags

Usually the upload-pack process running on the other side will give
us all the reachable tags we need during the primary object transfer
in do_fetch().  If that does not happen (e.g. the other side may be
running a third-party implementation of upload-pack), we will run
another fetch to pick up leftover tags that we know point at the
commits reachable from our updated tips.

Separate out the code to run this second fetch into a helper
function.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: work around "transport-take-over" hack

A Git-aware "connect" transport allows the "transport_take_over" to
redirect generic transport requests like fetch(), push_refs() and
get_refs_list() to the native Git transport handling methods.  The
take-over process replaces transport->data with a fake data that
these method implementations understand.

While this hack works OK for a single request, it breaks when the
transport needs to make more than one requests.  transport->data
that used to hold necessary information for the specific helper to
work correctly is destroyed during the take-over process.

One codepath that this matters is "git fetch" in auto-follow mode;
when it does not get all the tags that ought to point at the history
it got (which can be determined by looking at the peeled tags in the
initial advertisement) from the primary transfer, it internally
makes a second request to complete the fetch.  Because "take-over"
hack has already destroyed the data necessary to talk to the
transport helper by the time this happens, the second request cannot
make a request to the helper to make another connection to fetch
these additional tags.

Mark such a transport as "cannot_reuse", and use a separate
transport to perform the backfill fetch in order to work around
this breakage.

Note that this problem does not manifest itself when running t5802,
because our upload-pack gives you all the necessary auto-followed
tags during the primary transfer.  You would need to step through
"git fetch" in a debugger, stop immediately after the primary
transfer finishes and writes these auto-followed tags, remove the
tag references and repack/prune the repository to convince the
"find-non-local-tags" procedure that the primary transfer failed to
give us all the necessary tags, and then let it continue, in order
to trigger the bug in the secondary transfer this patch fixes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: use the cache_* macro to access the current index

These codepaths always start from the_index and use index_*
functions, but there is no reason to do so.  Use the compatibility
cache_* macro to access the current in-core index like everybody
else.

While at it, fix typo in the comment for a function to check if a
path within a directory appears in the index.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
ls-files -k: a directory only can be killed if the index has a non-directory

"ls-files -o" and "ls-files -k" both traverse the working tree down
to find either all untracked paths or those that will be "killed"
(removed from the working tree to make room) when the paths recorded
in the index are checked out.  It is necessary to traverse the
working tree fully when enumerating all the "other" paths, but when
we are only interested in "killed" paths, we can take advantage of
the fact that paths that do not overlap with entries in the index
can never be killed.

The treat_one_path() helper function, which is called during the
recursive traversal, is the ideal place to implement an
optimization.

When we are looking at a directory P in the working tree, there are
three cases:

 (1) P exists in the index.  Everything inside the directory P in
     the working tree needs to go when P is checked out from the
     index.

 (2) P does not exist in the index, but there is P/Q in the index.
     We know P will stay a directory when we check out the contents
     of the index, but we do not know yet if there is a directory
     P/Q in the working tree to be killed, so we need to recurse.

 (3) P does not exist in the index, and there is no P/Q in the index
     to require P to be a directory, either.  Only in this case, we
     know that everything inside P will not be killed without
     recursing.

Note that this helper is called by treat_leading_path() that decides
if we need to traverse only subdirectories of a single common
leading directory, which is essential for this optimization to be
correct.  This caller checks each level of the leading path
component from shallower directory to deeper ones, and that is what
allows us to only check if the path appears in the index.  If the
call to treat_one_path() weren't there, given a path P/Q/R, the real
traversal may start from directory P/Q/R, even when the index
records P as a regular file, and we would end up having to check if
any leading subpath in P/Q/R, e.g. P, appears in the index.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3010: update to demonstrate "ls-files -k" optimization pitfalls

An earlier draft of the previous step used cache_name_exists() to
check the directory we were looking at, which missed the second case
described in its log message.  Demonstrate why it is not sufficient.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
create_delta_index: simplify condition always evaluating to true

The code sequence  ' (1u << i) < hsize && i < 31 ' is a multi step
process, whose first step requires that 'i' is already less that 31,
otherwise the result (1u << i)  is undefined (and  'undef_val < hsize'
can therefore be assumed to be 'false'), and so the later test  i < 31
can always be optimized away as dead code ('i' is already less than 31,
or the short circuit 'and' applies).

So we need to get rid of that code. One way would be to exchange the
order of the conditions, so the expression 'i < 31 && (1u << i) < hsize'
would remove that optimized unstable code already.

However when checking the previous lines in that function, we can deduce
that 'hsize' must always be smaller than (1u<<31), since 506049c7df2c6
(fix >4GiB source delta assertion failure), because 'entries' is
capped at an upper bound of 0xfffffffeU, so 'hsize' contains a maximum
value of 0x3fffffff, which is smaller than (1u<<31), so the value of
'i' will never be larger than 31 and we can remove that condition
entirely.

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
Acked-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rebase -i: fix cases ignoring core.commentchar

180bad3d (rebase -i: respect core.commentchar, 2013-02-11) updated
"rebase -i" to honor core.commentchar but missed one instance of
hard-coded '#' comment character in skip_unnecessary_picks().

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rebase -i: fix core.commentchar regression

9a46c25bdbf79744 (rebase: interactive: fix short SHA-1 collision,
2013-08-12) added transform_todo_ids() which assumes incorrectly that
the comment character is always '#', however, this has not been the case
since 180bad3d10fe3a7f (rebase -i: respect core.commentchar, 2013-02-11)
patched rebase -i to respect core.commentchar. Fix this regression.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
bash prompt: test the prompt with newline in repository path

Newlines in the path to a git repository were not an issue for the
git-specific bash prompt before commit efaa0c1532 (bash prompt:
combine 'git rev-parse' executions in the main code path, 2013-06-17),
because the path returned by 'git rev-parse --git-dir' was directly
stored in a variable, and this variable was later always accessed
inside double quotes.

Newlines are not an issue after commit efaa0c1532 either, but it's
more subtle.  Since efaa0c1532 we use the following single 'git
rev-parse' execution to query various info about the repository:

  git rev-parse --git-dir --is-inside-git-dir \
          --is-bare-repository --is-inside-work-tree

The results to these queries are separated by a newline character in
the output, e.g.:

  /home/szeder/src/git/.git
  false
  false
  true

A newline in the path to the git repository could potentially break
the parsing of these results and ultimately the bash prompt, unless
the parsing is done right.  Commit efaa0c1532 got it right, as I
consciously started parsing 'git rev-parse's output from the end,
where each record is a single line containing either 'true' or 'false'
or, after e3e0b9378b (bash prompt: combine 'git rev-parse' for
detached head, 2013-06-24), the abbreviated commit object name, and
all what remains at the beginning is the path to the git repository,
no matter how many lines it is.

This subtlety really warrants its own test, especially since I didn't
explain it in the log message or in an in-code comment back then, so
add a test to excercise the prompt with newline characters in the path
to the repository.  Guard this test with the FUNNYNAMES prerequisite,
because not all filesystems support newlines in filenames.  Note that
'git rev-parse --git-dir' prints '.git' or '.' when at the top of the
worktree or the repository, respectively, and only prints the full
path to the repository when in a subdirectory, hence the need for
changing into a subdir in the test.

Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
avoid segfault on submodule.*.path set to an empty "true"

Git fails due to a segmentation fault if a submodule path is empty.
Here is an example .gitmodules that will cause a segmentation fault:

    [submodule "foo-module"]
      path
      url = http://host/repo.git
    $ git status
    Segmentation fault (core dumped)

This is because the parsing of "submodule.*.path" is not prepared to
see a value-less "true" and assumes that the value is always
non-NULL (parsing of "ignore" has the same problem).

Fix it by checking the NULL-ness of value and complain with
config_error_nonbool().

Signed-off-by: Jharrod LaFon <jlafon@eyesopen.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
mailmap: remove redundant check for freeing memory

The condition as it is written in that line has already been checked
in the beginning of the function, which was introduced in
8503ee4 (2007-05-01, Fix read_mailmap to handle a caller uninterested
in repo abbreviation)

Helped-by: Jeff King <peff@peff.net>
Helped-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
xread, xwrite: limit size of IO to 8MB

Checking out 2GB or more through an external filter (see test) fails
on Mac OS X 10.8.4 (12E55) for a 64-bit executable with:

    error: read from external filter cat failed
    error: cannot feed the input to external filter cat
    error: cat died of signal 13
    error: external filter cat failed 141
    error: external filter cat failed

The reason is that read() immediately returns with EINVAL when asked
to read more than 2GB.  According to POSIX [1], if the value of
nbyte passed to read() is greater than SSIZE_MAX, the result is
implementation-defined.  The write function has the same restriction
[2].  Since OS X still supports running 32-bit executables, the
32-bit limit (SSIZE_MAX = INT_MAX = 2GB - 1) seems to be also
imposed on 64-bit executables under certain conditions.  For write,
the problem has been addressed earlier [6c642a].

Address the problem for read() and write() differently, by limiting
size of IO chunks unconditionally on all platforms in xread() and
xwrite().  Large chunks only cause problems, like causing latencies
when killing the process, even if OS X was not buggy.  Doing IO in
reasonably sized smaller chunks should have no negative impact on
performance.

The compat wrapper clipped_write() introduced earlier [6c642a] is
not needed anymore.  It will be reverted in a separate commit.  The
new test catches read and write problems.

Note that 'git add' exits with 0 even if it prints filtering errors
to stderr.  The test, therefore, checks stderr.  'git add' should
probably be changed (sometime in another commit) to exit with
nonzero if filtering fails.  The test could then be changed to use
test_must_fail.

Thanks to the following people for suggestions and testing:

    Johannes Sixt <j6t@kdbg.org>
    John Keeping <john@keeping.me.uk>
    Jonathan Nieder <jrnieder@gmail.com>
    Kyle J. McKay <mackyle@gmail.com>
    Linus Torvalds <torvalds@linux-foundation.org>
    Torsten Bögershausen <tboegi@web.de>

[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/read.html
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/write.html

[6c642a] commit 6c642a878688adf46b226903858b53e2d31ac5c3
    compate/clipped-write.c: large write(2) fails on Mac OS X/XNU

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Revert "compat/clipped-write.c: large write(2) fails on Mac OS X/XNU"

This reverts commit 6c642a878688adf46b226903858b53e2d31ac5c3.

The previous commit introduced a size limit on IO chunks on all
platforms.  The compat clipped_write() is not needed anymore.

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
submodule: fix confusing variable name

cmd_summary reads the output of git diff, but reads in the submodule path into a
variable called name.  Since this variable does not contain the name of the
submodule, but the path, rename it to be clearer what data it actually holds.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>