hooks/post-receive-email: use plumbing instead of git log/show

This way the hook doesn't have to keep being tweaked as porcelain
learns new features like color and pagination.

While at it, replace the "git rev-list | git shortlog" idiom with
plain "git shortlog" for simplicity.

Except for depending less on the value of settings like '[log]
abbrevCommit', no change in output intended.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hooks/post-receive-email: force log messages in UTF-8

Git commands write commit messages in UTF-8 by default, but that
default can be overridden by the [i18n] commitEncoding and
logOutputEncoding settings.  With such a setting, the emails written
by the post-receive-email hook use a mixture of encodings:

 1. Log messages use the configured log output encoding, which is
    meant to be whatever encoding works best with local terminals
    (and does not have much to do with what encoding should be used
    for email)

 2. Filenames are left as is: on Linux, usually UTF-8, and in the Mingw
    port (which uses Unicode filesystem APIs), always UTF-8

 3. The "This is an automated email" preface uses a project description
    from .git/description, which is typically in UTF-8 to support
    gitweb.

So (1) is configurable, and (2) and (3) are unconfigurable and
typically UTF-8.  Override the log output encoding to always use UTF-8
when writing the email to get the best chance of a comprehensible
single-encoding email.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hooks/post-receive-email: set declared encoding to utf-8

Some email clients (e.g., claws-mail) display the message body
incorrectly when the charset is not defined explicitly in a
Content-Type header.  "git log" generates logs in UTF-8 encoding by
default, so add a Content-Type header declaring that encoding to
the emails the post-receive-email example hook sends.

[jn: also setting the Content-Transfer-Encoding so MTAs know what
 kind of mangling might be needed when sending to a non 8-bit clean
 SMTP host]

Requested-by: Alexander Gerasiov <gq@debian.org>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5802: add test for connect helper

This is an attempt to reproduce a problem reported for a third-party
custom "connect" remote helper.  The conjecture is that sometimes
"git fetch" wants to make two connections (one for the primary
transfer with 'follow-tags' option set, and then after noticing that
some tags are not packed because the primary transfer did not have
to send any commit that is pointed by them, another to explicitly
ask for the missing tags), and their "connect" helper is not called
in the second request, breaking the "fetch" as a whole.

Unfortunately this test script does not trigger the alleged failure
and happily passes when talking to upload-pack from git-core (see
patch 5/5 for details).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: rename file-scope global "transport" to "gtransport"

Although many functions in this file take a "struct transport" as a
parameter, "fetch_one()" assigns to the global singleton instance
which is a file-scope static, in order to allow a parameterless
signal handler unlock_pack() to access it.

Rename the variable to gtransport to make sure these uses stand out.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that prepares a transport

Make a helper function prepare_transport() that returns a transport
to talk to a given remote.

The set_option() helper that used to always affect the file-scope
global "gtransport" now takes a transport as its parameter.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that fetches leftover tags

Usually the upload-pack process running on the other side will give
us all the reachable tags we need during the primary object transfer
in do_fetch().  If that does not happen (e.g. the other side may be
running a third-party implementation of upload-pack), we will run
another fetch to pick up leftover tags that we know point at the
commits reachable from our updated tips.

Separate out the code to run this second fetch into a helper
function.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: work around "transport-take-over" hack

A Git-aware "connect" transport allows the "transport_take_over" to
redirect generic transport requests like fetch(), push_refs() and
get_refs_list() to the native Git transport handling methods.  The
take-over process replaces transport->data with a fake data that
these method implementations understand.

While this hack works OK for a single request, it breaks when the
transport needs to make more than one requests.  transport->data
that used to hold necessary information for the specific helper to
work correctly is destroyed during the take-over process.

One codepath that this matters is "git fetch" in auto-follow mode;
when it does not get all the tags that ought to point at the history
it got (which can be determined by looking at the peeled tags in the
initial advertisement) from the primary transfer, it internally
makes a second request to complete the fetch.  Because "take-over"
hack has already destroyed the data necessary to talk to the
transport helper by the time this happens, the second request cannot
make a request to the helper to make another connection to fetch
these additional tags.

Mark such a transport as "cannot_reuse", and use a separate
transport to perform the backfill fetch in order to work around
this breakage.

Note that this problem does not manifest itself when running t5802,
because our upload-pack gives you all the necessary auto-followed
tags during the primary transfer.  You would need to step through
"git fetch" in a debugger, stop immediately after the primary
transfer finishes and writes these auto-followed tags, remove the
tag references and repack/prune the repository to convince the
"find-non-local-tags" procedure that the primary transfer failed to
give us all the necessary tags, and then let it continue, in order
to trigger the bug in the secondary transfer this patch fixes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: sanitize P4CHARSET

In the tests, p4d is started without using "internationalized
mode".  Make sure this environment variable is unset, otherwise
a mis-matched user setting would break the tests.  The error
message would be "Unicode clients require a unicode enabled server."

[pw: use unset, add commit text]

Signed-off-by: Kazuki Saitoh <ksaitoh560@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: implement view spec wildcards with "p4 where"

Currently, git p4 does not support many of the view
wildcards, such as * and %%n.  It only knows the
common ... mapping, and exclusions.

Redo the entire wildcard code around the idea of
directly querying the p4 server for the mapping.  For each
unique file, invoke "p4 where //depot/path" and use
the client mapping to decide where the file goes in git.

This simplifies a lot of code, and adds support for all
wildcards supported by p4.  The downside is that each file
triggers a query to the p4 server, possibly making high file
count changes very slow.  If it turns out to be a problem,
we might consider resurrecting the old wildcard code just
for use on the "easy" cases it understands.

[pw: redo code and tests]

Signed-off-by: Kazuki Saitoh <ksaitoh560@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
WAITING FOR ACK
Merge branch 'km/svn-1.8-serf-only' into jch

Subversion 1.8.0 that was recently released breaks older subversion
clients coming over http/https in various ways.

* km/svn-1.8-serf-only:
  Git.pm: revert _temp_cache use of temp_is_locked
  git-svn: allow git-svn fetching to work using serf
  Git.pm: add new temp_is_locked function
Merge branch 'jc/check-x-z' into jch

"git check-ignore -z" applied the NUL termination to both its input
(with --stdin) and its output, but "git check-attr -z" ignored the
option on the output side.

This is potentially a backward incompatible fix.  I am tempted to
merge this to and keep it in 'next' for a while to see if anybody
screams before deciding if we want to do anything to help existing
users (there may be none).

* jc/check-x-z:
  check-attr -z: a single -z should apply to both input and output
  check-ignore -z: a single -z should apply to both input and output
  check-attr: the name of the character is NUL, not NULL
  check-ignore: the name of the character is NUL, not NULL

# Conflicts:
# 	builtin/check-ignore.c
Merge branch 'ms/fetch-prune-configuration' into jch

Allow fetch.prune and remote.*.prune configuration variables to be set,
and "git fetch" to behave as if "--prune" is given.

"git fetch" that honors remote.*.prune is fine, but I wonder if we
should somehow make "git push" aware of it as well.  Perhaps
remote.*.prune should not be just a boolean, but a 4-way "none",
"push", "fetch", "both"?

* ms/fetch-prune-configuration:
  fetch: make --prune configurable
Merge branch 'rj/doc-rev-parse' into jch

* rj/doc-rev-parse:
  rev-parse(1): logically group options
  rev-parse: remove restrictions on some options
Merge branch 'jc/pull-training-wheel' into jch

Make "git pull" (without arguments that say what branch to
integrate from where) refuse with "it does not fast forward; choose
between 'pull --merge' and 'pull --rebase'".

See

http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=225326
http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=229221

for full discussion.

* jc/pull-training-wheel:
  pull: require choice between rebase/merge on non-fast-forward pull
Merge branch 'jc/diff-filter-negation' into jch

Teach "git diff --diff-filter" to express "I do not want to see
these classes of changes" more directly by listing only the
unwanted ones in lowercase (e.g. "--diff-filter=d" will show
everything but deletion) and deprecate "diff-files -q" which did
the same thing as "--diff-filter=d".

* jc/diff-filter-negation:
  diff: deprecate -q option to diff-files
  diff: allow lowercase letter to specify what change class to exclude
  diff: reject unknown change class given to --diff-filter
  diff: preparse --diff-filter string argument
  diff: factor out match_filter()
  diff: pass the whole diff_options to diffcore_apply_filter()

# Conflicts:
# 	diff.c
Merge branch 'jk/fast-import-empty-ls' into jch

* jk/fast-import-empty-ls:
  fast-import: allow moving the root tree
  fast-import: allow ls or filecopy of the root tree
  fast-import: set valid mode on root tree in "ls"
  t9300: document fast-import empty path issues
Merge branch 'nd/magic-pathspec' into jch

Use "struct pathspec" interface in more places, instead of array of
characters, the latter of which cannot express magic pathspecs
(e.g. ":(icase)makefile" that matches both Makefile and makefile).

* nd/magic-pathspec: (48 commits)
  rm: do not set a variable twice without intermediate reading.
  t6131 - skip tests if on case-insensitive file system
  parse_pathspec: accept :(icase)path syntax
  pathspec: support :(glob) syntax
  pathspec: make --literal-pathspecs disable pathspec magic
  pathspec: support :(literal) syntax for noglob pathspec
  kill limit_pathspec_to_literal() as it's only used by parse_pathspec()
  parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN
  parse_pathspec: make sure the prefix part is wildcard-free
  rename field "raw" to "_raw" in struct pathspec
  tree-diff: remove the use of pathspec's raw[] in follow-rename codepath
  remove match_pathspec() in favor of match_pathspec_depth()
  remove init_pathspec() in favor of parse_pathspec()
  remove diff_tree_{setup,release}_paths
  convert common_prefix() to use struct pathspec
  convert add_files_to_cache to take struct pathspec
  convert {read,fill}_directory to take struct pathspec
  convert refresh_index to take struct pathspec
  convert report_path_error to take struct pathspec
  checkout: convert read_tree_some to take struct pathspec
  ...
Merge branch 'jc/push-cas' into jch

Allow a safer "rewind of the remote tip" push than blind "--force",
by requiring that the overwritten remote ref to be unchanged since
the new history to replace it was prepared.

The machinery is more or less ready.  The "--force" option is again
the big red button to override any safety, thanks to J6t's sanity
(the original round allowed --lockref to defeat --force).

The logic to choose the default implemented here is fragile
(e.g. "git fetch" after seeing a failure will update the
remote-tracking branch and will make the next "push" pass,
defeating the safety pretty easily).  It is suitable only for the
simplest workflows, and it may hurt users more than it helps them.

* jc/push-cas:
  push: teach --force-with-lease to smart-http transport
  send-pack: fix parsing of --force-with-lease option
  t5540/5541: smart-http does not support "--force-with-lease"
  t5533: test "push --force-with-lease"
  push --force-with-lease: tie it all together
  push --force-with-lease: implement logic to populate old_sha1_expect[]
  remote.c: add command line option parser for "--force-with-lease"
  builtin/push.c: use OPT_BOOL, not OPT_BOOLEAN
  cache.h: move remote/connect API out of it

# Conflicts:
# 	remote-curl.c
# 	remote.c