t5802: add test for connect helper

This is an attempt to reproduce a problem reported for a third-party
custom "connect" remote helper.  The conjecture is that sometimes
"git fetch" wants to make two connections (one for the primary
transfer with 'follow-tags' option set, and then after noticing that
some tags are not packed because the primary transfer did not have
to send any commit that is pointed by them, another to explicitly
ask for the missing tags), and their "connect" helper is not called
in the second request, breaking the "fetch" as a whole.

Unfortunately this test script does not trigger the alleged failure
and happily passes when talking to upload-pack from git-core (see
patch 5/5 for details).

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: rename file-scope global "transport" to "gtransport"

Although many functions in this file take a "struct transport" as a
parameter, "fetch_one()" assigns to the global singleton instance
which is a file-scope static, in order to allow a parameterless
signal handler unlock_pack() to access it.

Rename the variable to gtransport to make sure these uses stand out.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that prepares a transport

Make a helper function prepare_transport() that returns a transport
to talk to a given remote.

The set_option() helper that used to always affect the file-scope
global "gtransport" now takes a transport as its parameter.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: refactor code that fetches leftover tags

Usually the upload-pack process running on the other side will give
us all the reachable tags we need during the primary object transfer
in do_fetch().  If that does not happen (e.g. the other side may be
running a third-party implementation of upload-pack), we will run
another fetch to pick up leftover tags that we know point at the
commits reachable from our updated tips.

Separate out the code to run this second fetch into a helper
function.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: work around "transport-take-over" hack

A Git-aware "connect" transport allows the "transport_take_over" to
redirect generic transport requests like fetch(), push_refs() and
get_refs_list() to the native Git transport handling methods.  The
take-over process replaces transport->data with a fake data that
these method implementations understand.

While this hack works OK for a single request, it breaks when the
transport needs to make more than one requests.  transport->data
that used to hold necessary information for the specific helper to
work correctly is destroyed during the take-over process.

One codepath that this matters is "git fetch" in auto-follow mode;
when it does not get all the tags that ought to point at the history
it got (which can be determined by looking at the peeled tags in the
initial advertisement) from the primary transfer, it internally
makes a second request to complete the fetch.  Because "take-over"
hack has already destroyed the data necessary to talk to the
transport helper by the time this happens, the second request cannot
make a request to the helper to make another connection to fetch
these additional tags.

Mark such a transport as "cannot_reuse", and use a separate
transport to perform the backfill fetch in order to work around
this breakage.

Note that this problem does not manifest itself when running t5802,
because our upload-pack gives you all the necessary auto-followed
tags during the primary transfer.  You would need to step through
"git fetch" in a debugger, stop immediately after the primary
transfer finishes and writes these auto-followed tags, remove the
tag references and repack/prune the repository to convince the
"find-non-local-tags" procedure that the primary transfer failed to
give us all the necessary tags, and then let it continue, in order
to trigger the bug in the secondary transfer this patch fixes.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'km/svn-1.8-serf-only' into jch

Subversion 1.8.0 that was recently released breaks older subversion
clients coming over http/https in various ways.

* km/svn-1.8-serf-only:
  Git.pm: revert _temp_cache use of temp_is_locked
  git-svn: allow git-svn fetching to work using serf
  Git.pm: add new temp_is_locked function
Merge branch 'jc/check-x-z' into jch

"git check-ignore -z" applied the NUL termination to both its input
(with --stdin) and its output, but "git check-attr -z" ignored the
option on the output side.

This is potentially a backward incompatible fix.  I am tempted to
merge this to and keep it in 'next' for a while to see if anybody
screams before deciding if we want to do anything to help existing
users (there may be none).

* jc/check-x-z:
  check-attr -z: a single -z should apply to both input and output
  check-ignore -z: a single -z should apply to both input and output
  check-attr: the name of the character is NUL, not NULL
  check-ignore: the name of the character is NUL, not NULL

# Conflicts:
# 	builtin/check-ignore.c
Merge branch 'ms/fetch-prune-configuration' into jch

Allow fetch.prune and remote.*.prune configuration variables to be set,
and "git fetch" to behave as if "--prune" is given.

"git fetch" that honors remote.*.prune is fine, but I wonder if we
should somehow make "git push" aware of it as well.  Perhaps
remote.*.prune should not be just a boolean, but a 4-way "none",
"push", "fetch", "both"?

* ms/fetch-prune-configuration:
  fetch: make --prune configurable
Merge branch 'rj/doc-rev-parse' into jch

* rj/doc-rev-parse:
  rev-parse(1): logically group options
  rev-parse: remove restrictions on some options
Merge branch 'jc/pull-training-wheel' into jch

Make "git pull" (without arguments that say what branch to
integrate from where) refuse with "it does not fast forward; choose
between 'pull --merge' and 'pull --rebase'".

See

http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=225326
http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=229221

for full discussion.

* jc/pull-training-wheel:
  pull: require choice between rebase/merge on non-fast-forward pull
Merge branch 'jc/diff-filter-negation' into jch

Teach "git diff --diff-filter" to express "I do not want to see
these classes of changes" more directly by listing only the
unwanted ones in lowercase (e.g. "--diff-filter=d" will show
everything but deletion) and deprecate "diff-files -q" which did
the same thing as "--diff-filter=d".

* jc/diff-filter-negation:
  diff: deprecate -q option to diff-files
  diff: allow lowercase letter to specify what change class to exclude
  diff: reject unknown change class given to --diff-filter
  diff: preparse --diff-filter string argument
  diff: factor out match_filter()
  diff: pass the whole diff_options to diffcore_apply_filter()

# Conflicts:
# 	diff.c
Merge branch 'jk/fast-import-empty-ls' into jch

* jk/fast-import-empty-ls:
  fast-import: allow moving the root tree
  fast-import: allow ls or filecopy of the root tree
  fast-import: set valid mode on root tree in "ls"
  t9300: document fast-import empty path issues
Merge branch 'nd/magic-pathspec' into jch

Use "struct pathspec" interface in more places, instead of array of
characters, the latter of which cannot express magic pathspecs
(e.g. ":(icase)makefile" that matches both Makefile and makefile).

* nd/magic-pathspec: (48 commits)
  rm: do not set a variable twice without intermediate reading.
  t6131 - skip tests if on case-insensitive file system
  parse_pathspec: accept :(icase)path syntax
  pathspec: support :(glob) syntax
  pathspec: make --literal-pathspecs disable pathspec magic
  pathspec: support :(literal) syntax for noglob pathspec
  kill limit_pathspec_to_literal() as it's only used by parse_pathspec()
  parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN
  parse_pathspec: make sure the prefix part is wildcard-free
  rename field "raw" to "_raw" in struct pathspec
  tree-diff: remove the use of pathspec's raw[] in follow-rename codepath
  remove match_pathspec() in favor of match_pathspec_depth()
  remove init_pathspec() in favor of parse_pathspec()
  remove diff_tree_{setup,release}_paths
  convert common_prefix() to use struct pathspec
  convert add_files_to_cache to take struct pathspec
  convert {read,fill}_directory to take struct pathspec
  convert refresh_index to take struct pathspec
  convert report_path_error to take struct pathspec
  checkout: convert read_tree_some to take struct pathspec
  ...
Merge branch 'jc/push-cas' into jch

Allow a safer "rewind of the remote tip" push than blind "--force",
by requiring that the overwritten remote ref to be unchanged since
the new history to replace it was prepared.

The machinery is more or less ready.  The "--force" option is again
the big red button to override any safety, thanks to J6t's sanity
(the original round allowed --lockref to defeat --force).

The logic to choose the default implemented here is fragile
(e.g. "git fetch" after seeing a failure will update the
remote-tracking branch and will make the next "push" pass,
defeating the safety pretty easily).  It is suitable only for the
simplest workflows, and it may hurt users more than it helps them.

* jc/push-cas:
  push: teach --force-with-lease to smart-http transport
  send-pack: fix parsing of --force-with-lease option
  t5540/5541: smart-http does not support "--force-with-lease"
  t5533: test "push --force-with-lease"
  push --force-with-lease: tie it all together
  push --force-with-lease: implement logic to populate old_sha1_expect[]
  remote.c: add command line option parser for "--force-with-lease"
  builtin/push.c: use OPT_BOOL, not OPT_BOOLEAN
  cache.h: move remote/connect API out of it

# Conflicts:
# 	remote-curl.c
# 	remote.c
Merge branch 'nd/clone-connectivity-shortcut' into jch

* nd/clone-connectivity-shortcut:
  smart http: use the same connectivity check on cloning

# Conflicts:
# 	remote-curl.c
Merge branch 'sb/fsck-opt-bool' into jch

* sb/fsck-opt-bool:
  fsck: Replace deprecated OPT_BOOLEAN by OPT_BOOL
Merge branch 'rr/feed-real-path-to-editor' into jch

* rr/feed-real-path-to-editor:
  editor: use canonicalized absolute path
Merge branch 'db/http-savecookies' into jch

* db/http-savecookies:
  t5551: Remove header from curl cookie file
  http: add http.savecookies option to write out HTTP cookies
Merge branch 'nd/sq-quote-buf' into jch

Code simplification as a preparatory step to something larger.

* nd/sq-quote-buf:
  quote: remove sq_quote_print()
  tar-tree: remove dependency on sq_quote_print()
  for-each-ref, quote: convert *_quote_print -> *_quote_buf
Merge branch 'da/darwin' into jch

* da/darwin:
  OS X: Fix redeclaration of die warning
  Makefile: Fix APPLE_COMMON_CRYPTO with BLK_SHA1
  imap-send: use Apple's Security framework for base64 encoding