mailmap: handle mailmap blobs without trailing newlines

The read_mailmap_buf function reads each line of the mailmap
using strchrnul, like:

    const char *end = strchrnul(buf, '\n');
    unsigned long linelen = end - buf + 1;

But that's off-by-one when we actually hit the NUL byte; our
line does not have a terminator, and so is only "end - buf"
bytes long. As a result, when we subtract the linelen from
the total len, we end up with (unsigned long)-1 bytes left
in the buffer, and we start reading random junk from memory.

We could fix it with:

    unsigned long linelen = end - buf + !!*end;

but it is questionable for a function called read_mailmap_buf
to be using strchrnul in the first place.  It happens to work
because our buffers come from blobs, and read_sha1_file always
NUL-terminates the object data. But let's future-proof the
function by actually handling non-terminated strings
correctly, and fix the off-by-one at the same time.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
mailmap: avoid allocation when reading from blob

The read_mailmap_blob function reads the whole blob into a
buffer, then calls read_mailmap_buf to do the heavy lifting.
The latter function ends up making a NUL-terminated copy of
each line of the blob in order to call read_mailmap_line,
which was originally written to handle the line-at-a-time
input from fgets().

We can avoid the extra copy if we simply NUL-terminate each
line in place, and assume that the input buffer is itself
NUL-terminated. Neither of these is a problem, since our
only caller is read_mailmap_blob, which has a non-const
NUL-terminated buffer already.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch-pack: do not remove .git/shallow file when --depth is not specified

fetch_pack() can remove .git/shallow file when a shallow repository
becomes a full one again. This behavior is triggered incorrectly when
tags are also fetched because fetch_pack() will be called twice. At
the first fetch_pack() call:

 - shallow_lock is set up
 - alternate_shallow_file points to shallow_lock.filename, which is
   "shallow.lock"
 - commit_lock_file is called, which sets shallow_lock.filename to "".
   alternate_shallow_file also becomes "" because it points to the
   same memory.

At the second call, setup_alternate_shallow() is not called and
alternate_shallow_file remains "". It's mistaken as unshallow case and
.git/shallow is removed. The end result is a broken repository.

Fix this by always initializing alternate_shallow_file when
fetch_pack() is called. As an extra measure, check if args->depth > 0
before commit/rollback shallow file.

Reported-by: Kacper Kornet <kornet@camk.edu.pl>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
write_index: optionally allow broken null sha1s

Commit 4337b58 (do not write null sha1s to on-disk index, 2012-07-28)
added a safety check preventing git from writing null sha1s into the
index. The intent was to catch errors in other parts of the code that
might let such an entry slip into the index (or worse, a tree).

Some existing repositories have some invalid trees that contain null
sha1s already, though.  Until 4337b58, a common way to clean this up
would be to use git-filter-branch's index-filter to repair such broken
entries.  That now fails when filter-branch tries to write out the
index.

Introduce a GIT_ALLOW_NULL_SHA1 environment variable to relax this check
and make it easier to recover from such a history.

It is tempting to not involve filter-branch in this commit at all, and
instead require the user to manually invoke

	GIT_ALLOW_NULL_SHA1=1 git filter-branch ...

to perform an index-filter on a history with trees with null sha1s.
That would be slightly safer, but requires some specialized knowledge
from the user.  So let's set the GIT_ALLOW_NULL_SHA1 variable
automatically when checking out the to-be-filtered trees.  Advice on
using filter-branch to remove such entries already exists on places like
stackoverflow, and this patch makes it Just Work again on recent
versions of git.

Further commands that touch the index will still notice and fail, unless
they actually remove the broken entries.  A filter-branch whose filters
do not touch the index at all will not error out (since we complain of
the null sha1 only on writing, not when making a tree out of the index),
but this is acceptable, as we still print a loud warning, so the problem
is unlikely to go unnoticed.

Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
branch: report invalid tracking branch as gone

Command "git branch -vv" will report tracking branches, but invalid
tracking branches are also reported. This is because the function
stat_tracking_info() can not distinguish invalid tracking branch
from other cases which it would not like to report, such as
there is no upstream settings at all, or nothing is changed between
one branch and its upstream.

Junio suggested missing upstream should be reported [1] like:

    $ git branch -v -v
      master    e67ac84 initial
    * topic     3fc0f2a [topicbase: gone] topic

    $ git status
    # On branch topic
    # Your branch is based on 'topicbase', but the upstream is gone.
    #   (use "git branch --unset-upstream" to fixup)
    ...

    $ git status -b -s
    ## topic...topicbase [gone]
    ...

In order to do like that, we need to distinguish these three cases
(i.e. no tracking, with configured but no longer valid tracking, and
with tracking) in function stat_tracking_info(). So the refactored
function stat_tracking_info() has three return values: -1 (with "gone"
base), 0 (no base), and 1 (with base).

If the caller does not like to report tracking info when nothing
changed between the branch and its upstream, simply checks if
num_theirs and num_ours are both 0.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/231830/focus=232288

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
status: always show tracking branch even no change

In order to see what the current branch is tracking, one way is using
"git branch -v -v", but branches other than the current are also
reported. Another way is using "git status", such as:

    $ git status
    # On branch master
    # Your branch is ahead of 'origin/master' by 1 commit.
    ...

But this will not work if there is no change between the current
branch and its upstream. Always report upstream tracking info
even if there is no difference, so that "git status" is consistent
for checking tracking info for current branch. E.g.

    $ git status
    # On branch feature1
    # Your branch is up-to-date with 'github/feature1'.
    ...

    $ git status -bs
    ## feature1...github/feature1
    ...

    $ git checkout feature1
    Already on 'feature1'
    Your branch is up-to-date with 'github/feature1'.
    ...

Also add some test cases in t6040.

Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation/remote-helpers: document common use-case for private ref

The current documentation mentions the private ref namespace, but does
not really explain why it can be useful.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'km/svn-1.8-serf-only' into jch

Subversion 1.8.0 that was recently released breaks older subversion
clients coming over http/https in various ways.

* km/svn-1.8-serf-only:
  Git.pm: revert _temp_cache use of temp_is_locked
  git-svn: allow git-svn fetching to work using serf
  Git.pm: add new temp_is_locked function
Merge branch 'jc/check-x-z' into jch

"git check-ignore -z" applied the NUL termination to both its input
(with --stdin) and its output, but "git check-attr -z" ignored the
option on the output side.

This is potentially a backward incompatible fix.  I am tempted to
merge this to and keep it in 'next' for a while to see if anybody
screams before deciding if we want to do anything to help existing
users (there may be none).

* jc/check-x-z:
  check-attr -z: a single -z should apply to both input and output
  check-ignore -z: a single -z should apply to both input and output
  check-attr: the name of the character is NUL, not NULL
  check-ignore: the name of the character is NUL, not NULL

# Conflicts:
# 	builtin/check-ignore.c
Merge branch 'ms/fetch-prune-configuration' into jch

Allow fetch.prune and remote.*.prune configuration variables to be set,
and "git fetch" to behave as if "--prune" is given.

"git fetch" that honors remote.*.prune is fine, but I wonder if we
should somehow make "git push" aware of it as well.  Perhaps
remote.*.prune should not be just a boolean, but a 4-way "none",
"push", "fetch", "both"?

* ms/fetch-prune-configuration:
  fetch: make --prune configurable
Merge branch 'rj/doc-rev-parse' into jch

* rj/doc-rev-parse:
  rev-parse(1): logically group options
  rev-parse: remove restrictions on some options
Merge branch 'jc/pull-training-wheel' into jch

Make "git pull" (without arguments that say what branch to
integrate from where) refuse with "it does not fast forward; choose
between 'pull --merge' and 'pull --rebase'".

See

http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=225326
http://thread.gmane.org/gmane.comp.version-control.git/225146/focus=229221

for full discussion.

* jc/pull-training-wheel:
  pull: require choice between rebase/merge on non-fast-forward pull
Merge branch 'jc/diff-filter-negation' into jch

Teach "git diff --diff-filter" to express "I do not want to see
these classes of changes" more directly by listing only the
unwanted ones in lowercase (e.g. "--diff-filter=d" will show
everything but deletion) and deprecate "diff-files -q" which did
the same thing as "--diff-filter=d".

* jc/diff-filter-negation:
  diff: deprecate -q option to diff-files
  diff: allow lowercase letter to specify what change class to exclude
  diff: reject unknown change class given to --diff-filter
  diff: preparse --diff-filter string argument
  diff: factor out match_filter()
  diff: pass the whole diff_options to diffcore_apply_filter()

# Conflicts:
# 	diff.c
Merge branch 'jk/fast-import-empty-ls' into jch

* jk/fast-import-empty-ls:
  fast-import: allow moving the root tree
  fast-import: allow ls or filecopy of the root tree
  fast-import: set valid mode on root tree in "ls"
  t9300: document fast-import empty path issues
Merge branch 'nd/magic-pathspec' into jch

Use "struct pathspec" interface in more places, instead of array of
characters, the latter of which cannot express magic pathspecs
(e.g. ":(icase)makefile" that matches both Makefile and makefile).

* nd/magic-pathspec: (48 commits)
  rm: do not set a variable twice without intermediate reading.
  t6131 - skip tests if on case-insensitive file system
  parse_pathspec: accept :(icase)path syntax
  pathspec: support :(glob) syntax
  pathspec: make --literal-pathspecs disable pathspec magic
  pathspec: support :(literal) syntax for noglob pathspec
  kill limit_pathspec_to_literal() as it's only used by parse_pathspec()
  parse_pathspec: preserve prefix length via PATHSPEC_PREFIX_ORIGIN
  parse_pathspec: make sure the prefix part is wildcard-free
  rename field "raw" to "_raw" in struct pathspec
  tree-diff: remove the use of pathspec's raw[] in follow-rename codepath
  remove match_pathspec() in favor of match_pathspec_depth()
  remove init_pathspec() in favor of parse_pathspec()
  remove diff_tree_{setup,release}_paths
  convert common_prefix() to use struct pathspec
  convert add_files_to_cache to take struct pathspec
  convert {read,fill}_directory to take struct pathspec
  convert refresh_index to take struct pathspec
  convert report_path_error to take struct pathspec
  checkout: convert read_tree_some to take struct pathspec
  ...
Merge branch 'jc/push-cas' into jch

Allow a safer "rewind of the remote tip" push than blind "--force",
by requiring that the overwritten remote ref to be unchanged since
the new history to replace it was prepared.

The machinery is more or less ready.  The "--force" option is again
the big red button to override any safety, thanks to J6t's sanity
(the original round allowed --lockref to defeat --force).

The logic to choose the default implemented here is fragile
(e.g. "git fetch" after seeing a failure will update the
remote-tracking branch and will make the next "push" pass,
defeating the safety pretty easily).  It is suitable only for the
simplest workflows, and it may hurt users more than it helps them.

* jc/push-cas:
  push: teach --force-with-lease to smart-http transport
  send-pack: fix parsing of --force-with-lease option
  t5540/5541: smart-http does not support "--force-with-lease"
  t5533: test "push --force-with-lease"
  push --force-with-lease: tie it all together
  push --force-with-lease: implement logic to populate old_sha1_expect[]
  remote.c: add command line option parser for "--force-with-lease"
  builtin/push.c: use OPT_BOOL, not OPT_BOOLEAN
  cache.h: move remote/connect API out of it

# Conflicts:
# 	remote-curl.c
# 	remote.c
Merge branch 'nd/clone-connectivity-shortcut' into jch

* nd/clone-connectivity-shortcut:
  smart http: use the same connectivity check on cloning

# Conflicts:
# 	remote-curl.c
Merge branch 'rr/feed-real-path-to-editor' into jch

* rr/feed-real-path-to-editor:
  editor: use canonicalized absolute path
Merge branch 'db/http-savecookies' into jch

* db/http-savecookies:
  t5551: Remove header from curl cookie file
  http: add http.savecookies option to write out HTTP cookies
Merge branch 'nd/sq-quote-buf' into jch

Code simplification as a preparatory step to something larger.

* nd/sq-quote-buf:
  quote: remove sq_quote_print()
  tar-tree: remove dependency on sq_quote_print()
  for-each-ref, quote: convert *_quote_print -> *_quote_buf