git-completion.bash: add support for path completion

The git-completion.bash script did not implemented full, git aware,
support to complete paths, for git commands that operate on files within
the current working directory or the index.

As an example:

	git add <TAB>

will suggest all files in the current working directory, including
ignored files and files that have not been modified.

Support path completion, for git commands where the non-option arguments
always refer to paths within the current working directory or the index,
as follows:

* the path completion for the "git rm" and "git ls-files"
  commands will suggest all cached files.

* the path completion for the "git add" command will suggest all
  untracked and modified files.  Ignored files are excluded.

* the path completion for the "git clean" command will suggest all
  untracked files.  Ignored files are excluded.

* the path completion for the "git mv" command will suggest all cached
  files when expanding the first argument, and all untracked and cached
  files for subsequent arguments.  In the latter case, empty directories
  are included and ignored files are excluded.

* the path completion for the "git commit" command will suggest all
  files that have been modified from the HEAD, if HEAD exists, otherwise
  it will suggest all cached files.

For all affected commands, completion will always stop at directory
boundary.  Only standard ignored files are excluded, using the
--exclude-standard option of the ls-files command.

When using a recent Bash version, Git path completion will be the same
as builtin file completion, e.g.

	git add contrib/

will suggest relative file names.

Signed-off-by: Manlio Perillo <manlio.perillo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
simplify-merges: drop merge from irrelevant side branch

The merge simplification rule stated in 6546b59 (revision traversal:
show full history with merge simplification, 2008-07-31) still
treated merge commits too specially.  Namely, in a history with this
shape:

	---o---o---M
	          /
         x---x---x

where three 'x' were on a history completely unrelated to the main
history 'o' and do not touch any of the paths we are following, we
still said that after simplifying all of the parents of M, 'x'
(which is the leftmost 'x' that rightmost 'x simplifies down to) and
'o' (which would be the last commit on the main history that touches
the paths we are following) are independent from each other, and
both need to be kept.

That is incorrect; when the side branch 'x' never touches the paths,
it should be removed to allow M to simplify down to the last commit
on the main history that touches the paths.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit: drop useless xstrdup of commit message

When git-commit is asked to reuse a commit message via "-c",
we call read_commit_message, which looks up the commit and
hands back either the re-encoded result, or a copy of the
original. We make a copy in the latter case so that the
ownership semantics of the return value are clear (in either
case, it can be freed).

However, since we return a "const char *", and since the
resulting buffer's lifetime is the same as that of the whole
program, we never bother to free it at all.

Let's just drop the copy. That saves us a copy in the common
case. While it does mean we leak in the re-encode case, it
doesn't matter, since we are relying on program exit to free
the memory anyway.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
logmsg_reencode: never return NULL

The logmsg_reencode function will return the reencoded
commit buffer, or NULL if reencoding failed or no reencoding
was necessary. Since every caller then ends up checking for NULL
and just using the commit's original buffer, anyway, we can
be a bit more helpful and just return that buffer when we
would have returned NULL.

Since the resulting string may or may not need to be freed,
we introduce a logmsg_free, which checks whether the buffer
came from the commit object or not (callers either
implemented the same check already, or kept two separate
pointers, one to mark the buffer to be used, and one for the
to-be-freed string).

Pushing this logic into logmsg_* simplifies the callers, and
will let future patches lazily load the commit buffer in a
single place.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
logmsg_reencode: lazily load missing commit buffers

Usually a commit that makes it to logmsg_reencode will have
been parsed, and the commit->buffer struct member will be
valid. However, some code paths will free commit buffers
after having used them (for example, the log traversal
machinery will do so to keep memory usage down).

Most of the time this is fine; log should only show a commit
once, and then exits. However, there are some code paths
where this does not work. At least two are known:

  1. A commit may be shown as part of a regular ref, and
     then it may be shown again as part of a submodule diff
     (e.g., if a repo contains refs to both the superproject
     and subproject).

  2. A notes-cache commit may be shown during "log --all",
     and then later used to access a textconv cache during a
     diff.

Lazily loading in logmsg_reencode does not necessarily catch
all such cases, but it should catch most of them. Users of
the commit buffer tend to be either parsing for structure
(in which they will call parse_commit, and either we will
already have parsed, or we will load commit->buffer lazily
there), or outputting (either to the user, or fetching a
part of the commit message via format_commit_message). In
the latter case, we should always be using logmsg_reencode
anyway (and typically we do so via the pretty-print
machinery).

If there are any cases that this misses, we can fix them up
to use logmsg_reencode (or handle them on a case-by-case
basis if that is inappropriate).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4.py: support Python 2.5

Python 2.5 and older do not accept None as the first argument to
translate() and complain with:

   TypeError: expected a character buffer object

As suggested by Pete Wyckoff, let's just replace the call to translate()
with a regex search which should be more clear and more portable.

This allows git-p4 to be used with Python 2.5.

Signed-off-by: Brandon Casey <bcasey@nvidia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-p4.py: support Python 2.4

Python 2.4 lacks the following features:

   subprocess.check_call
   struct.pack_into

Take a cue from 460d1026 and provide an implementation of the
CalledProcessError exception.  Then replace the calls to
subproccess.check_call with calls to subprocess.call that check the return
status and raise a CalledProcessError exception if necessary.

The struct.pack_into in t/9802 can be converted into a single struct.pack
call which is available in Python 2.4.

Signed-off-by: Brandon Casey <bcasey@nvidia.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: run gc --auto after fetching

We generally try to run "gc --auto" after any commands that
might introduce a large number of new objects. An obvious
place to do so is after running "fetch", which may introduce
new loose objects or packs (depending on the size of the
fetch).

While an active developer repository will probably
eventually trigger a "gc --auto" on another action (e.g.,
git-rebase), there are two good reasons why it is nicer to
do it at fetch time:

  1. Read-only repositories which track an upstream (e.g., a
     continuous integration server which fetches and builds,
     but never makes new commits) will accrue loose objects
     and small packs, but never coalesce them into a more
     efficient larger pack.

  2. Fetching is often already perceived to be slow to the
     user, since they have to wait on the network. It's much
     more pleasant to include a potentially slow auto-gc as
     part of the already-long network fetch than in the
     middle of productive work with git-rebase or similar.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch-pack: avoid repeatedly re-scanning pack directory

When we look up a sha1 object for reading via parse_object() =>
read_sha1_file() => read_object() callpath, we first check
packfiles, and then loose objects. If we still haven't found it, we
re-scan the list of packfiles in `objects/pack`. This final step
ensures that we can co-exist with a simultaneous repack process
which creates a new pack and then prunes the old object.

This extra re-scan usually does not have a performance impact for
two reasons:

  1. If an object is missing, then typically the re-scan will find a
     new pack, then no more misses will occur.  Or if it truly is
     missing, then our next step is usually to die().

  2. Re-scanning is cheap enough that we do not even notice.

However, these do not always hold. The assumption in (1) is that the
caller is expecting to find the object. This is usually the case,
but the call to `parse_object` in `everything_local` does not follow
this pattern. It is looking to see whether we have objects that the
remote side is advertising, not something we expect to
have. Therefore if we are fetching from a remote which has many refs
pointing to objects we do not have, we may end up re-scanning the
pack directory many times.

Even with this extra re-scanning, the impact is often not noticeable
due to (2); we just readdir() the packs directory and skip any packs
that are already loaded. However, if there are a large number of
packs, even enumerating the directory can be expensive, especially
if we do it repeatedly.

Having this many packs is a good sign the user should run `git gc`,
but it would still be nice to avoid having to scan the directory at
all.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jk/maint-gc-auto-after-fetch' into jk/gc-auto-after-fetch

* jk/maint-gc-auto-after-fetch:
  fetch-pack: avoid repeatedly re-scanning pack directory
  fetch: run gc --auto after fetching
git p4: temp branch name should use / even on windows

Commit fed2369 (git-p4: Search for parent commit on branch creation,
2012-01-25) uses temporary branches to help find the parent of a
new p4 branch.  The temp branches are of the form "git-p4-tmp/%d"
for some p4 change number.  Mistakenly, this string was made
using os.path.join() instead of just string concatenation.  On
windows, this turns into a backslash (\), which is not allowed in
git branch names.

Reported-by: Casey McGinty <casey.mcginty@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: remove unused imports

Found by "pyflakes" checker tool.
Modules shelve, getopt were unused.
Module os.path is exported by os.
Reformat one-per-line as is PEP008 suggested style.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: generate better error message for bad depot path

Depot paths must start with //.  Exit with a better explanation
when a bad depot path is supplied.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: use client_view to build the initial client

Simplify the code a bit by using an existing function.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: avoid loop in client_view

The printf command re-interprets the format string as
long as there are arguments to consume.  Use this to
simplify a for loop in the client_view() library function.

This requires a fix to one of the client_view callers.
An errant \n in the string was converted into a harmless
newline in the input to "p4 client -i", but now shows up
as a literal \n as passed through by "%s".  Remove the \n.

Based-on-patch-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: use client_view in t9806

Use the standard client_view function from lib-git-p4.sh
instead of building one by hand.  This requires a bit of
rework, using the current value of $P4CLIENT for the client
name.  It also reorganizes the test to isolate changes to
$P4CLIENT and $cli in a subshell.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: start p4d inside its db dir

This will avoid having to do native path conversion for
windows.  Also may be a bit cleaner always to know that p4d
has that working directory, instead of wherever the function
was called from.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4 test: translate windows paths for cygwin

Native windows binaries do not understand posix-like
path mapping offered by cygwin.  Convert paths to native
using "cygpath --windows" before presenting them to p4d.

This is done using the AltRoots mechanism of p4.  Both the
posix and windows forms are put in the client specification,
allowing p4 to find its location by native path even though
the environment reports a different PWD.

Shell operations in tests will use the normal form of $cli,
which will look like a posix path in cygwin, while p4 will
use AltRoots to match against the windows form of the working
directory.

This mechanism also handles the symlink issue that was fixed in
23bd0c9 (git p4 test: use real_path to resolve p4 client
symlinks, 2012-06-27).  Now that every p4 client view has
an AltRoots with the real_path in it, explicitly calculating
the real_path elsewhere is not necessary.

Thanks-to: Sebastian Schuberth <sschuberth@gmail.com>
Thanks-to: Johannes Sixt <j6t@kdbg.org>

fixup! git p4 test: translate windows paths for cygwin

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: remove unreachable windows \r\n conversion code

Replacing \r\n with \n on windows was added in c1f9197 (Replace
\r\n with \n when importing from p4 on Windows, 2007-05-24), to
work around an oddity with "p4 print" on windows.  Text files
are printed with "\r\r\n" endings, regardless of whether they
were created on unix or windows, and regardless of the client
LineEnd setting.

As of d2c6dd3 (use p4CmdList() to get file contents in Python
dicts. This is more robust., 2007-05-23), git-p4 uses "p4 -G
print", which generates files in a raw format.  As the native
line ending format if p4 is \n, there will be no \r\n in the
raw text.

Actually, it is possible to generate a text file so that the
p4 representation includes embedded \r\n, even though this is not
normal on either windows or unix.  In that case the code would
have mistakenly stripped them out, but now they will be left
intact.

More information on how p4 deals with line endings is here:

    http://kb.perforce.com/article/63

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git p4: scrub crlf for utf16 files on windows

Files of type utf16 are handled with "p4 print" instead of the
normal "p4 -G print" interface due to how the latter does not
produce correct output.  See 55aa571 (git-p4: handle utf16
filetype properly, 2011-09-17) for details.

On windows, though, "p4 print" can not be told which line
endings to use, as there is no underlying client, and always
chooses crlf, even for utf16 files.  Convert the \r\n into \n
when importing utf16 files.

The fix for this is complex, in that the problem is a property
of the NT version of p4.  There are old versions of p4 that
were compiled directly for cygwin that should not be subjected
to text replacement.  The right check here, then, is to look
at the p4 version, not the OS version.  Note also that on cygwin,
platform.system() is "CYGWIN_NT-5.1" or similar, not "Windows".

Add a function to memoize the p4 version string and use it to
check for "/NT", indicating the Windows build of p4.

Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>