status/commit: show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for the diff family,
status and commit. For status and commit this is really confusing, as it
even when the user chooses to record a new commit for an ignored submodule
by adding it manually this change won't show up under the to-be-committed
changes. To add insult to injury, a later "git commit" will error out with
"nothing to commit" when only ignored submodules are staged.

Fix that by making wt_status always print staged submodule changes, no
matter what ignore settings are configured. The only exception is when the
user explicitly uses the "--ignore-submodules=all" command line option, in
that case the submodule output is still suppressed. This also makes "git
commit" work again when only modifications of ignored submodules are
staged, as that command uses the "commitable" member of the wt_status
struct to determine if staged changes are present. But this only happens
when the commit command uses the wt_status* functions to produce status
output for human consumption (when forking an editor or with --dry-run),
in all other cases (e.g. when run in a script with '-m') another code path
is taken which uses index_differs_from() to determine if any changes are
staged which still ignores submodules according to their configuration.
This will be fixed in a follow-up commit.

Change t7508 to reflect this new behavior and add three new tests to show
that a single staged submodule configured to be ignored will be committed
when the status output is generated and won't be if not. Also update the
documentation of the ignore config options accordingly.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit -m: commit staged submodules regardless of ignore config

The previous commit fixed the problem that the staged but that ignored
submodules did not show up in the status output of the commit command and
weren't committed afterwards either. But when commit doesn't generate the
status output (e.g. when used in a script with '-m') the ignored submodule
will still not be committed. This is because in that case a different code
path is taken which calls index_differs_from() instead of calling the
wt_status functions.

Fix that by calling index_differs_from() from builtin/commit.c with a
diff_options argument value that tells it not ignore any submodule changes
unless the '--ignore-submodules' option is used. Even though this option
isn't yet implemented for cmd_commit() but only for cmd_status() this
prepares cmd_commit() to correctly handle the '--ignore-submodules' option
later. As status and commit share the same ignore_submodule_arg variable
this makes the code more robust against accidental breakage and documents
how to correctly call index_differs_from().

Change the expected result of the test documenting this problem from
failure to success.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-export: improve argument parsing

We don't want to pass arguments specific to fast-export to
setup_revisions.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-export: add new --refspec option

So that we can convert the exported ref names.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
transport-helper: add support for old:new refspec

By using fast-export's new --refspec option.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
transport-helper: add support to push symbolic refs

For example 'HEAD'.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-import: add support to delete refs

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-export: add support to delete refs

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
transport-helper: add support to delete branches

For remote-helpers that use 'export' to push.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
transport-helper: remove unnecessary strbuf resets

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit: do not complain of empty messages from -C

When we pick another commit's message, we die() immediately
if we find that it's empty and we are not going to run an
editor (i.e., when running "-C" instead of "-c").  However,
this check is redundant and harmful.

It's redundant because we will already notice the empty
message later, after we would have run the editor, and die
there (just as we would for a regular, not "-C" case, where
the user provided an empty message in the editor).

It's harmful for a few reasons:

  1. It does not respect --allow-empty-message. As a result,
     a "git rebase -i" cannot "pick" such a commit. So you
     cannot even go back in time to fix it with a "reword"
     or "edit" instruction.

  2. It does not take into account other ways besides the
     editor to modify the message. For example, "git commit
     -C empty-commit -m foo" could take the author
     information from empty-commit, but add a message to it.
     There's more to do to make that work correctly (and
     right now we explicitly forbid "-C with -m"), but this
     removes one roadblock.

  3. The existing check is not enough to prevent segfaults.
     We try to find the "\n\n" header/body boundary in the
     commit. If it is at the end of the string (i.e., no
     body), _or_ if we cannot find it at all (i.e., a
     truncated commit object), we consider the message
     empty. With "-C", that's OK; we die in either case. But
     with "-c", we continue on, and in the case of a
     truncated commit may end up dereferencing NULL+2.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: large-scale performance rewrite

The previous implementation used a single sorted linear list of blame
entries for organizing all partial or completed work.  Every subtask had
to scan the whole list, with most entries not being relevant to the
task.  The resulting run-time was quadratic to the number of separate
chunks.

This change gives every subtask its own data to work with.  Subtasks are
organized into "struct origin" chains hanging off particular commits.
Commits are organized into a priority queue, processing them in commit
date order in order to keep most of the work affecting a particular blob
collated even in the presence of an extensive merge history.

For large files with a diversified history, a speedup by a factor of 3
or more is not unusual.

Signed-off-by: David Kastrup <dak@gnu.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pager: do allow spawning pager recursively

This reverts commit 88e8f908f2b0c56f9ccf8134d8ff9f689af9cc84, which
tried to allow

    GIT_PAGER="git -p column --mode='dense color'" git -p branch

and still wanted to avoid "git -p column" to invoke itself.  However,
this falls into "don't do that -p then" category.

In particular, inside "git log", with results going through less, a
potentially interesting commit may be found and from there inside
"less", the user may want to execute "git show <commit>".  Before
the commit being reverted, this used to show the patch in less but
it no longer does.

Signed-off-by: Jörn Engel <joern@logfs.org>
Reviewed-by: Jeff King <peff@peff.net>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Acked-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
compat/poll: sleep 1 millisecond to avoid busy wait

SwitchToThread() only gives away the rest of the current time slice
to another thread in the current process. So if the thread that feeds
the file decscriptor we're polling is not in the current process, we
get busy-waiting.

I played around with this quite a bit. After trying some more complex
schemes, I found that what worked best is to just sleep 1 millisecond
between iterations. Though it's a very short time, it still completely
eliminates the busy wait condition, without hurting perf.

There code uses SleepEx(1, TRUE) to sleep. See this page for a good
discussion of why that is better than calling SwitchToThread, which
is what was used previously:
http://stackoverflow.com/questions/1383943/switchtothread-vs-sleep1

Note that calling SleepEx(0, TRUE) does *not* solve the busy wait.

The most striking case was when testing on a UNC share with a large repo,
on a single CPU machine. Without the fix, it took 4 minutes 15 seconds,
and with the fix it took just 1:08! I think it's because git-upload-pack's
busy wait was eating the CPU away from the git process that's doing the
real work. With multi-proc, the timing is not much different, but tons of
CPU time is still wasted, which can be a killer on a server that needs to
do bunch of other things.

I also tested the very fast local case, and didn't see any measurable
difference. On a big repo with 4500 files, the upload-pack took about 2
seconds with and without the fix.

[jc: this was first accepted in msysgit tree in May 2012 via a pull
request and Paolo Bonzini has also accepted the same fix to Gnulib
around the same time; see $gmane/247518 for a bit more detail]

Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Acked-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t3910: show failure of core.precomposeunicode with decomposed filenames

If you have existing decomposed filenames in your git
repository (e.g., that were created with older versions of
git that did not precompose unicode), a modern git with
core.precomposeunicode set does not handle them well.

The problem is that we normalize the paths coming from the
disk into their precomposed form, and then compare them
against the literal bytes in the index. This makes things
better if you have the precomposed form in the index. It
makes things worse if you actually have the decomposed form
in the index.

As a result, paths with decomposed filenames may have their
precomposed variants listed as untracked files (even though
the precomposed variants do not exist on-disk at all).

This patch just adds a test to demonstrate the breakage.
Some possible fixes are:

  1. Tell everyone that NFD in the git repo is wrong, and
     they should make a new commit to normalize all their
     in-repo files to be precomposed.

     This is probably not the right thing to do, because it
     still doesn't fix checkouts of old history. And it
     spreads the problem to people on byte-preserving
     filesystems (like ext4), because now they have to start
     precomposing their filenames as they are adde to git.

  2. Do all index filename comparisons using a UTF-8 aware
     comparison function when core.precomposeunicode is set.
     This would probably have bad performance, and somewhat
     defeats the point of converting the filenames at the
     readdir level in the first place.

  3. Convert index filenames to their precomposed form when
     we read the index from disk. This would be efficient,
     but we would have to be careful not to write the
     precomposed forms back out to disk.

  4. Introduce some infrastructure to efficiently match up
     the precomposed/decomposed forms. We already do
     something similar for case-insensitive files using
     name-hash.c. We might be able to adapt that strategy
     here.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
imap-send: use git-credential

git-imap-send was directly prompting for a password rather than using
git-credential. git-send-email, on the other hand, supports git-credential.

This is a necessary improvement for users that use two factor authentication, as
they should not be expected to remember all of their app specific passwords.

Signed-off-by: Dan Albert <danalbert@google.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
replace: refactor command-mode determination

The git-replace command has three modes: listing, deleting,
and replacing. The first two are selected explicitly. If
none is selected, we fallback to listing when there are no
arguments, and replacing otherwise.

Let's figure out up front which operation we are going to
do, before getting into the application logic. That lets us
simplify our option checks (e.g., we currently have to check
whether a useless "--force" is given both along with an
explicit list, as well as with an implicit one).

This saves some lines, makes the logic easier to follow, and
will facilitate further cleanups.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
replace: use OPT_CMDMODE to handle modes

By using OPT_CMDMODE, the mutual exclusion between modes is
taken care of for us. It also makes it easy for us to
maintain a single variable with the mode, which makes its
intent more clear. We can use a single switch() to make sure
we have covered all of the modes.

This ends up breaking even in code size, but the win will be
much bigger when we start adding more modes.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
replace: factor object resolution out of replace_object

As we add new options that operate on objects before
replacing them, we'll want to be able to feed raw sha1s
straight into replace_object. Split replace_object into the
object-resolution part and the actual replacement.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
replace: add --edit option

This allows you to run:

    git replace --edit SHA1

to get dumped in an editor with the contents of the object
for SHA1. The result is then read back in and used as a
"replace" object for SHA1. The writing/reading is
type-aware, so you get to edit "ls-tree" output rather than
the binary tree format.

Missing documentation and tests.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>