apply --ignore-space-change: lines with and without leading whitespaces do not match

The fuzzy_matchlines() function is used when attempting to resurrect
a patch that is whitespace-damaged, or when applying a patch that
was produced against an old codebase to the codebase after
indentation change.

The patch may want to change a line "a_bc" ("_" is used throught
this description for a whitespace to make it stand out) in the
original into something else, and we may not find "a_bc" in the
current source, but there may be "a__bc" (two spaces instead of one
the whitespace-damaged patch claims to expect).  By ignoring the
amount of whitespaces, it forces "git apply" to consider that "a_bc"
in the broken patch meant to refer to "a__bc" in reality.

However, the implementation special cases a run of whitespaces at
the beginning of a line and makes "abc" match "_abc", even though a
whitespace in the middle of string never matches a 0-width gap,
e.g. "a_bc" does not match "abc".  A run of whitespace at the end of
one string does not match a 0-width end of line on the other line,
either, e.g. "abc_" does not match "abc".

Fix this inconsistency by making the code skip leading whitespaces
only when both strings begin with a whitespace.  This makes the
option mean the same as the option of the same name in "diff" and
"git diff".

Note that I am not sure if anybody sane should use this option in
the first place.  The fuzzy match logic may be able to find the
original line that the patch author may have meant to touch because
it does not fully trust what the original lines say (i.e. context
lines prefixed by " " and old lines prefixed by "-" does not have to
exactly match the contents the patch is applied to).  There is no
reason for us to trust what the replacement lines (i.e. new lines
prefixed by "+") say, either, but with this option enabled, we end
up copying these new lines with suspicious whitespace distributions
literally into the patched result.  But as long as we keep it, we
should make it do its insane thing consistently.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
status/commit: show staged submodules regardless of ignore config

Currently setting submodule.<name>.ignore and/or diff.ignoreSubmodules to
"all" suppresses all output of submodule changes for the diff family,
status and commit. For status and commit this is really confusing, as it
even when the user chooses to record a new commit for an ignored submodule
by adding it manually this change won't show up under the to-be-committed
changes. To add insult to injury, a later "git commit" will error out with
"nothing to commit" when only ignored submodules are staged.

Fix that by making wt_status always print staged submodule changes, no
matter what ignore settings are configured. The only exception is when the
user explicitly uses the "--ignore-submodules=all" command line option, in
that case the submodule output is still suppressed. This also makes "git
commit" work again when only modifications of ignored submodules are
staged, as that command uses the "commitable" member of the wt_status
struct to determine if staged changes are present. But this only happens
when the commit command uses the wt_status* functions to produce status
output for human consumption (when forking an editor or with --dry-run),
in all other cases (e.g. when run in a script with '-m') another code path
is taken which uses index_differs_from() to determine if any changes are
staged which still ignores submodules according to their configuration.
This will be fixed in a follow-up commit.

Change t7508 to reflect this new behavior and add three new tests to show
that a single staged submodule configured to be ignored will be committed
when the status output is generated and won't be if not. Also update the
documentation of the ignore config options accordingly.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit -m: commit staged submodules regardless of ignore config

The previous commit fixed the problem that the staged but that ignored
submodules did not show up in the status output of the commit command and
weren't committed afterwards either. But when commit doesn't generate the
status output (e.g. when used in a script with '-m') the ignored submodule
will still not be committed. This is because in that case a different code
path is taken which calls index_differs_from() instead of calling the
wt_status functions.

Fix that by calling index_differs_from() from builtin/commit.c with a
diff_options argument value that tells it not ignore any submodule changes
unless the '--ignore-submodules' option is used. Even though this option
isn't yet implemented for cmd_commit() but only for cmd_status() this
prepares cmd_commit() to correctly handle the '--ignore-submodules' option
later. As status and commit share the same ignore_submodule_arg variable
this makes the code more robust against accidental breakage and documents
how to correctly call index_differs_from().

Change the expected result of the test documenting this problem from
failure to success.

Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
wrapper.c: add xpread() similar to xread()

It is a common mistake to call read(2)/pread(2) and forget to
anticipate that they may return error with EAGAIN/EINTR when the
system call is interrupted.

We have xread() helper to relieve callers of read(2) from having to
worry about it; add xpread() helper to do the same for pread(2).

Update the caller in the builtin/index-pack.c and the mmap emulation
in compat/.

Signed-off-by: Yiannis Marangos <yiannis.marangos@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read-cache.c: verify index file before we opportunistically update it

Before we proceed to opportunistically update the index (often done
by an otherwise read-only operation like "git status" and "git diff"
that internally refreshes the index), we must verify that the
current index file is the same as the one that we read earlier
before we took the lock on it, in order to avoid a possible race.

In the example below git-status does "opportunistic update" and
git-rebase updates the index, but the race can happen in general.

  1. process A calls git-rebase (or does anything that uses the index)

  2. process A applies 1st commit

  3. process B calls git-status (or does anything that updates the index)

  4. process B reads index

  5. process A applies 2nd commit

  6. process B takes the lock, then overwrites process A's changes.

  7. process A applies 3rd commit

As an end result the 3rd commit will have a revert of the 2nd commit.
When process B takes the lock, it needs to make sure that the index
hasn't changed since step 4.

Signed-off-by: Yiannis Marangos <yiannis.marangos@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
index-pack: work around thread-unsafe pread()

Multi-threaing of index-pack was disabled with c0f8654
(index-pack: Disable threading on cygwin - 2012-06-26), because
pread() implementations for Cygwin and MSYS were not thread
safe.  Recent Cygwin does offer usable pread() and we enabled
multi-threading with 103d530f (Cygwin 1.7 has thread-safe pread,
2013-07-19).

Work around this problem on platforms with a thread-unsafe
pread() emulation by opening one file handle per thread; it
would prevent parallel pread() on different file handles from
stepping on each other.

Also remove NO_THREAD_SAFE_PREAD that was introduced in c0f8654
because it's no longer used anywhere.

This workaround is unconditional, even for platforms with
thread-safe pread() because the overhead is small (a couple file
handles more) and not worth fragmenting the code.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Tested-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: fix broken time_buf paddings in relative timestamp

Command `git blame --date relative` aligns the date field with a
fixed-width (defined by blame_date_width), and if time_str is shorter
than that, it adds spaces for padding.  But there are two bugs in the
following codes:

        time_len = strlen(time_str);
        ...
        memset(time_buf + time_len, ' ', blame_date_width - time_len);

 1. The type of blame_date_width is size_t, which is unsigned.  If
    time_len is greater than blame_date_width, the result of
    "blame_date_width - time_len" will never be a negative number, but a
    really big positive number, and will cause memory overwrite.

    This bug can be triggered if either l10n message for function
    show_date_relative() in date.c is longer than 30 characters, then
    `git blame --date relative` may exit abnormally.

 2. When show blame information with relative time, the UTF-8 characters
    in time_str will break the alignment of columns after the date field.
    This is because the time_buf padding with spaces should have a
    constant display width, not a fixed strlen size.  So we should call
    utf8_strwidth() instead of strlen() for width calibration.

Helped-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: dynamic blame_date_width for different locales

When show date in relative date format for git-blame, the max display
width of datetime is set as the length of the string "Thu Oct 19
16:00:04 2006 -0700" (30 characters long).  But actually the max width
for C locale is only 22 (the length of string "x years, xx months ago").
And for other locale, it maybe smaller.  E.g. For Chinese locale, only
needs a half (16-character width).

Set blame_date_width as the display width of _("4 years, 11 months
ago"), so that translators can make the choice.

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit: do not complain of empty messages from -C

When we pick another commit's message, we die() immediately
if we find that it's empty and we are not going to run an
editor (i.e., when running "-C" instead of "-c").  However,
this check is redundant and harmful.

It's redundant because we will already notice the empty
message later, after we would have run the editor, and die
there (just as we would for a regular, not "-C" case, where
the user provided an empty message in the editor).

It's harmful for a few reasons:

  1. It does not respect --allow-empty-message. As a result,
     a "git rebase -i" cannot "pick" such a commit. So you
     cannot even go back in time to fix it with a "reword"
     or "edit" instruction.

  2. It does not take into account other ways besides the
     editor to modify the message. For example, "git commit
     -C empty-commit -m foo" could take the author
     information from empty-commit, but add a message to it.
     There's more to do to make that work correctly (and
     right now we explicitly forbid "-C with -m"), but this
     removes one roadblock.

  3. The existing check is not enough to prevent segfaults.
     We try to find the "\n\n" header/body boundary in the
     commit. If it is at the end of the string (i.e., no
     body), _or_ if we cannot find it at all (i.e., a
     truncated commit object), we consider the message
     empty. With "-C", that's OK; we die in either case. But
     with "-c", we continue on, and in the case of a
     truncated commit may end up dereferencing NULL+2.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pager: do allow spawning pager recursively

This reverts commit 88e8f908f2b0c56f9ccf8134d8ff9f689af9cc84, which
tried to allow

    GIT_PAGER="git -p column --mode='dense color'" git -p branch

and still wanted to avoid "git -p column" to invoke itself.  However,
this falls into "don't do that -p then" category.

In particular, inside "git log", with results going through less, a
potentially interesting commit may be found and from there inside
"less", the user may want to execute "git show <commit>".  Before
the commit being reverted, this used to show the patch in less but
it no longer does.

Signed-off-by: Jörn Engel <joern@logfs.org>
Reviewed-by: Jeff King <peff@peff.net>
Reviewed-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Acked-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
rerere: fix for merge.conflictstyle

If we use a different conflict style `git rerere forget` is not able
to find the matching conflict SHA-1 because the diff generated is
actually different from what `git merge` generated, due to the
XDL_MERGE_* option differences among the codepaths.

The fix is to call git_xmerge_config() so that git_xmerge_style is set
properly and the diffs match.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
compat/bswap.h: detect endianness on more platforms that don't use BYTE_ORDER

Signed-off-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
compat/bswap.h: restore preference __BIG_ENDIAN over BIG_ENDIAN

The previous commit swaps the order we check the macros defined by
the compiler and the system headers from the original.  Since the
order of check should not matter (i.e. it is insane to define both
__BIG_ENDIAN and friends and BIG_ENDIAN and friends and in a
conflicting way), it is the most conservative thing to do not to
change it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
merge-recursive.c: fix case-changing merge bug

On a case-insensitive filesystem, when merging, a file would be
wrongly deleted from the working tree if an incoming commit had
renamed it changing only its case.  When merging a rename, the file
with the old name would be deleted -- but since the filesystem
considers the old name to be the same as the new name, the new
file would in fact be deleted.

We avoid this by not deleting files that have a case-clone in the
index at stage 0.

Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
mv: allow renaming to fix case on case insensitive filesystems

"git mv hello.txt Hello.txt" on a case insensitive filesystem
always triggers "destination already exists" error, because these
two names refer to the same path from the filesystem's point of
view, and requires the user to give "--force" when correcting the
case of the path recorded in the index and in the next commit.

Detect this case and allow it without requiring "--force".

Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: correctly handle files regardless of autocrlf

If a file contained CRLF line endings in a repository with
core.autocrlf=input, then blame always marked lines as "Not
Committed Yet", even if they were unmodified.  Don't attempt to
convert the line endings when creating the fake commit so that blame
works correctly regardless of the autocrlf setting.

Reported-by: Ephrim Khong <dr.khong@gmail.com>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
index-pack: distinguish missing objects from type errors

When we fetch a pack that does not contain an object we
expected to receive, we get an error like:

  $ git init --bare tmp.git && cd tmp.git
  $ git fetch ../parent.git
  [...]
  error: Could not read 964953ec7bcc0245cb1d0db4095455edd21a2f2e
  fatal: Failed to traverse parents of commit b8247b40caf6704fe52736cdece6d6aae87471aa
  error: ../parent.git did not send all necessary objects

This comes from the check_everything_connected rev-list. If
we try cloning the same repo (rather than a fetch), we end
up using index-pack's --check-self-contained-and-connected
option instead, which produces output like:

  $ git clone --no-local --bare parent.git tmp.git
  [...]
  fatal: object of unexpected type
  fatal: index-pack failed

Not only is the sha1 missing, but it's a misleading message.
There's no type problem, but rather a missing object
problem; we don't notice the difference because we simply
compare OBJ_BAD != OBJ_BLOB.  Let's provide a different
message for this case:

  $ git clone --no-local --bare parent.git tmp.git
  fatal: did not receive expected object 6b00a8c61ed379d5f925a72c1987c9c52129d364
  fatal: index-pack failed

While we're at it, let's also improve a true type mismatch
error to look like

  fatal: object 6b00a8c61ed379d5f925a72c1987c9c52129d364: expected type blob, got tree

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-show: fix 'git show -s' to not add extra terminator after merge commit

When git show -s is called for merge commit it prints extra newline
after any merge commit. This differs from output for commits with one
parent. Fix it by more thorough checking that diff output is disabled.

The code in question exists since commit 3969cf7db1. The additional
newline is really needed for cases when patch is requested, test
t4013-diff-various.sh contains cases which can demonstrate behavior when
the condition is restricted further.

Tests:

Added merge commit to 'set up a bit of history' case in t7007-show.sh to
cover the fix.

Existing tests are updated to demonstrate the new behaviour.  Earlier,
the tests that used "git show -s --pretty=format:%s", even though
"--pretty=format:%s" calls for item separator semantics and does not ask
for the terminating newline after the last item, expected the output to
end with such a newline.  They were relying on the buggy behaviour.  Use
of "--format=%s", which is equivalent to "--pretty=tformat:%s" that asks
for a terminating newline after each item, is a more realistic way to
use the command.

In the test 'merge log messages' the expected data is changed, because
it was explicitly listing the extra newline. Also the msg.nologff and
msg.nolognoff expected files are replaced by one msg.nolog, because they
were diffing because of the bug, and now there should be no difference.

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
run_diff_files: do not look at uninitialized stat data

If we try to diff an index entry marked CE_VALID (because it
was marked with --assume-unchanged), we do not bother even
running stat() on the file to see if it was removed. This
started long ago with 540e694 (Prevent diff machinery from
examining assume-unchanged entries on worktree, 2009-08-11).

However, the subsequent code may look at our "struct stat"
and expect to find actual data; currently it will find
whatever cruft was left on the stack. This can cause
problems in two situations:

  1. We call match_stat_with_submodule with the stat data,
     so a submodule may be erroneously marked as changed.

  2. If --find-copies-harder is in effect, we pass all
     entries, even unchanged ones, to diff_change, so it can
     list them as rename/copy sources. Since we found no
     change, we assume that function will realize it and not
     actually display any diff output. However, we end up
     feeding it a bogus mode, leading it to sometimes claim
     there was a mode change.

We can fix both by splitting the CE_VALID and regular code
paths, and making sure only to look at the stat information
in the latter. Furthermore, we push the declaration of our
"struct stat" down into the code paths that actually set it,
so we cannot accidentally access it uninitialized in future
code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
open_sha1_file: report "most interesting" errno

When we try to open a loose object file, we first attempt to
open in the local object database, and then try any
alternates. This means that the errno value when we return
will be from the last place we looked (and due to the way
the code is structured, simply ENOENT if we do not have have
any alternates).

This can cause confusing error messages, as read_sha1_file
checks for ENOENT when reporting a missing object. If errno
is something else, we report that. If it is ENOENT, but
has_loose_object reports that we have it, then we claim the
object is corrupted. For example:

    $ chmod 0 .git/objects/??/*
    $ git rev-list --all
    fatal: loose object b2d6fab18b92d49eac46dc3c5a0bcafabda20131 (stored in .git/objects/b2/d6fab18b92d49eac46dc3c5a0bcafabda20131) is corrupt

This patch instead keeps track of the "most interesting"
errno we receive during our search. We consider ENOENT to be
the least interesting of all, and otherwise report the first
error found (so problems in the object database take
precedence over ones in alternates). Here it is with this
patch:

    $ git rev-list --all
    fatal: failed to read object b2d6fab18b92d49eac46dc3c5a0bcafabda20131: Permission denied

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>