[NEEDSTEST] checkout: avoid unnecessary match_pathspec calls

In checkout_paths() we do this

 - for all updated items, call match_pathspec
 - for all items, call match_pathspec (inside unmerge_cache)
 - for all items, call match_pathspec (for showing "path .. is unmerged)
 - for updated items, call match_pathspec and update paths

That's a lot of duplicate match_pathspec(s) and the function is not
exactly cheap to be called so many times, especially on a large index.
Call match_pathspec once per updated index entry, save the result in
ce_flags and reuse the results in the following loops instead.

The changes in 0a1283b (checkout $tree $path: do not clobber local
changes in $path not in $tree - 2011-09-30) limit the affected paths
to ones we read from $tree. We do not do anything to other modified
entries in this case, so the "for all items" above could be modified
to "for all updated items". But..

The command's behavior now is modified slightly: unmerged entries that
match $path, but not updated by $tree, are now NOT touched.  Although
this should be considered a bug fix, not a regression.

And while at there, free ps_matched after use.

The following command is tested on webkit, 215k entries. The pattern
is chosen mainly to make match_pathspec sweat:

git checkout -- "*[a-zA-Z]*[a-zA-Z]*[a-zA-Z]*"

        before      after
real    0m3.493s    0m2.737s
user    0m2.239s    0m1.586s
sys     0m1.252s    0m1.151s

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
attr.c::path_matches(): the basename is part of the pathname

The function takes two strings (pathname and basename) as if they
are independent strings, but in reality, the latter is always
pointing into a substring in the former.

Clarify this relationship by expressing the latter as an offset into
the former.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c::match_basename(): pay attention to the length of string parameters

The function takes two counted strings (<basename, basenamelen> and
<pattern, patternlen>) as parameters, together with prefix (the
length of the prefix in pattern that is to be matched literally
without globbing against the basename) and EXC_* flags that tells it
how to match the pattern against the basename.

However, it did not pay attention to the length of these counted
strings.  Update them to do the following:

 * When the entire pattern is to be matched literally, the pattern
   matches the basename only when the lengths of them are the same,
   and they match up to that length.

 * When the pattern is "*" followed by a string to be matched
   literally, make sure that the basenamelen is equal or longer than
   the "literal" part of the pattern, and the tail of the basename
   string matches that literal part.

 * Otherwise, make sure we use only the counted part of the strings
   when calling fnmatch_icase().  Because these counted strings are
   full strings most of the time, avoid unnecessary allocation.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
attr.c::path_matches(): special case paths that end with a slash

The function is given a string that ends with a slash to signal that
the path is a directory to make sure that a pattern that ends with a
slash (i.e. MUSTBEDIR) can tell directories and non-directories
apart.  However, the pattern itself (pat->pattern and
pat->patternlen) that came from such a MUSTBEDIR pattern is
represented as a string that ends with a slash, but patternlen does
not count that trailing slash. A MUSTBEDIR pattern "element/" is
represented as a counted string <"element/", 7> and this must match
match pathname "element/".

Because match_basename() wants to see pathname "element" to match
against the pattern <"element/", 7>, reduce the length of the path
to exclude the trailing slash when calling match_basename().

A similar adjustment for match_pathname() might be needed, but I
didn't look into it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
make sure a pattern without trailing slash matches a directory

Prior to v1.8.1.1, with:

  git init
  echo content >foo &&
  mkdir subdir &&
  echo content >subdir/bar &&
  echo "subdir export-ignore" >.gitattributes
  git add . &&
  git commit -m one &&
  git archive HEAD | tar tf -

the resulting archive would contain only "foo" and ".gitattributes",
not subdir.  This was broken with a recent change that intended to
allow "subdir/ export-ignore" to also exclude the directory, but
instead ended up _requiring_ the trailing slash by mistake.

A pattern "subdir" should match any path "subdir", whether it is a
directory or a non-diretory.  A pattern "subdir/" insists that a
path "subdir" must be a directory for it to match.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jk/checkout-attribute-lookup' into jch

Codepath to stream blob object contents directly from the object
store to filesystem did not use the correct path to find conversion
filters when writing to temporary files.

* jk/checkout-attribute-lookup:
  t2003: work around path mangling issue on Windows
  entry: fix filter lookup
  t2003: modernize style
Merge branch 'jk/difftool-dir-diff-edit-fix' into jch

"git difftool --dir-diff" made symlinks to working tree files when
preparing a temporary directory structure, so that accidental edits
of these files in the difftool are reflected back to the working
tree, but the logic to decide when to do so was not quite right.

* jk/difftool-dir-diff-edit-fix:
  difftool --dir-diff: symlink all files matching the working tree
  difftool: avoid double slashes in symlink targets
  git-difftool(1): fix formatting of --symlink description
Merge branch 'ap/combine-diff-ignore-whitespace' into jch

Teach "diff --cc" output to honor options to ignore various forms
of whitespace changes.

* ap/combine-diff-ignore-whitespace:
  Allow combined diff to ignore white-spaces
Merge branch 'rs/archive-zip-raw-compression' into jch

* rs/archive-zip-raw-compression:
  archive-zip: use deflateInit2() to ask for raw compressed data
Merge branch 'rr/tests-dedup-test-config' into jch

* rr/tests-dedup-test-config:
  t4018,7810,7811: remove test_config() redefinition

# Conflicts:
# 	t/t7810-grep.sh
Merge branch 'rr/test-3200-style' into jch

Churns.

* rr/test-3200-style:
  t3200 (branch): modernize style

# Conflicts:
# 	t/t3200-branch.sh
Merge branch 'sr/am-show-final-message-in-applying-indicator' into jch

In addition to the case where the user edits the log message with
the "e)dit" option of "am -i", replace the "Applying: this patch"
message with the final log message contents after applymsg hook
munges it.

* sr/am-show-final-message-in-applying-indicator:
  git-am: show the final log message on "Applying:" indicator
Merge branch 'yd/doc-merge-annotated-tag' into jch

Document the 1.7.9 feature to merge a signed tag and keep that in
the mergetag header in the resulting commit better.

* yd/doc-merge-annotated-tag:
  Documentation: merging a tag is a special case
Merge branch 'yd/doc-is-in-asciidoc' into jch

* yd/doc-is-in-asciidoc:
  CodingGuidelines: our documents are in AsciiDoc
Merge branch 'kk/revwalk-slop-too-many-commit-within-a-second' into jch

Allow the revision "slop" code to look deeper while commits with
exactly the same timestamps come next to each other (which can
often happen after a large "am" and "rebase" session).

* kk/revwalk-slop-too-many-commit-within-a-second:
  Fix revision walk for commits with the same dates
Merge branch 'jk/no-more-self-assignment' into jch

This started as a topic to reduce "type var = var" self assignment
tricks that were used to squelch "variable used uninitialized perhaps?"
warning from some compilers, but resulted in rewriting logic with
a version that is simpler and easier to understand for humans.

* jk/no-more-self-assignment:
  match-trees: simplify score_trees() using tree_entry()
  submodule: clarify logic in show_submodule_summary
Merge branch 'bk/document-commit-tree-S' into jch

* bk/document-commit-tree-S:
  commit-tree: document -S option consistently
### match next
Merge branch 'jc/remove-treesame-parent-in-simplify-merges' into jch

The --simplify-merges logic did not cull irrelevant parents from a
merge that is otherwise not interesting with respect to the paths
we are following.

This touches a fairly core part of the revision traversal
infrastructure; even though I think this change is correct, please
report immediately if you find any unintended side effect.

* jc/remove-treesame-parent-in-simplify-merges:
  simplify-merges: drop merge from irrelevant side branch

# Conflicts:
# 	revision.c
Merge branch 'bc/append-signed-off-by' into jch

Consolidates codepaths that inspect log-message-to-be and decide to
add a new Signed-off-by line in various commands.  I think there is
no negative behaviour change in this series, but please report any
iffy behaviour change immediately if you notice one.

* bc/append-signed-off-by:
  git-commit: populate the edit buffer with 2 blank lines before s-o-b
  Unify appending signoff in format-patch, commit and sequencer
  format-patch: update append_signoff prototype
  t4014: more tests about appending s-o-b lines
  sequencer.c: teach append_signoff to avoid adding a duplicate newline
  sequencer.c: teach append_signoff how to detect duplicate s-o-b
  sequencer.c: always separate "(cherry picked from" from commit body
  sequencer.c: require a conforming footer to be preceded by a blank line
  sequencer.c: recognize "(cherry picked from ..." as part of s-o-b footer
  t/t3511: add some tests of 'cherry-pick -s' functionality
  t/test-lib-functions.sh: allow to specify the tag name to test_commit
  commit, cherry-pick -s: remove broken support for multiline rfc2822 fields
  sequencer.c: rework search for start of footer to improve clarity

# Conflicts:
# 	builtin/log.c
# 	t/t4014-format-patch.sh