libgit2/libgit2 — pull request #1507 by arrbee (2013-04-26 23:17:31+00:00)
Title: Update diff handling of "untracked" directories

When diff scans the working tree and finds a directory, it considers it either tracked, ignored, or untracked, as so:

1. If there is a tracked item inside the directory, it is tracked and diff recurses into it
2. If the directory is ignored, then either (A) if GIT_DIFF_RECURSE_IGNORED_DIRS is set, recurse into it, or (B) otherwise create an IGNORED item for the directory
3. Otherwise the directory is not ignored, so either (A) if GIT_DIFF_RECURSE_UNTRACKED_DIRS is set, recurse into it, or (B) otherwise create an UNTRACKED item for the directory

It turns out that case 3B does not match core Git. Even if you know that there are no tracked items inside a directory, core Git will *always* scan it to see if everything under it is ignored or not. If all of the files under the directory are ignored or if there are no files under the directory, then core Git will consider the directory IGNORED (not UNTRACKED). This is relevant for the `--ignored` and `-u` options to diff, but has a larger impact on submodule status where an UNTRACKED item can mean that the whole submodule is considered dirty, but an IGNORED item won't.

This issue is at the root of the problems mentioned in issue #1486 (although there are some other oddities at play there as well).

This PR breaks up the code in `git_diff__from_iterators()` into three separate routines, because it was growing too large. The routines handle the three cases: an unmatched item on left of the diff, an unmatched item on the right of the diff, or a matched item in the diff. Then, the actual fixes for the above case are all contained in the function that handles an unmatched item on the right side of the diff (which is where all the complexity of dealing with a working directory is found).

There is a new diff flag `GIT_DIFF_FAST_UNTRACKED_DIRS` that preserves the old behavior and immediately creates a single UNTRACKED entry in the diff list when an untracked directory is found. This is potentially much faster (for example, if you have a deep build directory with thousands of `.o` files and you ignored `*.o` instead of ignoring the build directory itself) in the cases where you don't actually care about the contents of the untracked directory. But for compatibility with core Git, that flag is not the default and is not used by `git_status_foreach()`.

- [x] split long `git_diff__from_iterators()` routine
- [x] implement scanning of untracked directory content for non-ignored items
- [ ] tests for diff with various combinations of untracked and ignored items
- [x] tests for submodule status with various combinations of untracked and ignored items