builtin/add.c: simplify boolean variables

Do not to explicitly initialize static variables to 0 and instead
let BSS take care of it.  Also use OPT_BOOL() to let the command
line arguments set these variables to 0 or 1, instead of the
deprecated OPT_BOOLEAN() aka OPT_COUNTUP().

Signed-off-by: Junio C Hamano <gitster@pobox.com>
git add: start preparing for "git add <pathspec>..." to default to "-A"

When "git add subdir/" is run without "-u" or "-A" option, e.g.

    $ edit subdir/x
    $ create subdir/y
    $ rm subdir/z
    $ git add subdir/

the command does not notice removal of paths (e.g. subdir/z) from
the working tree.  This sometimes confuses new people, as arguably
"git add" is told to record the current state of "subdir/" as a
whole, not the current state of the paths that exist in the working
tree that matches that pathspec (the latter by definition excludes
the state of "subdir/z" because it does not exist in the working
tree).

Plan to eventually make "git add" pretend as if "-A" is given when
there is a pathspec on the command line.  When resolving a conflict
to remove a path, the current code tells you to "git rm $path", but
with such a change, you will be able to say "git add $path" (of
course you can do "git add -A $path" today).  That means that we can
simplify the advice messages given by "git status".  That all will
be in Git 2.0 or later, if we are going to do so.

For that transition to work, people need to learn either to say "git
add --no-all subdir/" when they want to ignore the removed paths
like "subdir/z", or to say "git add -A subdir/" when they want to
take the state of the directory as a whole.

"git add" without any argument will continue to be a no-op.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't drop ignored directories

'git-status --ignored' drops ignored directories if they contain untracked
files in an untracked sub directory.

Fix it by getting exact (recursive) excluded status in treat_directory.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't list files in ignored directories

'git-status --ignored' lists both the ignored directory and the ignored
files if the files are in a tracked sub directory.

When recursing into sub directories in read_directory_recursive, pass on
the check_only parameter so that we don't accidentally add the files.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't list empty ignored directories

'git-status --ignored' lists ignored tracked directories without any
ignored files if a tracked file happens to match an exclude pattern.

Always exclude tracked files.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status --ignored: don't list empty directories as ignored

'git-status --ignored' lists empty untracked directories as ignored, even
though they don't have any ignored files.

When checking if a directory is already listed as untracked (i.e. shouldn't
be listed as ignored as well), don't assume that the dirctory has only
ignored files if it doesn't have untracked files, as the directory may be
empty.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: move prep_exclude and factor out parts of last_exclude_matching

Move code around in preparation for the next patch.

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: unify is_excluded and is_path_excluded APIs

The is_excluded and is_path_excluded APIs are very similar, except for a
few noteworthy differences:

is_excluded doesn't handle ignored directories, results for paths within
ignored directories are incorrect. This is probably based on the premise
that recursive directory scans should stop at ignored directories, which
is no longer true (in certain cases, read_directory_recursive currently
calls is_excluded *and* is_path_excluded to get correct ignored state).

is_excluded caches parsed .gitignore files of the last directory in struct
dir_struct. If the directory changes, it finds a common parent directory
and is very careful to drop only as much state as necessary. On the other
hand, is_excluded will also read and parse .gitignore files in already
ignored directories, which are completely irrelevant.

is_path_excluded correctly handles ignored directories by checking if any
component in the path is excluded. As it uses is_excluded internally, this
unfortunately forces is_excluded to drop and re-read all .gitignore files,
as there is no common parent directory for the root dir.

is_path_excluded tracks state in a separate struct path_exclude_check,
which is essentially a wrapper of dir_struct with two more fields. However,
as is_path_excluded also modifies dir_struct, it is not possible to e.g.
use multiple path_exclude_check structures with the same dir_struct in
parallel. The additional structure just unnecessarily complicates the API.

Teach is_excluded / prep_exclude about ignored directories: whenever
entering a new directory, first check if the entire directory is excluded.
Remember the excluded state in dir_struct. Don't traverse into already
ignored directories (i.e. don't read irrelevant .gitignore files).

Directories could also be excluded by exclude patterns specified on the
command line or .git/info/exclude, so we cannot simply skip prep_exclude
entirely if there's no .gitignore file name (dir_struct.exclude_per_dir).
Move this check to just before acutally reading the file.

is_path_excluded is now equivalent to is_excluded, so we can simply
redirect to it (the public API is cleaned up in the next patch).

The performance impact of the additional ignored check per directory is
hardly noticeable when reading directories recursively (e.g. 'git status').
However, performance of git commands using the is_path_excluded API (e.g.
'git ls-files --cached --ignored --exclude-standard') is greatly improved
as this no longer re-reads .gitignore files on each call.

Here's some performance data from the linux and WebKit repos (best of 10
runs on a Debian Linux on SSD, core.preloadIndex=true):

       | ls-files -ci   |    status      | status --ignored
       | linux | WebKit | linux | WebKit | linux | WebKit
-------+-------+--------+-------+--------+-------+---------
before | 0.506 |  6.539 | 0.212 |  1.555 | 0.323 |  2.541
after  | 0.080 |  1.191 | 0.218 |  1.583 | 0.321 |  2.579
gain   | 6.325 |  5.490 | 0.972 |  0.982 | 1.006 |  0.985

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: replace is_path_excluded with now equivalent is_excluded API

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir.c: git-status: avoid is_excluded checks for tracked files

Checking if a file is in the index is much faster (hashtable lookup) than
checking if the file is excluded (linear search over exclude patterns).

Skip is_excluded checks for files: move the cache_name_exists check from
treat_file to treat_one_path and return early if the file is tracked.

This can safely be done as all other code paths also return path_ignored
for tracked files, and dir_add_ignored skips tracked files as well.

There's just one line left in treat_file, so move this to treat_one_path
as well.

Here's some performance data for git-status from the linux and WebKit
repos (best of 10 runs on a Debian Linux on SSD, core.preloadIndex=true):

       |    status      | status --ignored
       | linux | WebKit | linux | WebKit
-------+-------+--------+-------+---------
before | 0.218 |  1.583 | 0.321 |  2.579
after  | 0.156 |  0.988 | 0.202 |  1.279
gain   | 1.397 |  1.602 | 1.589 |  2.016

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'jc/fetch-raw-sha1' into next

Allows requests to fetch objects at any tip of refs (including
hidden ones).  It seems that there may be use cases even outside
Gerrit (e.g. $gmane/215701).

* jc/fetch-raw-sha1:
  fetch: fetch objects by their exact SHA-1 object names
  upload-pack: optionally allow fetching from the tips of hidden refs
  fetch: use struct ref to represent refs to be fetched
  parse_fetch_refspec(): clarify the codeflow a bit
Merge branch 'nd/count-garbage' into next

"git count-objects -v" did not count leftover temporary packfiles
and other kinds of garbage.

* nd/count-garbage:
  count-objects: report how much disk space taken by garbage files
  count-objects: report garbage files in pack directory too
  sha1_file: reorder code in prepare_packed_git_one()
  git-count-objects.txt: describe each line in -v output
Merge branch 'bc/commit-complete-lines-given-via-m-option' into next

'git commit -m "$str"' when $str was already terminated with a LF
now avoids adding an extra LF to the message.

* bc/commit-complete-lines-given-via-m-option:
  Documentation/git-commit.txt: rework the --cleanup section
  git-commit: only append a newline to -m mesg if necessary
  t7502: demonstrate breakage with a commit message with trailing newlines
  t/t7502: compare entire commit message with what was expected
Merge branch 'ob/imap-send-ssl-verify' into next

Correctly connect to SSL/TLS sites that serve multiple hostnames on
a single IP by including Server Name Indication in the client-hello.

* ob/imap-send-ssl-verify:
  imap-send: support Server Name Indication (RFC4366)
Merge branch 'da/difftool-fixes' into next

Minor maintenance updates to difftool, and updates to its tests.

* da/difftool-fixes:
  t7800: "defaults" is no longer a builtin tool name
  t7800: modernize tests
  t7800: update copyright notice
  difftool: silence uninitialized variable warning
Merge branch 'jc/perl-cat-blob' into next

perl/Git.pm::cat_blob slurped everything in core only to write it
out to a file descriptor, which was not a very smart thing to do.

* jc/perl-cat-blob:
  Git.pm: fix cat_blob crashes on large files
Merge branch 'ap/maint-update-index-h-is-for-help' into next

"git update-index -h" did not do the usual "-h(elp)" thing.

* ap/maint-update-index-h-is-for-help:
  update-index: allow "-h" to also display options
Merge branch 'nd/branch-error-cases' into next

"git branch" had more cases where it did not bother to check
nonsense command line parameters.

* nd/branch-error-cases:
  branch: segfault fixes and validation
Merge branch 'rt/commit-cleanup-config' into next

Fix tests that contaminated their environments and affected new
tests introduced later in the sequence by containing their effects
in their own subshells.

* rt/commit-cleanup-config:
  t7502: perform commits using alternate editor in a subshell
Merge branch 'mg/gpg-interface-using-status' into next

Call "gpg" using the right API when validating the signature on
tags.

* mg/gpg-interface-using-status:
  pretty: make %GK output the signing key for signed commits
  pretty: parse the gpg status lines rather than the output
  gpg_interface: allow to request status return
  log-tree: rely upon the check in the gpg_interface
  gpg-interface: check good signature in a reliable way