git-count-objects.txt: describe each line in -v output

The current description requires a bit of guessing (what clause
corresponds to what printed line?) and lacks information, such as
the unit of size and size-pack.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1_file: reorder code in prepare_packed_git_one()

The current loop does

	while (...) {
		if (it is not an .idx file)
			continue;
		process .idx file;
	}

and is reordered to

	while (...) {
		if (it is an .idx file) {
			process .idx file;
		}
	}

This makes it easier to add new extension file processing.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
count-objects: report garbage files in pack directory too

prepare_packed_git_one() is modified to allow count-objects to hook a
report function to so we don't need to duplicate the pack searching
logic in count-objects.c. When report_pack_garbage is NULL, the
overhead is insignificant.

The garbage is reported with warning() instead of error() in packed
garbage case because it's not an error to have garbage. Loose garbage
is still reported as errors and will be converted to warnings later.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
count-objects: report how much disk space taken by garbage files

Also issue warnings on loose garbages instead of errors as a result of
using report_garbage() function in count_objects()

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
read_directory: avoid invoking exclude machinery on tracked files

read_directory() (and its friendly wrapper fill_directory) collects
untracked/ignored files by traversing through the whole worktree,
feeding every entry to treat_one_path(), where each entry is checked
against .gitignore patterns.

One may see that tracked files can't be excluded and we do not need to
run them through exclude machinery. On repos where there are many
.gitignore patterns and/or a lot of tracked files, this unnecessary
processing can become expensive.

This patch avoids it mostly for normal cases. Directories are still
processed as before. DIR_SHOW_IGNORED and DIR_COLLECT_IGNORED are not
normally used unless some options are given (e.g. "checkout
--overwrite-ignore", "add -f"...)

treat_one_path's behavior changes when taking this shortcut. With
current code, when a non-directory path is not excluded,
treat_one_path calls treat_file, which returns the initial value of
exclude_file and causes treat_one_path to return path_handled. With
this patch, on the same conditions, treat_one_path returns
path_ignored.

read_directory_recursive() cares about this difference. Check out the
snippet:

	while (...) {
		switch (treat_path(...)) {
		case path_ignored:
			continue;
		case path_handled:
			break;
		}
		contents++;
		if (check_only)
			break;
		dir_add_name(dir, path.buf, path.len);
	}

If path_handled is returned, contents goes up. And if check_only is
true, the loop could be broken early. These will not happen when
treat_one_path (and its wrapper treat_path) returns
path_ignored. dir_add_name internally does a cache_name_exists() check
so it makes no difference.

To avoid this behavior change, treat_one_path is instructed to skip
the optimization when check_only or contents is used.

Finally some numbers (best of 20 runs) that shows why it's worth all
the hassle:

git status   | webkit linux-2.6 libreoffice-core gentoo-x86
-------------+----------------------------------------------
before       | 1.097s    0.208s           0.399s     0.539s
after        | 0.736s    0.159s           0.248s     0.501s
nr. patterns |    89       376               19          0
nr. tracked  |   182k       40k              63k       101k

Tracked-down-by: Karsten Blees <karsten.blees@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sync with 1.8.2-rc0
Merge branch 'mm/config-intro-in-git-doc' into next

* mm/config-intro-in-git-doc:
  git.txt: update description of the configuration mechanism
Merge branch 'jc/mention-tracking-for-pull-default' into next

We stopped mentioning `tracking` is a deprecated but supported
synonym for `upstream` in pull.default even though we have no
intention of removing the support for it.

* jc/mention-tracking-for-pull-default:
  doc: mention tracking for pull.default
Merge branch 'nd/count-garbage' into next

* nd/count-garbage:
  count-objects: report how much disk space taken by garbage files
  count-objects: report garbage files in pack directory too
  sha1_file: reorder code in prepare_packed_git_one()
  git-count-objects.txt: describe each line in -v output
Merge branch 'nd/read-directory-recursive-optim' into next

"git status" has been optimized by taking advantage of the fact
that paths that are already known to the index do not have to be
checked against the .gitignore mechanism under some conditions.

* nd/read-directory-recursive-optim:
  read_directory: avoid invoking exclude machinery on tracked files
Merge branch 'master' into next

* master:
  user-manual: use -o latest.tar.gz to create a gzipped tarball
  user-manual: use 'git config --global user.*' for setup
  user-manual: mention 'git remote add' for remote branch config
  user-manual: give 'git push -f' as an alternative to +master
  user-manual: use 'remote add' to setup push URLs
  l10n: zh_CN.po: translate 35 new messages
  l10n: vi.po: update new strings (2004t0u0f)
  l10n: Update git.pot (35 new, 14 removed messages)
Merge branch 'master' into next

* master:
  RelNotes 1.8.2: push-simple will not be in effect in this release
  shell-prompt: clean up nested if-then
Merge branch 'jk/blame-tree' into HEAD

* jk/blame-tree:
  add blame-tree command
  combine-diff: zero memory used for callback filepairs
  teach tree-diff a max-depth parameter
  within_depth: fix return for empty path
Merge branch 'jk/builtin-attr' into HEAD

* jk/builtin-attr:
  attr: map builtin userdiff drivers to well-known extensions
Merge branch 'jk/bundle-fetch' into HEAD

* jk/bundle-fetch:
  clone: give advice on how to resume a failed clone
  remote-curl: resume interrupted bundle transfers
  remote-curl: show progress for bundle downloads
  progress: allow pure-throughput progress meters
  remote-curl: try base $URL after $URL/info/refs
  remote-curl: auto-detect bundles when fetching refs
  remote-curl: free "discovery" object
  bundle: add is_bundle_buf helper
  transport: factor out bundle to ref list conversion
  remote-curl: use http callback for requesting refs
  http: add a public function for arbitrary-callback request
  http: refactor http_request function
  http: turn off curl signals
  t/lib-httpd: check for NO_CURL
Merge branch 'jk/cache-patch-id' into HEAD

* jk/cache-patch-id:
  cache patch ids on disk
  cherry: read default config
  implement metadata cache subsystem
  map: implement persistent maps
  decorate: use "map" for the underlying implementation
  fast-export: use object to uint32 map instead of "decorate"
  map: add helper functions for objects as keys
  implement generic key/value map
Merge branch 'jk/child-cleanup' into HEAD

* jk/child-cleanup:
  send-pack: kill pack-objects helper on signal or exit
Merge branch 'jk/clone-file-local' into HEAD

* jk/clone-file-local:
  clone: send diagnostic messages to stderr
Merge branch 'jk/completion-quoting' into HEAD

* jk/completion-quoting:
  don't use quotemeta
  super speed with sed and perl
  completion: quote completions we find
  completion: improve shell expansion of items
Merge branch 'jk/config-sources' into HEAD

* jk/config-sources:
  show sources of config options (WIP)