add `config_set` API for caching config-like files

Currently `git_config()` uses a callback mechanism and file rereads for
config values. Due to this approach, it is not uncommon for the config
files to be parsed several times during the run of a git program, with
different callbacks picking out different variables useful to themselves.

Add a `config_set`, that can be used to construct an in-memory cache for
config-like files that the caller specifies (i.e., files like `.gitmodules`,
`~/.gitconfig` etc.). Add two external functions `git_configset_get_value`
and `git_configset_get_value_multi` for querying from the config sets.
`git_configset_get_value` follows `last one wins` semantic (i.e. if there
are multiple matches for the queried key in the files of the configset the
value returned will be the last entry in `value_list`).
`git_configset_get_value_multi` returns a list of values sorted in order of
increasing priority (i.e. last match will be at the end of the list). Add
type specific query functions like `git_configset_get_bool` and similar.

Add a default `config_set`, `the_config_set` to cache all key-value pairs
read from usual config files (repo specific .git/config, user wide
~/.gitconfig, XDG config and the global /etc/gitconfig). `the_config_set`
is populated using `git_config()`.

Add two external functions `git_config_get_value` and
`git_config_get_value_multi` for querying in a non-callback manner from
`the_config_set`. Also, add type specific query functions that are
implemented as a thin wrapper around the `config_set` API.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-config: add tests for the config_set API

Expose the `config_set` C API as a set of simple commands in order to
facilitate testing. Add tests for the `config_set` API as well as for
`git_config_get_*()` family for the usual config files.

Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Tanay Abhra <tanayabh@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'hv/submodule-config' into jch

* hv/submodule-config:
  do not die on error of parsing fetchrecursesubmodules option
  use new config API for worktree configurations of submodules
  extract functions for submodule config set and lookup
  implement submodule config cache for lookup of submodule names
Merge branch 'jc/reopen-lock-file' into jch

Needed in the "commit -p" code path to update the cache tree in the
index.

* jc/reopen-lock-file:
  lockfile: allow reopening a closed but still locked file
Merge branch 'dt/cache-tree-repair' into jch

Add a few more places in "commit" and "checkout" that make sure
that the cache-tree is fully populated in the index.

* dt/cache-tree-repair:
  cache-tree: Write updated cache-tree after commit
  cache-tree: subdirectory tests
  test-dump-cache-tree: invalid trees are not errors
  cache-tree: create/update cache-tree on checkout

# Conflicts:
# 	builtin/checkout.c
# 	builtin/commit.c
Merge branch 'nd/multiple-work-trees' into jch

A replacement for contrib/workdir/git-new-workdir that does not
rely on symbolic links and make sharing of objects and refs safer
by making the borrowee and borrowers aware of each other.

* nd/multiple-work-trees: (31 commits)
  checkout: don't require a work tree when checking out into a new one
  git_path(): keep "info/sparse-checkout" per work-tree
  count-objects: report unused files in $GIT_DIR/repos/...
  gc: support prune --repos
  gc: factor out gc.pruneexpire parsing code
  gc: style change -- no SP before closing parenthesis
  prune: strategies for linked checkouts
  checkout: detach if the branch is already checked out elsewhere
  checkout: clean up half-prepared directories in --to mode
  checkout: support checking out into a new working directory
  use new wrapper write_file() for simple file writing
  wrapper.c: wrapper to open a file, fprintf then close
  setup.c: support multi-checkout repo setup
  setup.c: detect $GIT_COMMON_DIR check_repository_format_gently()
  setup.c: convert check_repository_format_gently to use strbuf
  setup.c: detect $GIT_COMMON_DIR in is_git_directory()
  setup.c: convert is_git_directory() to use strbuf
  git-stash: avoid hardcoding $GIT_DIR/logs/....
  *.sh: avoid hardcoding $GIT_DIR/hooks/...
  git-sh-setup.sh: use rev-parse --git-path to get $GIT_DIR/objects
  ...

# Conflicts:
# 	refs.c
# 	refs.h
# 	trace.c
Merge branch 'jk/stable-prio-queue' into jch

* jk/stable-prio-queue:
  t5539: update a flaky test
  paint_down_to_common: use prio_queue
  prio-queue: make output stable with respect to insertion
  prio-queue: factor out compare and swap operations
Merge branch 'cc/replace-graft' into jch

"git replace" learned a "--graft" option to rewrite parents of a
commit.

* cc/replace-graft:
  replace: add test for --graft with a mergetag
  replace: check mergetags when using --graft
  replace: add test for --graft with signed commit
  replace: remove signature when using --graft
  contrib: add convert-grafts-to-replace-refs.sh
  Documentation: replace: add --graft option
  replace: add test for --graft
  replace: add --graft option
  replace: cleanup redirection style in tests

# Conflicts:
# 	Documentation/git-replace.txt
Merge branch 'jk/rebase-am-fork-point' into jch

"git rebase --fork-point" did not filter out patch-identical
commits correctly.

* jk/rebase-am-fork-point:
  rebase: omit patch-identical commits with --fork-point
  rebase--am: use --cherry-pick instead of --ignore-if-in-upstream
### match next
Merge branch 'sk/mingw-uni-fix-more' into jch

Most of these are battle-tested in msysgit and are needed to
complete what has been merged to 'master' already.

A fix has been squashed into "Unicode environ (outgoing)"; is this
now ready to go?

* sk/mingw-uni-fix-more:
  Win32: enable color output in Windows cmd.exe
  Win32: patch Windows environment on startup
  Win32: keep the environment sorted
  Win32: use low-level memory allocation during initialization
  Win32: reduce environment array reallocations
  Win32: don't copy the environment twice when spawning child processes
  Win32: factor out environment block creation
  Win32: unify environment function names
  Win32: unify environment case-sensitivity
  Win32: fix environment memory leaks
  Win32: Unicode environment (incoming)
  Win32: Unicode environment (outgoing)
  Revert "Windows: teach getenv to do a case-sensitive search"
  tests: do not pass iso8859-1 encoded parameter
Merge branch 'sk/mingw-tests-workaround' into jch

Make tests pass on msysgit by mostly disabling ones that are
infeasible on that platform.

The t0110 one has been replaced; is this now ready to go?

* sk/mingw-tests-workaround:
  t800[12]: work around MSys limitation
  t9902: mingw-specific fix for gitfile link files
  t4210: skip command-line encoding tests on mingw
  MinGW: disable legacy encoding tests
  t0110/MinGW: skip tests that pass arbitrary bytes on the command line
  MinGW: Skip test redirecting to fd 4
Merge branch 'ta/config-set' into jch

Almost there.

* ta/config-set:
  test-config: add tests for the config_set API
  add `config_set` API for caching config-like files
Merge branch 'jc/test-lazy-prereq' into jch

Test-script clean-up.

Will hold.

* jc/test-lazy-prereq:
  tests: drop GIT_*_TIMING_TESTS environment variable support
Merge branch 'mt/patch-id-stable' into jch

Teaches "git patch-id" to compute the patch ID that does not change
when the files in a single patch is reordered. As this new algorithm
is backward incompatible, the last bit to flip it to be the default
is left out of 'master' for now.

Will hold.

* mt/patch-id-stable:
  patch-id: change default to stable
Merge branch 'cc/interpret-trailers' into jch

A new filter to programatically edit the tail end of the commit log
messages.

What is the status of this one?  I think I saw reviews by Michael
but after that I do not recall seeing any updates.

* cc/interpret-trailers:
  Documentation: add documentation for 'git interpret-trailers'
  trailer: add tests for commands in config file
  trailer: execute command from 'trailer.<name>.command'
  trailer: add tests for "git interpret-trailers"
  trailer: add interpret-trailers command
  trailer: put all the processing together and print
  trailer: parse trailers from file or stdin
  trailer: process command line trailer arguments
  trailer: read and process config information
  trailer: process trailers from input message and arguments
  trailer: add data structures and basic functions
Merge branch 'rs/ref-transaction-1' into jch

The second batch of the transactional ref update series.

* rs/ref-transaction-1:
  refs.c: make delete_ref use a transaction
  refs.c: make prune_ref use a transaction to delete the ref
  refs.c: remove lock_ref_sha1
  refs.c: remove the update_ref_write function
  refs.c: remove the update_ref_lock function
  refs.c: make lock_ref_sha1 static
  walker.c: use ref transaction for ref updates
  fast-import.c: use a ref transaction when dumping tags
  receive-pack.c: use a reference transaction for updating the refs
  refs.c: change update_ref to use a transaction
  branch.c: use ref transaction for all ref updates
  fast-import.c: change update_branch to use ref transactions
  sequencer.c: use ref transactions for all ref updates
  commit.c: use ref transactions for updates
  replace.c: use the ref transaction functions for updates
  tag.c: use ref transactions when doing updates
  refs.c: add transaction.status and track OPEN/CLOSED/ERROR
  refs.c: make ref_transaction_begin take an err argument
  refs.c: update ref_transaction_delete to check for error and return status
  refs.c: change ref_transaction_create to do error checking and return status

# Conflicts:
# 	sequencer.c
# 	walker.c
Merge branch 'rs/ref-transaction' into jch

* rs/ref-transaction:
  refs.c: fix handling of badly named refs
  refs.c: make write_ref_sha1 static
  fetch.c: change s_update_ref to use a ref transaction
  refs.c: propagate any errno==ENOTDIR from _commit back to the callers
  refs.c: pass a skip list to name_conflict_fn
  refs.c: call lock_ref_sha1_basic directly from commit
  refs.c: move the check for valid refname to lock_ref_sha1_basic
  refs.c: pass NULL as *flags to read_ref_full
  refs.c: pass the ref log message to _create/delete/update instead of _commit
  refs.c: add an err argument to delete_ref_loose
  wrapper.c: add a new function unlink_or_msg
  wrapper.c: simplify warn_if_unremovable

# Conflicts:
# 	builtin/log.c
# 	git-compat-util.h
# 	refs.c
# 	walker.c
Merge branch 'nd/path-max-is-better-than-hardcoded-magic-1024' into jch

Still being discussed.

* nd/path-max-is-better-than-hardcoded-magic-1024:
  abspath.c: use PATH_MAX in real_path_internal()
Merge branch 'cb/mergetool-difftool' into jch

* cb/mergetool-difftool:
  difftool: don't assume that default sh is sane
  mergetool: don't require a work tree for --tool-help