refs: write packed_refs file using stdio

We write each line of a new packed-refs file individually
using a write() syscall (and sometimes 2, if the ref is
peeled). Since each line is only about 50-100 bytes long,
this creates a lot of system call overhead.

We can instead open a stdio handle around our descriptor and
use fprintf to write to it. The extra buffering is not a
problem for us, because nobody will read our new packed-refs
file until we call commit_lock_file (by which point we have
flushed everything).

On a pathological repository with 8.5 million refs, this
dropped the time to run `git pack-refs` from 20s to 6s.

Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'ta/config-set-1'

Use the new caching config-set API in git_config() calls.

* ta/config-set-1:
  add tests for `git_config_get_string_const()`
  add a test for semantic errors in config files
  rewrite git_config() to use the config-set API
  config: add `git_die_config()` to the config-set API
  change `git_config()` return value to void
  add line number and file name info to `config_set`
  config.c: fix accuracy of line number in errors
  config.c: mark error and warnings strings for translation
Merge branch 'ta/config-set-2'

Update git_config() users with callback functions for a very narrow
scope with calls to config-set API that lets us query a single
variable.

* ta/config-set-2:
  builtin/apply.c: replace `git_config()` with `git_config_get_string_const()`
  merge-recursive.c: replace `git_config()` with `git_config_get_int()`
  ll-merge.c: refactor `read_merge_config()` to use `git_config_string()`
  fast-import.c: replace `git_config()` with `git_config_get_*()` family
  branch.c: replace `git_config()` with `git_config_get_string()
  alias.c: replace `git_config()` with `git_config_get_string()`
  imap-send.c: replace `git_config()` with `git_config_get_*()` family
  pager.c: replace `git_config()` with `git_config_get_value()`
  builtin/gc.c: replace `git_config()` with `git_config_get_*()` family
  rerere.c: replace `git_config()` with `git_config_get_*()` family
  fetchpack.c: replace `git_config()` with `git_config_get_*()` family
  archive.c: replace `git_config()` with `git_config_get_bool()` family
  read-cache.c: replace `git_config()` with `git_config_get_*()` family
  http-backend.c: replace `git_config()` with `git_config_get_bool()` family
  daemon.c: replace `git_config()` with `git_config_get_bool()` family
Merge branch 'jk/contrib-subtree-make-all'

* jk/contrib-subtree-make-all:
  subtree: make "all" default target of Makefile
Merge branch 'rs/child-process-init'

Code clean-up.

* rs/child-process-init:
  run-command: inline prepare_run_command_v_opt()
  run-command: call run_command_v_opt_cd_env() instead of duplicating it
  run-command: introduce child_process_init()
  run-command: introduce CHILD_PROCESS_INIT
Merge branch 'br/http-init-fix'

Code clean-up.

* br/http-init-fix:
  http: style fixes for curl_multi_init error check
  http.c: die if curl_*_init fails
Merge branch 'et/spell-poll-infinite-with-minus-one-only'

We used to pass -1000 to poll(2), expecting it to also mean "no
timeout", which should be spelled as -1.

* et/spell-poll-infinite-with-minus-one-only:
  upload-pack: keep poll(2)'s timeout to -1
Merge branch 'jk/make-simplify-dependencies'

Admit that keeping LIB_H up-to-date, only for those that do not use
the automatically generated dependencies, is a losing battle, and
make it conservative by making everything depend on anything.

* jk/make-simplify-dependencies:
  Makefile: drop CHECK_HEADER_DEPENDENCIES code
  Makefile: use `find` to determine static header dependencies
  i18n: treat "make pot" as an explicitly-invoked target
Merge branch 'mm/discourage-commit-a-to-finish-conflict-resolution'

* mm/discourage-commit-a-to-finish-conflict-resolution:
  merge, pull: stop advising 'commit -a' in case of conflict
Merge branch 'nd/mv-code-cleaning'

Code clean-up.

* nd/mv-code-cleaning:
  mv: no SP between function name and the first opening parenthese
  mv: combine two if(s)
  mv: unindent one level for directory move code
  mv: move index search code out
  mv: remove an "if" that's always true
  mv: split submodule move preparation code out
  mv: flatten error handling code block
  mv: mark strings for translations
Merge branch 'rs/ref-transaction-1'

The second batch of the transactional ref update series.

* rs/ref-transaction-1: (22 commits)
  update-ref --stdin: pass transaction around explicitly
  update-ref --stdin: narrow scope of err strbuf
  refs.c: make delete_ref use a transaction
  refs.c: make prune_ref use a transaction to delete the ref
  refs.c: remove lock_ref_sha1
  refs.c: remove the update_ref_write function
  refs.c: remove the update_ref_lock function
  refs.c: make lock_ref_sha1 static
  walker.c: use ref transaction for ref updates
  fast-import.c: use a ref transaction when dumping tags
  receive-pack.c: use a reference transaction for updating the refs
  refs.c: change update_ref to use a transaction
  branch.c: use ref transaction for all ref updates
  fast-import.c: change update_branch to use ref transactions
  sequencer.c: use ref transactions for all ref updates
  commit.c: use ref transactions for updates
  replace.c: use the ref transaction functions for updates
  tag.c: use ref transactions when doing updates
  refs.c: add transaction.status and track OPEN/CLOSED
  refs.c: make ref_transaction_begin take an err argument
  ...
Merge branch 'dt/cache-tree-repair'

Add a few more places in "commit" and "checkout" that make sure
that the cache-tree is fully populated in the index.

* dt/cache-tree-repair:
  cache-tree: do not try to use an invalidated subtree info to build a tree
  cache-tree: Write updated cache-tree after commit
  cache-tree: subdirectory tests
  test-dump-cache-tree: invalid trees are not errors
  cache-tree: create/update cache-tree on checkout
Merge branch 'nd/fetch-pass-quiet-to-gc-child-process'

Progress output from "git gc --auto" was visible in "git fetch -q".

* nd/fetch-pass-quiet-to-gc-child-process:
  fetch: silence git-gc if --quiet is given
  fetch: convert argv_gc_auto to struct argv_array
Merge branch 'nd/large-blobs'

Teach a few codepaths to punt (instead of dying) when large blobs
that would not fit in core are involved in the operation.

* nd/large-blobs:
  diff: shortcut for diff'ing two binary SHA-1 objects
  diff --stat: mark any file larger than core.bigfilethreshold binary
  diff.c: allow to pass more flags to diff_populate_filespec
  sha1_file.c: do not die failing to malloc in unpack_compressed_entry
  wrapper.c: introduce gentle xmallocz that does not die()
Merge branch 'jk/prune-top-level-refs-after-packing'

After "pack-refs --prune" packed refs at the top-level, it failed
to prune them.

* jk/prune-top-level-refs-after-packing:
  pack-refs: prune top-level refs like "refs/foo"
Merge branch 'jk/fast-import-fixes'

With sufficiently long refnames, fast-import could have overflown
an on-stack buffer.

* jk/fast-import-fixes:
  fast-import: fix buffer overflow in dump_tags
  fast-import: clean up pack_data pointer in end_packfile
Merge branch 'rs/list-optim'

Fix a couple of "accumulate into a sorted list" to "accumulate and
then sort the list".

* rs/list-optim:
  walker: avoid quadratic list insertion in mark_complete
  sha1_name: avoid quadratic list insertion in handle_one_ref
Merge branch 'jn/unpack-trees-checkout-m-carry-deletion'

"git checkout -m" did not switch to another branch while carrying
the local changes forward when a path was deleted from the index.

* jn/unpack-trees-checkout-m-carry-deletion:
  checkout -m: attempt merge when deletion of path was staged
  unpack-trees: use 'cuddled' style for if-else cascade
  unpack-trees: simplify 'all other failures' case
Merge branch 'jk/name-decoration-alloc'

The API to allocate the structure to keep track of commit
decoration was cumbersome to use, inviting lazy code to
overallocate memory.

* jk/name-decoration-alloc:
  log-tree: use FLEX_ARRAY in name_decoration
  log-tree: make name_decoration hash static
  log-tree: make add_name_decoration a public function
Merge branch 'br/imap-send-simplify-tunnel-child-process'

Code clean-up.

* br/imap-send-simplify-tunnel-child-process:
  imap-send: simplify v_issue_imap_cmd() and get_cmd_result() using starts_with()
  imap-send.c: imap_folder -> imap_server_conf.folder
  git-imap-send: simplify tunnel construction