Provide some linguistic guidance for the documentation.

This will hopefully avoid questions over which spelling and grammar should
be used.  Translators are of course free to create localizations for
specific English dialects.

Signed-off-by: Marc Branchaud <marcnarc@xiplink.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1_file: introduce close_one_pack() to close packs on fd pressure

When the number of open packs exceeds pack_max_fds, unuse_one_window()
is called repeatedly to attempt to release the least-recently-used
pack windows, which, as a side-effect, will also close a pack file
after closing its last open window.  If a pack file has been opened,
but no windows have been allocated into it, it will never be selected
by unuse_one_window() and hence its file descriptor will not be
closed.  When this happens, git may exceed the number of file
descriptors permitted by the system.

This latter situation can occur in show-ref or receive-pack during ref
advertisement.  During ref advertisement, receive-pack will iterate
over every ref in the repository and advertise it to the client after
ensuring that the ref exists in the local repository.  If the ref is
located inside a pack, then the pack is opened to ensure that it
exists, but since the object is not actually read from the pack, no
mmap windows are allocated.  When the number of open packs exceeds
pack_max_fds, unuse_one_window() will not be able to find any windows to
free and will not be able to close any packs.  Once the per-process
file descriptor limit is exceeded, receive-pack will produce a warning,
not an error, for each pack it cannot open, and will then most likely
fail with an error to spawn rev-list or index-pack like:

   error: cannot create standard input pipe for rev-list: Too many open files
   error: Could not run 'git rev-list'

This may also occur during upload-pack when refs are packed (in the
packed-refs file) and the number of packs that must be opened to
verify that these packed refs exist exceeds the file descriptor
limit.  If the refs are loose, then upload-pack will read each ref
from the object database (if the object is in a pack, allocating one
or more mmap windows for it) in order to peel tags and advertise the
underlying object.  But when the refs are packed and peeled,
upload-pack will use the peeled sha1 in the packed-refs file and
will not need to read from the pack files, so no mmap windows will
be allocated and just like with receive-pack, unuse_one_window()
will never select these opened packs to close.

When we have file descriptor pressure, we just need to find an open
pack to close.  We can leave the existing mmap windows open.  If
additional windows need to be mapped into the pack file, it will be
reopened when necessary.  If the pack file has been rewritten in the
mean time, open_packed_git_1() should notice when it compares the file
size or the pack's sha1 checksum to what was previously read from the
pack index, and reject it.

Let's introduce a new function close_one_pack() designed specifically
for this purpose to search for and close the least-recently-used pack,
where LRU is defined as (in order of preference):

   * pack with oldest mtime and no allocated mmap windows
   * pack with the least-recently-used windows, i.e. the pack
     with the oldest most-recently-used window, where none of
     the windows are in use
   * pack with the least-recently-used windows

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't close pack fd when free'ing pack windows

Now that close_one_pack() has been introduced to handle file
descriptor pressure, it is not strictly necessary to close the
pack file descriptor in unuse_one_window() when we're under memory
pressure.

Jeff King provided a justification for leaving the pack file open:

   If you close packfile descriptors, you can run into racy situations
   where somebody else is repacking and deleting packs, and they go away
   while you are trying to access them. If you keep a descriptor open,
   you're fine; they last to the end of the process. If you don't, then
   they disappear from under you.

   For normal object access, this isn't that big a deal; we just rescan
   the packs and retry. But if you are packing yourself (e.g., because
   you are a pack-objects started by upload-pack for a clone or fetch),
   it's much harder to recover (and we print some warnings).

Let's do so (or uh, not do so).

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cat-file: only split on whitespace when %(rest) is used

Commit c334b87b (cat-file: split --batch input lines on whitespace,
2013-07-11) taught `cat-file --batch-check` to split input lines on
the first whitespace, and stash everything after the first token
into the %(rest) output format element.  It claimed:

   Object names cannot contain spaces, so any input with
   spaces would have resulted in a "missing" line.

But that is not correct.  Refs, object sha1s, and various peeling
suffixes cannot contain spaces, but some object names can. In
particular:

  1. Tree paths like "[<tree>]:path with whitespace"

  2. Reflog specifications like "@{2 days ago}"

  3. Commit searches like "rev^{/grep me}" or ":/grep me"

To remain backwards compatible, we cannot split on whitespace by
default, hence we will ship 1.8.4 with the commit reverted.

Resurrect its attempt but in a weaker form; only do the splitting
when "%(rest)" is used in the output format. Since that element did
not exist at all before c334b87, old scripts cannot be affected.

The existence of object names with spaces does mean that you
cannot reliably do:

  echo ":path with space and other data" |
  git cat-file --batch-check="%(objectname) %(rest)"

as it would split the path and feed only ":path" to get_sha1. But
that command is nonsensical. If you wanted to see "and other data"
in "%(rest)", git cannot possibly know where the filename ends and
the "rest" begins.

It might be more robust to have something like "-z" to separate the
input elements. But this patch is still a reasonable step before
having that.  It makes the easy cases easy; people who do not care
about %(rest) do not have to consider it, and the %(rest) code
handles the spaces and newlines of "rev-list --objects" correctly.

Hard cases remain hard but possible (if you might get whitespace in
your input, you do not get to use %(rest) and must split and join
the output yourself using more flexible tools). And most
importantly, it does not preclude us from having different splitting
rules later if a "-z" (or similar) option is added.  So we can make
the hard cases easier later, if we choose to.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t8001/t8002: blame: decompose overly-large test

Checking all bogus -L syntax forms in a single test makes it difficult
to identify the offender when one case fails. Decompose this
conglomerate test in order to check each bad syntax case separately.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t8001/t8002: blame: demonstrate -L bounds checking bug

A bounds checking bug allows the X in -LX to extend one line past the
end of file. For example, given a file with 5 lines, -L6 is accepted as
valid. Demonstrate this problem.

While here, also add tests to check that the remaining cases of X and Y
in -LX,Y are handled correctly at and in the vicinity of end-of-file.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t8001/t8002: blame: add empty file & partial-line tests

Add boundary case tests, with and without -L, for empty file; file with
one partial line; file with one full line.

The empty file test without -L is of particular interest. Historically,
this case has been supported (empty blame output) and this test protects
against regression by a subsequent patch fixing an off-by-one bug which
incorrectly accepts -LX where X is one past end-of-file.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: fix -L bounds checking bug

Since inception, -LX,Y has correctly reported an out-of-range error when
Y is beyond end of file, however, X was not checked, and an out-of-range
X would cause a crash.  92f9e273 (blame: prevent a segv when -L given
start > EOF; 2010-02-08) attempted to rectify this shortcoming but has
its own off-by-one error which allows X to extend one line past end of
file.  For example, given a file with 5 lines:

  git blame -L5 foo  # OK, blames line 5
  git blame -L6 foo  # accepted, no error, no output, huh?
  git blame -L7 foo  # error "fatal: file foo has only 5 lines"

Fix this bug.

In order to avoid regressing "blame foo" when foo is an empty file, the
fix is slightly more complicated than changing '<' to '<='.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4211: log: demonstrate -L bounds checking bug

A bounds checking bug allows the X in -LX to extend one line past the
end of file. For example, given a file with 5 lines, -L6 is accepted as
valid. Demonstrate this problem.

While here, also add tests to check that the remaining cases of X and Y
in -LX,Y are handled correctly at and in the vicinity of end-of-file.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t4211: retire soon-to-be unimplementable tests

58960978 and 99780b0a added tests which demonstrated bugs (crashes) in
range-set and line-log when handed empty ranges specified via "log
-LX:file" where X is one greater than the last line of the file.  After
these tests were added, it was realized that the ability to specify an
empty range is a loophole due to a bug in -L bounds checking. That bug
is slated to be fixed in a subsequent patch.

Unfortunately, the closure of this loophole makes it impossible to
continue checking range-set and line-log behavior with regard to empty
ranges since there is no other way to specify empty ranges via the
command-line.  APIs of both facilities are private (file static) so
there likewise is no way to test their behaviors programmatically.
Consequently, retire these two tests.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
log: fix -L bounds checking bug

When 12da1d1f added -L support to git-log, a broken bounds check was
copied from git-blame -L which incorrectly allows -LX to extend one line
past end of file without reporting an error.  Instead, it generates an
empty range.  Fix this bug.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t8001/t8002: blame: demonstrate acceptance of bogus -LX,+0 and -LX,-0

Empty ranges -LX,+0 and -LX,-0 are nonsensical in the context of blame
yet they are accepted. They should be errors. Demonstrate this
shortcoming.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: reject empty ranges -LX,+0 and -LX,-0

Empty ranges -LX,+0 and -LX,-0 are nonsensical in the context of blame
yet they are accepted (in fact, both are interpreted as -LX,+2).  Report
them as invalid.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t8001/t8002: blame: demonstrate acceptance of bogus -L,+0 and -L,-0

Empty ranges -L,+0 and -L,-0 are nonsensical in the context of blame yet
they are accepted. They should be errors. Demonstrate this shortcoming.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
blame: reject empty ranges -L,+0 and -L,-0

Empty ranges -L,+0 and -L,-0 are nonsensical in the context of blame yet
they are accepted (in fact, both are interpreted as -L1,Y where Y is
end-of-file). Report them as invalid.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'mb/docs-favor-en-us' into next

Declare that the official grammar & spelling of the source of this
project is en_US, but strongly discourage patches only to "fix"
existing en_UK strings to avoid unnecessary churns.

* mb/docs-favor-en-us:
  Provide some linguistic guidance for the documentation.
Merge branch 'bc/unuse-packfile' into next

Handle memory pressure and file descriptor pressure separately when
deciding to release pack windows to honor resource limits.

* bc/unuse-packfile:
  Don't close pack fd when free'ing pack windows
  sha1_file: introduce close_one_pack() to close packs on fd pressure
Merge branch 'es/blame-L-more' into next

More fixes to the code to parse the "-L" option in "log" and "blame".

* es/blame-L-more:
  blame: reject empty ranges -L,+0 and -L,-0
  t8001/t8002: blame: demonstrate acceptance of bogus -L,+0 and -L,-0
  blame: reject empty ranges -LX,+0 and -LX,-0
  t8001/t8002: blame: demonstrate acceptance of bogus -LX,+0 and -LX,-0
  log: fix -L bounds checking bug
  t4211: retire soon-to-be unimplementable tests
  t4211: log: demonstrate -L bounds checking bug
  blame: fix -L bounds checking bug
  t8001/t8002: blame: add empty file & partial-line tests
  t8001/t8002: blame: demonstrate -L bounds checking bug
  t8001/t8002: blame: decompose overly-large test
Merge branch 'jk/cat-file-batch-optim' into next

Rework the reverted change to `cat-file --batch-check`.

* jk/cat-file-batch-optim:
  cat-file: only split on whitespace when %(rest) is used
Merge branch 'reset-submodule-trailing-slash' into jk/pu

* branch reset-submodule-trailing-slash:
  WIP built on nd/magic-pathspecs

Waiting for nd/magic-pathspecs to graduate to master.