t9300: document fast-import empty path issues

When given an empty path, fast-import sometimes reports "missing"
instead of using the root tree object.  On top of this, for "ls" and
file copy (but not move) it dies with "Empty path component found in
input".

Document this behaviour with failing test cases.

Reported-by: Dave Abrahams <dave@boostpro.com>
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-import: set valid mode on root tree in "ls"

This prevents a failure later when we lift the restriction on ls with
the empty path.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-import: allow ls or filecopy of the root tree

Commit 178e1de (fast-import: don't allow 'ls' of path with empty
components, 2012-03-09) restricted paths which:

    . contain an empty directory component (e.g. foo//bar is invalid),
    . end with a directory separator (e.g. foo/ is invalid),
    . start with a directory separator (e.g. /foo is invalid).

However, the implementation also caught the empty path, which should
represent the root tree.  Relax this restriction so that the empty path
is explicitly allowed and refers to the root tree.

Reported-by: Dave Abrahams <dave@boostpro.com>
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast-import: allow moving the root tree

Because fast-import.c::tree_content_remove does not check for the empty
path, it is not possible to move the root tree to a subdirectory.
Instead the error "Path  not in branch" is produced (note the double
space where the empty path has been inserted).

Fix this by explicitly checking for the empty path and handling it.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git.pm: add new temp_is_locked function

The temp_is_locked function can be used to determine whether
or not a given name previously passed to temp_acquire is
currently locked.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-svn: allow git-svn fetching to work using serf

When attempting to git-svn fetch files from an svn https?: url using
the serf library (the only choice starting with svn 1.8) the following
errors can occur:

Temp file with moniker 'svn_delta' already in use at Git.pm line 1250
Temp file with moniker 'git_blob' already in use at Git.pm line 1250

David Rothenberger <daveroth@acm.org> has determined the cause to
be that ra_serf does not drive the delta editor in a depth-first
manner [...]. Instead, the calls come in this order:

1. open_root
2. open_directory
3. add_file
4. apply_textdelta
5. add_file
6. apply_textdelta

When using the ra_serf access method, git-svn can end up needing
to create several temp files before the first one is closed.

This change causes a new temp file moniker to be generated if the
one that would otherwise have been used is currently locked.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
check-ignore: the name of the character is NUL, not NULL

Signed-off-by: Junio C Hamano <gitster@pobox.com>
check-attr: the name of the character is NUL, not NULL

Signed-off-by: Junio C Hamano <gitster@pobox.com>
check-ignore -z: a single -z should apply to both input and output

Unless a command has separate --nul-terminated-{input,output}
options, the --nul-terminated-records (-z) option should apply
to both input and output for consistency.  The caller knows that its
input paths may need to be protected for LF, and the program shows
these problematic paths to its output.

The code already did the right thing.  Only the help text needs
fixing.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
check-attr -z: a single -z should apply to both input and output

Unless a command has separate --nul-terminated-{input,output}
options, the --nul-terminated-records (-z) option should apply
to both input and output for consistency.  The caller knows that its
input paths may need to be protected for LF, and the program shows
these problematic paths to its output.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git.pm: revert _temp_cache use of temp_is_locked

When the temp_is_locked function was introduced, there was
a desire to make _temp_cache use it.  Unfortunately due to the
various tests and logic flow involved changing the _temp_cache
function to use the new temp_is_locked function is problematic
as _temp_cache needs a slightly different test than is provided
by the temp_is_locked function.

This change reverts use of temp_is_locked in the _temp_cache
function and restores the original code that existed there
before the temp_is_locked function was added.

Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
editor: use canonicalized absolute path

By improving the relative_path() algorithm, e02ca72 (path.c:
refactor relative_path(), not only strip prefix, 2013-06-25)
uncovered a latent bug in Emacs.  While most editor applications
like cat and vim handle non-canonicalized relative paths fine, emacs
does not.  This is due to a long-standing bug in emacs, where it
refuses to resolve symlinks in the supplied path:

  #!/bin/sh
  cd /tmp
  mkdir z z/a z/b
  echo moodle >z/a/file
  ln -s z/b
  cd b
  emacs ../a/file # fail: attempts to open /tmp/a/file

Even if emacs were to be patched to fix this bug, it may be nicer to
help users running older versions.

Note that this can potentially regress for users of all editors,
when they ask "what file am I editing?" to the editor, as it is
likely to answer with an unsightly long full path.

Co-authored-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
for-each-ref, quote: convert *_quote_print -> *_quote_buf

The print_value() function in for-each-ref.c prints values to stdout
immediately using {sq|perl|python|tcl}_quote_print().  Change these
lower-level quote functions to instead leave their results in strbuf
so that we can later add post-processing to the results of them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
tar-tree: remove dependency on sq_quote_print()

By rewriting the loop that formats the argv[] in cmd_tar_tree()
function using sq_quote_argv() for code simplicity, the last use of
sq_quote_print() goes away.

Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
quote: remove sq_quote_print()

Remove sq_quote_print() since it has no callers.

Signed-off-by: Ramkumar Ramachandra <artagnon@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
imap-send: use Apple's Security framework for base64 encoding

Use Apple's supported functions for base64 encoding instead
of the deprecated OpenSSL functions.

Signed-off-by: Jeremy Huddleston <jeremyhu@apple.com>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
parse-options: add OPT_CMDMODE()

This can be used to define a set of mutually exclusive "command
mode" options, and automatically catch use of more than one from
that set as an error.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
tag: use OPT_CMDMODE

This is just a demonstration of how the code would look like; I do
not think it is particularly easier to read than before myself.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
sha1_file: introduce close_one_pack() to close packs on fd pressure

When the number of open packs exceeds pack_max_fds, unuse_one_window()
is called repeatedly to attempt to release the least-recently-used
pack windows, which, as a side-effect, will also close a pack file
after closing its last open window.  If a pack file has been opened,
but no windows have been allocated into it, it will never be selected
by unuse_one_window() and hence its file descriptor will not be
closed.  When this happens, git may exceed the number of file
descriptors permitted by the system.

This latter situation can occur in show-ref or receive-pack during ref
advertisement.  During ref advertisement, receive-pack will iterate
over every ref in the repository and advertise it to the client after
ensuring that the ref exists in the local repository.  If the ref is
located inside a pack, then the pack is opened to ensure that it
exists, but since the object is not actually read from the pack, no
mmap windows are allocated.  When the number of open packs exceeds
pack_max_fds, unuse_one_window() will not be able to find any windows to
free and will not be able to close any packs.  Once the per-process
file descriptor limit is exceeded, receive-pack will produce a warning,
not an error, for each pack it cannot open, and will then most likely
fail with an error to spawn rev-list or index-pack like:

   error: cannot create standard input pipe for rev-list: Too many open files
   error: Could not run 'git rev-list'

This may also occur during upload-pack when refs are packed (in the
packed-refs file) and the number of packs that must be opened to
verify that these packed refs exist exceeds the file descriptor
limit.  If the refs are loose, then upload-pack will read each ref
from the object database (if the object is in a pack, allocating one
or more mmap windows for it) in order to peel tags and advertise the
underlying object.  But when the refs are packed and peeled,
upload-pack will use the peeled sha1 in the packed-refs file and
will not need to read from the pack files, so no mmap windows will
be allocated and just like with receive-pack, unuse_one_window()
will never select these opened packs to close.

When we have file descriptor pressure, we just need to find an open
pack to close.  We can leave the existing mmap windows open.  If
additional windows need to be mapped into the pack file, it will be
reopened when necessary.  If the pack file has been rewritten in the
mean time, open_packed_git_1() should notice when it compares the file
size or the pack's sha1 checksum to what was previously read from the
pack index, and reject it.

Let's introduce a new function close_one_pack() designed specifically
for this purpose to search for and close the least-recently-used pack,
where LRU is defined as (in order of preference):

   * pack with oldest mtime and no allocated mmap windows
   * pack with the least-recently-used windows, i.e. the pack
     with the oldest most-recently-used window, where none of
     the windows are in use
   * pack with the least-recently-used windows

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Don't close pack fd when free'ing pack windows

Now that close_one_pack() has been introduced to handle file
descriptor pressure, it is not strictly necessary to close the
pack file descriptor in unuse_one_window() when we're under memory
pressure.

Jeff King provided a justification for leaving the pack file open:

   If you close packfile descriptors, you can run into racy situations
   where somebody else is repacking and deleting packs, and they go away
   while you are trying to access them. If you keep a descriptor open,
   you're fine; they last to the end of the process. If you don't, then
   they disappear from under you.

   For normal object access, this isn't that big a deal; we just rescan
   the packs and retry. But if you are packing yourself (e.g., because
   you are a pack-objects started by upload-pack for a clone or fetch),
   it's much harder to recover (and we print some warnings).

Let's do so (or uh, not do so).

Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>