Fix git_commit_dates() to handle (obvious) duplicates

Since git show will suppress duplicate results, change
git_commit_dates() to detect that situation and repair the damage.

Though currently, it can only handle cases where the duplicates are
obvious, i.e. have the same exact string in the argument list.  It
won't be able to handle cases where two arguments differ, but resolve
to the same underlying hash.  For our current purposes, that should be
fine since we only pass in hashes.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move VFS cp() to git.py; handle repodir changes

Move the VFS's cp() to git.py and reset the cp() CatPipe whenever the
global repodir changes.

Previously, if two different lib/bup/t/ tests (for example) needed to
use two different repositories, and both happened to indirectly call
cp(), the second test would end up with a CatPipe() connected to the
wrong repository.

In the longer run, we may want to consider further cleanup here, but
this should fix the immediate problem without too much risk.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Use CatPipe, not show, in git_commit_dates()

Import the commit parser from the experimental bup-get branch and use
it along with CatPipe to produce the git commit dates in
git_commit_dates().

This appears to resolve the performance problem with real archives
that was introduced by the use of "git show -s --pretty=format:%ct
..." (cf. 00ba9fb811e71bb6182b9379461bc6b493e3b7a4), and is still much
faster than 0.25 for at least a 1000 branch synthetic repository here.

Thanks to Gabriel Filion <gabster@lelutin.ca> for reporting the
problem, and to him and Patrick Rouleau <prouleau72@gmail.com> for
helping test the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't call _mksubs() in VFS Node nlink()

The VFS Node nlink() code calls _mksubs(), but then just returns 1.
It looks like that may be unnecessary work, so stop doing it.

Thanks to William Stein <wstein@gmail.com> for pointing it out.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
midx: close the mmap before erasing an midx file

Linux allows erasing a file even if it is still memory mapped, but
Windows does not. However, Cygwin tries to emulate Linux and, under
certain conditions, it silently moves the file to $RECYCLED.BIN/, but
not in a way where Windows takes care of it when we use the "empty the
recycled bin" command. The erased midx ends up comsuming more and more
disk space.

To solve this, we have to close the midx's mmap before erasing
the midx.

Cygwin users: you can use this command to clean up your
$RECYCLE.BIN/ directories:
find /cygdrive/?/\$RECYCLE.BIN/ -name "\.???[[:xdigit:]]*" -print -delete

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Keep PackMidx pointers until finished with mmap

Now that PackMidx objects close their mmap, don't drop the pointers to
them in midx-cmd.py until finished with the mmaps.

The isinstance() call may need revisiting later.  i.e. should all
indexes support close()?

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Merge fixes for orphaned midx files on Cygwin

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
t/force-delete: bash is not always at /bin/bash

Signed-off-by: Brandon Smith <freedom@reardencode.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
US/Central is not a common timezone, use America/Chicago

Signed-off-by: Brandon Smith <freedom@reardencode.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
import-rdiff-backup-cmd: invoke mktemp more portably

This should fix a problem on NetBSD 6, reported by Greg Troxel
<gdt@lexort.com>.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Greg Troxel <gdt@lexort.com>
Tested-by: Greg Troxel <gdt@lexort.com>
git.py: fix minor typos in comments

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit summary]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
mangle_path: verify chunked name refers to git dir

Before returning the mangled name.bup for a chunked item, ensure that
the name really does refer to a dir in the git tree via asssert.

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Makefile: add missing t/test-save-creates-no-unrefs.sh

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit summary]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
fuse-cmd.py: given --meta, report original metadata

Add a new --meta option to "bup fuse", and when specified, report the
original mode, uid, gid, atime, mtime, and ctime for the mounted
paths.

Since negative timestamps cause access errors, and cause "ls -l" to
report question marks for the stat values, set a floor of 0 for all
timestamps reported to python-fuse.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
web-cmd.py: add --browser option to open the site

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
web-cmd.py: open browser from ioloop callback

Assuming the callback won't be called until tornado is ready to
receive requests.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add --graft and --regraft support to bup-index and bup-save

Support for highly graftable indexes to enable easy use of bup with
relocateable roots (like snapshots with different paths), or directory
trees with subtle differences (i.e. when backing up home directories
across Windows and Linux)

The principle effect of this branch is to repurpose the metadata cache
objects path field during indexing to store the real path of any given
entry in the bupindex. The bupindex represents what the final bup
archive will look like.
Add documentation for --graft, --regraft commands to bup-index manpage
Add --rename and --delete options to bup-index

--rename allows a user or application invoking bup-index to explicitly
mark files as being renamed to another, preserving the original hash
value. This is useful for inotify driven applications watching for
rename operations.

--delete allows a user or application invoking bup-index to explicitly
mark files as being deleted on the disk, without needing to run a full
bup-index update operation. This is useful for inotify driven
applications watching for unlink operations.
Add --rename and --delete documentation to bup-index.md