midx: close the mmap before erasing an midx file

Linux allows erasing a file even if it is still memory mapped, but
Windows does not. However, Cygwin tries to emulate Linux and, under
certain conditions, it silently moves the file to $RECYCLED.BIN/, but
not in a way where Windows takes care of it when we use the "empty the
recycled bin" command. The erased midx ends up comsuming more and more
disk space.

To solve this, we have to close the midx's mmap before erasing
the midx.

Cygwin users: you can use this command to clean up your
$RECYCLE.BIN/ directories:
find /cygdrive/?/\$RECYCLE.BIN/ -name "\.???[[:xdigit:]]*" -print -delete

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Keep PackMidx pointers until finished with mmap

Now that PackMidx objects close their mmap, don't drop the pointers to
them in midx-cmd.py until finished with the mmaps.

The isinstance() call may need revisiting later.  i.e. should all
indexes support close()?

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Merge fixes for orphaned midx files on Cygwin

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
t/force-delete: bash is not always at /bin/bash

Signed-off-by: Brandon Smith <freedom@reardencode.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
US/Central is not a common timezone, use America/Chicago

Signed-off-by: Brandon Smith <freedom@reardencode.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
import-rdiff-backup-cmd: invoke mktemp more portably

This should fix a problem on NetBSD 6, reported by Greg Troxel
<gdt@lexort.com>.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Greg Troxel <gdt@lexort.com>
Tested-by: Greg Troxel <gdt@lexort.com>
git.py: fix minor typos in comments

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit summary]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
mangle_path: verify chunked name refers to git dir

Before returning the mangled name.bup for a chunked item, ensure that
the name really does refer to a dir in the git tree via asssert.

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Makefile: add missing t/test-save-creates-no-unrefs.sh

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit summary]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
fuse-cmd.py: given --meta, report original metadata

Add a new --meta option to "bup fuse", and when specified, report the
original mode, uid, gid, atime, mtime, and ctime for the mounted
paths.

Since negative timestamps cause access errors, and cause "ls -l" to
report question marks for the stat values, set a floor of 0 for all
timestamps reported to python-fuse.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
web-cmd.py: add --browser option to open the site

Signed-off-by: Holger Dell <github@holgerdell.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
web-cmd.py: open browser from ioloop callback

Assuming the callback won't be called until tornado is ready to
receive requests.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Avoid using incomplete utimensat implementations.

At least NetBSD 6 has partial support for utimensat, implementing the
function but not providing needed definitions via sys/stat.h as POSIX
requires.  Check for needed defines, and if missing, undefine
HAVE_UTIMENSAT.

Signed-Off-By: Greg Troxel <gdt@ir.bbn.com>
Tested-By: Greg Troxel <gdt@ir.bbn.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Initialize _file in MetaStoreReader before open

So that __del__() will work properly if open fails.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
save-cmd.py: don't test for meta index via access()

Instead just try to open the metadata index and catch any EACCES.
There may be an issue with access on Cygwin
(http://bugs.python.org/issue2528), and even if not, this approach
removes a potential race between access() and open().

Thanks to Mark J Hewitt <mjh@idnet.com> for reporting the problem and
helping track down the cause.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Ignore empty lines in --exclude-rx-from files

Thanks to Daniel Hahler Daniel Hahler <git@thequod.de> for suggesting
the change.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
parse_excludes: drop empty --exclude-from paths

Empty paths shouldn't match during drecurse, but there's no point in
including them.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
format-subst.pl: check for subcommand failure

Thanks to karl.richter722@gmail.com for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
test-meta.sh: use user's uid/gid, not 0

Cygwin does not fully support uid and gid of 0. Instead, use the
current user's ids to avoid portability problems. Since this test is
not run for ordinary users, the result is the same.

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Avoid uid/gid 0 metadata tests when ids don't exist

Cygwin may not have a 0 uid/gid, so skip the relevant tests whenever
it doesn't.

We have to explicitly exclude the gid 0 from other_group, because
/etc/group may define a group named root with the same id as
Administrators. We cannot use "root" instead of 0, because root may
not be defined.

Cygwin users: If you want to define the root group, add this line
at the begining of /etc/group:
  root:S-1-5-32-544:0:

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>