Don't crash; defer an error when a path changes while indexing it.

Mirror the error handling already used by save.

With this change the result for a path that changes or is deleted
between the drecurse traversal and the from_path() call should be the
same as the result for a path that changes or is deleted between index
and save.

Thanks to Yann Autissier <yann.autissier@gmail.com> for reporting the
problem and Gabriel Filion <gabster@lelutin.ca> for helping figure out
the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Makefile: simplify runtests-python since metadata tests aren't conditional.

Since the metadata tests are no longer conditional, revert
runtests-python to the simpler implementation.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Manually set TMPDIR to $(CURDIR)/tmp for all tests for now.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
metadata.py: fix variable access in Writer.add(); tmax -> self.tmax.

Thanks to Patrick Rouleau <prouleau72@gmail.com> for reporting the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
metadata.py: fix incorrectly indented line in Writer._add().

Thanks to Patrick Rouleau <prouleau72@gmail.com> for reporting the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Completely disable ACL support on NetBSD for now.

Signed-off-by: Thomas Klausner <tk@giga.or.at>
[rlb@defaultvalue.org: adjust commit message; wrap code.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
DESIGN: fix typo.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Remove redundant *bits assignment in bupsplit_find_ofs().

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Enforce MAX_PER_TREE by always _squish()ing in spit_to_shalist().

Previously bup would ignore MAX_PER_TREE whenever it hit a long run of
data that didn't produce a non-zero hashplit "level" (see the
discussion of fanout in DESIGN).  That can happen, for example, when
traversing a file containing large zero regions (sparse or not).

As a result, bup could produce an arbitrarily large number of blobs at
level 0 of the hashsplit tree, causing it to consume increasing memory
during split/save, and to behave badly during join/restore.

To fix that, don't try to outsmart _squish() -- just call it every
time, and let it enforce MAX_PER_TREE when appropriate.

Thanks to trebor <robert.rebstock@tempelhof-projekt.de> for reporting
the problem a long while back, Yung-Chin Oei <yungchin@yungchin.nl>
for tracking down the cause and proposing a slightly different fix,
and Aidan Hobson Sayers <aidanphs@gmail.com> for suggesting this
particular approach.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix off-by-one in MAX_PER_TREE guard in _squish().

Previously bup would allow MAX_PER_TREE + 1 items in a tree.  Change
the guard to ensure that trees really are limited to MAX_PER_TREE.

Thanks to Patrick Rouleau <prouleau72@gmail.com> for reporting the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't check ACLs in t/compare-trees on FreeBSD.

Treat FreeBSD like Cygwin and Darwin (OS X) in t/compare-trees.  This
should finish the process of disabling POSIX ACL support on FreeBSD
that was started here: 349ff15c7db09883abc20bdf3e8a4df6bff12cd3.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Revert "Don't check ACLs in t/compare-trees on FreeBSD."

This reverts commit 985b17f45707fe3759ca13f005625934f5f63b4c.

That should have been *NetBSD*.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't check ACLs in t/compare-trees on NetBSD.

Treat NetBSD like Cygwin and Darwin (OS X) in t/compare-trees.  This
should finish the process of disabling POSIX ACL support on NetBSD
that was started here: 349ff15c7db09883abc20bdf3e8a4df6bff12cd3.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Use the correct types when calling FS_IOC_GETFLAGS/FS_IOC_SETFLAGS.

Despite the definitions in <linux/fs.h> these ioctls take int
arguments, not longs. This is important for 64-bit big endian
machines.

See http://marc.info/?l=linux-fsdevel&m=138552482917220&w=2

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
[rlb@defaultvalue.org: adjust commit message; use "I" conversion
 to/from Python instead of "k".]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't compute local script_home in lib.sh realpath (depended on pwd).

Compute bup_t_lib_script_home once at load time so that it won't be
affected by changes to pwd, and use it in realpath.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Create t/force-delete from t/lib.sh force-delete and use it everywhere.

Move the t/lib.sh force-delete code to t/force-delete as a standalone
command, and use it everywhere -- particularly to delete t/tmp during
"make clean", given the recent move to run all tests inside t/ via
TMPDIR (18f838c9a66d521f9fa042eead3c5771ca99b03a).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rebasing fixups in index-cmd.py
Add atomic rename support to bup-index

This commit adds preliminary support to bup-index for atomically
renaming files in the index.

This allows an inotify or similar application to detect atomic rename
operations, and request the index be updated to reflect the new file
location without a full rehash of the file.

Files copied moved into the directory tree are detected and added by the
normal update_index() function.
Add atomic unlink support to bup-index.

This allows an index-only delete operation to be requested from the
bup-index command line. This is intended to be used with file watcher
daemons which will have already confirmed the deleted file operation.
Remove t/ from the test-meta.sh test tree now that it contains t/tmp/.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>