Change index; include atime; pack time as xstat timespec; use ns in memory.

Update the index format header to 'BUPI\0\0\0\4' (version 4).

Change the index to include atimes, and write all times to bupindex as
xstat timespecs.  This is in preparation for indexing all metadata.
After moving all of the times to the index, there should be far fewer
unique instances of the remaining metadata in many/most cases.

Change the index and index-related code to handle all time
values as integer nanoseconds since the epoch, excepting the packed
format, which (as mentioned above) is now an xstat timespec.

After these changes the index-related in-memory and on-disk time value
handling should match that of the metadata code (i.e. metadata.py,
.bupm, etc.).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: change 10e8 to 10**9 for ns per second.]
Document accommodations for filesystems with low-resolution timestamps.

...and some effects thereof.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Store metadata in the index, in bupindex.meta; only store unique values.

See DESIGN for more information.

Update the index format header to 'BUPI\0\0\0\5' (version 5).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Add the filename/fd and invalid value to stat_struc_to_py error messages.

Thanks to Nathan Bird <ecthellion@gmail.com> for the initial report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Add "bup index --clear" to clear the index.

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
bloom-cmd: don't explicitly create the bloom file (allow bloom.create()).

Don't explicitly create the bloom file since bloom.create() will
handle it, and will actually use the correct (binary) mode instead of
"w+".

The binary mode has no effect under Linux or Cygwin, but is important
under Windows.

Signed-off-by: Patrick Rouleau <prouleau72@gmail.com>
Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
clear_index(): remove unused "cleared" variable.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add 'x' to the getopts call in t/compare-trees (i.e. allow -x).

Signed-off-by: Anton Eliasson <devel@antoneliasson.se>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Attempt to fix HACKING link in README.md.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Update HACKING to reflect the glorious death of tmp/pending/meta.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Officially drop support for Python 2.4.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Alexander Barton <alex@barton.de>
Mention what's "in-scope" for 0.25 in HACKING.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
wvtestrun: invoke perl via /usr/bin/env for portability.

This also matches format-subst.pl.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
bup-save.md: fix --graft documentation bug; use "chroot" consistently.

Change mixture of "chroot" and "chroots" in examples to "chroot".

Thanks to "jon.seymour" <jon.seymour@gmail.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Update bup-restore.md to match reality; document "/." handling.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move parse_excludes to helpers and rework it a bit.

Don't use f before it's initialized.

Narrow the catch to open(), and only catch IOException.

Call Options.fatal() when there's an error (as does
helpers.parse_date_or_fatal()).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix use of Python assert() with respect to optimization.

Fix a number of places where bup's assertions had material
side-effects, or where other code expected to see the AssertionError,
neither of which happen when optimization is enabled.

Reported-by: Jon Dowland <jmtd@debian.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a force-delete function to t/test.sh

Replace "rm -rf" with force-delete because "rm -rf" will proceed even
if it doesn't succeed.  This masked a problem (reported on Fedora)
where the root directory and other system directories had "r-xr-xr-x"
(0555) permissions, causing bup restore to create directories that "rm
-rf" couldn't remove.  As a result, subsequent tests would fail.

Signed-off-by: Ryan Brown <ryansb@csh.rit.edu>
[rlb@defaultvalue.org: edited commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
t/test.sh: remove a few calls to force-delete that shouldn't be needed.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move force-delete to t/lib.sh from test.sh and test-meta.sh; improve.

Only call chattr and setfacl on Linux for now.

Test that each item is actually gone.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>