Mention what's "in-scope" for 0.25 in HACKING.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
wvtestrun: invoke perl via /usr/bin/env for portability.

This also matches format-subst.pl.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
bup-save.md: fix --graft documentation bug; use "chroot" consistently.

Change mixture of "chroot" and "chroots" in examples to "chroot".

Thanks to "jon.seymour" <jon.seymour@gmail.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Update bup-restore.md to match reality; document "/." handling.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move parse_excludes to helpers and rework it a bit.

Don't use f before it's initialized.

Narrow the catch to open(), and only catch IOException.

Call Options.fatal() when there's an error (as does
helpers.parse_date_or_fatal()).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix use of Python assert() with respect to optimization.

Fix a number of places where bup's assertions had material
side-effects, or where other code expected to see the AssertionError,
neither of which happen when optimization is enabled.

Reported-by: Jon Dowland <jmtd@debian.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a force-delete function to t/test.sh

Replace "rm -rf" with force-delete because "rm -rf" will proceed even
if it doesn't succeed.  This masked a problem (reported on Fedora)
where the root directory and other system directories had "r-xr-xr-x"
(0555) permissions, causing bup restore to create directories that "rm
-rf" couldn't remove.  As a result, subsequent tests would fail.

Signed-off-by: Ryan Brown <ryansb@csh.rit.edu>
[rlb@defaultvalue.org: edited commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
t/test.sh: remove a few calls to force-delete that shouldn't be needed.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move force-delete to t/lib.sh from test.sh and test-meta.sh; improve.

Only call chattr and setfacl on Linux for now.

Test that each item is actually gone.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Include metadata when asked to restore individual non-directory paths.

Adjust bup restore to handle cases like this:

  bup restore -C tmp backup/latest/somewhere/not-a-directory

Previously bup wouldn't restore the metadata for not-a-directory; now
it should.

Add wvtest-bup.sh which includes wvmktempdir().  This function creates
all temporary directories under t/tmp, making cleanup easier.

Add t/test-restore-single-file.sh

Reported-by: Tim Riemenschneider <t.riemenschneider@detco.de>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
find_dir_item_metadata_by_name(): wait for the right item name.

Fix a bug in the previous patch:

  Include metadata when asked to restore individual non-directory paths.
  53649c18ec3db90f80ff4657315009538d4333f3

Don't return whatever metadata you have the first time you hit a
directory in the dir.  Wait until you find the right name.

Reported-by: varacanero <varacanero@zeromail.org>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Normalize stat(2) timespecs in _helpers.c.

Previously, the code required timespecs to have a nanosecond value
between 0 and 999999999, because that's what Linux appeared to
produce, but other platforms don't (i.e. Cygwin) -- so normalize the
system values to match expectations.

In particular, look for negative nanosecond values (must be greater
than -10**9), and if found, rewrite the timespec with non-negative
nanoseconds (still less than 10**9).

Thanks to Will Rouesnel <w.rouesnel@gmail.com> for reporting the
problem and posting an initial patch.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
save-cmd.py: insist on having an index; check for the primary index file.

Insist on having an index, and report an error (with a suggestion to
run "bup index") if one isn't found.

Previously (at least as recently as 0.25-rc1), bup save would just
create an empty tree if there was no index.  The metadata work
unintentionally broke that behavior.

For now, treat a missing index as an error under the assumption that
it's not likely to be intentional.  If a good use case for the
previous behavior is presented, we can always restore it.

Signed-off-by: Jean-Baptiste Denis <jbd@jbdenis.net>
[rlb@defaultvalue.org: adjust commit message and save-cmd.py error message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a test to check that bup save without an index fails.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Use the correct C types when converting between C and Python in _helpers.c.

Previously, bup might fail when handling large repositories.  For
example:

    Traceback (most recent call last):
     File "/usr/lib/bup/cmd/bup-midx", line 258, in <module>
       do_midx_dir(path)
     File "/usr/lib/bup/cmd/bup-midx", line 199, in do_midx_dir
       all = list(do_midx_group(path, part1)) + part2
     File "/usr/lib/bup/cmd/bup-midx", line 216, in do_midx_group
       rv = _do_midx(path, None, sublist, gprefix)
     File "/usr/lib/bup/cmd/bup-midx", line 123, in _do_midx
       count = merge_into(fmap, bits, total, inp)
  OverflowError: size does not fit in an int
  ['/usr/bin/bup', 'midx', '--auto', '--dir',
  '/backup/bup/repository/objects/pack']: returned 1

Signed-off-by: Mark J Hewitt <m.hewitt@computer.org>
[rlb@defaultvalue.org: Remove vestigal header includes.
 Remove Py_ssize_t definition for Python < 2.5.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Take additional precautions with the Python/C conversions in _helpers.c.

bitmatch(): Use Py_ssize_t for byte, to match the values it's compared
against.  Make sure the result will fit in the Python destination.

bloom_add(): Use "n" for the len conversion since len is Py_ssize_t.

bloom_contains(): Make sure the Python -> C converted values' ranges
are appropriate before using them.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Mark J Hewitt <mjh@idnet.com>
Don't even test for Linux xattrs if platform.system() doesn't include Linux.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
metadata.py: always set xattr to None by default.

Reported-by: Will Rouesnel <w.rouesnel@gmail.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
tmetadata.py: skip test_apply_to_path_restricted_access() on Cygwin.

Reported-by: Will Rouesnel <w.rouesnel@gmail.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
metadata.py: ignore posix1e if platform.system() includes CYGWIN.

Reported-by: Will Rouesnel <w.rouesnel@gmail.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>