HACKING: update to reflect 0.25 release.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Require utimensat or utimes/lutimes.

This was already true (tests would fail), so make it explicit, and if
we do encounter any platforms where none of these functions are
available, we can worry about it then.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Stop testing negative timestamps -- they're not necessarily valid.

Both timevals and timespecs use a time_t for the seconds which POSIX
requires to be an "integer or real-floating type", so unsigned integer
types are perfectly valid.  (See FUSE for example -- though it's not
even using a time_t; at the moment it's just using uint64_t...).

Thanks to Tim Riemenschneider <t.riemenschneider@detco.de> for
reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add index option "--exclude-rx-from FILE".

Signed-off-by: Thomas Klausner <tk@giga.or.at>
[rlb@defaultvalue.org: adjust commit message; change "path" to
 "pattern" in bup-index.md description; adjust bup-index.md text;
 adjust optspec text.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move the --exclude-rx tests from test.sh to test-save-restore-exclude-rx.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add simple test for "index --exclude-rx-from FILE".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add restore option "--exclude-rx-from FILE".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't fail tests when the timestamp read resolution is higher than write.

Previously, if bup was able to read path timestamps at a higher
resolution than it could write them, tests would fail.  This situation
can occur (for example) when the stat() resolution is 1ns, but either
the underlying filesystem's isn't, or bup wasn't able to find
utimensat at build time.

To fix this, compute the maximum resolution of the test filesystem
(via "t/ns-timestamp-resolutions HERE"), and then limit the "bup
xstat" timestamp resolution in the tests to match via a new
--mtime-resolution argument.

For completeness, also add --atime-resolution and --ctime-resolution
arguments.

Thanks to Tim Riemenschneider <t.riemenschneider@detco.de> for
reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Always publish utimensat in helpers when available and fix type conversions.

Previously, bup assumed a timespec was (long, long), which agrees with
glibc.info, but utimensat(2) and POSIX say it's (time_t, long).

Since the types vary (and even if they didn't, Python doesn't provide
safe "time_t" conversions), add ASSIGN_PYLONG_TO_INTEGRAL(), which
should handle safely assigning a PyLong value to any given C integral
destination, whenever it will fit -- then use that macro to handle the
utimensat arguments.

Whenever it's available, publish utimensat independently in helpers.
For now, this allows us to test utimensat directly, and in the future,
it may allow us to test bup's behavior with either utimensat or
utimes/lutimes without recompilation, whenever both are available.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add -n, -A, -F, --file-type, --numeric-ids and detailed -l to "bup ls".

With the addition of -A, change -a to behave more conventionally,
i.e. to show "." and "..".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Alexander Barton <alex@barton.de>
Reviewed-By: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: fall back to n.mode when n.metadata() isn't
 available in node_info() so that CommitDirs ("ls -AF /"), etc. will
 still work; thanks to Gabriel Filion <gabster@lelutin.ca> for the
 report; fix the rdiff-backup test to use -A.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Remove vestigial and inappropriate isnan()/isinf() timestamp tests.

I'm fairly sure this is an historical accident, but regardless, remove
the (my) obviously inappropriate floating point tests on integer
values from bup_parse_xutime_args().  Nothing to see here -- move
along.

I've broken this change out from a larger post-0.25 patch, where I'd
already fixed it, so that it'll be easy to pull into stable if we
like.

Thanks to Ed Maste <carpeddiem@gmail.com> for reporting the problem
(which prompted me to go ahead and create the separate fix).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Always publish (l)utimes in helpers when available and fix type conversions.

Previously, bup assumed a timeval was (long, long), which agrees with
glibc.info and (Linux) utimes(2), but POSIX says it's (time_t,
suseconds_t).

Since the types vary (and even if they didn't, Python doesn't provide
safe "time_t" conversions), handle the type conversions more
generically, via ASSIGN_PYLONG_TO_INTEGRAL() and
INTEGRAL_ASSIGNMENT_FITS().

Optimistically assume the latter is sufficient for tv_usec (where we
convert it to a long long) because POSIX guarantees that tv_usec will
be a signed integral.

Whenever available, publish utimes and lutimes independently in
helpers, and run the relevant t/txstat.py tests.  So now, we'll be
running at least some tests on utimensat, lutimes, and utimes whenever
they're available.

A more significant improvement would be to run some broader bup tests
twice, once with utimensat and once with utimes/lutimes, whenever both
are available.

cf. 9ce545703dda3c888081c504863308902ce8c56b

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Remove ancient lib/tornado in favor of an external dependency.

Our lib/tornado dates from over three years ago -- drop it in favor of
an external dependency, and update the install instructions
accordingly.

Anyone trying to run "bup web" without tornado installed will see
this:

  error: cannot find the python "tornado" module; please install it

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't require non-negative timespec ns; fix stat timestamp conversions.

Previously we required that the system timespec tv_nsec values be
non-negative -- stop that.  POSIX doesn't specify, and there's no real
need for us to care, since all significant timestamp manipulations are
now handled as integer nanosecond values, not (sec, ns) pairs.

Change stat_struct_to_py() (used by bup_stat()/bup_lstat()) to use
INTEGER_TO_PY() to convert the timestamp st_atime/st_mtime/st_ctime
values so that we don't have to care about the sign/size of the
underlying type (which should be time_t).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move save --strip/--graft tests from test.sh to test-save-strip-graft.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move import-rdiff-backup tests from test.sh to test-import-rdiff-backup.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move compression tests from test.sh to test-compression.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move test.sh exclude tests to renamed test-save-restore-excludes.sh.

Rename test-save-restore-exclude-rx.sh to
test-save-restore-excludes.sh, and move test.sh exclude tests there.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move clear-index tests from test.sh to test-index-clear.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move the main fsck tests from test.sh to test-fsck.sh.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>