Drop helpers.next() and just use Python's built-in.

The python built-in 'next' function has an optional second parameter
that specifies what to return instead of excepting with StopIteration.
There is therefore no need for the function in helpers.py.

Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Only define helpers.next() if Python's isn't new enough.

Since helpers.next() now matches the Python built-in's semantics
(since Python 2.6), just rely on the built-in when it's new enough.
Otherwise, set up a matching replacement.

Borrow the fallback next documentation string from Python.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Merge work to make helpers.next() match the newer Python next().
Write the system ARG_MAX to an autogenerated config.py at configure time.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add a batchpipe() command to helpers that behaves somewhat like xargs(1).

Add batchpipe(), which will yield the output produced by calling a
given external command with a given list of arguments.

The resulting output may be provided in chunks, from multiple
invocations of the command, if the limits imposed by ARG_MAX make that
necessary.

See http://www.in-ulm.de/~mascheck/various/argmax/ for details, but
note that batchpipe() takes the additional precaution of adding room
for the argv pointers in addition to the envp pointers.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Retrieve the dates for all branches with one bulk git call in the VFS.

Instead of calling "git rev-list" once per branch to get the branch
tip dates for the VFS branch list, make a single call to "git show"
using a suitable pretty format, and pass all the branch refs as
arguments.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Revert "Write the system ARG_MAX to an autogenerated config.py..."

Prepare to try a more direct approach.

This reverts commit b72365ad597edbc528da5b93528fc3a67a18d58e.
Read ARG_MAX directly via os.sysconf('SC_ARG_MAX').

For now, this code also adds an optional arg_max parameter to
batchpipe(), exclusively for testing.

There appears to be an issue with wvtest at the moment that causes
assignments to something like helpers.arg_max to have no effect from
the importers perspective.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Find SC_ARG_MAX via C sysconf().

Apparently os.sysconf('SC_ARG_MAX') returns -1, even in some common
cases.

Thanks to Alexander Barton <alex@barton.de> for reporting that at
least when run under make 3.81, os.sysconf('SC_ARG_MAX') was just
returning -1.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Always line-buffer bup restore stdout

Flush stdout after every line, even when stdio is not a tty, to
provide more regular progress information.  Useful for progress
monitoring by a parent process or when watching a logfile with "tail".

Signed-off-by: Simon Persson <simonpersson1@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Return integers, not strings from get_commit_dates()

Returning strings broke bup fuse (surprising that it worked
elsewhere), and since we have no fuse tests, it wasn't immediately
apparent.

So fix the bug and add some initial (trivial) fuse tests so that at
least this particular problem doesn't happen again.

To cleanup after the fuse tests, detect fuse mounts in
t/cleanup-mounts-under, and try to unmount them.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move unshared version code from helpers to version-cmd.py

While this is relatively "harmless", the import * directive brings in a
whole lot of unused code into the version subcommand.

At the same time, the three version_* functions are only used in the
version subcommand, so their place inside helpers.py is unjustified. The
same goes for the _version module.

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: adjust commit message]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Re-allow backup set names containing "/"

Revert the prohibition because all releases up to now have allowed
"/", and so that's nothing new.  Disabling "/" in 0.26 would be a
regression, and one we're not ready to commit to -- in fact, we may
eventually do the opposite, and add comprehensive support for "/".

This reverts commit 52a98179490c6ccb990e221bbdeb511f1de59de4.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Initialize MetaStoreWriter self._file to None

Without this, if something goes wrong in the constructor self._file
may not be initialized, which causes close(), invoked via __del__(),
to crash.

Thanks to Fabian Bakkum <fabianbakkum@hotmail.com> for reporting the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add --graft and --regraft support to bup-index and bup-save

Support for highly graftable indexes to enable easy use of bup with
relocateable roots (like snapshots with different paths), or directory
trees with subtle differences (i.e. when backing up home directories
across Windows and Linux)

The principle effect of this branch is to repurpose the metadata cache
objects path field during indexing to store the real path of any given
entry in the bupindex. The bupindex represents what the final bup
archive will look like.
Add documentation for --graft, --regraft commands to bup-index manpage
Add --rename and --delete options to bup-index

--rename allows a user or application invoking bup-index to explicitly
mark files as being renamed to another, preserving the original hash
value. This is useful for inotify driven applications watching for
rename operations.

--delete allows a user or application invoking bup-index to explicitly
mark files as being deleted on the disk, without needing to run a full
bup-index update operation. This is useful for inotify driven
applications watching for unlink operations.
Add --rename and --delete documentation to bup-index.md