Stop interleaving stream and mmap IO operations when writing the index.

Previously bup would write the index via a simultaneous combination of
stream operations and mmap writes to non-overlapping, but adjacent
regions of the same file.  This was causing index corruption in some
cases.

Instead, make an extra pass over the data in memory to precompute the
size of the final index section, which contains any 31+-bit offsets.
Then mmap and write the entire set of tables directly, avoiding the
need for simultaneous stream operations.

Reported-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Greg Troxel <gdt@lexort.com>
Reviewed-by: Greg Troxel <gdt@lexort.com>
bup-init.md: update after dropping automatic ~/.bup init.

Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
[rlb@defaultvalue.org: adjust commit message; remove escape from
 BUP\_DIR after list discussion; remove additional "bup init" from
 README.md example.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't try to load POSIX1e ACLs when the library isn't installed.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Always fall back to socket()/bind() when os.mknod(...S_IFSOCK) fails.

Previously bup would use socket()/bind() instead of os.mknod(... |
stat.S_IFSOCK) on Cygwin, but this issue isn't Cygwin specific.
Remove the platform conditionalization, and fall back to
socket()/bind() any time mknod() fails with EINVAL.

Thanks to Robert Edmonds <edmonds@debian.org> for reporting the
relevant failure on a Debian kFreeBSD buildd.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix minor typo in Metadata.read() in metadata.py.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Treat ENOSYS like EOPNOTSUPP in _add_posix1e_acl().

Thanks to Robert Edmonds <edmonds@debian.org> for reporting the
relevant failure on a Debian kFreeBSD buildd.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix path ownership restoration problems on Cygwin.

It turns out that Cygwin won't allow you to chown() a path to an
unknown uid or gid, even when "root".

For now, make that a deferred error on Cygwin, rework the tests to
avoid it when possible, and disable the tests (on Cygwin) that require
it.

For the record, it appears that tar doesn't normally hit this problem
on Cygwin because it uses "geteuid() == 0" to detect super-user
status, which won't be true in the normal case, even if the user is an
administrator.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Fix ./configure handling of OS_* for GNU/kFreeBSD.

Thanks to Robert Edmonds <edmonds@debian.org> for the initial report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Update bup-split(1); document -d, clarify the split "modes", and reorganize.

Update the synopsis, document the "-d DATE" option, and make a clearer
distinction between normal options and the options that select the
split "mode".

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: add dashes to seconds-since-epoch; adjust the
 MODES text a bit and move -b down; make a few other edits and adjust
 commit message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Update bup split -h output (optspec) to distinguish between split "modes".

Split it into three lines to clearly show that some modes don't fare
well with others.

Signed-off-by: Gabriel Filion <gabster@lelutin.ca>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Mention --git-ids in bup split -h output sample commands.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
write_idx: make filename char* non-const (fixes python 2.6.6 compatibility).

Thanks to Alexander Barton <alex@barton.de> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Quash stat(2) st_rdev unless we're going to need it for mknod().

On some platforms (i.e. kFreeBSD), the st_rdev value isn't completely
stable.  For example, given "date > foo; cp -a foo foo-2", the st_rdev
value may differ between the two files.

Since we only need the st_rdev value for the call to mknod() when
restoring character and block special files, set it to zero for
anything else.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Ignore unimplemented metadata record during read (_rec_tag_nfsv4_acl).

Don't test for _rec_tag_nfsv4_acl and then try to call an
unimplemented method in Metadata.read().  Otherwise, if/when we do add
support, older versions of bup will choke on it.

Thanks to Alexander Barton <alex@barton.de> for pointing this out.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
configure: since the *BSDs have GNU make as gmake, look for that too.

Thanks to Ryan Hinton <iobass@email.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move non-system includes after system includes in _helpers.c.

Having Python.h before assert.h was apparently causing trouble on
OpenBSD:

  _helpers.c: In function 'bup_xutime_ns':
  _helpers.c:842: warning: implicit declaration of function 'utimensat'
  _helpers.c:842: error: 'AT_FDCWD' undeclared (first use in this function)
  _helpers.c:842: error: (Each undeclared identifier is reported only once
  _helpers.c:842: error: for each function it appears in.)
  _helpers.c:843: error: 'AT_SYMLINK_NOFOLLOW' undeclared (first use in this function)
  error: command 'cc' failed with exit status 1

...and moving the non-sytem includes lower is reasonable anyway.

Thanks to Ryan Hinton <iobass@email.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Move _helper.c's Python.h back up (now to the top), because Python.

Move the Python.h include in _helper.c to the top because Python says
so:

  http://docs.python.org/2/c-api/intro.html#include-files

Though according to this, it's been fixed upstream:

  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=206805

Which might explain why I didn't see the problem here.

Thanks to Chandan Kumar <chandank.kumar@gmail.com> for the report.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>