Call date with -d since it's more portable than --date.

Signed-off-by: Thomas Klausner <wiz@NetBSD.org>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
test-meta.sh: prefer the more portable test '=' to '=='.

Signed-off-by: Thomas Klausner <wiz@NetBSD.org>
[rlb@defaultvalue.org: adjust commit message.]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
test.sh: remove "wc -l" quoting until/unless we fix the broader issue.

On some platforms "wc -l" returns whitespace along with the count,
which breaks the one case where we quoted the result, i.e. "$(... | wc
-l").  For now, remove the quoting until/unless we decide to fix all
of the affected invocations.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem on
NetBSD.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
_apply_linux_xattr_rec(): do nothing if no rec; fix restricted access test.

If the metadata object has no linux_xattr, don't do anything at all in
_apply_linux_xattr_rec(), and don't test the filesystem for xattr
support to determine whether or not to expect an xattr error, because
EACCES trumps the other errors.

This should be more efficient and fix a
test_apply_to_path_restricted_access() failure.

Thanks to Sebastian Schumb <sebastian@sebastians-site.de> for
reporting the problem and helping track down the fix.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Include the chattr value in the ApplyError.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add current-filesystem and path-filesystems to t/lib.sh; use to guard test.

Add current-filesystem which reports the current filesystem type, and
path-filesystems which reports all the filesystems back up to the
root, and use the latter to skip the "save --strip-path (no match)"
test whenever all the filesystems involved aren't the same type.

This issue was discovered while testing on
/some/ext4/tree/containing/a/btrfs/tree.  When the test indexes/saves,
it picks up the metadata for the parents above the btrfs, which may
have (for example) linux +e attrs.  Then when it tries to restore the
tree fully within the btrfs, the restore defers an error when it tries
to restore the +e, which isn't valid for btrfs.

Normally we can avoid this problem via --strip, but this particular
test is actually checking the behavior when --strip doesn't apply.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Use "type -p" instead of "which" in test.sh.

Which isn't portable (if I recall correctly), and we know we're in
Bash.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Don't include atime when determining hardlink compatibility.

Don't include the atimes since we're comparing two paths stat()ed at
different times, and depending on the
platform/filesystem/mount-options, that may result in two different
atimes, even when the paths refer to the same file.

Thanks to Alexander Barton <alex@barton.de> for tracking down the
problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
Correct thanks for commit ff5e91840c92fbb3af70731ede8696114dfee7aa.

Thanks to Ben Kelly <btk@google.com> for tracking down the problem,
rather than Alexander Barton <alex@barton.de>.

Original message:

  Author: Rob Browning <rlb@defaultvalue.org>
  Date:   Tue Nov 5 17:17:45 2013 -0600

    Don't include atime when determining hardlink compatibility.

    Don't include the atimes since we're comparing two paths stat()ed at
    different times, and depending on the
    platform/filesystem/mount-options, that may result in two different
    atimes, even when the paths refer to the same file.

    Thanks to Alexander Barton <alex@barton.de> for tracking down the
    problem.

    Signed-off-by: Rob Browning <rlb@defaultvalue.org>
    Reviewed-by: Gabriel Filion <gabster@lelutin.ca>

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Treat stat fields correctly as signed, unsigned, or unspecified (cf. POSIX).

Patrick reported a problem with restore on Cygwin that was being
caused by bup's handling of unknown uid/gid values; it turns out that
Cygwin returns -1 in that situation:

  http://www.cygwin.com/cygwin-ug-net/ntsec.html
  "Special values of user and group ids"

However, bup didn't see -1; bup saw 4294967295, because it was
incorrectly converting some stat field values as unsigned when POSIX
says that they should be signed (or doesn't specify):

  http://pubs.opengroup.org/onlinepubs/009695299/basedefs/sys/types.h.html

To fix the problem, convert the stat fields correctly -- using the
widest type available (PY_LONG_LONG), and add a new common metadata
record (_rec_tag_common_v2) to store them correctly.  Test at runtime
for potential overflows that we can't rule out at compile time.

Store all of the fields (excepting the timestamp ns values, which we
already force to be non-negative) as signed vints because the overhead
should be negligible, and doing so means that we won't have to break
backward compatibility again if we ever decide we actually want to
store negative values.

Continue to read the legacy format as before, with the realization
that this may leave some existing archives broken.  At the moment, the
only known case is the Cygwin "unknown uid/gid" problem cited above,
and to help with that and similar cases, we'll be adding
--map-{user,group,uid,gid} options to restore.

Thanks to Patrick Rouleau <prouleau72@gmail.com> for the initial
report leading to the deeper problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
Add --map-user --map-group --map-uid and --map-gid options to restore.

Note that the usual metadata rules still appply, so a user or group
entry will normally take precedence over a uid/gid unless
--numeric-ids is specified.  So if you want to map a uid, for example,
you'll either need --numeric-ids, or you'll need to clear the
user/group like this:

  bup restore ... --map-user rlb= --map-uid 1000=2000

These options should also make it possible to recover from archives
that were broken as a result of our incorrect handling of
signed/unsigned stat values (recently fixed).

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
bup-restore.md: add more detail to the restore --map-* documentation.

Thanks to Gabriel Filion <gabster@lelutin.ca> for the suggestions.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Gabriel Filion <gabster@lelutin.ca>
current-filesystem(): accommodate NetBSD and SunOS.

Thanks to Thomas Klausner <tk@giga.or.at> for reporting the problem,
and providing the alternate commands.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
index.py: compute the index metadata offset more carefully.

Previously when reading bupindex.meta (or similar), bup computed the
metadata file offset for it's internal lookup table by just
subtracting the size of a re-encoding of the metadata from file.tell()
after the read.  This works fine until we change the length of the
metadata representation (which we just did); then bup will get the
wrong length from the re-encoded data.

To fix this, do what we should have done originally and capture the
index metadata offset before reading the metadata via tell(), and use
that instead.

Thanks to Mark J Hewitt <mjh@idnet.com> for being the first (known)
victim and for helping track down the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
test-meta.sh: add a missing WVPASS in the ownership restoration tests.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
xstat.py: include "s" for socket in mode strings, i.e. "srw-r-----".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
bup-index.md: make it clear that *bup* doesn't expand --exclude paths.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Add 'tag -f' support.

Add a new flag, -f/--force, to bup tag. By analogy with 'git tag',
'bup tag -f foo commit' will create tag *foo* even if a tag with that
name already exists; the existing tag will be silently overwritten.

By analogy with 'rm -f', add 'bup tag -d -f'. All this does is
suppress the error when you attempt to 'bup tag -d' a tag that doesn't
exist; it is primarily for use in scripts that want to delete a tag
that may or may not exist in the first place without 'bup tag'
reporting a failure.

Signed-off-by: Ben Kelly <btk@google.com>
[rlb@defaultvalue.org: add missing "tag" and wrap text in bup-tag.md;
 put "-d" first in tag-cmd.py optspec line and bup-tag.md; adjust
 commit message.]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
tag-cmd.py: add "FIXME: review for safe writes.".

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Makefile: avoid using -Werror with clang for now.

Apparently the default compiler on OS X and FreeBSD is now clang, and
it doesn't like our CHECK_VALUE_FITS() definition.  So for now, just
don't specify -Werror when we detect clang.

Thanks to Alexander Barton <alex@barton.de>, Thomas Klausner
<tk@giga.or.at>, Sebastian Schumb <sebastian@sebastians-site.de>, and
Zoran Zaric <zz@zoranzaric.de> for reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Alexander Barton <alex@barton.de>