test-fuse.sh: set TZ=UTC for ls dates

Thanks to Mark J Hewitt <mjh@idnet.com> and Scott Sugar
<scottsugar@outlook.com> for reporting the problem, and Patrick
Rouleau <prouleau72@gmail.com> for helping test the solution.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
HACKING: update to reflect 0.26 release

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
test-fuse.sh: use invariant test data

Thanks to Tim Riemenschneider <t.riemenschneider@detco.de> for
reporting the problem.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Fix HEAD request handling in bup-web

HEAD requests to files did not set self.finished(). This causes problems
with user-agents that expect some type of request since it never closes
(OpenOffice/LibreOffice will notably stall opening files from a URL).

This patch stands alone from WebDAV but also fixes the problem there.

The problem can be demonstrated like this:

    bup web
    curl -I 'http://127.0.0.1:8080/some/file'

Signed-off-by: Will Rouesnel <w.rouesnel@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Just log when skipping a large file during save

If bup-save --smaller= is specified, the user *expects* large files
to be skipped.  Having bup report errors and return exitcode 1 in this
case hides real errors.  Instead, log it when running verbosely, but
not otherwise.

Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
[rlb@defaultvalue.org: adjust commit summary; simplify opt.verbose >= 1
 to opt.verbose]
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Correct author for 497505901824dd7382d1d80e152f6b6d810a2844

The author for 497505901824dd7382d1d80e152f6b6d810a2844 was incorrect.
It should have been "Nick Alcock <nick.alcock@oracle.com>", matching
the Signed-off-by.

[rlb@defaultvalue.org: add correction on behalf of author]
Omit htonll when already #defined

OS X 10.10 Yosemite defines it as a macro.

Signed-off-by: R. Andrew Ohana <andrew.ohana@gmail.com>
[rlb@defaultvalue.org: adjust comments; adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
test-fuse.sh: make ls output locale independent

Call "ls" with LC_ALL set to "C".

(Otherwise it reports "total" for English, "insgesamt" for German;
other locales would be similar.)

Signed-off-by: Tim Riemenschneider <git@tim-riemenschneider.de>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Add atomically_replaced_file for safer output

Use `tempfile.mkstemp` to write files safely and then rename them
atomically to the target filename at the end. This guarantees that two
concurrent processes trying to write the same target file will result
in one process "winning" with a consistent version.

Use this to protect:
 * midx writing
 * .idx cache writing

Previously both of these processes `open(destination + '.tmp', 'w')`
which would lead to concurrent processes writing the same tmp file at
the same time.

See also https://groups.google.com/d/msg/bup-list/qdPUaXGO1cI/g8K2aFaaXz0J

Signed-off-by: Nathan Bird <ecthellion@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Adjust atomically_replaced_file documentation.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
thelpers.py: use t/tmp as the tmpdir parent

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Merge work on atomic writes
atomically_replaced_file: ensure error leaves original

Make sure an exception inside the body leaves the original file
content.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Only silence ENOENT in helpers.unlink

This is what it is documented as doing and what makes sense in this
case.

Signed-off-by: Nathan Bird <ecthellion@gmail.com>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>
Handle kernel/FUSE disagreement over Linux attrs

Use long for the Linux attr type, so that it should work on all
little-endian systems, across both normal and FUSE-backed filesystems.

Disable Linux attr support for now on big-endian systems where
sizeof(long) > sizeof(int).  See the changes to bup-index.md and
_helpers.c for more information.

Eventually, we may want to add an argument to allow re-enabling
support on affected systems, when the user can guarantee a homogeneous
filesystem type -- or better yet, the kernel and FUSE will finally
sort out their issues.

Thanks to Tilo Schwarz <mail@tilo-schwarz.de> and daryl5@arcor.de for
reporting and pursuing the problems that lead to this patch.

Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Assume FS_IOC_GETFLAGS may trash output on error

Don't assert that the attr value is reasonable (<= UINT_MAX) when
FS_IOC_GETFLAGS returns an error, because it may change the the value
to something invalid in that situation.

This has been observed to occur when, for example, the underlying
filesystem does not support the operation,

Signed-off-by: Mark J Hewitt <m.hewitt@computer.org>
[rlb@defaultvalue.org: adjust commit message]
Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Tested-by: Rob Browning <rlb@defaultvalue.org>