Repository: borgbackup/borg

Issue:
testsuite failure on hurd-i386
Hi, following the error:
(note: hurd-i386 is not a release architecture, so you can just close this issue)
https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=hurd-i386&ver=1.0.5-2&stamp=1468279201

```
plugins: cov-2.2.1
collected 554 items

borg/testsuite/archive.py ...............................
borg/testsuite/archiver.py .s..F.............s..............X.........sssssssssssssssssssssssssssssssssssssssss........F......s......s..............X...........
borg/testsuite/chunker.py ...
borg/testsuite/compress.py ........
borg/testsuite/crypto.py .....
borg/testsuite/hashindex.py .....................
borg/testsuite/helpers.py ...............................................................................................................................................................
borg/testsuite/key.py ....
borg/testsuite/locking.py ..............
borg/testsuite/logger.py ....
borg/testsuite/lrucache.py ..
borg/testsuite/platform.py sssss
borg/testsuite/repository.py .........................................
borg/testsuite/shellpattern.py .............................................................
borg/testsuite/upgrader.py sssss..
borg/testsuite/xattr.py s

=================================== FAILURES ===================================
_________________________ ArchiverTestCase.test_atime __________________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        self.create_test_files()
        atime, mtime = 123456780, 234567890
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if hasattr(os, 'O_NOATIME'):
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1468278979000000000 == 123456780000000000
E            +  where 1468278979000000000 = os.stat_result(st_mode=35309, st_ino=253216, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1468278979, st_mtime=234567890, st_ctime=1468278979).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=32704, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1468278979).st_atime_ns

borg/testsuite/archiver.py:369: AssertionError
______________________ RemoteArchiverTestCase.test_atime _______________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        self.create_test_files()
        atime, mtime = 123456780, 234567890
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if hasattr(os, 'O_NOATIME'):
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1468279041000000000 == 123456780000000000
E            +  where 1468279041000000000 = os.stat_result(st_mode=35309, st_ino=539078, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1468279041, st_mtime=234567890, st_ctime=1468279041).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=552, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1468279042).st_atime_ns

borg/testsuite/archiver.py:369: AssertionError
========= 55 tests deselected by '-knot benchmark and not fuse_mount' ==========
= 2 failed, 439 passed, 56 skipped, 55 deselected, 2 xpassed in 221.08 seconds =
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd /«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount' --pyargs borg.testsuite
dh_auto_test: pybuild --test -i python{version} -p 3.5 returned exit code 13
/bin/fakeauth: Error 13 for child 4500
/bin/settrans: Error 13 for child 4499
debian/rules:63: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 13
make[1]: Leaving directory '/«PKGBUILDDIR»'
```



Discussion:
the above fix seems to be not complete?
https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=hurd-i386&ver=1.0.5-3&stamp=1468322970

@LocutusOfBorg if that happens only on that platform, it might mean that on this platform they offer O_NOATIME, but they did not really implement it correctly.

Hehe, search for O_NOATIME there: https://www.gnu.org/software/hurd/contributing.html

lol thanks :+1: 

c07d91e kills the atime asserts for "gnu0" / Hurd. Should be done. Thanks for your testing :)

interestingly enough, now kfreebsd is failing lol
https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=kfreebsd-amd64&ver=1.0.5-3&stamp=1468330206&file=log

platform gnukfreebsd10 -- Python 3.5.2, pytest-2.9.2, py-1.4.31, pluggy-0.3.1

4891d33  -- just check that NOATIME is there _and actually works_. Test may false-positive on systems with coarse atime granularity (seconds...days) and NOATIME. But in that case the test case itself should also pass -- most of the time.

this is the build for 1.0.6 and hurd
https://buildd.debian.org/status/fetch.php?pkg=borgbackup&arch=hurd-i386&ver=1.0.6-2&stamp=1468410860

```
borg/testsuite/repository.py .........................................
borg/testsuite/shellpattern.py .............................................................
borg/testsuite/upgrader.py sssss..
borg/testsuite/xattr.py s

=================================== FAILURES ===================================
_________________________ ArchiverTestCase.test_atime __________________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                os.close(os.open(some_file, flags_noatime))
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1468410612000000000 == 123456780000000000
E            +  where 1468410612000000000 = os.stat_result(st_mode=35309, st_ino=236834, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1468410612, st_mtime=234567890, st_ctime=1468410612).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=236900, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1468410612).st_atime_ns

borg/testsuite/archiver.py:389: AssertionError
______________________ RemoteArchiverTestCase.test_atime _______________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                os.close(os.open(some_file, flags_noatime))
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1468410680000000000 == 123456780000000000
E            +  where 1468410680000000000 = os.stat_result(st_mode=35309, st_ino=220675, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1468410680, st_mtime=234567890, st_ctime=1468410680).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=237077, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1468410681).st_atime_ns

borg/testsuite/archiver.py:389: AssertionError
 55 tests deselected by '-knot benchmark and not fuse_mount and not test_fuse' =
= 2 failed, 439 passed, 56 skipped, 55 deselected, 2 xpassed in 265.29 seconds =
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd /«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount and not test_fuse' --pyargs borg.testsuite
```

somewhat it is still failing

I really don't get why this is still failing. Hurd's implementation of noatime must be very flawed (or not existing at all, while proclaiming support, nice). The only hypothesis I have here is that some other syscall besides open() triggers an atime update on Hurd, say read() or fadvise().

At this point I feel like there is no point in pursuing this further. I'd say we either just backlist _gnu0_ from the test or leave that blacklisting to the one running it on hurd (you ;)

Moving to low-priority 1.0.7.

Can you check if this is working with #1367 merged?

mmm I used 1.0.6 with that two lines patch

```
borg/testsuite/crypto.py .....
borg/testsuite/hashindex.py .....................
borg/testsuite/helpers.py ...............................................................................................................................................................
borg/testsuite/key.py ....
borg/testsuite/locking.py ..............
borg/testsuite/logger.py ....
borg/testsuite/lrucache.py ..
borg/testsuite/platform.py sssss
borg/testsuite/repository.py .........................................
borg/testsuite/shellpattern.py .............................................................
borg/testsuite/upgrader.py sssss..
borg/testsuite/xattr.py s

===================================================================================================== FAILURES =====================================================================================================
___________________________________________________________________________________________ ArchiverTestCase.test_atime ____________________________________________________________________________________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                with open(os.open(some_file, flags_noatime)) as file:
                    file.read()
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1469436215000000000 == 123456780000000000
E            +  where 1469436215000000000 = os.stat_result(st_mode=35309, st_ino=425182, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1469436215, st_mtime=234567890, st_ctime=1469436215).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=425291, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1469436216).st_atime_ns

borg/testsuite/archiver.py:390: AssertionError
________________________________________________________________________________________ RemoteArchiverTestCase.test_atime _________________________________________________________________________________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                with open(os.open(some_file, flags_noatime)) as file:
                    file.read()
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1469436282000000000 == 123456780000000000
E            +  where 1469436282000000000 = os.stat_result(st_mode=35309, st_ino=425429, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1469436282, st_mtime=234567890, st_ctime=1469436282).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=375926, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1469436283).st_atime_ns

borg/testsuite/archiver.py:390: AssertionError
================================================================== 55 tests deselected by '-knot benchmark and not fuse_mount and not test_fuse' ===================================================================
=================================================================== 2 failed, 439 passed, 56 skipped, 55 deselected, 2 xpassed in 237.65 seconds ===================================================================
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd /home/locutusofborg/borgbackup-1.0.6/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount and not test_fuse' --pyargs borg.testsuite
dh_auto_test: pybuild --test -i python{version} -p 3.5 returned exit code 13
/bin/fakeauth: Error 13 for child 19351
/bin/settrans: Error 13 for child 19350
debian/rules:63: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 13
make[1]: Leaving directory '/home/locutusofborg/borgbackup-1.0.6'
debian/rules:46: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

```

Thanks for testing!

So there's still some other read-operation on GNU HURD that triggers an atime update with O_NOATIME set. Since there's no ACL/xattr support there I'm wondering which one that could be...

according to the build logs, the problem seems to have appeared between 1.0.0rc1 and 1.0.0
git diff 1.0.0rc1 1.0.0

https://buildd.debian.org/status/logs.php?pkg=borgbackup&arch=hurd-i386

failure with 1.0.7-rc1

```
=================================== FAILURES ===================================
_________________________ ArchiverTestCase.test_atime __________________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                os.close(os.open(some_file, flags_noatime))
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1470744219000000000 == 123456780000000000
E            +  where 1470744219000000000 = os.stat_result(st_mode=35309, st_ino=220760, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1470744219, st_mtime=234567890, st_ctime=1470744219).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=32690, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1470744219).st_atime_ns

borg/testsuite/archiver.py:389: AssertionError
______________________ RemoteArchiverTestCase.test_atime _______________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_atime>

    def test_atime(self):
        def has_noatime(some_file):
            atime_before = os.stat(some_file).st_atime_ns
            try:
                os.close(os.open(some_file, flags_noatime))
            except PermissionError:
                return False
            else:
                atime_after = os.stat(some_file).st_atime_ns
                noatime_used = flags_noatime != flags_normal
                return noatime_used and atime_before == atime_after

        self.create_test_files()
        atime, mtime = 123456780, 234567890
        have_noatime = has_noatime('input/file1')
        os.utime('input/file1', (atime, mtime))
        self.cmd('init', self.repository_location)
        self.cmd('create', self.repository_location + '::test', 'input')
        with changedir('output'):
            self.cmd('extract', self.repository_location + '::test')
        sti = os.stat('input/file1')
        sto = os.stat('output/input/file1')
        assert sti.st_mtime_ns == sto.st_mtime_ns == mtime * 1e9
        if have_noatime:
>           assert sti.st_atime_ns == sto.st_atime_ns == atime * 1e9
E           AssertionError: assert 1470744278000000000 == 123456780000000000
E            +  where 1470744278000000000 = os.stat_result(st_mode=35309, st_ino=204347, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=1470744278, st_mtime=234567890, st_ctime=1470744278).st_atime_ns
E            +  and   123456780000000000 = os.stat_result(st_mode=35309, st_ino=294227, st_dev=5, st_nlink=2, st_uid=100, st_gid=200, st_size=81920, st_atime=123456780, st_mtime=234567890, st_ctime=1470744279).st_atime_ns

borg/testsuite/archiver.py:389: AssertionError
 55 tests deselected by '-knot benchmark and not fuse_mount and not test_fuse' =
= 2 failed, 443 passed, 57 skipped, 55 deselected, 2 xpassed in 201.20 seconds =
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd '/«PKGBUILDDIR»/.pybuild/pythonX.Y_3.5/build'; python3.5 -m pytest -k 'not benchmark and not fuse_mount and not test_fuse' --pyargs borg.testsuite
dh_auto_test: pybuild --test -i python{version} -p 3.5 returned exit code 13
/bin/fakeauth: Error 13 for child 3514
/bin/settrans: Error 13 for child 3513
```

hmm, we just open/close in the detection code. do we need to read?

That was already tried and failed: https://github.com/borgbackup/borg/issues/1315#issuecomment-234892613

Edit: Did someone report this yet to the gnu0 guys?


Resolving patch:
diff --git a/borg/testsuite/archiver.py b/borg/testsuite/archiver.py
--- a/borg/testsuite/archiver.py
+++ b/borg/testsuite/archiver.py
@@ -362,6 +362,9 @@ def filter(output):
         # the interesting parts of info_output2 and info_output should be same
         self.assert_equal(filter(info_output), filter(info_output2))
 
+    # Search for O_NOATIME there: https://www.gnu.org/software/hurd/contributing.html - we just
+    # skip the test on Hurd, it is not critical anyway, just testing a performance optimization.
+    @pytest.mark.skipif(sys.platform == 'gnu0', reason="O_NOATIME is strangely broken on GNU Hurd")
     def test_atime(self):
         def has_noatime(some_file):
             atime_before = os.stat(some_file).st_atime_ns


Test patch: