Repository: borgbackup/borg

Issue:
ubuntu/debian testing issue with 1.0.8
```
12:53  LocutusOfBorg$ ThomasWaldmann, latest borgbackup breaks testsuite with older pyversions
12:53  LocutusOfBorg$ I mean with older python-py
12:53  LocutusOfBorg$ __________________ ERROR collecting borg/testsuite/archive.py __________________
12:53  LocutusOfBorg$ import file mismatch:
12:53  LocutusOfBorg$ imported module 'borg.testsuite.archive' has this __file__ attribute:
12:53  LocutusOfBorg$   /<<PKGBUILDDIR>>/borg/testsuite/archive.py
12:53  LocutusOfBorg$ which is not the same as the test file we want to collect:
12:53  LocutusOfBorg$   /<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.4/build/borg/testsuite/archive.py
12:53  LocutusOfBorg$ HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
12:53  LocutusOfBorg$ _________________ ERROR collecting borg/testsuite/archiver.py __________________
12:53  LocutusOfBorg$ import file mismatch:
12:53  LocutusOfBorg$ I backported python-py to my ppa, but please fix if possible
12:54  LocutusOfBorg$ I had to backport  python-hypothesis - 3.4.2-2~ubuntu16.10.1~ppa1   python-py - 1.4.31-2~ubuntu16.10.1~ppa1  
                      pytest - 3.0.3-1~ubuntu16.10.1~ppa1 
12:54  LocutusOfBorg$ to yakkety, even more to xenial
12:54  LocutusOfBorg$ and trusty is still ongoing
12:55  LocutusOfBorg$ this will make impossible to stable-release upgrade it in the future, because I can't backport half python to official releases
14:57  ThomasWaldmann$ LocutusOfBorg: i've also seen that now and then, but i fixed it by cleaning up the __pycache__
14:58  ThomasWaldmann$ not sure what is causing this, did you find out more? I suspect conftest.py.
15:00  ThomasWaldmann$ python-py is the name of the pytest (py.test) package?
15:01  ThomasWaldmann$ we don't use hypothesis for borg
15:02  ThomasWaldmann$ and, what precisely do you mean with "latest borgbackup"? 1.0.x or 1.1 / master branch / 1.1.0bN?
15:45  LocutusOfBorg$ ThomasWaldmann, https://packages.qa.debian.org/p/python-py.html I need at least 1.4.31-2
15:45  LocutusOfBorg$ and python-py 1.4.31 needs hypothesis and pytest 3.0.3-1
15:45  LocutusOfBorg$ so, even if not directly, such issue is dragging 3 updates, on yakkety, and many more on trusty of course
15:46  LocutusOfBorg$ talking about 1.0.8
15:46  LocutusOfBorg$ uploaded yesterday in unstable
15:59  ThomasWaldmann$ i wonder a bit about that python-py - it is nothing we require directly
15:59  ThomasWaldmann$ same for hypothesis. so wouldn't pytest be enough?
16:00  ThomasWaldmann$ (and maybe tox, depending on how you invoke the tests)
16:00  ThomasWaldmann$ LocutusOfBorg: ^
16:01  ThomasWaldmann$ https://github.com/borgbackup/borg/commits/1.0-maint/conftest.py maybe it is triggered by that
```



Discussion:
Not sure about the root cause(*), but maybe an `export PYTHONDONTWRITEBYTECODE=1` in the build environment works as quick fix.

python-py is probably the "py" library, used by py.test. That's the thing where py.path.local yielded by the tmpdir fixture and others etc come from.

(*) something something tox?

@enkore ah, good idea. @LocutusOfBorg can you try that?

The conftest.py changes I made were for other reasons, not just for fun. :)

this is on a clean xenial installation /chroot

```
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/borg/hashindex.o -o /borgbackup-1.0.8/borg/hashindex.cpython-35m-x86_64-linux-gnu.so
skipping 'borg/platform_linux.c' Cython extension (up-to-date)
building 'borg.platform_linux' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -c borg/platform_linux.c -o build/temp.linux-x86_64-3.5/borg/platform_linux.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.5/borg/platform_linux.o -lacl -o /borgbackup-1.0.8/borg/platform_linux.cpython-35m-x86_64-linux-gnu.so
fakeroot dh_auto_test
I: pybuild base:184: cd /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount and not test_fuse' --pyargs borg.testsuite
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.5.2, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /borgbackup-1.0.8, inifile: setup.cfg
plugins: cov-2.2.1
collected 0 items / 17 errors 

====================================================================================================== ERRORS ======================================================================================================
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/archive.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.archive' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/archive.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/archive.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/archiver.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.archiver' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/archiver.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/archiver.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/benchmark.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.benchmark' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/benchmark.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/benchmark.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/chunker.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.chunker' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/chunker.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/chunker.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/compress.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.compress' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/compress.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/compress.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
______________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/crypto.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.crypto' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/crypto.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/crypto.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/hashindex.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.hashindex' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/hashindex.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/hashindex.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/helpers.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.helpers' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/helpers.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/helpers.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_______________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/key.py ________________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.key' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/key.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/key.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/locking.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.locking' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/locking.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/locking.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
______________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/logger.py ______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.logger' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/logger.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/logger.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/lrucache.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.lrucache' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/lrucache.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/lrucache.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/platform.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.platform' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/platform.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/platform.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/repository.py ____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.repository' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/repository.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/repository.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
___________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/shellpattern.py ___________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.shellpattern' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/shellpattern.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/shellpattern.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
_____________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/upgrader.py _____________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.upgrader' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/upgrader.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/upgrader.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
______________________________________________________________________ ERROR collecting .pybuild/pythonX.Y_3.5/build/borg/testsuite/xattr.py _______________________________________________________________________
import file mismatch:
imported module 'borg.testsuite.xattr' has this __file__ attribute:
  /borgbackup-1.0.8/borg/testsuite/xattr.py
which is not the same as the test file we want to collect:
  /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build/borg/testsuite/xattr.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
============================================================================================= 17 error in 0.51 seconds =============================================================================================
E: pybuild pybuild:274: test: plugin distutils failed with: exit code=1: cd /borgbackup-1.0.8/.pybuild/pythonX.Y_3.5/build; python3.5 -m pytest -k 'not benchmark and not fuse_mount and not test_fuse' --pyargs borg.testsuite
dh_auto_test: pybuild --test -i python{version} -p 3.5 --dir . returned exit code 13
debian/rules:66: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 25
make[1]: Leaving directory '/borgbackup-1.0.8'
debian/rules:46: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
```

@LocutusOfBorg was that with the suggested env var?

yes

Perhaps this fix which we also needed previously solves it: https://github.com/borgbackup/borg/commit/4f1157c3a49ee68c350dc8b8aa68e9dbc7f20093

@enkore it works!
thanks!


Resolving patch:
diff --git a/conftest.py b/conftest.py
--- a/conftest.py
+++ b/conftest.py
@@ -1,4 +1,5 @@
 import os
+import os.path
 import sys
 
 import pytest
@@ -10,12 +11,10 @@
 # The workaround is to remove entries pointing there from the path and check whether "borg"
 # is still importable. If it is not, then it has not been installed in the environment
 # and the entries are put back.
-#
-# TODO: After moving the package to src/: remove this.
 
 original_path = list(sys.path)
 for entry in original_path:
-    if entry == '' or entry.endswith('/borg'):
+    if entry == '' or entry == os.path.dirname(__file__):
         sys.path.remove(entry)
 
 try:


Test patch: