Repository: rdiff-backup/rdiff-backup (language: python)

Issue:
[ENH] make tox.ini only write to readable places with --current-env
## What is it about?

When trying to add a `%check` option to RPM spec file, using the current `tox.ini` file it attempts to write to a directory that it does not have permission to do, i.e. `open("{envsitepackagesdir}/coverage.pth","w")`

## Version, Python, Operating System

Current github version, i.e. 2.4.1+

## rdiff-backup call

Adding the following to rdiff-backup RPM fails:

```
%check
%tox
```

Causes a failure on writing:

```
+ /usr/bin/python3 -m tox --current-env -q --recreate -e py311
rdiff-backup 2.4.1.dev14+gf27110de3
Traceback (most recent call last):
  File "<string>", line 1, in <module>
PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.11/site-packages/coverage.pth'
py311: exit 1 (0.01 seconds) /builddir/build/BUILD/rdiff-backup-weekly> python -c 'with open("/usr/lib/python3.11/site-packages/coverage.pth","w") as fd: fd.write("import coverage; coverage.process_startup()\n")' pid=1235
  py311: FAIL code 1 (0.19 seconds)
  evaluation failed :( (0.21 seconds)
error: Bad exit status from /var/tmp/rpm-tmp.HDMejv (%check)
    Bad exit status from /var/tmp/rpm-tmp.HDMejv (%check)

```

The question is `tox.ini` suitable for use for an RPM `%check`, or what can be used, and should it be attempting to write to that directory?


Discussion / hints:
I think that the issue is the `--current-env`. I was wondering about this option, but this seems to come from a plug-in, not tox itself: https://pypi.org/project/tox-current-env/

Without this option/plugin, `{envsitepackagesdir}` is within the virtualenv created by tox and hence writable. From me the look of it, it needs to be there and nowhere else:

```shell
# write the hook file which will make sure that coverage is loaded
# also for sub-processes, like for "client/server" rdiff-backup
python -c 'with open("{envsitepackagesdir}/coverage.pth","w") as fd: fd.write("import coverage; coverage.process_startup()\n")'
```

Ah, that sounds like the solution to your issue: https://pypi.org/project/coverage_pth/ resp. https://github.com/dougn/coverage_pth.

Quick and dirty:

```diff
$ git diff
diff --git a/tox.ini b/tox.ini
index 836bec4..8bdcaec 100644
--- a/tox.ini
+++ b/tox.ini
@@ -22,15 +22,15 @@ setenv =
 deps = -r{toxinidir}/requs/base.txt
        -r{toxinidir}/requs/optional.txt
        -r{toxinidir}/requs/test.txt
+# write the hook file which will make sure that coverage is loaded
+# also for sub-processes, like for "client/server" rdiff-backup
+       coverage_pth
 allowlist_externals = sh
 commands_pre =
        rdiff-backup --version
 # must be the first command to setup the test environment
        python testing/commontest.py
        coverage erase
-# write the hook file which will make sure that coverage is loaded
-# also for sub-processes, like for "client/server" rdiff-backup
-       python -c 'with open("{envsitepackagesdir}/coverage.pth","w") as fd: fd.write("import coverage; coverage.process_startup()\n")'
 commands =
        coverage run testing/action_backuprestore_test.py --verbose --buffer
        coverage run testing/action_calculate_test.py --verbose --buffer
```

Let me know if this fixes your issue. This should, but I fear that you'll choke then on `COVERAGE_FILE = {envlogdir}/coverage.sqlite`. I guess that it'll need replacement with `{envtmpdir}`, which should be everywhere writable by a normal user.
One more thing: it's perhaps obvious but `coverage_pth` can't become a solid dependency, as it would change the behaviour of any usage of coverage.
More difficult than expected, the coverage_pth package is bogus... OK, need to find an alternative.

Fix patch:
diff --git a/.gitignore b/.gitignore
index 423039446..f33a142e8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,7 +12,7 @@ MANIFEST
 *[._]local[._]*
 
 # setup.py installation files
-src/rdiff_backup.egg-info/
+*.egg-info/
 .eggs/
 
 # Windows build file
diff --git a/tools/coverage_pth/MANIFEST.in b/tools/coverage_pth/MANIFEST.in
new file mode 100644
index 000000000..5b7a6ee95
--- /dev/null
+++ b/tools/coverage_pth/MANIFEST.in
@@ -0,0 +1,1 @@
+include **/*.pth
diff --git a/tools/coverage_pth/pyproject.toml b/tools/coverage_pth/pyproject.toml
new file mode 100644
index 000000000..10207d380
--- /dev/null
+++ b/tools/coverage_pth/pyproject.toml
@@ -0,0 +1,21 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "coverage-pth"
+version = "0.0.1"
+authors = [
+    {name = "The rdiff-backup project", email = "rdiff-backup-users@nongnu.org"}
+]
+description = "write the hook file which will make sure that coverage is loaded"
+
+[tool.setuptools]
+package-dir = {""="src"}
+include-package-data = true
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+"*" = ["*.pth"]
diff --git a/tools/coverage_pth/src/coverage.pth b/tools/coverage_pth/src/coverage.pth
new file mode 100644
index 000000000..493469d7e
--- /dev/null
+++ b/tools/coverage_pth/src/coverage.pth
@@ -0,0 +1,1 @@
+import coverage; coverage.process_startup()
diff --git a/tox.ini b/tox.ini
index 2a39103bd..4f4876f8b 100644
--- a/tox.ini
+++ b/tox.ini
@@ -17,7 +17,7 @@ passenv = RDIFF_TEST_*, RDIFF_BACKUP_*
 setenv =
 # paths for coverage must be absolute so that sub-processes find them
 # even if they're started from another location.
-	COVERAGE_FILE = {envlogdir}/coverage.sqlite
+	COVERAGE_FILE = {envtmpdir}/coverage.sqlite
 	COVERAGE_PROCESS_START = {toxinidir}/tox.ini
 deps = -r{toxinidir}/requs/base.txt
        -r{toxinidir}/requs/optional.txt
@@ -30,7 +30,7 @@ commands_pre =
 	coverage erase
 # write the hook file which will make sure that coverage is loaded
 # also for sub-processes, like for "client/server" rdiff-backup
-	python -c 'with open("{envsitepackagesdir}/coverage.pth","w") as fd: fd.write("import coverage; coverage.process_startup()\n")'
+	python testing/coverage_pth.py
 commands =
 	coverage run testing/action_backuprestore_test.py --verbose --buffer
 	coverage run testing/action_calculate_test.py --verbose --buffer
diff --git a/tox_smoke.ini b/tox_smoke.ini
new file mode 100644
index 000000000..8fecf7990
--- /dev/null
+++ b/tox_smoke.ini
@@ -0,0 +1,51 @@
+# tox (https://tox.readthedocs.io/) is a tool for running tests
+# in multiple virtualenvs. This configuration file will run the
+# test suite on all supported python versions. To use it, "pip install tox"
+# and then run "tox -c tox_smoke.ini" from this directory.
+
+# Configuration file for smoke tests with no coverage.
+
+[tox]
+envlist = py
+
+[testenv]
+# make sure those variables are passed down; you should define 
+# either explicitly the RDIFF_TEST_* variables or rely on the current
+# user being correctly identified (which might not happen in a container)
+passenv = RDIFF_TEST_*, RDIFF_BACKUP_*
+deps = -r{toxinidir}/requs/base.txt
+       -r{toxinidir}/requs/optional.txt
+       -r{toxinidir}/requs/test.txt
+allowlist_externals = sh
+commands_pre =
+	rdiff-backup --version
+# must be the first command to setup the test environment
+	python testing/commontest.py
+commands =
+	python testing/action_backuprestore_test.py --verbose --buffer
+	python testing/action_calculate_test.py --verbose --buffer
+	python testing/action_compare_test.py --verbose --buffer
+	python testing/action_complete_test.py --verbose --buffer
+	python testing/action_list_test.py --verbose --buffer
+	python testing/action_regress_test.py --verbose --buffer
+	python testing/action_remove_test.py --verbose --buffer
+	python testing/action_test_test.py --verbose --buffer
+	python testing/action_verify_test.py --verbose --buffer
+	python testing/readonly_actions_test.py --verbose --buffer
+	python testing/api_test.py --verbose --buffer
+	python testing/location_map_filenames_test.py --verbose --buffer
+	python testing/location_map_hardlinks_test.py --verbose --buffer
+	python testing/location_lock_test.py --verbose --buffer
+	python testing/utils_simpleps_test.py --verbose --buffer
+
+	python testing/ctest.py --verbose --buffer
+	python testing/timetest.py --verbose --buffer
+	python testing/librsynctest.py --verbose --buffer
+	python testing/user_grouptest.py --verbose --buffer
+	python testing/setconnectionstest.py --verbose --buffer
+	python testing/iterfiletest.py --verbose --buffer
+	python testing/eas_aclstest.py --verbose --buffer
+	python testing/fs_abilitiestest.py --verbose --buffer
+	python testing/hashtest.py --verbose --buffer
+	python testing/errorsrecovertest.py --verbose --buffer
+	python testing/rdb_arguments.py --verbose --buffer


Test patch:
diff --git a/testing/api_test.py b/testing/api_test.py
index 2f6a3b340..2c5b6415a 100644
--- a/testing/api_test.py
+++ b/testing/api_test.py
@@ -17,17 +17,19 @@ def test_runtime_info_calling(self):
         Globals.api_version["actual"] = 201
         info = Globals.get_runtime_info()
 
-        # because the current test will have a different call than rdiff-backup itself
-        # we can't compare certain keys
+        # because the current test will have a different call than rdiff-backup
+        # itself, we can't compare certain keys
         self.assertIn("exec", out_info)
         self.assertIn("argv", out_info["exec"])
         out_info["exec"].pop("argv")
         info["exec"].pop("argv")
-        # info['python']['executable'] could also be different but I think that
-        # our test environments make sure that it doesn't happen, unless Windows
-        if os.name == "nt":
-            info["python"]["executable"] = info["python"]["executable"].lower()
-            out_info["python"]["executable"] = out_info["python"]["executable"].lower()
+
+        # info['python'] could also be different, under special conditions
+        self.assertIn("python", out_info)
+        self.assertIn("executable", out_info["python"])
+        out_info.pop("python")
+        info.pop("python")
+
         self.assertEqual(info, out_info)
 
     def test_default_actual_api(self):
@@ -41,7 +43,7 @@ def test_default_actual_api(self):
         self.assertEqual(out_info["exec"]["api_version"]["actual"], api_version["max"])
 
     def test_debug_output(self):
-        """we use verbosity 9 only to cover debug functions in logging"""
+        """Use verbosity 9 to cover debug functions in logging"""
         output = subprocess.check_output(
             [RBBin, b"-v", b"9", b"--terminal-verbosity", b"9", b"info"]
         )
diff --git a/testing/coverage_pth.py b/testing/coverage_pth.py
new file mode 100644
index 000000000..ac8ff6ac8
--- /dev/null
+++ b/testing/coverage_pth.py
@@ -0,0 +1,31 @@
+# write the hook file which will make sure that coverage is loaded
+# also for sub-processes, like for "client/server" rdiff-backup
+
+import os
+import site
+import sys
+
+for site_path in sys.path:
+    coverage_pth = os.path.join(site_path, "coverage.pth")
+    if os.path.isfile(coverage_pth):
+        print("The file {} exists already".format(coverage_pth))
+        sys.exit(0)
+
+if site.ENABLE_USER_SITE:
+    site_packages = [site.getusersitepackages()]
+else:  # we're probably in a virtualenv
+    site_packages = reversed(site.getsitepackages())
+
+for site_path in site_packages:
+    coverage_pth = os.path.join(site_path, "coverage.pth")
+    try:
+        os.makedirs(site_path, exist_ok=True)
+        with open(coverage_pth, "w") as fd:
+            fd.write("import coverage; coverage.process_startup()\n")
+            print("The file {} has been written".format(coverage_pth))
+            sys.exit(0)
+    except OSError as exc:
+        print("Couldn't write the file {} due to {}".format(coverage_pth, exc))
+
+print("The coverage.pth file could be written nowhere")
+sys.exit(1)