Repository: pypa/virtualenv

Issue:
PermissionError when using venv creator on Solaris
**Issue**
Several tests are failing for us on Solaris when `venv` creator is being used (and not just the test suite, installed virtualenv as well):
```console
python3.7 -m virtualenv -vvv --with-traceback --creator venv xxxx
```
From what I found out, this is because `--creator venv` uses the files delivered with Python runtime itself (those are /usr/lib/python3.x/venv/scripts/*) and there is a difference between how Linux (at least RHEL based ones I checked) an Solaris deliver those files. On Linux they have write permission for the owner and on Solaris not - this is how we deliver all the files by default and from my quick check it seems that e.g. both OpenBSD and FreeBSD do the same thing.

The workaround is pretty simple for us - delivering those files with write permission fixes the issue. But I still wanted to report this in case this uncovers some wrong assumptions in virtualenv.

**Environment**
OS: Oracle Solaris 11.4

**Output of the virtual environment creation**
Make sure to run the creation with `-vvv --with-traceback`:

```console
2111 setup logging to NOTSET [DEBUG report:35]
2114 created app data folder /home/user/.local/share/virtualenv [DEBUG __init__:45]
2156 find interpreter for spec PythonSpec(path=/bin/python3.7) [INFO builtin:56]
2157 proposed PythonInfo(spec=CPython3.7.13.final.0-64, exe=/bin/python3.7, platform=sunos5, version='3.7.13 (default, Jul 29 2022, 13:27:54) \n[GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [INFO builtin:63]
2157 accepted PythonInfo(spec=CPython3.7.13.final.0-64, exe=/bin/python3.7, platform=sunos5, version='3.7.13 (default, Jul 29 2022, 13:27:54) \n[GCC 11.3.0]', encoding_fs_io=utf-8-utf-8) [DEBUG builtin:65]
2162 filesystem is case-sensitive [DEBUG info:24]
2251 create virtual environment via Venv(dest=/..../userland-gate/components/python/virtualenv/xxxxx, clear=False, no_vcs_ignore=False, global=False, describe=CPython3Posix) [INFO session:48]
2256 create virtualenv import hook file /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/_virtualenv.pth [DEBUG api:89]
2256 create /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/_virtualenv.py [DEBUG api:92]
2257 ============================== target debug ============================== [DEBUG session:50]
2258 debug via /..../userland-gate/components/python/virtualenv/xxxxx/bin/python /usr/lib/python3.7/vendor-packages/virtualenv/create/debug.py [DEBUG creator:197]
2258 {
  "sys": {
    "executable": "/..../userland-gate/components/python/virtualenv/xxxxx/bin/python",
    "_base_executable": null,
    "prefix": "/..../userland-gate/components/python/virtualenv/xxxxx",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/..../userland-gate/components/python/virtualenv/xxxxx",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib/python37.zip",
      "/usr/lib/python3.7",
      "/usr/lib/python3.7/lib-dynload",
      "/..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.7.13 (default, Jul 29 2022, 13:27:54) \n[GCC 11.3.0]",
  "makefile_filename": "/usr/lib/python3.7/config-3.7m/Makefile",
  "os": "<module 'os' from '/usr/lib/python3.7/os.py'>",
  "site": "<module 'site' from '/usr/lib/python3.7/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib/python3.7/datetime.py'>",
  "math": "<module 'math' from '/usr/lib/python3.7/lib-dynload/math.cpython-37m.so'>",
  "json": "<module 'json' from '/usr/lib/python3.7/json/__init__.py'>"
} [DEBUG session:51]
2349 add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/user/.local/share/virtualenv) [INFO session:55]
2365 wrote embed update of distribution pip at /home/user/.local/share/virtualenv/wheel/3.7/embed/3/pip.json [DEBUG via_disk_folder:155]
2365 wrote embed update of distribution wheel at /home/user/.local/share/virtualenv/wheel/3.7/embed/3/wheel.json [DEBUG via_disk_folder:155]
2366 wrote embed update of distribution setuptools at /home/user/.local/share/virtualenv/wheel/3.7/embed/3/setuptools.json [DEBUG via_disk_folder:155]
2392 triggered periodic upgrade of wheel==0.37.1 (for python 3.7) via background process having PID 14966 [INFO periodic_update:223]
2392 triggered periodic upgrade of pip==22.2.2 (for python 3.7) via background process having PID 14967 [INFO periodic_update:223]
2395 triggered periodic upgrade of setuptools==63.4.1 (for python 3.7) via background process having PID 14968 [INFO periodic_update:223]
2401 got embed update of distribution setuptools from /home/user/.local/share/virtualenv/wheel/3.7/embed/3/setuptools.json [DEBUG via_disk_folder:129]
2401 got embed update of distribution pip from /home/user/.local/share/virtualenv/wheel/3.7/embed/3/pip.json [DEBUG via_disk_folder:129]
2402 got embed update of distribution wheel from /home/user/.local/share/virtualenv/wheel/3.7/embed/3/wheel.json [DEBUG via_disk_folder:129]
2410 install setuptools from wheel /usr/lib/python3.7/vendor-packages/virtualenv/seed/wheels/embed/setuptools-63.4.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:47]
2410 install pip from wheel /usr/lib/python3.7/vendor-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:47]
2411 install wheel from wheel /usr/lib/python3.7/vendor-packages/virtualenv/seed/wheels/embed/wheel-0.37.1-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:47]
2418 Attempting to acquire lock 2247434926886480 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any.lock [DEBUG _api:172]
2418 Attempting to acquire lock 2247434926817040 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:172]
2418 Attempting to acquire lock 2247434926817104 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any.lock [DEBUG _api:172]
2421 Lock 2247434926817104 acquired on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any.lock [DEBUG _api:176]
2422 Lock 2247434926886480 acquired on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any.lock [DEBUG _api:176]
2422 Lock 2247434926817040 acquired on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:176]
2422 build install image for setuptools-63.4.1-py3-none-any.whl to /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any [DEBUG base:45]
2422 build install image for pip-22.2.2-py3-none-any.whl to /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any [DEBUG base:45]
2423 build install image for wheel-0.37.1-py2.py3-none-any.whl to /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any [DEBUG base:45]
2568 Attempting to release lock 2247434926817040 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:209]
2569 Lock 2247434926817040 released on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any.lock [DEBUG _api:212]
2571 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/wheel [DEBUG _sync:36]
2597 copy /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.virtualenv to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/wheel-0.37.1.virtualenv [DEBUG _sync:36]
2598 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/wheel-0.37.1-py2.py3-none-any/wheel-0.37.1.dist-info to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/wheel-0.37.1.dist-info [DEBUG _sync:36]
2609 generated console scripts wheel3.7 wheel3 wheel wheel-3.7 [DEBUG base:41]
3489 Attempting to release lock 2247434926817104 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any.lock [DEBUG _api:209]
3490 Lock 2247434926817104 released on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any.lock [DEBUG _api:212]
3492 copy /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/distutils-precedence.pth to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/distutils-precedence.pth [DEBUG _sync:36]
3493 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/_distutils_hack to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/_distutils_hack [DEBUG _sync:36]
3497 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/pkg_resources to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/pkg_resources[DEBUG _sync:36]
3541 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/setuptools-63.4.1.dist-info to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/setuptools-63.4.1.dist-info [DEBUG _sync:36]
3548 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/setuptools to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/setuptools [DEBUG _sync:36]
3743 copy /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/setuptools-63.4.1-py3-none-any/setuptools-63.4.1.virtualenv to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/setuptools-63.4.1.virtualenv [DEBUG _sync:36]
3745 generated console scripts  [DEBUG base:41]
4447 Attempting to release lock 2247434926886480 on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any.lock [DEBUG _api:209]
4448 Lock 2247434926886480 released on /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any.lock [DEBUG _api:212]
4449 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any/pip-22.2.2.dist-info to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/pip-22.2.2.dist-info [DEBUG _sync:36]
4455 copy directory /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any/pip to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/pip [DEBUG _sync:36]
4784 copy /home/user/.local/share/virtualenv/wheel/3.7/image/1/CopyPipInstall/pip-22.2.2-py3-none-any/pip-22.2.2.virtualenv to /..../userland-gate/components/python/virtualenv/xxxxx/lib/python3.7/site-packages/pip-22.2.2.virtualenv [DEBUG _sync:36]
4787 generated console scripts pip3 pip3.7 pip-3.7 pip [DEBUG base:41]
4787 add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [INFO session:61]
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.7/vendor-packages/virtualenv/__main__.py", line 69, in <module>
    run_with_catch()  # pragma: no cov
  File "/usr/lib/python3.7/vendor-packages/virtualenv/__main__.py", line 54, in run_with_catch
    run(args, options, env)
  File "/usr/lib/python3.7/vendor-packages/virtualenv/__main__.py", line 16, in run
    session = cli_run(args, options, env)
  File "/usr/lib/python3.7/vendor-packages/virtualenv/run/__init__.py", line 30, in cli_run
    of_session.run()
  File "/usr/lib/python3.7/vendor-packages/virtualenv/run/session.py", line 44, in run
    self._activate()
  File "/usr/lib/python3.7/vendor-packages/virtualenv/run/session.py", line 63, in _activate
    activator.generate(self.creator)
  File "/usr/lib/python3.7/vendor-packages/virtualenv/activation/via_template.py", line 21, in generate
    generated = self._generate(replacements, self.templates(), dest_folder, creator)
  File "/usr/lib/python3.7/vendor-packages/virtualenv/activation/via_template.py", line 41, in _generate
    dest.write_bytes(text.encode("utf-8"))
  File "/usr/lib/python3.7/pathlib.py", line 1230, in write_bytes
    with self.open(mode='wb') as f:
  File "/usr/lib/python3.7/pathlib.py", line 1208, in open
    opener=self._opener)
  File "/usr/lib/python3.7/pathlib.py", line 1063, in _opener
    return self._accessor.open(self, flags, mode)
PermissionError: [Errno 13] Permission denied: '/..../userland-gate/components/python/virtualenv/xxxxx/bin/activate'
```



Discussion:
Seems upstream venv issue.
Hi, I am sorry for the very delayed response. While I won't deny that it's possible that it is an upstream issue, I am not necessarily sure.

When I run
```
python3.7 -m venv venv
```
everything works as expected. The reason it works is that standard venv (and I hope I am not missing something) seems to just create `activate*` files with the same permissions as in `/usr/lib/python3.7/venv/scripts/` (it specifically copies the permission bits with `shutil.copymode` [here](https://github.com/python/cpython/blob/7522ff72484fab740a85f27f2326336906a69674/Lib/venv/__init__.py#L381)).

This works fine because after it creates them, it doesn't touch them anymore (and thus, no write permission is fine).

However, virtualenv first calls `self.creator.run()` [here](https://github.com/pypa/virtualenv/blob/fd7e6fc5a1f42441c409d56d3b14a26e55e39a8d/src/virtualenv/run/session.py#L49) which does what I described above, and then (as can be seen from the traceback) in `activator.generate(self.creator)` [here](https://github.com/pypa/virtualenv/blob/fd7e6fc5a1f42441c409d56d3b14a26e55e39a8d/src/virtualenv/run/session.py#L63) and all the way in `ViaTemplateActivator._generate()` [here](https://github.com/pypa/virtualenv/blob/fd7e6fc5a1f42441c409d56d3b14a26e55e39a8d/src/virtualenv/activation/via_template.py#L45) it calls `dest.write_bytes` and tries to overwrite those, and that fails because they don't have write permission bits.

I don't see how this can be an upstream issue.

I think that virtualenv should check whether those files exist and either change permission bits or remove them as they will be overwritten anyway. It seems that the following change fixes the issue (although it would need to be tested more):
```diff
--- virtualenv-20.16.5/src/virtualenv/activation/via_template.py
+++ virtualenv-20.16.5/src/virtualenv/activation/via_template.py
@@ -37,6 +37,8 @@ class ViaTemplateActivator(Activator, me
         for template in templates:
             text = self.instantiate_template(replacements, template, creator)
             dest = to_folder / self.as_name(template)
+            if dest.exists():
+                dest.unlink()
             # use write_bytes to avoid platform specific line normalization (\n -> \r\n)
             dest.write_bytes(text.encode("utf-8"))
             generated.append(dest)
```

Resolving patch:
diff --git a/src/virtualenv/activation/via_template.py b/src/virtualenv/activation/via_template.py
--- a/src/virtualenv/activation/via_template.py
+++ b/src/virtualenv/activation/via_template.py
@@ -43,6 +43,10 @@ def _generate(self, replacements, templates, to_folder, creator):
         for template in templates:
             text = self.instantiate_template(replacements, template, creator)
             dest = to_folder / self.as_name(template)
+            # remove the file if it already exists - this prevents permission
+            # errors when the dest is not writable
+            if dest.exists():
+                dest.unlink()
             # use write_bytes to avoid platform specific line normalization (\n -> \r\n)
             dest.write_bytes(text.encode("utf-8"))
             generated.append(dest)


Test patch:
diff --git a/tests/unit/create/test_creator.py b/tests/unit/create/test_creator.py
--- a/tests/unit/create/test_creator.py
+++ b/tests/unit/create/test_creator.py
@@ -611,3 +611,25 @@ def _get_sys_path(flag=None):
         assert non_python_path == [i for i in base if i not in extra_as_python_path]
     else:
         assert base == extra_all
+
+
+# Make sure that the venv creator works on systems where vendor-delivered files
+# (specifically venv scripts delivered with Python itself) are not writable.
+#
+# https://github.com/pypa/virtualenv/issues/2419
+@pytest.mark.skipif("venv" not in CURRENT_CREATORS, reason="test needs venv creator")
+def test_venv_creator_without_write_perms(tmp_path, mocker):
+    from virtualenv.run.session import Session
+
+    prev = Session._create
+
+    def func(self):
+        prev(self)
+        scripts_dir = self.creator.dest / "bin"
+        for script in scripts_dir.glob("*ctivate*"):
+            script.chmod(stat.S_IREAD | stat.S_IRGRP | stat.S_IROTH)
+
+    mocker.patch("virtualenv.run.session.Session._create", side_effect=func, autospec=True)
+
+    cmd = [str(tmp_path), "--seeder", "app-data", "--without-pip", "--creator", "venv"]
+    cli_run(cmd)