Repository: lektor/lektor (python)

Issue:
Lektor cached venv is broken on macOS with Python installed via Homebrew
I'm using the latest version (Lektor-3.4.0b9.dev7+gbfce91a.d20230827 as of now).

python3 is installed via homebrew (`python@3.11`) and is symlinked:

`/usr/local/bin/python3` -> `/usr/local/Cellar/python@3.11/3.11.5/bin/python3` -> ... -> `/usr/local/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/bin/python3.11`

I have created a venv using `python3 -m venv env`, everything worked fine in that venv until I installed a plugin (`lektor plugins add lektor-index-pages`). After that, trying to run the `lektor server` started raising an exception:
```
...
  File "/Users/igor/.virtualenvs/lektor/lib/python3.11/site-packages/lektor/packages.py", line 125, in create
    EnvBuilder(upgrade_deps=upgrade_deps, **options).create(self.path)
...
subprocess.CalledProcessError: Command '['/Users/igor/Library/Caches/Lektor/venvs/d63a0228b40929c7189d2c99d18e960b/bin/python3.11', '-m', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGABRT: 6>.
```

When I tried to run python3 using that path, I saw this error:
```
dyld[34820]: Library not loaded: @loader_path/../../../../Python.framework/Versions/3.11/Python
  Referenced from: <430E089A-35B0-314B-9FD1-1E60F9CE5756> /Users/igor/Library/Caches/Lektor/venvs/d63a0228b40929c7189d2c99d18e960b/bin/python3.11
  Reason: tried: '/Users/igor/Library/Caches/Lektor/venvs/d63a0228b40929c7189d2c99d18e960b/bin/../../../../Python.framework/Versions/3.11/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file), '/System/Library/Frameworks/Python.framework/Versions/3.11/Python' (no such file, not in dyld cache)
[1]    34820 abort
```

So, looks like the homebrew version of Python was built with relative paths in mind and when EnvBuilder copies that binary to another path, things go south.

I have successfully fixed that for myself by editing the EnvBuilder options in [lektor/packages.py:123](https://github.com/lektor/lektor/blob/master/lektor/packages.py#L123), adding an `symlinks=True` option there:

```
options: Dict[str, Any] = {"clear": True, "with_pip": with_pip, "symlinks": True}
```

This has solved the issue for me - now, when Lektor creates a cached venv, it symlinks the used python binary in there and everything seems to work fine.

I've never heard of Lektor until today and I don't feel comfortable with making a pull request since my fix could break something else. Sorry!

Pull request:
Use symlinks when creating virtual environments — fixes #1159
This changes things to use symlinks when constructing Lektor's private virtual environment.

By default, `python -m venv` uses symlinks, except when running on Windows (where symlinks can be problematic).  However when creating a virtual environment programmatically, `venv.EnvBuilder` defaults to not using symlinks.

Using symlinks is better (everywhere except maybe on Windows) because:

- It uses less disk space
- That's what `python -m venv` does
- Apparently, that's what Python on macOS expects, at least when involved via Homebrew (see #1159)

<!---
Remember to please check that your code passes tests locally. Test with the Makefile
commands `make test`, or if you only need to test just Python / JS `make test-python`
or `make test-js`.
--->

### Issue(s) Resolved

<!--- List the issue(s) below, in the form "Fixes #1234"; one per line --->

Fixes #1159

### Related Issues / Links

<!---
Are there any similar or related issues or pull requests?
Did you make a pull request to update the docs?
--->

### Description of Changes

- [x] Added unit test(s) covering the changes (if testable)
<!--- Remember that an image/animation is worth a thousand words! --->


<!--- Explain what you've done and why --->

<!--- Thanks for your help making Lektor better for everyone! --->


Patch:
diff --git a/lektor/packages.py b/lektor/packages.py
index 93d6ece..b7515ba 100644
--- a/lektor/packages.py
+++ b/lektor/packages.py
@@ -87,6 +87,12 @@ def remove_package_from_project(project, name):
     return None
 
 
+if os.name == "nt":
+    _default_venv_symlinks = False
+else:
+    _default_venv_symlinks = True
+
+
 class VirtualEnv:
     """A helper for manipulating our private package cache virtual environment.
 
@@ -100,7 +106,12 @@ class VirtualEnv:
     def __init__(self, path: StrPath):
         self.path = Path(path)
 
-    def create(self, with_pip: bool = True, upgrade_deps: bool = True) -> None:
+    def create(
+        self,
+        with_pip: bool = True,
+        upgrade_deps: bool = True,
+        symlinks: bool = _default_venv_symlinks,
+    ) -> None:
         """(Re-)Create a new virtual environment.
 
         This will remove any existing virtual environment and create a new one.
@@ -120,7 +131,11 @@ class VirtualEnv:
         #
         # Note that, e.g., pip>=21.3 is required to support PEP660 editable
         # installs.
-        options: Dict[str, Any] = {"clear": True, "with_pip": with_pip}
+        options: Dict[str, Any] = {
+            "clear": True,
+            "with_pip": with_pip,
+            "symlinks": symlinks,
+        }
         if sys.version_info >= (3, 9):
             EnvBuilder(upgrade_deps=upgrade_deps, **options).create(self.path)
         else:


Test patch:
diff --git a/tests/test_packages.py b/tests/test_packages.py
index 8c6d6c4..55bcc94 100644
--- a/tests/test_packages.py
+++ b/tests/test_packages.py
@@ -53,6 +53,12 @@ def test_VirtualEnv_addsitedir(nested_venv: VirtualEnv) -> None:
     assert proc.returncode == 0
 
 
+@pytest.mark.skipif(os.name == "nt", reason="Windows")
+def test_VirtualEnv_uses_symlinks(nested_venv: VirtualEnv) -> None:
+    executable = Path(nested_venv.executable)
+    assert executable.resolve(strict=True).parent != executable.parent
+
+
 @pytest.mark.requiresinternet
 @pytest.mark.slowtest
 def test_VirtualEnv_run_pip_install(tmp_path: Path) -> None: