Repository: pipxproject/pipx
Pull request #465 (2020-08-13 22:06:15)

Support running with the Windows embeddable Python distribution
<!-- add an 'x' in the brackets below -->
* [x] I have added an entry to `docs/changelog.md`

## Summary of changes

Adds support for using pipx with the Windows embeddable distribution, which doesn't include a `venv` module itself. The change makes pipx use the embeddable distribution for running its own code, but uses the system default Python for creating virtual environments.

The code is not specific to the embeddable distribution, but is triggered if the `venv` module is not importable, which may be useful for other limited Python distributions.

## Test plan

There is a version of pipx installable using the [scoop](https://scoop.sh/) package manager which uses this patch. I have been using this distribution for some time now.

cs01: > There is a version of pipx installable using the scoop package manager which uses this patch. I have been using this distribution for some time now.

Perhaps documentation for this should be added to pipx?
pfmoore: > Perhaps documentation for this should be added to pipx?

Perhaps. It's not my project, though, so I'd be reluctant to give it more publicity without the owner (@uranusjr) confirming he was OK with that.
uranusjr: I’ve mentioned the project in both #244 and #338. Neither message got any feedback, and I assumed the interest has been minimal.

It’s perfectly fine if it’s added to the documentation, but I’d caution that the maintenance has been low-effort (I regularly skip pipx updates I don’t care to follow), and anyone wanting to use it should be ready to contribute 🙂 
pfmoore: (@uranusjr Offtopic here, but I've got some scripts that do auto-updates of a scoop bucket - I'm pretty sure I could set up an auto-update process that tracked new pipx releases. It's a bit fiddly, though, so I can't promise how soon I'd be able to get to it)
pfmoore: Sorry for the delay. New version uploaded. I had to rebase and force-push as I got in a mess merging with master (black changed its behaviour and I did a whole lot of fixups before I merged and found master handled it differently...)

Anyway, the resulting change seems cleaner as a single commit, so hopefully no harm done.

Patch:
diff --git a/docs/changelog.md b/docs/changelog.md
index aa43b41..f19b37a 100644
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -19,6 +19,7 @@ dev
 - [bugfix] Requiring userpath v1.4.1 or later so ensure Windows bug is fixed for `ensurepath` (#437)
 - [feature] log pipx version (#423)
 - [feature] `--suffix` option for `install` to allow multiple versions of same tool to be installed (#445)
+- [feature] pipx can now be used with the Windows embeddable Python distribution
 
 0.15.4.0
 
diff --git a/src/pipx/constants.py b/src/pipx/constants.py
index d05c668..e8a3870 100644
--- a/src/pipx/constants.py
+++ b/src/pipx/constants.py
@@ -3,7 +3,6 @@ import sys
 from pathlib import Path
 from textwrap import dedent
 
-DEFAULT_PYTHON = sys.executable
 DEFAULT_PIPX_HOME = Path.home() / ".local/pipx"
 DEFAULT_PIPX_BIN_DIR = Path.home() / ".local/bin"
 PIPX_HOME = Path(os.environ.get("PIPX_HOME", DEFAULT_PIPX_HOME)).resolve()
diff --git a/src/pipx/interpreter.py b/src/pipx/interpreter.py
new file mode 100644
index 0000000..cb876c4
--- /dev/null
+++ b/src/pipx/interpreter.py
@@ -0,0 +1,62 @@
+import shutil
+import subprocess
+import sys
+
+from pipx.constants import WINDOWS
+from pipx.util import PipxError
+
+
+def has_venv() -> bool:
+    try:
+        import venv  # noqa
+
+        return True
+    except ImportError:
+        return False
+
+
+# The following code was copied from https://github.com/uranusjr/pipx-standalone
+# which uses the same technique to build a completely standalone pipx
+# distribution.
+#
+# If we are running under the Windows embeddable distribution,
+# venv isn't available (and we probably don't want to use the
+# embeddable distribution as our applications' base Python anyway)
+# so we try to locate the system Python and use that instead.
+
+
+def _find_default_windows_python() -> str:
+
+    if has_venv():
+        return sys.executable
+
+    py = shutil.which("py")
+    if py:
+        return py
+    python = shutil.which("python")
+    if python is None:
+        raise PipxError("no suitable Python found")
+
+    # If the path contains "WindowsApps", it's the store python
+    if "WindowsApps" not in python:
+        return python
+
+    # Special treatment to detect Windows Store stub.
+    # https://twitter.com/zooba/status/1212454929379581952
+
+    proc = subprocess.run(
+        [python, "-V"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
+    )
+    if proc.returncode != 0:
+        # Cover the 9009 return code pre-emptively.
+        raise PipxError("no suitable Python found")
+    if not proc.stdout.strip():
+        # A real Python should print version, Windows Store stub won't.
+        raise PipxError("no suitable Python found")
+    return python  # This executable seems to work.
+
+
+if WINDOWS:
+    DEFAULT_PYTHON = _find_default_windows_python()
+else:
+    DEFAULT_PYTHON = sys.executable
diff --git a/src/pipx/main.py b/src/pipx/main.py
index 6820694..72c68ed 100644
--- a/src/pipx/main.py
+++ b/src/pipx/main.py
@@ -17,6 +17,7 @@ import argcomplete  # type: ignore
 from . import commands, constants
 from .animate import hide_cursor, show_cursor
 from .colors import bold, green
+from .interpreter import DEFAULT_PYTHON
 from .util import PipxError, mkdir
 from .venv import VenvContainer
 from .version import __version__
@@ -266,7 +267,7 @@ def _add_install(subparsers):
     )
     p.add_argument(
         "--python",
-        default=constants.DEFAULT_PYTHON,
+        default=DEFAULT_PYTHON,
         help=(
             "The Python executable used to create the Virtual Environment and run the "
             "associated app/apps. Must be v3.5+."
@@ -379,7 +380,7 @@ def _add_reinstall_all(subparsers):
     )
     p.add_argument(
         "--python",
-        default=constants.DEFAULT_PYTHON,
+        default=DEFAULT_PYTHON,
         help=(
             "The Python executable used to recreate the Virtual Environment "
             "and run the associated app/apps. Must be v3.5+."
@@ -448,7 +449,7 @@ def _add_run(subparsers):
     p.add_argument("--verbose", action="store_true")
     p.add_argument(
         "--python",
-        default=constants.DEFAULT_PYTHON,
+        default=DEFAULT_PYTHON,
         help="The Python version to run package's CLI app with. Must be v3.5+.",
     )
     add_pip_venv_args(p)
diff --git a/src/pipx/shared_libs.py b/src/pipx/shared_libs.py
index 43231b5..93c3268 100644
--- a/src/pipx/shared_libs.py
+++ b/src/pipx/shared_libs.py
@@ -5,7 +5,8 @@ import time
 import datetime
 
 from pipx.animate import animate
-from pipx.constants import DEFAULT_PYTHON, PIPX_SHARED_LIBS, WINDOWS
+from pipx.constants import PIPX_SHARED_LIBS, WINDOWS
+from pipx.interpreter import DEFAULT_PYTHON
 from pipx.util import get_site_packages, get_venv_paths, run
 
 SHARED_LIBS_MAX_AGE_SEC = datetime.timedelta(days=30).total_seconds()
diff --git a/src/pipx/venv.py b/src/pipx/venv.py
index 92d8821..eeae8a7 100644
--- a/src/pipx/venv.py
+++ b/src/pipx/venv.py
@@ -5,7 +5,8 @@ from pathlib import Path
 from typing import Dict, Generator, List, NamedTuple, Set
 
 from pipx.animate import animate
-from pipx.constants import DEFAULT_PYTHON, PIPX_SHARED_PTH
+from pipx.constants import PIPX_SHARED_PTH
+from pipx.interpreter import DEFAULT_PYTHON
 from pipx.package_specifier import (
     parse_specifier_for_install,
     parse_specifier_for_metadata,


Test patch:
diff --git a/tests/test_interpreter.py b/tests/test_interpreter.py
new file mode 100644
index 0000000..c7a93e2
--- /dev/null
+++ b/tests/test_interpreter.py
@@ -0,0 +1,81 @@
+import shutil
+import subprocess
+import sys
+import pytest  # type: ignore
+import pipx.interpreter
+from pipx.interpreter import _find_default_windows_python
+from pipx.util import PipxError
+
+
+def test_windows_python_venv_present(monkeypatch):
+    monkeypatch.setattr(pipx.interpreter, "has_venv", lambda: True)
+    assert _find_default_windows_python() == sys.executable
+
+
+def test_windows_python_no_venv_py_present(monkeypatch):
+    def which(name):
+        if name == "py":
+            return "py"
+
+    monkeypatch.setattr(pipx.interpreter, "has_venv", lambda: False)
+    monkeypatch.setattr(shutil, "which", which)
+    assert _find_default_windows_python() == "py"
+
+
+def test_windows_python_no_venv_python_present(monkeypatch):
+    def which(name):
+        if name == "python":
+            return "python"
+        # Note: returns False for "py"
+
+    monkeypatch.setattr(pipx.interpreter, "has_venv", lambda: False)
+    monkeypatch.setattr(shutil, "which", which)
+    assert _find_default_windows_python() == "python"
+
+
+def test_windows_python_no_venv_no_python(monkeypatch):
+    def which(name):
+        return None
+
+    monkeypatch.setattr(pipx.interpreter, "has_venv", lambda: False)
+    monkeypatch.setattr(shutil, "which", which)
+    with pytest.raises(PipxError):
+        _find_default_windows_python()
+
+
+# Test the checks for the store Python.
+def test_windows_python_no_venv_store_python(monkeypatch):
+    def which(name):
+        if name == "python":
+            return "WindowsApps"
+
+    class dummy_runner:
+        def __init__(self, rc, out):
+            self.rc = rc
+            self.out = out
+
+        def __call__(self, *args, **kw):
+            class Ret:
+                pass
+
+            ret = Ret()
+            ret.returncode = self.rc
+            ret.stdout = self.out
+            return ret
+
+    monkeypatch.setattr(pipx.interpreter, "has_venv", lambda: False)
+    monkeypatch.setattr(shutil, "which", which)
+
+    # Store version stub gives return code 9009
+    monkeypatch.setattr(subprocess, "run", dummy_runner(9009, ""))
+    with pytest.raises(PipxError):
+        _find_default_windows_python()
+
+    # Even if it doesn't, it returns no output
+    monkeypatch.setattr(subprocess, "run", dummy_runner(0, ""))
+    with pytest.raises(PipxError):
+        _find_default_windows_python()
+
+    # If it *does* pass the tests, we use it as it's not the stub
+    monkeypatch.setattr(subprocess, "run", dummy_runner(0, "3.8"))
+    assert _find_default_windows_python() == "WindowsApps"