Repository: wntrblm/nox
Pull request #762 (2024-02-16 22:46:44)

feat: add a uv backend
UV just came out from the Ruff folks, and it's insanely fast - it makes venvs faster (25ms) than Python can start up (50ms). And much faster than virtualenv (400ms) and venv (nearly 5 seconds). The package install speeds are also unreal compared to pip.

There are some differences - namely `name @ .` is [required](https://github.com/astral-sh/uv/issues/1499) instead of `.` if an install is not editable (for now). But it's really nice to try it out!


https://github.com/wntrblm/nox/assets/4616906/1fe4c981-9d9a-4aaf-b19e-94c79daeebe7


Nox's docs build in 4 seconds instead of 22 seconds using the uv backend.



henryiii: I pulled your changes in here, thanks @alex! (Nice username, by the way!)
alex: Happy to help! Looking forward to this.
henryiii: I'm thinking a nice followup could be to allow fallback backends. So `uv|virtualenv` would use UV if available, and fallback to virtualenv if not available. Same with `mamba|conda`.
henryiii: Is there a way to ignore a line from coverage only on Python 3.7?
edgarrmondragon: > Is there a way to ignore a line from coverage only on Python 3.7?

Maybe https://github.com/asottile/covdefaults?tab=readme-ov-file#version-specific--pragma-no-cover?
alex: I don't believe there's a way to skip coverage by python version with just coverage.py. I think you have two choices:

1) `#pragma: no cover` the line always
2) Install `uv` outside the virtualenv, with the system python interpreter, so that it's on the `$PATH` in CI.
henryiii: @FollowTheProcess, @DiddiLeija, @cjolowicz, or @theacodes, I think this is ready. Also, there are quite a few other PRs that are basically ready and useful to have open too!
henryiii: Also, Tox added UV support yesterday: https://github.com/tox-dev/tox-uv
slafs: Fantastic work! Thank you all! My 3 cents:

1. [minor] The `uv` option is not mentioned in the **help text** of `--default-venv-backend` nor `--force-venv-backend` CLI options.
2. It's also not mentioned in the docs for those options ("Changing the sessions default backend" and "Forcing the sessions backend" in `usage.rst`).
3. Speaking of docs, would it make sense to add a note that `uv venv` does NOT install `pip` by default? Unlike `venv` or `virtualenv`. I've run into a (rather obscure) caveat when trying out this branch. Namely you can't switch backends from `uv` (fresh) to `venv` (re-use) without re-creating the whole virtual environment, because `pip` is missing. It's OK to go from `venv` (fresh) to `uv` (re-use), though. Again, this is probably a non-issue in a day-to-day usage and only relevant when exploring backends or seeing that ridiculous speed difference 😅. Nevertheless, I feel like it wouldn't hurt to mention that in the docs.

Also, thanks @skshetry for pointing out that bug in `uv` 😬.

> I'm thinking a nice followup could be to allow fallback backends. So uv|virtualenv would use UV if available, and fallback to virtualenv if not available. Same with mamba|conda.

Yes! 💯 That would be great!
henryiii: I kind of forgot about docs because I was initially just making it work, will add!
FollowTheProcess: @henryiii nice work! Sorry been a bit swamped recently and haven't had much time to look at Nox too much. I think once the docs get an update calling out the "no pip" behaviour as mentioned above then I'm happy and excited to get this in 👍🏻 
henryiii: @slafs: Thanks! I've addressed (1) and (2). I think I also fixed (3) by detecting `uv` created venvs, would be great if you could check!
henryiii: CI is busted on Python 3.7 until https://github.com/conda-forge/conda-forge-repodata-patches-feedstock/pull/667 goes in.
henryiii: CI should be restartable now, fix looks like it's available.
henryiii: > LGTM, but I don't think (3) is fixed, right? 

(3) should be "fixed" in the case that if you switch environments, it should rebuild the environment now, as it can recognize that the environment was created by virtualenv or nox. You won't have `pip` inside the `uv` environment unless you install it, like any other dependency. But `.install(...)` should always work.
slafs: Hmmm... I might be holding it wrong, but with this `noxfile.py`:
```py
import nox

@nox.session()
def xxx(session):
    session.install("cowsay")
    session.run("cowsay", "-t", f"Hello world")
```
running:
```sh
nox -N -db uv -s xxx
```
and then:
```sh
nox -r -s xxx
```
gives me:
```
nox > Running session xxx
nox > Re-using existing virtual environment at .nox/xxx.
nox > python -m pip install cowsay
nox > Command python -m pip install cowsay failed with exit code 1:
/private/tmp/noxuvtest/.nox/xxx/bin/python: No module named pip
nox > Session xxx failed.
```
I think `reuse_existing_virtualenvs` option is the key here (we have it on by default in our noxfile - via global `nox.options.reuse_existing_virtualenvs = True`).
AFAIU without this option nox always recreates the environment, no? 🤔 
henryiii: Hmm, that's exactly what I thought should work now, it should recreate the environment since the backend changed. I'll investigate.
layday: Feel like nox should either error out or recreate the venv if the active venv backend != backend the venv was created with.  A marker file in the venv folder might work, or reading `pyvenv.cfg` - both virtualenv and uv add a key with the backend's name. I don't know about conda and mamba.
cjolowicz: This is a known issue. That environment variable is a feature flag for some incomplete work to fix it, and was never publicly announced. I think this PR is good as is.
cjolowicz: Could you re-base this?
henryiii: Requiring 100% coverage for each job is causing problems, the tox 3 jobs are not running the tox 4 parts. The total coverage is fine, but not the per job coverage.
henryiii: And the "passing" tox 4 one looks very suspicions!

![Screenshot 2024-02-20 at 3 06 14 PM](https://github.com/wntrblm/nox/assets/4616906/59f1f2f3-111d-40ee-b80a-e098784e8a66)

cjolowicz: > Requiring 100% coverage for each job is causing problems, the tox 3 jobs are not running the tox 4 parts. The total coverage is fine, but not the per job coverage.

This is unfortunate, we shouldn't require 100% coverage for individual jobs. We should only gather the coverage data in those jobs. In my own projects, I omit the coverage report unless `session.interactive` is true, which achieves this behavior. But I see you're already on to this. :+1: 
henryiii: Yeah, it looks like we did and I broke it somehow. Going to try to revert the coverage changes (in a few hours probably)
henryiii: I can remove the covdefaults & conditional pragmas, they aren't needed, it's being checked at the end again.
cjolowicz: I'd be happy to reset to 9c935699987a04bf8a96f6bc04ba50927b903d9f and clean up the coverage-related code in a follow-up PR if that's easier.
henryiii: I've pulled out the fix #778 (which is then running into issues merging due to Windows paths, I think). Currently there are a lot of "no covers" due to this on different OSs, and I'd like to not have to add more for the new code.
cjolowicz: Would something like this help with merging the Windows paths?

```toml
[tool.coverage.paths]
source = ["src", "*/site-packages"]

[tool.coverage.run]
source = ["nox"]
```
henryiii: I'll rebase on #778 and remove coverage changes when that goes in.

Patch:
diff --git a/docs/usage.rst b/docs/usage.rst
index bc23761..dc1e03d 100644
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -126,25 +126,43 @@ Then running ``nox --session tests`` will actually run all parametrized versions
 Changing the sessions default backend
 -------------------------------------
 
-By default Nox uses ``virtualenv`` as the virtual environment backend for the sessions, but it also supports ``conda``, ``mamba``, and ``venv`` as well as no backend (passthrough to whatever python environment Nox is running on). You can change the default behaviour by using ``-db <backend>`` or ``--default-venv-backend <backend>``. Supported names are ``('none', 'virtualenv', 'conda', 'mamba', 'venv')``.
+By default Nox uses ``virtualenv`` as the virtual environment backend for the sessions, but it also supports ``uv``, ``conda``, ``mamba``, and ``venv`` as well as no backend (passthrough to whatever python environment Nox is running on). You can change the default behaviour by using ``-db <backend>`` or ``--default-venv-backend <backend>``. Supported names are ``('none', 'uv', 'virtualenv', 'conda', 'mamba', 'venv')``.
 
 .. code-block:: console
 
     nox -db conda
     nox --default-venv-backend conda
 
+.. note::
+
+   The ``uv``, ``conda``, and ``mamba`` backends require their respective
+   programs be pre-installed. ``uv`` is distributed as a Python package
+   and can be installed with the ``nox[uv]`` extra.
 
 You can also set this option in the Noxfile with ``nox.options.default_venv_backend``. In case both are provided, the commandline argument takes precedence.
 
 Note that using this option does not change the backend for sessions where ``venv_backend`` is explicitly set.
 
+.. warning::
+
+   The ``uv`` backend does not install anything by default, including ``pip``,
+   as ``uv pip`` is used to install programs instead. If you need to manually
+   interact with pip, you should install it with ``session.install("pip")``.
+
+.. warning::
+
+   Currently the ``uv`` backend requires the ``<program name> @ .`` syntax to
+   install a local folder in non-editable mode; it does not (yet) compute the
+   name from the install process like pip does if the name is omitted. Editable
+   installs do not require a name.
+
 
 .. _opt-force-venv-backend:
 
 Forcing the sessions backend
 ----------------------------
 
-You might work in a different environment than a project's default continuous integration settings, and might wish to get a quick way to execute the same tasks but on a different venv backend. For this purpose, you can temporarily force the backend used by **all** sessions in the current Nox execution by using ``-fb <backend>`` or ``--force-venv-backend <backend>``. No exceptions are made, the backend will be forced for all sessions run whatever the other options values and Noxfile configuration. Supported names are ``('none', 'virtualenv', 'conda', 'venv')``.
+You might work in a different environment than a project's default continuous integration settings, and might wish to get a quick way to execute the same tasks but on a different venv backend. For this purpose, you can temporarily force the backend used by **all** sessions in the current Nox execution by using ``-fb <backend>`` or ``--force-venv-backend <backend>``. No exceptions are made, the backend will be forced for all sessions run whatever the other options values and Noxfile configuration. Supported names are ``('none', 'uv', 'virtualenv', 'conda', 'mamba', 'venv')``.
 
 .. code-block:: console
 
diff --git a/nox/_options.py b/nox/_options.py
index 39ebd4c..7c4b89e 100644
--- a/nox/_options.py
+++ b/nox/_options.py
@@ -383,10 +383,10 @@ options.add_options(
         merge_func=_default_venv_backend_merge_func,
         help=(
             "Virtual environment backend to use by default for Nox sessions, this is"
-            " ``'virtualenv'`` by default but any of ``('virtualenv', 'conda', 'mamba',"
-            " 'venv')`` are accepted."
+            " ``'virtualenv'`` by default but any of ``('uv, 'virtualenv',"
+            " 'conda', 'mamba', 'venv')`` are accepted."
         ),
-        choices=["none", "virtualenv", "conda", "mamba", "venv"],
+        choices=["none", "virtualenv", "conda", "mamba", "venv", "uv"],
     ),
     _option_set.Option(
         "force_venv_backend",
@@ -398,10 +398,10 @@ options.add_options(
         help=(
             "Virtual environment backend to force-use for all Nox sessions in this run,"
             " overriding any other venv backend declared in the Noxfile and ignoring"
-            " the default backend. Any of ``('virtualenv', 'conda', 'mamba', 'venv')``"
-            " are accepted."
+            " the default backend. Any of ``('uv', 'virtualenv', 'conda', 'mamba',"
+            " 'venv')`` are accepted."
         ),
-        choices=["none", "virtualenv", "conda", "mamba", "venv"],
+        choices=["none", "virtualenv", "conda", "mamba", "venv", "uv"],
     ),
     _option_set.Option(
         "no_venv",
diff --git a/nox/sessions.py b/nox/sessions.py
index 02e1e69..f29e6bf 100644
--- a/nox/sessions.py
+++ b/nox/sessions.py
@@ -650,7 +650,12 @@ class Session:
         if "silent" not in kwargs:
             kwargs["silent"] = True
 
-        self._run("python", "-m", "pip", "install", *args, external="error", **kwargs)
+        if isinstance(venv, VirtualEnv) and venv.venv_backend == "uv":
+            self._run("uv", "pip", "install", *args, external="error", **kwargs)
+        else:
+            self._run(
+                "python", "-m", "pip", "install", *args, external="error", **kwargs
+            )
 
     def notify(
         self,
@@ -766,11 +771,12 @@ class SessionRunner:
             self.func.reuse_venv or self.global_config.reuse_existing_virtualenvs
         )
 
-        if backend is None or backend == "virtualenv":
+        if backend is None or backend in {"virtualenv", "venv", "uv"}:
             self.venv = VirtualEnv(
                 self.envdir,
                 interpreter=self.func.python,  # type: ignore[arg-type]
                 reuse_existing=reuse_existing,
+                venv_backend=backend or "virtualenv",
                 venv_params=self.func.venv_params,
             )
         elif backend in {"conda", "mamba"}:
@@ -781,14 +787,6 @@ class SessionRunner:
                 venv_params=self.func.venv_params,
                 conda_cmd=backend,
             )
-        elif backend == "venv":
-            self.venv = VirtualEnv(
-                self.envdir,
-                interpreter=self.func.python,  # type: ignore[arg-type]
-                reuse_existing=reuse_existing,
-                venv=True,
-                venv_params=self.func.venv_params,
-            )
         else:
             raise ValueError(
                 "Expected venv_backend one of ('virtualenv', 'conda', 'mamba',"
diff --git a/nox/virtualenv.py b/nox/virtualenv.py
index 431e8f2..d11db74 100644
--- a/nox/virtualenv.py
+++ b/nox/virtualenv.py
@@ -312,6 +312,7 @@ class VirtualEnv(ProcessEnv):
     """
 
     is_sandboxed = True
+    allowed_globals = ("uv",)
 
     def __init__(
         self,
@@ -319,7 +320,7 @@ class VirtualEnv(ProcessEnv):
         interpreter: str | None = None,
         reuse_existing: bool = False,
         *,
-        venv: bool = False,
+        venv_backend: str = "virtualenv",
         venv_params: Any = None,
     ):
         self.location_name = location
@@ -327,7 +328,7 @@ class VirtualEnv(ProcessEnv):
         self.interpreter = interpreter
         self._resolved: None | str | InterpreterNotFound = None
         self.reuse_existing = reuse_existing
-        self.venv_or_virtualenv = "venv" if venv else "virtualenv"
+        self.venv_backend = venv_backend
         self.venv_params = venv_params or []
         super().__init__(env={"VIRTUAL_ENV": self.location})
 
@@ -349,17 +350,21 @@ class VirtualEnv(ProcessEnv):
 
     def _check_reused_environment_type(self) -> bool:
         """Check if reused environment type is the same."""
-        path = os.path.join(self.location, "pyvenv.cfg")
-        if not os.path.isfile(path):
+        try:
+            with open(os.path.join(self.location, "pyvenv.cfg")) as fp:
+                parts = (x.partition("=") for x in fp if "=" in x)
+                config = {k.strip(): v.strip() for k, _, v in parts}
+            if "uv" in config or "gourgeist" in config:
+                old_env = "uv"
+            elif "virtualenv" in config:
+                old_env = "virtualenv"
+            else:
+                old_env = "venv"
+        except FileNotFoundError:  # pragma: no cover
             # virtualenv < 20.0 does not create pyvenv.cfg
             old_env = "virtualenv"
-        else:
-            pattern = re.compile("virtualenv[ \t]*=")
-            with open(path) as fp:
-                old_env = (
-                    "virtualenv" if any(pattern.match(line) for line in fp) else "venv"
-                )
-        return old_env == self.venv_or_virtualenv
+
+        return old_env == self.venv_backend
 
     def _check_reused_environment_interpreter(self) -> bool:
         """Check if reused environment interpreter is the same."""
@@ -474,10 +479,18 @@ class VirtualEnv(ProcessEnv):
 
             return False
 
-        if self.venv_or_virtualenv == "virtualenv":
+        if self.venv_backend == "virtualenv":
             cmd = [sys.executable, "-m", "virtualenv", self.location]
             if self.interpreter:
                 cmd.extend(["-p", self._resolved_interpreter])
+        elif self.venv_backend == "uv":
+            cmd = [
+                "uv",
+                "venv",
+                "-p",
+                self._resolved_interpreter if self.interpreter else sys.executable,
+                self.location,
+            ]
         else:
             cmd = [self._resolved_interpreter, "-m", "venv", self.location]
         cmd.extend(self.venv_params)
@@ -485,7 +498,7 @@ class VirtualEnv(ProcessEnv):
         resolved_interpreter_name = os.path.basename(self._resolved_interpreter)
 
         logger.info(
-            f"Creating virtual environment ({self.venv_or_virtualenv}) using"
+            f"Creating virtual environment ({self.venv_backend}) using"
             f" {resolved_interpreter_name} in {self.location_name}"
         )
         nox.command.run(cmd, silent=True, log=nox.options.verbose or False)
diff --git a/pyproject.toml b/pyproject.toml
index e7cda19..f3dc726 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -52,6 +52,9 @@ tox_to_nox = [
   "jinja2",
   "tox",
 ]
+uv = [
+  "uv",
+]
 [project.urls]
 bug-tracker = "https://github.com/wntrblm/nox/issues"
 documentation = "https://nox.thea.codes"


Test patch:
diff --git a/requirements-test.txt b/requirements-test.txt
index 9886e7a..eb05f07 100644
--- a/requirements-test.txt
+++ b/requirements-test.txt
@@ -6,4 +6,5 @@ pytest-cov
 sphinx>=3.0
 sphinx-autobuild
 sphinx-tabs
+uv; python_version>='3.8'
 witchhazel
diff --git a/tests/test_sessions.py b/tests/test_sessions.py
index 1926a33..4b83146 100644
--- a/tests/test_sessions.py
+++ b/tests/test_sessions.py
@@ -79,6 +79,7 @@ class TestSession:
         runner.venv = mock.create_autospec(nox.virtualenv.VirtualEnv)
         runner.venv.env = {}
         runner.venv.bin_paths = ["/no/bin/for/you"]
+        runner.venv.venv_backend = "venv"
         return nox.sessions.Session(runner=runner), runner
 
     def test_create_tmp(self):
@@ -633,6 +634,7 @@ class TestSession:
         )
         runner.venv = mock.create_autospec(nox.virtualenv.VirtualEnv)
         runner.venv.env = {}
+        runner.venv.venv_backend = "venv"
 
         class SessionNoSlots(nox.sessions.Session):
             pass
@@ -662,6 +664,7 @@ class TestSession:
         )
         runner.venv = mock.create_autospec(nox.virtualenv.VirtualEnv)
         runner.venv.env = {}
+        runner.venv.venv_backend = "venv"
 
         class SessionNoSlots(nox.sessions.Session):
             pass
@@ -798,6 +801,35 @@ class TestSession:
 
         assert run.called is run_called
 
+    def test_install_uv(self):
+        runner = nox.sessions.SessionRunner(
+            name="test",
+            signatures=["test"],
+            func=mock.sentinel.func,
+            global_config=_options.options.namespace(posargs=[]),
+            manifest=mock.create_autospec(nox.manifest.Manifest),
+        )
+        runner.venv = mock.create_autospec(nox.virtualenv.VirtualEnv)
+        runner.venv.env = {}
+        runner.venv.venv_backend = "uv"
+
+        class SessionNoSlots(nox.sessions.Session):
+            pass
+
+        session = SessionNoSlots(runner=runner)
+
+        with mock.patch.object(session, "_run", autospec=True) as run:
+            session.install("requests", "urllib3", silent=False)
+            run.assert_called_once_with(
+                "uv",
+                "pip",
+                "install",
+                "requests",
+                "urllib3",
+                silent=False,
+                external="error",
+            )
+
     def test___slots__(self):
         session, _ = self.make_session_and_runner()
         with pytest.raises(AttributeError):
diff --git a/tests/test_virtualenv.py b/tests/test_virtualenv.py
index 1578c58..b5cf49a 100644
--- a/tests/test_virtualenv.py
+++ b/tests/test_virtualenv.py
@@ -31,6 +31,7 @@ import nox.virtualenv
 
 IS_WINDOWS = nox.virtualenv._SYSTEM == "Windows"
 HAS_CONDA = shutil.which("conda") is not None
+HAS_UV = shutil.which("uv") is not None
 RAISE_ERROR = "RAISE_ERROR"
 VIRTUALENV_VERSION = virtualenv.__version__
 
@@ -240,12 +241,24 @@ def test_condaenv_detection(make_conda):
     assert path_regex.search(output).group("env_dir") == dir_.strpath
 
 
+@pytest.mark.skipif(not HAS_UV, reason="Missing uv command.")
+def test_uv_creation(make_one):
+    venv, _ = make_one(venv_backend="uv")
+    assert venv.location
+    assert venv.interpreter is None
+    assert venv.reuse_existing is False
+    assert venv.venv_backend == "uv"
+
+    venv.create()
+    assert venv._check_reused_environment_type()
+
+
 def test_constructor_defaults(make_one):
     venv, _ = make_one()
     assert venv.location
     assert venv.interpreter is None
     assert venv.reuse_existing is False
-    assert venv.venv_or_virtualenv == "virtualenv"
+    assert venv.venv_backend == "virtualenv"
 
 
 @pytest.mark.skipif(IS_WINDOWS, reason="Not testing multiple interpreters on Windows.")
@@ -417,7 +430,7 @@ def test_create_reuse_stale_venv_environment(make_one):
 
 @enable_staleness_check
 def test_create_reuse_stale_virtualenv_environment(make_one):
-    venv, location = make_one(reuse_existing=True, venv=True)
+    venv, location = make_one(reuse_existing=True, venv_backend="venv")
     venv.create()
 
     # Drop a virtualenv-style pyvenv.cfg into the environment.
@@ -442,7 +455,7 @@ def test_create_reuse_stale_virtualenv_environment(make_one):
 
 @enable_staleness_check
 def test_create_reuse_venv_environment(make_one):
-    venv, location = make_one(reuse_existing=True, venv=True)
+    venv, location = make_one(reuse_existing=True, venv_backend="venv")
     venv.create()
 
     # Place a spurious occurrence of "virtualenv" in the pyvenv.cfg.
@@ -516,7 +529,7 @@ def test_create_reuse_python2_environment(make_one):
 
 
 def test_create_venv_backend(make_one):
-    venv, dir_ = make_one(venv=True)
+    venv, dir_ = make_one(venv_backend="venv")
     venv.create()