Repository: wntrblm/nox
Pull request #842 (2024-08-27 22:06:31)

feat: support for uv-installed pythons
A question more than a PR, now that `uv` supports bringing its own pythons, i wanted to try simplify a CI and not have to install python from other sources, but `nox` is currently now able to find uv-installed pythons  (someone else's issue @uv about this same thing  https://github.com/astral-sh/uv/issues/6579 )

would it make sense to add a more fleshed out version of this? meaning passing the desired interpreter to `uv venv -p {interpreter}`, or should this be done by other mechanism that adds uv's python interpreters to `$PATH` ?


note: this is just a draft to show an example, kind of works but also breaks tests and probably breaks `uv|virtualenv` and other things

henryiii: I think it makes sense, especially if it's roughly this easy. :)
saucoide: @henryiii i've updated this a bit, but there is a failing test locally `test__resolved_interpreter_windows_path_and_version_fails` which i dont quite understand though :/ 
saucoide: im not happy about ignoring the uv-python cases for those windows tests, but i am not familiar with how nox works on windows to touch that part, so if anyone has any ideas it'll be helpful
nugend: @saucoide Just want to express my appreciation for this PR. This has already made my life *way* easier!
saucoide: thanks but tbh i wouldn't use it just yet 😅 there is still something wrong with it, the tox-to-nox tests fails mixing up the interpreter depending on which order the tests are run, so there is something broken still
saucoide: ok so this may be an issue with uv (or with how i thought it would behave) looks like if you run

`uv python install pypy3.8`  (there is a test that does it)

and then later on

`uv venv -p python3.8 .venv`

uv will go ahead and use the existing pypy to make the venv, will try to figure out if this is intended from uv's side or not
henryiii: I wonder if we could make pythonX -> cpythonX for uv? Just a thought.
saucoide: > I wonder if we could make pythonX -> cpythonX for uv? Just a thought.

yes I think we might have to, originally i thought passing it as-is to uv would be nice, because you can then specify the python with uv's own version syntax, and whatever versions they support e.g. `3.13t`, without having to do any translation. But it looks like it it might not work out, depending on what they do here https://github.com/astral-sh/uv/issues/7118

ill try to test tomorrow with replacing python/cpython
saucoide: looks the uv people were faster so it works without changes now, this latest version seems ok 
saucoide: @henryiii could you review and let me know if you think this approach is ok?
henryiii: I think this is great, my only question is if we should have some control for this. We could in theory add downloading to the other backends (similar to how PDM and Hatch download Pythons - and it comes from the same place as uv's), so maybe we should have `--download-python=auto,always,never`? Not necessary for this PR unless the default wasn't auto.

Patch:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c26408d..c684e5e 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -19,7 +19,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-20.04, windows-latest, macos-13]
+        os: [ubuntu-22.04, windows-latest, macos-13]
         python-version:
           - "3.8"
           - "3.9"
@@ -38,13 +38,12 @@ jobs:
           python-version: ${{ matrix.python-version }}
           allow-prereleases: true
       - name: Setup uv
-        uses: yezz123/setup-uv@v4
+        uses: astral-sh/setup-uv@v3
       - name: Set up Miniconda
         uses: conda-incubator/setup-miniconda@v3
         with:
           auto-update-conda: true
           python-version: ${{ matrix.python-version }}
-          miniforge-variant: Mambaforge
           use-mamba: true
           channels: conda-forge/label/python_rc,conda-forge
       - name: Install Nox-under-test (uv)
@@ -71,7 +70,7 @@ jobs:
         with:
           python-version: "3.12"
       - name: Setup uv
-        uses: yezz123/setup-uv@v4
+        uses: astral-sh/setup-uv@v3
       - name: Install Nox-under-test
         run:  uv pip install --system .
       - name: Download individual coverage reports
@@ -105,7 +104,7 @@ jobs:
         with:
           python-version: "3.12"
       - name: Setup uv
-        uses: yezz123/setup-uv@v4
+        uses: astral-sh/setup-uv@v3
       - name: Install Nox-under-test
         run:  uv pip install --system .
       - name: Docs
diff --git a/docs/config.rst b/docs/config.rst
index 1a08aa6..04c0997 100644
--- a/docs/config.rst
+++ b/docs/config.rst
@@ -397,6 +397,63 @@ Pythons:
         ...
 
 
+Assigning tags to parametrized sessions
+---------------------------------------
+
+Just as tags can be :ref:`assigned to normal sessions <session tags>`, they can also be assigned to parametrized sessions.  The following examples are both equivalent:
+
+.. code-block:: python
+
+    @nox.session
+    @nox.parametrize('dependency',
+        ['1.0', '2.0'],
+        tags=[['old'], ['new']])
+    @nox.parametrize('database'
+        ['postgres', 'mysql'],
+        tags=[['psql'], ['mysql']])
+    def tests(session, dependency, database):
+        ...
+
+.. code-block:: python
+
+    @nox.session
+    @nox.parametrize('dependency', [
+        nox.param('1.0', tags=['old']),
+        nox.param('2.0', tags=['new']),
+    ])
+    @nox.parametrize('database', [
+        nox.param('postgres', tags=['psql']),
+        nox.param('mysql', tags=['mysql']),
+    ])
+    def tests(session, dependency, database):
+        ...
+
+In either case, running ``nox --tags old`` will run the tests using version 1.0 of the dependency against both database backends, while running ``nox --tags psql`` will run the tests using both versions of the dependency, but only against PostgreSQL.
+
+More sophisticated tag assignment can be performed by passing a generator to the ``@nox.parametrize`` decorator, as seen in the following example:
+
+.. code-block:: python
+
+    def generate_params():
+        for dependency in ["1.0", "1.1", "2.0"]:
+            for database in ["sqlite", "postgresql", "mysql"]:
+                tags = []
+                if dependency == "2.0" and database == "sqlite":
+                    tags.append("quick")
+                if dependency == "2.0" or database == "sqlite":
+                    tags.append("standard")
+                yield nox.param((dependency, database), tags)
+
+    @nox.session
+    @nox.parametrize(
+        ["dependency", "database"], generate_params(),
+    )
+    def tests(session, dependency, database):
+        ...
+
+In this example, the ``quick`` tag is assigned to the single combination of the latest version of the dependency along with the SQLite database backend, allowing a developer to run the tests in a single configuration as a basic sanity test.  The ``standard`` tag, in contrast, selects combinations targeting either the latest version of the dependency *or* the SQLite database backend.  If the developer runs ``tox --tags standard``, the tests will be run against all supported versions of the dependency with the SQLite backend, as well as against all supported database backends under the latest version of the dependency, giving much more comprehensive test coverage while using only five of the potential nine test matrix combinations.
+
+
 The session object
 ------------------
 
diff --git a/docs/cookbook.rst b/docs/cookbook.rst
index 7489b70..eb93fdd 100644
--- a/docs/cookbook.rst
+++ b/docs/cookbook.rst
@@ -40,6 +40,7 @@ Enter the ``dev`` nox session:
 .. code-block:: python
 
     import os
+    import pathlib
 
     import nox
 
@@ -87,6 +88,7 @@ Even more so with a sprinkling of Nox:
 
 .. code-block:: python
 
+    import argparse
     import nox
 
     @nox.session
diff --git a/docs/tutorial.rst b/docs/tutorial.rst
index 75677dd..2634bb4 100644
--- a/docs/tutorial.rst
+++ b/docs/tutorial.rst
@@ -496,6 +496,8 @@ read more about parametrization and see more examples over at
 .. _pytest's parametrize: https://pytest.org/latest/parametrize.html#_pytest.python.Metafunc.parametrize
 
 
+.. _session tags:
+
 Session tags
 ------------
 
diff --git a/nox/_decorators.py b/nox/_decorators.py
index 25f13de..c21f166 100644
--- a/nox/_decorators.py
+++ b/nox/_decorators.py
@@ -128,7 +128,7 @@ class Call(Func):
             func.venv_backend,
             func.venv_params,
             func.should_warn,
-            func.tags,
+            func.tags + param_spec.tags,
             default=func.default,
         )
         self.call_spec = call_spec
diff --git a/nox/_parametrize.py b/nox/_parametrize.py
index 865cb19..24b8786 100644
--- a/nox/_parametrize.py
+++ b/nox/_parametrize.py
@@ -29,6 +29,8 @@ class Param:
         arg_names (Sequence[str]): The names of the args.
         id (str): An optional ID for this set of parameters. If unspecified,
             it will be generated from the parameters.
+        tags (Sequence[str]): Optional tags to associate with this set of
+            parameters.
     """
 
     def __init__(
@@ -36,6 +38,7 @@ class Param:
         *args: Any,
         arg_names: Sequence[str] | None = None,
         id: str | None = None,
+        tags: Sequence[str] | None = None,
     ) -> None:
         self.args = args
         self.id = id
@@ -45,6 +48,11 @@ class Param:
 
         self.arg_names = tuple(arg_names)
 
+        if tags is None:
+            tags = []
+
+        self.tags = list(tags)
+
     @property
     def call_spec(self) -> dict[str, Any]:
         return dict(zip(self.arg_names, self.args))
@@ -60,13 +68,16 @@ class Param:
     __repr__ = __str__
 
     def copy(self) -> Param:
-        new = self.__class__(*self.args, arg_names=self.arg_names, id=self.id)
+        new = self.__class__(
+            *self.args, arg_names=self.arg_names, id=self.id, tags=self.tags
+        )
         return new
 
     def update(self, other: Param) -> None:
         self.id = ", ".join([str(self), str(other)])
         self.args = self.args + other.args
         self.arg_names = self.arg_names + other.arg_names
+        self.tags = self.tags + other.tags
 
     def __eq__(self, other: object) -> bool:
         if isinstance(other, self.__class__):
@@ -74,6 +85,7 @@ class Param:
                 self.args == other.args
                 and self.arg_names == other.arg_names
                 and self.id == other.id
+                and self.tags == other.tags
             )
         elif isinstance(other, dict):
             return dict(zip(self.arg_names, self.args)) == other
@@ -95,6 +107,7 @@ def parametrize_decorator(
     arg_names: str | Sequence[str],
     arg_values_list: Iterable[ArgValue] | ArgValue,
     ids: Iterable[str | None] | None = None,
+    tags: Iterable[Sequence[str]] | None = None,
 ) -> Callable[[Any], Any]:
     """Parametrize a session.
 
@@ -114,6 +127,8 @@ def parametrize_decorator(
             argument name, for example ``[(1, 'a'), (2, 'b')]``.
         ids (Sequence[str]): Optional sequence of test IDs to use for the
             parametrized arguments.
+        tags (Iterable[Sequence[str]]): Optional iterable of tags to associate
+            with the parametrized arguments.
     """
 
     # Allow args names to be specified as any of 'arg', 'arg,arg2' or ('arg', 'arg2')
@@ -143,14 +158,21 @@ def parametrize_decorator(
     if not ids:
         ids = []
 
+    if tags is None:
+        tags = []
+
     # Generate params for each item in the param_args_values list.
     param_specs: list[Param] = []
-    for param_arg_values, param_id in itertools.zip_longest(_arg_values_list, ids):
+    for param_arg_values, param_id, param_tags in itertools.zip_longest(
+        _arg_values_list, ids, tags
+    ):
         if isinstance(param_arg_values, Param):
             param_spec = param_arg_values
             param_spec.arg_names = tuple(arg_names)
         else:
-            param_spec = Param(*param_arg_values, arg_names=arg_names, id=param_id)
+            param_spec = Param(
+                *param_arg_values, arg_names=arg_names, id=param_id, tags=param_tags
+            )
 
         param_specs.append(param_spec)
 
diff --git a/nox/virtualenv.py b/nox/virtualenv.py
index adf6033..aa5f3ed 100644
--- a/nox/virtualenv.py
+++ b/nox/virtualenv.py
@@ -17,6 +17,7 @@ from __future__ import annotations
 import abc
 import contextlib
 import functools
+import json
 import os
 import platform
 import re
@@ -28,6 +29,8 @@ from pathlib import Path
 from socket import gethostbyname
 from typing import Any, ClassVar
 
+from packaging import version
+
 import nox
 import nox.command
 from nox.logger import logger
@@ -65,7 +68,39 @@ def find_uv() -> tuple[bool, str]:
     return uv_on_path is not None, "uv"
 
 
+def uv_version() -> version.Version:
+    """Returns uv's version defaulting to 0.0 if uv is not available"""
+    try:
+        ret = subprocess.run(
+            [UV, "version", "--output-format", "json"],
+            check=False,
+            text=True,
+            capture_output=True,
+        )
+    except FileNotFoundError:
+        logger.info("uv binary not found.")
+        return version.Version("0.0")
+
+    if ret.returncode == 0 and ret.stdout:
+        return version.Version(json.loads(ret.stdout).get("version"))
+    else:
+        logger.info("Failed to establish uv's version.")
+        return version.Version("0.0")
+
+
+def uv_install_python(python_version: str) -> bool:
+    """Attempts to install a given python version with uv"""
+    ret = subprocess.run(
+        [UV, "python", "install", python_version],
+        check=False,
+    )
+    return ret.returncode == 0
+
+
 HAS_UV, UV = find_uv()
+# supported since uv 0.3 but 0.4.16 is the first version that doesn't cause
+# issues for nox with pypy/cpython confusion
+UV_PYTHON_SUPPORT = uv_version() >= version.Version("0.4.16")
 
 
 class InterpreterNotFound(OSError):
@@ -526,6 +561,12 @@ class VirtualEnv(ProcessEnv):
             self._resolved = cleaned_interpreter
             return self._resolved
 
+        if HAS_UV and UV_PYTHON_SUPPORT:
+            uv_python_success = uv_install_python(cleaned_interpreter)
+            if uv_python_success:
+                self._resolved = cleaned_interpreter
+                return self._resolved
+
         # The rest of this is only applicable to Windows, so if we don't have
         # an interpreter by now, raise.
         if _SYSTEM != "Windows":


Test patch:
diff --git a/tests/resources/noxfile_tags.py b/tests/resources/noxfile_tags.py
index 960ec52..10c85fa 100644
--- a/tests/resources/noxfile_tags.py
+++ b/tests/resources/noxfile_tags.py
@@ -16,3 +16,10 @@ def one_tag(unused_session):
 @nox.session(tags=["tag1", "tag2", "tag3"])
 def more_tags(unused_session):
     print("Some more tags here.")
+
+
+@nox.session(tags=["tag4"])
+@nox.parametrize("foo", [nox.param(1, tags=["tag5", "tag6"])])
+@nox.parametrize("bar", [2, 3], tags=[["tag7"]])
+def parametrized_tags(unused_session):
+    print("Parametrized tags here.")
diff --git a/tests/test__option_set.py b/tests/test__option_set.py
index c474d12..73f7aff 100644
--- a/tests/test__option_set.py
+++ b/tests/test__option_set.py
@@ -127,5 +127,5 @@ class TestOptionSet:
             prefix=None, parsed_args=parsed_args
         )
 
-        expected_tags = {"tag1", "tag2", "tag3"}
+        expected_tags = {f"tag{n}" for n in range(1, 8)}
         assert expected_tags == set(actual_tags_from_file)
diff --git a/tests/test__parametrize.py b/tests/test__parametrize.py
index 41ea423..a09c6e1 100644
--- a/tests/test__parametrize.py
+++ b/tests/test__parametrize.py
@@ -197,7 +197,7 @@ def test_generate_calls_simple():
 
 
 def test_generate_calls_multiple_args():
-    f = mock.Mock(should_warn=None, tags=None)
+    f = mock.Mock(should_warn=None, tags=[])
     f.__name__ = "f"
 
     arg_names = ("foo", "abc")
@@ -244,6 +244,44 @@ def test_generate_calls_ids():
     f.assert_called_with(foo=2)
 
 
+def test_generate_calls_tags():
+    f = mock.Mock(should_warn={}, tags=[])
+    f.__name__ = "f"
+
+    arg_names = ("foo",)
+    call_specs = [
+        _parametrize.Param(1, arg_names=arg_names, tags=["tag3"]),
+        _parametrize.Param(1, arg_names=arg_names),
+        _parametrize.Param(2, arg_names=arg_names, tags=["tag4", "tag5"]),
+    ]
+
+    calls = _decorators.Call.generate_calls(f, call_specs)
+
+    assert len(calls) == 3
+    assert calls[0].tags == ["tag3"]
+    assert calls[1].tags == []
+    assert calls[2].tags == ["tag4", "tag5"]
+
+
+def test_generate_calls_merge_tags():
+    f = mock.Mock(should_warn={}, tags=["tag1", "tag2"])
+    f.__name__ = "f"
+
+    arg_names = ("foo",)
+    call_specs = [
+        _parametrize.Param(1, arg_names=arg_names, tags=["tag3"]),
+        _parametrize.Param(1, arg_names=arg_names),
+        _parametrize.Param(2, arg_names=arg_names, tags=["tag4", "tag5"]),
+    ]
+
+    calls = _decorators.Call.generate_calls(f, call_specs)
+
+    assert len(calls) == 3
+    assert calls[0].tags == ["tag1", "tag2", "tag3"]
+    assert calls[1].tags == ["tag1", "tag2"]
+    assert calls[2].tags == ["tag1", "tag2", "tag4", "tag5"]
+
+
 def test_generate_calls_session_python():
     called_with = []
 
diff --git a/tests/test_tasks.py b/tests/test_tasks.py
index 122cc0e..fdfad62 100644
--- a/tests/test_tasks.py
+++ b/tests/test_tasks.py
@@ -254,13 +254,14 @@ def test_filter_manifest_keywords_syntax_error():
 @pytest.mark.parametrize(
     "tags,session_count",
     [
-        (None, 4),
-        (["foo"], 3),
-        (["bar"], 3),
-        (["baz"], 1),
-        (["foo", "bar"], 4),
-        (["foo", "baz"], 3),
-        (["foo", "bar", "baz"], 4),
+        (None, 8),
+        (["foo"], 7),
+        (["bar"], 5),
+        (["baz"], 3),
+        (["foo", "bar"], 8),
+        (["foo", "baz"], 7),
+        (["bar", "baz"], 6),
+        (["foo", "bar", "baz"], 8),
     ],
 )
 def test_filter_manifest_tags(tags, session_count):
@@ -280,6 +281,12 @@ def test_filter_manifest_tags(tags, session_count):
     def corge():
         pass
 
+    @nox.session(tags=["foo"])
+    @nox.parametrize("a", [1, nox.param(2, tags=["bar"])])
+    @nox.parametrize("b", [3, 4], tags=[["baz"]])
+    def grault():
+        pass
+
     config = _options.options.namespace(
         sessions=None, pythons=(), posargs=[], tags=tags
     )
@@ -289,6 +296,7 @@ def test_filter_manifest_tags(tags, session_count):
             "quux": quux,
             "quuz": quuz,
             "corge": corge,
+            "grault": grault,
         },
         config,
     )
diff --git a/tests/test_virtualenv.py b/tests/test_virtualenv.py
index fd5bfc9..962f3e9 100644
--- a/tests/test_virtualenv.py
+++ b/tests/test_virtualenv.py
@@ -269,6 +269,11 @@ def test_uv_creation(make_one):
     assert venv._check_reused_environment_type()
 
 
+@has_uv
+def test_uv_managed_python(make_one):
+    make_one(interpreter="cpython3.12", venv_backend="uv")
+
+
 def test_constructor_defaults(make_one):
     venv, _ = make_one()
     assert venv.location
@@ -620,6 +625,50 @@ def test_find_uv(monkeypatch, which_result, find_uv_bin_result, found, path):
     assert nox.virtualenv.find_uv() == (found, path)
 
 
+@pytest.mark.parametrize(
+    ["return_code", "stdout", "expected_result"],
+    [
+        (0, '{"version": "0.2.3", "commit_info": null}', "0.2.3"),
+        (1, None, "0.0"),
+        (1, '{"version": "9.9.9", "commit_info": null}', "0.0"),
+    ],
+)
+def test_uv_version(monkeypatch, return_code, stdout, expected_result):
+    def mock_run(*args, **kwargs):
+        return subprocess.CompletedProcess(
+            args=["uv", "version", "--output-format", "json"],
+            stdout=stdout,
+            returncode=return_code,
+        )
+
+    monkeypatch.setattr(subprocess, "run", mock_run)
+    assert nox.virtualenv.uv_version() == version.Version(expected_result)
+
+
+def test_uv_version_no_uv(monkeypatch):
+    def mock_exception(*args, **kwargs):
+        raise FileNotFoundError
+
+    monkeypatch.setattr(subprocess, "run", mock_exception)
+    assert nox.virtualenv.uv_version() == version.Version("0.0")
+
+
+@pytest.mark.parametrize(
+    ["requested_python", "expected_result"],
+    [
+        ("3.11", True),
+        ("pypy3.8", True),
+        ("cpython3.9", True),
+        ("python3.12", True),
+        ("nonpython9.22", False),
+        ("java11", False),
+    ],
+)
+@has_uv
+def test_uv_install(requested_python, expected_result):
+    assert nox.virtualenv.uv_install_python(requested_python) == expected_result
+
+
 def test_create_reuse_venv_environment(make_one, monkeypatch):
     # Making the reuse requirement more strict
     monkeypatch.setenv("NOX_ENABLE_STALENESS_CHECK", "1")
@@ -840,6 +889,7 @@ def test__resolved_interpreter_windows_pyexe_fails(which, run, make_one):
 
 
 @mock.patch("nox.virtualenv._SYSTEM", new="Windows")
+@mock.patch("nox.virtualenv.UV_PYTHON_SUPPORT", new=False)
 def test__resolved_interpreter_windows_path_and_version(make_one, patch_sysfind):
     # Establish that if we get a standard pythonX.Y path, we look it
     # up via the path on Windows.
@@ -865,6 +915,7 @@ def test__resolved_interpreter_windows_path_and_version(make_one, patch_sysfind)
 @pytest.mark.parametrize("sysfind_result", [r"c:\python37-x64\python.exe", None])
 @pytest.mark.parametrize("sysexec_result", ["3.7.3\\n", RAISE_ERROR])
 @mock.patch("nox.virtualenv._SYSTEM", new="Windows")
+@mock.patch("nox.virtualenv.UV_PYTHON_SUPPORT", new=False)
 def test__resolved_interpreter_windows_path_and_version_fails(
     input_, sysfind_result, sysexec_result, make_one, patch_sysfind
 ):