Repository: tox-dev/tox

Issue:
Numeric factor in environment name wrongly causes "conflicting factors" error
https://github.com/tox-dev/tox/pull/2597 had the unintended effect of interpreting a numeric factor (e.g. `2105`) as a candidate base python, breaking previously valid environment names (e.g. `py37-foo-2105`) by raising `ValueError: conflicting factors py37, 2105 in py37-foo-2105`.
Example GitHub workflow build: https://github.com/galaxyproject/planemo/actions/runs/3654111042/jobs/6174245903 for `tox.ini` https://github.com/galaxyproject/planemo/blob/nsoranzo-patch-1/tox.ini

_Originally posted by @nsoranzo in https://github.com/tox-dev/tox/pull/2597#discussion_r1044065606_

Discussion:
stephenfin: Other points:

- There's no technical reason we couldn't support python factors with periods in them (e.g. `py3.8`) though it was never supported before tox 4 and I don't really get _why_ you'd want this extra complexity
- We could probably expand the list of interpreter prefixes to include e.g. `cpthon`, `ironpython` (is that still a thing?), etc. Again though, it feels like cpython and pypy are the only two players in town these days.
gaborbernat: > * There's no technical reason we couldn't support python factors with periods in them (e.g. `py3.8`) though it was never supported before tox 4 and I don't really get _why_ you'd want this extra complexity

We do want this. This will make Github actions simple to setup because you can use the same spec for loading the python version in github as the env you need to run. Removing the gymanstic needed for tox 3 to map 3.8 to py38 👍 This feature will make CI configs much simpler so I'll need to insist on it.

> * We could probably expand the list of interpreter prefixes to include e.g. `cpython`, `ironpython` (is that still a thing?), etc. Again though, it feels like cpython and pypy are the only two players in town these days so you'd have to question what this gives you.

Today perhaps. There's also rustpython. I don't want to be the gatekeeper of what's reasonable python implementation. We also have ironpython and JVM pythons too. Most people don't use this, but I want tox to allow using these.

We need to be permissive to make people's lives easier, even if they are not in majority.
stephenfin: > > * There's no technical reason we couldn't support python factors with periods in them (e.g. `py3.8`) though it was never supported before tox 4 and I don't really get _why_ you'd want this extra complexity
> 
> We do want this. This will make Github actions simple to setup because you can use the same spec for loading the python version in github as the env you need to run. Removing the gymanstic needed for tox 3 to map 3.8 to py38 +1 This feature will make CI configs much simpler so I'll need to insist on it. See e.g. [`main`/.github/workflows/check.yml#L20-L25](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-10T18%3A45%3A56Z#L20-L25) and then [`main`/.github/workflows/check.yml#L43-L50](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-10T18%3A45%3A56Z#L43-L50) we need to do just because tox does not support `3.8` as env name and spec. So yes, `tox -e 3.8` should work.

Okay, so we need to incorporate periods into the env names. Given it doesn't break the *hyphens as delimiters* idea, I could live with that.

> > * We could probably expand the list of interpreter prefixes to include e.g. `cpython`, `ironpython` (is that still a thing?), etc. Again though, it feels like cpython and pypy are the only two players in town these days so you'd have to question what this gives you.
> 
> Today perhaps. There's also rustpython. I don't want to be the gatekeeper of what's reasonable python implementation.

`packaging` is already gatekeeping via  [`INTERPRETER_SHORT_NAMES` from `packaging.tags`](https://github.com/pypa/packaging/blob/63e657138d30063bd46ed16a2e10cfe279a1d7c3/src/packaging/tags.py#L31-L37) so I don't think this is all that relevant?

(BTW, `rustpython` isn't listed in that dict so you might need to add that?)

> We also have ironpython and JVM pythons too. Most people don't use this, but I want tox to allow using these.
> 
> We need to be permissive to make people's lives easier, even if they are not in majority.

So we can't use a wildcard because that would falsely identify e.g. `django32` or `django3.2`, which clearly isn't a Python version. I also don't want to rely on `INTERPRETER_SHORT_NAMES` to build the regex in case it's ever modified to include characters that would lead to an invalid regex (unlikely but possible) or in other ways that could break *tox* (like removing `py`). I guess I could expand our hardcoded list to include these additional interpreters, but it seems silly to do this when we could simply document that these users (who are going to be in a minority of a minority) should simply use `pyXY` factors and rely on `base_python` to support other interpreters, e.g.

```ini
[testenv:py38]
base_python = rustpython3.8
```

We could also introduce a top-level or testenv-level `python_interpreter` setting, e.g.

```ini
[tox]
python_interpreter = rustpython

[testenv]
commands = ...
```

Would this be a reasonable compromise: support `py`, `pypy` and (purely for legacy reasons) `jython` but insist on `base_python` for anything else? We can always expand this regex if a particular interpreter gains enough of a following to warrant first-class support in the future.
stephenfin: > > > * There's no technical reason we couldn't support python factors with periods in them (e.g. `py3.8`) though it was never supported before tox 4 and I don't really get _why_ you'd want this extra complexity
> > 
> > 
> > We do want this. This will make Github actions simple to setup because you can use the same spec for loading the python version in github as the env you need to run. Removing the gymanstic needed for tox 3 to map 3.8 to py38 +1 This feature will make CI configs much simpler so I'll need to insist on it. See e.g. [`main`/.github/workflows/check.yml#L20-L25](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-11T11%3A15%3A27Z#L20-L25) and then [`main`/.github/workflows/check.yml#L43-L50](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-11T11%3A15%3A27Z#L43-L50) we need to do just because tox does not support `3.8` as env name and spec. So yes, `tox -e 3.8` should work.
> 
> Okay, so we need to incorporate periods into the env names. Given it doesn't break the _hyphens as delimiters_ idea, I could live with that.

Oh, wait, you're arguing for `tox -e 3.8` instead of `tox -e py3.8`. That's going to be really error prone. The `py` prefix makes this unambiguous and unlikely to conflict with other factors. We can't say the same for something as arbitrary as `XY`. What are the expected outcomes for these, for example?

- `tox -e 2000`
- `tox -e 4.foo`
- `tox -e 123`
- `tox -e django-23`
- `tox -e py-310`
- `tox -e catch-22`
- `tox -e eslint-8.3`

It's also unnecessary. For one, we could easily do this via a plugin like the old `tox-travis` plugin. A big part of the tox 4 rewrite (as I understand it) was to improve the plugin architecture so use of plugins should be encouraged. You don't need the plugin though: all I've been doing in my CI configurations is running `tox -e py` because `py` implies whatever `sys.executable` points to which, as we all know, is determined by the version `tox` is running (and was installed) under. Install tox under the correct Python version and it'll happily do its job:

```ini
[tox]
testenv = py, linters, coverage, docs, typing
```

```yaml
jobs:
  test:
    runs-on: ubuntu-22.04
    strategy:
      matrix:
        py:
          - "3.11"
          - "3.10"
          - "3.9"
          - "3.8"
          - "3.7"
    steps:
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: ${{ matrix.py }}
      - name: Install tox
        run: python -m pip install tox
      - name: Run unit tests
        run: tox
```
stephenfin: > > * There's no technical reason we couldn't support python factors with periods in them (e.g. `py3.8`) though it was never supported before tox 4 and I don't really get _why_ you'd want this extra complexity
> 
> 
> We do want this. This will make Github actions simple to setup because you can use the same spec for loading the python version in github as the env you need to run. Removing the gymanstic needed for tox 3 to map 3.8 to py38 +1 This feature will make CI configs much simpler so I'll need to insist on it. See e.g. [`main`/.github/workflows/check.yml#L20-L25](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-11T11%3A15%3A27Z#L20-L25) and then [`main`/.github/workflows/check.yml#L43-L50](https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml?rgh-link-date=2023-01-11T11%3A15%3A27Z#L43-L50) we need to do just because tox does not support `3.8` as env name and spec. So yes, `tox -e 3.8` should work.

Another point on this: this only works if you're using simple, single-factor testenvs. It doesn't work when you start taking advantage of combinatorial factors, which is an first class feature of tox.

```ini
[testenv:py{38,39,310}-django{40,41}]
commands = ...
```

The https://github.com/ymyzk/tox-gh-actions project already exists and supports tox 4 (I'm using it in a few projects of my own, funnily enough). It's far more powerful than simple support for e.g. `3.8` factors. Perhaps we could reach out to the author and ask if they'd like to move it to the https://github.com/tox-dev organisation, making it a "blessed" plugin (similar to https://github.com/pypa/setuptools_scm in the https://github.com/pypa organisation)? We could also just link to it from the docs.
stephenfin: Added support for period in Python factors, along with interpreter implementation prefixes for all of the interpreters listed in [`INTERPRETER_SHORT_NAMES` in `packaging.tags`](https://github.com/pypa/packaging/blob/63e657138d30063bd46ed16a2e10cfe279a1d7c3/src/packaging/tags.py#L31-L37) + `rustpython`. Let me know what you think. Happy to follow this up with a patch to reintroduce documentation explaining factors and what's allowed.
gaborbernat: > Would this be a reasonable compromise: support `py`, `pypy` and (purely for legacy reasons) `jython` but insist on `base_python` for anything else? We can always expand this regex if a particular interpreter gains enough of a following to warrant first-class support in the future.

Perhaps we can use for start the same thing packaging defines here ttps://github.com/pypa/packaging/blob/63e657138d30063bd46ed16a2e10cfe279a1d7c3/src/packaging/tags.py#L31-L37?

> Oh, wait, you're arguing for `tox -e 3.8` instead of `tox -e py3.8`

Ideally but I can live with `py3.8` as a compromise.

> It's also unnecessary. For one, we could easily do this via a plugin like the old `tox-travis` plugin. A big part of the tox 4 rewrite (as I understand it) was to improve the plugin architecture: surely the use of plugins should be encouraged?

For this simple use, I'd instead not require users to use a plugin.

>  You don't need the plugin though. All I've been doing in my CI jobs is running `tox -e py`. The `py` environment is special since it implies `base_python` is whatever `sys.executable` points to which, as we all know, is determined by the version `tox` is running (and was installed) under. Install tox under the correct Python version and it'll happily do its job:

This works as long as `tox` support your targeted Python, but what if you want to run your test suite against say `3.6`? I'd rather use a different paradigm, that is more flexible,  use the latest python to run tox, and make only the test env run in the target Python - this is what this project does too, check out https://github.com/tox-dev/tox/blob/main/.github/workflows/check.yml#L30-L43; bonus point for tox can take advantage of speed improvements in newer Pythons even when testing previous versions.

> Is this somehow insufficient? I mean, couldn't we document this as our recommended practice?

We can do that, but I'd add my approach for reasons outlined in my previous paragraph.

> Another point on this: this only works if you're using simple, single-factor testenvs. It doesn't work when you start taking advantage of combinatorial factors, which is an first class feature of tox.

Yeah, but those cases are still, in my experience 20%. They can have more complicated CI configs, but for the 80%, I prefer my approach.

> The [ymyzk/tox-gh-actions](https://github.com/ymyzk/tox-gh-actions) project already exists and supports tox 4 (I'm using it in a few projects of my own, funnily enough). It's far more powerful than simple support for e.g. 3.8 factors. Perhaps we could reach out to the author and ask if they'd like to move it to the [@tox-dev](https://github.com/tox-dev) organisation, making it a "blessed" plugin (similar to [pypa/setuptools_scm](https://github.com/pypa/setuptools_scm) in the [@pypa](https://github.com/pypa) organisation)? We could also just link to it from the docs?

I don't like how that project achieves this, and I don't believe we'd need a plugin for this. So I'm soft against it. [I reached out](https://github.com/ymyzk/tox-gh-actions/issues/74) to the person over a year ago and they refused, so alas 🦐 , hence why we have the competing https://github.com/tox-dev/tox-gh.  But again I'd prefer to not need a plugin for the 80% case.



Patch:
diff --git a/.github/ISSUE_TEMPLATE/feature-request.md b/.github/ISSUE_TEMPLATE/feature-request.md
index d065c219..3d35f697 100644
--- a/.github/ISSUE_TEMPLATE/feature-request.md
+++ b/.github/ISSUE_TEMPLATE/feature-request.md
@@ -18,7 +18,8 @@ assignees: ""
 
 ## Alternative Solutions
 
-<!-- Have you tried to workaround the problem using tox or other tools? Or a different approach to solving this issue? Please elaborate here. -->
+<!-- Have you tried to workaround the problem using tox or other tools? Or a different approach to solving this issue?
+Please elaborate here. -->
 
 ## Additional context
 
diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml
index 43c6d09c..4a25b76c 100644
--- a/.github/workflows/check.yml
+++ b/.github/workflows/check.yml
@@ -40,18 +40,10 @@ jobs:
         uses: actions/setup-python@v4
         with:
           python-version: ${{ matrix.py }}
-      - name: Pick environment to run
-        run: |
-          import os; import platform; import sys; from pathlib import Path
-          env = f'TOXENV=py{"" if platform.python_implementation() == "CPython" else "py"}3{sys.version_info.minor}'
-          print(f"Picked: {env} for {sys.version} based of {sys.executable}")
-          with Path(os.environ["GITHUB_ENV"]).open("ta") as file_handler:
-               file_handler.write(env)
-        shell: python
       - name: Setup test suite
-        run: tox r -vv --notest
+        run: tox r -e py${{ matrix.py }} -vv --notest
       - name: Run test suite
-        run: tox r --skip-pkg-install
+        run: tox r -e py${{ matrix.py }} --skip-pkg-install
         env:
           CI_RUN: "yes"
           DIFF_AGAINST: HEAD
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index d153df00..ebccbdc3 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -43,7 +43,7 @@ repos:
     hooks:
       - id: rst-backticks
   - repo: https://github.com/tox-dev/tox-ini-fmt
-    rev: "0.5.2"
+    rev: "0.6.0"
     hooks:
       - id: tox-ini-fmt
         args: ["-p", "fix"]
@@ -69,7 +69,7 @@ repos:
           - "@prettier/plugin-xml@2.2"
         args: ["--print-width=120", "--prose-wrap=always"]
   - repo: https://github.com/igorshubovych/markdownlint-cli
-    rev: v0.32.2
+    rev: v0.33.0
     hooks:
       - id: markdownlint
   - repo: local
diff --git a/docs/changelog/2848.bugfix.rst b/docs/changelog/2848.bugfix.rst
new file mode 100644
index 00000000..9220c483
--- /dev/null
+++ b/docs/changelog/2848.bugfix.rst
@@ -0,0 +1,6 @@
+tox has reverted support for Python factors that include PATCH release info (e.g. ``py3.10.1``), build architecture
+(e.g. ``pypy3-64``) or do not define a ``py`` prefix or other supported prefix (e.g. ``3.10``). These complex factors
+were initially supported with the release of tox 4.0 but has proven complicated to support. Instead, the simple factors
+supported by tox 3 e.g. (``py310``, ``pypy3``) or period-separated equivalent (``py3.10``) introduced in tox 4 should be
+used. Users who wish to specify more specific Python version information should configure the :ref:`base_python` setting
+- by :user:`stephenfin`.
diff --git a/pyproject.toml b/pyproject.toml
index e4d5ad96..e8be85ac 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
 [build-system]
 build-backend = "hatchling.build"
-requires = ["hatchling>=1.12", "hatch-vcs>=0.3"]
+requires = ["hatchling>=1.12.2", "hatch-vcs>=0.3"]
 
 [project]
 name = "tox"
@@ -22,24 +22,24 @@ maintainers = [
 ]
 requires-python = ">=3.7"
 dependencies = [
-  "cachetools>=5.2",
+  "cachetools>=5.2.1",
   "chardet>=5.1",
   "colorama>=0.4.6",
-  "packaging>=22",
+  "packaging>=23",
   "platformdirs>=2.6.2",
   "pluggy>=1",
-  "pyproject-api>=1.2.1",
+  "pyproject-api>=1.4",
   'tomli>=2.0.1; python_version < "3.11"',
   "virtualenv>=20.17.1",
   "filelock>=3.9",
-  'importlib-metadata>=5.2; python_version < "3.8"',
+  'importlib-metadata>=6; python_version < "3.8"',
   'typing-extensions>=4.4; python_version < "3.8"',
 ]
 optional-dependencies.docs = [
   "furo>=2022.12.7",
-  "sphinx>=6",
-  "sphinx-argparse-cli>=1.10",
-  "sphinx-autodoc-typehints>=1.19.5",
+  "sphinx>=6.1.3",
+  "sphinx-argparse-cli>=1.11",
+  "sphinx-autodoc-typehints>=1.20.1",
   "sphinx-copybutton>=0.5.1",
   "sphinx-inline-tabs>=2022.1.2b11",
   "sphinxcontrib-towncrier>=0.2.1a0",
@@ -53,7 +53,7 @@ optional-dependencies.testing = [
   "distlib>=0.3.6",
   "flaky>=3.7",
   "hatch-vcs>=0.3",
-  "hatchling>=1.12",
+  "hatchling>=1.12.2",
   "psutil>=5.9.4",
   "pytest>=7.2",
   "pytest-cov>=4",
diff --git a/src/tox/tox_env/python/api.py b/src/tox/tox_env/python/api.py
index 271a3803..da9e435f 100644
--- a/src/tox/tox_env/python/api.py
+++ b/src/tox/tox_env/python/api.py
@@ -4,12 +4,12 @@ Declare the abstract base class for tox environments that handle the Python lang
 from __future__ import annotations
 
 import logging
+import re
 import sys
 from abc import ABC, abstractmethod
 from pathlib import Path
 from typing import Any, List, NamedTuple, cast
 
-from packaging.tags import INTERPRETER_SHORT_NAMES
 from virtualenv.discovery.py_spec import PythonSpec
 
 from tox.config.main import Config
@@ -47,6 +47,16 @@ class PythonInfo(NamedTuple):
         return f"{self.version_info.major}.{self.version_info.minor}"
 
 
+PY_FACTORS_RE = re.compile(
+    r"""
+    ^(?!py$)                                               # don't match 'py' as it doesn't provide any info
+    (?P<impl>py|pypy|cpython|jython|rustpython|ironpython) # the interpeter; most users will simply use 'py'
+    (?P<version>[2-9]\.?[0-9]?[0-9]?)?$                    # the version; one of: MAJORMINOR, MAJOR.MINOR
+    """,
+    re.VERBOSE,
+)
+
+
 class Python(ToxEnv, ABC):
     def __init__(self, create_args: ToxEnvCreateArgs) -> None:
         self._base_python: PythonInfo | None = None
@@ -131,9 +141,8 @@ class Python(ToxEnv, ABC):
     def extract_base_python(cls, env_name: str) -> str | None:
         candidates: list[str] = []
         for factor in env_name.split("-"):
-            spec = PythonSpec.from_string_spec(factor)
-            impl = spec.implementation or "python"
-            if impl.lower() in INTERPRETER_SHORT_NAMES and env_name is not None and spec.path is None:
+            match = PY_FACTORS_RE.match(factor)
+            if match:
                 candidates.append(factor)
         if candidates:
             if len(candidates) > 1:
diff --git a/tox.ini b/tox.ini
index d7bbdf47..2f803f23 100644
--- a/tox.ini
+++ b/tox.ini
@@ -83,7 +83,7 @@ description = do a release, required posarg of the version number
 skip_install = true
 deps =
     gitpython>=3.1.30
-    packaging>=22
+    packaging>=23
     towncrier>=22.12
 commands =
     python {toxinidir}/tasks/release.py --version {posargs}


Test patch:
diff --git a/tests/tox_env/python/test_python_api.py b/tests/tox_env/python/test_python_api.py
index 725eab05..3348acda 100644
--- a/tests/tox_env/python/test_python_api.py
+++ b/tests/tox_env/python/test_python_api.py
@@ -71,6 +71,36 @@ def test_diff_msg_no_diff() -> None:
     assert Python._diff_msg({}, {}) == "python "
 
 
+@pytest.mark.parametrize(
+    ("env", "base_python"),
+    [
+        ("py3", "py3"),
+        ("py311", "py311"),
+        ("py3.12", "py3.12"),
+        ("pypy2", "pypy2"),
+        ("rustpython3", "rustpython3"),
+        ("cpython3.8", "cpython3.8"),
+        ("ironpython2.7", "ironpython2.7"),
+        ("functional-py310", "py310"),
+        ("bar-pypy2-foo", "pypy2"),
+        ("py", None),
+        ("django-32", None),
+        ("eslint-8.3", None),
+        ("py-310", None),
+        ("py3000", None),
+        ("4.foo", None),
+        ("310", None),
+        ("5", None),
+        ("2000", None),
+        ("4000", None),
+    ],
+    ids=lambda a: "|".join(a) if isinstance(a, list) else str(a),
+)
+def test_extract_base_python(env: str, base_python: str | None) -> None:
+    result = Python.extract_base_python(env)
+    assert result == base_python
+
+
 @pytest.mark.parametrize("ignore_conflict", [True, False])
 @pytest.mark.parametrize(
     ("env", "base_python"),
@@ -89,7 +119,6 @@ def test_base_python_env_no_conflict(env: str, base_python: list[str], ignore_co
 @pytest.mark.parametrize(
     ("env", "base_python", "expected", "conflict"),
     [
-        ("cpython", ["pypy"], "cpython", ["pypy"]),
         ("pypy", ["cpython"], "pypy", ["cpython"]),
         ("pypy2", ["pypy3"], "pypy2", ["pypy3"]),
         ("py3", ["py2"], "py3", ["py2"]),
@@ -97,8 +126,7 @@ def test_base_python_env_no_conflict(env: str, base_python: list[str], ignore_co
         ("py38", ["py38", "py39"], "py38", ["py39"]),
         ("py38", ["python3"], "py38", ["python3"]),
         ("py310", ["py38", "py39"], "py310", ["py38", "py39"]),
-        ("py3.11.1", ["py3.11.2"], "py3.11.1", ["py3.11.2"]),
-        ("py3-64", ["py3-32"], "py3-64", ["py3-32"]),
+        ("py3.11", ["py310"], "py3.11", ["py310"]),
         ("py310-magic", ["py39"], "py310", ["py39"]),
     ],
     ids=lambda a: "|".join(a) if isinstance(a, list) else str(a),
@@ -110,9 +138,6 @@ def test_base_python_env_conflict(
     conflict: list[str],
     ignore_conflict: bool,
 ) -> None:
-    if env == "py3-64":
-        raise pytest.skip("bug #2657")
-
     if ignore_conflict:
         result = Python._validate_base_python(env, base_python, ignore_conflict)
         assert result == [expected]
diff --git a/tests/tox_env/python/virtual_env/test_virtualenv_api.py b/tests/tox_env/python/virtual_env/test_virtualenv_api.py
index 2d84dafd..cd57fd05 100644
--- a/tests/tox_env/python/virtual_env/test_virtualenv_api.py
+++ b/tests/tox_env/python/virtual_env/test_virtualenv_api.py
@@ -33,10 +33,10 @@ def virtualenv_opt(monkeypatch: MonkeyPatch, mocker: MockerFixture) -> VirtualEn
 
 def test_virtualenv_default_settings(tox_project: ToxProjectCreator, virtualenv_opt: VirtualEnvOptions) -> None:
     proj = tox_project({"tox.ini": "[testenv]\npackage=skip"})
-    result = proj.run("r", "-e", "py", "--discover", sys.executable, str(proj.path / "a"))
+    result = proj.run("r", "-e", "py3", "--discover", sys.executable, str(proj.path / "a"))
     result.assert_success()
 
-    conf = result.env_conf("py")
+    conf = result.env_conf("py3")
     assert conf["system_site_packages"] is False
     assert conf["always_copy"] is False
     assert conf["download"] is False
@@ -46,7 +46,7 @@ def test_virtualenv_default_settings(tox_project: ToxProjectCreator, virtualenv_
     assert virtualenv_opt.download is False
     assert virtualenv_opt.copies is False
     assert virtualenv_opt.no_periodic_update is True
-    assert virtualenv_opt.python == ["py"]
+    assert virtualenv_opt.python == ["py3"]
     assert virtualenv_opt.try_first_with == [str(sys.executable), str(proj.path / "a")]
 
 
@@ -60,10 +60,10 @@ def test_virtualenv_flipped_settings(
     )
     monkeypatch.setenv("VIRTUALENV_CLEAR", "0")
 
-    result = proj.run("r", "-e", "py")
+    result = proj.run("r", "-e", "py3")
     result.assert_success()
 
-    conf = result.env_conf("py")
+    conf = result.env_conf("py3")
     assert conf["system_site_packages"] is True
     assert conf["always_copy"] is True
     assert conf["download"] is True
@@ -72,7 +72,7 @@ def test_virtualenv_flipped_settings(
     assert virtualenv_opt.system_site is True
     assert virtualenv_opt.download is True
     assert virtualenv_opt.copies is True
-    assert virtualenv_opt.python == ["py"]
+    assert virtualenv_opt.python == ["py3"]
 
 
 def test_virtualenv_env_ignored_if_set(