Repository: pypa/pip
Pull request: Move from tox to Nox
Closes #6721. Move all our CI from tox to Nox, and update documentation.

@pypa/pip-team @theacodes

<!---
Thank you for your soon to be pull request. Before you submit this, please
double check to make sure that you've added a news file fragment. In pip we
generate our NEWS.rst from multiple news fragment files, and all pull requests
require either a news file fragment or a marker to indicate they don't require
one.

To read more about adding a news file fragment for your PR, please check out
our documentation at: https://pip.pypa.io/en/latest/development/contributing/#news-entries
-->

Problem statement:
Investigate if we can replace our tox+invoke setup with nox
My understanding is that nox is good enough to cover our use-cases for both tox and invoke (eg. the test stuff can possibly be easier, and the vendoring / generation logic).

It'll might be worth the effort to invest time to switch from those two tools to using just nox.

/cc @crwilcox @theacodes

Hints from issue discussion:
Happy to help in any way. :) Nox is pretty well suited for what y'all are doing.
I think if we do this -- we see most benefit from replacing our invoke stuff.

I imagine having the chunk of the code living in tools/automation/ and having the noxfile.py just contain the "overview" of what those commands need.
@xavfernandez answering here.

`nox` has functionality from tox and invoke.

As such, the reason I'm considering switching us over, is that nox lets us create isolated environments like tox, with a python syntax like invoke and in general is more comfortable to work with in my experience. These combined, will let us transition our invoke scripts (where we currently have an extra step of install dependencies before running) pretty easily and our test-run-related customizations in tox, would likely be cleaner and easier to work with in nox.

AFAICT, we should also be able to switch our entire automation setup, to use nox, which then gives us 1 tool for all our scripting/automation needs. Which, is a good place to be in. 🙃
The answer is: yes, but we're blocked on https://github.com/theacodes/nox/issues/158 before making it the default.
FYI, I created a draft PR. Will want to chat with others on `nox` about the approach, but this does allow me to do what I think you are asking for? You could always try installing from my branch as a test of sorts :) `pip install git+https://github.com/crwilcox/nox.git@allow-non-specified-python-versions`
This was added to nox several months ago, FYI. :)
Hi everybody! I don't know how is this issue going, but I think we can use Nox a bit more. I'll be more than happy to help with anything you need ;)
There's 2 places we still use tox:

- In our CI system.
- In our contributor documentation.

We need to change those over to nox.
Ok. In the next days, I will take my time to investigate about this.
I think we can just remove `tox.ini` in favor of `noxfile.py`? Or should we transfer some items to `noxfile.py`?

Commit message: Fix the news entry

Patch to review:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 6d3f70fcabe..e9885a499ae 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -121,17 +121,17 @@ jobs:
         if: matrix.os == 'MacOS'
         run: brew install bzr
 
-      - run: pip install tox 'virtualenv<20'
+      - run: pip install nox 'virtualenv<20'
 
       # Main check
       - name: Run unit tests
         run: >-
-          tox -e py --
+          nox -s test-${{ matrix.python }} --
           -m unit
           --verbose --numprocesses auto --showlocals
       - name: Run integration tests
         run: >-
-          tox -e py --
+          nox -s test-${{ matrix.python }} --
           -m integration
           --verbose --numprocesses auto --showlocals
           --durations=5
@@ -178,7 +178,7 @@ jobs:
           $acl.AddAccessRule($rule)
           Set-Acl "R:\Temp" $acl
 
-      - run: pip install tox 'virtualenv<20'
+      - run: pip install nox 'virtualenv<20'
         env:
           TEMP: "R:\\Temp"
 
@@ -186,7 +186,7 @@ jobs:
       - name: Run unit tests
         if: matrix.group == 1
         run: >-
-          tox -e py --
+          nox -s test-${{ matrix.python }} --
           -m unit
           --verbose --numprocesses auto --showlocals
         env:
@@ -195,7 +195,7 @@ jobs:
       - name: Run integration tests (group 1)
         if: matrix.group == 1
         run: >-
-          tox -e py --
+          nox -s test-${{ matrix.python }} --
           -m integration -k "not test_install"
           --verbose --numprocesses auto --showlocals
         env:
@@ -204,7 +204,7 @@ jobs:
       - name: Run integration tests (group 2)
         if: matrix.group == 2
         run: >-
-          tox -e py --
+          nox -s test-${{ matrix.python }} --
           -m integration -k "test_install"
           --verbose --numprocesses auto --showlocals
         env:
diff --git a/docs/html/development/getting-started.rst b/docs/html/development/getting-started.rst
index 3ba664f1231..730f5ece08f 100644
--- a/docs/html/development/getting-started.rst
+++ b/docs/html/development/getting-started.rst
@@ -27,8 +27,8 @@ Development Environment
 pip is a command line application written in Python. For developing pip,
 you should `install Python`_ on your computer.
 
-For developing pip, you need to install :pypi:`tox`. Often, you can run
-``python -m pip install tox`` to install and use it.
+For developing pip, you need to install :pypi:`nox`. Often, you can run
+``python -m pip install nox`` to install and use it.
 
 
 Running pip From Source Tree
@@ -60,7 +60,7 @@ Running Tests
 =============
 
 pip's tests are written using the :pypi:`pytest` test framework and
-:mod:`unittest.mock`. :pypi:`tox` is used to automate the setup and execution
+:mod:`unittest.mock`. :pypi:`nox` is used to automate the setup and execution
 of pip's tests.
 
 It is preferable to run the tests in parallel for better experience during development,
@@ -70,29 +70,29 @@ To run tests:
 
 .. code-block:: console
 
-    $ tox -e py310 -- -n auto
+    $ nox -s test-3.10 -- -n auto
 
 To run tests without parallelization, run:
 
 .. code-block:: console
 
-    $ tox -e py310
+    $ nox -s test-3.10
 
 The example above runs tests against Python 3.10. You can also use other
-versions like ``py39`` and ``pypy3``.
+versions like ``3.9`` and ``pypy3``.
 
-``tox`` has been configured to forward any additional arguments it is given to
+``nox`` has been configured to forward any additional arguments it is given to
 ``pytest``. This enables the use of pytest's `rich CLI`_. As an example, you
 can select tests using the various ways that pytest provides:
 
 .. code-block:: console
 
     $ # Using file name
-    $ tox -e py310 -- tests/functional/test_install.py
+    $ nox -s test-3.10 -- tests/functional/test_install.py
     $ # Using markers
-    $ tox -e py310 -- -m unit
+    $ nox -s test-3.10 -- -m unit
     $ # Using keywords
-    $ tox -e py310 -- -k "install and not wheel"
+    $ nox -s test-3.10 -- -k "install and not wheel"
 
 Running pip's entire test suite requires supported version control tools
 (subversion, bazaar, git, and mercurial) to be installed. If you are missing
@@ -101,8 +101,8 @@ explicitly tell pytest to skip those tests:
 
 .. code-block:: console
 
-    $ tox -e py310 -- -k "not svn"
-    $ tox -e py310 -- -k "not (svn or git)"
+    $ nox -s test-3.10 -- -k "not svn"
+    $ nox -s test-3.10 -- -k "not (svn or git)"
 
 
 Running Linters
@@ -116,7 +116,7 @@ To use linters locally, run:
 
 .. code-block:: console
 
-    $ tox -e lint
+    $ nox -s lint
 
 .. note::
 
@@ -155,7 +155,7 @@ To build it locally, run:
 
 .. code-block:: console
 
-    $ tox -e docs
+    $ nox -s docs
 
 The built documentation can be found in the ``docs/build`` folder.
 
diff --git a/news/10693.trivial.rst b/news/10693.trivial.rst
new file mode 100644
index 00000000000..5138d9d7117
--- /dev/null
+++ b/news/10693.trivial.rst
@@ -0,0 +1 @@
+Replace all our tox operations with Nox.
diff --git a/noxfile.py b/noxfile.py
index 06ad8d5493e..8fe16606322 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -63,12 +63,6 @@ def should_update_common_wheels() -> bool:
     return need_to_repopulate
 
 
-# -----------------------------------------------------------------------------
-# Development Commands
-#   These are currently prototypes to evaluate whether we want to switch over
-#   completely to nox for all our automation. Contributors should prefer using
-#   `tox -e ...` until this note is removed.
-# -----------------------------------------------------------------------------
 @nox.session(python=["3.7", "3.8", "3.9", "3.10", "pypy3"])
 def test(session: nox.Session) -> None:
     # Get the common wheels.
@@ -223,6 +217,21 @@ def pinned_requirements(path: Path) -> Iterator[Tuple[str, str]]:
         release.commit_file(session, ".", message=message)
 
 
+@nox.session
+def coverage(session: nox.Session) -> None:
+    if not os.path.exists("./.coverage-output"):
+        os.mkdir("./coverage-output")
+    session.run(
+        "pytest",
+        "--cov=pip",
+        "--cov-config=./setup.cfg",
+        env={
+            "COVERAGE_OUTPUT_DIR": "./coverage-output",
+            "COVERAGE_PROCESS_START": "./setup.cfg",
+        },
+    )
+
+
 # -----------------------------------------------------------------------------
 # Release Commands
 # -----------------------------------------------------------------------------
diff --git a/tox.ini b/tox.ini
deleted file mode 100644
index 23208736df8..00000000000
--- a/tox.ini
+++ /dev/null
@@ -1,74 +0,0 @@
-[tox]
-minversion = 3.4.0
-envlist =
-    docs, packaging, lint, vendoring,
-    py37, py38, py39, py310, pypy3
-
-[helpers]
-# Wrapper for calls to pip that make sure the version being used is the
-# original virtualenv (stable) version, and not the code being tested.
-pip = python {toxinidir}/tools/tox_pip.py
-mkdirp = python -c 'import os, sys; os.path.exists(sys.argv[1]) or os.mkdir(sys.argv[1])'
-
-[testenv]
-passenv =
-    CI
-    GIT_SSL_CAINFO
-    HTTP_PROXY
-    HTTPS_PROXY
-    NO_PROXY
-setenv =
-    # This is required in order to get UTF-8 output inside of the subprocesses
-    # that our tests use.
-    LC_CTYPE = en_US.UTF-8
-deps = -r{toxinidir}/tests/requirements.txt
-commands_pre =
-    python -c 'import shutil, sys; shutil.rmtree(sys.argv[1], ignore_errors=True)' {toxinidir}/tests/data/common_wheels
-    {[helpers]pip} wheel -w {toxinidir}/tests/data/common_wheels -r {toxinidir}/tests/requirements-common_wheels.txt
-commands = pytest []
-install_command = {[helpers]pip} install {opts} {packages}
-list_dependencies_command = {[helpers]pip} freeze --all
-
-[testenv:coverage]
-basepython = python3
-commands =
-    {[helpers]mkdirp} {toxinidir}/.coverage-output
-    pytest --cov=pip --cov-config={toxinidir}/setup.cfg []
-
-setenv =
-    # Used in coverage configuration in setup.cfg.
-    COVERAGE_OUTPUT_DIR = {toxinidir}/.coverage-output
-    # Ensure coverage is enabled in child processes in virtual environments
-    # since they won't already have been enabled by pytest-cov.
-    COVERAGE_PROCESS_START = {toxinidir}/setup.cfg
-
-[testenv:docs]
-# Don't skip install here since pip_sphinxext uses pip's internals.
-deps = -r{toxinidir}/docs/requirements.txt
-basepython = python3
-commands =
-    sphinx-build -W -j auto -d {envtmpdir}/doctrees/html -b html docs/html docs/build/html
-    # Having the conf.py in the docs/html is weird but needed because we
-    # can not use a different configuration directory vs source directory on RTD
-    # currently -- https://github.com/rtfd/readthedocs.org/issues/1543.
-    # That is why we have a "-c docs/html" in the next line.
-    sphinx-build -W -j auto -d {envtmpdir}/doctrees/man -b man docs/man docs/build/man -c docs/html
-
-[testenv:lint]
-skip_install = True
-commands_pre =
-deps = pre-commit
-commands =
-   pre-commit run [] --all-files --show-diff-on-failure --hook-stage=manual
-
-[testenv:vendoring]
-basepython = python3
-skip_install = True
-commands_pre =
-deps =
-    vendoring~=1.2.0
-whitelist_externals = git
-commands =
-    # Check that the vendoring is up-to-date
-    vendoring sync . -v
-    git diff --exit-code


Code review comments:
diff_hunk: @@ -63,12 +63,6 @@ def should_update_common_wheels() -> bool:
     return need_to_repopulate
 
 
-# -----------------------------------------------------------------------------
path: noxfile.py
text: @user1:
Let's keep the separator that these are development commands and only drop the note about not using them.

diff_hunk: @@ -223,6 +217,21 @@ def pinned_requirements(path: Path) -> Iterator[Tuple[str, str]]:
         release.commit_file(session, ".", message=message)
 
 
+@nox.session
+def coverage(session: nox.Session) -> None:
+    if not os.path.exists("./.coverage-output"):
+        os.mkdir("./coverage-output")
+    session.run(
+        "pytest",
+        "--cov=pip",
+        "--cov-config=./setup.cfg",
+        env={
+            "COVERAGE_OUTPUT_DIR": "./coverage-output",
path: noxfile.py
text: @user1:
```suggestion
            "COVERAGE_OUTPUT_DIR": "./.coverage-output",
```

diff_hunk: @@ -223,6 +217,21 @@ def pinned_requirements(path: Path) -> Iterator[Tuple[str, str]]:
         release.commit_file(session, ".", message=message)
 
 
+@nox.session
+def coverage(session: nox.Session) -> None:
+    if not os.path.exists("./.coverage-output"):
+        os.mkdir("./coverage-output")
path: noxfile.py
text: @user1:
```suggestion
        os.mkdir("./.coverage-output")
```