Repository: pypa/pip
Pull request #13356 (2025-04-28 15:52:19)

Fix new dependency-groups feature to use the stdlib tomllib where possible
Previously, commit 88c9f31ad8a5ffe0bb31ab500b8ddd1b9ff6a5dd modified pip to use the stdlib on versions of python where this module is in the stdlib. As justified there:

Although a tomli copy is vendored, doing this conditional import allows:
- automatically upgrading the code, when the time comes to drop py3.10 support

- slightly simplifying debundling support, as it's no longer necessary to depend on a tomli(-wheel)? package on sufficiently newer versions of python.

https://github.com/pypa/pip/pull/13065 added a new feature, including a vendored "dependency_groups" library that likewise supports using the stdlib tomllib via `dependency_groups/_toml_compat.py`. But the code in pip itself to use dependency_groups manually loads pyproject.toml and passes it to dependency_groups, and fails to use the same compatibility dispatch as both the pre-existing pip code and dependency_groups itself.

Add back the conditional logic.

<!---
Thank you for your soon to be pull request. Before you submit this, please
double check to make sure that you've added a news file fragment. In pip we
generate our NEWS.rst from multiple news fragment files, and all pull requests
require either a news file fragment or a marker to indicate they don't require
one.

To read more about adding a news file fragment for your PR, please check out
our documentation at: https://pip.pypa.io/en/latest/development/contributing/#news-entries
-->


pfmoore: Please hold off on merging this. I don't feel that it counts as a bug fix, and I want to avoid introducing extra changes into a potential 25.1.1 release, if needed.

I'm happy to review whether this should be included *if* we need a 25.1.1, but let's not pre-judge that question for now.
NHOrus: It caused a problem for me
https://bugs.gentoo.org/955029
```
python3.13 -m pip install --user --upgrade pip
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib/python3.13/site-packages/pip/__main__.py", line 24, in <module>
    sys.exit(_main())
             ~~~~~^^
  File "/usr/lib/python3.13/site-packages/pip/_internal/cli/main.py", line 77, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/usr/lib/python3.13/site-packages/pip/_internal/commands/__init__.py", line 119, in create_command
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1026, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/lib/python3.13/site-packages/pip/_internal/commands/install.py", line 24, in <module>
    from pip._internal.cli.req_command import (
    ...<2 lines>...
    )
  File "/usr/lib/python3.13/site-packages/pip/_internal/cli/req_command.py", line 31, in <module>
    from pip._internal.req.req_dependency_group import parse_dependency_groups
  File "/usr/lib/python3.13/site-packages/pip/_internal/req/req_dependency_group.py", line 3, in <module>
    import tomli
ModuleNotFoundError: No module named 'tomli'
```
eli-schwartz: ```
_____________________ test_basic_install_editable_from_git _____________________
[...]

E               tests.lib.TestFailure: expected egg link file missing: [PosixPath('venv/lib/python3.11/site-packages/testpackage.egg-link'), PosixPath('venv/lib/python3.11/site-packages/testpackage.egg-link')]
```

I don't understand this test failure, it doesn't seem possible to have happened because of the changes in this PR.

It is also reproducible in this scheduled cron job: https://github.com/pypa/pip/actions/runs/14697699388
sbidoul: > I don't understand this test failure, it doesn't seem possible to have happened because of the changes in this PR.

Maybe due to setuptools 80, released yesterday, which changed editable installs.
eli-schwartz: Until I hear otherwise, I'll assume my PR is entirely fine and passes all tests. They pass in my local environment to my satisfaction, though it's not the same as the CI environment so I cannot assert any guarantees.
sbidoul: Attempt to make tests green again in https://github.com/pypa/pip/pull/13357
notatallshaw: @NHOrus that looks like a bug with the Gentoo distribution of pip, if you think there is a bug with source pip or are asking to make a change based on helping distributions out please make a new issue and we can discuss there. 
sbidoul: @eli-schwartz can you rebase?

Patch:
diff --git a/news/13356.vendor.rst b/news/13356.vendor.rst
new file mode 100644
index 000000000..856642228
--- /dev/null
+++ b/news/13356.vendor.rst
@@ -0,0 +1,1 @@
+Fix issues with using tomllib from the stdlib if available, rather than tomli
diff --git a/src/pip/_internal/req/req_dependency_group.py b/src/pip/_internal/req/req_dependency_group.py
index 8f124de5b..e81dd4552 100644
--- a/src/pip/_internal/req/req_dependency_group.py
+++ b/src/pip/_internal/req/req_dependency_group.py
@@ -1,6 +1,11 @@
+import sys
 from typing import Any, Dict, Iterable, Iterator, List, Tuple
 
-from pip._vendor import tomli
+if sys.version_info >= (3, 11):
+    import tomllib
+else:
+    from pip._vendor import tomli as tomllib
+
 from pip._vendor.dependency_groups import DependencyGroupResolver
 
 from pip._internal.exceptions import InstallationError
@@ -65,10 +70,10 @@ def _load_pyproject(path: str) -> Dict[str, Any]:
     """
     try:
         with open(path, "rb") as fp:
-            return tomli.load(fp)
+            return tomllib.load(fp)
     except FileNotFoundError:
         raise InstallationError(f"{path} not found. Cannot resolve '--group' option.")
-    except tomli.TOMLDecodeError as e:
+    except tomllib.TOMLDecodeError as e:
         raise InstallationError(f"Error parsing {path}: {e}") from e
     except OSError as e:
         raise InstallationError(f"Error reading {path}: {e}") from e


Test patch:
diff --git a/tests/unit/test_req_dependency_group.py b/tests/unit/test_req_dependency_group.py
index b596f6fc5..1b180f8d7 100644
--- a/tests/unit/test_req_dependency_group.py
+++ b/tests/unit/test_req_dependency_group.py
@@ -120,7 +120,7 @@ def test_parse_gets_unexpected_oserror(
         raise OSError(errno.EPIPE, "Broken pipe")
 
     monkeypatch.setattr(
-        "pip._internal.req.req_dependency_group.tomli.load", epipe_toml_load
+        "pip._internal.req.req_dependency_group.tomllib.load", epipe_toml_load
     )
 
     with pytest.raises(InstallationError, match=r"Error reading pyproject\.toml"):