Repository: pypa/pip

Issue:
pip imports the deprecated distutils module
### Description

The ``pip/_internal/locations/_distutils.py`` file imports the distutils module: https://github.com/pypa/pip/blob/cb24fb4052ca8ab8009866b0de61980c81a7e13c/src/pip/_internal/locations/_distutils.py#L9-L12

This module is deprecated since Python 3.10 and is going to be removed in Python 3.12: https://github.com/python/cpython/issues/92584#issue-1230326042

### Expected behavior

_No response_

### pip version

22.0.4

### Python version

3.12.0a0

### OS

Linux

### How to Reproduce

1. Build Python 3.12 with https://github.com/python/cpython/pull/92585 PR
2. Create a venv without pip: ``./python -m venv env``
3. Install pip: ``./env/bin/python -m ensurepip -v``

### Output

```sh-session
vstinner@apu$ ./env/bin/python -m ensurepip -v
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/home/vstinner/python/main/Lib/runpy.py", line 226, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/runpy.py", line 98, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/__main__.py", line 29, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/cli/main.py", line 9, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/cli/autocompletion.py", line 10, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/cli/main_parser.py", line 8, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/cli/cmdoptions.py", line 23, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/cli/parser.py", line 12, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/configuration.py", line 26, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/utils/logging.py", line 27, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/utils/misc.py", line 39, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/locations/__init__.py", line 14, in <module>
  File "/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl/pip/_internal/locations/_distutils.py", line 9, in <module>
ModuleNotFoundError: No module named 'distutils'
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
             ^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ensurepip/__init__.py", line 276, in _main
    return _bootstrap(
           ^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ensurepip/__init__.py", line 192, in _bootstrap
    return _run_pip([*args, *_PACKAGE_NAMES], additional_paths)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/ensurepip/__init__.py", line 92, in _run_pip
    return subprocess.run([sys.executable, '-W', 'ignore::DeprecationWarning',
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/subprocess.py", line 558, in run
    raise CalledProcessError(retcode, process.args,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
subprocess.CalledProcessError: Command '['/home/vstinner/python/main/env/bin/python', '-W', 'ignore::DeprecationWarning', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/tmp/tmps2pazori/setuptools-58.1.0-py3-none-any.whl\', \'/tmp/tmps2pazori/pip-22.0.4-py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/tmp/tmps2pazori\', \'-v\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
```


### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).


Discussion:
This module is provided for backward compatibility; on 3.10 it should never be used. So I guess (assuming there's no real bug where we still use it) we simply need to do conditional imports better.
Maybe only import ``locations/_distutils.py`` in ``locations/__init__.py`` if ``_USE_SYSCONFIG`` is false?
I looked at the code and pip in fact still relies on some distutils internals beside the `locations` module. I’ll try to either remove them, or at least localise so `distutils` is imported only when relevant functionalities are accessed (the most significant usage of it is to parse `--install-option`).
Hi, is there an update on this issue? How can I help?
No updates (since those would be reflected on this issue).

Looking at this now... There's 5 major files that this needs changes for:

- src/pip/_internal/locations/__init__.py: this needs a conditional import of `pip._internal.locations._distutils`
- src/pip/_internal/locations/_distutils.py: well... this should be imported conditionally and never imported on 3.12+.
- src/pip/_internal/locations/_sysconfig.py: this needs `change_root` from distutils.util
- src/pip/_internal/operations/install/legacy.py: this needs `change_root` from distutils.util
- src/pip/_internal/utils/distutils_args.py: this uses `FancyGetopt` from distutils to parse out distutils arguments.

Help would be appreciated with all of these. There's one import/use of distutils in a vendored package -- for `distilb.get_archive_formats` -- and that code path shouldn't be executed in the real world (that function is not used by pip, AFAIK).

I'll pick up the `change_root` changes, partly because that's the easiest of the lot. :)
Alrighty, other than the open PRs, the only thing remaining here is moving to a conditional import of the `_distutils` module, in the locations package. If someone else could pick that up, that'd be great! ^>^

distutils.sysconfig import get_python_lib, what should be new import for get_python_lib or what else should be used?
`sysconfig.get_path("purelib")` and `sysconfig.get_path("platlib")`.
and what for `get_prefixed_libs`?
`sysconfig` does not provide a native way to supply a custom `prefix`, but you can use the `vars` argument and supply custom `base`, `platbase`, etc. values. (The semantic of “prefix” in distutils is split into quite several variables in sysconfig; you need to hunt them all up from the sysconfig source.)

https://github.com/python/cpython/blob/main/Lib/sysconfig.py
This still needs a PR adding a conditional import to `pip._internal.locations`, to not import distutils if sysconfig is going to be used.
like this?
```python
if _USE_SYSCONFIG:
    from . import _distutils
```
It'd likely need more changes, but yes -- we basically want to avoid importing distutils if `_USE_SYSCONFIG` is true.

We should also have corresponding tests, to catch us regressing on this FWIW.
Apparently the problem still reproduces with py311 from deadsnakes on ubuntu while trying to install a package.
```
cmd: /media/psf/c/a/ansible-navigator/.tox/u1/py311/bin/python -m pip install '.[test]'
Traceback (most recent call last):
...
  File "/media/psf/c/a/ansible-navigator/.tox/u1/py311/lib/python3.11/site-packages/pip/_internal/locations/__init__.py", line 14, in <module>
    from . import _distutils, _sysconfig
  File "/media/psf/c/a/ansible-navigator/.tox/u1/py311/lib/python3.11/site-packages/pip/_internal/locations/_distutils.py", line 9, in <module>
    from distutils.cmd import Command as DistutilsCommand
ModuleNotFoundError: No module named 'distutils.cmd'
```

Does anyone know a workaround for it as I would really want to enable CI/CD pipeline for py311. I already had success on other projects but on this project, it even fails to install dependencies.
Unless someone plans to work on this in the coming days, I'm going to push this to down to 22.3.
I’ll try to find some time this weekend.
@uranusjr Hmm... does this need anything more than the following?

```diff
diff --git a/src/pip/_internal/locations/__init__.py b/src/pip/_internal/locations/__init__.py
index 99312d77a..22e90a00f 100644
--- a/src/pip/_internal/locations/__init__.py
+++ b/src/pip/_internal/locations/__init__.py
@@ -11,7 +11,7 @@
 from pip._internal.utils.deprecation import deprecated
 from pip._internal.utils.virtualenv import running_under_virtualenv
 
-from . import _distutils, _sysconfig
+from . import _sysconfig
 from .base import (
     USER_CACHE_DIR,
     get_major_minor_version,
@@ -241,6 +241,8 @@ def get_scheme(
     if _USE_SYSCONFIG:
         return new
 
+    from . import _distutils
+
     old = _distutils.get_scheme(
         dist_name,
         user=user,
```
Maybe not? I am not sure.

Resolving patch:
diff --git a/src/pip/_internal/locations/__init__.py b/src/pip/_internal/locations/__init__.py
--- a/src/pip/_internal/locations/__init__.py
+++ b/src/pip/_internal/locations/__init__.py
@@ -11,7 +11,7 @@
 from pip._internal.utils.deprecation import deprecated
 from pip._internal.utils.virtualenv import running_under_virtualenv
 
-from . import _distutils, _sysconfig
+from . import _sysconfig
 from .base import (
     USER_CACHE_DIR,
     get_major_minor_version,
@@ -241,6 +241,8 @@ def get_scheme(
     if _USE_SYSCONFIG:
         return new
 
+    from . import _distutils
+
     old = _distutils.get_scheme(
         dist_name,
         user=user,
@@ -405,6 +407,8 @@ def get_bin_prefix() -> str:
     if _USE_SYSCONFIG:
         return new
 
+    from . import _distutils
+
     old = _distutils.get_bin_prefix()
     if _warn_if_mismatch(pathlib.Path(old), pathlib.Path(new), key="bin_prefix"):
         _log_context()
@@ -438,6 +442,8 @@ def get_purelib() -> str:
     if _USE_SYSCONFIG:
         return new
 
+    from . import _distutils
+
     old = _distutils.get_purelib()
     if _looks_like_deb_system_dist_packages(old):
         return old
@@ -452,6 +458,8 @@ def get_platlib() -> str:
     if _USE_SYSCONFIG:
         return new
 
+    from . import _distutils
+
     old = _distutils.get_platlib()
     if _looks_like_deb_system_dist_packages(old):
         return old
@@ -480,6 +488,8 @@ def get_prefixed_libs(prefix: str) -> List[str]:
     if _USE_SYSCONFIG:
         return _deduplicated(new_pure, new_plat)
 
+    from . import _distutils
+
     old_pure, old_plat = _distutils.get_prefixed_libs(prefix)
     old_lib_paths = _deduplicated(old_pure, old_plat)
 


Test patch: