Repository: pypa/setuptools

Issue:
[BUG] distutils cannot be imported on Python 3.12 (without distutils) if pip was imported
### setuptools version

65.5.0

### Python version

3.12 (dev, without distutils)

### OS

Linux

### Additional environment information

_No response_

### Description

I just removed the distutils packages from Python 3.12: [commit](https://github.com/python/cpython/commit/0faa0ba240e815614e5a2900e48007acac41b214). setuptools works as expected, it provides its copy of distutils. Good.

But if pip is imported first, ``import setuptools`` or  ``import distutils`` fails on import setuptools, since _distutils_hack has a special path when pip is installed which requires to get the stdlib distutils... which no longer exists.

Extract of ``_distutils_hack/__init__.py``:

```
    def spec_for_pip(self):
        """
        Ensure stdlib distutils when running under pip.
        See pypa/pip#8761 for rationale.
        """
```

See https://github.com/pypa/pip/issues/8761 for the rationale.

Well, this special case should be revisited for Python >= 3.12.0 alpha 2.

### Expected behavior

Importing setuptools on Python 3.12 after pip was imported should just work.

### How to Reproduce

```
$ python3.12
Python 3.12.0a1+ (heads/main:f09da28768, Nov  4 2022, 17:46:22) [GCC 12.2.1 20220819 (Red Hat 12.2.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip
>>> import setuptools
```

### Output

```console
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/py312/lib/python3.12/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 77, in do_override
    ensure_local_distutils()
  File "/opt/py312/lib/python3.12/site-packages/_distutils_hack/__init__.py", line 60, in ensure_local_distutils
    importlib.import_module('distutils')
  File "/opt/py312/lib/python3.12/importlib/__init__.py", line 124, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils'
```



Discussion:
It doesn't seem to be related to pip, setuptools just do not support python 3.12. When running setuptools' `setup.py install` with python 3.12.0a3, as it is done when installing py-setuptools package on FreeBSD:

```
...
byte-compiling /work/usr/ports/devel/py-setuptools/work-py312/stage/usr/local/lib/python3.12/site-packages/_distutils_hack/override.py to override.cpython-312.pyc
writing byte-compilation script '/tmp/tmph5sr5lui.py'
/usr/local/bin/python3.12 /tmp/tmph5sr5lui.py
Traceback (most recent call last):
  File "/tmp/tmph5sr5lui.py", line 1, in <module>
    from distutils.util import byte_compile
ModuleNotFoundError: No module named 'distutils'
error: command '/usr/local/bin/python3.12' failed with exit code 1
```
I get this even without explicitly importing pip (i.e. just importing setuptools). Any further information on this?
Hi,
I'm trying to take up this PR. Would it be fair to move distutils modules from cpython to setuptools to handle this?
Hi @sathyz, thank you very much for working on this. If you need to change anything of distutils please submit a PR to https://github.com/pypa/distutils.

But on a first glance I think we could tackle this issue by modifying https://github.com/pypa/setuptools/blob/main/_distutils_hack/__init__.py... What if the finder is always enabled for Python >= 3.12?
Hi @abravalheri 
I'm able to get this working by installing pypa/distutils and with the following change,

```
@@ -61,7 +61,8 @@ def ensure_local_distutils():

     # check that submodules load as expected
     core = importlib.import_module('distutils.core')
-    assert '_distutils' in core.__file__, core.__file__
+    if sys.version_info < (3, 12):
+        assert '_distutils' in core.__file__, core.__file__
     assert 'setuptools._distutils.log' not in sys.modules
```

I tried installing a package that was breaking earlier and is now working fine. One of the problem that I see is, pypa/distutils is not in pypi.org that I couldn't do `pip install distutils`, until this is available in PyPi repository we wouldn't be able to have it as dependency, am I correct? Any thoughts?

Resolving patch:
diff --git a/_distutils_hack/__init__.py b/_distutils_hack/__init__.py
--- a/_distutils_hack/__init__.py
+++ b/_distutils_hack/__init__.py
@@ -142,7 +142,7 @@ def spec_for_pip(self):
         Ensure stdlib distutils when running under pip.
         See pypa/pip#8761 for rationale.
         """
-        if self.pip_imported_during_build():
+        if sys.version_info >= (3, 12) or self.pip_imported_during_build():
             return
         clear_distutils()
         self.spec_for_distutils = lambda: None
@@ -208,15 +208,20 @@ def __enter__(self):
         insert_shim()
 
     def __exit__(self, exc, value, tb):
-        remove_shim()
+        _remove_shim()
 
 
 def insert_shim():
     sys.meta_path.insert(0, DISTUTILS_FINDER)
 
 
-def remove_shim():
+def _remove_shim():
     try:
         sys.meta_path.remove(DISTUTILS_FINDER)
     except ValueError:
         pass
+
+
+if sys.version_info < (3, 12):
+    # DistutilsMetaFinder can only be disabled in Python < 3.12 (PEP 632)
+    remove_shim = _remove_shim
diff --git a/setuptools/sandbox.py b/setuptools/sandbox.py
--- a/setuptools/sandbox.py
+++ b/setuptools/sandbox.py
@@ -237,7 +237,7 @@ def hide_setuptools():
     """
     _distutils_hack = sys.modules.get('_distutils_hack', None)
     if _distutils_hack is not None:
-        _distutils_hack.remove_shim()
+        _distutils_hack._remove_shim()
 
     modules = filter(_needs_hiding, sys.modules)
     _clear_modules(modules)


Test patch: