Repository: pypa/setuptools

Issue:
[BUG] Issue with local distutils override, duplicate module entries
### setuptools version

60.5.0

### Python version

3.9.9

### OS

macOS

### Additional environment information

edit(jaraco): ~~I explicitly state that I DO NOT AGREE with being forced to agree with the PSF Code of Conduct to submit this issue. I ticked the box just because I cannot continue otherwise. I hope this requirement for issue submission is an oversight. If not, please close this issue and ignore my report.~~

### Description

There are issues with the local `distutils` override leading to duplicated `distutils.log` module entries in `sys.modules`. As a side effect, the verbose flag (`-v`) to `setup.py` does not work (i.e passing `-v` DOES NOT increase verbosity).

### Expected behavior

1. After `import setuptools`, there should be no entries in `sys.modules` referring to `setuptools._distutils`.
2. Running `python setup.py -vv <command>` should increase verbosity.

### How to Reproduce

Run the following Python code:

```python
import setuptools
import sys
for k in sys.modules.keys():
    if 'distutils.log' in k:
        print(k)
```

### Output

```console
setuptools._distutils.log
distutils.log
```

Obviously, the first entry is bogus and should not be printed.

Adding `print(log)` after the `from distutils import log` import statement in `setuptools/_distutils/dist.py` will help with debugging the problem. You should get the bogus `setuptools._distutils.log` module.

I believe this module duplication issue happens with other distutils submodules, you can check with:

```python
import setuptools
import sys
for k in sys.modules.keys():
    if '_distutils' in k:
        print(k)
```

which produces

```console
_distutils_hack
setuptools._distutils
setuptools._distutils.log
setuptools._distutils.dir_util
setuptools._distutils.file_util
setuptools._distutils.archive_util
_distutils_hack.override
```

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:

Resolving patch:
diff --git a/_distutils_hack/__init__.py b/_distutils_hack/__init__.py
--- a/_distutils_hack/__init__.py
+++ b/_distutils_hack/__init__.py
@@ -57,6 +57,7 @@ def ensure_local_distutils():
     # check that submodules load as expected
     core = importlib.import_module('distutils.core')
     assert '_distutils' in core.__file__, core.__file__
+    assert 'setuptools._distutils.log' not in sys.modules
 
 
 def do_override():
@@ -112,6 +113,7 @@ def spec_for_distutils(self):
         class DistutilsLoader(importlib.abc.Loader):
 
             def create_module(self, spec):
+                mod.__name__ = 'distutils'
                 return mod
 
             def exec_module(self, module):


Test patch:
diff --git a/setuptools/tests/test_distutils_adoption.py b/setuptools/tests/test_distutils_adoption.py
--- a/setuptools/tests/test_distutils_adoption.py
+++ b/setuptools/tests/test_distutils_adoption.py
@@ -93,3 +93,66 @@ def test_distutils_has_origin():
     Distutils module spec should have an origin. #2990.
     """
     assert __import__('distutils').__spec__.origin
+
+
+ENSURE_IMPORTS_ARE_NOT_DUPLICATED = r"""
+# Depending on the importlib machinery and _distutils_hack, some imports are
+# duplicated resulting in different module objects being loaded, which prevents
+# patches as shown in #3042.
+# This script provides a way of verifying if this duplication is happening.
+
+from distutils import cmd
+import distutils.command.sdist as sdist
+
+# import last to prevent caching
+from distutils import {imported_module}
+
+for mod in (cmd, sdist):
+    assert mod.{imported_module} == {imported_module}, (
+        f"\n{{mod.dir_util}}\n!=\n{{{imported_module}}}"
+    )
+
+print("success")
+"""
+
+
+@pytest.mark.parametrize(
+    "distutils_version, imported_module",
+    [
+        ("stdlib", "dir_util"),
+        ("stdlib", "file_util"),
+        ("stdlib", "archive_util"),
+        ("local", "dir_util"),
+        ("local", "file_util"),
+        ("local", "archive_util"),
+    ]
+)
+def test_modules_are_not_duplicated_on_import(
+        distutils_version, imported_module, tmpdir_cwd, venv
+):
+    env = dict(SETUPTOOLS_USE_DISTUTILS=distutils_version)
+    script = ENSURE_IMPORTS_ARE_NOT_DUPLICATED.format(imported_module=imported_module)
+    cmd = ['python', '-c', script]
+    output = popen_text(venv.run)(cmd, env=win_sr(env)).strip()
+    assert output == "success"
+
+
+ENSURE_LOG_IMPORT_IS_NOT_DUPLICATED = r"""
+# Similar to ENSURE_IMPORTS_ARE_NOT_DUPLICATED
+import distutils.dist as dist
+from distutils import log
+
+assert dist.log == log, (
+    f"\n{dist.log}\n!=\n{log}"
+)
+
+print("success")
+"""
+
+
+@pytest.mark.parametrize("distutils_version", "local stdlib".split())
+def test_log_module_is_not_duplicated_on_import(distutils_version, tmpdir_cwd, venv):
+    env = dict(SETUPTOOLS_USE_DISTUTILS=distutils_version)
+    cmd = ['python', '-c', ENSURE_LOG_IMPORT_IS_NOT_DUPLICATED]
+    output = popen_text(venv.run)(cmd, env=win_sr(env)).strip()
+    assert output == "success"