Repository: pypa/setuptools

Issue:
[BUG] DistutilsMetaFinder appears multiple times on sys.meta_path
### setuptools version

setuptools==60.0.4

### Python version

python3.8

### OS

Any

### Additional environment information

N/A

### Description

The _DistutilsMetaFinder meta_path finder shim is installed multiple times, at least twice by `site.py` via the `.pth` shim, and once by `ensure_local_distutils` when `setuptools` is actually imported. It's unlikely to be a problem, especially since it's literally the same instance, but `add_shim` should be idempotent instead of blindly inserting the shim each time it's called. It also appears that the intent was to remove the shim entirely once distutils has been loaded, but since it's added more than once, the removal in the import leaves two references to it in `sys.meta_path` after setuptools has been imported.

### Expected behavior

After setuptools has been imported, the distutils shim finder is no longer on the meta path.

### How to Reproduce

(with setuptools>60 installed):
python -c "import sys; print(f'Before: {sys.meta_path}'); import setuptools; print(f'After: {sys.meta_path}')"


### Output

```console
Before: [<_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_virtualenv._Finder object at 0x7fe4d2806f70>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>]
After: [<_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_virtualenv._Finder object at 0x7fe4d2806f70>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>, <pkg_resources.extern.VendorImporter object at 0x7fe4d21c5460>, <setuptools.extern.VendorImporter object at 0x7fe4d1f2a100>]
```


### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct
[BUG] DistutilsMetaFinder appears multiple times on sys.meta_path
### setuptools version

setuptools==60.0.4

### Python version

python3.8

### OS

Any

### Additional environment information

N/A

### Description

The _DistutilsMetaFinder meta_path finder shim is installed multiple times, at least twice by `site.py` via the `.pth` shim, and once by `ensure_local_distutils` when `setuptools` is actually imported. It's unlikely to be a problem, especially since it's literally the same instance, but `add_shim` should be idempotent instead of blindly inserting the shim each time it's called. It also appears that the intent was to remove the shim entirely once distutils has been loaded, but since it's added more than once, the removal in the import leaves two references to it in `sys.meta_path` after setuptools has been imported.

### Expected behavior

After setuptools has been imported, the distutils shim finder is no longer on the meta path.

### How to Reproduce

(with setuptools>60 installed):
python -c "import sys; print(f'Before: {sys.meta_path}'); import setuptools; print(f'After: {sys.meta_path}')"


### Output

```console
Before: [<_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_virtualenv._Finder object at 0x7fe4d2806f70>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>]
After: [<_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_distutils_hack.DistutilsMetaFinder object at 0x7fe4d2688700>, <_virtualenv._Finder object at 0x7fe4d2806f70>, <class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib_external.PathFinder'>, <pkg_resources.extern.VendorImporter object at 0x7fe4d21c5460>, <setuptools.extern.VendorImporter object at 0x7fe4d1f2a100>]
```


### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:
I observed this behavior today too (https://github.com/pypa/pip/issues/10743#issuecomment-999768459). My instinct was that under normal circumstances that it would just get installed once. I was surprised to see that as a matter of course, it could get installed two or three times. I agree it might be preferable to be idempotent. My concern was with stability if it's added twice but removed once - what should happen?

One correction:

> It also appears that the intent was to remove the shim entirely once distutils has been loaded, but since it's added more than once, the removal in the import leaves two references to it in `sys.meta_path` after setuptools has been imported.

Originally, Setuptools was being cautious about the override and only invoking it during setuptools operations, but it became clear that it's necessary to supply 'distutils' more broadly and before setuptools is imported. At the same time, Setuptools wanted to be sure that if enabled, the local copy would be guaranteed to be present/imported, even if the shim had somehow not been installed previously (such as with python -S). So the short explanation is that it's complicated and there's no simple expectation but more of a mishmash of expectations.

I'd be happy to make it idempotent, but I'd also like to be sure the instance added by the .pth file is not removed by `ensure_local_distutils`.
I'll probably get to this in the future, but for now, there are more pressing issues with the distutils adoption, so I welcome others to devise an implementation that meets the above goals.
> So the short explanation is that it's complicated and there's no simple expectation but more of a mishmash of expectations.

@jaraco Yeah, I wondered about that, especially with a couple of the other methods in the metafinder that alter the behavior depending on what else is going on. There's also the question to whether you want it to be resilient to things like people clearing out `sys.modules` then re-importing it - not that anyone [would](https://github.com/ansible/ansible/blob/bcb64054edaa7cf636bd38b8ab0259f6fb93f3f9/test/units/_vendor/test_vendor.py#L22-L24) [ever](https://github.com/ansible/ansible/blob/ce6d8a143c6d0458c548f273d8564e4c2fac209f/test/units/module_utils/basic/test_imports.py#L23-L26) do such a silly thing... :stuck_out_tongue_winking_eye: 

#2959 is the dirt-simplest thing that keeps it from appearing multiple times, though to jive with your previous statement, it should probably also remove the code from `ensure_local_distutils` that zaps the loader off the metapath?
Yep. That's simple and would likely work, but isn't quite what I had in mind. Let me see if I can throw something together.
I observed this behavior today too (https://github.com/pypa/pip/issues/10743#issuecomment-999768459). My instinct was that under normal circumstances that it would just get installed once. I was surprised to see that as a matter of course, it could get installed two or three times. I agree it might be preferable to be idempotent. My concern was with stability if it's added twice but removed once - what should happen?

One correction:

> It also appears that the intent was to remove the shim entirely once distutils has been loaded, but since it's added more than once, the removal in the import leaves two references to it in `sys.meta_path` after setuptools has been imported.

Originally, Setuptools was being cautious about the override and only invoking it during setuptools operations, but it became clear that it's necessary to supply 'distutils' more broadly and before setuptools is imported. At the same time, Setuptools wanted to be sure that if enabled, the local copy would be guaranteed to be present/imported, even if the shim had somehow not been installed previously (such as with python -S). So the short explanation is that it's complicated and there's no simple expectation but more of a mishmash of expectations.

I'd be happy to make it idempotent, but I'd also like to be sure the instance added by the .pth file is not removed by `ensure_local_distutils`.
I'll probably get to this in the future, but for now, there are more pressing issues with the distutils adoption, so I welcome others to devise an implementation that meets the above goals.
> So the short explanation is that it's complicated and there's no simple expectation but more of a mishmash of expectations.

@jaraco Yeah, I wondered about that, especially with a couple of the other methods in the metafinder that alter the behavior depending on what else is going on. There's also the question to whether you want it to be resilient to things like people clearing out `sys.modules` then re-importing it - not that anyone [would](https://github.com/ansible/ansible/blob/bcb64054edaa7cf636bd38b8ab0259f6fb93f3f9/test/units/_vendor/test_vendor.py#L22-L24) [ever](https://github.com/ansible/ansible/blob/ce6d8a143c6d0458c548f273d8564e4c2fac209f/test/units/module_utils/basic/test_imports.py#L23-L26) do such a silly thing... :stuck_out_tongue_winking_eye: 

#2959 is the dirt-simplest thing that keeps it from appearing multiple times, though to jive with your previous statement, it should probably also remove the code from `ensure_local_distutils` that zaps the loader off the metapath?
Yep. That's simple and would likely work, but isn't quite what I had in mind. Let me see if I can throw something together.

Resolving patch:
diff --git a/_distutils_hack/__init__.py b/_distutils_hack/__init__.py
--- a/_distutils_hack/__init__.py
+++ b/_distutils_hack/__init__.py
@@ -3,6 +3,7 @@
 import re
 import importlib
 import warnings
+import contextlib
 
 
 is_pypy = '__pypy__' in sys.builtin_module_names
@@ -52,9 +53,8 @@ def ensure_local_distutils():
     # With the DistutilsMetaFinder in place,
     # perform an import to cause distutils to be
     # loaded from setuptools._distutils. Ref #2906.
-    add_shim()
-    importlib.import_module('distutils')
-    remove_shim()
+    with shim():
+        importlib.import_module('distutils')
 
     # check that submodules load as expected
     core = importlib.import_module('distutils.core')
@@ -129,6 +129,19 @@ def frame_file_is_setup(frame):
 DISTUTILS_FINDER = DistutilsMetaFinder()
 
 
+def ensure_shim():
+    DISTUTILS_FINDER in sys.meta_path or add_shim()
+
+
+@contextlib.contextmanager
+def shim():
+    add_shim()
+    try:
+        yield
+    finally:
+        remove_shim()
+
+
 def add_shim():
     sys.meta_path.insert(0, DISTUTILS_FINDER)
 
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -55,7 +55,7 @@ class install_with_pth(install):
         import os
         var = 'SETUPTOOLS_USE_DISTUTILS'
         enabled = os.environ.get(var, 'local') == 'local'
-        enabled and __import__('_distutils_hack').add_shim()
+        enabled and __import__('_distutils_hack').ensure_shim()
         """).lstrip().replace('\n', '; ')
 
     def initialize_options(self):


Test patch:
diff --git a/setuptools/tests/test_distutils_adoption.py b/setuptools/tests/test_distutils_adoption.py
--- a/setuptools/tests/test_distutils_adoption.py
+++ b/setuptools/tests/test_distutils_adoption.py
@@ -3,6 +3,7 @@
 import functools
 import subprocess
 import platform
+import textwrap
 
 import pytest
 import jaraco.envs
@@ -49,12 +50,24 @@ def find_distutils(venv, imports='distutils', env=None, **kwargs):
     return popen_text(venv.run)(cmd, env=env, **kwargs)
 
 
+def count_meta_path(venv, env=None):
+    py_cmd = textwrap.dedent(
+        """
+        import sys
+        is_distutils = lambda finder: finder.__class__.__name__ == "DistutilsMetaFinder"
+        print(len(list(filter(is_distutils, sys.meta_path))))
+        """)
+    cmd = ['python', '-c', py_cmd]
+    return int(popen_text(venv.run)(cmd, env=env))
+
+
 def test_distutils_stdlib(venv):
     """
     Ensure stdlib distutils is used when appropriate.
     """
     env = dict(SETUPTOOLS_USE_DISTUTILS='stdlib')
     assert venv.name not in find_distutils(venv, env=env).split(os.sep)
+    assert count_meta_path(venv, env=env) == 0
 
 
 def test_distutils_local_with_setuptools(venv):
@@ -64,6 +77,7 @@ def test_distutils_local_with_setuptools(venv):
     env = dict(SETUPTOOLS_USE_DISTUTILS='local')
     loc = find_distutils(venv, imports='setuptools, distutils', env=env)
     assert venv.name in loc.split(os.sep)
+    assert count_meta_path(venv, env=env) <= 1
 
 
 @pytest.mark.xfail('IS_PYPY', reason='pypy imports distutils on startup')
@@ -74,3 +88,4 @@ def test_distutils_local(venv):
     """
     env = dict(SETUPTOOLS_USE_DISTUTILS='local')
     assert venv.name in find_distutils(venv, env=env).split(os.sep)
+    assert count_meta_path(venv, env=env) <= 1