Repository: pex-tool/pex

Issue:
Replace deprecated imp module with importlib

This project uses the [`imp` module](https://docs.python.org/3/library/imp.html) which has been deprecated since Python 3.4 and removed in 3.12:

* Raised `PendingDeprecationWarning` since [3.4 (2014)](https://github.com/python/cpython/commit/e4f41deccf94ccc798b1eb1f44657ade66669a60)
* Raised `DeprecationWarning` since [3.5 (2015)](https://github.com/python/cpython/commit/c0d91aff9a3b91307b26e8b7c34dfbf27bbdd43a)
* Updated `DeprecationWarning` to say removal in 3.12 since [3.10 (2021)](https://github.com/python/cpython/commit/dc6d3e1e4c0c1e4b2210edab8fb4762569dc2936)
* Removal planned for [3.12](https://github.com/python/cpython/issues/98040) [(2023)](https://github.com/python/cpython/pull/98573)

Python 3.12 is [set for release on 2023-10-02](https://devguide.python.org/versions/) and this library is one of the [top 5,000 most-downloaded from PyPI](https://hugovk.github.io/top-pypi-packages/).

Please could you upgrade to use `importlib`? The [`imp` docs](https://docs.python.org/3.11/library/imp.html) have suggestions on what to use to replace each function and constant.




Discussion:
Hi @hugovk 

Please note that `pex` still supports Python 2, so it's not as easy to just remove the use of `imp` as `importlib` was introduced first in Python 3.1.

https://github.com/pantsbuild/pex/blob/ff220b9e41484450e0c78136f98d6899f2e2325c/pyproject.toml#L19-L28

Fair enough, some compatibility code will be needed.

Heads up: GitHub Actions is removing Python 2.7 in a couple of weeks: https://github.com/actions/runner-images/issues/7401
Unfortunately compatibility code will not do the trick. All the imp use is in vendored code:
```
$ git grep -e "import imp\b" -e "from imp import"
pex/vendor/_vendored/packaging_20_9/packaging/tags.py:    import imp
pex/vendor/_vendored/pip/pip/_internal/utils/compat.py:    import imp
pex/vendor/_vendored/pip/pip/_vendor/distlib/_backport/misc.py:    from imp import cache_from_source
pex/vendor/_vendored/pip/pip/_vendor/distlib/compat.py:        from imp import cache_from_source
pex/vendor/_vendored/pip/pip/_vendor/distlib/wheel.py:import imp
pex/vendor/_vendored/pip/pip/_vendor/packaging/tags.py:    import imp
pex/vendor/_vendored/pip/pip/_vendor/pkg_resources/__init__.py:    import imp as _imp
pex/vendor/_vendored/setuptools/pkg_resources/__init__.py:    import imp as _imp
pex/vendor/_vendored/setuptools/setuptools/_vendor/packaging/tags.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/command/build_ext.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/py27compat.py:    import imp
pex/vendor/_vendored/setuptools/setuptools/py27compat.py:    from imp import PY_COMPILED, PY_FROZEN, PY_SOURCE  # noqa
pex/vendor/_vendored/setuptools/setuptools/site-patch.py:                import imp  # Avoid import loop in Python 3
pex/vendor/_vendored/wheel/wheel/vendored/packaging/tags.py:    import imp
pex/venv/virtualenv_16.7.12_py:        import imp
```

IOW Pex vendors an old single-file version of virtualenv to support Pythons without -mvenv support and a bootstrap Pip that works with all versions of Python Pex supports. It uses the vendored Pip to upgrade to newer versions of Pip it supports when asked, but the baseline vendored Pip must work with all versions of Python we support.

I had not created a tracking issue yet for the Pex 3.x release, but that fundamentally needs to decide what to do about vendored Pip and legacy support; i.e.: maintain two release branches - one supporting `>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,<3.12` and one supporting `>=3.7,<3.next` - or just cut the Pex 2.x series off and our existing Python 2.7 users just have to deal with that.

Thanks for the GHA heads up, I'll switch to [pyenv-action](https://github.com/gabrielfalcao/pyenv-action) where needed. Pex already uses pyenv to maintain a pool of always-available `2.7` and `3.{8,9,10}` CPythons and that works well.
Ah - this threw me for a loop. It's true that pre-installed Python 2.7 in the tool cache on the image goes away, but then `actions/setup-python` just grabs the release from https://github.com/actions/python-versions/releases via https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json. I have a hard time believing they'll nuke pre-built binary releases, at least I'm not worried about them doing that prior to Pex deciding and implementing a 3.x plan.

Resolving patch:
diff --git a/pex/pex_builder.py b/pex/pex_builder.py
--- a/pex/pex_builder.py
+++ b/pex/pex_builder.py
@@ -119,10 +119,12 @@ def __entry_point_from_filename__(filename):
 if '__file__' in locals() and __file__ is not None and os.path.exists(__file__):
   __entry_point__ = __entry_point_from_filename__(__file__)
 elif '__loader__' in locals():
-  from pkgutil import ImpLoader
   if hasattr(__loader__, 'archive'):
     __entry_point__ = __loader__.archive
-  elif isinstance(__loader__, ImpLoader):
+  elif hasattr(__loader__, 'get_filename'):
+    # The source of the loader interface has changed over the course of Python history from
+    # `pkgutil.ImpLoader` to `importlib.abc.Loader`, but the existence and semantics of
+    # `get_filename` has remained constant; so we just check for the method.
     __entry_point__ = __entry_point_from_filename__(__loader__.get_filename())
 
 if __entry_point__ is None:
diff --git a/pex/third_party/__init__.py b/pex/third_party/__init__.py
--- a/pex/third_party/__init__.py
+++ b/pex/third_party/__init__.py
@@ -364,7 +364,16 @@ def uninstall(self):
                 loader.unload()
             _tracer().log("Uninstalled {}".format(self), V=3)
 
-    # The PEP-302 finder API.
+    def find_spec(self, fullname, path, target=None):
+        # Python 2.7 does not know about this API and does not use it.
+        from importlib.util import spec_from_loader  # type: ignore[import]
+
+        loader = self.find_module(fullname, path)
+        if loader:
+            return spec_from_loader(fullname, loader)
+        return None
+
+    # The Legacy PEP-302 finder API.
     # See: https://www.python.org/dev/peps/pep-0302/#specification-part-1-the-importer-protocol
     def find_module(self, fullname, path=None):
         for importable in self._importables:


Test patch: