Repository: pex-tool/pex

Issue:
Pex fails to find RECORD for python-certifi-win32 1.6.1
Looks like:
```
$ pex python-certifi-win32==1.6.1
Traceback (most recent call last):
  File "/home/jsirois/.venv/pex/bin/pex", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/bin/pex.py", line 775, in main
    catch(
    ^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/result.py", line 103, in catch
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/bin/pex.py", line 798, in do_main
    pex_builder = build_pex(
                  ^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/bin/pex.py", line 681, in build_pex
    result = resolve(
             ^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/resolver.py", line 948, in resolve
    build_and_install_request.install_distributions(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/resolver.py", line 714, in install_distributions
    for install_result in execute_parallel(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/jobs.py", line 541, in execute_parallel
    yield spawn_result.spawned_job.await_result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/jobs.py", line 220, in await_result
    job.wait()
    ^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/jobs.py", line 81, in wait
    self._finalize_job()
    ^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/jobs.py", line 139, in _finalize_job
    self._finalizer(self._process.returncode)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/pip/tool.py", line 711, in fixup_install
    record = Record.from_prefix_install(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jsirois/.venv/pex/lib/python3.11/site-packages/pex/pep_376.py", line 445, in from_prefix_install
    raise RecordNotFoundError(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
pex.pep_376.RecordNotFoundError: Could not find the installation RECORD for python-certifi-win32 1.6.1 under /home/jsirois/.pex/installed_wheels/508fd4fb1730cad2d9dada061df737650c8cfaa205d64657faa4cc6a55384402/python_certifi_win32-1.6.1-py2.py3-none-any.whl.353db331daf74436a74d93cf324aa9a9
```


Discussion:
The `--prefix` install is:
```
$ pip install --prefix chroot --no-deps python_certifi_win32-1.6.1-py2.py3-none-any.whl
$ tree -L 4 chroot/
chroot/
└── lib
    ├── python3.10
    │   └── site-packages
    │       ├── certifi_win32
    │       └── python_certifi_win32-1.6.1.dist-info
    └── site-packages
        └── python-certifi-win32-init.pth

6 directories, 1 file
```

So the distribution tries to install a `.pth` file and misses. Those must belong to the site-packages directory of a venv which is `chroot/lib/python3.10/site-packages` in this case and not `chroot/lib/site-packages`.  The issue is in how they package the `.pth` file:
```
$ zipinfo -1 python_certifi_win32-1.6.1-py2.py3-none-any.whl 
certifi_win32/__init__.py
certifi_win32/bootstrap.py
certifi_win32/wincerts.py
certifi_win32/wrapt_certifi.py
certifi_win32/wrapt_pip.py
python_certifi_win32-1.6.1.data/data/lib/site-packages/python-certifi-win32-init.pth
python_certifi_win32-1.6.1.dist-info/LICENSE
python_certifi_win32-1.6.1.dist-info/METADATA
python_certifi_win32-1.6.1.dist-info/WHEEL
python_certifi_win32-1.6.1.dist-info/top_level.txt
python_certifi_win32-1.6.1.dist-info/RECORD
```
Instead of storing it as `python-certifi-win32-init.pth` at the top-level, they attempt to use the data files mechanism.

Pex will need to get smarter about finding the site-packages directory here:
https://github.com/pantsbuild/pex/blob/ddd3bd807856788d32037a539b5a36f28198dbf9/pex/common.py#L52-L59

Right now it finds the miss at `chroot/lib/site-packages` which just has the `.pth` and, indeed, no `RECORD`.
Ok, throwing various versions of PyPy in the mix:
```
$ for chroot in chroot*; do tree -L 4 -d $chroot && echo "---"; done
chroot
└── lib
    ├── python3.10
    │   └── site-packages
    │       ├── certifi_win32
    │       └── python_certifi_win32-1.6.1.dist-info
    └── site-packages

6 directories
---
chroot-pypy2.7
├── lib
│   └── site-packages
└── site-packages
    ├── certifi_win32
    └── python_certifi_win32-1.6.1.dist-info

5 directories
---
chroot-pypy3.6
├── lib
│   └── site-packages
└── site-packages
    ├── certifi_win32
    │   └── __pycache__
    └── python_certifi_win32-1.6.1.dist-info

6 directories
---
chroot-pypy3.7
├── lib
│   └── site-packages
└── site-packages
    ├── certifi_win32
    │   └── __pycache__
    └── python_certifi_win32-1.6.1.dist-info

6 directories
---
chroot-pypy3.8
└── lib
    ├── pypy3.8
    │   └── site-packages
    │       ├── certifi_win32
    │       └── python_certifi_win32-1.6.1.dist-info
    └── site-packages

6 directories
---
chroot-pypy3.9
└── lib
    ├── pypy3.9
    │   └── site-packages
    │       ├── certifi_win32
    │       └── python_certifi_win32-1.6.1.dist-info
    └── site-packages

6 directories
---
```

So there are two different layouts for the legitimate site-packages depending on the interpreter used to run `pip install`. Either `chroot/site-packages` for old PyPy or `chroot/lib/<py major minor>/site-packages` for newer PyPy and all CPython.

Resolving patch:
diff --git a/pex/pep_376.py b/pex/pep_376.py
--- a/pex/pep_376.py
+++ b/pex/pep_376.py
@@ -15,19 +15,12 @@
 from fileinput import FileInput
 
 from pex import dist_metadata, hashing
-from pex.common import (
-    filter_pyc_dirs,
-    filter_pyc_files,
-    find_site_packages,
-    is_python_script,
-    safe_mkdir,
-    safe_open,
-)
+from pex.common import filter_pyc_dirs, filter_pyc_files, is_python_script, safe_mkdir, safe_open
 from pex.compatibility import get_stdout_bytes_buffer, urlparse
 from pex.dist_metadata import Distribution, EntryPoint
 from pex.interpreter import PythonInterpreter
 from pex.typing import TYPE_CHECKING, cast
-from pex.venv.virtualenv import Virtualenv
+from pex.venv.virtualenv import InvalidVirtualenvError, Virtualenv, find_site_packages_dir
 
 if TYPE_CHECKING:
     from typing import (
@@ -424,13 +417,14 @@ def from_prefix_install(
         prefix_dir,  # type: str
         project_name,  # type: str
         version,  # type: str
+        interpreter=None,  # type: Optional[PythonInterpreter]
     ):
-
-        site_packages = find_site_packages(prefix_dir=prefix_dir)
-        if site_packages is None:
+        try:
+            site_packages = find_site_packages_dir(venv_dir=prefix_dir, interpreter=interpreter)
+        except InvalidVirtualenvError as e:
             raise RecordNotFoundError(
                 "Could not find a site-packages directory under installation prefix "
-                "{prefix_dir}.".format(prefix_dir=prefix_dir)
+                "{prefix_dir}: {err}".format(prefix_dir=prefix_dir, err=e)
             )
 
         site_packages_listing = [
diff --git a/pex/pip/tool.py b/pex/pip/tool.py
--- a/pex/pip/tool.py
+++ b/pex/pip/tool.py
@@ -712,6 +712,7 @@ def fixup_install(returncode):
                 prefix_dir=install_dir,
                 project_name=project_name_and_version.project_name,
                 version=project_name_and_version.version,
+                interpreter=interpreter,
             )
             record.fixup_install(interpreter=interpreter)
 
diff --git a/pex/venv/virtualenv.py b/pex/venv/virtualenv.py
--- a/pex/venv/virtualenv.py
+++ b/pex/venv/virtualenv.py
@@ -77,6 +77,37 @@ class InvalidVirtualenvError(Exception):
     """Indicates a virtualenv is malformed."""
 
 
+def find_site_packages_dir(
+    venv_dir,  # type: str
+    interpreter=None,  # type: Optional[PythonInterpreter]
+):
+    # type: (...) -> str
+    interpreter = interpreter or PythonInterpreter.get()
+    if (
+        interpreter.identity.interpreter == "PyPy"
+        and interpreter.version[:2] <= (3, 7)
+    ):
+        site_packages_dir = os.path.join(venv_dir, "site-packages")
+    else:
+        site_packages_dir = os.path.join(
+            venv_dir,
+            "lib",
+            "{python}{major_minor}".format(
+                python="pypy" if interpreter.identity.interpreter == "PyPy" else "python",
+                major_minor=".".join(map(str, interpreter.version[:2]))
+            ),
+            "site-packages",
+        )
+    if not os.path.isdir(site_packages_dir):
+        raise InvalidVirtualenvError(
+            "The virtualenv at {venv_dir} is not valid. The expected site-packages directory "
+            "at {site_packages_dir} does not exist.".format(
+                venv_dir=venv_dir, site_packages_dir=site_packages_dir
+            )
+        )
+    return site_packages_dir
+
+
 class Virtualenv(object):
     @classmethod
     def enclosing(cls, python):
@@ -198,28 +229,7 @@ def __init__(
                     venv_dir=self._venv_dir, python_exe_path=python_exe_path, err=e
                 )
             )
-        if (
-            self._interpreter.identity.interpreter == "PyPy"
-            and self._interpreter.version[:2] <= (3, 7)
-        ):
-            self._site_packages_dir = os.path.join(venv_dir, "site-packages")
-        else:
-            self._site_packages_dir = os.path.join(
-                venv_dir,
-                "lib",
-                "{python}{major_minor}".format(
-                    python="pypy" if self._interpreter.identity.interpreter == "PyPy" else "python",
-                    major_minor=".".join(map(str, self._interpreter.version[:2]))
-                ),
-                "site-packages",
-            )
-        if not os.path.isdir(self._site_packages_dir):
-            raise InvalidVirtualenvError(
-                "The virtualenv at {venv_dir} is not valid. The expected site-packages directory "
-                "at {site_packages_dir} does not exist.".format(
-                    venv_dir=venv_dir, site_packages_dir=self._site_packages_dir
-                )
-            )
+        self._site_packages_dir = find_site_packages_dir(venv_dir, self._interpreter)
         self._base_bin = frozenset(_iter_files(self._bin_dir))
         self._sys_path = None  # type: Optional[Tuple[str, ...]]
 


Test patch:
diff --git a/tests/integration/test_issue_1861.py b/tests/integration/test_issue_1861.py
new file mode 100644
--- /dev/null
+++ b/tests/integration/test_issue_1861.py
@@ -0,0 +1,33 @@
+# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+
+import os
+
+from pex.testing import make_env, run_pex_command
+from pex.typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from typing import Any
+
+
+def test_confounding_site_packages_directory(tmpdir):
+    # type: (Any) -> None
+
+    pex_root = os.path.join(str(tmpdir), "pex_root")
+    local_app_data = os.path.join(str(tmpdir), "local_app_data")
+    result = run_pex_command(
+        args=[
+            "--pex-root",
+            pex_root,
+            "--runtime-pex-root",
+            pex_root,
+            "python-certifi-win32==1.6.1",
+            "--",
+            "-c",
+            "import certifi_win32; print(certifi_win32.__file__)",
+        ],
+        # N.B.: certifi_win32 requires `LOCALAPPDATA` be set in the env to import at all.
+        env=make_env(LOCALAPPDATA=local_app_data),
+    )
+    result.assert_success()
+    assert result.output.startswith(os.path.join(pex_root, "installed_wheels"))