Repository: pex-tool/pex

Issue:
Universal lockfile contains the wrong dists
Generating this universal lockfile on MacOS:

```
python3 -m pex.cli lock create $'--output=lock.json' $'--style=universal' --resolver-version pip-2020-resolver $'--indent=2'  --manylinux manylinux2014 --interpreter-constraint $'CPython<3.10,>=3.7' $'SecretStorage==3.3.1' $'apache-airflow==2.1.4' $'authlib==0.15.2' $'moto==1.3.14' $'paramiko==2.7.2' $'snowflake-connector-python[secure-local-storage]==2.4.6'
```

creates a lockfile in which the `cryptography` package has only one artifact, which is an arbitrary wheel (pypy 3.7 on windows on amd64):

```
$ cat lock.json | jq '.locked_resolves[].locked_requirements[] | select(.project_name == "cryptography")'
{
  "artifacts": [
    {
      "algorithm": "sha256",
      "hash": "cd65b60cfe004790c795cc35f272e41a3df4631e2fb6b35aa7ac6ef2859d554e",
      "url": "https://files.pythonhosted.org/packages/35/23/b7dc7a019e1a34b024498510686a6b6a400caf6ecd1c0ba102e27dd3658f/cryptography-3.4.8-pp37-pypy37_pp73-win_amd64.whl"
    }
  ],
  ...,
  "project_name": "cryptography",
  "requires_python": ">=3.6",
  "version": "3.4.8"
}
```

Note that cryptography 3.4.8 has many wheels, and an sdist: https://pypi.org/project/cryptography/3.4.8/#files

Note also that the only valid dist for the specified interpreter constraints (CPython<3.10,>=3.7) is the sdist, so the number of dists is right, just not the one provided. It may or may not be coincidence that the arbitrary wheel is the first wheel listed on PyPI.


Discussion:
I repro on a linux box - which should be the case for any universal lock.
Note that naively resolving just `cryptography==3.4.8` instead of the exact combination of requirements above yields a lockfile with 16 dists (out of the 19 available) many of which do not match the interpreter constraints.  (But also removing the interpreter constraints yields a lockfile with all 19 dists).
Also note that removing the interpreter constraints from the original resolve, so:

```python3 -m pex.cli lock create $'--output=lock.json' $'--style=universal' --resolver-version pip-2020-resolver $'--indent=2'  --manylinux manylinux2014  $'SecretStorage==3.3.1' $'apache-airflow==2.1.4' $'authlib==0.15.2' $'moto==1.3.14' $'paramiko==2.7.2' $'snowflake-connector-python[secure-local-storage]==2.4.6'```

takes a very long time to run (I have not yet seen it complete and have been watching it for over 20 minutes).
Yeah, almost any universal lock without ICs is asking alot.
The pip log used to derive the artifacts does in fact have them listed as the code expects FWICT:
```
$ grep -n "Found link" pip.log | grep -i cryptography | grep -i "3\.4\.8"
6216:2022-04-07T15:08:20,477   Found link https://files.pythonhosted.org/packages/00/e4/da1509e64a92e32ec8df97f5c4372e7f0e56b5b0bad299da61a9632b900c/cryptography-3.4.8-cp36-abi3-macosx_10_10_x86_64.whl#sha256=a00cf305f07b26c351d8d4e1af84ad7501eca8a342dedf24a7acb0e7b7406e14 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6217:2022-04-07T15:08:20,480   Found link https://files.pythonhosted.org/packages/7d/d1/41c840255bd2b416e3653f4fef31224ba87576cc6b75b791de5c752126cb/cryptography-3.4.8-cp36-abi3-macosx_11_0_arm64.whl#sha256=f44d141b8c4ea5eb4dbc9b3ad992d45580c1d22bf5e24363f2fbf50c2d7ae8a7 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6218:2022-04-07T15:08:20,483   Found link https://files.pythonhosted.org/packages/28/d4/35dc171f470704403e35d639b1b4ad38f6a6e07c39aab2fad9437b88d4c3/cryptography-3.4.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl#sha256=0a7dcbcd3f1913f664aca35d47c1331fce738d44ec34b7be8b9d332151b0b01e (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6219:2022-04-07T15:08:20,487   Found link https://files.pythonhosted.org/packages/f5/2e/a61d992645593be442b2666252f5d2fceb0777dfd2675a75d8914a353cd3/cryptography-3.4.8-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl#sha256=34dae04a0dce5730d8eb7894eab617d8a70d0c97da76b905de9efb7128ad7085 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6220:2022-04-07T15:08:20,490   Found link https://files.pythonhosted.org/packages/96/07/4d23f8e34e56d8eeb2c37cd5924928a01c3dd756a1d99e470181bc57551e/cryptography-3.4.8-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=1eb7bb0df6f6f583dd8e054689def236255161ebbcf62b226454ab9ec663746b (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6221:2022-04-07T15:08:20,493   Found link https://files.pythonhosted.org/packages/17/7e/cec41c164b42db6364b10e13d218c40e604cc6745448b9fcd9318f2d9556/cryptography-3.4.8-cp36-abi3-manylinux_2_24_x86_64.whl#sha256=9965c46c674ba8cc572bc09a03f4c649292ee73e1b683adb1ce81e82e9a6a0fb (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6222:2022-04-07T15:08:20,497   Found link https://files.pythonhosted.org/packages/31/6b/12678b9f4aade4cdd322d2366421cb9bca978d82a4b2ad556f7dd80811ed/cryptography-3.4.8-cp36-abi3-musllinux_1_1_aarch64.whl#sha256=3c4129fc3fdc0fa8e40861b5ac0c673315b3c902bbdc05fc176764815b43dd1d (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6223:2022-04-07T15:08:20,500   Found link https://files.pythonhosted.org/packages/d7/05/01b2482b88edc0ad8f99bbf763dd34895c71f0a87ce61e89813eba481187/cryptography-3.4.8-cp36-abi3-musllinux_1_1_x86_64.whl#sha256=695104a9223a7239d155d7627ad912953b540929ef97ae0c34c7b8bf30857e89 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6224:2022-04-07T15:08:20,503   Found link https://files.pythonhosted.org/packages/03/27/e2ccbfcdaa6768110676bfdc60012322e0d09b83227d52b25a89cb6e2914/cryptography-3.4.8-cp36-abi3-win32.whl#sha256=21ca464b3a4b8d8e86ba0ee5045e103a1fcfac3b39319727bc0fc58c09c6aff7 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6225:2022-04-07T15:08:20,507   Found link https://files.pythonhosted.org/packages/b6/32/e82e51a64280b16647af538bbfe4424285a41f01687899405e997a6e39b4/cryptography-3.4.8-cp36-abi3-win_amd64.whl#sha256=3520667fda779eb788ea00080124875be18f2d8f0848ec00733c0ec3bb8219fc (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6229:2022-04-07T15:08:20,510   Found link https://files.pythonhosted.org/packages/51/07/e7fd9090777196af438661b0c6471a0bab2e8697c26747b8f400b1715668/cryptography-3.4.8-pp37-pypy37_pp73-macosx_10_10_x86_64.whl#sha256=d9ec0e67a14f9d1d48dd87a2531009a9b251c02ea42851c060b25c782516ff06 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6230:2022-04-07T15:08:20,513   Found link https://files.pythonhosted.org/packages/8b/41/6eefbe046f7d49beeb17d3387e22cf8fc656c683ef4e08694b74834f82fb/cryptography-3.4.8-pp37-pypy37_pp73-manylinux_2_12_x86_64.manylinux2010_x86_64.whl#sha256=5b0fbfae7ff7febdb74b574055c7466da334a5371f253732d7e2e7525d570498 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6231:2022-04-07T15:08:20,517   Found link https://files.pythonhosted.org/packages/77/1b/01aa2cacdba8c8debdd5a550dcc8f00fbef19d603d9ce5a2c26c2a133056/cryptography-3.4.8-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=94fff993ee9bc1b2440d3b7243d488c6a3d9724cc2b09cdb297f6a886d040ef7 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6232:2022-04-07T15:08:20,520   Found link https://files.pythonhosted.org/packages/d8/e1/4bd5056f436b23f9059db1ff897ab3747fa7726a12c6ec96486386222b52/cryptography-3.4.8-pp37-pypy37_pp73-manylinux_2_24_x86_64.whl#sha256=8695456444f277af73a4877db9fc979849cd3ee74c198d04fc0776ebc3db52b9 (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6233:2022-04-07T15:08:20,523   Found link https://files.pythonhosted.org/packages/35/23/b7dc7a019e1a34b024498510686a6b6a400caf6ecd1c0ba102e27dd3658f/cryptography-3.4.8-pp37-pypy37_pp73-win_amd64.whl#sha256=cd65b60cfe004790c795cc35f272e41a3df4631e2fb6b35aa7ac6ef2859d554e (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
6234:2022-04-07T15:08:20,527   Found link https://files.pythonhosted.org/packages/cc/98/8a258ab4787e6f835d350639792527d2eb7946ff9fc0caca9c3f4cf5dcfe/cryptography-3.4.8.tar.gz#sha256=94cc5ed4ceaefcbe5bf38c8fba6a21fc1d365bb8fb826ea1688e3370b2e24a1c (from https://pypi.org/simple/cryptography/) (requires-python:>=3.6), version: 3.4.8
```

I'll debug the `Locker` to see why its missing associating these links with the finished resolve.

Resolving patch:
diff --git a/pex/pip/tool.py b/pex/pip/tool.py
--- a/pex/pip/tool.py
+++ b/pex/pip/tool.py
@@ -415,8 +415,6 @@ def analyze(self, line):
 
                     # VCS requirements are satisfied by a singular source; so we need not consult
                     # links collected in this round.
-                    self._links.clear()
-
                     self._resolved_requirements.append(
                         ResolvedRequirement(
                             requirement=requirement,
@@ -467,7 +465,6 @@ def analyze(self, line):
 
                 additional_artifacts = self._links[project_name_and_version]
                 additional_artifacts.discard(partial_artifact)
-                self._links.clear()
 
                 self._resolved_requirements.append(
                     ResolvedRequirement(


Test patch:
diff --git a/tests/integration/cli/commands/test_issue_1711.py b/tests/integration/cli/commands/test_issue_1711.py
new file mode 100644
--- /dev/null
+++ b/tests/integration/cli/commands/test_issue_1711.py
@@ -0,0 +1,114 @@
+# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+
+import os
+
+from pex.cli.testing import run_pex3
+from pex.compatibility import PY3
+from pex.pep_440 import Version
+from pex.pep_503 import ProjectName
+from pex.resolve import lockfile
+from pex.resolve.locked_resolve import Artifact, LockedRequirement
+from pex.resolve.resolved_requirement import Fingerprint
+from pex.testing import run_pex_command
+from pex.typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from typing import Any
+
+
+def pypi_artifact(
+    hash,  # type: str
+    path,  # type: str
+):
+    # type: (...) -> Artifact
+    return Artifact.from_url(
+        url="https://files.pythonhosted.org/packages/{}".format(path),
+        fingerprint=Fingerprint(algorithm="sha256", hash=hash),
+    )
+
+
+def test_backtrack_links_preserved(tmpdir):
+    # type: (Any) -> None
+
+    lock = os.path.join(str(tmpdir), "lock")
+    create_lock_args = [
+        "lock",
+        "create",
+        "--resolver-version",
+        "pip-2020-resolver",
+        "--style",
+        "universal",
+        "--interpreter-constraint",
+        ">=3.7,<3.10",
+        "psutil",
+        "psutil<5.5",  # force a back-track
+        "-o",
+        lock,
+        "--indent",
+        "2",
+    ]
+
+    def lock_as_json():
+        with open(lock) as fp:
+            return fp.read()
+
+    def assert_psutil_basics():
+        # type: () -> LockedRequirement
+        lock_file = lockfile.load(lock)
+        assert 1 == len(
+            lock_file.locked_resolves
+        ), "Expected 1 resolve for universal style:\n{json}".format(json=lock_as_json())
+        locked_resolve = lock_file.locked_resolves[0]
+
+        locked_requirements_by_project_name = {
+            locked_requirement.pin.project_name: locked_requirement
+            for locked_requirement in locked_resolve.locked_requirements
+        }
+        psutil = locked_requirements_by_project_name.get(ProjectName("psutil"))
+        assert psutil is not None, "Expected lock to resolve psutil:\n{json}".format(
+            json=lock_as_json()
+        )
+        assert Version("5.4.8") == psutil.pin.version, (
+            "Expected lock to resolve psutil to <5.5 due to the second requirement but otherwise "
+            "as high as possible, which should be 5.4.8 but was: {version}\n{json}".format(
+                version=psutil.pin.version, json=lock_as_json()
+            )
+        )
+        return psutil
+
+    # 1st prove this does the wrong thing on prior broken versions of Pex.
+    # N.B.: For some reason, this works with old Pex under Python 2.7; i.e.: It appears Pip behaves
+    # differently - likely because of some collection implementation difference.
+    if PY3:
+        run_pex_command(
+            args=["pex==2.1.77", "-c", "pex3", "--"] + create_lock_args
+        ).assert_success()
+        psutil_old = assert_psutil_basics()
+        assert 0 == len(psutil_old.additional_artifacts), (
+            "Expected old versions of Pex to incorrectly wipe out the additional artifacts "
+            "database when backtracking needs to retrieve saved links later:\n{json}".format(
+                json=lock_as_json()
+            )
+        )
+
+    # Now show it currently works.
+    run_pex3(*create_lock_args).assert_success()
+    psutil_current = assert_psutil_basics()
+    assert {
+        pypi_artifact(
+            hash="1c71b9716790e202a00ab0931a6d1e25db1aa1198bcacaea2f5329f75d257fff",
+            path="50/00/ae52663b879333aa5c65fc9a87ddc24169f8fdd1831762a1ba9c9be7740d/psutil-5.4.8-cp37-cp37m-win_amd64.whl",
+        ),
+        pypi_artifact(
+            hash="bfcea4f189177b2d2ce4a34b03c4ac32c5b4c22e21f5b093d9d315e6e253cd81",
+            path="21/1e/fe6731e5f03ddf2e57d5b307f25bba294262bc88e27a0fbefdb3515d1727/psutil-5.4.8-cp37-cp37m-win32.whl",
+        ),
+        pypi_artifact(
+            hash="6e265c8f3da00b015d24b842bfeb111f856b13d24f2c57036582568dc650d6c3",
+            path="e3/58/0eae6e4466e5abf779d7e2b71fac7fba5f59e00ea36ddb3ed690419ccb0f/psutil-5.4.8.tar.gz",
+        ),
+    } == set(psutil_current.iter_artifacts()), (
+        "Expected a full set of artifacts even after the lock resolve backtracked from "
+        "psutil latest to psutil<5.5 before settling:\n{json}".format(json=lock_as_json())
+    )