Repository: pantsbuild/pants

Issue:
pants 2.5 dependency issues with cross-platform targets
I have a pants build with one target that tries to build a cross-platform pex file:

```
   platforms = [
      'linux-x86_64-cp-37-m',
      'macosx-10.14-x86_64-cp-37-m'
   ]
```

That built fine on pants 2.4 (and still does) but started to fail when I upgraded to pants 2.5 with the following:

```
(venv) oliver@marmaduke:~/Documents/code/central2/python$ ./pants package db_cli/::
10:51:26.74 [INFO] Completed: Resolving constraints.txt
10:51:26.74 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 234, in _run_inner
    return self._perform_run(goals)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 173, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/bin/local_pants_runner.py", line 190, in _perform_run_body
    return self.graph_session.run_goal_rules(
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/init/engine_initializer.py", line 135, in run_goal_rules
    exit_code = self.scheduler_session.run_goal_rule(
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 530, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/oliver/.cache/pants/setup/bootstrap-Linux-x86_64/2.5.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 498, in _raise_on_error
    raise ExecutionError(
Exception message: 1 Exception encountered:
  ProcessExecutionFailure: Process 'Resolving constraints.txt' failed with exit code 1.
stdout:
stderr:
ERROR: Could not find a version that satisfies the requirement gast==0.2.2
ERROR: No matching distribution found for gast==0.2.2
pid 795584 -> /home/oliver/.cache/pants/named_caches/pex_root/venvs/7f48ea14fe9a31ad7cc2422df7a949fe8899372d/cc48858524bf3820a737c19c7f14d57d4a5c4208/pex --disable-pip-version-check --no-python-version-warning --exists-action a --isolated -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root --log /tmp/process-executionlRKkVb/.tmp/tmpyfr58o_p/pip.log download --dest /tmp/process-executionlRKkVb/.tmp/tmpjmhkeakd/macosx_10_14_x86_64-cp-37-cp37m --platform macosx_10_14_x86_64 --implementation cp --python-version 37 --abi cp37m --only-binary :all: --constraint constraints.txt absl-py==0.13.0 astor==0.8.1 attrs==21.2.0 backoff==1.10.0 cachetools==4.2.2 certifi==2021.5.30 chardet==4.0.0 gast==0.2.2 google-api-core==1.30.0 google-auth==1.31.0 google-cloud-core==1.7.0 google-cloud-firestore==2.1.3 google-pasta==0.2.0 googleapis-common-protos==1.53.0 grpcio==1.38.0 h5py==2.10.0 idna==2.10 importlib-metadata==4.5.0 iniconfig==1.1.1 keras-applications==1.0.8 keras-preprocessing==1.1.2 labelbox==2.6.0 markdown==3.3.4 ndjson==0.3.1 numpy==1.18.5 opencv-python-headless==4.5.2.54 opt-einsum==3.3.0 packaging==20.9 pluggy==0.13.1 proto-plus==1.18.1 protobuf==3.17.3 py==1.10.0 pyasn1-modules==0.2.8 pyasn1==0.4.8 pydantic==1.8.2 pyparsing==2.4.7 pytest==6.2.4 pytz==2021.1 requests==2.25.1 rsa==4.7.2 six==1.16.0 tensorboard==1.15.0 tensorflow-estimator==1.15.1 tensorflow==1.15.5 termcolor==1.1.0 toml==0.10.2 typing-extensions==3.10.0.0 urllib3==1.26.5 werkzeug==2.0.1 wheel==0.36.2 wrapt==1.12.1 zipp==3.4.1 --index-url https://pypi.org/simple/ --retries 5 --timeout 15 exited with 1 and STDERR:
None
(Use --print-stacktrace to see more error details.)
```

Relevant bits of `pants.toml`:
```
[python-setup]
interpreter_constraints = ['CPython==3.7.*']
requirement_constraints = 'constraints.txt'
```

Also note:

1. We depend on `gast` only transitively via TensorFlow and this target does not depend on TensorFlow or gast so there's no need to resolve that dependency
2. [gast](https://github.com/serge-sans-paille/gast/) appears to be pure-python so there shouldn't be any issue finding a compatible version for OSX.


Discussion:
Because this repo sets `[python-setup] requirement_constraints`, I think that this is related to the changes that lead to #11985. We're now always executing a single resolve (i.e., the `nondeployables` setting was deprecated), and in this context, that means that all of the requirements in the constraints file must be resolvable on all of the platforms used in the repo. cc @jsirois , @Eric-Arellano 

A hitch in this hypothesis though is that `gast` should be just fine resolving for both platforms, so it's odd that that was the reported-missing package.
There is no hitch. There is no wheel: https://pypi.org/project/gast/0.2.2/#files

Foreign resolves (platforms are used here instead of ICs) must have all dists available as pre-built wheels. Pex has an option to try to turn platforms into local interpreters so it can build sdists when that's all that's available, but Pants doesn't use that option or plumb it.
So the bug here is a "lock file" really only works for a single interpreter (or a single platform - aka foreign interpreter). I had pointed out Pants own lockfile is buggy. If you regenerate Pants constraints.txt with python 3.7 and python 3.9 you get different results. That much is correct! Pants internals though assume 1 "lockfile" works for many interpreters, that can be incorrect. And this bug shows we further don't consider the platform case.
See #12200
Thanks @oliverdain! 

In the meantime to redesigning our lockfile support in 2.6 or 2.7, I think the best fix is to change our `pex_from_targets` code to not use the global resolve (but still use constraints file) if `platforms` is specified. Thoughts?

That sounds right. The bug is still present for IC ranges as layed out in #12200 after that fix, but this gets back status quo.
Well, using constraints at all really isn't correct, since who knows what the constraints are for that Linux platform when I'm running on Mac, but I think that was status quo.
Cool. Ack that the status quo is still broken. I'll pair with @wilsonliam today on fixing this and cherry-pick to 2.5.

Resolving patch:
diff --git a/src/python/pants/backend/python/util_rules/pex_from_targets.py b/src/python/pants/backend/python/util_rules/pex_from_targets.py
--- a/src/python/pants/backend/python/util_rules/pex_from_targets.py
+++ b/src/python/pants/backend/python/util_rules/pex_from_targets.py
@@ -271,7 +271,9 @@ async def pex_from_targets(request: PexFromTargetsRequest, python_setup: PythonS
                 f"entries for the following requirements: {', '.join(unconstrained_projects)}"
             )
 
-        if python_setup.resolve_all_constraints:
+        # NB: it isn't safe to resolve against the whole constraints file if
+        # platforms are in use. See https://github.com/pantsbuild/pants/issues/12222.
+        if python_setup.resolve_all_constraints and not request.platforms:
             if unconstrained_projects:
                 logger.warning(
                     "Ignoring `[python_setup].resolve_all_constraints` option because constraints "


Test patch:
diff --git a/src/python/pants/backend/python/util_rules/pex_from_targets_test.py b/src/python/pants/backend/python/util_rules/pex_from_targets_test.py
--- a/src/python/pants/backend/python/util_rules/pex_from_targets_test.py
+++ b/src/python/pants/backend/python/util_rules/pex_from_targets_test.py
@@ -16,7 +16,7 @@
 
 from pants.backend.python.target_types import PythonLibrary, PythonRequirementLibrary
 from pants.backend.python.util_rules import pex_from_targets
-from pants.backend.python.util_rules.pex import Pex, PexRequest, PexRequirements
+from pants.backend.python.util_rules.pex import Pex, PexPlatforms, PexRequest, PexRequirements
 from pants.backend.python.util_rules.pex_from_targets import PexFromTargetsRequest
 from pants.build_graph.address import Address
 from pants.engine.internals.scheduler import ExecutionError
@@ -267,3 +267,34 @@ def get_pex_request(
 
     # Shouldn't error, as we don't explicitly set --resolve-all-constraints.
     get_pex_request(None, resolve_all_constraints=None)
+
+
+def test_issue_12222(rule_runner: RuleRunner) -> None:
+    rule_runner.write_files(
+        {
+            "constraints.txt": "foo==1.0\nbar==1.0",
+            "BUILD": dedent(
+                """
+            python_requirement_library(name="foo",requirements=["foo"])
+            python_requirement_library(name="bar",requirements=["bar"])
+            python_library(name="lib",sources=[],dependencies=[":foo"])
+            """
+            ),
+        }
+    )
+    request = PexFromTargetsRequest(
+        [Address("", target_name="lib")],
+        output_filename="demo.pex",
+        internal_only=False,
+        platforms=PexPlatforms(["some-platform-x86_64"]),
+    )
+    rule_runner.set_options(
+        [
+            "--python-setup-requirement-constraints=constraints.txt",
+            "--python-setup-resolve-all-constraints",
+        ]
+    )
+    result = rule_runner.request(PexRequest, [request])
+
+    assert result.repository_pex is None
+    assert result.requirements == PexRequirements(["foo"])