Repository: pantsbuild/pants

Issue:
awslambda's configuration of the Pex resolve to target linux is flawed
It looks like there are two bugs here:
https://github.com/pantsbuild/pants/blob/39ae9b9a8be2f14343dde4198043c339148d58f4/src/python/pants/backend/awslambda/python/awslambda_python_rules.py#L62-L79

1. When using --platform with Pex, Pex does not check to see if the given platform is available locally via a discoverable interpreter that matches and instead assumes the platform is foreign, requiring pre-built wheels to be present.
2. The platform used is limited to manylinux ruling out use of non-manylinux wheels. Since the target OSes are ~known (https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html) its definitely possible to have non-manylinux wheels pre-built that work on the target AMIs.

To fix 1 we could try to detect the current platform if running locally and otherwise looks at the target platform we're requesting in remote execution. There is currently no reasonable way to do this that I'm aware of. Alternatively, Pex would need a feature that allows scanning all discoverable interpreters to find a local one matching --platform, and if found, loosen the `--no-build` constraint implicitly applied to `--platform` resolves.

To fix 2 just requires `s/manylinux2014/linux/` and possibly redundantly setting the already default `--manylinux=manylinux2014` Pex option for documentation purposes / emphasis.




Discussion:
> Alternatively, Pex would need a feature that allows scanning all discoverable interpreters to find a local one matching --platform, and if found, loosen the --no-build constraint implicitly applied to --platform resolves.

How viable is this? That sounds very useful to Pex users and for `./pants binary`.
> How viable is this?

This actually looks pretty simple. `PythonInterpreter.all(path=None)` gets you all local interpreters and this goes through a cache already. Then `Platform.of_interpreter(interpreter)` gets you the Platform (whose `__str__` is the platform string in the Pex API/CLI) that corresponds to that interpreter. The only consideration would be whether the scan is default behavior or is flagged on. Filed: https://github.com/pantsbuild/pex/issues/996

Resolving patch:
diff --git a/src/python/pants/backend/awslambda/python/awslambda_python_rules.py b/src/python/pants/backend/awslambda/python/awslambda_python_rules.py
--- a/src/python/pants/backend/awslambda/python/awslambda_python_rules.py
+++ b/src/python/pants/backend/awslambda/python/awslambda_python_rules.py
@@ -63,7 +63,7 @@ async def create_python_awslambda(
     # (Running the "hello world" lambda in the example code will report the platform, and can be
     # used to verify correctness of these platform strings.)
     py_major, py_minor = field_set.runtime.to_interpreter_version()
-    platform = f"manylinux2014_x86_64-cp-{py_major}{py_minor}-cp{py_major}{py_minor}"
+    platform = f"linux_x86_64-cp-{py_major}{py_minor}-cp{py_major}{py_minor}"
     # set pymalloc ABI flag - this was removed in python 3.8 https://bugs.python.org/issue36707
     if py_major <= 3 and py_minor < 8:
         platform += "m"
@@ -75,6 +75,13 @@ async def create_python_awslambda(
             entry_point=None,
             output_filename=pex_filename,
             platforms=PexPlatforms([platform]),
+            additional_args=[
+                # Ensure we can resolve manylinux wheels in addition to any AMI-specific wheels.
+                "--manylinux=manylinux2014",
+                # When we're executing Pex on Linux, allow a local interpreter to be resolved if
+                # available and matching the AMI platform.
+                "--resolve-local-platforms",
+            ],
         )
     )
 
diff --git a/src/python/pants/backend/python/rules/download_pex_bin.py b/src/python/pants/backend/python/rules/download_pex_bin.py
--- a/src/python/pants/backend/python/rules/download_pex_bin.py
+++ b/src/python/pants/backend/python/rules/download_pex_bin.py
@@ -23,9 +23,9 @@
 class PexBin(ExternalTool):
     options_scope = "download-pex-bin"
     name = "pex"
-    default_version = "v2.1.12"
+    default_version = "v2.1.13"
     default_known_versions = [
-        f"v2.1.12|{plat}|98cee30e00bfad8279159390139844f9154d9505f98cdbf35733f92412e26910|2631935"
+        f"v2.1.13|{plat}|240712c75bb7c7cdfe3dd808ad6e6f186182e6aea3efeea5760683bb0fe89198|2633838"
         for plat in ["darwin", "linux"]
     ]
 


Test patch: