Repository: pantsbuild/pants

Issue:
`./pants run ...` does not work for non-venv-mode `pex_binary` targets that re-exec sys.argv[0].
In short, the `run` goal executes via ~:
```
export PEX_PATH=/path/to/requirements.pex
export PEX_EXTRA_SYS_PATH=/path/to/source_root_1:/path/to/cource_root2
./only-contains-entry-point-metadata.pex
```

If the executed code then tries to re-execute via argv[0] (the PEX file itself), then sys.path scrubbing is engaged which strips back off the PEX_PATH and PEX_EXTRA_SYS_PATH triggered sys.path additions since those two env vars are also stripped by default. Either Pants needs to expose the `--no-strip-pex-env` option as a `pex_binary` parameter or else it needs to set this option for `pants run` unconditionally.

The concrete example of apps that re-exec via sys.argv[0] are django manage.py apps.

See https://github.com/pantsbuild/pex/issues/1349 where @asherf discovered the issue and the mechanism behind it was all worked out.


Discussion:

Resolving patch:
diff --git a/src/python/pants/backend/python/goals/run_pex_binary.py b/src/python/pants/backend/python/goals/run_pex_binary.py
--- a/src/python/pants/backend/python/goals/run_pex_binary.py
+++ b/src/python/pants/backend/python/goals/run_pex_binary.py
@@ -58,7 +58,13 @@ async def create_pex_binary_run_request(
         PexRequest(
             output_filename=output_filename,
             interpreter_constraints=requirements_pex_request.interpreter_constraints,
-            additional_args=field_set.generate_additional_args(pex_binary_defaults),
+            additional_args=(
+                *field_set.generate_additional_args(pex_binary_defaults),
+                # N.B.: Since we cobble together the runtime environment via PEX_PATH and
+                # PEX_EXTRA_SYS_PATH below, it's important for any app that re-executes itself that
+                # these environment variables are not stripped.
+                "--no-strip-pex-env",
+            ),
             internal_only=True,
             # Note that the entry point file is not in the PEX itself. It's loaded by setting
             # `PEX_EXTRA_SYS_PATH`.


Test patch:
diff --git a/src/python/pants/backend/python/goals/run_pex_binary_integration_test.py b/src/python/pants/backend/python/goals/run_pex_binary_integration_test.py
--- a/src/python/pants/backend/python/goals/run_pex_binary_integration_test.py
+++ b/src/python/pants/backend/python/goals/run_pex_binary_integration_test.py
@@ -1,5 +1,6 @@
 # Copyright 2020 Pants project contributors (see CONTRIBUTORS.md).
 # Licensed under the Apache License, Version 2.0 (see LICENSE).
+
 import json
 from textwrap import dedent
 from typing import Optional
@@ -88,3 +89,38 @@ def run(*extra_args: str, **extra_env: str) -> PantsResult:
         pex_info = json.loads(result.stdout)
         assert (execution_mode is PexExecutionMode.UNZIP) == pex_info["unzip"]
         assert (execution_mode is PexExecutionMode.VENV) == pex_info["venv"]
+        assert pex_info["strip_pex_env"] is False
+
+
+def test_no_strip_pex_env_issues_12057() -> None:
+    sources = {
+        "src/app.py": dedent(
+            """\
+            import os
+            import sys
+
+
+            if __name__ == "__main__":
+                exit_code = os.environ.get("PANTS_ISSUES_12057")
+                if exit_code is None:
+                    os.environ["PANTS_ISSUES_12057"] = "42"
+                    os.execv(sys.executable, [sys.executable, *sys.argv])
+                sys.exit(int(exit_code))
+            """
+        ),
+        "src/BUILD": dedent(
+            """\
+            python_library(name="lib")
+            pex_binary(entry_point="app.py")
+            """
+        ),
+    }
+    with setup_tmpdir(sources) as tmpdir:
+        args = [
+            "--backend-packages=pants.backend.python",
+            f"--source-root-patterns=['/{tmpdir}/src']",
+            "run",
+            f"{tmpdir}/src/app.py",
+        ]
+        result = run_pants(args)
+        assert result.exit_code == 42, result.stderr