Repository: pex-tool/pex

Issue:
Interactive interpreter hangs if pex called with a python cli option
## Issue description and reproduction steps

A pex that has no pre-defined entry point or console script is expected to open up an interactive interpreter if it isn't passed a script, module, or command as an argument:
```
~/code/pantsbuild/pex (main) $ python -m pex -o testpex.pex
~/code/pantsbuild/pex (main) $ ./testpex.pex
Python 3.9.17 (main, Aug  8 2023, 12:13:14) 
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> print('hello')
hello
>>> quit()
~/code/pantsbuild/pex (main) $ 
```

If we pass a [python command line option](https://docs.python.org/3/using/cmdline.html) like `-q`, which normally quiets the repl preamble, the pex command hangs indefinitely:
```
~/code/pantsbuild/pex (main) $ python -q
>>> quit()
~/code/pantsbuild/pex (main) $ ./testpex.pex -q
# hangs ... and requires Ctrl-C
```

## Motivation

This came up while testing a fix to address 
 https://github.com/pantsbuild/pants/issues/3423 so we can pass through args to `pants repl` for the python shell. For a python shell, pants runs a `requirements.pex` as an interactive process to give a repl to the user.

Usually if you want to pass args to a python repl it is going to be either interpreter options like these or it will be `-i` to run a script/module in interactive mode.

## Diagnosis

Under the hood it hangs because pex gets into an infinite loop of re-executing itself with the same argument. We deal with python options inside `Pex.execute_interpreter`:

```python
args = sys.argv[1:]
python_options = []
for index, arg in enumerate(args):
    # Check if the arg is an expected startup arg.
    if arg.startswith("-") and arg not in ("-", "-c", "-m"):
        python_options.append(arg)
    else:
        args = args[index:]
        break

# The pex was called with Python interpreter options
if python_options:
    return self.execute_with_options(python_options, args)
```
The first block of code is attempting to partition the input args into python options like `-q` or `-B` and a remainder of command line args that should be passed through when we `execute_with_options`. But when the args supplied are _only python options_, we do not set `args = []` because the `args = args[index:]` does not get triggered. Therefore we end up execing something like this:
```python
# cmdline
['<path_to_python>/python3.9', '-q', '<path_to_unzipped_pex>/__main__.py', '-q']
```
later in `os.execv(python, cmdline)` within `execute_with_options` which puts us in an infinite loop because the pex will encounter the `-q` again as a python option.

This is simple to fix, a PR will be forthcoming.


Discussion:

Resolving patch:
diff --git a/pex/pex.py b/pex/pex.py
--- a/pex/pex.py
+++ b/pex/pex.py
@@ -620,6 +620,9 @@ def execute_interpreter(self):
             else:
                 args = args[index:]
                 break
+        else:
+            # All the args were python options
+            args = []
 
         # The pex was called with Python interpreter options
         if python_options:
diff --git a/pex/venv/installer.py b/pex/venv/installer.py
--- a/pex/venv/installer.py
+++ b/pex/venv/installer.py
@@ -849,6 +849,9 @@ def sys_executable_paths():
                     else:
                         args = args[index:]
                         break
+                else:
+                    # All the args were python options
+                    args = []
 
                 # The pex was called with Python interpreter options, so we need to re-exec to
                 # respect those:


Test patch:
diff --git a/tests/integration/venv_ITs/test_issue_2248.py b/tests/integration/venv_ITs/test_issue_2248.py
new file mode 100644
--- /dev/null
+++ b/tests/integration/venv_ITs/test_issue_2248.py
@@ -0,0 +1,82 @@
+# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
+# Licensed under the Apache License, Version 2.0 (see LICENSE).
+
+import os
+import subprocess
+from textwrap import dedent
+
+import pytest
+from colors import colors
+
+from pex.typing import TYPE_CHECKING
+from testing import IntegResults, run_pex_command
+
+if TYPE_CHECKING:
+    from typing import Any, List
+
+
+# N.B.: To test that we are running a REPL we'll show that it computes result even after
+# assertion errors and exceptions that would normally halt a script.
+# To check that it forwards python options we use -O to deactivate asserts
+# See: https://docs.python.org/3/using/cmdline.html#cmdoption-O
+@pytest.mark.parametrize(
+    "execution_mode_args",
+    [
+        pytest.param([], id="ZIPAPP"),
+        pytest.param(["--venv"], id="VENV"),
+    ],
+)
+def test_repl_python_options(
+    execution_mode_args,  # type: List[str]
+    tmpdir,  # type: Any
+):
+    # type: (...) -> None
+
+    pex = os.path.join(str(tmpdir), "pex")
+    run_pex_command(args=["ansicolors==1.1.8", "-o", pex] + execution_mode_args).assert_success()
+
+    repl_commands = dedent(
+        """
+        import colors
+        assert False
+        raise Exception("customexc")
+        result = 20 + 103
+        print(colors.green("Worked: {}".format(result)))
+        quit()
+        """
+    )
+
+    def execute_pex(disable_assertions):
+        # type: (bool) -> IntegResults
+        args = [pex]
+        if disable_assertions:
+            args.append("-O")
+        process = subprocess.Popen(
+            args=args,
+            stdin=subprocess.PIPE,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+        )
+        stdout, stderr = process.communicate(input=repl_commands.encode("utf-8"))
+        return IntegResults(
+            output=stdout.decode("utf-8"),
+            error=stderr.decode("utf-8"),
+            return_code=process.returncode,
+        )
+
+    # The assertion will fail and print but since it is a REPL it will keep going
+    # and compute the result
+    result = execute_pex(disable_assertions=False)
+    result.assert_success()
+    assert "InteractiveConsole" in result.error
+    assert "AssertionError" in result.error
+    assert "customexc" in result.error
+    assert colors.green("Worked: 123") in result.output
+
+    # The -O will disable the assertion, but the regular exception will still get raised.
+    result = execute_pex(disable_assertions=True)
+    result.assert_success()
+    assert "InteractiveConsole" in result.error
+    assert "AssertionError" not in result.error
+    assert "customexc" in result.error
+    assert colors.green("Worked: 123") in result.output
diff --git a/tests/test_pex.py b/tests/test_pex.py
--- a/tests/test_pex.py
+++ b/tests/test_pex.py
@@ -789,6 +789,63 @@ def test_execute_interpreter_file_program():
             assert b"" == stderr
 
 
+def test_execute_repl():
+    with temporary_dir() as pex_chroot:
+        pex_builder = PEXBuilder(path=pex_chroot)
+        pex_builder.freeze()
+        process = PEX(pex_chroot).run(
+            args=[],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            stdin=subprocess.PIPE,
+            blocking=False,
+        )
+        commands = dedent(
+            """
+            assert False
+            20 + 103
+            quit()
+            """
+        )
+        stdout, stderr = process.communicate(input=commands.encode("utf-8"))
+
+        assert b"(InteractiveConsole)" in stderr
+        assert b"AssertionError" in stderr
+        assert b">>> " in stdout
+        assert b"123" in stdout
+        assert 0 == process.returncode
+
+
+def test_execute_repl_with_python_options():
+    with temporary_dir() as pex_chroot:
+        pex_builder = PEXBuilder(path=pex_chroot)
+        pex_builder.freeze()
+        process = PEX(pex_chroot).run(
+            args=["-O"],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            stdin=subprocess.PIPE,
+            blocking=False,
+        )
+        # adding the -O option will ignore that assertion
+        # see: https://docs.python.org/3/using/cmdline.html#cmdoption-O
+        # We should not see the AssertionError in the output anymore.
+        commands = dedent(
+            """
+            assert False
+            20 + 103
+            quit()
+            """
+        )
+        stdout, stderr = process.communicate(input=commands.encode("utf-8"))
+
+        assert b"(InteractiveConsole)" in stderr
+        assert b"AssertionError" not in stderr
+        assert b">>> " in stdout
+        assert b"123" in stdout
+        assert 0 == process.returncode
+
+
 def test_pex_run_strip_env():
     # type: () -> None
     with temporary_dir() as pex_root: