Repository: pypa/cibuildwheel

Issue:
CIBW_ENVIRONMENT broken in 1.5.3
Somehow, my line of `export CIBW_ENVIRONMENT='... CCACHE_BASEDIR=`python -c "import tempfile; import os; print(os.path.realpath(tempfile.gettempdir()))"` ...'` (see https://github.com/YannickJadoul/Parselmouth/blob/1f3dd9abc63afdf4a1d26db40d210bb4187118a8/.travis.yml#L143; yes, I know I ought to clean this up) breaks:

```
    + /bin/true
    + mkdir -p /project
    + /opt/python/cp38-cp38/bin/python -c 'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
    + uname -i
x86_64
    + python -c import 'tempfile;' import 'os;' 'print(os.path.realpath(tempfile.gettempdir()))'
  File "<string>", line 1
    import
```

See https://travis-ci.org/github/YannickJadoul/Parselmouth/jobs/709679114#L392

I'll start investigating, but is this perhaps related to #403, @joerick?


Discussion:
OK, got it.

Simplified example:
```python
from cibuildwheel import environment
print(environment.parse_environment('X=`python -c "import antigravity"`').as_dictionary({}))
```

What's going wrong is that `node.word` is not the same as `node.word` is not the same as `context.input[part.pos[0]:part.pos[1]]` (the latter includes the quotes, the first one doesn't), so cibuildwheel loses the quotes, ends up with `python -c import antigravity`, which gets split by `shlex.split` into 4 parts, rather than 3.

Let's see if I can make a PR.

Resolving patch:
diff --git a/cibuildwheel/bashlex_eval.py b/cibuildwheel/bashlex_eval.py
--- a/cibuildwheel/bashlex_eval.py
+++ b/cibuildwheel/bashlex_eval.py
@@ -1,15 +1,14 @@
-import shlex
 import subprocess
 from typing import Callable, Dict, List, NamedTuple, Optional, Sequence
 
 import bashlex  # type: ignore
 
-# a function that takes a shell command and the environment, and returns the result
-EnvironmentExecutor = Callable[[str, Dict[str, str]], str]
+# a function that takes a command and the environment, and returns the result
+EnvironmentExecutor = Callable[[List[str], Dict[str, str]], str]
 
 
-def local_environment_executor(command: str, env: Dict[str, str]) -> str:
-    return subprocess.check_output(shlex.split(command), env=env, universal_newlines=True)
+def local_environment_executor(command: List[str], env: Dict[str, str]) -> str:
+    return subprocess.check_output(command, env=env, universal_newlines=True)
 
 
 class NodeExecutionContext(NamedTuple):
@@ -97,8 +96,7 @@ def evaluate_nodes_as_compound_command(nodes: Sequence[bashlex.ast.node], contex
 
 
 def evaluate_nodes_as_simple_command(nodes: List[bashlex.ast.node], context: NodeExecutionContext):
-    words = [evaluate_node(part, context=context) for part in nodes]
-    command = ' '.join(words)
+    command = [evaluate_node(part, context=context) for part in nodes]
     return context.executor(command, context.environment)
 
 
diff --git a/cibuildwheel/docker_container.py b/cibuildwheel/docker_container.py
--- a/cibuildwheel/docker_container.py
+++ b/cibuildwheel/docker_container.py
@@ -180,9 +180,9 @@ def get_environment(self) -> Dict[str, str]:
             'import sys, json, os; json.dump(os.environ.copy(), sys.stdout)'
         ], capture_output=True))
 
-    def environment_executor(self, command: str, environment: Dict[str, str]) -> str:
+    def environment_executor(self, command: List[str], environment: Dict[str, str]) -> str:
         # used as an EnvironmentExecutor to evaluate commands and capture output
-        return self.call(shlex.split(command), env=environment)
+        return self.call(command, env=environment)
 
 
 def shell_quote(path: PurePath) -> str:


Test patch:
diff --git a/unit_test/environment_test.py b/unit_test/environment_test.py
--- a/unit_test/environment_test.py
+++ b/unit_test/environment_test.py
@@ -40,7 +40,7 @@ def test_inheritance():
 
 
 def test_shell_eval():
-    environment_recipe = parse_environment('VAR="$(echo "a test" string)"')
+    environment_recipe = parse_environment('VAR="$(echo "a   test" string)"')
 
     env_copy = os.environ.copy()
     env_copy.pop('VAR', None)
@@ -50,8 +50,8 @@ def test_shell_eval():
     )
     environment_cmds = environment_recipe.as_shell_commands()
 
-    assert environment_dict['VAR'] == 'a test string'
-    assert environment_cmds == ['export VAR="$(echo "a test" string)"']
+    assert environment_dict['VAR'] == 'a   test string'
+    assert environment_cmds == ['export VAR="$(echo "a   test" string)"']
 
 
 def test_shell_eval_and_env():