Repository: xonsh/xonsh

Issue:
Unexpected newlines in @$(...) substitution
<!--- Provide a general summary of the issue in the Title above -->
<!--- If you have a question along the lines of "How do I do this Bash command in xonsh"
please first look over the Bash to Xonsh translation guide: https://xon.sh/bash_to_xsh.html
If you don't find an answer there, please do open an issue! -->

Seems to be related to #2021, at least in part. 

Whenever I run `docker stop @$(docker ps -aq)`, I receive a Docker error (`404 page not found`), which appears to be caused by the fact that Docker is not receiving a valid container image.

Further experimentation shows this happens with basically everything that uses `@$` notation, which [appears to go against documentation](https://xon.sh/tutorial.html#command-substitution-with) as well as what was mentioned in #2021. 

Am I missing something here?

## xonfig

<details>

```
$ xonfig
+------------------+----------------------+
| xonsh            | 0.9.13.dev1          |
| Git SHA          | 9f7ccc65             |
| Commit Date      | Oct 15 17:14:50 2019 |
| Python           | 3.7.6                |
| PLY              | 3.11                 |
| have readline    | True                 |
| prompt toolkit   | 2.0.9                |
| shell type       | prompt_toolkit2      |
| pygments         | 2.4.2                |
| on posix         | True                 |
| on linux         | False                |
| on darwin        | True                 |
| on windows       | False                |
| on cygwin        | False                |
| on msys2         | False                |
| is superuser     | False                |
| default encoding | utf-8                |
| xonsh encoding   | utf-8                |
| encoding errors  | surrogateescape      |
+------------------+----------------------+
```

</details>

## Expected Behavior
<!--- Tell us what should happen -->

Commands should be split properly and should execute normally, like Bash's own output:

```
$ ls $(ls)
1:

2:

3:

4:

5:
```

## Current Behavior

```
$ ls @$(ls)
ls: 1
2
3
4
5: No such file or directory
$ ls
1 2 3 4 5
```

`argv` view:

```
$ python -c "import sys; print(sys.argv)" @$(ls)
['-c', '1\n2\n3\n4\n5']
$ python -c "import sys; print(sys.argv)" @([i.strip() for i in $(ls).split()])
['-c', '1', '2', '3', '4', '5']
```


## Steps to Reproduce

1. Navigate to test directory - `cd @$(mktemp -d)`
2. Create test folders - `mkdir 1 2 3 4 5`
3. Run `ls @$(ls)`, observe failure


Discussion:

Resolving patch:
diff --git a/xonsh/built_ins.py b/xonsh/built_ins.py
--- a/xonsh/built_ins.py
+++ b/xonsh/built_ins.py
@@ -992,8 +992,12 @@ def subproc_captured_inject(*cmds):
     The string is split using xonsh's lexer, rather than Python's str.split()
     or shlex.split().
     """
-    s = run_subproc(cmds, captured="stdout")
-    toks = builtins.__xonsh__.execer.parser.lexer.split(s.strip())
+    o = run_subproc(cmds, captured="object")
+    o.end()
+    toks = []
+    for line in o:
+        line = line.rstrip(os.linesep)
+        toks.extend(builtins.__xonsh__.execer.parser.lexer.split(line))
     return toks
 
 
diff --git a/xonsh/parsers/base.py b/xonsh/parsers/base.py
--- a/xonsh/parsers/base.py
+++ b/xonsh/parsers/base.py
@@ -3140,7 +3140,9 @@ def p_subproc_atom_pyeval(self, p):
         p[0] = p0
 
     def p_subproc_atom_subproc_inject(self, p):
-        """subproc_atom : atdollar_lparen_tok subproc RPAREN"""
+        """subproc_atom : atdollar_lparen_tok subproc RPAREN
+           subproc_arg_part : atdollar_lparen_tok subproc RPAREN
+        """
         p1 = p[1]
         p0 = xonsh_call(
             "__xonsh__.subproc_captured_inject", p[2], lineno=p1.lineno, col=p1.lexpos
@@ -3149,12 +3151,16 @@ def p_subproc_atom_subproc_inject(self, p):
         p[0] = p0
 
     def p_subproc_atom_subproc_inject_bang_empty(self, p):
-        """subproc_atom : atdollar_lparen_tok subproc bang_tok RPAREN"""
+        """subproc_atom : atdollar_lparen_tok subproc bang_tok RPAREN
+           subproc_arg_part : atdollar_lparen_tok subproc bang_tok RPAREN
+        """
         self._append_subproc_bang_empty(p)
         self.p_subproc_atom_subproc_inject(p)
 
     def p_subproc_atom_subproc_inject_bang(self, p):
-        """subproc_atom : atdollar_lparen_tok subproc bang_tok nocloser rparen_tok"""
+        """subproc_atom : atdollar_lparen_tok subproc bang_tok nocloser rparen_tok
+           subproc_arg_part : atdollar_lparen_tok subproc bang_tok nocloser rparen_tok
+        """
         self._append_subproc_bang(p)
         self.p_subproc_atom_subproc_inject(p)
 


Test patch:
diff --git a/tests/test_integrations.py b/tests/test_integrations.py
--- a/tests/test_integrations.py
+++ b/tests/test_integrations.py
@@ -312,6 +312,34 @@ def _echo(args):
         0,
     ),
     #
+    # test @$() without leading/trailig WS
+    #
+    (
+        """
+def _echo(args):
+    print(' '.join(args))
+aliases['echo'] = _echo
+
+echo foo_@$(echo spam)_bar
+""",
+        "foo_spam_bar\n",
+        0,
+    ),
+    #
+    # test @$() outer product
+    #
+    (
+        """
+def _echo(args):
+    print(' '.join(args))
+aliases['echo'] = _echo
+
+echo foo_@$(echo spam sausage)_bar
+""",
+        "foo_spam_bar foo_sausage_bar\n",
+        0,
+    ),
+    #
     # test redirection
     #
     (
diff --git a/tests/test_parser.py b/tests/test_parser.py
--- a/tests/test_parser.py
+++ b/tests/test_parser.py
@@ -2250,6 +2250,13 @@ def test_rhs_nested_injection():
     check_xonsh_ast({}, "$[ls @$(dirname @$(which python))]", False)
 
 
+def test_merged_injection():
+    tree = check_xonsh_ast({}, "![a@$(echo 1 2)b]", False, return_obs=True)
+    assert isinstance(tree, AST)
+    func = tree.body.args[0].right.func
+    assert func.attr == "list_of_list_of_strs_outer_product"
+
+
 def test_backtick_octothorpe():
     check_xonsh_ast({}, "print(`#.*`)", False)