Repository: xonsh/xonsh

Issue:
Syntax errors in subprocess commands containing []
I am trying to run a command which should be getting passed to a subprocess (in this case rake). The command itself includes square brackets (`[]`) which results in a traceback.

The command: `rake so_thing[some,parameters]`
The error:

<details>

```
Traceback (most recent call last):
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/ptk/shell.py", line 137, in _push
    locs=None)
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/execer.py", line 110, in compile
    transform=transform)
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/execer.py", line 79, in parse
    tree, input = self._parse_ctx_free(input, mode=mode, filename=filename)
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/execer.py", line 179, in _parse_ctx_free
    raise original_error from None
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/execer.py", line 166, in _parse_ctx_free
    debug_level=(self.debug_level > 2))
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/parsers/base.py", line 348, in parse
    tree = self.parser.parse(input=s, lexer=self.lexer, debug=debug_level)
  File "/usr/lib/python3.6/site-packages/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/lib/python3.6/site-packages/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/lib/python3.6/site-packages/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/parsers/base.py", line 2726, in p_error
    column=p.lexpos))
  File "/home/tmacey/.local/lib/python3.6/site-packages/xonsh/parsers/base.py", line 479, in _parse_error
    raise err
  File "<string>", line None
SyntaxError: /home/tmacey/.local/lib/python3.6/site-packages/xontrib/jedi.xsh:1:21: ('code: [',)
rake generate_stack[01,dev2-useast1]
                   ^
```

</details>

## For community
⬇️  **Please click the 👍 reaction instead of leaving a `+1` or 👍  comment**


Discussion:
codecov-commenter: ## [Codecov](https://app.codecov.io/gh/xonsh/xonsh/pull/5326?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 64.99%. Comparing base [(`7c23508`)](https://app.codecov.io/gh/xonsh/xonsh/commit/7c23508afc1e753701eb36ab03aa63bdfd753cd7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) to head [(`0cbf87b`)](https://app.codecov.io/gh/xonsh/xonsh/pull/5326?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh).
> Report is 29 commits behind head on main.


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #5326      +/-   ##
==========================================
- Coverage   67.14%   64.99%   -2.16%     
==========================================
  Files         119      119              
  Lines       23125    23142      +17     
  Branches     4855     4859       +4     
==========================================
- Hits        15528    15040     -488     
- Misses       6391     6864     +473     
- Partials     1206     1238      +32     
```

| [Flag](https://app.codecov.io/gh/xonsh/xonsh/pull/5326/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) | Coverage Δ | |
|---|---|---|
| [macOS-latest](https://app.codecov.io/gh/xonsh/xonsh/pull/5326/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) | `?` | |
| [ubuntu-latest](https://app.codecov.io/gh/xonsh/xonsh/pull/5326/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) | `64.99% <100.00%> (+0.27%)` | :arrow_up: |
| [windows-latest](https://app.codecov.io/gh/xonsh/xonsh/pull/5326/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh) | `?` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/xonsh/xonsh/pull/5326?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xonsh).


Patch:
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 9f429925..0ec50cb0 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -14,7 +14,7 @@ repos:
       pass_filenames: false
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: 'v0.3.4'
+    rev: 'v0.3.5'
     hooks:
       - id: ruff
         args: [., --fix, --exit-non-zero-on-fix]
@@ -41,7 +41,7 @@ repos:
         additional_dependencies:
           - types-ujson
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.5.0
+    rev: v4.6.0
     hooks:
       - id: trailing-whitespace
         exclude: |
diff --git a/news/brackets-in-args.rst b/news/brackets-in-args.rst
new file mode 100644
index 00000000..1478e6c5
--- /dev/null
+++ b/news/brackets-in-args.rst
@@ -0,0 +1,23 @@
+**Added:**
+
+* Square brackets can now be used in command arguments without quotes (e.g. `echo a[b]`)
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* <news item>
+
+**Fixed:**
+
+* <news item>
+
+**Security:**
+
+* <news item>
diff --git a/xonsh/parsers/base.py b/xonsh/parsers/base.py
index c4a6c524..76c2ba4f 100644
--- a/xonsh/parsers/base.py
+++ b/xonsh/parsers/base.py
@@ -1,5 +1,6 @@
 """Implements the base xonsh parser."""
 
+import itertools
 import os
 import re
 import textwrap
@@ -3496,27 +3497,43 @@ class BaseParser:
         """subproc_arg : subproc_arg_part"""
         p[0] = p[1]
 
+    def _arg_part_combine(self, *arg_parts):
+        """Combines arg_parts. If all arg_parts are strings, concatenate the strings.
+        Otherwise, return a list of arg_parts."""
+        if all(ast.is_const_str(ap) for ap in arg_parts):
+            return ast.const_str(
+                "".join(ap.value for ap in arg_parts),
+                lineno=arg_parts[0].lineno,
+                col_offset=arg_parts[0].col_offset,
+            )
+        else:
+            return list(
+                itertools.chain.from_iterable(
+                    ap if isinstance(ap, list) else [ap] for ap in arg_parts
+                )
+            )
+
     def p_subproc_arg_many(self, p):
         """subproc_arg : subproc_arg subproc_arg_part"""
         # This glues the string together after parsing
+        p[0] = self._arg_part_combine(p[1], p[2])
+
+    def p_subproc_arg_part_brackets(self, p):
+        """subproc_arg_part : lbracket_tok subproc_arg rbracket_tok"""
         p1 = p[1]
         p2 = p[2]
-        if ast.is_const_str(p1) and ast.is_const_str(p2):
-            p0 = ast.const_str(
-                p1.value + p2.value, lineno=p1.lineno, col_offset=p1.col_offset
-            )
-        elif isinstance(p1, list):
-            if isinstance(p2, list):
-                p1.extend(p2)
-            else:
-                p1.append(p2)
-            p0 = p1
-        elif isinstance(p2, list):
-            p2.insert(0, p1)
-            p0 = p2
-        else:
-            p0 = [p1, p2]
-        p[0] = p0
+        p3 = p[3]
+        p1 = ast.const_str(s=p1.value, lineno=p1.lineno, col_offset=p1.lexpos)
+        p3 = ast.const_str(s=p3.value, lineno=p3.lineno, col_offset=p3.lexpos)
+        p[0] = self._arg_part_combine(p1, p2, p3)
+
+    def p_subproc_arg_part_brackets_empty(self, p):
+        """subproc_arg_part : lbracket_tok rbracket_tok"""
+        p1 = p[1]
+        p2 = p[2]
+        p1 = ast.const_str(s=p1.value, lineno=p1.lineno, col_offset=p1.lexpos)
+        p2 = ast.const_str(s=p2.value, lineno=p2.lineno, col_offset=p2.lexpos)
+        p[0] = self._arg_part_combine(p1, p2)
 
     def _attach_subproc_arg_part_rules(self):
         toks = set(self.tokens)
diff --git a/xonsh/procs/specs.py b/xonsh/procs/specs.py
index 7c538a1b..660cfc85 100644
--- a/xonsh/procs/specs.py
+++ b/xonsh/procs/specs.py
@@ -250,6 +250,17 @@ def _redirect_streams(r, loc=None):
     return stdin, stdout, stderr
 
 
+def _flatten_cmd_redirects(cmd):
+    """Transforms a command like ['ls', ('>', '/dev/null')] into ['ls', '>', '/dev/null']."""
+    new_cmd = []
+    for c in cmd:
+        if isinstance(c, tuple):
+            new_cmd.extend(c)
+        else:
+            new_cmd.append(c)
+    return new_cmd
+
+
 def default_signal_pauser(n, f):
     """Pauses a signal, as needed."""
     signal.pause()
@@ -352,7 +363,7 @@ class SubprocSpec:
         else:
             self.env = None
         # pure attrs
-        self.args = list(cmd)
+        self.args = _flatten_cmd_redirects(cmd)
         self.alias = None
         self.alias_name = None
         self.alias_stack = XSH.env.get("__ALIAS_STACK", "").split(":")
@@ -433,9 +444,7 @@ class SubprocSpec:
             raise xt.XonshError(msg)
 
     def get_command_str(self):
-        return " ".join(
-            " ".join(arg) if isinstance(arg, tuple) else arg for arg in self.args
-        )
+        return " ".join(arg for arg in self.args)
 
     #
     # Execution methods
@@ -883,6 +892,9 @@ def run_subproc(cmds, captured=False, envs=None):
             print(f"TRACE SUBPROC: {cmds}, captured={captured}", file=sys.stderr)
 
     specs = cmds_to_specs(cmds, captured=captured, envs=envs)
+    cmds = [
+        _flatten_cmd_redirects(cmd) if isinstance(cmd, list) else cmd for cmd in cmds
+    ]
     if _should_set_title():
         # context manager updates the command information that gets
         # accessed by CurrentJobField when setting the terminal's title


Test patch:
diff --git a/tests/test_parser.py b/tests/test_parser.py
index ae57dfa2..ee131ed8 100644
--- a/tests/test_parser.py
+++ b/tests/test_parser.py
@@ -2649,6 +2649,25 @@ def test_echo_slash_question(check_xonsh_ast):
     check_xonsh_ast({}, "![echo /?]", False)
 
 
+@pytest.mark.parametrize(
+    "case",
+    [
+        "[]",
+        "[[]]",
+        "[a]",
+        "[a][b]",
+        "a[b]",
+        "[a]b",
+        "a[b]c",
+        "a[b[c]]",
+        "[a]b[[]c[d,e]f[]g,h]",
+        "[a@([1,2])]@([3,4])",
+    ],
+)
+def test_echo_brackets(case, check_xonsh_ast):
+    check_xonsh_ast({}, f"![echo {case}]")
+
+
 def test_bad_quotes(check_xonsh_ast):
     with pytest.raises(SyntaxError):
         check_xonsh_ast({}, '![echo """hello]', False)