Repository: facelessuser/bracex (language: python)

Issue:
Feature proposal/offer: command line interface
I've been [seeking a command `x`, like `seq` but doing brace expansion](https://stackoverflow.com/questions/69383322/is-there-command-like-seq-that-performs-full-bash-like-brace-expansion-to-std). It could be used like

```bash
x "{a..z}/{a..z}/{a..z}/{a..z}/{a..z}/{a..z}" | xargs -d $'\n' mkdir -p
```

I've not found a satisfactory `x`, so I'd like to offer a PR allowing `python -m bracex` to serve as `x`. Would you be interested?

I imagine the major elements would be
- a new `__main__.py` wrapping `bracex.iexpand()`
- minimal argument parsing using `argparse`
- updates to documentation
- additional tests to maintain code coverage

I totally understand if the answer is no, so please don't feel pressured. I know every new feature creates support costs. As plan B I could release a separate package that wraps bracex, keeping a loose coupling.

Thanks for your consideration, Alex


Discussion / hints:
I think a command streaming the expansions to stdout would be useful because it wouldn't require O(n**m) memory for n terms, each with m elements. Shells I've tested (bash, zsh, ..) implement brace expansion by trying to allocate the full array at once. For brace expressions with many terms that can exceed available RAM.
I'm curious, have you tested the differences between `iexpand` and `expand`and verified that it takes less memory?  I only say this as I haven't. Yeah, everything is wrapped up in generators and such, but I've never sat down to verify that generators don't just unravel to be pretty big when spitting out the iterated values. Sure we don't allocate the whole result array, but I've just never sat down to really analyze the results. 

Ultimately, I needed brace expansion, which I pulled off pretty decently,  and the idea to have one as an iterator was nice in theory, but I never proved out whether it gave a substantial advantage one way or the other 🙃.
> have you tested the differences between iexpand and expandand verified that it takes less memory?

I've not. I will. Thanks for the heads up.
While I'm currently uncertain as to whether `iexpand` ends up doing much better as the necessary context may cause a lot of it to unravel, as far as a CLI is concerned, assuming you find the results worth the effort, I'll have to think about it.

@gir-bot remove S: triage
@gir-bot add T: feature, S: needs-decision
`bracex.iexpand()` is not O(m**n) with memory. On my laptop
- bash fails at 5 * `{a..z}` (26**5, 11.8 million)
- `bracex.iexpand()` works to at least 6 * `{a..z}` (26**6, 308 million).

### Test script
```python
# bx
import sys

import bracex

for entry in bracex.iexpand(sys.argv[1], limit=0):
    print(entry)
```
### Bash fails
```
$ time bash -c "printf '%s\n' {a..z}/{a..z}/{a..z}/{a..z}/{a..z}"
Killed

real	0m5.676s
user	0m3.673s
sys	0m1.011s
```
### bracex succeeds
```
$ time v38/bin/python bx "{a..z}/{a..z}/{a..z}/{a..z}/{a..z}" | wc -l
11881376

real	0m7.901s
user	0m8.003s
sys	0m0.158s
$ time v38/bin/python bx "{a..z}/{a..z}/{a..z}/{a..z}/{a..z}/{a..z}" | wc -l
308915776

real	3m47.018s
user	3m51.033s
sys	0m7.881s
```
Edit 1: 1 Oct 2021 Improve labelling of results
Yep, this actually doesn't surprise me. I suspected, due to context needed when expanding that the generators would probably fully unfold. Add to that the extra overhead that python naturally adds.... You get the idea.

The reason no one iterates is because you can't expand without having all the context. In our case, the attempt to iterate may even add more overhead.

Python is always going to be less efficient than good, optimized C code. This at least confirms what I kind of already suspected. We'd probably be more efficient if we didn't add the overhead of generators, but probably never as good as C.

This has been a useful exercise, but probably didn't give you what you were going for 🙂.
I should probably, at the very least, set people's expectations for what `iexpand` can actually do in the documents. Like you, I imagine some people came in thinking they'd get a far greater boost, but I know when I finally got iterating working, I was looking at the code very skeptical of how much memory would actually get saved. At the time though, it was good enough to get me what I needed for https://github.com/facelessuser/wcmatch.
I think there's a misunderstanding. In my view Bash failed the experiment,
and bracex.iexpand() passed.

I agree python will be slowing it down, vs e.g. Rust, but it's good enough
for me.

On Fri, 1 Oct 2021, 11:56 Isaac Muse, ***@***.***> wrote:

> I should probably, at the very least, set people's expectations for what
> iexpand can actually do in the documents. Like you, I imagine some people
> came in thinking they'd get a far greater boost, but I know when I finally
> got iterating working, I was looking at the code very skeptical of how much
> memory would actually get saved. At the time though, it was good enough to
> get me what I needed for https://github.com/facelessuser/wcmatch.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/facelessuser/bracex/issues/27#issuecomment-932127248>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AABKS4UYSVL6SEYOPX6A3DLUEWHV5ANCNFSM5FDJTMKA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
>

Sorry, I read that at 5 in the morning. I now see it handled many millions more now :). 
Thanks for confirming that it does what I hoped! Clearly, I was more pessimistic about my own work.
I think I may be open to a CLI interface. It's probably not something many are specifically looking for, but it should require extremely minimal maintenance.

What do you propose the command be called, and what options?

@gir-bot remove S: needs-decision
I was thinking something similar to `seq`
```
usage: python -m bracex [OPTION]... <expression>

Options:
    -s, --seperator=STRING
        use STRING to separate entries (default: \n)
    --help ...
    --version ...
```
I'm not proposing to create a new command in $PATH. That could be done later, if there's demand.

An example invocation would look like
```
$ python -m bracex "{a..c}/{01..03}"
a/01
a/02
...
c/02
c/03
```
> I'm not proposing to create a new command in $PATH. That could be done later, if there's demand.

This is trivial to do. And I don't think it would conflict with anything. We could do it with very little effort.

Can you elaborate on the `-s` option? Are speaking of how the info is returned? I'm not sure I understand.
> This is trivial to do. And I don't think it would conflict with anything. We could do it with very little effort.

I'm aware, but I'd rather not do it from the start because it's easier/cleaner to add something later than to try and remove it because your earlier choice turned out wrong.
 
> Can you elaborate on the `-s` option? Are speaking of how the info is returned? I'm not sure I understand.

E.g.
```
$ python -m bracex "{a..c}/{01..03}"
a/01
a/02
...
c/02
c/03
$ # ^--- implicit newline, like find -print
```
```
$ python -m bracex -s ' ' "{a..c}/{01..03}"
a/01 a/02 a/03 b/01 b/02 b/03 c/01 c/02 c/03$ # <--- intentionally no trailing new line, like find -print0
```
```
$ python -m bracex -s $'\0' "base/{a..c}/{01..03}" | xargs -0 mkdir -p
$ tree base/
base/
├── a
│   ├── 01
...
└── c
    ├── 01
    ├── 02
    └── 03

12 directories, 0 files
```
Gotcha. Yeah, I'm okay moving forward.
Great, thank you for your time. ETA this weekend.

Fix patch:
diff --git a/README.md b/README.md
index 0f42bbf..a34881e 100644
--- a/README.md
+++ b/README.md
@@ -21,6 +21,20 @@ Bracex adds this ability to Python:
 ['file-ad.txt', 'file-bd.txt', 'file-cd.txt']
 ```
 
+and as a command:
+
+```console
+$ python3 -m bracex -0 "base/{a,b}/{1..2}" | xargs -0 mkdir -p
+$ tree base/
+base/
+├── a
+│   ├── 1
+│   └── 2
+└── b
+    ├── 1
+    └── 2
+```
+
 - **Why Bracex over other solutions?**
 
     Bracex actually follows pretty closely to how Bash processes braces. It is not a 1:1 implementation of how Bash
@@ -29,6 +43,8 @@ Bracex adds this ability to Python:
     deviated. For instance, we are not handling Bash's command line inputs, so we are not giving special meaning to back
     ticks and quotes at this time.
 
+    On the command line Bracex can handle more expansions than Bash itself.
+
 ## Install
 
 ```console
diff --git a/bracex/__main__.py b/bracex/__main__.py
new file mode 100644
index 0000000..8c62511
--- /dev/null
+++ b/bracex/__main__.py
@@ -0,0 +1,66 @@
+"""
+Expands a bash-style brace expression, and outputs each expansion.
+
+Licensed under MIT
+Copyright (c) 2018 - 2020 Isaac Muse <isaacmuse@gmail.com>
+Copyright (c) 2021 Alex Willmer <alex@moreati.org.uk>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
+documentation files (the "Software"), to deal in the Software without restriction, including without limitation
+the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software,
+and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions
+of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
+TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
+CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.
+"""
+import argparse
+import bracex
+
+
+def main(argv=None):
+    """Accept command line arguments and output brace expansion to stdout."""
+    parser = argparse.ArgumentParser(
+        prog='python -m bracex',
+        description='Expands a bash-style brace expression, and outputs each expansion.',
+        allow_abbrev=False,
+    )
+    parser.add_argument(
+        'expression',
+        help="Brace expression to expand",
+    )
+    terminators = parser.add_mutually_exclusive_group()
+    terminators.add_argument(
+        '--terminator', '-t',
+        default='\n',
+        metavar='STR',
+        help="Terminate each expansion with string STR (default: \\n)",
+    )
+    terminators.add_argument(
+        '-0',
+        action='store_const',
+        const='\0',
+        dest='terminator',
+        help="Terminate each expansion with a NUL character",
+    )
+    parser.add_argument(
+        '--version',
+        action='version',
+        version=bracex.__version__,
+    )
+
+    args = parser.parse_args(argv)
+
+    for expansion in bracex.iexpand(args.expression, limit=0):
+        print(expansion, end=args.terminator)
+
+    raise SystemExit(0)
+
+
+if __name__ == '__main__':
+    main()  # pragma: no cover
diff --git a/docs/src/dictionary/en-custom.txt b/docs/src/dictionary/en-custom.txt
index 8106e31..c95b79f 100644
--- a/docs/src/dictionary/en-custom.txt
+++ b/docs/src/dictionary/en-custom.txt
@@ -8,12 +8,14 @@ Gitter
 MERCHANTABILITY
 MkDocs
 NONINFRINGEMENT
+NUL
 Preprocess
 PyPI
 TODO
 Tox
 Twemoji
 Virtualenv
+Willmer
 accessor
 deprecations
 expander
@@ -26,6 +28,7 @@ prerelease
 prereleases
 pytest
 regex
+stdout
 sublicense
 tuple
 wordlist
diff --git a/docs/src/markdown/about/changelog.md b/docs/src/markdown/about/changelog.md
index e403dce..cfab47b 100644
--- a/docs/src/markdown/about/changelog.md
+++ b/docs/src/markdown/about/changelog.md
@@ -2,7 +2,8 @@
 
 ## 2.2
 
-- **NEW** Support Python 3.10
+- **NEW**: Support Python 3.10
+- **NEW**: Command line interface using `python3 -m bracex`
 
 ## 2.1.1
 
diff --git a/docs/src/markdown/index.md b/docs/src/markdown/index.md
index ce0358b..7c24f1f 100644
--- a/docs/src/markdown/index.md
+++ b/docs/src/markdown/index.md
@@ -17,6 +17,20 @@ Bracex adds this ability to Python:
 ['file-1.txt', 'file-2.txt', 'file-3.txt']
 ```
 
+and as a command:
+
+```shell-session
+$ python3 -m bracex -0 "base/{a,b}/{1..2}" | xargs -0 mkdir -p
+$ tree base/
+base/
+├── a
+│   ├── 1
+│   └── 2
+└── b
+    ├── 1
+    └── 2
+```
+
 - **Why Bracex over other solutions?**
 
     Bracex actually follows pretty closely to how Bash processes braces. It is not a 1:1 implementation of how Bash
@@ -25,6 +39,8 @@ Bracex adds this ability to Python:
     deviated. For instance, we are not handling Bash's command line inputs, so we are not giving special meaning to back
     ticks and quotes at this time.
 
+    On the command line Bracex can handle more expansions than Bash itself.
+
 More Examples:
 
 ```pycon3
@@ -115,3 +131,22 @@ def iexpand(string, keep_escapes=False, limit=1000):
 ```
 
 `iexpand` is just like `expand` except it returns a generator.
+
+## Command line interface
+
+```shell-session
+$ python3 -m bracex --help
+usage: python -mbracex [-h] [--terminator STR | -0] [--version] expression
+
+Expands a bash-style brace expression, and outputs each expansion.
+
+positional arguments:
+  expression            Brace expression to expand
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --terminator STR, -t STR
+                        Terminate each expansion with string STR (default: \n)
+  -0                    Terminate each expansion with a NUL character
+  --version             show program's version number and exit
+```


Test patch:
diff --git a/tests/test_main.py b/tests/test_main.py
new file mode 100644
index 0000000..335b5a4
--- /dev/null
+++ b/tests/test_main.py
@@ -0,0 +1,95 @@
+"""Test command module and argument handling."""
+import pytest
+from bracex.__main__ import main
+from bracex import __version__
+
+
+def test_expand_with_default_terminator(capsys):
+    """Test that by default one expansion is printed per line."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['{a..c}'])
+    capture = capsys.readouterr()
+    assert capture.out == "a\nb\nc\n"
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_expand_with_spaces(capsys):
+    """Test that expansions can be space terminated."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['-t', ' ', '{a..c}'])
+    capture = capsys.readouterr()
+    assert capture.out == "a b c "
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_expand_with_empty_terminators(capsys):
+    """Test that expansions can be terminated by an empty string."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['-t', '', '{a..c}'])
+    capture = capsys.readouterr()
+    assert capture.out == "abc"
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_expand_with_nul_terminators(capsys):
+    """Test that expansions can be terminated by a NUL character."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['-0', '{a..c}'])
+    capture = capsys.readouterr()
+    assert capture.out == "a\x00b\x00c\x00"
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_terminator_arguments_are_mutually_exclusive(capsys):
+    """Test that contradicting terminators raise an error."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['-0', '--terminator', ' ', '{a..c}'])
+
+    capture = capsys.readouterr()
+    assert capture.out == ""
+    assert capture.err.find("error: argument --terminator/-t: not allowed with argument -0") > 0
+    assert exinfo.value.code > 0
+
+
+def test_help(capsys):
+    """Test that help is available."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['--help'])
+    capture = capsys.readouterr()
+    assert capture.out.startswith("usage:")
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_version(capsys):
+    """Test that the version is available."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['--version'])
+    capture = capsys.readouterr()
+    assert capture.out == f"{__version__}\n"
+    assert capture.err == ""
+    assert exinfo.value.code == 0
+
+
+def test_no_args_is_considered_an_error(capsys):
+    """Test that an error is reported when no expression is provided."""
+    with pytest.raises(SystemExit) as exinfo:
+        main([])
+    capture = capsys.readouterr()
+    assert capture.out == ""
+    assert capture.err.endswith("error: the following arguments are required: expression\n")
+    assert exinfo.value.code > 0
+
+
+def test_excess_args_is_considered_an_error(capsys):
+    """Test that an error is reported when too many arguments are provided."""
+    with pytest.raises(SystemExit) as exinfo:
+        main(['{a,b,c}', '{1..3}'])
+    capture = capsys.readouterr()
+    assert capture.out == ""
+    assert capture.err.find("error: unrecognized arguments") > 0
+    assert exinfo.value.code > 0