Repository: Textualize/rich

Issue:
[BUG] ANSI sequences parsed incorrectly: \x1b(B\x1b[m
- [x] I've checked [docs](https://rich.readthedocs.io/en/latest/introduction.html) and [closed issues](https://github.com/Textualize/rich/issues?q=is%3Aissue+is%3Aclosed) for possible solutions.
- [x] I can't find my issue in the [FAQ](https://github.com/Textualize/rich/blob/master/FAQ.md).

**Describe the bug**

Some ANSI sequences (which I didn't know about) are not correctly parsed.
Here is a text example with such ANSI sequences:

```
src/failprint/capture.py:57: \x1b[1m\x1b[31merror:\x1b(B\x1b[m Property \x1b(B\x1b[m\x1b[1m"buffer"\x1b(B\x1b[m defined in \x1b(B\x1b[m\x1b[1m"TextIOWrapper"\x1b(B\x1b[m is read-only  \x1b(B\x1b[m\x1b[33m[misc]\x1b(B\x1b[m
src/failprint/capture.py:57: \x1b[1m\x1b[31merror:\x1b(B\x1b[m Incompatible types in assignment (expression has type \x1b(B\x1b[m\x1b[1m"_BytesBuffer"\x1b(B\x1b[m, variable has type \x1b(B\x1b[m\x1b[1m"BinaryIO"\x1b(B\x1b[m)  \x1b(B\x1b[m\x1b[33m[assignment]\x1b(B\x1b[m
src/failprint/capture.py:77: \x1b[1m\x1b[31merror:\x1b(B\x1b[m Argument 1 to \x1b(B\x1b[m\x1b[1m"StringIO"\x1b(B\x1b[m has incompatible type \x1b(B\x1b[m\x1b[1m"StringIO"\x1b(B\x1b[m; expected \x1b(B\x1b[m\x1b[1m"Optional[str]"\x1b(B\x1b[m  \x1b(B\x1b[m\x1b[33m[arg-type]\x1b(B\x1b[m
src/failprint/runners.py:221: \x1b[1m\x1b[31merror:\x1b(B\x1b[m Incompatible return value type (got \x1b(B\x1b[m\x1b[1m"Union[str, int, None]"\x1b(B\x1b[m, expected \x1b(B\x1b[m\x1b[1m"int"\x1b(B\x1b[m)  \x1b(B\x1b[m\x1b[33m[return-value]\x1b(B\x1b[m
\x1b[1m\x1b[31mFound 4 errors in 2 files (checked 18 source files)\x1b(B\x1b[m
```

This is the (ANSI colored) output of mypy.
Other libraries like `ansimarkup` are able to print it.

```python
from ansimarkup import ansiprint
ansiprint('\x1b[31mFound 4 errors in 2 files (checked 18 source files)\x1b(B\x1b[m\n')
```

```python
from rich.text import Text
from rich import print as richprint
richprint(Text.from_ansi('\x1b[31mFound 4 errors in 2 files (checked 18 source files)\x1b(B\x1b[m\n'))
```

![richansi](https://user-images.githubusercontent.com/3999221/206017227-0d275ae4-667b-4fb3-a82f-96ef255b543d.png)

Note the trailing `B` in the above screenshot.

**Platform**
<details>
<summary>Click to expand</summary>

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you're using Rich in a terminal:

```
python -m rich.diagnose
pip freeze | grep rich
```

```
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                  │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=191 ColorSystem.TRUECOLOR>                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                  │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 19                                                            │
│    is_alt_screen = False                                                         │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=191, height=19),             │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=191,                                            │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        max_height=19,                                            │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        markup=None,                                              │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=191, height=19)                       │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 191                                                           │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭─── <class 'rich._windows.WindowsConsoleFeatures'> ────╮
│ Windows features available.                           │
│                                                       │
│ ╭───────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=False, truecolor=False) │ │
│ ╰───────────────────────────────────────────────────╯ │
│                                                       │
│ truecolor = False                                     │
│        vt = False                                     │
╰───────────────────────────────────────────────────────╯
╭────── Environment Variables ───────╮
│ {                                  │
│     'TERM': 'xterm-256color',      │
│     'COLORTERM': 'truecolor',      │
│     'CLICOLOR': None,              │
│     'NO_COLOR': None,              │
│     'TERM_PROGRAM': 'vscode',      │
│     'COLUMNS': None,               │
│     'LINES': None,                 │
│     'JUPYTER_COLUMNS': None,       │
│     'JUPYTER_LINES': None,         │
│     'JPY_PARENT_PID': None,        │
│     'VSCODE_VERBOSE_LOGGING': None │
│ }                                  │
╰────────────────────────────────────╯
platform="Linux"
rich==12.6.0
```

</details>



Discussion:
Thank you for your issue. Give us a little time to review it.

PS. You might want to check the [FAQ](https://github.com/textualize/rich/blob/master/FAQ.md) if you haven't done so already.

This is an automated reply, generated by [FAQtory](https://github.com/willmcgugan/faqtory)
@pawamoy hi - I'm not a maintainer/contributor to `rich` so far, just a curious learner.

I haven't known much about ANSI sequences in the past either, although I was _vaguely_ aware of them.  So far I'm thinking that the `\x1(` prefix before the `B` corresponds to the ["nF escape sequences"](https://en.wikipedia.org/wiki/ANSI_escape_code#nF_Escape_sequences) grouping, as described on wikipedia.  Does that match your assessment too?  If possible it'd be nice to confirm this from some other places!

(and to explain the comment: I'm trying to figure out what the expected behaviour should be in this case -- and then we can write a test case, and if necessary fix any incorrect behaviour of the code)
Here is the [code](https://github.com/Textualize/rich/blob/master/rich/ansi.py) in question. It is somewhat limited in the ansi codes it processes. It would be nice to handle what makes sense, but it should also strip out unfamiliar codes.
@jayaddison thanks for the investigation!
On my end I verified that these codes really come from mypy, and they do; looking at their [FancyFormatter](https://github.com/python/mypy/blob/04d44c12cfbb3a469a96253d4656d16c21be41b9/mypy/util.py#L530), I ran this in a Python REPL:

```pycon
>>> import sys
>>> from mypy.util import FancyFormatter
>>> ff = FancyFormatter(sys.stdout, sys.stderr, hide_error_codes=False)
>>> ff.NORMAL
'\x1b(B\x1b[m'
```

Since Rich seems to render the right colors/style, it seems fine to me if it also strips these unfamiliar codes.
Nice, thanks @pawamoy - great to know where the sequence is being emitted from. :+1: 

It looks like the [component of the regex](https://github.com/Textualize/rich/blob/8bcb7011cfb01462fcd08c7298e756c5e4486617/rich/ansi.py#L13) that matches the `(` is the first item in the character class `[(@-Z\\-_]` (`0x28, 0x40-to-0x51, 0x5c, 0x2d, 0x5f`).
I believe I am having a similar problem when trying to read the output of the Tree() module. Running the script through the python console results in everything formatted correctly. However, running through powershell (I am doing everything through pycharm) results in the same formatting issues as above, which is also rednered properly with ansiprint

```
from rich import get_console
from rich.console import Console
from rich.markup import escape
from rich.tree import Tree
from rich import print as rich_print
from pathlib import Path
from ansimarkup import ansiprint

import logging
from logging import FileHandler

DEBUG = True

path_to_log = r"C:\Users\User\test\log.log"

FORMAT = "%(message)s"
logging.basicConfig(
    level="NOTSET", format=FORMAT, datefmt="[%X]", handlers=[FileHandler(path_to_log, encoding='utf-8')],
)

log = logging.getLogger("rich")

path = Path(r"C:\Users\User\test")  # path to arbitrary folder

tree = Tree(label="Test")
style = "dim" if path.name.startswith("__") else ""
tree.add(
    f"[bold magenta]:open_file_folder: [link file://{path}]{escape(path.name)}",
    style=style,
    guide_style=style,
)

console = get_console()
with console.capture() as capture:
    console.print(tree, markup=True)

if DEBUG:
    breakpoint()
    rich_print(capture.get())
    ansiprint(capture.get())

log.info(capture.get())
```
To run this just need to point path_to_log and path_ to a log file output and path to arbitary folder, respectively. 

Running this through powershell with `python -m test_tree' results in the log-file displaying the tree formatting incorrectly (it displays correctly if using 'Run' through pycharm / python console.

![image](https://user-images.githubusercontent.com/55797454/208241455-11cafc1a-31e9-4ca9-befd-dce32be5989b.png)

![image](https://user-images.githubusercontent.com/55797454/208241473-4e1716fc-71f8-4703-94c9-c8c147a8e2b5.png)

Platform:
<details>
```
(datashuttle_conda) C:\fMRIData\git-repo\datashuttle\datashuttle\utils>python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                  │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=266 ColorSystem.TRUECOLOR>                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                  │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 16                                                            │
│    is_alt_screen = False                                                         │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=266, height=16),             │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=266,                                            │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        max_height=16,                                            │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        markup=None,                                              │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=266, height=16)                       │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 266                                                           │
╰──────────────────────────────────────────────────────────────────────────────────╯
╭── <class 'rich._windows.WindowsConsoleFeatures'> ───╮
│ Windows features available.                         │
│                                                     │
│ ╭─────────────────────────────────────────────────╮ │
│ │ WindowsConsoleFeatures(vt=True, truecolor=True) │ │
│ ╰─────────────────────────────────────────────────╯ │
│                                                     │
│ truecolor = True                                    │
│        vt = True                                    │
╰─────────────────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────────────── Environment Variables ───────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ {'TERM': None, 'COLORTERM': None, 'CLICOLOR': None, 'NO_COLOR': None, 'TERM_PROGRAM': None, 'COLUMNS': None, 'LINES': None, 'JUPYTER_COLUMNS': None, 'JUPYTER_LINES': None, 'JPY_PARENT_PID': None, 'VSCODE_VERBOSE_LOGGING': None} │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
platform="Windows"
```
</details>






Resolving patch:
diff --git a/rich/ansi.py b/rich/ansi.py
--- a/rich/ansi.py
+++ b/rich/ansi.py
@@ -43,6 +43,9 @@ def _ansi_tokenize(ansi_text: str) -> Iterable[_AnsiToken]:
         if start > position:
             yield _AnsiToken(ansi_text[position:start])
         if sgr:
+            if sgr == "(":
+                position = end + 1
+                continue
             if sgr.endswith("m"):
                 yield _AnsiToken("", sgr[1:-1], osc)
         else:


Test patch:
diff --git a/tests/test_ansi.py b/tests/test_ansi.py
--- a/tests/test_ansi.py
+++ b/tests/test_ansi.py
@@ -1,3 +1,5 @@
+import pytest
+
 from rich.ansi import AnsiDecoder
 from rich.console import Console
 from rich.style import Style
@@ -45,3 +47,24 @@ def test_decode_example():
     print(repr(result))
     expected = "\x1b[1mC:\\Users\\stefa\\AppData\\Local\\Temp\\tmp3ydingba:\x1b[0m In function '\x1b[1mmain\x1b[0m':\n\x1b[1mC:\\Users\\stefa\\AppData\\Local\\Temp\\tmp3ydingba:3:5:\x1b[0m \x1b[1;35mwarning: \x1b[0munused variable '\x1b[1ma\x1b[0m' \n[\x1b[1;35m-Wunused-variable\x1b[0m]\n    3 | int \x1b[1;35ma\x1b[0m=1;\n      |     \x1b[1;35m^\x1b[0m\n"
     assert result == expected
+
+
+@pytest.mark.parametrize(
+    "ansi_bytes, expected_text",
+    [
+        # https://github.com/Textualize/rich/issues/2688
+        (
+            b"\x1b[31mFound 4 errors in 2 files (checked 18 source files)\x1b(B\x1b[m\n",
+            "Found 4 errors in 2 files (checked 18 source files)",
+        ),
+        # https://mail.python.org/pipermail/python-list/2007-December/424756.html
+        (b"Hallo", "Hallo"),
+        (b"\x1b(BHallo", "Hallo"),
+        (b"\x1b(JHallo", "Hallo"),
+        (b"\x1b(BHal\x1b(Jlo", "Hallo"),
+    ],
+)
+def test_decode_issue_2688(ansi_bytes, expected_text):
+    text = Text.from_ansi(ansi_bytes.decode())
+
+    assert str(text) == expected_text