Repository: psf/black

Issue:
Black --preview (ESP) crashes with implicitly concatenated stringified generic type params
<!--
Please make sure that the bug is not already fixed either in newer versions or the
current development version. To confirm this, you have three options:

1. Update Black's version if a newer release exists: `pip install -U black`
2. Use the online formatter at <https://black.vercel.app/?version=main>, which will use
   the latest main branch.
3. Or run _Black_ on your machine:
   - create a new virtualenv (make sure it's the same Python version);
   - clone this repository;
   - run `pip install -e .[d]`;
   - run `pip install -r test_requirements.txt`
   - make sure it's sane by running `python -m pytest`; and
   - run `black` like you did last time.
-->

**Describe the bug**

<!-- A clear and concise description of what the bug is. -->

**To Reproduce**

<!--
Minimal steps to reproduce the behavior with source code and Black's configuration.
-->

For example, take this code:

```python
def func(
    arg1,
    arg2,
) -> Set["this_is_a_very_long_module_name.AndAVeryLongClasName"
         ".WithASubClass"]:
  pass
```

And run it with these arguments:

```sh
$ black --preview t.py
```

The resulting error is:

```
error: cannot format /Users/yileiyang/Downloads/t.py: Unable to match a closing bracket to the following opening bracket: )

Oh no! 💥 💔 💥
1 file would fail to reformat.
```

**Expected behavior**

<!-- A clear and concise description of what you expected to happen. -->

**Environment**

<!-- Please complete the following information: -->

- Black's version: this crash existed since v22.1.0, doesn't affect v23.1a1 since it's ESP

**Additional context**

<!-- Add any other context about the problem here. -->

```
Traceback (most recent call last):
  File "/Users/yileiyang/github.com/yilei/black/src/black/brackets.py", line 99, in mark
    opening_bracket = self.bracket_match.pop((self.depth, leaf.type))
KeyError: (-1, 8)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/yileiyang/github.com/yilei/black/src/black/__init__.py", line 1077, in format_str
    dst_contents = _format_str_once(src_contents, mode=mode)
  File "/Users/yileiyang/github.com/yilei/black/src/black/__init__.py", line 1109, in _format_str_once
    for line in transform_line(
  File "/Users/yileiyang/github.com/yilei/black/src/black/linegen.py", line 577, in transform_line
    result = run_transformer(line, transform, mode, features, line_str=line_str)
  File "/Users/yileiyang/github.com/yilei/black/src/black/linegen.py", line 1420, in run_transformer
    result.extend(transform_line(transformed_line, mode=mode, features=features))
  File "/Users/yileiyang/github.com/yilei/black/src/black/linegen.py", line 577, in transform_line
    result = run_transformer(line, transform, mode, features, line_str=line_str)
  File "/Users/yileiyang/github.com/yilei/black/src/black/linegen.py", line 1416, in run_transformer
    for transformed_line in transform(line, features):
  File "/Users/yileiyang/github.com/yilei/black/src/black/trans.py", line 251, in __call__
    for line_result in self.do_transform(line, string_idx):
  File "/Users/yileiyang/github.com/yilei/black/src/black/trans.py", line 395, in do_transform
    msg_result = self._merge_string_group(new_line, string_idx)
  File "/Users/yileiyang/github.com/yilei/black/src/black/trans.py", line 602, in _merge_string_group
    append_leaves(new_line, line, [leaf])
  File "/Users/yileiyang/github.com/yilei/black/src/black/lines.py", line 707, in append_leaves
    new_line.append(new_leaf, preformatted=preformatted)
  File "/Users/yileiyang/github.com/yilei/black/src/black/lines.py", line 81, in append
    self.bracket_tracker.mark(leaf)
  File "/Users/yileiyang/github.com/yilei/black/src/black/brackets.py", line 101, in mark
    raise BracketMatchError(
black.brackets.BracketMatchError: Unable to match a closing bracket to the following opening bracket: )
```


Discussion:

Resolving patch:
diff --git a/src/black/brackets.py b/src/black/brackets.py
--- a/src/black/brackets.py
+++ b/src/black/brackets.py
@@ -80,9 +80,12 @@ def mark(self, leaf: Leaf) -> None:
         within brackets a given leaf is. 0 means there are no enclosing brackets
         that started on this line.
 
-        If a leaf is itself a closing bracket, it receives an `opening_bracket`
-        field that it forms a pair with. This is a one-directional link to
-        avoid reference cycles.
+        If a leaf is itself a closing bracket and there is a matching opening
+        bracket earlier, it receives an `opening_bracket` field with which it forms a
+        pair. This is a one-directional link to avoid reference cycles. Closing
+        bracket without opening happens on lines continued from previous
+        breaks, e.g. `) -> "ReturnType":` as part of a funcdef where we place
+        the return type annotation on its own line of the previous closing RPAR.
 
         If a leaf is a delimiter (a token on which Black can split the line if
         needed) and it's on depth 0, its `id()` is stored in the tracker's
@@ -91,6 +94,13 @@ def mark(self, leaf: Leaf) -> None:
         if leaf.type == token.COMMENT:
             return
 
+        if (
+            self.depth == 0
+            and leaf.type in CLOSING_BRACKETS
+            and (self.depth, leaf.type) not in self.bracket_match
+        ):
+            return
+
         self.maybe_decrement_after_for_loop_variable(leaf)
         self.maybe_decrement_after_lambda_arguments(leaf)
         if leaf.type in CLOSING_BRACKETS:
diff --git a/src/black/nodes.py b/src/black/nodes.py
--- a/src/black/nodes.py
+++ b/src/black/nodes.py
@@ -848,3 +848,15 @@ def is_string_token(nl: NL) -> TypeGuard[Leaf]:
 
 def is_number_token(nl: NL) -> TypeGuard[Leaf]:
     return nl.type == token.NUMBER
+
+
+def is_part_of_annotation(leaf: Leaf) -> bool:
+    """Returns whether this leaf is part of type annotations."""
+    ancestor = leaf.parent
+    while ancestor is not None:
+        if ancestor.prev_sibling and ancestor.prev_sibling.type == token.RARROW:
+            return True
+        if ancestor.parent and ancestor.parent.type == syms.tname:
+            return True
+        ancestor = ancestor.parent
+    return False
diff --git a/src/black/trans.py b/src/black/trans.py
--- a/src/black/trans.py
+++ b/src/black/trans.py
@@ -30,7 +30,6 @@
 
 from mypy_extensions import trait
 
-from black.brackets import BracketMatchError
 from black.comments import contains_pragma_comment
 from black.lines import Line, append_leaves
 from black.mode import Feature
@@ -41,6 +40,7 @@
     is_empty_lpar,
     is_empty_par,
     is_empty_rpar,
+    is_part_of_annotation,
     parent_type,
     replace_child,
     syms,
@@ -351,7 +351,7 @@ class StringMerger(StringTransformer, CustomSplitMapMixin):
 
     Requirements:
         (A) The line contains adjacent strings such that ALL of the validation checks
-        listed in StringMerger.__validate_msg(...)'s docstring pass.
+        listed in StringMerger._validate_msg(...)'s docstring pass.
             OR
         (B) The line contains a string which uses line continuation backslashes.
 
@@ -377,6 +377,8 @@ def do_match(self, line: Line) -> TMatchResult:
                 and is_valid_index(i + 1)
                 and LL[i + 1].type == token.STRING
             ):
+                if is_part_of_annotation(leaf):
+                    return TErr("String is part of type annotation.")
                 return Ok(i)
 
             if leaf.type == token.STRING and "\\\n" in leaf.value:
@@ -454,7 +456,7 @@ def _merge_string_group(self, line: Line, string_idx: int) -> TResult[Line]:
 
         Returns:
             Ok(new_line), if ALL of the validation checks found in
-            __validate_msg(...) pass.
+            _validate_msg(...) pass.
                 OR
             Err(CannotTransform), otherwise.
         """
@@ -608,7 +610,7 @@ def make_naked(string: str, string_prefix: str) -> str:
     def _validate_msg(line: Line, string_idx: int) -> TResult[None]:
         """Validate (M)erge (S)tring (G)roup
 
-        Transform-time string validation logic for __merge_string_group(...).
+        Transform-time string validation logic for _merge_string_group(...).
 
         Returns:
             * Ok(None), if ALL validation checks (listed below) pass.
@@ -622,6 +624,11 @@ def _validate_msg(line: Line, string_idx: int) -> TResult[None]:
                 - The set of all string prefixes in the string group is of
                   length greater than one and is not equal to {"", "f"}.
                 - The string group consists of raw strings.
+                - The string group is stringified type annotations. We don't want to
+                  process stringified type annotations since pyright doesn't support
+                  them spanning multiple string values. (NOTE: mypy, pytype, pyre do
+                  support them, so we can change if pyright also gains support in the
+                  future. See https://github.com/microsoft/pyright/issues/4359.)
         """
         # We first check for "inner" stand-alone comments (i.e. stand-alone
         # comments that have a string leaf before them AND after them).
@@ -812,13 +819,7 @@ def do_transform(self, line: Line, string_idx: int) -> Iterator[TResult[Line]]:
 
         new_line = line.clone()
         new_line.comments = line.comments.copy()
-        try:
-            append_leaves(new_line, line, LL[: string_idx - 1])
-        except BracketMatchError:
-            # HACK: I believe there is currently a bug somewhere in
-            # right_hand_split() that is causing brackets to not be tracked
-            # properly by a shared BracketTracker.
-            append_leaves(new_line, line, LL[: string_idx - 1], preformatted=True)
+        append_leaves(new_line, line, LL[: string_idx - 1])
 
         string_leaf = Leaf(token.STRING, LL[string_idx].value)
         LL[string_idx - 1].remove()


Test patch:
diff --git a/tests/data/preview/long_strings__type_annotations.py b/tests/data/preview/long_strings__type_annotations.py
new file mode 100644
--- /dev/null
+++ b/tests/data/preview/long_strings__type_annotations.py
@@ -0,0 +1,59 @@
+def func(
+    arg1,
+    arg2,
+) -> Set["this_is_a_very_long_module_name.AndAVeryLongClasName"
+         ".WithAVeryVeryVeryVeryVeryLongSubClassName"]:
+  pass
+
+
+def func(
+    argument: (
+        "VeryLongClassNameWithAwkwardGenericSubtype[int] |"
+        "VeryLongClassNameWithAwkwardGenericSubtype[str]"
+    ),
+) -> (
+    "VeryLongClassNameWithAwkwardGenericSubtype[int] |"
+    "VeryLongClassNameWithAwkwardGenericSubtype[str]"
+):
+  pass
+
+
+def func(
+    argument: (
+        "int |"
+        "str"
+    ),
+) -> Set["int |"
+         " str"]:
+  pass
+
+
+# output
+
+
+def func(
+    arg1,
+    arg2,
+) -> Set[
+    "this_is_a_very_long_module_name.AndAVeryLongClasName"
+    ".WithAVeryVeryVeryVeryVeryLongSubClassName"
+]:
+    pass
+
+
+def func(
+    argument: (
+        "VeryLongClassNameWithAwkwardGenericSubtype[int] |"
+        "VeryLongClassNameWithAwkwardGenericSubtype[str]"
+    ),
+) -> (
+    "VeryLongClassNameWithAwkwardGenericSubtype[int] |"
+    "VeryLongClassNameWithAwkwardGenericSubtype[str]"
+):
+    pass
+
+
+def func(
+    argument: ("int |" "str"),
+) -> Set["int |" " str"]:
+    pass