Repository: psf/black

Issue:
Black can break strings immediately after an escape
**Describe the bug**
Black does not produce identical code if it unluckily chooses to split a regex pattern right after an escape.

**To Reproduce** Steps to reproduce the behavior:
Given black installed via a particular git commit - accomplished via `python3 -m pip install git+git://github.com/psf/black.git@ea03b6c5e1f96aa2569f0080a61577b368b7af0d`
```
black                    19.10b1.dev118+gea03b6c
```

and a long regex string, as found in this example file foo.py:
```python
REGEX_PATTERNS = {
    "asdf_hjkl_jkl": re.compile(
        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
    ),
}
```

Black will error s follows:
```
python3 -m black --diff foo.py
error: cannot format foo.py: INTERNAL ERROR: Black produced invalid code: EOL while scanning string literal (<unknown>, line 3). Please report a bug on https://github.com/psf/black/issues.  This invalid output might be helpful: /var/folders/1z/64vyvtv136v6mfhfr16hws3c0000gn/T/blk_bbdmeyyx.log
Oh no! 💥 💔 💥
1 file would fail to reformat.
```

The contents of that log are as follows:

```shell
% cat /var/folders/1z/64vyvtv136v6mfhfr16hws3c0000gn/T/blk_bbdmeyyx.log
  File "/usr/local/lib/python3.7/site-packages/black/__init__.py", line 6042, in assert_equivalent
    dst_ast = parse_ast(dst)
  File "/usr/local/lib/python3.7/site-packages/black/__init__.py", line 5953, in parse_ast
    return ast27.parse(src)
  File "/Users/matt/Library/Python/3.7/lib/python/site-packages/typed_ast/ast27.py", line 50, in parse
    return _ast27.parse(source, filename, mode)
REGEX_PATTERNS = {
    "asdf_hjkl_jkl": re.compile(
        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\"
        r" )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
    ),
}
```

**Expected behavior**

Don't break strings immediately after an escape. This seems like it requires inspection of the string contents to avoid this.

**Environment:**

- Version: master
- OS and Python version: Mac OS X / python 3.7.7

**Does this bug also happen on master?** 
Yes. In fact, using https://black.now.sh/?version=master it ignores the error and produces invalid python code, which is a UI issue with that site.



Discussion:
I tried to run with the latest version of black on this code on my computer and it worked just fine.
I can confirm this is now fixed
Reopening as this isn't actually fixed. The additional string processing Black used to do has been locked away under a hidden flag (`--experimental-string-processing`) as we were in the process of cutting a new release and we didn't want to include new features without it being stable. If we turn on the extra string processing the exact same bug reappears:

```console
(black) ichard26@acer-ubuntu:~/programming/oss/black$ cat temp.py
REGEX_PATTERNS = {
    "asdf_hjkl_jkl": re.compile(
        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
    ),
}
(black) ichard26@acer-ubuntu:~/programming/oss/black$ black temp.py --experimental-string-processing --diff --color
error: cannot format temp.py: INTERNAL ERROR: Black produced invalid code: EOL while scanning string literal (<unknown>, line 3). Please report a bug on https://github.com/psf/black/issues.  This invalid output might be helpful: /tmp/blk_3ys_qh0e.log
Oh no! 💥 💔 💥
1 file would fail to reformat.
(black) ichard26@acer-ubuntu:~/programming/oss/black$ cat /tmp/blk_3ys_qh0e.log 
  File "/home/ichard26/programming/oss/black/src/black/__init__.py", line 6244, in assert_equivalent
    dst_ast = parse_ast(dst)
  File "/home/ichard26/programming/oss/black/src/black/__init__.py", line 6155, in parse_ast
    return ast27.parse(src)
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/typed_ast/ast27.py", line 50, in parse
    return _ast27.parse(source, filename, mode)
REGEX_PATTERNS = {
    "asdf_hjkl_jkl": re.compile(
        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\"
        r" )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
    ),
}
```
<details>
<summary><b>Environment</b></summary>

* Black version: `master` (f311d82569b9595d85c08cc8fcf5250de525e7a0)
* Python version: `CPython 3.8.5`
* OS version: `Ubuntu 20.04.01 LTS`

</details>

There are plans to make the extra string processing significantly more stable (actually [there have been efforts courtesy of bbugyi200](https://github.com/psf/black/pulls?q=is%3Apr+author%3Abbugyi200)) and then have the features turned on by default.
cc @bbugyi200 in case you're interested in fixing this one too (thanks for your work on the other string issues!)

Resolving patch:
diff --git a/src/black/__init__.py b/src/black/__init__.py
--- a/src/black/__init__.py
+++ b/src/black/__init__.py
@@ -3590,7 +3590,8 @@ class StringSplitter(CustomSplitMapMixin, BaseStringSplitter):
         MIN_SUBSTR_SIZE characters.
 
         The string will ONLY be split on spaces (i.e. each new substring should
-        start with a space).
+        start with a space). Note that the string will NOT be split on a space
+        which is escaped with a backslash.
 
         If the string is an f-string, it will NOT be split in the middle of an
         f-expression (e.g. in f"FooBar: {foo() if x else bar()}", {foo() if x
@@ -3930,11 +3931,23 @@ def passes_all_checks(i: Index) -> bool:
                 section of this classes' docstring would be be met by returning @i.
             """
             is_space = string[i] == " "
+
+            is_not_escaped = True
+            j = i - 1
+            while is_valid_index(j) and string[j] == "\\":
+                is_not_escaped = not is_not_escaped
+                j -= 1
+
             is_big_enough = (
                 len(string[i:]) >= self.MIN_SUBSTR_SIZE
                 and len(string[:i]) >= self.MIN_SUBSTR_SIZE
             )
-            return is_space and is_big_enough and not breaks_fstring_expression(i)
+            return (
+                is_space
+                and is_not_escaped
+                and is_big_enough
+                and not breaks_fstring_expression(i)
+            )
 
         # First, we check all indices BELOW @max_break_idx.
         break_idx = max_break_idx


Test patch:
diff --git a/tests/data/long_strings__regression.py b/tests/data/long_strings__regression.py
--- a/tests/data/long_strings__regression.py
+++ b/tests/data/long_strings__regression.py
@@ -353,6 +353,24 @@ def xxxxxxx_xxxxxx(xxxx):
     key
 ] = "test"  # set some Thrift field to non-None in the struct aa bb cc dd ee
 
+RE_ONE_BACKSLASH = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}
+
+RE_TWO_BACKSLASHES = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}
+
+RE_THREE_BACKSLASHES = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\\\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}
+
 # output
 
 
@@ -793,3 +811,22 @@ def xxxxxxx_xxxxxx(xxxx):
 value.__dict__[
     key
 ] = "test"  # set some Thrift field to non-None in the struct aa bb cc dd ee
+
+RE_ONE_BACKSLASH = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}
+
+RE_TWO_BACKSLASHES = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\\"
+        r" )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}
+
+RE_THREE_BACKSLASHES = {
+    "asdf_hjkl_jkl": re.compile(
+        r"(?<!([0-9]\ ))(?<=(^|\ ))([A-Z]+(\ )?|[0-9](\ )|[a-z](\\\ )){4,7}([A-Z]|[0-9]|[a-z])($|\b)(?!(\ ?([0-9]\ )|(\.)))"
+    ),
+}