Repository: psf/black

Issue:
Fix Bug exposed by Fuzz Tests
**Describe the bug**
Fuzz testing is failing with libcst errors I don't get.

```
fuzz run-test: commands[2] | coverage run fuzz.py
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
<string>:1: SyntaxWarning: "is" with a literal. Did you mean "=="?
Highest target scores:
               5  (label='(hypothesmith) number of unique ast node types')
               9  (label='(hypothesmith) instructions in bytecode')
              10  (label='(hypothesmith from_node) number of unique ast node types')
              13  (label='(hypothesmith) total number of ast nodes')
              35  (label='(hypothesmith from_node) total number of ast nodes')
              48  (label='(hypothesmith from_node) instructions in bytecode')

Traceback (most recent call last):
  File "<string>", line 7, in __init__
  File "/home/cooper/repos/black/.tox/fuzz/lib/python3.9/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/cooper/repos/black/.tox/fuzz/lib/python3.9/site-packages/libcst/_nodes/expression.py", line 1148, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Must have at least one space around comparison operator.
```

**To Reproduce** 
Run `tox -e fuzz` on head

**Expected behavior**
Fuzz tests to pass.


Discussion:
Would love some pointers here @Zac-HD if you get 5 mins :1st_place_medal: 
<details>
<summary>Console output of <code>python fuzz.py</code> on <code>master</code> as I write (warning: there's a lot of text)</summary>

```console
Highest target scores:
               6  (label='(hypothesmith) number of unique ast node types')
               7  (label='(hypothesmith) instructions in bytecode')
              10  (label='(hypothesmith) total number of ast nodes')
              11  (label='(hypothesmith from_node) number of unique ast node types')
              23  (label='(hypothesmith from_node) total number of ast nodes')
              55  (label='(hypothesmith from_node) instructions in bytecode')

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/op.py", line 62, in _validate
    raise CSTValidationError("Must have at least one space between not and in.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Must have at least one space between not and in.

Traceback (most recent call last):
  File "<string>", line 5, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 1064, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Must have at least one space around comparison operator.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 2920, in _validate
    super(Tuple, self)._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 235, in _validate
    raise CSTValidationError("Cannot have unbalanced parens.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Cannot have unbalanced parens.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 2924, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: A zero-length tuple must be wrapped in parentheses.

Traceback (most recent call last):
  File "<string>", line 8, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 233, in _validate
    raise CSTValidationError("Cannot have right paren without left paren.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Cannot have right paren without left paren.

Traceback (most recent call last):
  File "<string>", line 9, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 783, in _validate
    raise CSTValidationError("Cannot have a name for an empty type.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Cannot have a name for an empty type.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/op.py", line 692, in _validate
    raise CSTValidationError("Invalid value for NotEqual node.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Invalid value for NotEqual node.

Falsifying example: test_idempotent_any_syntatically_valid_python(
    src_contents='\t\n\\\r\n',
    mode=Mode(target_versions=set(), line_length=88, string_normalization=True, experimental_string_processing=False, is_pyi=False),
)
Traceback (most recent call last):
  File "fuzz.py", line 43, in test_idempotent_any_syntatically_valid_python
    dst_contents = black.format_str(src_contents, mode=mode)
  File "/home/ichard26/programming/oss/black/src/black/__init__.py", line 1016, in format_str
    src_node = lib2to3_parse(src_contents.lstrip(), mode.target_versions)
  File "/home/ichard26/programming/oss/black/src/black/__init__.py", line 1113, in lib2to3_parse
    result = drv.parse_string(src_txt, True)
  File "/home/ichard26/programming/oss/black/src/blib2to3/pgen2/driver.py", line 142, in parse_string
    return self.parse_tokens(tokens, debug)
  File "/home/ichard26/programming/oss/black/src/blib2to3/pgen2/driver.py", line 70, in parse_tokens
    for quintuple in tokens:
  File "/home/ichard26/programming/oss/black/src/blib2to3/pgen2/tokenize.py", line 542, in generate_tokens
    raise TokenError("EOF in multi-line statement", (lnum, 0))
blib2to3.pgen2.tokenize.TokenError: ('EOF in multi-line statement', (2, 0))

Traceback (most recent call last):
  File "<string>", line 9, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 1873, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: The last WithItem in a With cannot have a trailing comma.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 2387, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: The last NameItem in a Nonlocal cannot have a trailing comma.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 2329, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: The last NameItem in a Global cannot have a trailing comma.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 733, in _validate
    raise CSTValidationError("There must be at least one space before 'as'.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: There must be at least one space before 'as'.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 729, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: There must be at least one space between 'as' and name.

Traceback (most recent call last):
  File "<string>", line 6, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 2920, in _validate
    super(Tuple, self)._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/expression.py", line 231, in _validate
    raise CSTValidationError("Cannot have left paren without right paren.")
  File "<string>", line None
libcst._nodes.base.CSTValidationError: Cannot have left paren without right paren.

Traceback (most recent call last):
  File "<string>", line 7, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 669, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: An indent must be composed of only whitespace characters.

Traceback (most recent call last):
  File "<string>", line 7, in __init__
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/base.py", line 115, in __post_init__
    self._validate()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/libcst/_nodes/statement.py", line 665, in _validate
    raise CSTValidationError(
  File "<string>", line None
libcst._nodes.base.CSTValidationError: An indented block must have a non-zero width indent.

Traceback (most recent call last):
  File "fuzz.py", line 59, in <module>
    test_idempotent_any_syntatically_valid_python()
  File "fuzz.py", line 17, in test_idempotent_any_syntatically_valid_python
    max_examples=1000,  # roughly 1k tests/minute, or half that under coverage
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/hypothesis/core.py", line 1163, in wrapped_test
    raise the_error_hypothesis_found
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/hypothesis/core.py", line 1132, in wrapped_test
    state.run_engine()
  File "/home/ichard26/.local/share/virtualenvs/black-Q9x7i-w-/lib/python3.8/site-packages/hypothesis/core.py", line 860, in run_engine
    raise MultipleFailures(
hypothesis.errors.MultipleFailures: Hypothesis found 16 distinct failures.
```

</details>
Ah, the root of the problem here is that Hypothesmith is relying on a particular behaviour of `st.from_type()` which was never guaranteed and [changed in Hypothesis 5.46](https://hypothesis.readthedocs.io/en/latest/changes.html#v5-46-0) a few days ago.  I'll have a fixed version out in the next few days.
I've now published Hypothesmith 0.1.8, which fixes these issues, and confirmed that `tox -re fuzz` now passes on `master` 🎉
fuzz tests now pass again on master for me with latest hyposmithy.
```
fuzz run-test: commands[1] | coverage erase
fuzz run-test: commands[2] | coverage run fuzz.py
fuzz run-test: commands[3] | coverage report
Name                    Stmts   Miss  Cover
-------------------------------------------
fuzz.py                    15      0   100%
src/_black_version.py       1      0   100%
src/black/__init__.py    3014   1981    34%
-------------------------------------------
TOTAL                    3030   1981    35%
__________________________________________________________________________________________________________________ summary ___________________________________________________________________________________________________________________
  fuzz: commands succeeded
  congratulations :)
  ```
  
  Thanks!
These have now become flaky, the error looks like a real problem tho:
(Have seen it with Python 3.6 and 3.9)
```
AssertionError: INTERNAL ERROR: Black produced different code on the second pass of the formatter.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk__o2fa65u.log
```
Example: https://github.com/psf/black/pull/1918/checks?check_run_id=1679427463
```python
class A:\
# type: ignore
 pass
```

That's the Python code it generated that caused Black to fail. I have a feeling that I've seen something similar before, I'll have to do some searching later.
Right right right, #1425 appears to exhibit the same underlying issue. Also I'mma reopen it cause it's a valid bug report that needs to be resolved by a patch.
What should this format to anyway?

```
class A:\
# type: ignore
 pass
```

If you run this on black.now.sh (or via the black CLI) it doesn't error at all, but instead produces:
```
class A:  # type: ignore
    pass
```

Because of the way `decode_bytes` is written, the actual string that gets passed to `format_str` is `'if ...:\\\n# type: ignore\n pass\n'` which doesn't error.

If this is intended behavior, perhaps strings passed to `format_str` could be "normalized" somehow to this representation.

Adding a couple more instances of this occurring:

- https://github.com/psf/black/runs/1834260212#step:5:71
- https://github.com/psf/black/runs/1849391379#step:5:71

Hoping to find some time to investigate this soon; unless anything else appears preferable until then, the `black.now.sh` format mentioned by @xrisk seems to be the format to aim for.

Resolving patch:
diff --git a/src/black/__init__.py b/src/black/__init__.py
--- a/src/black/__init__.py
+++ b/src/black/__init__.py
@@ -2635,7 +2635,7 @@ def list_comments(prefix: str, *, is_endmarker: bool) -> List[ProtoComment]:
     consumed = 0
     nlines = 0
     ignored_lines = 0
-    for index, line in enumerate(prefix.split("\n")):
+    for index, line in enumerate(re.split("\r?\n", prefix)):
         consumed += len(line) + 1  # adding the length of the split '\n'
         line = line.lstrip()
         if not line:


Test patch:
diff --git a/tests/test_black.py b/tests/test_black.py
--- a/tests/test_black.py
+++ b/tests/test_black.py
@@ -1793,6 +1793,11 @@ def test_bpo_33660_workaround(self) -> None:
         finally:
             os.chdir(str(old_cwd))
 
+    def test_newline_comment_interaction(self) -> None:
+        source = "class A:\\\r\n# type: ignore\n pass\n"
+        output = black.format_str(source, mode=DEFAULT_MODE)
+        black.assert_stable(source, output, mode=DEFAULT_MODE)
+
 
 with open(black.__file__, "r", encoding="utf-8") as _bf:
     black_source_lines = _bf.readlines()