Repository: psf/black

Issue:
Newlines after code block open, once and for all?
Background:
- https://github.com/psf/black/issues/902
  Original issue, with some follow-up comments post merge
- https://github.com/psf/black/pull/3035
  Original implementation
- https://github.com/psf/black/issues/3551
  Follow-up issue
- https://github.com/psf/black/pull/3967 
  Recent preview change partially walking things back
- https://github.com/psf/black/issues/619#issuecomment-1792996633
  Discussion about PEP 8

(there's discussion elsewhere too, but I think these are the most important links)

---

tldr; the 2023 stable style strips newlines after code block opens. This has proven controversial (and was most of the change from 22 to 23 by volume). Recently, we walked this back a little bit in preview style, but we should try to get it right for 24.

The main argument against is it can hurt readability, see https://github.com/psf/black/issues/902#issuecomment-1425713025 . I'm quite sympathetic to this. For instance, functions with a complicated return type and first statement can get quite dense. Similarly, there are examples with comments in a new code block where things can get too dense.

An auxiliary argument against it is that stripping new lines between `class` and first method is in violation of a close reading and changelog analysis of PEP 8, see https://github.com/psf/black/issues/619#issuecomment-1792996633. I don't care much for PEP 8 absolutism (and certainly not at the expense of churning established behaviour), but I consider it a nice-to-have to not reformat away from it.

Recently in https://github.com/psf/black/pull/3967, Henri contributed a nice change that walks this back in some cases, involving comments or compound statements. However, I'm not sure this is enough. [Here are two examples from the first few pages of Black 22 -> 23 diff in my work codebase where I feel readability really suffers](https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ARDAb9dAD2IimZxl1N_WlbvK5V-fnaWIwL5TYBadve5dORVlMpn2CM7Ex6036-fmBvZZxxU7M4SzueDveXQyyVCyVj-_SI8G5MhyXK3nNLHUmR1qpfo3F4ciXU1BMotQs0XgWgkBVKnr_rKC7aIr11sgQsmB5Vap7pzA0R4M5007ghia_366U1-hEjwpaB66-Lkb9ZsEQGd_I3HS8XCUyn1O9pe47U_JM9qrygevkU8y8MHi42r1FAGT-KAUg8Rw7JMVMPhspr1cpS3BzKbGToLSkpjYcCJCj70GUsfc-SKPPQf6n9nHSnwHTurwtoO8zChj8UkwEuEK_PzkD2G_pjTZTi2pwy8tqqwW4_Sg7NqddQaPn1JDfnMvajayo16d3g90jDv7alCLxL4CghJEpl5sU2gaWINat_wzfhsHjDOd77doe6GceCBcWuklNcdz1KAT8N7e_h68QFQcENH51Z8nzi6Nsk0ViOJ7WcYti9iYfIqrura2wKKSmLKz1FVErFWWojvd07WdSB1M2nGrsld_aGwOC7R7wb84ks5wpovY1jSZdnLj1QgrWWAB8ns3Zvi7HnLsmBGxIFV2EtqyAyKkQswAAAAs1goWJRFiskAAdsDxAgAAAhqMAuxxGf7AgAAAAAEWVo=) (scrolling through I see many others). Henri's change won't save us here. And fundamentally this code is quite simple, it just has slightly long type and variable names, i.e. it's not the kind of code where there should have been a comment or something.

---

Jelle nudged me to think about this, and I now feel we should just walk this change back entirely. We should allow users to have either 0 or 1 blank newlines. This is also simpler and more explainable than the partially walked back logic.

I am fine with continuing to strip blank newlines between a function declaration and its docstring (and preview does this for classes as well). We currently enforce one blank newline after class and module docstrings. We don't have an opinion on blank newlines after function docstrings (see https://github.com/psf/black/issues/2370). I think this is probably fine as-is and not having an opinion would be consistent with the proposed no docstring case. (I'll grant that it feels easier to come up with a good opinion for the post docstring blank line in functions. At the very least, we certainly should not resolve that issue by always removing blank lines).

This will not create any churn going from 23 -> 24 (in fact, most of this change technically even meets our stability policy (although any change here should certainly only be made in preview style)).

---

As a meta thing, as a relatively new maintainer of Black, I'm still unsure of my decision making framework. I feel the original change had at most marginal benefit, especially given its churn (I've never browsed code and been like ugh one extra blank line, nor has any code reviewer or linter ever yelled at me for it. These are the things Black usually saves me from. All the motivating changes I found have multiple blank newlines or are toy code). On the other hand, it's very costly to revert changes in stable style. The only thing that makes it possible in this case is that both unreverted and reverted formatting will be valid and inconsistency in this specific case is not costly.


Discussion:
I would support a blank line after the definition of functions without a docstring. I find the two examples provided convincing in that regard.

---

> I am fine with continuing to strip blank newlines between a function declaration and its docstring

I think this behavior should be kept, too. As I've said in [#2370 (comment)](https://github.com/psf/black/issues/2370#issuecomment-1487068337), having a blank line after a docstring would be in violation with all of the major Python docstring conventions (PEP 257, numpydoc and Google). Pydocstyle files "No blank lines allowed after function docstring" under error code D202. D202 is included in all the docstring conventions mentioned above (source: http://www.pydocstyle.org/en/stable/error_codes.html#default-conventions).
My guess would be that it's not uncommon for projects to use both pydocstyle and black and in that case the two should ideally not be in conflict with each other.
OK, let's walk back the change.

@mcnoat reading your comment, there might be confusion between blank lines before and after docstrings? Given this function:

```python
def f():
    
    """
    doc
    """
    
    hi
```

Currently we remove the blank line before the docstring (and Shantanu says we should keep doing that; I agree), but not the one after the docstring. Possibly we should remove the one after the docstring, but that's a discussion for #2370.
> As a meta thing, as a relatively new maintainer of Black, I'm still unsure of my decision making framework.

I feel like I owe you a response to this, but I'm not sure I have a good one :) Some of the things I look at are:

- How disruptive is this change? I generally want to make changes to the stable style only if they feel like clear improvements; when people upgrade Black and see the changes, they should feel it made things better.
- Do I like the change? That's obviously subjective, but I feel part of the point of Black is that instead of the entire Python ecosystem arguing over style decisions, it's just us on this tracker who have to do that.
- Does it feel like the community supports the change? For example, if there are lots of people who comment/like comments in favor of a change, that's a good signal.
- Does the change feel consistent with Black's overall style? Here's a couple of principles that I feel we generally follow, though some may be controversial:
   - No backslashes for formatting, ever
   - Indentation should always be by a multiple of four
   - Formatting should generally not depend on previous formatting, though with obvious exceptions like the magic trailing comma
   - Diffs should be easy to read
   - Fewer lines is better than more

In this particular case, I think a few of us simply misjudged how controversial this change would be. Personally I liked most of the changes Black 22 -> 23 made to our work codebase; many looked like this:

```
 def _source_query_sql_fn(dt: str) -> str:
-
     sql = f"""
         SELECT DISTINCT
```

Where the extra newline just felt like a waste of space.

Hopefully to prevent a repeat of this experience, we can get more eyes on the current preview style (thanks for opening an issue for that!) before promoting it to stable.
> @mcnoat reading your comment, there might be confusion between blank lines before and after docstrings? Given this function:

You are absolutely correct, thanks for pointing it out. Since [hauntsaninja](https://github.com/hauntsaninja) left a comment on #2370, I presumed that this issue was about the same thing and - despite honest effort - did not read his comment or the original post here close enough.

Resolving patch:
diff --git a/src/black/lines.py b/src/black/lines.py
--- a/src/black/lines.py
+++ b/src/black/lines.py
@@ -689,18 +689,14 @@ def _maybe_empty_lines(self, current_line: Line) -> Tuple[int, int]:
                 return 0, 1
             return before, 1
 
+        # In preview mode, always allow blank lines, except right before a function
+        # docstring
         is_empty_first_line_ok = (
-            Preview.allow_empty_first_line_before_new_block_or_comment
-            in current_line.mode
+            Preview.allow_empty_first_line_in_block in current_line.mode
             and (
-                # If it's a standalone comment
-                current_line.leaves[0].type == STANDALONE_COMMENT
-                # If it opens a new block
-                or current_line.opens_block
-                # If it's a triple quote comment (but not at the start of a funcdef)
+                not is_docstring(current_line.leaves[0])
                 or (
-                    is_docstring(current_line.leaves[0])
-                    and self.previous_line
+                    self.previous_line
                     and self.previous_line.leaves[0]
                     and self.previous_line.leaves[0].parent
                     and not is_funcdef(self.previous_line.leaves[0].parent)
diff --git a/src/black/mode.py b/src/black/mode.py
--- a/src/black/mode.py
+++ b/src/black/mode.py
@@ -191,7 +191,7 @@ class Preview(Enum):
     accept_raw_docstrings = auto()
     fix_power_op_line_length = auto()
     hug_parens_with_braces_and_square_brackets = auto()
-    allow_empty_first_line_before_new_block_or_comment = auto()
+    allow_empty_first_line_in_block = auto()
     single_line_format_skip_with_multiple_comments = auto()
     long_case_block_line_splitting = auto()
     allow_form_feeds = auto()


Test patch:
diff --git a/tests/data/cases/preview_allow_empty_first_line_in_special_cases.py b/tests/data/cases/preview_allow_empty_first_line.py
similarity index 87%
rename from tests/data/cases/preview_allow_empty_first_line_in_special_cases.py
rename to tests/data/cases/preview_allow_empty_first_line.py
--- a/tests/data/cases/preview_allow_empty_first_line_in_special_cases.py
+++ b/tests/data/cases/preview_allow_empty_first_line.py
@@ -51,6 +51,17 @@ def baz():
     if x:
         a = 123
 
+def quux():
+
+    new_line = here
+
+
+class Cls:
+
+    def method(self):
+
+        pass
+
 # output
 
 def foo():
@@ -104,3 +115,14 @@ def baz():
     # OK
     if x:
         a = 123
+
+
+def quux():
+
+    new_line = here
+
+
+class Cls:
+    def method(self):
+
+        pass
diff --git a/tests/data/cases/preview_form_feeds.py b/tests/data/cases/preview_form_feeds.py
--- a/tests/data/cases/preview_form_feeds.py
+++ b/tests/data/cases/preview_form_feeds.py
@@ -198,6 +198,7 @@ def foo():
 
 # form feeds are prohibited inside blocks, or on a line with nonwhitespace
 def bar(a=1, b: bool = False):
+
     pass