Repository: psf/black

Issue:
Newlines after code block open, once and for all?
Background:
- https://github.com/psf/black/issues/902
  Original issue, with some follow-up comments post merge
- https://github.com/psf/black/pull/3035
  Original implementation
- https://github.com/psf/black/issues/3551
  Follow-up issue
- https://github.com/psf/black/pull/3967 
  Recent preview change partially walking things back
- https://github.com/psf/black/issues/619#issuecomment-1792996633
  Discussion about PEP 8

(there's discussion elsewhere too, but I think these are the most important links)

---

tldr; the 2023 stable style strips newlines after code block opens. This has proven controversial (and was most of the change from 22 to 23 by volume). Recently, we walked this back a little bit in preview style, but we should try to get it right for 24.

The main argument against is it can hurt readability, see https://github.com/psf/black/issues/902#issuecomment-1425713025 . I'm quite sympathetic to this. For instance, functions with a complicated return type and first statement can get quite dense. Similarly, there are examples with comments in a new code block where things can get too dense.

An auxiliary argument against it is that stripping new lines between `class` and first method is in violation of a close reading and changelog analysis of PEP 8, see https://github.com/psf/black/issues/619#issuecomment-1792996633. I don't care much for PEP 8 absolutism (and certainly not at the expense of churning established behaviour), but I consider it a nice-to-have to not reformat away from it.

Recently in https://github.com/psf/black/pull/3967, Henri contributed a nice change that walks this back in some cases, involving comments or compound statements. However, I'm not sure this is enough. [Here are two examples from the first few pages of Black 22 -> 23 diff in my work codebase where I feel readability really suffers](https://black.vercel.app/?version=stable&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4ARDAb9dAD2IimZxl1N_WlbvK5V-fnaWIwL5TYBadve5dORVlMpn2CM7Ex6036-fmBvZZxxU7M4SzueDveXQyyVCyVj-_SI8G5MhyXK3nNLHUmR1qpfo3F4ciXU1BMotQs0XgWgkBVKnr_rKC7aIr11sgQsmB5Vap7pzA0R4M5007ghia_366U1-hEjwpaB66-Lkb9ZsEQGd_I3HS8XCUyn1O9pe47U_JM9qrygevkU8y8MHi42r1FAGT-KAUg8Rw7JMVMPhspr1cpS3BzKbGToLSkpjYcCJCj70GUsfc-SKPPQf6n9nHSnwHTurwtoO8zChj8UkwEuEK_PzkD2G_pjTZTi2pwy8tqqwW4_Sg7NqddQaPn1JDfnMvajayo16d3g90jDv7alCLxL4CghJEpl5sU2gaWINat_wzfhsHjDOd77doe6GceCBcWuklNcdz1KAT8N7e_h68QFQcENH51Z8nzi6Nsk0ViOJ7WcYti9iYfIqrura2wKKSmLKz1FVErFWWojvd07WdSB1M2nGrsld_aGwOC7R7wb84ks5wpovY1jSZdnLj1QgrWWAB8ns3Zvi7HnLsmBGxIFV2EtqyAyKkQswAAAAs1goWJRFiskAAdsDxAgAAAhqMAuxxGf7AgAAAAAEWVo=) (scrolling through I see many others). Henri's change won't save us here. And fundamentally this code is quite simple, it just has slightly long type and variable names, i.e. it's not the kind of code where there should have been a comment or something.

---

Jelle nudged me to think about this, and I now feel we should just walk this change back entirely. We should allow users to have either 0 or 1 blank newlines. This is also simpler and more explainable than the partially walked back logic.

I am fine with continuing to strip blank newlines between a function declaration and its docstring (and preview does this for classes as well). We currently enforce one blank newline after class and module docstrings. We don't have an opinion on blank newlines after function docstrings (see https://github.com/psf/black/issues/2370). I think this is probably fine as-is and not having an opinion would be consistent with the proposed no docstring case. (I'll grant that it feels easier to come up with a good opinion for the post docstring blank line in functions. At the very least, we certainly should not resolve that issue by always removing blank lines).

This will not create any churn going from 23 -> 24 (in fact, most of this change technically even meets our stability policy (although any change here should certainly only be made in preview style)).

---

As a meta thing, as a relatively new maintainer of Black, I'm still unsure of my decision making framework. I feel the original change had at most marginal benefit, especially given its churn (I've never browsed code and been like ugh one extra blank line, nor has any code reviewer or linter ever yelled at me for it. These are the things Black usually saves me from. All the motivating changes I found have multiple blank newlines or are toy code). On the other hand, it's very costly to revert changes in stable style. The only thing that makes it possible in this case is that both unreverted and reverted formatting will be valid and inconsistency in this specific case is not costly.
One space between class definition and the first inner method
Can Black have a rule to **not remove** the whitespace at line `2` from here:
```python
class C:
    
    def __init__(self):
        pass
```
so that it will keep the blank line between class definition and any first inner entity (as it does already with the docstrings and assignment attributes)?

It's extremely ugly to have that whitespace consistent in the entire codebase and just in this particular situation, where you don't have anything to add between class definition and the first inner method (which usually is the `__init__` one), to end up with "glued" headers of class and method definition.

Now if I'm formatting this:
```python
class C:
    
    def __init__(self):
        pass
    def func(self):
        pass
```
I get this:
```python
class C:
    def __init__(self):
        pass

    def func(self):
        pass
```
So I expect the rule of _one empty line before and after class methods_ to be consistent here. Everyone likes to have symmetry in their code.


Discussion:
I would support a blank line after the definition of functions without a docstring. I find the two examples provided convincing in that regard.

---

> I am fine with continuing to strip blank newlines between a function declaration and its docstring

I think this behavior should be kept, too. As I've said in [#2370 (comment)](https://github.com/psf/black/issues/2370#issuecomment-1487068337), having a blank line after a docstring would be in violation with all of the major Python docstring conventions (PEP 257, numpydoc and Google). Pydocstyle files "No blank lines allowed after function docstring" under error code D202. D202 is included in all the docstring conventions mentioned above (source: http://www.pydocstyle.org/en/stable/error_codes.html#default-conventions).
My guess would be that it's not uncommon for projects to use both pydocstyle and black and in that case the two should ideally not be in conflict with each other.
OK, let's walk back the change.

@mcnoat reading your comment, there might be confusion between blank lines before and after docstrings? Given this function:

```python
def f():
    
    """
    doc
    """
    
    hi
```

Currently we remove the blank line before the docstring (and Shantanu says we should keep doing that; I agree), but not the one after the docstring. Possibly we should remove the one after the docstring, but that's a discussion for #2370.
> As a meta thing, as a relatively new maintainer of Black, I'm still unsure of my decision making framework.

I feel like I owe you a response to this, but I'm not sure I have a good one :) Some of the things I look at are:

- How disruptive is this change? I generally want to make changes to the stable style only if they feel like clear improvements; when people upgrade Black and see the changes, they should feel it made things better.
- Do I like the change? That's obviously subjective, but I feel part of the point of Black is that instead of the entire Python ecosystem arguing over style decisions, it's just us on this tracker who have to do that.
- Does it feel like the community supports the change? For example, if there are lots of people who comment/like comments in favor of a change, that's a good signal.
- Does the change feel consistent with Black's overall style? Here's a couple of principles that I feel we generally follow, though some may be controversial:
   - No backslashes for formatting, ever
   - Indentation should always be by a multiple of four
   - Formatting should generally not depend on previous formatting, though with obvious exceptions like the magic trailing comma
   - Diffs should be easy to read
   - Fewer lines is better than more

In this particular case, I think a few of us simply misjudged how controversial this change would be. Personally I liked most of the changes Black 22 -> 23 made to our work codebase; many looked like this:

```
 def _source_query_sql_fn(dt: str) -> str:
-
     sql = f"""
         SELECT DISTINCT
```

Where the extra newline just felt like a waste of space.

Hopefully to prevent a repeat of this experience, we can get more eyes on the current preview style (thanks for opening an issue for that!) before promoting it to stable.
> @mcnoat reading your comment, there might be confusion between blank lines before and after docstrings? Given this function:

You are absolutely correct, thanks for pointing it out. Since [hauntsaninja](https://github.com/hauntsaninja) left a comment on #2370, I presumed that this issue was about the same thing and - despite honest effort - did not read his comment or the original post here close enough.
This is great news. Thanks guys.

One thing though: #4060 does not fix #619. That pre-#3035 behavior will now remain the odd man out.

Yes, we still remove the first blank line here:

```
class Cls:

    def method(self):

        pass
```

That seems fine to me.
Right, I had gotten the impression that addressing the `def` after `class` case was part of the plan, as the initial comment specifically referenced #619.

And from [this comment](https://github.com/psf/black/issues/3551#issuecomment-1749856113) I understood that the inconsistency bothered @JelleZijlstra as well.

> Yes, we still remove the first blank line here:
> 
> ```
> class Cls:
> 
>     def method(self):
> 
>         pass
> ```
> 

So we get:

```python
class Cls:    
	def method(self):

		pass
```

- From a typographic perspective this doesn’t look logical or visually satisfying.
- This goes against PEP 8 as explained in [#619](https://github.com/psf/black/issues/619).
- Were the first inner statement a class variable instead of `def`, the empty line would not be removed.
- Empty lines preceding subsequent `def` statements are not removed (in fact, they are enforced).

I’m struggling to understand the motivation for special-casing a `class`-adjacent `def` and treating it differently than everything else. This certainly _feels like_ a bug even if it’s planned.

What a coincidence, I used `black` for the first time a few minutes ago and I had the same remark. I didn't bother opening an issue though because I thought it has been suggested before but refused for some reason?

PEP8 states that:

> Method definitions inside a class are surrounded by a single blank line.

So I guess this applies for the first method too, including `__init__()`.
Just tested how black would format [Robot Framework source code](https://github.com/robotframework/robotframework) and this was the biggest problem. I found it important to surround methods with an empty line and think it should be done consistently. I understand that the motivation behind black is remove the need to discuss about different formatting alternatives, but in this particular I think black just behaves wrong. As already pointed above, PEP-8 explicitly states that

> "Method definitions inside a class are surrounded by a single blank line."

In my opinion the PEP-8 compatible behavior should be the default. There could be a command line option to omit the empty line to avoid changes with existing blackened code bases. Having an option to preserve/add it would be the minimum.
I think black’s behaviour make sense for classes that start with a docstring or comment.  In this case, the empty line is wanted:

```python
class C:
    """Abstraction for B to do spam."""

    def __init__(self):
        ...
````
@merwok I totally agree there should be an empty line after the docstring. I think the docstring PEP also explicitly says so. This issue is about black _removing_ the empty line if there's no docstring, even though that's against PEP-8. Without the docstring, black would format you example like this:

```python
class C:
    def __init__(self):
        ...
```
This is interesting. While your reading of PEP 8 is correct to the letter, I'm not quite sure if it was intended by the PEP 8 authors. Case in point: the PEP also says the following:
> Surround top-level function and class definitions with two blank lines.

Following your logic, if a Python file starts with a function or a class definition, we would have to put two blank lines before it, no?

I'd like to understand your need for this blank line better. What purpose does it serve? There's clear delineation due to the increased indentation. In fact, I'm leaning towards implementing #450 so that we are consistent in that regard.
> if a Python file starts with a function or a class definition, we would have to put two blank lines before it

pep8 or flake8 used to require that and it was annoying!
FWIW my take on the question here:

```python
class CoolRegularClass:
    """Nice docstring.

    More info.
    Cool.
    """

    def __init__(self):
        # nice to have one blank after the docstring
        ...


class SomeInternalClass:
    def __init__(self):
        # no docstring, no needless blank line
        ...
```

In other words: the `class` statement is directly followed by the first interesting thing (docstring, attribute, method), then usual blank lines conventions applies.
Some reasons why I prefer an empty line before the first method in a class:
- There are empty lines above other methods as well. I like consistency in this regard.
- PEP-8 says so. There are likely lot of projects following that rule and Black forcing other style causes unnecessary code churn (or just prevents taking the tool into use).
- I find it odd if docstrings or class attributes change this behaviour.
- For my eye it just looks better.

I guess this is something where opinions differ and in my opinion stuff like this should be configurable. I know Black tries to be uncompromising, but there's already similar `--skip-string-normalization` option. It would be awesome if projects could have their own `black.toml` for options like this. Black could set the overall rules and let project decide on nuances like this. Black doing all formatting automatically would give you its normal benefits within a project. Two projects not having exactly same styles shouldn't be a big problem.
Reasons for blank line, IMHO:
* I interpret PEP 8's:
  > Method definitions inside a class are surrounded by a single blank line.
  
  ...to include the first method.

* Reduces diff size by one when adding/removing a docstring or class attribute(s), i.e. no blank-line insertion/removal churn.
* Would make consistent with current behavior for leading class attributes or function calls. Currently, these are left as is:
  ```python
  class LeadingAttributeAfterBlankLine:

      a = 1
  ```
  ```python
  class LeadingFunctionCallAfterBlankLine:

      print("defining")
  ```
  ...but these classes with a leading method/class have the blank line removed:
  ```python
  class LeadingMethodAfterBlankLine:

      def something(self):
          pass
  ```
  ```python
  class InnerClassAfterBlankLine:

      class Inner:
          pass
  ```
  If going to remove blank line, then I'd probably prefer the opposite in these cases, i.e. blank line removed for leading class attributes and kept for method/class blocks.

Instead of forcing blank line or not, what if it were just left alone? I.e. allow zero or one blank line, as is currently done for the `LeadingAttributeAfterBlankLine` example.
I'd prefer our current formatting: [playground](https://black.vercel.app/?version=main&state=_Td6WFoAAATm1rRGAgAhARYAAAB0L-Wj4AG4APBdAD2IimZxl1N_WlXnON2nzNKQkw9URT6V0G1AmdTrd0aaIKGnbyxJwRyj96Y2sgnFk8Yd5zWyGyK50sf0jNjTsnTImGkHERHLqTtW-i54fZN7K8EmUAYM3DP_1LqtP4E00cd8c8-_kQgn3wN_4VeyO5VE43OkhSfh2G79HdiO67x2aZQA77K-5Csi90H56MmlSYFCLSibu8EMLAB23eOjQk-WlIHsxcXh2zm0lUzAlrRZqszerbv61PGmFaEdi5RfLeQonJ16MDYG8ZRkOXrToyNZE0itGuXsvyb0vWsBPSbKFXab67n8nJ1jVoqFyjoTAAAesN4BpFi0uwABjAK5AwAAV-zbsrHEZ_sCAAAAAARZWg==). Which is no empty line before functions, but one before other body statements. After discussing with others as well, we'll not be making any changes here.
https://discord.com/channels/267624335836053506/846434317021741086/937021595933884487
What is the private discord link for?
It's a link to a public channel where we discuss development-related things.
yeah my bad for not providing context, we are partnered with Python Discord (https://discord.gg/python) so we have our own `#black-formatter` text channel there for runtime communication.
It's unfortunate you don't want to make this configurable for projects like you've done at least with quotes. Being able to configure should there be empty lines or not, or at least having an option to just leave them as-is, would have been great. I like the idea of uniform formatting a lot, but I consider it a lot more important within a project than with all different Python projects. Having some things carved to stone is fine as well, but at least with behavior where PEP-8 is ambiguous configuration would make sense. Anyway, it's your project and your rules. For me this just means I won't be using Black.
This unfortunate rule seems to be based on a flawed reading of PEP 8.

PEP 8 states:

> Method definitions inside a class are surrounded by a single blank line.

To me, this directive seems pretty unambiguous, but the maintainers have interpreted it as:

> Consecutive method definitions are separated by a single blank line.

This, of course, is not the same thing. It turns out that the directive actually used to say “separated”, but in 2015 it was specifically changed to “surrounded”. See https://hg.python.org/peps/rev/5784.

Thus, the current wording appears to be quite deliberate and, in all likelihood, is intended to apply to the first method definition as well. I mean, why wouldn’t it be?

> [...] if a Python file starts with a function or a class definition, we would have to put two blank lines before it, no?

This argument is disingenuous. The purpose of vertical whitespace is to separate elements of code from one another. The rules governing it are not meant to apply at either end of a file.

I have read the prior discussion, and perhaps part of the problem is that the examples considered tend to be stubs and not representative of real code.

```
class DummyHelperClass:
    pass
```

Yes, I would agree that inserting a blank line there would be silly and ugly. But as soon as the class body grows to contain methods or other blocks that are themselves separated by blank lines, the leading blank line also becomes essential.

I hope that the above historical finding lends enough support to the literal interpretation of the PEP 8 directive in question that the decision to reject this issue can be reversed. Compliance with PEP 8 would seem to require that blank lines between adjoining `class` and `def` statements should not only be allowed, they should be *mandated*. If that is unpalatable, then authors should at the very least be given discretion to follow their chosen convention.

This whole thing popped up on my radar because the rule discussed here is now being used as an argument in favor of preserving PR #3035, which has proved a controversial and disruptive change.



Resolving patch:
diff --git a/src/black/lines.py b/src/black/lines.py
--- a/src/black/lines.py
+++ b/src/black/lines.py
@@ -745,7 +745,10 @@ def _maybe_empty_lines_for_class_or_def(  # noqa: C901
         if self.previous_line.depth < current_line.depth and (
             self.previous_line.is_class or self.previous_line.is_def
         ):
-            return 0, 0
+            if self.mode.is_pyi or not Preview.allow_empty_first_line_in_block:
+                return 0, 0
+            else:
+                return 1 if user_had_newline else 0, 0
 
         comment_to_add_newlines: Optional[LinesBlock] = None
         if (


Test patch:
diff --git a/tests/data/cases/class_blank_parentheses.py b/tests/data/cases/class_blank_parentheses.py
--- a/tests/data/cases/class_blank_parentheses.py
+++ b/tests/data/cases/class_blank_parentheses.py
@@ -39,6 +39,7 @@ def test_func(self):
 
 
 class ClassWithEmptyFunc(object):
+
     def func_with_blank_parentheses():
         return 5
 
diff --git a/tests/data/cases/preview_allow_empty_first_line.py b/tests/data/cases/preview_allow_empty_first_line.py
--- a/tests/data/cases/preview_allow_empty_first_line.py
+++ b/tests/data/cases/preview_allow_empty_first_line.py
@@ -62,6 +62,15 @@ def method(self):
 
         pass
 
+
+def top_level(
+    a: int,
+    b: str,
+) -> Whatever[Generic, Something]:
+
+    def nested(x: int) -> int:
+        pass
+
 # output
 
 def foo():
@@ -123,6 +132,16 @@ def quux():
 
 
 class Cls:
+
     def method(self):
 
         pass
+
+
+def top_level(
+    a: int,
+    b: str,
+) -> Whatever[Generic, Something]:
+
+    def nested(x: int) -> int:
+        pass
diff --git a/tests/data/cases/preview_form_feeds.py b/tests/data/cases/preview_form_feeds.py
--- a/tests/data/cases/preview_form_feeds.py
+++ b/tests/data/cases/preview_form_feeds.py
@@ -203,6 +203,7 @@ def bar(a=1, b: bool = False):
 
 
 class Baz:
+
     def __init__(self):
         pass