Repository: sunpy/sunpy
Pull request #8173: Made dynamically constructed docstrings compatible with all Python versions
Author: login: ayshih
Labels: name: Documentation
description: Affects the documentation.
Created: 2025-05-08T03:37:00Z | Merged: 2025-05-08T14:49:29Z

Python 3.13+ automatically cleans up common indentations in docstrings, which breaks the assumptions behind our `add_common_docstring()` decorator and `fix_duplicate_notes()` function.  This PR fixes them by always removing any common indentations prior to manipulating the docstring.  Along the way, I fixed the wildly wrong argument names and documentation for `fix_duplicate_notes()` and refactored its internals.

For Python 3.12 and earlier, this will technically change a couple of our docstrings when accessed directly via `.__doc__`, but it won't change docstrings as retrieved by `inspect` or as rendered by Sphinx (which uses `inspect`).  Thus, I do not consider this to be a breaking change.

Fixes https://github.com/sunpy/sunpy/issues/8172


Diff:
diff --git a/.rtd-environment.yml b/.rtd-environment.yml
index 74f873f1eb2..8b6aa8b5037 100644
--- a/.rtd-environment.yml
+++ b/.rtd-environment.yml
@@ -2,9 +2,7 @@ name: rtd_sunpy
 channels:
   - conda-forge
 dependencies:
-  # Python 3.13 strips indention in docstrings, which breaks our dynamically edited docstrings
-  # https://github.com/python/cpython/issues/81283
-  - python=3.12
+  - python=3.13
   - pip
   - openjpeg
   - graphviz!=2.42.*,!=2.43.*
diff --git a/changelog/8173.trivial.rst b/changelog/8173.trivial.rst
new file mode 100644
index 00000000000..bc2951fbcd6
--- /dev/null
+++ b/changelog/8173.trivial.rst
@@ -0,0 +1,2 @@
+In order to support multiple versions of Python, the indentation of some docstrings is now different when accessed directly (i.e., via ``__doc__``).
+The docstrings will look exactly the same when viewed in other ways, including in built documentation.
diff --git a/sunpy/coordinates/frames.py b/sunpy/coordinates/frames.py
index 8e1efd51cef..87e48ea856a 100644
--- a/sunpy/coordinates/frames.py
+++ b/sunpy/coordinates/frames.py
@@ -48,64 +48,63 @@ def _frame_parameters():
     """
     ret = {}
 
-    # Each text block is missing the first indent because it already exists in the frame docstring
     ret['data'] = ("data : `~astropy.coordinates.BaseRepresentation` or ``None``\n"
-                   "        A representation object or ``None`` to have no data\n"
-                   "        (or use the coordinate component arguments, see below).")
+                   "    A representation object or ``None`` to have no data\n"
+                   "    (or use the coordinate component arguments, see below).")
     ret['common'] = (f"obstime : {_variables_for_parse_time_docstring()['parse_time_types']}\n"
-                     "        The time of the observation. This is used to determine the\n"
-                     "        position of solar-system bodies (e.g., the Sun and the Earth) as\n"
-                     "        needed to define the origin and orientation of the frame.\n"
-                     "    representation_type : `~astropy.coordinates.BaseRepresentation`, str, optional\n"
-                     "        A representation class or string name of a representation class.\n"
-                     "        This may change the valid coordinate component arguments from the\n"
-                     "        defaults (see above). For example, passing\n"
-                     "        ``representation_type='cartesian'`` will make the frame expect\n"
-                     "        Cartesian coordinate component arguments (typically, ``x``, ``y``,\n"
-                     "        and ``z``).\n"
-                     "    copy : bool, optional\n"
-                     "        If `True` (default), make copies of the input coordinate arrays.")
+                     "    The time of the observation. This is used to determine the\n"
+                     "    position of solar-system bodies (e.g., the Sun and the Earth) as\n"
+                     "    needed to define the origin and orientation of the frame.\n"
+                     "representation_type : `~astropy.coordinates.BaseRepresentation`, str, optional\n"
+                     "    A representation class or string name of a representation class.\n"
+                     "    This may change the valid coordinate component arguments from the\n"
+                     "    defaults (see above). For example, passing\n"
+                     "    ``representation_type='cartesian'`` will make the frame expect\n"
+                     "    Cartesian coordinate component arguments (typically, ``x``, ``y``,\n"
+                     "    and ``z``).\n"
+                     "copy : bool, optional\n"
+                     "    If `True` (default), make copies of the input coordinate arrays.")
     ret['lonlat'] = ("lon : `~astropy.coordinates.Angle` or `~astropy.units.Quantity`, optional\n"
-                     "        The longitude coordinate for this object (``lat`` must also be\n"
-                     "        given and ``data`` must be ``None``).\n"
-                     "        Not needed if ``data`` is given.\n"
-                     "    lat : `~astropy.coordinates.Angle` or `~astropy.units.Quantity`, optional\n"
-                     "        The latitude coordinate for this object (``lon`` must also be\n"
-                     "        given and ``data`` must be ``None``).\n"
-                     "        Not needed if ``data`` is given.")
+                     "    The longitude coordinate for this object (``lat`` must also be\n"
+                     "    given and ``data`` must be ``None``).\n"
+                     "    Not needed if ``data`` is given.\n"
+                     "lat : `~astropy.coordinates.Angle` or `~astropy.units.Quantity`, optional\n"
+                     "    The latitude coordinate for this object (``lon`` must also be\n"
+                     "    given and ``data`` must be ``None``).\n"
+                     "    Not needed if ``data`` is given.")
     ret['radius'] = ("radius : `~astropy.units.Quantity`, optional\n"
-                     "        The radial distance coordinate from Sun center for this object.\n"
-                     "        Defaults to the radius of the Sun. Not needed if ``data`` is given.")
+                     "    The radial distance coordinate from Sun center for this object.\n"
+                     "    Defaults to the radius of the Sun. Not needed if ``data`` is given.")
     ret['distance_sun'] = ("distance : `~astropy.units.Quantity`, optional\n"
-                           "        The distance coordinate from Sun center for this object.\n"
-                           "        Not needed if ``data`` is given.")
+                           "    The distance coordinate from Sun center for this object.\n"
+                           "    Not needed if ``data`` is given.")
     ret['distance_earth'] = ("distance : `~astropy.units.Quantity`, optional\n"
-                             "        The distance coordinate from Earth center for this object.\n"
-                             "        Not needed if ``data`` is given.")
+                             "    The distance coordinate from Earth center for this object.\n"
+                             "    Not needed if ``data`` is given.")
     ret['xyz'] = ("x : `~astropy.units.Quantity`, optional\n"
-                  "        X-axis coordinate for this object. Not needed if ``data`` is given.\n"
-                  "    y : `~astropy.units.Quantity`, optional\n"
-                  "        Y-axis coordinate for this object. Not needed if ``data`` is given.\n"
-                  "    z : `~astropy.units.Quantity`, optional\n"
-                  "        Z-axis coordinate for this object. Not needed if ``data`` is given.")
+                  "    X-axis coordinate for this object. Not needed if ``data`` is given.\n"
+                  "y : `~astropy.units.Quantity`, optional\n"
+                  "    Y-axis coordinate for this object. Not needed if ``data`` is given.\n"
+                  "z : `~astropy.units.Quantity`, optional\n"
+                  "    Z-axis coordinate for this object. Not needed if ``data`` is given.")
     ret['observer'] = ("observer : `~sunpy.coordinates.frames.HeliographicStonyhurst`, str\n"
-                       "        The location of the observer. If a string is provided,\n"
-                       "        it must be a solar system body that can be parsed by\n"
-                       "        `~sunpy.coordinates.ephemeris.get_body_heliographic_stonyhurst`\n"
-                       "        at the time ``obstime``. Defaults to Earth center.")
+                       "    The location of the observer. If a string is provided,\n"
+                       "    it must be a solar system body that can be parsed by\n"
+                       "    `~sunpy.coordinates.ephemeris.get_body_heliographic_stonyhurst`\n"
+                       "    at the time ``obstime``. Defaults to Earth center.")
     ret['rsun'] = ("rsun : `~astropy.units.Quantity`\n"
-                   "        The radius of the Sun in length units. Used to convert a 2D\n"
-                   "        coordinate (i.e., no ``radius`` component) to a 3D coordinate by\n"
-                   "        assuming that the coordinate is on the surface of the Sun. Defaults\n"
-                   "        to the photospheric radius as defined in `sunpy.sun.constants`.")
+                   "    The radius of the Sun in length units. Used to convert a 2D\n"
+                   "    coordinate (i.e., no ``radius`` component) to a 3D coordinate by\n"
+                   "    assuming that the coordinate is on the surface of the Sun. Defaults\n"
+                   "    to the photospheric radius as defined in `sunpy.sun.constants`.")
     ret['equinox'] = (f"equinox : {_variables_for_parse_time_docstring()['parse_time_types']}\n"
-                      "        The date for the mean vernal equinox.\n"
-                      "        Defaults to the J2000.0 equinox.")
+                      "    The date for the mean vernal equinox.\n"
+                      "    Defaults to the J2000.0 equinox.")
     ret['magnetic_model'] = ("magnetic_model : `str`\n"
-                             "        The IGRF model to use for determining the orientation of\n"
-                             "        Earth's magnetic dipole pole. The supported options are\n"
-                             "        ``'igrf13'`` (default), ``'igrf12'``, ``'igrf11'``, and\n"
-                             "        ``'igrf10'``.")
+                             "    The IGRF model to use for determining the orientation of\n"
+                             "    Earth's magnetic dipole pole. The supported options are\n"
+                             "    ``'igrf13'`` (default), ``'igrf12'``, ``'igrf11'``, and\n"
+                             "    ``'igrf10'``.")
     ret['igrf_reference'] = ("* `International Geomagnetic Reference Field (IGRF) "
                              "<https://www.ngdc.noaa.gov/IAGA/vmod/igrf.html>`__")
 
diff --git a/sunpy/map/tests/test_mapbase.py b/sunpy/map/tests/test_mapbase.py
index d7e6f27afa6..2d1da9e443a 100644
--- a/sunpy/map/tests/test_mapbase.py
+++ b/sunpy/map/tests/test_mapbase.py
@@ -37,101 +37,9 @@
 from sunpy.util import SunpyUserWarning
 from sunpy.util.exceptions import SunpyDeprecationWarning, SunpyMetadataWarning
 from sunpy.util.metadata import ModifiedItem
-from sunpy.util.util import fix_duplicate_notes
 from .strategies import matrix_meta
 
 
-def test_notes_combined():
-    map_documentation = """
-    Class Info.
-
-    Notes
-    -----
-    This is a note.
-
-    References
-    ----------
-    This is reference.
-    """
-    extra_note_section= """\nNotes\n-----\nThis should be combined."""
-    updated_documentation= fix_duplicate_notes(extra_note_section, map_documentation)
-    expected_result = """
-    Class Info.
-
-    Notes
-    -----
-    This is a note.
-
-    This should be combined.
-
-    References
-    ----------
-    This is reference.
-    """
-    assert updated_documentation == expected_result
-
-def test_notes_combined_no_references():
-    map_documentation = """
-    Class Info.
-
-    Notes
-    -----
-    This is a note.
-    """
-    extra_note_section= """\nNotes\n-----\nThis should be combined."""
-    updated_documentation= fix_duplicate_notes(extra_note_section, map_documentation)
-    updated_documentation2=updated_documentation.replace("\n    \n    ","\n\n    ")
-    expected_result = """
-    Class Info.
-
-    Notes
-    -----
-    This is a note.
-
-    This should be combined.
-    """
-    assert updated_documentation2.strip() == expected_result.strip()
-
-def test_notes_combined_no_existing_notes():
-    map_documentation = """
-    Class Info.
-
-    References
-    ----------
-    This is reference.
-    """
-    extra_note_section= """\nNotes\n-----\nThis should be combined."""
-    updated_documentation= fix_duplicate_notes(extra_note_section, map_documentation)
-    expected_result = """
-    Class Info.
-
-    Notes
-    -----
-    This should be combined.
-
-    References
-    ----------
-    This is reference.
-    """
-    assert updated_documentation == expected_result
-
-def test_notes_combined_no_notes_no_references():
-    map_documentation = """
-    Class Info.
-    """
-    extra_note_section= """\nNotes\n-----\nThis should be combined."""
-    updated_documentation= fix_duplicate_notes(extra_note_section, map_documentation)
-    updated_documentation2=updated_documentation.replace("\n    \n    ","\n\n    ")
-    expected_result = """
-    Class Info.
-
-    Notes
-    -----
-    This should be combined.
-    """
-    assert updated_documentation2.strip() == expected_result.strip()
-
-
 def test_fits_data_comparison(aia171_test_map):
     """Make sure the data is the same when read with astropy.io.fits and sunpy"""
     with pytest.warns(VerifyWarning, match="Invalid 'BLANK' keyword in header."):
diff --git a/sunpy/util/decorators.py b/sunpy/util/decorators.py
index 6bda712418f..9cfb07617a0 100644
--- a/sunpy/util/decorators.py
+++ b/sunpy/util/decorators.py
@@ -1,6 +1,7 @@
 """
 This module provides sunpy specific decorators.
 """
+from inspect import cleandoc
 from functools import wraps
 from contextlib import contextmanager
 
@@ -301,5 +302,6 @@ def __call__(self, func):
         if self.prepend and isinstance(func.__doc__, str):
             func.__doc__ = self.prepend + func.__doc__
         if self.kwargs:
+            func.__doc__ = cleandoc(func.__doc__)  # not necessary on Python 3.13+
             func.__doc__ = func.__doc__.format(**self.kwargs)
         return func
diff --git a/sunpy/util/tests/test_util.py b/sunpy/util/tests/test_util.py
index 5b70b6bcb3c..8d37a45f293 100644
--- a/sunpy/util/tests/test_util.py
+++ b/sunpy/util/tests/test_util.py
@@ -1,3 +1,5 @@
+from inspect import cleandoc
+
 import pytest
 
 from astropy.io import fits
@@ -121,3 +123,95 @@ def set_test6_n(self, *args, **kwargs):
             pass
 
     assert util.get_set_methods(A()) == {'test2', 'test6_n'}
+
+
+def test_notes_combined():
+    original_documentation = """
+    Class Info.
+
+    Notes
+    -----
+    This is a note.
+
+    References
+    ----------
+    This is reference.
+    """
+    extra_note_section = """\nNotes\n-----\nThis should be combined."""
+    updated_documentation = util.fix_duplicate_notes(extra_note_section, original_documentation)
+    expected_result = """
+    Class Info.
+
+    Notes
+    -----
+    This is a note.
+
+    This should be combined.
+
+    References
+    ----------
+    This is reference.
+    """
+    assert updated_documentation == cleandoc(expected_result)
+
+
+def test_notes_combined_no_references():
+    original_documentation = """
+    Class Info.
+
+    Notes
+    -----
+    This is a note.
+    """
+    extra_note_section = """\nNotes\n-----\nThis should be combined."""
+    updated_documentation = util.fix_duplicate_notes(extra_note_section, original_documentation)
+    expected_result = """
+    Class Info.
+
+    Notes
+    -----
+    This is a note.
+
+    This should be combined.
+    """
+    assert updated_documentation == cleandoc(expected_result)
+
+
+def test_notes_combined_no_existing_notes():
+    original_documentation = """
+    Class Info.
+
+    References
+    ----------
+    This is reference.
+    """
+    extra_note_section= """\nNotes\n-----\nThis should be combined."""
+    updated_documentation = util.fix_duplicate_notes(extra_note_section, original_documentation)
+    expected_result = """
+    Class Info.
+
+    Notes
+    -----
+    This should be combined.
+
+    References
+    ----------
+    This is reference.
+    """
+    assert updated_documentation == cleandoc(expected_result)
+
+
+def test_notes_combined_no_notes_no_references():
+    original_documentation = """
+    Class Info.
+    """
+    extra_note_section= """\nNotes\n-----\nThis should be combined."""
+    updated_documentation= util.fix_duplicate_notes(extra_note_section, original_documentation)
+    expected_result = """
+    Class Info.
+
+    Notes
+    -----
+    This should be combined.
+    """
+    assert updated_documentation == cleandoc(expected_result)
diff --git a/sunpy/util/util.py b/sunpy/util/util.py
index d469453a5b1..11c56c2050b 100644
--- a/sunpy/util/util.py
+++ b/sunpy/util/util.py
@@ -4,7 +4,6 @@
 import os
 import hashlib
 import inspect
-import textwrap
 from io import BytesIO
 from base64 import b64encode
 from shutil import get_terminal_size
@@ -284,38 +283,37 @@ def _figure_to_base64(fig):
     return b64encode(buf.getvalue()).decode('utf-8')
 
 
-def fix_duplicate_notes(subclass_doc, cls_doc):
+def fix_duplicate_notes(notes_to_add, docstring):
     """
-    Returns a new documentation string such that there are notes section duplication in in `~sunpy.map.Map` subclasses.
+    Merges a note section into a docstring that may have a notes section.
 
     Parameters
     ----------
-    subclass_doc : str
-        The documentation that needs to be appended.
-    cls_doc
-        The original class's documentation.
+    notes_to_add : str
+        The notes that need to be added (starting with ``"Notes"``).
+    docstring
+        The original docstring.
 
     Returns
     -------
     str
-        Updated documentation that contains no note section duplication.
+        Updated docstring with a single notes section.
     """
-    existing_notes_pos = cls_doc.find('Notes\n    -----')
-    subclass_notes_pos = subclass_doc.find('Notes\n-----')
-    subclass_notes_data = textwrap.indent(subclass_doc[subclass_notes_pos + len('Notes\n-----'):].strip(), "    ")
-    references_pattern = "References\n    ----------"
-    examples_pattern = "Examples\n   -------"
-    start_index = cls_doc.find(references_pattern if references_pattern in cls_doc else examples_pattern)
-    if start_index!=-1:
-        next_pattern_pos = min(pos for pos in [cls_doc.find(references_pattern, start_index), cls_doc.find(examples_pattern, start_index)] if pos != -1)
-        other_patterns = cls_doc[:next_pattern_pos]
-        if existing_notes_pos!=-1:
-            cls_doc = other_patterns + subclass_notes_data.lstrip() + '\n\n    ' + cls_doc[next_pattern_pos:]
-        else:
-            cls_doc = other_patterns + 'Notes\n    -----\n' + subclass_notes_data + '\n\n    ' + cls_doc[next_pattern_pos:]
-    elif existing_notes_pos != -1:
-        cls_doc +="\n"+subclass_notes_data
-    else:
-        cls_doc += textwrap.indent(subclass_doc, "    ")
-
-    return cls_doc
+    docstring = inspect.cleandoc(docstring)  # not necessary on Python 3.13+
+    existing_notes_pos = docstring.find('Notes\n-----')
+    new_notes_pos = notes_to_add.find('Notes\n-----')
+    if new_notes_pos == -1:
+        raise RuntimeError("The notes to add does not appear to be a 'Notes' section.")
+    notes_to_add_data = notes_to_add[new_notes_pos + len('Notes\n-----'):].strip()
+
+    # Insert the notes before either numpydoc section that can come after 'Notes', otherwise at end
+    references_pos = docstring.find("References\n----------")
+    examples_pos = docstring.find("Examples\n--------")
+    index = min({len(docstring), references_pos, examples_pos} - {-1})
+
+    pieces = [docstring[:index],
+              "\n\n" if index == len(docstring) else "",
+              "Notes\n-----\n" if existing_notes_pos == -1 else "",
+              f"{notes_to_add_data}",
+              f"\n\n{docstring[index:]}" if len(docstring) > index else ""]
+    return "".join(pieces)


Reviews:
user: login: nabobalis
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-08T05:31:42Z
user: login: Cadair
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-08T14:08:16Z
user: login: ayshih
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T14:10:22Z
user: login: ayshih
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T14:16:52Z

Review comments:
diff_hunk: @@ -121,3 +121,88 @@ def set_test6_n(self, *args, **kwargs):
             pass
 
     assert util.get_set_methods(A()) == {'test2', 'test6_n'}
+
+
+def test_notes_combined():
+    original_documentation = """
path: sunpy/util/tests/test_util.py
user: login: Cadair
body: eugh I hate it when things are dedented like this, would using `textwrap.dedent` defeat the object of the exercise?
created_at: 2025-05-08T14:07:35Z
author_association: MEMBER
diff_hunk: @@ -121,3 +121,88 @@ def set_test6_n(self, *args, **kwargs):
             pass
 
     assert util.get_set_methods(A()) == {'test2', 'test6_n'}
+
+
+def test_notes_combined():
+    original_documentation = """
path: sunpy/util/tests/test_util.py
user: login: ayshih
body: That's fair.  It's ugly as hell.  I'll fix it.
created_at: 2025-05-08T14:10:21Z
author_association: MEMBER
diff_hunk: @@ -121,3 +121,88 @@ def set_test6_n(self, *args, **kwargs):
             pass
 
     assert util.get_set_methods(A()) == {'test2', 'test6_n'}
+
+
+def test_notes_combined():
+    original_documentation = """
path: sunpy/util/tests/test_util.py
user: login: ayshih
body: Done (using `inspect.cleandoc()`)
created_at: 2025-05-08T14:16:52Z
author_association: MEMBER

PR comments:
user: login: ayshih
created_at: 2025-05-08T13:13:16Z
body: Docs build confirmed to succeed on both Python 3.12 and 3.13 without warnings/errors associated with the docstring indentations.

I believe the just-merged #8165 means that the backports here will have to be manually done
author_association: MEMBER
user: login: lumberbot-app[bot]
created_at: 2025-05-08T14:49:38Z
body: Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 6.0
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 15e8f183cf4570f7b27dad11948aee1a24af3f88
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #8173: Made dynamically constructed docstrings compatible with all Python versions'
```

4. Push to a named branch:

```
git push YOURFORK 6.0:auto-backport-of-pr-8173-on-6.0
```

5. Create a PR against branch 6.0, I would have named this PR:

> "Backport PR #8173 on branch 6.0 (Made dynamically constructed docstrings compatible with all Python versions)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
author_association: NONE
user: login: lumberbot-app[bot]
created_at: 2025-05-08T14:49:45Z
body: Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 6.1
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 15e8f183cf4570f7b27dad11948aee1a24af3f88
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #8173: Made dynamically constructed docstrings compatible with all Python versions'
```

4. Push to a named branch:

```
git push YOURFORK 6.1:auto-backport-of-pr-8173-on-6.1
```

5. Create a PR against branch 6.1, I would have named this PR:

> "Backport PR #8173 on branch 6.1 (Made dynamically constructed docstrings compatible with all Python versions)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
author_association: NONE