Repository: pandas-dev/pandas

Issue:
DOC: Validate the order of the sections in the docstrings
`scripts/validate_docstrings.py` validates that the content of a docstring follows our standards. There are still some of those standards that the script does not validate, and this gives the users the wrong impression that the docstring is all right, when it's not really the case. All the missing validations are listed in #20298.

In the docstrings, the order of the sections should be:
- Short summary
- Summary
- Parameters
- Returns or Yields
- See Also
- Notes
- Examples

This is described in:
- Our documentation: https://pandas.pydata.org/pandas-docs/stable/contributing_docstring.html
- The numpydoc standard we follow: https://numpydoc.readthedocs.io/en/latest/format.html#docstring-standard

For this issue is required:
- Change `scripts/validate_docstings.py`
- Add tests in `scripts/tests/test_validate_docstrings.py`
- Run the script for all docstrings (i.e. `./scripts/validate_docstings.py`) and check in the resulting report how many of the docstrings in `pandas` have the sections in the wrong order (add that information to this issue)
- Test manually that running the validation for one of the wrong cases, produces an output that is useful for the users (the script can be run for one docstring using for example `./scripts/validate_docstings.py pandas.DataFrame.head`, and the output is a user-friendly explanation of the errors).


Discussion:
@datapythonista Assuming this is not taken - ill work on this  - any other similar ones I can work on - looking to hit my target of 5 PRs this month for the Hacktoberfest t-shirt :) 
Shouldn't be taken, thanks. Everything is in the issue tracker.

Resolving patch:
diff --git a/scripts/validate_docstrings.py b/scripts/validate_docstrings.py
--- a/scripts/validate_docstrings.py
+++ b/scripts/validate_docstrings.py
@@ -56,6 +56,9 @@
 
 PRIVATE_CLASSES = ['NDFrame', 'IndexOpsMixin']
 DIRECTIVES = ['versionadded', 'versionchanged', 'deprecated']
+ALLOWED_SECTIONS = ['Parameters', 'Attributes', 'Methods', 'Returns', 'Yields',
+                    'Other Parameters', 'Raises', 'Warns', 'See Also', 'Notes',
+                    'References', 'Examples']
 ERROR_MSGS = {
     'GL01': 'Docstring text (summary) should start in the line immediately '
             'after the opening quotes (not in the same line, or leaving a '
@@ -69,6 +72,10 @@
             'mentioned in public docstrings',
     'GL05': 'Tabs found at the start of line "{line_with_tabs}", please use '
             'whitespace only',
+    'GL06': 'Found unknown section "{section}". Allowed sections are: '
+            '{allowed_sections}',
+    'GL07': 'Wrong order of sections. "{wrong_section}" should be located '
+            'before "{goes_before}", the right order is: {sorted_sections}',
     'SS01': 'No summary found (a short summary in a single line should be '
             'present at the beginning of the docstring)',
     'SS02': 'Summary does not start with a capital letter',
@@ -353,6 +360,18 @@ def double_blank_lines(self):
             prev = row.strip()
         return False
 
+    @property
+    def section_titles(self):
+        sections = []
+        self.doc._doc.reset()
+        while not self.doc._doc.eof():
+            content = self.doc._read_to_next_section()
+            if (len(content) > 1
+                    and len(content[0]) == len(content[1])
+                    and set(content[1]) == {'-'}):
+                sections.append(content[0])
+        return sections
+
     @property
     def summary(self):
         return ' '.join(self.doc['Summary'])
@@ -580,6 +599,25 @@ def validate_one(func_name):
         if re.match("^ *\t", line):
             errs.append(error('GL05', line_with_tabs=line.lstrip()))
 
+    unseen_sections = list(ALLOWED_SECTIONS)
+    for section in doc.section_titles:
+        if section not in ALLOWED_SECTIONS:
+            errs.append(error('GL06',
+                              section=section,
+                              allowed_sections=', '.join(ALLOWED_SECTIONS)))
+        else:
+            if section in unseen_sections:
+                section_idx = unseen_sections.index(section)
+                unseen_sections = unseen_sections[section_idx + 1:]
+            else:
+                section_idx = ALLOWED_SECTIONS.index(section)
+                goes_before = ALLOWED_SECTIONS[section_idx + 1]
+                errs.append(error('GL07',
+                                  sorted_sections=' > '.join(ALLOWED_SECTIONS),
+                                  wrong_section=section,
+                                  goes_before=goes_before))
+                break
+
     if not doc.summary:
         errs.append(error('SS01'))
     else:


Test patch:
diff --git a/scripts/tests/test_validate_docstrings.py b/scripts/tests/test_validate_docstrings.py
--- a/scripts/tests/test_validate_docstrings.py
+++ b/scripts/tests/test_validate_docstrings.py
@@ -350,6 +350,35 @@ def private_classes(self):
         This mentions NDFrame, which is not correct.
         """
 
+    def unknown_section(self):
+        """
+        This section has an unknown section title.
+
+        Unknown Section
+        ---------------
+        This should raise an error in the validation.
+        """
+
+    def sections_in_wrong_order(self):
+        """
+        This docstring has the sections in the wrong order.
+
+        Parameters
+        ----------
+        name : str
+            This section is in the right position.
+
+        Examples
+        --------
+        >>> print('So far Examples is good, as it goes before Parameters')
+        So far Examples is good, as it goes before Parameters
+
+        See Also
+        --------
+        function : This should generate an error, as See Also needs to go
+            before Examples.
+        """
+
 
 class BadSummaries(object):
 
@@ -706,6 +735,11 @@ def test_bad_generic_functions(self, func):
         ('BadGenericDocStrings', 'private_classes',
          ("Private classes (NDFrame) should not be mentioned in public "
           'docstrings',)),
+        ('BadGenericDocStrings', 'unknown_section',
+         ('Found unknown section "Unknown Section".',)),
+        ('BadGenericDocStrings', 'sections_in_wrong_order',
+         ('Wrong order of sections. "See Also" should be located before '
+          '"Notes"',)),
         ('BadSeeAlso', 'desc_no_period',
          ('Missing period at end of description for See Also "Series.iloc"',)),
         ('BadSeeAlso', 'desc_first_letter_lowercase',