ENH: Use flake8 to check for PEP8 violations in doctests
State: closed

- [x] tests added 
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Improvement to PR #23381 regarding issue #23154 

Enables `--doctests` flag, resulting in doctests being checked by flake8
Uses [flake8.api.legacy](http://flake8.pycqa.org/en/latest/user/python-api.html) to invoke the tests.

Downsides are:
* > Flake8 3.0.0 presently does not have a public, stable Python API.
* provided options in `flake8.get_style_guide` don't have an effect. Thus `setup.cfg` had to be enhanced.
* returns the count of each error within the complete file, not just the requested function. [Might be confusing]

`python ./scripts/validate_docstrings.py pandas.read_excel`
```
                [...]
                Parameter "convert_float" description should finish with "."
        Errors in doctest sections
                2 F721 syntax error in doctest
                2 F821 undefined name '_make_signature'
```

Help in the form of `Use "flake8 pandas/util/_decorators.py" for further information.` could be added to the output:
```
pandas/util/_decorators.py:320:26: F721 syntax error in doctest
pandas/util/_decorators.py:321:13: F721 syntax error in doctest
pandas/util/_decorators.py:322:15: F821 undefined name '_make_signature'
pandas/util/_decorators.py:322:31: F821 undefined name 'f'
```


Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Hello @FHaase! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`scripts/validate_docstrings.py`](https://github.com/FHaase/pandas/blob/c87360d45b9d9452ead729c28fba68e3ce57c4fd/scripts/validate_docstrings.py) !


@FHaase : Thanks for the PR!  Given your first comment in the description, I would recommend opening an issue first.  That's generally what we would recommend instead of jumping right to the PR.

cc @datapythonista 
Thanks @FHaase, this looks much better. Can you mention the issue you close, and the other PR related to this in the description please.

I just saw couple of things. Doesn't feel like we should be using a module named legacy. I guess you checked it, but doesn't flake8 have a newer module if they call that one legacy?

Also, feels like when we validate a docstring, we validate all the pep8 problems in the file, right? We should find a way to validate a single docstring.

When this is done, we should also implement tests for this, as the rest of validations.

In any case, good work, this will be very helpful once is done.
> I just saw couple of things. Doesn't feel like we should be using a module named legacy. I guess you checked it, but doesn't flake8 have a newer module if they call that one legacy?

[`flake8.api`](https://gitlab.com/pycqa/flake8/tree/master/src/flake8/api) is empty besides the legacy submodule. `__init.py__` states:

> Module containing all public entry-points for Flake8.
This is the only submodule in Flake8 with a guaranteed stable API. All other
submodules are considered internal only and are subject to change.

> Also, feels like when we validate a docstring, we validate all the pep8 problems in the file, right? We should find a way to validate a single docstring.

Yes I mentioned it. The only way of fixing this might be to create a temp-file just with the docstring in it and run flake8 on that?  
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=h1) Report
> Merging [#23399](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/047242b59c15d9f184d32b025c3e545e57f6c50a?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23399/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #23399   +/-   ##
=======================================
  Coverage   92.23%   92.23%           
=======================================
  Files         161      161           
  Lines       51197    51197           
=======================================
  Hits        47220    47220           
  Misses       3977     3977
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.61% <ø> (ø)` | :arrow_up: |
| #single | `42.27% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/23399/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `96.41% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/23399/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.81% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=footer). Last update [047242b...1178cae](https://codecov.io/gh/pandas-dev/pandas/pull/23399?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> I just saw couple of things. Doesn't feel like we should be using a module named legacy. I guess you checked it, but doesn't flake8 have a newer module if they call that one legacy?

Dug down into the code and basically recreated the legacy api.
Thanks @FHaase, extremely useful PR