Changes to validate_docstring script to be able to check all docstrings at once
State: closed

- [ ] closes #xxxx
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

This PR includes two main changes:
1. When calling the `scripts/validate_docstrings.py` script without parameters, it returns a JSON with the validation of all docstrings.
2. Refactoring the structure of the code, mainly to be able to implement 1.

The previous behavior when calling the script with parameters was to generate a csv with basic information about each docstring (not the full validation, but basic things like whether sections existed, or the file where the function is defined).

The main change is that `validate_one()` now returns the results as Python objects, instead of printing directly to the console. Few other changes to make the code cleaner are implemented (mainly moving `_load_obj` and `_original_callable` inside the `Docstring` class.


Labels:
name: Docs

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=h1) Report
> Merging [#22408](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f771ef67e8fa3f395967d5c02918fbbc3410612a?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22408/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #22408      +/-   ##
==========================================
+ Coverage   92.18%   92.19%   +<.01%     
==========================================
  Files         169      169              
  Lines       50830    50873      +43     
==========================================
+ Hits        46860    46904      +44     
+ Misses       3970     3969       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.61% <ø> (+0.01%)` | :arrow_up: |
| #single | `42.32% <ø> (-0.05%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `95.56% <0%> (-0.56%)` | :arrow_down: |
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `93.48% <0%> (-0.37%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `93.89% <0%> (-0.26%)` | :arrow_down: |
| [pandas/core/ops.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvb3BzLnB5) | `97.19% <0%> (-0.19%)` | :arrow_down: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.62% <0%> (-0.13%)` | :arrow_down: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `92.44% <0%> (-0.05%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.65% <0%> (-0.02%)` | :arrow_down: |
| [pandas/core/strings.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3RyaW5ncy5weQ==) | `98.63% <0%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/core/tools/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvZGF0ZXRpbWVzLnB5) | `85.22% <0%> (ø)` | :arrow_up: |
| ... and [21 more](https://codecov.io/gh/pandas-dev/pandas/pull/22408/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=footer). Last update [f771ef6...db782fd](https://codecov.io/gh/pandas-dev/pandas/pull/22408?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Thanks for the great feedback. I addressed most of the points, and commented on the others.

@jorisvandenbossche agree some more info about the PR had to be provided. I edited the description to include what's the goal, and what else has been changed.
> I edited the description to include what's the goal, and what else has been changed.

OK, that sounds good to me!
@WillAyd let me know if there is any blocker to merge this
@datapythonista sorry I think the size of the diff is misleading; I didn't realize I was asking you to change things that were orthogonal to your change in the first place!

Is this addressing something critical that we need to get in quickly? On one side I suppose it's OK because this isn't going to break anything critical, but on the other hand I do think our test coverage here is pretty weak and think it would be easier to address sooner than later
Not critical, but it'll take me few days to have the time to write those tests. And as the PR modifies the whole script, it is blocking (or will generate conflicts) with any other change to it.

#22423 is already a case, and I'd like to start creating issues for tasks in #20298, as they will save us a lot of time when reviewing the docstring updates.

But it can surely wait a couple of weeks if you think it's worth adding the tests here.
Thanks @datapythonista - I'd prefer tests up front. Hate to be a blocker on this, but realistically the longer we wait for test coverage the harder it becomes to ever get there
Hello @datapythonista! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`scripts/tests/test_validate_docstrings.py`](https://github.com/datapythonista/pandas/blob/3a5e7fd823462f8b899bd154adc03280aa4b57d2/scripts/tests/test_validate_docstrings.py) !

 - There are no PEP8 issues in the file [`scripts/validate_docstrings.py`](https://github.com/datapythonista/pandas/blob/3a5e7fd823462f8b899bd154adc03280aa4b57d2/scripts/validate_docstrings.py) !



##### Comment last updated on September 28, 2018 at 15:36 Hours UTC
@WillAyd I added the requested tests, let me know if this can be merge now
@jorisvandenbossche can you take a look at this? it's the PR that makes the docstring validation script generate the list with the errors in all docstrings.

Once this is merged I'll start creating individual tickets for all the points, so we start validating all possible errors in the docstrings (and with some luck we can add it to the CI soon).
cc @jorisvandenbossche @WillAyd 
@WillAyd addressed or commented your review, can you take another look please?
Merging this, so we can finally unblock #20298. If there is any other comment I'll take care of it in a separate PR.
Sorry forgot about this one. Thanks for moving this forward!