Split test_categorical into subpackage (#18497)
State: closed

- [X] closes #18497
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Speaking to the methodology here, I created separate files mirroring what was provided in the other test packages. Within each file, there is one class for standard categorical tests and one for "block" tests, owing back to how this file is currently structured.

Because there were setup functions for only a very limited number of test cases, rather than creating those as separate classes within each module I lumped all of them into one ``test_generic.py`` file.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Categorical
description: Categorical Data Type

Comments:
you need to add an entry in ``setup.py`` FYI as we don't pick up the test sub-dirs automatically (maybe we should)
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=h1) Report
> Merging [#18508](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/674fb96b33c07c680844f674fcdf0767b6e3c2f9?src=pr&el=desc) will **decrease** coverage by `0.04%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18508/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18508      +/-   ##
==========================================
- Coverage   91.35%   91.31%   -0.05%     
==========================================
  Files         163      163              
  Lines       49801    49796       -5     
==========================================
- Hits        45496    45469      -27     
- Misses       4305     4327      +22
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.11% <ø> (-0.03%)` | :arrow_down: |
| #single | `40.79% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.44% <0%> (-1.82%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `96.94% <0%> (-0.09%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=footer). Last update [674fb96...574e6c3](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=h1) Report
> Merging [#18508](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9629fef912dbeca5154e9cf58970bad3e02657f4?src=pr&el=desc) will **decrease** coverage by `0.03%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18508/graphs/tree.svg?width=650&height=150&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18508      +/-   ##
==========================================
- Coverage   91.59%   91.55%   -0.04%     
==========================================
  Files         153      155       +2     
  Lines       51257    51255       -2     
==========================================
- Hits        46949    46929      -20     
- Misses       4308     4326      +18
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.42% <ø> (-0.02%)` | :arrow_down: |
| #single | `40.67% <ø> (-0.12%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.44% <0%> (-3.08%)` | :arrow_down: |
| [pandas/core/config\_init.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnX2luaXQucHk=) | `98.34% <0%> (-0.12%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/plotting/\_timeseries.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL190aW1lc2VyaWVzLnB5) | `60.73% <0%> (-0.1%)` | :arrow_down: |
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `96.9% <0%> (-0.05%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.68% <0%> (ø)` | :arrow_up: |
| [pandas/plotting/\_compat.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb21wYXQucHk=) | `62% <0%> (ø)` | :arrow_up: |
| [pandas/io/parquet.py](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnF1ZXQucHk=) | `65.38% <0%> (ø)` | :arrow_up: |
| ... and [9 more](https://codecov.io/gh/pandas-dev/pandas/pull/18508/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=footer). Last update [9629fef...5e9234a](https://codecov.io/gh/pandas-dev/pandas/pull/18508?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Do you think it makes sense to move the block constructors into the series and frame construction tests as well or would you rather keep them here?
yes i would ideally like to live any testing to series/frame as appropriate 
could add a series/test_categorical (and frame too)

we already segregate some testing by type somewhat (test_timestanp/timedelta) and not solely by function (in series/frame)
Hello @WillAyd! Thanks for updating the PR.





























 - In the file [`pandas/tests/frame/test_constructors.py`](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/frame/test_constructors.py), following are the PEP8 issues :

> [Line 1604:9](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/frame/test_constructors.py#L1604): [E741](https://duckduckgo.com/?q=pep8%20E741) ambiguous variable name 'l'
> [Line 1609:9](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/frame/test_constructors.py#L1609): [E741](https://duckduckgo.com/?q=pep8%20E741) ambiguous variable name 'l'



























 - In the file [`pandas/tests/series/test_dtypes.py`](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/series/test_dtypes.py), following are the PEP8 issues :

> [Line 234:9](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/series/test_dtypes.py#L234): [E741](https://duckduckgo.com/?q=pep8%20E741) ambiguous variable name 'l'
> [Line 240:9](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/series/test_dtypes.py#L240): [E741](https://duckduckgo.com/?q=pep8%20E741) ambiguous variable name 'l'
> [Line 261:9](https://github.com/pandas-dev/pandas/blob/e77f1987515550ac9a46157e116676f2aed4803e/pandas/tests/series/test_dtypes.py#L261): [E741](https://duckduckgo.com/?q=pep8%20E741) ambiguous variable name 'l'















##### Comment last updated on December 04, 2017 at 13:41 Hours UTC
This was a fairly involved change as the tests were scattered all over the place. To aid in tracking purposes, the attached file lists out every function that was in the original ``test_categories.py`` file and where it was moved to. Please note that in some cases I changed the function name for clarity.

In addition to the parameterization that was added to the ``test_comparisons`` function I ended up refactoring three new functions into place, which you'll see on the second tab of the attached. Let me know what you think.
[categorical_func_moves.xlsx](https://github.com/pandas-dev/pandas/files/1525212/categorical_func_moves.xlsx)

usually a quick check is to run the tests before and after and make sure we have the same number
When I compared the number of tests to upstream/master I got 4 more tests, which is what I expected (one from the parameterized test and the three new funcs)
Edits accounted for. Note that 5 new tests are showing up since the last commit, due to the parametrization of ``test_mode`` in ``test_analytics.py``. 

In case it is of use I updated the attachment with before/after locations of every func.

[categorical_func_moves.xlsx](https://github.com/pandas-dev/pandas/files/1527260/categorical_func_moves.xlsx)

Checked the AppVeyor error but it is not coming from a module that was changed as part of this, so I believe it is unrelated. Let me know any outstanding thoughts on the change
thanks @WillAyd nice patch!