API: expose public testing functions in pandas.testing (GH9895)
State: closed

Closes #9895

This just exposes a few of our testing functions publicly in `pandas.testing`.

- Alternative is to move the full testing module to `pandas.testing` (the actual code), question is then however how to only expose a subset of it publicly.
- I now only included the 3 most obvious ones (frame, series, index). Are there other ones that people would want to use? (the sparse ones?)
- We have quite a lot of keywords to those assert_* functions. We could op to only expose a subset of those publicly (redefining in pandas.testing instead of importing and only passing subset of the kwargs through)

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: API Design

Comments:
lgtm. need some whatsnew docs (but can prob add onto the ``pandas.errors`` section maybe (rename it though)
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=h1) Report
> Merging [#16003](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2d9451ddc36c194f1d65dab7568901c9ea998002?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16003/graphs/tree.svg?width=650&height=150&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16003      +/-   ##
==========================================
+ Coverage   91.02%   91.02%   +<.01%     
==========================================
  Files         145      146       +1     
  Lines       50391    50394       +3     
==========================================
+ Hits        45870    45873       +3     
  Misses       4521     4521
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.83% <100%> (ø)` | :arrow_up: |
| #single | `40.33% <100%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `81.01% <ø> (ø)` | :arrow_up: |
| [pandas/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL3Rlc3RpbmcucHk=) | `100% <100%> (ø)` | |
| [pandas/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL19faW5pdF9fLnB5) | `92.85% <100%> (+0.17%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=footer). Last update [2d9451d...bb65d12](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@1f812e3`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16003/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master   #16003   +/-   ##
=========================================
  Coverage          ?      91%           
=========================================
  Files             ?      147           
  Lines             ?    50422           
  Branches          ?        0           
=========================================
  Hits              ?    45887           
  Misses            ?     4535           
  Partials          ?        0
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.81% <100%> (?)` | |
| #single | `40.32% <100%> (?)` | |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `81.01% <ø> (ø)` | |
| [pandas/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL3Rlc3RpbmcucHk=) | `100% <100%> (ø)` | |
| [pandas/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=tree#diff-cGFuZGFzL19faW5pdF9fLnB5) | `92.85% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=footer). Last update [1f812e3...1bf8a93](https://codecov.io/gh/pandas-dev/pandas/pull/16003?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I did a quick search for which `pandas.util.testing` functionality is used by other projects (apart from the main 3 assert_frame/series/index_equal ones):

- dask:
  - makeDataFrame, makeTimeDataFrame, makeMissingDataframe, makeDateIndex, makeTimedeltaIndex, makeTimeSeries
  - assertRaisesRegexp, assertRaises, assert_produces_warning, assert_numpy_array_equal
- statsmodels:
  - makeDataFrame, makeTimeDataFrame, assert_produces_warning
- zipline:
  - assert_panel_equal (but since panel is deprecated, this is less important)

I am not sure whether we want to commit to keeping some of the `make..` functions stable? (and projects can of course keep using them at own risk as it is now)

For the raises and warning ones, since those are not pandas-objects specific test functions, I would also not include them in the official public ones?
xref https://github.com/pandas-dev/pandas/issues/15911

I think we will create a ``pandas.util.benchmarks`` sub-module that holds things like ``make*`` which we can make public, keeping  ``pandas.util.testing`` private (or dask can reach in, doesn't really matter). I don't think these need to be really public anyhow.

cc @mrocklin

``makeDataFrame`` in camelCase is a really old naming convention anyhow. 
pls rebase. lgtm. let's merge and can always amend/add to this. 
Does this mean pytest is now a dependency of pandas?

I just upgraded pandas through [AUR](https://aur.archlinux.org/packages/python-pandas-git/) and got this error message:

```  
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pandas/__init__.py", line 60, in <module>
    import pandas.testing
  File "/usr/lib/python3.6/site-packages/pandas/testing.py", line 7, in <module>
    from pandas.util.testing import (
  File "/usr/lib/python3.6/site-packages/pandas/util/testing.py", line 22, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'
```

I do not have pytest installed.  Is it now required?
pytest is not required unless you are actually testing things - this is fixed in the rc

unclear what version you actually have
I had 1b52b1218e973382ea958084c47c3f33eb0ed40c installed.  Looks like it was fixed in 70ae817a5d0bd5e11e30a2e0781afa98103d656f .  Re-reinstalled Pandas and it seems to import without pytest just fine now.  Thanks!