DEPR/CLN: Remove pd.rolling_*, pd.expanding* and pd.ewm*
State: closed

- [x] xref #18601, #18668 and #11603
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This PR removes ``pd.rolling_*``,`` pd.expanding_*`` ``pd.ewm*`` and the related ``pd.stats`` subpackage from the code base.

I have some doubts about some stuff:
* there are modules ``doc.plots.stats.*`` that I'm not sure of if they should be changed or removed. Currently they're not touched.
* In ``asv_bench.benchmarks.gil.py`` there are calls to ``pd.rolling_*`` etc. I've changed this to be method-based.
* ``pandas.tests.test_windows.py`` I felt was a bit complex to clean up.

I would appreciate inputs.

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Clean

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=h1) Report
> Merging [#18723](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/2aa4aa947df59d7f0a5bd86145394f5db124250a?src=pr&el=desc) will **increase** coverage by `0.06%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18723/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18723      +/-   ##
==========================================
+ Coverage   91.61%   91.67%   +0.06%     
==========================================
  Files         153      151       -2     
  Lines       51339    51120     -219     
==========================================
- Hits        47034    46865     -169     
+ Misses       4305     4255      -50
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.53% <ø> (+0.07%)` | :arrow_up: |
| #single | `40.69% <ø> (-0.17%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9jYXRlZ29yeS5weQ==) | `97.2% <0%> (-0.31%)` | :arrow_down: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.08% <0%> (-0.24%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `82.34% <0%> (-0.2%)` | :arrow_down: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `96.03% <0%> (-0.15%)` | :arrow_down: |
| [pandas/core/dtypes/dtypes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2R0eXBlcy5weQ==) | `95.14% <0%> (-0.14%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `91.21% <0%> (-0.06%)` | :arrow_down: |
| [pandas/core/indexes/numeric.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9udW1lcmljLnB5) | `97.33% <0%> (-0.04%)` | :arrow_down: |
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `92.9% <0%> (-0.04%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/pandas-dev/pandas/pull/18723/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=footer). Last update [2aa4aa9...e311f50](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=h1) Report
> Merging [#18723](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ca4ae4f1798fb63c707c6cca79e1f586cae63391?src=pr&el=desc) will **increase** coverage by `0.07%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18723/graphs/tree.svg?height=150&token=eZ4WkYLtcO&width=650&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18723      +/-   ##
==========================================
+ Coverage   91.62%   91.69%   +0.07%     
==========================================
  Files         150      148       -2     
  Lines       48728    48542     -186     
==========================================
- Hits        44645    44513     -132     
+ Misses       4083     4029      -54
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.06% <ø> (+0.07%)` | :arrow_up: |
| #single | `41.71% <ø> (-0.03%)` | :arrow_down: |


------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=footer). Last update [ca4ae4f...1443818](https://codecov.io/gh/pandas-dev/pandas/pull/18723?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

The rebase issue has been solved now. I'll look at your comments now,. If you've got additional comments, I'd be thakful, and I'll look at them
Hello @topper-123! Thanks for updating the PR.







Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on February 01, 2018 at 06:53 Hours UTC
Hi @jreback. An issue is that dask apparently still uses ``pd.rolling_*``. This means that ``pandas/tests/test_downstream.py`` currently fails. Should this PR be put on hold, or would notifying the dask guys about this removal be enough? This stuff has been deprecated since 0.18, so for quite a long time.

This PR doesn't touch too many files, so it would be ok to wait a bit, but given the simplification to the pandas API that this PR offers, it should IMO be included when 0.22 releases.

I've updated the PR, so be'd grateful for a review, so it'll be ready for an eventual pull.
@TomAugspurger @mrocklin @jcrist 
The dask versions are deprecated, just like pandas. I believe the best solution would be to

1. Use this PR to update all the test to the new syntax
2. Update dask to remove the `dd.rolling`, etc. methods (@topper-123 could I convince you to work on that? :) It'd be nice to have you as a contributor if you haven't already.)
3. Remove them from pandas once dask has a release

Depending on the timing of pandas 0.22, dask may not have a release out.
@TomAugspurger , the ``test_dask`` function doesn't explicitly call ``pd.rolling_*``, so changes must be made in the dask code base, correct?

I've never used dask, and have a feeling I'd have to learn some intricancies of that lib to make such a PR, and I could easily go down a rabbit's hole that I never intended to enter... Could someone familiar with dask not take it upon themselves to do those changes? I can sit on this PR until dask is updated, and rebase when this is ready to merge.
@topper-123 dask was importing them at the top level if `dask.dataframe`, which caused the failure. Next time dask wants to mirror a pandas deprecation, we'll want to put it in a `try` / `except` block.

https://github.com/dask/dask/pull/2995
@TomAugspurger , thanks for picking up on this, I will wait with my PR a while, until dask is updated. 

My PR lowers the number of top-level functions of pandas by some 30, so it would IMO be very beneficial to pandas' users to have this merged by 0.22, as it would make pandas look much more focused. Could dask have a release before pandas 0.22, eventually just a minor release?
dask 0.16.1 has now been released, and all tests now pass locally with dask version 0.16.1. The travis failure has to do with IO/S3 which I didn't touch. That's the only failure in travis.

So, I think everything is ok with my PR. Is there anything I can do to make this pass that travis error, or is this travis failure unrelated to my PR?
All green.
Can you add a whatsnew notice?
All green
@jorisvandenbossche , I've now replicated the code from ``_check_ew_ndarray`` into ``check_ew``. Could you review it? 
@topper-123 that specific one looks good now I think. 
Did you double check the other tests to see there were not similar problems?
I've added ``test_expanding_func`` (parametrized, was called ``test_parametrized_max`` previously) and ``test_expanding_apply``. Everything is now included AFAICT.
The tests have been running for 9 hours now and the Travis-CI tests have not yet finished. Same thing happened yesterday where after 20 hours one travis test-suite failed without console output (my guess is it timed out).  Is anything wrong with the travis suite? 

Everything passes locally and the errors that travis reported two days ago have been fixed (but the fixes not confirmed by travis, because I can't get travis to go through them...).

What should I do? Could a core dev run the travis tests locally and confirm they're ok, and merge them?
TravisCI had an outage yesterday, so all the builds were canceled and rescheduled. Looks like yours is 4th on deck at the moment: https://travis-ci.org/pandas-dev/pandas/pull_requests
Ok, thanks. I'll wait a bit more then.
All green now🤓
when @jorisvandenbossche approves can merge.
@jorisvandenbossche, I'd really appreciate if you could do this.
Changes suggested by @jorisvandenbossche implemented and all green.
Hi @jreback. I uploaded a new PR 12 days ago.

There were a few issues that were checked in ``_check_ndarray`` but were not in ``_check_moments`` previously:
* tests for whether Nans were excluded correctly
* tests for whether ``center`` worked correctly
* tests for what happens when the window is larger than the series

All of the above are inlined in ``_check_moments`` now (see 37abd2582b5a1043fe460f8793c4480116df31f3).

I've rebased a moments ago, without making code changes otherwise, so everything should turn up green & rebased when the tests are finished.
all green.
@topper-123 Thanks a lot for this!