DEPS: drop numpy < 1.12
State: closed

- [x] closes #21242
- [x] tests modified / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Spurred on also partly by compat failure of #22725. ;-)


Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
Hello @h-vetinari! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/compat/numpy/__init__.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/compat/numpy/__init__.py) !

 - There are no PEP8 issues in the file [`pandas/core/algorithms.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/core/algorithms.py) !

 - There are no PEP8 issues in the file [`pandas/core/computation/check.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/core/computation/check.py) !

 - There are no PEP8 issues in the file [`pandas/core/ops.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/core/ops.py) !

 - There are no PEP8 issues in the file [`pandas/plotting/_compat.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/plotting/_compat.py) !

 - There are no PEP8 issues in the file [`pandas/plotting/_converter.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/plotting/_converter.py) !

 - There are no PEP8 issues in the file [`pandas/plotting/_core.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/plotting/_core.py) !

 - There are no PEP8 issues in the file [`pandas/plotting/_style.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/plotting/_style.py) !

 - There are no PEP8 issues in the file [`pandas/tests/arithmetic/test_period.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/arithmetic/test_period.py) !

 - There are no PEP8 issues in the file [`pandas/tests/frame/test_analytics.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/frame/test_analytics.py) !

 - There are no PEP8 issues in the file [`pandas/tests/frame/test_quantile.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/frame/test_quantile.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/datetimes/test_ops.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/indexes/datetimes/test_ops.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/period/test_ops.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/indexes/period/test_ops.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/period/test_partial_slicing.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/indexes/period/test_partial_slicing.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/timedeltas/test_ops.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/indexes/timedeltas/test_ops.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/common.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/common.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_boxplot_method.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_boxplot_method.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_datetimelike.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_datetimelike.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_frame.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_frame.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_hist_method.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_hist_method.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_misc.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_misc.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_series.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/plotting/test_series.py) !

 - There are no PEP8 issues in the file [`pandas/tests/series/test_analytics.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/series/test_analytics.py) !

 - There are no PEP8 issues in the file [`pandas/tests/sparse/series/test_series.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/sparse/series/test_series.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_algos.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/test_algos.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_expressions.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/test_expressions.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_sorting.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/test_sorting.py) !

 - There are no PEP8 issues in the file [`pandas/tests/test_window.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/tests/test_window.py) !

 - There are no PEP8 issues in the file [`pandas/util/_test_decorators.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/pandas/util/_test_decorators.py) !

 - There are no PEP8 issues in the file [`setup.py`](https://github.com/h-vetinari/pandas/blob/e7f5bf2438c5ba8eca2eef0edf70c9608354b394/setup.py) !



##### Comment last updated on October 12, 2018 at 16:26 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=h1) Report
> Merging [#23062](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cf11f71e423be65e9c2ae83ea1bab8b69ee79ca4?src=pr&el=desc) will **increase** coverage by `0.04%`.
> The diff coverage is `94.73%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23062/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #23062      +/-   ##
==========================================
+ Coverage   92.14%   92.19%   +0.04%     
==========================================
  Files         170      170              
  Lines       51017    50963      -54     
==========================================
- Hits        47011    46984      -27     
+ Misses       4006     3979      -27
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.61% <94.73%> (+0.04%)` | :arrow_up: |
| #single | `42.29% <31.57%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.82% <ø> (-0.07%)` | :arrow_down: |
| [pandas/util/\_test\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX3Rlc3RfZGVjb3JhdG9ycy5weQ==) | `93.24% <ø> (+0.56%)` | :arrow_up: |
| [pandas/plotting/\_style.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19zdHlsZS5weQ==) | `75% <100%> (+0.71%)` | :arrow_up: |
| [pandas/plotting/\_compat.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb21wYXQucHk=) | `87.5% <100%> (-3.81%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.63% <100%> (+0.27%)` | :arrow_up: |
| [pandas/core/computation/check.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vY2hlY2sucHk=) | `90.9% <100%> (ø)` | :arrow_up: |
| [pandas/core/algorithms.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYWxnb3JpdGhtcy5weQ==) | `95.12% <100%> (+0.55%)` | :arrow_up: |
| [pandas/core/ops.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvb3BzLnB5) | `94.63% <100%> (+0.24%)` | :arrow_up: |
| [pandas/compat/numpy/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree#diff-cGFuZGFzL2NvbXBhdC9udW1weS9fX2luaXRfXy5weQ==) | `92.85% <80%> (-1.09%)` | :arrow_down: |
| ... and [1 more](https://codecov.io/gh/pandas-dev/pandas/pull/23062/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=footer). Last update [cf11f71...d097b43](https://codecov.io/gh/pandas-dev/pandas/pull/23062?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@WillAyd 
I did it right here (where it belongs IMO). Not sure I got everything, because there are some Win32 errors in the plotting tests that were already there before.
i don’t think any real urgency to bump numexpr or bottleneck mins

we generally try to allow packages to continue to work as much as possible
> i don’t think any real urgency to bump numexpr or bottleneck mins
we generally try to allow packages to continue to work as much as possible

I only bumped them because anything lower is not compatible with numpy>=1.12 (you can check the respective builds to see that always something was failing due to unresolvable dependencies)
In core.ops there is a try/except for `np.broadcast_to` that can be simplified along with this PR.
Clearly the sane option would have been not to touch the mpl compat code...

In any case, I investigated the conda resolvability things, and locally, I get:
```
(pandas-dev) C:\[...]\pddev>conda install "numpy>=1.12" "bottleneck<1.2.0"
UnsatisfiableError: The following specifications were found to be in conflict:
  - bottleneck[version='<1.2.0'] -> numpy=1.10
  - numpy[version='>=1.12']

(pandas-dev) C:\[...]\pddev>conda install "numpy>=1.12" "matplotlib<2.0.0"
UnsatisfiableError: The following specifications were found to be in conflict:
  - matplotlib[version='<2.0.0'] -> numpy=1.6
  - numpy[version='>=1.12']
  
(pandas-dev) C:\[...]\pddev>conda install "numpy>=1.12" "pytables<3.2.2"
UnsatisfiableError: The following specifications were found to be in conflict:
  - numpy[version='>=1.12']
  - pytables[version='<3.2.2'] -> numexpr=2.0 -> numpy=1.6

(pandas-dev) C:\[...]\pddev>conda install "numpy>=1.12" "scipy<0.18.1"
UnsatisfiableError: The following specifications were found to be in conflict:
  - numpy[version='>=1.12']
  - scipy[version='<0.18.1'] -> numpy=1.6
```

That's why reduced the version bump for `pytables`, but now https://circleci.com/gh/pandas-dev/pandas/24379
says:
```
UnsatisfiableError: The following specifications were found to be in conflict:
  - numpy=1.12.0
  - pytables=3.2.2 -> numexpr=2.5 -> numpy=1.10
```

Not sure where this is coming from, but going back up to `3.4.2`.
@WillAyd @jreback @jbrockmendel 

There's a *chance* that the last commit runs through without errors. If so, I would then obviously remove all the print statements again, but wanted to ask if would be desired to keep the matplotlib-pins I added to cover all the versions (`2.0.0`, `2.0.1`, `2.1.0`, `2.2.0`, `2.2.2`, `2.2.3` (unpinned; latest for Py2), `3.0.0`), or if I should revert those too.

There were a bunch of them that were not tested - with slightly different behaviours (as the diff in `test_datetimelike` shows)...
OK, https://github.com/pandas-dev/pandas/pull/23062/commits/b971bfba245efde575269eca38b09a614256e33b is finally green for all MPL versions. Removing the debugging stuff and reverting most of the pins (I left two to also test `2.0.1` and `2.2.2`, which would be untested otherwise).
lgtm. can you rebase. @TomAugspurger ok with the plotting cleanups? should merge this before SparseArray? (to avoid some hacks for backward compat)?
Plotting changes look fine. No need to merge before / after sparse array. The hacks there are minimal, and I think will be addressed in time anyway.
@TomAugspurger 
There's something wrong with the azure failure reporting (discovered by accident):
The last three runs reported success despite two failures in the Mac job:
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=961
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=964
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=1045
> There's something wrong with the azure failure reporting (discovered by accident):
The last three runs reported success despite two failures in the Mac job

I've been troubleshooting the same problem [here](https://github.com/jbrockmendel/sm2/pull/96), please let me know if you figure it out
I understand now why no-one touched the matplotlib compat for a long time (e.g. `_mpl_le_1_2_1` still around, while the lowest tested version prior to this PR was 1.4.3)...

In any case, I've now got a passing CI on all mpl builds again in https://github.com/pandas-dev/pandas/pull/23062/commits/079bdaf078733adc1faa1d076a43f8b86ea9a44c (after the false positive from azure; there are no pins or changed code paths for matplotlib in Circle, which is lagging behind due to not cancelling on new pushes):
`2.0.0`, `2.0.1`, `2.1.0`, `2.2.0`, `2.2.2`, `2.2.3`, `3.0.0`

Reverting the pins, then this thing should be done...

@TomAugspurger @jreback another quick look?
pls rebase

not you will have to rebase somewhat frequently until this is merged 
> note you will have to rebase somewhat frequently until this is merged

What's the hold-up on getting this merged soon?
PS. Can someone trigger the circleCI runs? not sure why they didn't start.
this. needs review for the plotting changes 
I don't see any way to trigger a circle build from their UI. Probably easiest to add an empty commit.
lgtm. @WillAyd if ok, pls accept, but after that @h-vetinari will need a rebase then, just before merging to make sure nothing has changed. (you can rebase now if you want).
Not much happened in the last 17h (3 commits), but ok.
thanks @h-vetinari 
This was quite the effort I have to say. MPL has some really weird behaviour in that (e.g.) several times in `test_datetimelike` there's:
* behaviour A in `2.0.0`, `2.2.0` and `3.0.0`
* behaviour B in `2.0.1`, `2.1.0`, `2.2.2`, `2.2.3`.

Not 100% sure this is not overlaid with some numpy/system dependencies as well, but happy that right now everything passes and got merged.