Repository: matplotlib/matplotlib
Pull request: FIX: catch warnings from pandas in cbook._check_1d
If we catch the warning or the exception, we need to cast to numpy
because later on in `_plot_args` we again use multi-dimensional
indexing to up-cast to a 2D array.

closes #16295


## PR Summary

## PR Checklist

- [x] Has Pytest style unit tests
- [x] Code is [Flake 8](http://flake8.pycqa.org/en/latest/) compliant

<!--
Thank you so much for your PR!  To help us review your contribution, please
consider the following points:

- A development guide is available at https://matplotlib.org/devdocs/devel/index.html.

- Help with git and github is available at
  https://matplotlib.org/devel/gitwash/development_workflow.html.

- Do not create the PR out of master, but out of a separate branch.

- The PR title should summarize the changes, for example "Raise ValueError on
  non-numeric input to set_xlim".  Avoid non-descriptive titles such as
  "Addresses issue #8576".

- The summary should provide at least 1-2 sentences describing the pull request
  in detail (Why is this change required?  What problem does it solve?) and
  link to any relevant issues.

- If you are contributing fixes to docstrings, please pay attention to
  http://matplotlib.org/devel/documenting_mpl.html#formatting.  In particular,
  note the difference between using single backquotes, double backquotes, and
  asterisks in the markup.

We understand that PRs can sometimes be overwhelming, especially as the
reviews start coming in.  Please let us know if the reviews are unclear or
the recommended next step seems overly demanding, if you would like help in
addressing a reviewer's comments, or if you have been waiting too long to hear
back on your PR.
-->

Problem statement:
pyplot.plot using pandas series raises DeprecationWarning with pandas=1.0.0rc0
<!--To help us understand and resolve your issue, please fill out the form to the best of your ability.-->
<!--You can feel free to delete the sections that do not apply.-->

### Bug report

**Bug summary**

<!--A short 1-2 sentences that succinctly describes the bug-->
The code below raises ``DeprecationWarning`` from ``pandas=1.0.0rc0`` when ``plt.plot`` is fed with two ``pandas.Series`` (using my matplotlib=3.1.0 instance). It reproduces also when the series is not empty, and also when plotting two different series as x and y. 

**Code for reproduction**
<!--A minimum code snippet required to reproduce the bug.
Please make sure to minimize the number of dependencies required, and provide
any necessary plotted data.
Avoid using threads, as Matplotlib is (explicitly) not thread-safe.-->

```python
import pandas as pd
import matplotlib.pyplot as plt

x = pd.Series([], dtype="float64")
plt.plot(x, x)
```

**Actual outcome**

<!--The output produced by the above code, which may be a screenshot, console output, etc.-->

```
Traceback (most recent call last):
  File "pandas1.py", line 5, in <module>
    plt.plot(x, x)
  File "/lib/python3.7/site-packages/matplotlib/pyplot.py", line 2789, in plot
    is not None else {}), **kwargs)
  File "/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 1666, in plot
    lines = [*self._get_lines(*args, data=data, **kwargs)]
  File "/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 225, in __call__
    yield from self._plot_args(this, kwargs)
  File "/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 386, in _plot_args
    x = _check_1d(tup[0])
  File "/lib/python3.7/site-packages/matplotlib/cbook/__init__.py", line 1402, in _check_1d
    x[:, None]
  File "/lib/python3.7/site-packages/pandas/core/series.py", line 865, in __getitem__
    return self._get_with(key)
  File "/lib/python3.7/site-packages/pandas/core/series.py", line 878, in _get_with
    return self._get_values_tuple(key)
  File "/lib/python3.7/site-packages/pandas/core/series.py", line 920, in _get_values_tuple
    return self._get_values(key)
  File "/lib/python3.7/site-packages/pandas/core/series.py", line 934, in _get_values
    self._data.get_slice(indexer), fastpath=True
  File "/lib/python3.7/site-packages/pandas/core/internals/managers.py", line 1542, in get_slice
    return type(self)(self._block._slice(slobj), self.index[slobj], fastpath=True)
  File "/lib/python3.7/site-packages/pandas/core/indexes/range.py", line 708, in __getitem__
    return super().__getitem__(key)
  File "/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 3908, in __getitem__
    deprecate_ndim_indexing(result)
  File "/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 5553, in deprecate_ndim_indexing
    stacklevel=3,
DeprecationWarning: Support for multi-dimensional indexing (e.g. `index[:, None]`) on an Index is deprecated and will be removed in a future version.  Convert to a numpy array before indexing instead.
```

**Expected outcome**

No warnings.

**Matplotlib version**
<!--Please specify your platform and versions of the relevant libraries you are using:-->
  * Operating system: linux
  * Matplotlib version: 3.1.0
  * Matplotlib backend (`print(matplotlib.get_backend())`): agg
  * Python version: 3.7.3
  * Other libraries: pandas=1.0.0rc0

<!--Please tell us how you installed matplotlib and python e.g., from source, pip, conda-->
<!--If you installed from conda, please specify which channel you used if not the default-->
python and matplotlib obtained through anaconda, pandas through pip

Hints from issue discussion:
This is something we have sort of dealt with before (https://github.com/matplotlib/matplotlib/issues/14992 / https://github.com/matplotlib/matplotlib/pull/15007 and https://github.com/pandas-dev/pandas/issues/27775).

Looks like we are ready for the far future after the deprecation is done, but not the near future when it is warning, working on a PR.

Commit message: FIX: catch warnings from pandas in cbook._check_1d

If we catch the warning or the exception, we need to cast to numpy
because later on in `_plot_args` we again use multi-dimensional
indexing to up-cast to a 2D array.

closes #16295

Patch to review:
diff --git a/lib/matplotlib/cbook/__init__.py b/lib/matplotlib/cbook/__init__.py
index fbb9ee5f1ec0..5617b5f071ba 100644
--- a/lib/matplotlib/cbook/__init__.py
+++ b/lib/matplotlib/cbook/__init__.py
@@ -1367,11 +1367,22 @@ def _check_1d(x):
         return np.atleast_1d(x)
     else:
         try:
-            ndim = x[:, None].ndim
             # work around https://github.com/pandas-dev/pandas/issues/27775
             # which mean the shape is not as expected. That this ever worked
             # was an unintentional quirk of pandas the above line will raise
             # an exception in the future.
+            # This warns in pandas >= 1.0 via
+            # https://github.com/pandas-dev/pandas/pull/30588
+            with warnings.catch_warnings(record=True) as w:
+                warnings.filterwarnings("always",
+                                        category=DeprecationWarning,
+                                        module='pandas[.*]')
+
+                ndim = x[:, None].ndim
+                # we have definitely hit a pandas index or series object
+                # cast to a numpy array.
+                if len(w) != 0:
+                    return np.asanyarray(x)
             if ndim < 2:
                 return np.atleast_1d(x)
             return x
diff --git a/lib/matplotlib/tests/test_axes.py b/lib/matplotlib/tests/test_axes.py
index 3c7d3c1adcdd..293b81141e26 100644
--- a/lib/matplotlib/tests/test_axes.py
+++ b/lib/matplotlib/tests/test_axes.py
@@ -1695,6 +1695,14 @@ def test_bar_pandas_indexed(pd):
     ax.bar(df.x, 1., width=df.width)
 
 
+def test_pandas_smoke(pd):
+    # This should not raise any warnings
+    x = pd.Series([], dtype="float64")
+    plt.plot(x, x)
+    plt.plot(x.index, x)
+    plt.plot(x)
+
+
 @image_comparison(['hist_log'], remove_text=True)
 def test_hist_log():
     data0 = np.linspace(0, 1, 200)**3


Code review comments:
diff_hunk: @@ -1367,11 +1367,22 @@ def _check_1d(x):
         return np.atleast_1d(x)
     else:
         try:
-            ndim = x[:, None].ndim
             # work around https://github.com/pandas-dev/pandas/issues/27775
             # which mean the shape is not as expected. That this ever worked
             # was an unintentional quirk of pandas the above line will raise
             # an exception in the future.
path: lib/matplotlib/cbook/__init__.py
text: @user1:
since the above line is getting deleted, this comment is no longer relevant. 

@author:
It is still relevant, the line above was moved down and put inside the context manager below.

@user1:
Ok, then it should be changed to "the line below" or moved to be next to the line it's referring to:
``` python
 ndim = x[:, None].ndim  #will raise exception in future
```

diff_hunk: @@ -1367,11 +1367,22 @@ def _check_1d(x):
         return np.atleast_1d(x)
     else:
         try:
-            ndim = x[:, None].ndim
             # work around https://github.com/pandas-dev/pandas/issues/27775
             # which mean the shape is not as expected. That this ever worked
             # was an unintentional quirk of pandas the above line will raise
             # an exception in the future.
+            # This warns in pandas >= 1.0 via
+            # https://github.com/pandas-dev/pandas/pull/30588
+            with warnings.catch_warnings(record=True) as w:
+                warnings.filterwarnings("always",
+                                        category=DeprecationWarning,
+                                        module='pandas[.*]')
+
+                ndim = x[:, None].ndim
+                # we have definitely hit a pandas index or series object
+                # cast to a numpy array.
+                if len(w) != 0:
path: lib/matplotlib/cbook/__init__.py
text: @user1:
little bit confused why having a warning of any length (and why can it be negative in this case?) means casting x to an array. Is there anything more specific going on here?

@author:
the pandas RC now warns (it didn't used to) so if we get a warning from pandas -> we know we have a pandas Index or Series so we need to cast it to an array so our down-stream code works.  `len` won't give us negative lengths.

In pandas < 1.0rc0 the `[:, None]` line does not warn, but returns something with a very funny behavior (ndim and shape are inconsistent, that is what the `ndim < 2` line below catches), for pandas >= 1.0rc0 the `[:, None]` now warns, but return a numpy array whos shape is consistent with ndim.  In either case, we need to convert the pandas object to an array so our downstream code works as expected.

@user1:
>len won't give us negative lengths.
so why not do `if len(w) >0`?

diff_hunk: @@ -1695,6 +1695,14 @@ def test_bar_pandas_indexed(pd):
     ax.bar(df.x, 1., width=df.width)
 
 
+def test_pandas_smoke(pd):
path: lib/matplotlib/tests/test_axes.py
text: @user1:
why is it called smoke?

@user2:
Smoke tests are simple tests to verify basic functionality, without checking details, such as „does it run without producing errors“. Not sure why it’s called that way. Maybe because the program smokes metaphorically if it’s not working properly or blows up?

https://en.wikipedia.org/wiki/Smoke_testing_(software)


@author:
My understanding of the history of that phrase is that with physical hardware if it fails the test it starts smoking.  Which leads to the joke that "electronics run on smoke" (like steam engines run on steam) and when it breaks "you let the smoke out". 

@user1:
Ok, then can we change the name of this function to something that's not idiomatic like `test_pandas_empty` or `test_accepts_pandas`?