Repository: pandas-dev/pandas

Issue:
Update plotting functions to use the new Matplotlib default styles
In version 2.0.0, Matplotlib introduced several changes in it's default style. Pandas plotting functions should be updated to use the new style.

http://matplotlib.org/users/dflt_style_changes.html


Discussion:
Can you give specific examples of where the plot needs to be updated? I don't think we are using much hard-coded style elements in the plot method.

If you are talking about the documentation (http://pandas.pydata.org/pandas-docs/stable/visualization.html), we are already using the `style` support of matplotlib, but using the 'ggplot' style instead of the default (but that is just for the docs, not for the default that users get when plotting). So for this, we could indeed switch to the default style again now this has been improved. 
One thing that I've noticed is that Pandas doesn't use the new behavior for axis limits:

Pandas:
![pandas](https://cloud.githubusercontent.com/assets/22940964/23305377/447bf42c-fa7d-11e6-8d2f-46a2aba318ef.png)

Matplotlib:
![matplotlib](https://cloud.githubusercontent.com/assets/22940964/23305376/447b70ce-fa7d-11e6-92f6-9ca0d578ffc9.png)

@apcamargo Thanks for the example.

Pandas does a `set_xlim` based on the data when plotting (see https://github.com/pandas-dev/pandas/blob/1400305899d55bee21253952de9f6f0cf245b089/pandas/tools/plotting.py#L1749), although I am not sure why this was initially done. 
I thought this was because matplotlib did not do that automatically when you added more lines to an `ax`, but that does not seem to be the case (unless that was the case for older matplotlib versions).

cc @TomAugspurger @sinhrks do you know the reason(ing) for updating the xlims based on the data?
> I thought this was because matplotlib did not do that automatically when you added more lines to an ax, but that does not seem to be the case (unless that was the case for older matplotlib versions).

I think that's correct. IIRC this was changed in MPL 1.5ish. We could probably skip the `set_xlim` on newer version of matplotlib.
@apcamargo Interested to make a PR to change this? 
If you have other examples where we conflict with the new defaults, that is also always welcome
FYI I think [this](
http://matplotlib.org/users/prev_whats_new/whats_new_1.5.html#interactive-oo-usage)
is the relevant change in matplotlib.

On Mon, Feb 27, 2017 at 11:29 AM, Joris Van den Bossche <
notifications@github.com> wrote:

> @apcamargo <https://github.com/apcamargo> Interested to make a PR to
> change this?
> If you have other examples where we conflict with the new defaults, that
> is also always welcome
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/issues/15495#issuecomment-282789816>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIitOji5ipApVokVXhkUbwKgrJLmAks5rgwfmgaJpZM4MKwlZ>
> .
>

In that whatsnew entry, they talk about 'interactive usage', so will that updating of the axes not be the case in non-interactive usage? That's not fully clear to me, but important for pandas I think if we could remove the `set_xlim`. 
I just tried, but still get updated xlims when not displaying the figure in the notebook but just saving it.
So, to fix this, the line `ax.set_xlim(left, right)` in plotting.py should be deleted. I've tested this change and the issue was fixed.
Should this behavior depend on the Matplotlib version? 
> Should this behavior depend on the Matplotlib version?

Yep. It should be in an `if` block that only applies to older versions. I think something like

```python
if not self.mpl_ge_1_5_0():
    ax.set_xlim(left, right)
```

should work.
We could also only do this for 2.0 and above to be safe (for lower versions it would also not have any impact, since the default xlim would be the same as what pandas now does).

Resolving patch:
diff --git a/pandas/plotting/_core.py b/pandas/plotting/_core.py
--- a/pandas/plotting/_core.py
+++ b/pandas/plotting/_core.py
@@ -31,7 +31,8 @@
 from pandas.util._decorators import Appender
 
 from pandas.plotting._compat import (_mpl_ge_1_3_1,
-                                     _mpl_ge_1_5_0)
+                                     _mpl_ge_1_5_0,
+                                     _mpl_ge_2_0_0)
 from pandas.plotting._style import (mpl_stylesheet, plot_params,
                                     _get_standard_colors)
 from pandas.plotting._tools import (_subplots, _flatten, table,
@@ -969,9 +970,10 @@ def _make_plot(self):
                              **kwds)
             self._add_legend_handle(newlines[0], label, index=i)
 
-            lines = _get_all_lines(ax)
-            left, right = _get_xlim(lines)
-            ax.set_xlim(left, right)
+            if not _mpl_ge_2_0_0():
+                lines = _get_all_lines(ax)
+                left, right = _get_xlim(lines)
+                ax.set_xlim(left, right)
 
     @classmethod
     def _plot(cls, ax, x, y, style=None, column_num=None,
diff --git a/pandas/plotting/_tools.py b/pandas/plotting/_tools.py
--- a/pandas/plotting/_tools.py
+++ b/pandas/plotting/_tools.py
@@ -361,8 +361,8 @@ def _get_xlim(lines):
     left, right = np.inf, -np.inf
     for l in lines:
         x = l.get_xdata(orig=False)
-        left = min(x[0], left)
-        right = max(x[-1], right)
+        left = min(np.nanmin(x), left)
+        right = max(np.nanmax(x), right)
     return left, right
 
 


Test patch:
diff --git a/pandas/tests/plotting/test_datetimelike.py b/pandas/tests/plotting/test_datetimelike.py
--- a/pandas/tests/plotting/test_datetimelike.py
+++ b/pandas/tests/plotting/test_datetimelike.py
@@ -386,9 +386,16 @@ def test_get_finder(self):
 
     @pytest.mark.slow
     def test_finder_daily(self):
-        xp = Period('1999-1-1', freq='B').ordinal
         day_lst = [10, 40, 252, 400, 950, 2750, 10000]
-        for n in day_lst:
+
+        if self.mpl_ge_2_0_0:
+            xpl1 = [7565, 7564, 7553, 7546, 7518, 7428, 7066]
+            xpl2 = [7566, 7564, 7554, 7546, 7519, 7429, 7066]
+        else:
+            xpl1 = xpl2 = [Period('1999-1-1', freq='B').ordinal] * len(day_lst)
+
+        for i, n in enumerate(day_lst):
+            xp = xpl1[i]
             rng = bdate_range('1999-1-1', periods=n)
             ser = Series(np.random.randn(len(rng)), rng)
             _, ax = self.plt.subplots()
@@ -396,6 +403,7 @@ def test_finder_daily(self):
             xaxis = ax.get_xaxis()
             rs = xaxis.get_majorticklocs()[0]
             assert xp == rs
+            xp = xpl2[i]
             vmin, vmax = ax.get_xlim()
             ax.set_xlim(vmin + 0.9, vmax)
             rs = xaxis.get_majorticklocs()[0]
@@ -404,9 +412,16 @@ def test_finder_daily(self):
 
     @pytest.mark.slow
     def test_finder_quarterly(self):
-        xp = Period('1988Q1').ordinal
         yrs = [3.5, 11]
-        for n in yrs:
+
+        if self.mpl_ge_2_0_0:
+            xpl1 = [68, 68]
+            xpl2 = [72, 68]
+        else:
+            xpl1 = xpl2 = [Period('1988Q1').ordinal] * len(yrs)
+
+        for i, n in enumerate(yrs):
+            xp = xpl1[i]
             rng = period_range('1987Q2', periods=int(n * 4), freq='Q')
             ser = Series(np.random.randn(len(rng)), rng)
             _, ax = self.plt.subplots()
@@ -414,6 +429,7 @@ def test_finder_quarterly(self):
             xaxis = ax.get_xaxis()
             rs = xaxis.get_majorticklocs()[0]
             assert rs == xp
+            xp = xpl2[i]
             (vmin, vmax) = ax.get_xlim()
             ax.set_xlim(vmin + 0.9, vmax)
             rs = xaxis.get_majorticklocs()[0]
@@ -422,9 +438,16 @@ def test_finder_quarterly(self):
 
     @pytest.mark.slow
     def test_finder_monthly(self):
-        xp = Period('Jan 1988').ordinal
         yrs = [1.15, 2.5, 4, 11]
-        for n in yrs:
+
+        if self.mpl_ge_2_0_0:
+            xpl1 = [216, 216, 204, 204]
+            xpl2 = [216, 216, 216, 204]
+        else:
+            xpl1 = xpl2 = [Period('Jan 1988').ordinal] * len(yrs)
+
+        for i, n in enumerate(yrs):
+            xp = xpl1[i]
             rng = period_range('1987Q2', periods=int(n * 12), freq='M')
             ser = Series(np.random.randn(len(rng)), rng)
             _, ax = self.plt.subplots()
@@ -432,6 +455,7 @@ def test_finder_monthly(self):
             xaxis = ax.get_xaxis()
             rs = xaxis.get_majorticklocs()[0]
             assert rs == xp
+            xp = xpl2[i]
             vmin, vmax = ax.get_xlim()
             ax.set_xlim(vmin + 0.9, vmax)
             rs = xaxis.get_majorticklocs()[0]
@@ -450,7 +474,11 @@ def test_finder_monthly_long(self):
 
     @pytest.mark.slow
     def test_finder_annual(self):
-        xp = [1987, 1988, 1990, 1990, 1995, 2020, 2070, 2170]
+        if self.mpl_ge_2_0_0:
+            xp = [1986, 1986, 1990, 1990, 1995, 2020, 1970, 1970]
+        else:
+            xp = [1987, 1988, 1990, 1990, 1995, 2020, 2070, 2170]
+
         for i, nyears in enumerate([5, 10, 19, 49, 99, 199, 599, 1001]):
             rng = period_range('1987', periods=nyears, freq='A')
             ser = Series(np.random.randn(len(rng)), rng)
@@ -470,7 +498,10 @@ def test_finder_minutely(self):
         ser.plot(ax=ax)
         xaxis = ax.get_xaxis()
         rs = xaxis.get_majorticklocs()[0]
-        xp = Period('1/1/1999', freq='Min').ordinal
+        if self.mpl_ge_2_0_0:
+            xp = Period('1998-12-29 12:00', freq='Min').ordinal
+        else:
+            xp = Period('1/1/1999', freq='Min').ordinal
         assert rs == xp
 
     def test_finder_hourly(self):
@@ -481,7 +512,10 @@ def test_finder_hourly(self):
         ser.plot(ax=ax)
         xaxis = ax.get_xaxis()
         rs = xaxis.get_majorticklocs()[0]
-        xp = Period('1/1/1999', freq='H').ordinal
+        if self.mpl_ge_2_0_0:
+            xp = Period('1998-12-31 22:00', freq='H').ordinal
+        else:
+            xp = Period('1/1/1999', freq='H').ordinal
         assert rs == xp
 
     @pytest.mark.slow
@@ -665,8 +699,8 @@ def test_mixed_freq_regular_first(self):
         assert idx2.equals(s2.index.to_period('B'))
         left, right = ax2.get_xlim()
         pidx = s1.index.to_period()
-        assert left == pidx[0].ordinal
-        assert right == pidx[-1].ordinal
+        assert left <= pidx[0].ordinal
+        assert right >= pidx[-1].ordinal
 
     @pytest.mark.slow
     def test_mixed_freq_irregular_first(self):
@@ -696,8 +730,8 @@ def test_mixed_freq_regular_first_df(self):
         assert idx2.equals(s2.index.to_period('B'))
         left, right = ax2.get_xlim()
         pidx = s1.index.to_period()
-        assert left == pidx[0].ordinal
-        assert right == pidx[-1].ordinal
+        assert left <= pidx[0].ordinal
+        assert right >= pidx[-1].ordinal
 
     @pytest.mark.slow
     def test_mixed_freq_irregular_first_df(self):
@@ -1211,8 +1245,8 @@ def test_irregular_ts_shared_ax_xlim(self):
 
         # check that axis limits are correct
         left, right = ax.get_xlim()
-        assert left == ts_irregular.index.min().toordinal()
-        assert right == ts_irregular.index.max().toordinal()
+        assert left <= ts_irregular.index.min().toordinal()
+        assert right >= ts_irregular.index.max().toordinal()
 
     @pytest.mark.slow
     def test_secondary_y_non_ts_xlim(self):
@@ -1228,7 +1262,7 @@ def test_secondary_y_non_ts_xlim(self):
         s2.plot(secondary_y=True, ax=ax)
         left_after, right_after = ax.get_xlim()
 
-        assert left_before == left_after
+        assert left_before >= left_after
         assert right_before < right_after
 
     @pytest.mark.slow
@@ -1245,7 +1279,7 @@ def test_secondary_y_regular_ts_xlim(self):
         s2.plot(secondary_y=True, ax=ax)
         left_after, right_after = ax.get_xlim()
 
-        assert left_before == left_after
+        assert left_before >= left_after
         assert right_before < right_after
 
     @pytest.mark.slow
@@ -1278,8 +1312,8 @@ def test_secondary_y_irregular_ts_xlim(self):
         ts_irregular[:5].plot(ax=ax)
 
         left, right = ax.get_xlim()
-        assert left == ts_irregular.index.min().toordinal()
-        assert right == ts_irregular.index.max().toordinal()
+        assert left <= ts_irregular.index.min().toordinal()
+        assert right >= ts_irregular.index.max().toordinal()
 
     def test_plot_outofbounds_datetime(self):
         # 2579 - checking this does not raise
@@ -1294,9 +1328,14 @@ def test_format_timedelta_ticks_narrow(self):
         if is_platform_mac():
             pytest.skip("skip on mac for precision display issue on older mpl")
 
-        expected_labels = [
-            '00:00:00.00000000{:d}'.format(i)
-            for i in range(10)]
+        if self.mpl_ge_2_0_0:
+            expected_labels = [''] + [
+                '00:00:00.00000000{:d}'.format(2 * i)
+                for i in range(5)] + ['']
+        else:
+            expected_labels = [
+                '00:00:00.00000000{:d}'.format(i)
+                for i in range(10)]
 
         rng = timedelta_range('0', periods=10, freq='ns')
         df = DataFrame(np.random.randn(len(rng), 3), rng)
@@ -1312,17 +1351,32 @@ def test_format_timedelta_ticks_wide(self):
         if is_platform_mac():
             pytest.skip("skip on mac for precision display issue on older mpl")
 
-        expected_labels = [
-            '00:00:00',
-            '1 days 03:46:40',
-            '2 days 07:33:20',
-            '3 days 11:20:00',
-            '4 days 15:06:40',
-            '5 days 18:53:20',
-            '6 days 22:40:00',
-            '8 days 02:26:40',
-            ''
-        ]
+        if self.mpl_ge_2_0_0:
+            expected_labels = [
+                '',
+                '00:00:00',
+                '1 days 03:46:40',
+                '2 days 07:33:20',
+                '3 days 11:20:00',
+                '4 days 15:06:40',
+                '5 days 18:53:20',
+                '6 days 22:40:00',
+                '8 days 02:26:40',
+                '9 days 06:13:20',
+                ''
+            ]
+        else:
+            expected_labels = [
+                '00:00:00',
+                '1 days 03:46:40',
+                '2 days 07:33:20',
+                '3 days 11:20:00',
+                '4 days 15:06:40',
+                '5 days 18:53:20',
+                '6 days 22:40:00',
+                '8 days 02:26:40',
+                ''
+            ]
 
         rng = timedelta_range('0', periods=10, freq='1 d')
         df = DataFrame(np.random.randn(len(rng), 3), rng)
diff --git a/pandas/tests/plotting/test_frame.py b/pandas/tests/plotting/test_frame.py
--- a/pandas/tests/plotting/test_frame.py
+++ b/pandas/tests/plotting/test_frame.py
@@ -304,6 +304,29 @@ def test_unsorted_index(self):
         rs = Series(rs[:, 1], rs[:, 0], dtype=np.int64, name='y')
         tm.assert_series_equal(rs, df.y)
 
+    def test_unsorted_index_lims(self):
+        df = DataFrame({'y': [0., 1., 2., 3.]}, index=[1., 0., 3., 2.])
+        ax = df.plot()
+        xmin, xmax = ax.get_xlim()
+        lines = ax.get_lines()
+        assert xmin <= np.nanmin(lines[0].get_data()[0])
+        assert xmax >= np.nanmax(lines[0].get_data()[0])
+
+        df = DataFrame({'y': [0., 1., np.nan, 3., 4., 5., 6.]},
+                       index=[1., 0., 3., 2., np.nan, 3., 2.])
+        ax = df.plot()
+        xmin, xmax = ax.get_xlim()
+        lines = ax.get_lines()
+        assert xmin <= np.nanmin(lines[0].get_data()[0])
+        assert xmax >= np.nanmax(lines[0].get_data()[0])
+
+        df = DataFrame({'y': [0., 1., 2., 3.], 'z': [91., 90., 93., 92.]})
+        ax = df.plot(x='z', y='y')
+        xmin, xmax = ax.get_xlim()
+        lines = ax.get_lines()
+        assert xmin <= np.nanmin(lines[0].get_data()[0])
+        assert xmax >= np.nanmax(lines[0].get_data()[0])
+
     @pytest.mark.slow
     def test_subplots(self):
         df = DataFrame(np.random.rand(10, 3),
@@ -735,14 +758,14 @@ def test_line_lim(self):
         ax = df.plot()
         xmin, xmax = ax.get_xlim()
         lines = ax.get_lines()
-        assert xmin == lines[0].get_data()[0][0]
-        assert xmax == lines[0].get_data()[0][-1]
+        assert xmin <= lines[0].get_data()[0][0]
+        assert xmax >= lines[0].get_data()[0][-1]
 
         ax = df.plot(secondary_y=True)
         xmin, xmax = ax.get_xlim()
         lines = ax.get_lines()
-        assert xmin == lines[0].get_data()[0][0]
-        assert xmax == lines[0].get_data()[0][-1]
+        assert xmin <= lines[0].get_data()[0][0]
+        assert xmax >= lines[0].get_data()[0][-1]
 
         axes = df.plot(secondary_y=True, subplots=True)
         self._check_axes_shape(axes, axes_num=3, layout=(3, 1))
@@ -751,8 +774,8 @@ def test_line_lim(self):
             assert not hasattr(ax, 'right_ax')
             xmin, xmax = ax.get_xlim()
             lines = ax.get_lines()
-            assert xmin == lines[0].get_data()[0][0]
-            assert xmax == lines[0].get_data()[0][-1]
+            assert xmin <= lines[0].get_data()[0][0]
+            assert xmax >= lines[0].get_data()[0][-1]
 
     def test_area_lim(self):
         df = DataFrame(rand(6, 4), columns=['x', 'y', 'z', 'four'])
@@ -763,8 +786,8 @@ def test_area_lim(self):
             xmin, xmax = ax.get_xlim()
             ymin, ymax = ax.get_ylim()
             lines = ax.get_lines()
-            assert xmin == lines[0].get_data()[0][0]
-            assert xmax == lines[0].get_data()[0][-1]
+            assert xmin <= lines[0].get_data()[0][0]
+            assert xmax >= lines[0].get_data()[0][-1]
             assert ymin == 0
 
             ax = _check_plot_works(neg_df.plot.area, stacked=stacked)
diff --git a/pandas/tests/plotting/test_series.py b/pandas/tests/plotting/test_series.py
--- a/pandas/tests/plotting/test_series.py
+++ b/pandas/tests/plotting/test_series.py
@@ -102,23 +102,23 @@ def test_ts_line_lim(self):
         ax = self.ts.plot(ax=ax)
         xmin, xmax = ax.get_xlim()
         lines = ax.get_lines()
-        assert xmin == lines[0].get_data(orig=False)[0][0]
-        assert xmax == lines[0].get_data(orig=False)[0][-1]
+        assert xmin <= lines[0].get_data(orig=False)[0][0]
+        assert xmax >= lines[0].get_data(orig=False)[0][-1]
         tm.close()
 
         ax = self.ts.plot(secondary_y=True, ax=ax)
         xmin, xmax = ax.get_xlim()
         lines = ax.get_lines()
-        assert xmin == lines[0].get_data(orig=False)[0][0]
-        assert xmax == lines[0].get_data(orig=False)[0][-1]
+        assert xmin <= lines[0].get_data(orig=False)[0][0]
+        assert xmax >= lines[0].get_data(orig=False)[0][-1]
 
     def test_ts_area_lim(self):
         _, ax = self.plt.subplots()
         ax = self.ts.plot.area(stacked=False, ax=ax)
         xmin, xmax = ax.get_xlim()
         line = ax.get_lines()[0].get_data(orig=False)[0]
-        assert xmin == line[0]
-        assert xmax == line[-1]
+        assert xmin <= line[0]
+        assert xmax >= line[-1]
         tm.close()
 
         # GH 7471
@@ -126,8 +126,8 @@ def test_ts_area_lim(self):
         ax = self.ts.plot.area(stacked=False, x_compat=True, ax=ax)
         xmin, xmax = ax.get_xlim()
         line = ax.get_lines()[0].get_data(orig=False)[0]
-        assert xmin == line[0]
-        assert xmax == line[-1]
+        assert xmin <= line[0]
+        assert xmax >= line[-1]
         tm.close()
 
         tz_ts = self.ts.copy()
@@ -136,16 +136,16 @@ def test_ts_area_lim(self):
         ax = tz_ts.plot.area(stacked=False, x_compat=True, ax=ax)
         xmin, xmax = ax.get_xlim()
         line = ax.get_lines()[0].get_data(orig=False)[0]
-        assert xmin == line[0]
-        assert xmax == line[-1]
+        assert xmin <= line[0]
+        assert xmax >= line[-1]
         tm.close()
 
         _, ax = self.plt.subplots()
         ax = tz_ts.plot.area(stacked=False, secondary_y=True, ax=ax)
         xmin, xmax = ax.get_xlim()
         line = ax.get_lines()[0].get_data(orig=False)[0]
-        assert xmin == line[0]
-        assert xmax == line[-1]
+        assert xmin <= line[0]
+        assert xmax >= line[-1]
 
     def test_label(self):
         s = Series([1, 2])
@@ -289,6 +289,16 @@ def test_irregular_datetime(self):
         ax.set_xlim('1/1/1999', '1/1/2001')
         assert xp == ax.get_xlim()[0]
 
+    def test_unsorted_index_xlim(self):
+        ser = Series([0., 1., np.nan, 3., 4., 5., 6.],
+                     index=[1., 0., 3., 2., np.nan, 3., 2.])
+        _, ax = self.plt.subplots()
+        ax = ser.plot(ax=ax)
+        xmin, xmax = ax.get_xlim()
+        lines = ax.get_lines()
+        assert xmin <= np.nanmin(lines[0].get_data(orig=False)[0])
+        assert xmax >= np.nanmax(lines[0].get_data(orig=False)[0])
+
     @pytest.mark.slow
     def test_pie_series(self):
         # if sum of values is less than 1.0, pie handle them as rate and draw