Fix automatic xlims in line plots
State: closed

 - [x] closes #11310, closes #11471, closes #15495
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [x] whatsnew entry

Labels:
name: Visualization
description: plotting

Comments:
So this breaks 18 Tests:
1. test_finder_annual - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_finder_daily - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_finder_hourly - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_finder_minutely - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_finder_monthly - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_finder_quarterly - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_format_timedelta_ticks_narrow - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_format_timedelta_ticks_wide - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_irregular_ts_shared_ax_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_mixed_freq_regular_first - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_mixed_freq_regular_first_df - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_secondary_y_irregular_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_secondary_y_non_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_secondary_y_regular_ts_xlim - pandas.tests.plotting.test_datetimelike.TestTSPlot
1. test_area_lim - pandas.tests.plotting.test_frame.TestDataFramePlots
1. test_line_lim - pandas.tests.plotting.test_frame.TestDataFramePlots
1. test_ts_area_lim - pandas.tests.plotting.test_series.TestSeriesPlots
1. test_ts_line_lim - pandas.tests.plotting.test_series.TestSeriesPlots

These tests hardcode expectations from previous MPL defaults. It is only the second commit (which allows MPL to apply its new defaults) that breaks the tests. Do they need a second set of expected results depending on which MPL version is used? Or can we simply update the expected results to what current MPL produces?

The existing tests do not yet cover the case of unsorted x axis (the subject of the first commit). This will need new tests. Is a single test sufficient or do we need different instances for datetimelike/DataFrame/Series?
Thanks for this. Your changes look correct.

> Do they need a second set of expected results depending on which MPL version is used? Or can we simply update the expected results to what current MPL produces?

I think we test on an old matplotlib (1.3.1) on python 2.7, and the most recent matplotlib otherwise. So yes, ideally we would cover both. If it's excessively difficult, you can raise skiptests for older matplotlibs where nescessary.

> Is a single test sufficient or do we need different instances for datetimelike/DataFrame/Series?

I think just a test for line plots covering a few cases

- `Series.plot` with unsorted index
- `DataFrame.plot` with unsorted index
- `DataFrame.plot(x='x', y='y')` with unsorted `x`
 
should be sufficient.

We'll also need a release note. If you rebase on master, there should be a `docs/source/whatsnew/v0.20.3.txt` you can add this to.
Actually, it's probably best to do this for 0.21.0, not 0.20.3, since the xlim will be now following matplotlib 2's.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=h1) Report
> Merging [#16600](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ceaf85233d434a226b23f891465b4abfdc602e46?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `16.66%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16600/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16600      +/-   ##
==========================================
+ Coverage   90.93%   90.95%   +0.01%     
==========================================
  Files         161      161              
  Lines       49266    49267       +1     
==========================================
+ Hits        44801    44810       +9     
+ Misses       4465     4457       -8
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.71% <16.66%> (+0.01%)` | :arrow_up: |
| #single | `40.22% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_tools.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL190b29scy5weQ==) | `72.92% <0%> (-6.08%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `82.38% <25%> (+0.46%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.2% <0%> (+1.96%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=footer). Last update [ceaf852...ce6bcb1](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=h1) Report
> Merging [#16600](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cbb090fac6a39d0e687a01b2ad1b7c136f5d92fc?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `16.66%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16600/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16600      +/-   ##
==========================================
- Coverage   91.22%    91.2%   -0.02%     
==========================================
  Files         163      163              
  Lines       49624    49625       +1     
==========================================
- Hits        45269    45260       -9     
- Misses       4355     4365      +10
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.99% <16.66%> (-0.01%)` | :arrow_down: |
| #single | `40.19% <0%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_tools.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL190b29scy5weQ==) | `72.92% <0%> (-6.08%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `82.52% <25%> (-0.21%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.77% <0%> (-0.1%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.2% <0%> (+1.96%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=footer). Last update [cbb090f...15eae82](https://codecov.io/gh/pandas-dev/pandas/pull/16600?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

So the codecov decrease in plotting/_tool.py is actually the point of one of the patches &ndash; xlims should only be adjusted with older matplotlib.

If you prefer this PR to be split into two (one for wrong xlims and one for MPL xlim defaults), or if you have other comments, let me know.
> So the codecov decrease...

That may be inaccurate. I don't recall which versions of matplotlib are installed on the runs that report code coverage. It should still be being tested on other runs.
can you rebase / update according to comments
I think you'll just need a rebase. IIRC, things looked pretty good last time I looked.
rebased
can you rebase / update once again
rebased
Thanks @nmartensen! (sorry about the extra rebase)