BUG: fixes #12405 by eliding values index by NaT in MPLPlot._get_xticks
State: closed

- [ ] closes #12405
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Proposed solution: elide rows where x value would be `NaT` in [MPLPlot._get_xticks](https://github.com/pandas-dev/pandas/blob/7f5a45c1b388b3f7f309f82bfa0733b7b9980c3a/pandas/tools/plotting.py#L1328) (which seems like a minor misnomer: its getting x values, not what will ultimately be ticks).

Reordering occurs in MPLPlot._get_xticks and it has a block that is conditional on `use_index` and `is_datetype`, so seems like a reasonable place to elide the values.  Mutation of `self.data` already occurs there (reordering) and a read through suggests nothing has saved info related to `self.data`'s shape before this point.


Labels:
name: Bug
name: Datetime
description: Datetime data dtype
name: Visualization
description: plotting

Comments:
Will add a test in [test_datetimelike.py](https://github.com/pandas-dev/pandas/blob/7f5a45c1b388b3f7f309f82bfa0733b7b9980c3a/pandas/tests/plotting/test_datetimelike.py) but it will probably just repeat the sorting and eliding that occurs in the `use_index`/`is_datetype` branch of _get_xticks.  Alternatively, could check that all the xvalues are within the nan{min,max} bounds of the original index.

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=h1) Report
> Merging [#14540](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0f55de1b334131dd0577b91eb061dcb8bc65ef78?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/14540/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #14540      +/-   ##
==========================================
+ Coverage   90.41%   90.41%   +<.01%     
==========================================
  Files         161      161              
  Lines       50997    50999       +2     
==========================================
+ Hits        46107    46109       +2     
  Misses       4890     4890
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.24% <100%> (ø)` | :arrow_up: |
| #single | `40.19% <50%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `81.89% <100%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=footer). Last update [0f55de1...eae5c0e](https://codecov.io/gh/pandas-dev/pandas/pull/14540?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

i would add some edge cases
e.g. all nans, single nan first, and single nan last

@jreback , I think all NaTs should be a failure:  `pd.Series().plot()` (data is all NaTs and we use the elide NaTs methodology) results in `TypeError: Empty 'DataFrame': no numeric data to plot`.  Perhaps we should alert that we've elided values and that's why there is no data left.

Otherwise, all cases are the same since `sort_values()` results in the same series regardless of number of non-NaTs, barring all NaTs.

incidentally, re @jorisvandenbossche's comment in the orignal bug report, matplotlib complains if you try to plot with `NaT` as an xvalue in a new plot.  But once you're plotting on an axis that has already plotted with a datetime on the index, it accepts `NaT` values (as the min datetime value).

pls add a release note, 0.19.1 bug fixes is fine.

The way matplotlib deals with missing values (for numerical data) is to leave a gap in the line. So matplotlib can actually deal with nan values. So therefore, we could also opt for this behaviour, rather than just removing those NaN values from the data before plotting.

@jorisvandenbossche , we [sort the index](https://github.com/pandas-dev/pandas/blob/v0.19.0/pandas/tools/plotting.py#L1328) before we plot the data.  Where would we insert the `NaT` index value?

@tsdlovell can you rebase / update
status of this?
@tsdlovell ping!
@tsdlovell can you rebase?
@jreback , I had some issues rebase'ing onto master.  I ended up just applying the changes manually to the current head and creating a new branch: [fix-gh12405-2](https://github.com/tsdlovell/pandas/tree/fix-gh12405-2).

Can I just push that to the original fork branch for this PR or will that mess something up?
yes i think u can just force push it and will work
@TomAugspurger how does this look?
@TomAugspurger , let me know if there's something I still need to do.
Thanks!