BUG: Fixes plotting with nullable integers (#32073)
State: closed

- [x] closes #32073
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

Similar to #32410 this tries to fix plotting with nullable integer extension data types by casting them to float before passing on to matplotlib.

Labels:
name: Visualization
description: plotting
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
@MarcoGorelli I'm not sure where to put the whatsnew entry. `doc/source/whatsnew/v1.1.0.rst`? Under "Other enhancements"?
> @MarcoGorelli I'm not sure where to put the whatsnew entry. `doc/source/whatsnew/v1.1.0.rst`? Under "Other enhancements"?

Thanks for your PR! I think this should go under the _plotting_ section of the bug fixes
I've added the `safe_convert_to_ndarray` that seems to work for both `GroupBy.quantile`, but I've got some doubts about intended behavior:
- `GroupBy.quantile` doesn't seem to handle `datetime64tz` dtypes, given that `is_datetime64_dtype` returns False for these types, see https://github.com/pandas-dev/pandas/blob/f5b2e5a9a2431165b1e8002219c5d3962d7bc920/pandas/core/groupby/groupby.py#L2060, I'm not sure if it should? If so it seems to be missing tests.
- Previously datetime (non-extension type) was converted to float in `GroupBy.quantile`, now they are just represented as datetime64 array but that doesn't seem to break any tests. I think I should figure out if that is just because of missing tests, or if it is actually working like this, but I'm not sure how to define the quantile of a set of dates.
- There is a case in the if statement in `safe_convert_to_ndarray` where the input dtype is an extension dtype, but not one of the known ones. Can this happen? If so, should it raise something like NotImplementedError, or should it try to convert like now with the possibility that you get an array of object dtype back?

Also any feedback on the docstring and typehints would be appreciated. Because of the many types that could be the input I wasn't sure what types to add there so I only added a hint for output type.
Ah I don't pass my own test it seems, was sure I reran it after the last change. :facepalm: The problem here is that for a Series object it'd need to call `Series.dt.tz_localize` or it will be called on the index instead of on the values, while for a `DatetimeIndex` it is `DatetimeIndex.tz_localize`.

Perhaps we can simplify it by making `safe_convert_to_ndarray` take only `pandas.array` input and have whatever calls it take care of extracting the data it wants to convert?
Mypy gives the following error:
```
pandas/core/dtypes/cast.py:1777: error: Item "ExtensionArray" of "Union[ExtensionArray, Index]" has no attribute "tz_localize"
pandas/core/dtypes/cast.py:1777: error: Item "Index" of "Union[ExtensionArray, Index]" has no attribute "tz_localize"
pandas/core/dtypes/cast.py:1777: error: Item "Index" of "Union[Any, Index]" has no attribute "tz_localize"
```
I could fix this by defering this to another function that only takes care of the `datetime64tz` conversion, but how do I type that for specific dtype?
The version of `safe_convert_to_ndarray` I've implemented now seems to work for the intended plotting bugfix, and for the `GroupBy.quantile` function. However, the more I worked on this the more I think that this shouldn't be it's own function:
- The real way to safely convert to `ndarray` should be the `to_numpy` method.
- `GroupBy.quantile` and `MPLPlot._compute_plot_data` have different motivations to subvert this default path and handle extension dtypes in an unsafe way (i.e. changing the data such as bool -> float).
- cast.py already seems to have quite some near-duplicate functionality and adding this function feels like adding more noise on top of that.

How I see it there are 2 options:
- A: we fix the type issue with the current code, maybe add a warning to docstring to use `to_numpy` instead, and merge this.
- B: we revert to the fix where we only touch `MPLPlot._compute_plot_data`. This has my preference given the above.

Regarding the dates quantile; in both this version and master quantile of `[2001, NaT, 2003]` is `1839-05-13 12:06:21.572612096`. I'll see if there is already an issue open for how `GroupBy.quantile` handles date series with `NaT` or else I'll open a new one for that. 

EDIT: the quantile issue seems too close to #33168 and #29485 to open a new issue for it.
Given that the requested direction to have the same code path shared between the plotting function and `GroupBy.quantile` doesn't sit well by me, I'm going to close this PR and unassign myself from the issue so that somebody else can pick it up that can either retry a minimal fix for the bug or that wants to do a bigger rewrite to clean up some of the issues with operations on nullable types.