DEPR: Change boxplot return_type kwarg
State: closed

Part of https://github.com/pydata/pandas/issues/6581
Deprecation started in https://github.com/pydata/pandas/pull/7096

Changes the default value of `return_type` in DataFrame.boxplot
and DataFrame.plot.box from None to 'axes'.


Labels:
name: Visualization
description: plotting
name: Deprecate
description: Functionality to remove in pandas

Comments:
Tests still need to be updated

@TomAugspurger pls update

@TomAugspurger can you update

Still working on this. It looks I didn't properly deprecate the `df.boxplot(by=)` (or my docs are wrong). According to the [docs](http://pandas-docs.github.io/pandas-docs-travis/visualization.html) `by=` is supposed to return an ordered dict of whatever `return_type` is. But it's returning a numpy array instead. I'll work on this more tonight, sorry.

@TomAugspurger if you can get to this would be great, otherwise push to 0.19 :<

I’ll update tonight or tomorrow morning. Might want to push anyway since I’ve found a discrepancy between our documented behavior and actual behavior for `groupby.boxplot`

> On Feb 15, 2016, at 2:22 PM, Jeff Reback notifications@github.com wrote:
> 
> @TomAugspurger https://github.com/TomAugspurger if you can get to this would be great, otherwise push to 0.19 :<
> 
> —
> Reply to this email directly or view it on GitHub https://github.com/pydata/pandas/pull/12216#issuecomment-184374754.

@TomAugspurger ok, we do have a little time, and this has been around for a while. I'll keep it here and move later (or you decide  its a real stumbling block)

Well, here's a not so great thing. I think we'll have to keep the behavior of `return_type=None` not being the same as `return_type='axes'` when `by` is not None.

``` python
# before
a = df.groupby('d').boxplot()  # OD[dict]  warned
b = df.boxplot()               # dict, warned
c = df.boxplot(by='d')         # [axes], didn't warn
c = df.boxplot(by='d', return_type='axes') # OD[axes]
```

and

``` python
# after
a = df.groupby('d').boxplot()  # OD[axes]
b = df.boxplot()               # axes
c = df.boxplot(by='d')         # [axes]
c = df.boxplot(by='d', return_type='axes') # OD[axes]
```

I had hoped to just make the default 'axes', but since I din't warn in the 3rd example, I don't think we can change that. Are we OK with that? Here's the updated docs:

``` rst
Plot functions return scalar or arrays of :class:`matplotlib Axes <matplotlib.axes.Axes>`.
In ``boxplot``, the return type can be controlled by the ``return_type``, keyword. The valid choices are ``{"axes", "dict", "both"}``.
``'axes'`` returns a single matplotlib axes.
``'dict'`` returns a dict of matplotlib artists, similar to the matplotlib boxplot function.
``'both'`` returns a named tuple of axes and dicts.
When ``by`` is not None, you get back an ``OrderedDict`` of whatever ``return_type`` is, unless ``return_type`` is just ``None``, in which case you get back an array of axes.
```

(maybe not fully on topic, but) Is there actually a reason we are using OrderDicts here ? For example why not a series? This would give both the key access of an OrderDict as the integer indexing of a an array (and seems more natural in pandas)

On returning a series: that is also what `df.groupby('d').plot()` does, so it seems logical that `df.groupby('d').boxplot()` does the same?

But more on topic: wouldn't it be possible to raise that warning now for the third case? (but keeps its behaviour as you proposed) So we can maybe in a future release align the behaviour fully.

@TomAugspurger lgtm. just some small doc comments. ping when updated.

lgtm. 

@jorisvandenbossche ?

Forgot to send the post I had started last night,

I could deprecate the third case (`df.boxplot(by='d')` -> [axes]), but at this point I feel a bit defeated by our boxplot implementations, plural, since we have 3(!), and none is equivalent to any other. It'd be nice to have them all aligned but I kind of feel like I've done enough API breaking changes here, especially when people should just use seaborn.

If we aren't warning on that case, this is good to go.

@TomAugspurger About my comment on using Series instead of OrderedDict, wouldn't that be a possible way to just change it?

To be clear: the third case is `c = df.boxplot(by='d')`, it currently gives you an array of axes (if there are multiple columns) and ideally we would like that it returns an OrderedDict of axes to be consistent with a default of 'axes'. 
Suppose we would return a Series of axes, wouldn't that cover most of the use cases for both cases? Or would a lot of people relied on the fact it was an actual numpy array it returned? I would think most use is accessing one of the axes with `c[0]`, which would still work with a Series as return value.

@jorisvandenbossche yes, that makes much more sense. It's not clear to me the best way to get to there though. Do we return a `Series` for `c = df.boxplot(by='d')` now, or have another FutureWarning?

To answer that question, we should have an idea to which extent it would break people's code. And that is of course difficult to guess, but I think in the case of `c = df.boxplot(by='d')` return an array of axes, if you do something with that return value, it will typically be accessing one of the axes to modify the subplot further with matplotlib. That case would keep working with returning a Series. 
Of course code like `isintance(c, np.ndarray)` would break, but would people do that?

The questions is also, do we return Series for the cases where now already OrderedDicts are returned? Again, the most logical usecase (accessing a value by its key) keeps working, but things might not.

My thinking was similar, the most common operations upon receiving the array or dict of axes is to 1. do nothing, 2. select individual axes by position or key. Thankfully Series handles both of those.

I'll update tonight to return Series of `return_type` wherever we returned an OrderedDict / numpy array before. It'll be nice to have this in a somewhat consistent and stable state.

I think returning `Series` is OK if plot method returns `Axes`. Needs to consider to change other plots than boxplot though.

But for subplots, we receive a dict (per Axes) of dict (per Artist kinds) which contains list (per Artists). Are we using `Series` to store this kind of non-structural data?

```
df = pd.DataFrame({'A': [1, 2] * 20,
                   'B': np.random.randn(40),
                   'C': np.random.randn(40)})
df.boxplot(by='A', return_type='dict')
# OrderedDict([('B',
#               {'boxes': [<matplotlib.lines.Line2D at 0x164cb748>,
#                 <matplotlib.lines.Line2D at 0x163c9e48>],
#                'caps': [<matplotlib.lines.Line2D at 0x164b75f8>,
#                 <matplotlib.lines.Line2D at 0x163ee898>,
#                 <matplotlib.lines.Line2D at 0x162d8d68>,
#                ...
```

@sinhrks  Correct, in the last case we return a dict.

http://nbviewer.jupyter.org/gist/TomAugspurger/9b0fb2309c7ac17ecf15

Thoughts? I think Series are dict-like enough to cover the common use-cases, selecting individual items for modification, or iteration through and modifying each.

I'm going to read through your groupby plotting enhancements PR next, to see how this would interact with it. I hadn't thought about changing the return type for all groupby.plot methods.

looks ok to me. @jorisvandenbossche ?

@TomAugspurger @jorisvandenbossche status on this?

I think the concern is about returning Series for `Groupby.boxplot` and but for any of the other Groubpy plotting methods.

why? previously these returned an `OrderedDict` or `numpy` array right?

What is the objection?

@TomAugspurger @jorisvandenbossche are we changing this? or defer to 0.19.0

Sorry, I was AWOL all saturday and yesterday.

I don't feel great about changing the groupby.boxplot() to return a Series without changing `groupby.plot` as well. We originally put the boxplot deprecation in place to make in more consistent with the rest of the plotting stuff, doesn't sense to introduce more inconsistencies in the process :)

I really need to review @sinhrks' groupby plotting changes, but I won't have a chance for at least a week. I would suggest pushing till 0.19 and doing all the groupby plotting changes then.

@TomAugspurger ok, moved to 0.19.0. Though if you do have something before next week that makes sense. ping.

Master is now targeted for 0.19.0, so we can revisit this

@TomAugspurger ?

## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/12216?src=pr) is 85.25% (diff: 37.50%)

> Merging [#12216](https://codecov.io/gh/pydata/pandas/pull/12216?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #12216   diff @@
==========================================
  Files           139        139          
  Lines         50501      50502     +1   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          43058      43057     -1   
- Misses         7443       7445     +2   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [4488f18...60c1c26](https://codecov.io/gh/pydata/pandas/compare/4488f18076226894c0f07be67ee700663e32623e...60c1c262ffc6336809b5b9dc2ef9ab0b0fd3bade?src=pr)