ENH: plot method accessors
State: closed

Fixes #9124

This PR adds plotting sub-methods like `df.plot.scatter()` as an alternative to using `df.plot(kind='scatter')`.

I've added meaningful function signatures and documentation for a few of these methods, but I would greatly appreciate help to fill in the rest -- this is a lot of documentation to assemble/reconstruct! The entire point of this PR, of course, is to have better introspection and docstrings.

Todo list:
- [x] Basic docstrings/signatures
  - [x] `area`
  - [x] `line`
  - [x] `bar`
  - [x] `barh`
  - [x] `box`
  - [x] `hexbin`
  - [x] `hist`
  - [x] `kde`/`density`
  - [x] `pie`
  - [x] `scatter`
- [x] Write tests for the methods
- [x] Fix groupby plots (tests currently failing)
- [x] Plotting docs
- [x] API docs
- [x] Release notes


Labels:
name: Docs
name: Visualization
description: plotting
name: API Design

Comments:
I'll do a PR against your branch for the rest of the docstrings this Saturday unless someone else gets there first.  

Thanks for doing this!

Current policy is nice for docstring thus for users, but I feel this makes consistent maintenance little difficult, because maintainer have to care all the separated functions. I think it is better to have a table which organizes what options are available on every plotting function. Maybe on Wiki?

I'm willing to do, but it takes a while as each function have options not listed in the signature, for example,
https://github.com/pydata/pandas/blob/master/pandas/tools/plotting.py#L1428

@sinhrks Agreed, this is going to be tricky. For a first draft (this PR), I think use of `**kwargs` for some options is unavoidable (every function will have `**kwargs` in the signature). Eventually, we can try to stop using it but practically that may not be possible.

I hope to get the docstrings done tonight / tomorrow.

This is a rough list of which `kinds` accept which kws. It should be conservative, I think we can prune a bit further.

I'm going to try to come up with a hierarchy later, to factor out common kws.

# line
- data
- x
- y
- use_index
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- position
- table
- yerr
- xerr
- stacked
- sort_columns
- kwds

# bar / barh
- data
- x
- y
- use_index
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- position
- table
- yerr
- xerr
- stacked
- sort_columns
- kwds

# hist
- column
- by
- ~~x~~
- ~~y~~
- ~~use_index~~
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- table
- stacked
- sort_columns
- kwds

# box
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- ~~logy~~ : isn't computed correctly.
- rot
- xlim
- ylim
- fontsize
- figsize
- table
- kwds

# kde / density
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- colorbar
- table
- kwds

# area
- x
- y
- use_index
- ax
- subplots
- sharex
- sharey
- layout
- grid
- title
- legend
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- table
- stacked
- sort_columns
- kwds

# pie
- y
- ax
- subplots
- layout
- grid
- title
- legend
- rot
- xlim
- ylim
- fontsize
- figsize
- colormap
- table
- kwds

# scatter
- x
- y
- ax
- grid
- title
- ~~legend~~
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- colorbar
- position
- table
- yerr
- xerr
- kwds

# hexbin
- x
- y
- ax
- grid
- title
- ~~legend~~
- logx
- logy
- loglog
- xticks
- yticks
- rot
- xlim
- ylim
- fontsize
- secondary_y
- mark_right
- figsize
- style
- colormap
- colorbar
- position
- table
- yerr
- xerr
- gridsize
- C
- reduce_C_function
- kwds

Unfortunately, the `__call__` signature is included in the docstring of the `.plot` method.

``` python
In [9]: df.plot?
Type:        FramePlotMethods
String form: <pandas.tools.plotting.FramePlotMethods object at 0x1087b9860>
File:        /Users/tom/miniconda3/envs/py3/lib/python3.4/site-packages/pandas/pandas/tools/plotting.py
Definition:  df.plot(self, x=None, y=None, kind='line', ax=None, subplots=False, sharex=True, sharey=False, layout=None, figsize=None, use_index=True, title=None, grid=None, legend=True, style=None, logx=False, logy=False, loglog=False, xticks=None, yticks=None, xlim=None, ylim=None, rot=None, fontsize=None, colormap=None, table=False, yerr=None, xerr=None, secondary_y=False, sort_columns=False, **kwds)
Docstring:
DataFrameFrame plotting accessor

Examples
--------
>>> df.plot.line()
>>> df.plot.scatter('x', 'y')
>>> df.plot.hexbin()

The plotting methods can also be accessed by directly calling the
accessor with the ``kind`` argument:
``df.plot(kind='line')`` is equivalent to ``df.plot.line()``
Call def:    df.plot(x=None, y=None, kind='line', ax=None, subplots=False, sharex=True, sharey=False, layout=None, figsize=None, use_index=True, title=None, grid=None, legend=True, style=None, logx=False, logy=False, loglog=False, xticks=None, yticks=None, xlim=None, ylim=None, rot=None, fontsize=None, colormap=None, table=False, yerr=None, xerr=None, secondary_y=False, sort_columns=False, **kwds)
```

I'm not sure this is avoidable.

I did some work on this a while back. I'll see how it looks and maybe open up a pull against your branch.

@shoyer nice start....for 0.17.0!

@jreback @TomAugspurger Yes, let's get this in for 0.17. Even it's just our best guess for the full doc strings, we can fill in the long tail of options later.

OK, I have been updating this. To keep the complexity under control, so far all I've done is add the kind specific arguments to each plotting method. So at least it's clear which arguments are relevant for each kind, and these can be used positionally.

I would rather get this out in its somewhat incomplete form and add the long tail of generic keyword arguments (as documented by @TomAugspurger above) later. That should be pretty straightforward at this point, if someone can figure out a way to consolidate the docstring and function signature generation without going insane.

Here's what the new API docs look like (edit: updated to latest function signatures):

![screen shot 2015-05-29 at 2 43 32 pm](https://cloud.githubusercontent.com/assets/1217238/7893293/1d42fa66-0611-11e5-8bfc-03fd95c57d8e.png)

It looks pretty good, except we get `alias of FramePlotMethods` as the API documentation for `DataFrame.plot`, without any docstring or generated page. @jorisvandenbossche do you know how to fix this? I can dig in if necessary.

@jorisvandenbossche any ideas for fixing the doc generation? Even just a pointer to relevant docs would Sphinx would be helpful... my blind experiments did not have much luck.

@shoyer I took a look at this, and I am afraid I don't directly see a way to solve this. 

The problem is that `DataFrame/Series.plot` is no longer a method, but a class.
So also when you do `df.plot?`, you get the class docstring. 
Just this fact is also already a bit annoying, as this will not show the signature as does `df.plot?` now.

To get the docstring page build for `Series.plot`, you have to use another template (see how it is done for cat/str/dt):

```
.. autosummary::
       :toctree: generated/
       :template: autosummary/accessor.rst

       Series.plot
```

But that does not solve the issue of above of the wrong docstring/wrong signature. The wrong docstring can of course easily be fixed by changing the docstring of the PlotMethods class. But the help of `df.plot?` will still look like that one of a class

Well, in the worst case scenario we could generate the rst on the API docs for .plot manually? That wouldn't be too bad

On Thu, Jun 25, 2015 at 5:29 AM, Joris Van den Bossche
notifications@github.com wrote:

> @shoyer I took a look at this, and I am afraid I don't directly see a way to solve this. 
> The problem is that `DataFrame/Series.plot` is no longer a method, but a class.
> So also when you do `df.plot?`, you get the class docstring. 
> Just this fact is also already a bit annoying, as this will not show the signature as does `df.plot?` now.
> To get the docstring page build for `Series.plot`, you have to use another template (see how it is done for cat/str/dt):
> 
> ```
> .. autosummary::
>        :toctree: generated/
>        :template: autosummary/accessor.rst
>        Series.plot
> ```
> 
> ## But that does not solve the issue of above of the wrong docstring/wrong signature. The wrong docstring can of course easily be fixed by changing the docstring of the PlotMethods class. But the help of `df.plot?` will still look like that one of a class
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/pydata/pandas/pull/9321#issuecomment-115234427

The generation of the API pages won't be the problem I think. With my suggestion of above (using the correct template), I think this should work. 
Where I am more worried about is the `df.plot?` help in the console. We can make the docstring give the correct information, but it will still say it is a class and show no signature, we seems a potential source of confusion to me ..

@shoyer status?

I think this just needs the fix to generate the right doc pages (and a rebase, of course). I'll need to experiment to figure that out, unless @jorisvandenbossche already know how...

I'm on vacation this week but will try to squeeze in some time :)

status?

Ugh, it is a major struggle to get autodoc to handle `Series.plot` properly.

At build time (for the docs) `Series.plot` is a class object, so autodoc insists on using the class template for the entry on `api.html`. I can use a custom template to make the subpage look reasonable, but it's still not looking right in the API docs table of contents...

This still needs a major rebase, but I have finally succeeded in generating the right API docs!

---

Here's what the relevant sections look like on the main API page:

For DataFrame:
<img width="695" alt="screen shot 2015-09-05 at 12 14 23 pm" src="https://cloud.githubusercontent.com/assets/1217238/9701120/26eb256c-53d1-11e5-873c-6fd7b56635de.png">

For Series:
<img width="674" alt="screen shot 2015-09-05 at 12 17 52 pm" src="https://cloud.githubusercontent.com/assets/1217238/9701124/37dc3ea6-53d1-11e5-8c59-7991542e9624.png">

---

Here's the documentation page for `DataFrame.plot`:
<img width="682" alt="screen shot 2015-09-05 at 12 14 40 pm" src="https://cloud.githubusercontent.com/assets/1217238/9701122/2d1c6aae-53d1-11e5-9684-78486f9e1168.png">

---

And here's an example of the doc pages for one of the submethods. Note that **`pandas.DataFrame.plot`** here is a link to the main `DataFrame.plot` page:
<img width="680" alt="screen shot 2015-09-05 at 12 15 01 pm" src="https://cloud.githubusercontent.com/assets/1217238/9701123/32e45fb4-53d1-11e5-8715-1e04f44e73c8.png">

Nice hacks to get the autosummary working! :-) Glad you found a solution

lgtm. 

@sinhrks @TomAugspurger @jorisvandenbossche 

needs a rebase / squash

existing docs in visualization.rst to change to the new accessing methods? in a follow-up PR (pls make an issue)

For the existing visualization.rst docs, I think it is OK to leave that for now

(also for later, question is one of the two we want to advertise the most)

Did the big rebase/squash -- let's see if things pass on Travis. Also added two followup issues for documentation. Assuming tests pass, this is ready for a final review.

lgtm. 

deprecate `.hist`? (and point to `.plot.hist`)
@jorisvandenbossche @sinhrks @TomAugspurger 

I don't think a deprecation of `hist` is directly needed at the moment (they also don't have fully compatible signatures). But could certainly add a pointer in the hist docstring.

:clap: 

Added another followup issue for deprecating `DataFrame.hist`: https://github.com/pydata/pandas/issues/11053

I think this would be a good idea if we can sort out the API differences.

ok, unless anyone has anything else, @shoyer merge on green.

I don't the image in the whatsnew got commited: `_static/whatsnew_plot_submethods.png`
can you push it up?