Feat/bubble plot
State: closed

- [x] Closes #16827 partially : makes bubble plots possible by passing a column name as an argument ''s''
- [x] Replaces PR #20572 (updated and improved)
- [x] 2 tests added & passed
- [x] Passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] Whatsnew entry added for 0.24.0


Labels:
name: Visualization
description: plotting

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=h1) Report
> Merging [#22403](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/9f6c02dd40e3ccd4dc13dcc2ab449b7aed2dc4ae?src=pr&el=desc) will **decrease** coverage by `0.08%`.
> The diff coverage is `12.72%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22403/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #22403      +/-   ##
==========================================
- Coverage   92.05%   91.96%   -0.09%     
==========================================
  Files         169      169              
  Lines       50709    50762      +53     
==========================================
+ Hits        46679    46684       +5     
- Misses       4030     4078      +48
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.37% <12.72%> (-0.09%)` | :arrow_down: |
| #single | `42.21% <7.27%> (-0.04%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `80.77% <12.72%> (-2.72%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=footer). Last update [9f6c02d...4d7fa1c](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Here are two examples of the bubble plots produced with numeric and categorical data.

**Numeric bubble size data:**

```python
df = pd.DataFrame(np.random.rand(50, 3), columns=['a', 'b', 'c'])
df.plot.scatter(x='a', y='b', s='c')
```
![image](https://user-images.githubusercontent.com/27647971/44311008-31e6a100-a3e0-11e8-93f0-0997c93e3a60.png)

The auto generated legend adapts to the maximum value to be represented and always has round values.

---
**Ordered categorical bubble size data:**

```python
surf_area = np.concatenate([40.0 + 80.0 * np.random.rand(30),
                            80.0 + 160.0 * np.random.rand(20),
                            100.0 + 200.0 * np.random.rand(10)])

types = np.array(30*['Flat'] + 20*['House'] + 10*['Castle'])
prices = 0.01 * surf_area * (np.random.rand(60) + 1.5) / 2
prices *= np.array([1]*30 + [1.4]*20 + [2]*10)
property_types = pd.Categorical(types, categories=['Flat', 'House', 'Castle'], ordered=True)
df = pd.DataFrame({
   'Surface area (sqm)': surf_area,
   'Price (M€)': prices,
   'Property type': property_types
})

df.loc[[0, 1, 30, 31, 50, 51], :]
```

![image](https://user-images.githubusercontent.com/27647971/44311122-02389880-a3e2-11e8-8dbd-7ce91da9e73e.png)


```python
df.plot.scatter(x='Surface area (sqm)', y='Price (M€)',
                s='Property type', alpha=.5);
```

![image](https://user-images.githubusercontent.com/27647971/44311107-aec64a80-a3e1-11e8-8966-e226761f58d3.png)

@TomAugspurger Since you followed this from the start your comments would be highly appreciated! Thx Vincent
Hello @VincentAntoine! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/plotting/_core.py`](https://github.com/VincentAntoine/pandas/blob/cd1a636bec61743922b3f7e9dbc63f5d6c874379/pandas/plotting/_core.py) !

 - There are no PEP8 issues in the file [`pandas/tests/plotting/test_frame.py`](https://github.com/VincentAntoine/pandas/blob/cd1a636bec61743922b3f7e9dbc63f5d6c874379/pandas/tests/plotting/test_frame.py) !



##### Comment last updated on September 18, 2018 at 20:11 Hours UTC
lgtm. @TomAugspurger over to you.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=h1) Report
> Merging [#22403](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/1c500fb7b3fa08c163e13375d01b9607fcdac0d6?src=pr&el=desc) will **decrease** coverage by `0.09%`.
> The diff coverage is `15.87%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22403/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #22403     +/-   ##
=========================================
- Coverage   92.17%   92.07%   -0.1%     
=========================================
  Files         169      169             
  Lines       50721    50780     +59     
=========================================
+ Hits        46753    46757      +4     
- Misses       3968     4023     +55
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.49% <15.87%> (-0.1%)` | :arrow_down: |
| #single | `42.3% <9.52%> (-0.05%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `80.56% <15.87%> (-2.99%)` | :arrow_down: |
| [pandas/io/clipboard/clipboards.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2NsaXBib2FyZC9jbGlwYm9hcmRzLnB5) | `28.23% <0%> (-2.36%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=footer). Last update [1c500fb...cd1a636](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I would tend to follow matplotlib's lead here.

https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.html does say
scatter plots are "sometimes also called bubble chart" in an aside. We can
do something similar.

On Tue, Sep 18, 2018 at 3:23 PM Vincent Chéry <notifications@github.com>
wrote:

> *@VincentAntoine* commented on this pull request.
> ------------------------------
>
> In doc/source/whatsnew/v0.24.0.txt
> <https://github.com/pandas-dev/pandas/pull/22403#discussion_r218580824>:
>
> > @@ -159,6 +161,24 @@ This is the same behavior as ``Series.values`` for categorical data. See
>  :ref:`whatsnew_0240.api_breaking.interval_values` for more.
>
>
> +.. _whatsnew_0240.enhancements.bubble_plots:
> +
> +Bubble plots
>
> Don't you think we should mention somewhere that these are bubble plots ?
> My concern is that "bubble plot" is the accurate name for this kind of
> graph, so if someone googles "pandas bubble plot", it should be easy to
> find, and for that reason it would probably need to appear somewhere in the
> documentation.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22403#discussion_r218580824>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHItQItGTz_78QC28rrztguIeQ1YEEks5ucVYwgaJpZM4WCIIO>
> .
>

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=h1) Report
> Merging [#22403](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/960a73f0c81d877fc40c77f66c834fe6af51760f?src=pr&el=desc) will **decrease** coverage by `0.06%`.
> The diff coverage is `18.75%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22403/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #22403      +/-   ##
==========================================
- Coverage   92.24%   92.18%   -0.07%     
==========================================
  Files         161      161              
  Lines       51431    51471      +40     
==========================================
+ Hits        47444    47447       +3     
- Misses       3987     4024      +37
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.57% <18.75%> (-0.07%)` | :arrow_down: |
| #single | `42.26% <12.5%> (-0.03%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `80.7% <18.75%> (-2.93%)` | :arrow_down: |
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.35% <0%> (ø)` | :arrow_up: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/22403/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `97.68% <0%> (+4.49%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=footer). Last update [960a73f...6bf9699](https://codecov.io/gh/pandas-dev/pandas/pull/22403?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Keep both I think. They're both valuable in different ways.

On Wed, Sep 19, 2018 at 6:37 PM Vincent Chéry <notifications@github.com>
wrote:

> *@VincentAntoine* commented on this pull request.
> ------------------------------
>
> In pandas/tests/plotting/test_frame.py
> <https://github.com/pandas-dev/pandas/pull/22403#discussion_r218996881>:
>
> > +        data = np.array([[3.1, 4.2],
> +                        [1.9, 2.8],
> +                        [5.4, 4.32],
> +                        [0.4, 3.4],
> +                        [4.4, 4.9],
> +                        [2.7, 6.2]])
> +        df = DataFrame(data, columns=['x', 'y'])
> +        df['z'] = pd.Categorical(['a', 'b', 'c', 'a', 'b', 'c'], ordered=True)
> +        ax = df.plot.scatter(x='x', y='y', s='z', size_factor=4)
> +        bubbles = ax.collections[0]
> +        bubble_sizes = bubbles.get_sizes()
> +        max_data = df['z'].cat.codes.max() + 1.0
> +        expected_sizes = 200.0 * 4 * (df['z'].cat.codes.values + 1)**2 / \
> +            max_data**2
> +        tm.assert_numpy_array_equal(bubble_sizes, expected_sizes)
> +
>
> Shall I *replace* this test by a test that calls _get_plot_bubbles, or
> *add* such a test and keep both?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22403#discussion_r218996881>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIvAiapT_dyBNPSxlCubFn_AmmRbKks5uctVHgaJpZM4WCIIO>
> .
>

@datapythonista not at the moment. When is the new version scheduled?
Was scheduled for September I think. No worries, I'll see if I have time to push the changes myself, as it's almost done.
rebase. @VincentAntoine @datapythonista @TomAugspurger if you'd have a look.
I've been checking this, and personally I don't think we should add these changes to pandas. If I'm not wrong, generating the desired plots in pandas is possible with `s=df['col']` and adding the legend after the plot has been generated. For a bit of syntactic sugar of being able to use `s='col'` and implementing a legend that won't be customizable, we are adding a lot of complexity (including a function `_sci_notation` that shouldn't be in the scatter, or in plots in general).
> I'm not wrong, generating the desired plots in pandas is possible with
s=df['col'] and adding the legend after the plot has been generated.

Not quite. That generates an exact mapping between the value in `df['col']`
and the radius of the marker. This isn't user-friendly since
you need to scale the data to something reasonable for matplotlib before
doing the plot. This may be a complex non-linear transformation,
depending on the data.

This PR implements a kind of binning (which libraries like ggplot do),
which can make the visualization easier to understand.

FWIW, I notice now that seaborn has a scatterplot method:
https://seaborn.pydata.org/generated/seaborn.scatterplot.html

On Wed, Nov 21, 2018 at 3:21 PM Marc Garcia <notifications@github.com>
wrote:

> I've been checking this, and personally I don't think we should add these
> changes to pandas. If I'm not wrong, generating the desired plots in pandas
> is possible with s=df['col'] and adding the legend after the plot has
> been generated. For a bit of syntactic sugar of being able to use s='col'
> and implementing a legend that won't be customizable, we are adding a lot
> of complexity (including a function _sci_notation that shouldn't be in
> the scatter, or in plots in general).
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22403#issuecomment-440812137>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIoFjy5-N5eQ7qfBXvlTFy7SzWQWGks5uxcPYgaJpZM4WCIIO>
> .
>

I missed that part, thanks for clarifying.

Wouldn't be a better approach to simply have a method that does the binning, and apply it always to `s` (I assume that we want that `s='col'` and `s=df['col']` returns the same thing).

I think that is much simpler than what's implemented here.
closing. if you want to continue, pls ping. needs to merge master and update to comments.