scatter plot and hexbin plot lose x-axis when colorbar is included.
State: closed

closes https://github.com/pandas-dev/pandas/issues/10611 
closes https://github.com/pandas-dev/pandas/issues/10678. 

The x-axis for scatter plot and hexbin plot disappears when colorbar is included. This seems to be because colorbar axis is looped through in `_handle_shared_axes`: https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218104ffdfc49/pandas/plotting/_core.py#L426
after discussing with @TomAugspurger (issue https://github.com/pandas-dev/pandas/issues/10611), we decided to try adding the attribute `__is_pandas_colorbar` to colorbar axis object and skipping it during handling of shared axes. 

I've done some tests that seem to fix the issue. But we may need more tests:
```
%matplotlib inline
import matplotlib.pylab as pl
from mypandas import pandas as pd
import numpy as np
random_array = np.random.random((1000,3))

df = pd.DataFrame(random_array,columns=['A label','B label','C label'])
df.plot.scatter('A label','B label',c='C label', PATCH_MODE_FLAG = False);pl.title('pandas current version');
df.plot.scatter('A label','B label',c='C label', PATCH_MODE_FLAG = True);pl.title('patch fixing x-axis');

df.plot.hexbin('A label','B label', gridsize=25, PATCH_MODE_FLAG = False);pl.title('pandas current version');
df.plot.hexbin('A label','B label', gridsize=25, PATCH_MODE_FLAG = True);pl.title('patch fixing x-axis');
```

![image](https://user-images.githubusercontent.com/26352146/37747996-49642266-2d58-11e8-8c6f-c382267e79db.png)
![image](https://user-images.githubusercontent.com/26352146/37747999-4d88d986-2d58-11e8-8445-f1348c35d892.png)
![image](https://user-images.githubusercontent.com/26352146/37748001-5109d93e-2d58-11e8-9f6f-73a68329f731.png)
![image](https://user-images.githubusercontent.com/26352146/37748005-54beb590-2d58-11e8-8233-4e26b135fdad.png)


Labels:
name: Visualization
description: plotting

Comments:
Thanks, let us know when you're ready for us to take a look.

How does this work with `subplots=True`? Do we not have to worry about that since it's a scatter plot?

How does this work with user-specified axes via `ax=`? Especially with multiple subplots. It may already be broken, so no worries if it doesn't work well.
Yes, it's already broken. Please feel free to take a look:

```
def make_multiplots(numrows=2,numcols=2,sharey=True,sharex=True,PATCH_MODE_FLAG = True, is_scatter = True):
    fig, axes = pl.subplots(numrows,numcols,sharey=sharey,sharex=sharex )
    for i in range(axes.shape[0]):
        for j in range(axes.shape[1]):
            random_array = np.random.random((1000,3))
            df = pd.DataFrame(random_array,columns=['A label','B label','C label'])
            if is_scatter:
                df.plot.scatter('A label','B label',c='C label', PATCH_MODE_FLAG = PATCH_MODE_FLAG,ax=axes[i,j]);
            else:
                df.plot.hexbin('A label','B label',gridsize=20, PATCH_MODE_FLAG = PATCH_MODE_FLAG,ax=axes[i,j]);
            
            
make_multiplots(numrows=2,numcols=2,sharey=True, sharex=True, PATCH_MODE_FLAG = False, is_scatter=True);pl.suptitle('pandas current version');            
make_multiplots(numrows=2,numcols=2,sharey=True, sharex=True, PATCH_MODE_FLAG = True, is_scatter=True);pl.suptitle('patch fixing x-axis');            
make_multiplots(numrows=2,numcols=2,sharey=True, sharex=True, PATCH_MODE_FLAG = False, is_scatter=False);pl.suptitle('pandas current version');            
make_multiplots(numrows=2,numcols=2,sharey=True, sharex=True, PATCH_MODE_FLAG = True, is_scatter=False);pl.suptitle('patch fixing x-axis');  
```          
![image](https://user-images.githubusercontent.com/26352146/37773292-97f0a5a2-2db3-11e8-9912-8f2f558f5740.png)
![image](https://user-images.githubusercontent.com/26352146/37773303-9eaa8e08-2db3-11e8-975b-ce454a33e277.png)
![image](https://user-images.githubusercontent.com/26352146/37773312-a30c9b62-2db3-11e8-9040-864b16ebf59a.png)
![image](https://user-images.githubusercontent.com/26352146/37773316-a6259916-2db3-11e8-9be2-592295972768.png)

As of subplots=True, I've only used it with histograms and line plots. I'm not familiar with their usage with scatter plots/hexbin plots. So I'm not sure how to test them. Simply using it with single plot doesn't interfere with the outcome, but I'm assuming that's not what you were referring to. I think we'll need your expertise here.

`df.plot.scatter('A label','B label',c='C label', PATCH_MODE_FLAG = True, subplots=True);`
![image](https://user-images.githubusercontent.com/26352146/37774409-7f994c9a-2db6-11e8-804b-ddfaadea46e5.png)

Hello @javadnoorb! Thanks for updating the PR.





Cheers ! There are no PEP8 issues in this Pull Request. :beers: 

##### Comment last updated on July 04, 2018 at 03:04 Hours UTC
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=h1) Report
> Merging [#20446](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc45fbafef172e357cb5decdeab22de67160f5b7?src=pr&el=desc) will **increase** coverage by `0.02%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/20446/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #20446      +/-   ##
==========================================
+ Coverage    91.9%   91.92%   +0.02%     
==========================================
  Files         154      158       +4     
  Lines       49657    49705      +48     
==========================================
+ Hits        45638    45693      +55     
+ Misses       4019     4012       -7
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.3% <100%> (+0.02%)` | :arrow_up: |
| #single | `41.99% <11.11%> (+0.09%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.57% <100%> (+0.03%)` | :arrow_up: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.48% <0%> (-0.18%)` | :arrow_down: |
| [pandas/core/indexes/timedeltas.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy90aW1lZGVsdGFzLnB5) | `91.08% <0%> (-0.16%)` | :arrow_down: |
| [pandas/core/indexes/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9wZXJpb2QucHk=) | `92.53% <0%> (-0.14%)` | :arrow_down: |
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `88.36% <0%> (-0.13%)` | :arrow_down: |
| [pandas/core/indexes/accessors.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9hY2Nlc3NvcnMucHk=) | `90.09% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `95.53% <0%> (-0.07%)` | :arrow_down: |
| [pandas/core/arrays/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL19faW5pdF9fLnB5) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/timedelta.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3RpbWVkZWx0YS5weQ==) | `100% <0%> (ø)` | |
| [pandas/core/arrays/period.py](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3BlcmlvZC5weQ==) | `100% <0%> (ø)` | |
| ... and [17 more](https://codecov.io/gh/pandas-dev/pandas/pull/20446/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=footer). Last update [dc45fba...7196d6e](https://codecov.io/gh/pandas-dev/pandas/pull/20446?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Don't worry about subplots. It would be something like `df.plot.scatter(x=0, y=[1, 2], subplots=True)`, which currently isn't allowed.
That test with `assert` doesn't deal with the issue. It seems like if you make a subplot and pass the axis to pandas, the x-axis comes out fine. I didn't use environments, and don't know how to switch from my development pandas to the native version without an uninstall. But with the `PATCH_MODE_FLAG=False` you can see it:

```
fig, ax1 = pl.subplots()
df.plot.scatter(x='A label', y='B label', c='C label',ax=ax1, PATCH_MODE_FLAG=False)
```
![image](https://user-images.githubusercontent.com/26352146/37789044-48531978-2dd9-11e8-85d0-4562a26ca296.png)


`df.plot.scatter(x='A label', y='B label', c='C label', PATCH_MODE_FLAG=False)`
![image](https://user-images.githubusercontent.com/26352146/37788928-0c47e0bc-2dd9-11e8-997b-50028f9edc15.png)

This works:

```
PATCH_MODE_FLAG = False
ax1 = df.plot.scatter(x='A label', y='B label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)
ax2 = df.plot.scatter(x='A label', y='B label', c='C label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)
assert (ax1.get_xticks()==ax2.get_xticks()).all(), "axis labels are not the same"
```

but regardless of `PATCH_MODE_FLAG` value it doesn't throw an error. 

I think the issue is not with the xticks attribute but rather showing them.
Trying a few of these. Quite bizarre:

```
PATCH_MODE_FLAG = False
ax1 = df.plot.scatter(x='A label', y='B label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)
ax2 = df.plot.scatter(x='A label', y='B label', c='C label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)
assert (ax1.get_xticks()==ax2.get_xticks()).all(), "Axis xticks are not the same"
assert ax1.xaxis.get_visible()==ax2.xaxis.get_visible(), "x-axis visbilities are not the same"
print(ax1.get_xticklabels(),ax2.get_xticklabels())
print(ax1.get_xticks(),ax2.get_xticks())
print(ax1.xaxis.get_visible(),ax2.xaxis.get_visible())
```

![image](https://user-images.githubusercontent.com/26352146/37790413-d5a3cbf8-2ddc-11e8-9dfd-30c1e97b6724.png)

@javadnoorb how's this going? For this PR I would ignore user-provided axes, and only worry about the common case of `df.plot(x='x', y='y', c='c')`.
@TomAugspurger I haven't worked on this for a few days. Let me see if I can come up with a reasonable test. `get_xticklabels` returns an object, so I think we can't use it for comparison. I think the reason the x-labels disappear should be because of these three lines:

https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218104ffdfc49/pandas/plotting/_tools.py#L282
https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218104ffdfc49/pandas/plotting/_tools.py#L296
https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218104ffdfc49/pandas/plotting/_tools.py#L293
my guess is that something like:

`assert ax1.xaxis.get_majorticklabels().get_visible()==ax2.xaxis.get_majorticklabels().get_visible()` 

and 

`assert ax1.xaxis.get_label().get_visible()==ax2.xaxis.get_label().get_visible()` 

and 

`assert ax1.xaxis.get_minorticklabels().get_visible()==ax2.xaxis.get_minorticklabels().get_visible()` 

or something like that, should work for the test. 

Let me give it a shot.
Sounds good. Let me know if that doesn't work and I can take a look.

On Wed, Mar 28, 2018 at 11:48 AM, Javad Noorbakhsh <notifications@github.com
> wrote:

> @TomAugspurger <https://github.com/TomAugspurger> I haven't worked on
> this for a few days. Let me see if I can come up with a reasonable test.
> get_xticklabels returns an object, so I think we can't use it for
> comparison. I think the reason the x-labels disappear should be because of
> these two lines:
>
> https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218
> 104ffdfc49/pandas/plotting/_tools.py#L282
> https://github.com/pandas-dev/pandas/blob/8a5830305d2d9a087fbe46bd968218
> 104ffdfc49/pandas/plotting/_tools.py#L296
>
> my guess is that something like:
>
> assert ax1.get_majorticklabels().get_visible()==ax2.get_
> majorticklabels().get_visible()
>
> and
>
> assert ax1.get_label().get_visible()==ax2.get_label().get_visible()
>
> or something like that, should work for the test.
>
> Let me give it a shot.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/20446#issuecomment-376955988>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIkLc6e7wel8eSgSbmlptCfR_HWFaks5ti77NgaJpZM4S2XhP>
> .
>

Ok, I wrote a test that seems to work:
```
def test_if_colorbar_affects_xaxis_visibility(PATCH_MODE_FLAG=True):
    random_array = np.random.random((1000,3))
    df = pd.DataFrame(random_array,columns=['A label','B label','C label'])

    ax1 = df.plot.scatter(x='A label', y='B label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)
    ax2 = df.plot.scatter(x='A label', y='B label', c='C label', PATCH_MODE_FLAG=PATCH_MODE_FLAG)

    assert all([vis[0].get_visible()==vis[1].get_visible() for vis in zip(ax1.xaxis.get_minorticklabels(),ax2.xaxis.get_minorticklabels())]), 'minor x-axis tick labels visibility changes when colorbar included'
    assert all([vis[0].get_visible()==vis[1].get_visible() for vis in zip(ax1.xaxis.get_majorticklabels(),ax2.xaxis.get_majorticklabels())]), 'major x-axis tick labels visibility changes when colorbar included'
    assert ax1.xaxis.get_label().get_visible()==ax2.xaxis.get_label().get_visible(),'x-axis label visibility changes when colorbar included'
```
```    
test_if_colorbar_affects_xaxis_visibility(PATCH_MODE_FLAG=True)    
```
![image](https://user-images.githubusercontent.com/26352146/38047415-6f046c82-3290-11e8-9f00-4d7f504cf6d4.png)
![image](https://user-images.githubusercontent.com/26352146/38047422-75e82192-3290-11e8-807c-3679d298e2d7.png)
```
test_if_colorbar_affects_xaxis_visibility(PATCH_MODE_FLAG=False)
```

```---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-47-0cc25b4e00f4> in <module>()
     10     assert ax1.xaxis.get_label().get_visible()==ax2.xaxis.get_label().get_visible(),'x-axis label visibility changes when colorbar included'
     11 
---> 12 test_if_colorbar_affects_xaxis_visibility(PATCH_MODE_FLAG=False)

<ipython-input-47-0cc25b4e00f4> in test_if_colorbar_affects_xaxis_visibility(PATCH_MODE_FLAG)
      7 
      8     assert all([vis[0].get_visible()==vis[1].get_visible() for vis in zip(ax1.xaxis.get_minorticklabels(),ax2.xaxis.get_minorticklabels())]), 'minor x-axis tick labels visibility changes when colorbar included'
----> 9     assert all([vis[0].get_visible()==vis[1].get_visible() for vis in zip(ax1.xaxis.get_majorticklabels(),ax2.xaxis.get_majorticklabels())]), 'major x-axis tick labels visibility changes when colorbar included'
     10     assert ax1.xaxis.get_label().get_visible()==ax2.xaxis.get_label().get_visible(),'x-axis label visibility changes when colorbar included'
     11 

AssertionError: major x-axis tick labels visibility changes when colorbar included
```

![image](https://user-images.githubusercontent.com/26352146/38047483-afb01362-3290-11e8-847f-5c05308c5637.png)
![image](https://user-images.githubusercontent.com/26352146/38047488-b2f6755c-3290-11e8-811c-03a729cf3f08.png)


If that seems fine I'll include the changes and remove the PATCH_MODE flag. 

I'm not familiar with writing tests. Is this where it goes?
https://github.com/pandas-dev/pandas/blob/master/pandas/tests/plotting/test_misc.py
That test seems sensible.

Tests can go in `pandas/pandas/tests/plotting/test_frame.py` under TestFrame. There are some other ones there for `scatter`.
Ok, great. I'll check if it passes the regular checks before including the test.
Also need a test for hexbin. This one seems to work:

```
def test_if_hexbin_xaxis_label_is_visible(PATCH_MODE_FLAG=True):
    random_array = np.random.random((1000,3))
    df = pd.DataFrame(random_array,columns=['A label','B label','C label'])

    ax = df.plot.hexbin('A label','B label', gridsize=12, PATCH_MODE_FLAG = PATCH_MODE_FLAG);    
    assert all([vis.get_visible() for vis in ax.xaxis.get_minorticklabels()]), 'minor x-axis tick labels are not visible'
    assert all([vis.get_visible() for vis in ax.xaxis.get_majorticklabels()]), 'major x-axis tick labels are not visible'
    assert ax.xaxis.get_label().get_visible(),'x-axis label is not visible'
```

```
test_if_hexbin_xaxis_label_is_visible(PATCH_MODE_FLAG=True)   
```
![image](https://user-images.githubusercontent.com/26352146/38051264-c750eea0-329b-11e8-9bae-856e60e094c7.png)

```
test_if_hexbin_xaxis_label_is_visible(PATCH_MODE_FLAG=False)   
```
```
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-66-cd06ece40a4d> in <module>()
      9 
     10 
---> 11 test_if_hexbin_xaxis_label_is_visible(PATCH_MODE_FLAG=False)

<ipython-input-66-cd06ece40a4d> in test_if_hexbin_xaxis_label_is_visible(PATCH_MODE_FLAG)
      4 
      5     ax = df.plot.hexbin('A label','B label', gridsize=12, PATCH_MODE_FLAG = PATCH_MODE_FLAG);
----> 6     assert all([vis.get_visible() for vis in ax.xaxis.get_minorticklabels()]), 'minor x-axis tick labels are not visible'
      7     assert all([vis.get_visible() for vis in ax.xaxis.get_majorticklabels()]), 'major x-axis tick labels are not visible'
      8     assert ax.xaxis.get_label().get_visible(),'x-axis label is not visible'

AssertionError: minor x-axis tick labels are not visible
```
![image](https://user-images.githubusercontent.com/26352146/38051297-dc3f9d70-329b-11e8-9e8e-01fb6b249fd6.png)

These changes seem pretty benign. Is there any specific reason for Travis-CI to throw an error?
Where are the release notes?
https://github.com/pandas-dev/pandas/blob/master/doc/source/whatsnew/v0.23.0.txt

On Thu, Mar 29, 2018 at 1:23 PM, Javad Noorbakhsh <notifications@github.com>
wrote:

> Where are the release notes?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/20446#issuecomment-377327656>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIhj82OsukyNfHE-kQT8ohbcDzhN1ks5tjSaHgaJpZM4S2XhP>
> .
>

I've been trying the tests with the inline and pdf backend. Tests pass depending on the backend. So I'm guessing that my PR doesn't really address the issue. 
> Tests pass depending on the backend. So I'm guessing that my PR doesn't really address the issue.

Can you rephrase that? This behavior only manifests with specific backends. Ideally it would always fail on a given backend (e.g. notebook) without your changes, and always pass with it.

If the backend is too much of an issue, we can adapt the test. The change is clearly an improvement.
What I mean is that the test always passes when my changes in this PR are applied regardless of the backend. But without those changes it passes for the pdf backend, but not the inline backend. I was assuming the tests are run by pytest in some default backend which is not inline. Do you know if that is the case?  If so, then they will pass regardless of my changes, unless we explicitly force them to run with the inline backend.

On a separate note, this issue with x-axis labels seems to stem from how a non-native backend interacts with axes in ways that I don't quite understand. But it might be an IPython problem. I'm wondering if we don't address the underlying issue it may pop up later in a different context. If we're gonna go ahead with this PR, just to be safe, should we include something like `matplotlib.get_backend() == 'module://ipykernel.pylab.backend_inline'` for deciding whether the changes should go into effect?
@TomAugspurger. Seems like the issue is because in the inline backend floats corresponding to location of axes have too many arbitrary trailing digits. For example two axes that have their bottom left corners at 0.3 may appear as 0.3 and 0.30000001. I rounded them in `_core.py`.

This new commit works for single plots with pdf and inline backends. It still fails for multiplots. We'll need to dig deeper for that. But it seems like we don't need the previous hack anymore.
I just remembered about this. The last commit seemed to have passed all the checks. Is there any interest to follow up on it?
Thanks @TomAugspurger. I cleaned up the code. It's passing the tests. Here's what the output in jupyterlab looks like:

```python
%matplotlib inline
import numpy as np
import pandas as pd
random_array = np.random.random((1000, 3))
df = pd.DataFrame(random_array,columns=['A label','B label','C label'])
df.plot.scatter('A label', 'B label', c='C label')
```
![image](https://user-images.githubusercontent.com/26352146/42129418-e9df0f20-7c91-11e8-989a-1aaf01f4bcc4.png)

```python
df.plot.hexbin('A label','B label', gridsize=12);
```
![image](https://user-images.githubusercontent.com/26352146/42129422-fd31c11c-7c91-11e8-97a9-b7152d63ff07.png)

thanks @javadnoorb very nice!
thanks @jreback!