Repository: pandas-dev/pandas

Issue:
DataFrame.hist() does not get along with matplotlib.pyplot.tight_layout()
When `tight_layout()` is called after `DataFrame.hist()` it raises `AttributeError: 'NoneType' object has no attribute 'is_bbox` if any of subplots is empty (number of charts with histograms is smaller than number os subplots).

I.e. the following code:

```
pd.DataFrame({'a':np.random.randn(100),
              'b':np.random.randn(100),
              }).hist()
plt.tight_layout()
```

works fine, but this code:

```
pd.DataFrame({'a':np.random.randn(100),
              'b':np.random.randn(100),
              'c':np.random.randn(100),
              }).hist()
plt.tight_layout()
```

raises `AttributeError`.

My versions:
pandas: 0.15.2
numpy: 1.9.1
scipy: 0.14.1
IPython: 2.3.1
matplotlib: 1.4.2



Discussion:
Ditto here, I can't seem to figure out what's going on.

Would either of you want to dig into what's causing the problem?

It seems that

``` python
    129         tight_bbox_raw = union([ax.get_tightbbox(renderer) for ax in subplots])
```

in `matplotlib.tight_layout.auto_adjust_subplotpars` returns None when it's expected to return a `BBox`.

Actually, just before the error is raised, `subplots` is a list of one `AxesSubplot` instance for which `get_visible()` returns `False`. `ax.get_tightbbox()` returns `None` for invisible axes and `auto_adjust_subplotpars()` pass this `None` to `TransformedBbox()`, which expects a real `Bbox` and raises the exception.

Looks like a matplotlib issue to me, or maybe pandas should not be creating that invisible `Axes`.

A quick workaround would be to add something like this:

```
if tight_bbox_raw is None:
    continue
```

But it might have side effects.

+1

As of `matplotlib 2.0` this now works as expected.
@ResidentMario can you add a test to confirm this behavior (that skips if mpl < 2.0)?
This might be a different cause, but tight_layout is still not playing well with `pandas.plot(kind='hist', ...)`. A plot that I am generating works fine, but as soon as I add a `fig.tight_layout()` call, I get the following error:

```python
$ scripts/exploratory_analysis/site_analysis.py QC_histogram
/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/__init__.py:898: MatplotlibDeprecationWarning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.
  mplDeprecation)
Traceback (most recent call last):
  File "scripts/exploratory_analysis/site_analysis.py", line 122, in <module>
    main(args)
  File "scripts/exploratory_analysis/site_analysis.py", line 112, in main
    QC_plot(sites, variables, variant='histogram')
  File "scripts/exploratory_analysis/site_analysis.py", line 94, in QC_plot
    fig.tight_layout()
  File "/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/figure.py", line 1943, in tight_layout
    rect=rect)
  File "/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/tight_layout.py", line 345, in get_tight_layout_figure
    pad=pad, h_pad=h_pad, w_pad=w_pad)
  File "/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/tight_layout.py", line 126, in auto_adjust_subplotpars
    tight_bbox_raw = union([ax.get_tightbbox(renderer) for ax in subplots])
  File "/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/transforms.py", line 723, in union
    x0 = np.min([bbox.xmin for bbox in bboxes])
  File "/home/naught101/miniconda3/envs/science/lib/python3.5/site-packages/matplotlib/transforms.py", line 723, in <listcomp>
    x0 = np.min([bbox.xmin for bbox in bboxes])
AttributeError: 'NoneType' object has no attribute 'xmin'
```

I figured this might have been fixed in newer versions, so I updated most things to git versions (which is more painful that I'd have liked, with conda...). Current versions are:

```
matplotlib                2.0.0               np110py35_1  
numpy                     1.12.0                   py35_0  
pandas                    0.19.2              np112py35_1  
python                    3.5.3                         1  
scipy                     0.16.0             np110py35_p1  [mkl]
```
:disappointed: I'll try looking for a fix, then, now that I've mucked this issue up.
OK, so the non-working code is:

```
df.hist()
plt.tight_layout()
```

I consciously avoid working with plot methods directly, preferring to use the plot accessor methods instead (so `df.plot.hist` instead of `df.hist`), so I wasn't aware of the rather important fact that although `df.plot.hist` and `df.hist` are the same method internally, they have different default parameter signatures. I think I was fiddling with something else involving histograms at the time, because I just plugged this into `.plot.hist` and said hey, it works, must have been a `mpl 2.0.0` fix:

```
df.plot.hist()
plt.tight_layout()
```

TIL these actually result in radically different output! To replicate this bug using the accessor method you have to further specifiy:

```
df.plot.hist(subplots=True, layout=(2, 2))
plt.tight_layout()
```

The test I added in the closing commit *did* use `hist`, but for *some reason* in my test `DataFrame` a `3` became a `2`:

```
df = DataFrame(randn(100, 2))
```

Which passes, because it creates a `(1, 2)` layout that works. Ouch. My apologies.
So I haven't quite finished investigating this, but here's what I think so far: the core of `pandas` plotting methodology is very old, predating the current `matplotlib` orthodoxy. It looks like  `pandas` uses a lower-level API&mdash;adding artists to the canvas&mdash;to do what would today be done using something like `plt.subplots`.

Since it does this in a loop which iterates over the data, this means that in this example case it only does so three times, leaving the last layout element undefined. That axis then has a `BBox` of `None`, which `tight_layout` trips over, which then causes everything else to fail.

This may be reportable as a bug in `matplotlib`. A band-aid can definitely be made in `pandas` though. In the long term, `pandas` defo needs to move on to doing something like `plt.subplots(2, 2)` instead.
ok reopening
This boils down to a `matplotlib` bug. This is what `pandas` is doing internally, which fails:

```python
import matplotlib.pyplot as plt
f, axarr = plt.subplots(2, 2)
axarr[1][1].set_visible(False)
plt.tight_layout()
```

Going about it a slightly different way, this works:

```python
import matplotlib.pyplot as plt
f, axarr = plt.subplots(2, 2)
axarr[1][1].axis('off')
plt.tight_layout()
```
This is now fixed downstream in `matplotlib` as [#8255](https://github.com/matplotlib/matplotlib/issues/8225) by [PR#8244](https://github.com/matplotlib/matplotlib/pull/8244). Once `matplotlib 2.0.1` lands this issue will be fixed and can be closed (use `mpl > 2.0.1`). For the moment best to leave it open though.
thanks @ResidentMario ok can update the release note at some point. any additional tests needed?
@jreback I would say no, the PR included a regression test for this in the `matplotlib` test suite.
@ResidentMario the regression test passed though, contrary to the bug fix that was needed, right? (IIRC you did another PR which added a slightly different test which failed though..?)
this was corrected in https://github.com/pandas-dev/pandas/commit/03dca9610b6ad91538c1cd1da71fb5196d7bb3f7

any reason not to close this issue?

Resolving patch:
diff --git a/pandas/tools/plotting.py b/pandas/tools/plotting.py
--- a/pandas/tools/plotting.py
+++ b/pandas/tools/plotting.py
@@ -150,6 +150,14 @@ def _mpl_ge_2_0_0():
         return False
 
 
+def _mpl_ge_2_0_1():
+    try:
+        import matplotlib
+        return matplotlib.__version__ >= LooseVersion('2.0.1')
+    except ImportError:
+        return False
+
+
 if _mpl_ge_1_5_0():
     # Compat with mp 1.5, which uses cycler.
     import cycler


Test patch:
diff --git a/pandas/tests/plotting/common.py b/pandas/tests/plotting/common.py
--- a/pandas/tests/plotting/common.py
+++ b/pandas/tests/plotting/common.py
@@ -53,6 +53,7 @@ def setUp(self):
         self.mpl_ge_1_4_0 = plotting._mpl_ge_1_4_0()
         self.mpl_ge_1_5_0 = plotting._mpl_ge_1_5_0()
         self.mpl_ge_2_0_0 = plotting._mpl_ge_2_0_0()
+        self.mpl_ge_2_0_1 = plotting._mpl_ge_2_0_1()
 
         if self.mpl_ge_1_4_0:
             self.bp_n_objects = 7
diff --git a/pandas/tests/plotting/test_hist_method.py b/pandas/tests/plotting/test_hist_method.py
--- a/pandas/tests/plotting/test_hist_method.py
+++ b/pandas/tests/plotting/test_hist_method.py
@@ -241,8 +241,8 @@ def test_hist_layout(self):
     @slow
     # GH 9351
     def test_tight_layout(self):
-        if self.mpl_ge_2_0_0:
-            df = DataFrame(randn(100, 2))
+        if self.mpl_ge_2_0_1:
+            df = DataFrame(randn(100, 3))
             _check_plot_works(df.hist)
             self.plt.tight_layout()