Repository: pandas-dev/pandas
Pull request: DataFrameGroupby.boxplot fails when subplots=False
- [x] closes #16748
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

![Screen Shot 2020-10-13 at 8 06 48 PM](https://user-images.githubusercontent.com/9269816/95898874-037e6980-0d90-11eb-85b3-e00fe36611b9.png)


Problem statement:
DataFrameGroupBy.boxplot with subplots=False fails when using column param
#### Code Sample, a copy-pastable example if possible

```python
df = pd.DataFrame({'cat':np.random.choice(list('abcde'), 100), 
                    'v':np.random.rand(100), 
                    'v1':np.random.rand(100)})
df.groupby('cat').boxplot(subplots=False, column='v')
```
outputs
`KeyError: "['v'] not in index"`
#### Problem description
The boxplot works when either `subplots=False` or `column='v'` but not when they are both specified.

#### Expected Output
A single axes plot with each group having its own boxplot. The column 'cat' would label the x-axis.
#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.20.2
pytest: 3.0.7
pip: 9.0.1
setuptools: 35.0.2
Cython: 0.25.2
numpy: 1.13.0
scipy: 0.19.0
xarray: None
IPython: 6.0.0
sphinx: 1.5.5
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.7
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: 0.3.0.post

</details>

Hints from issue discussion:
How could i start working on this issue ? 
Experiencing the same issue on pandas 0.20.3
I also ran into the issue on 0.20.3. Inspired by [your own Stack Overflow post](https://stackoverflow.com/questions/44677690/boxplot-on-groupby-timegrouper-without-subplots-using-pandas), I ended up going with the hacky solution of first boiling down the data frame to what's necessary, plotting through 

    df[['cat', 'v']].groupby('cat').boxplot(subplots=False)
With this diff, the example seems to work

```diff
diff --git a/pandas/plotting/_core.py b/pandas/plotting/_core.py
index 0d77b5f41..eead95ad3 100644
--- a/pandas/plotting/_core.py
+++ b/pandas/plotting/_core.py
@@ -2373,6 +2373,8 @@ def boxplot_frame_groupby(grouped, subplots=True, column=None, fontsize=None,
                             right=0.9, wspace=0.2)
     else:
         from pandas.core.reshape.concat import concat
+        from pandas import IndexSlice
+
         keys, frames = zip(*grouped)
         if grouped.axis == 0:
             df = concat(frames, keys=keys, axis=1)
@@ -2381,7 +2383,9 @@ def boxplot_frame_groupby(grouped, subplots=True, column=None, fontsize=None,
                 df = frames[0].join(frames[1::])
             else:
                 df = frames[0]
-        ret = df.boxplot(column=column, fontsize=fontsize, rot=rot,
+        if column:
+            df = df.loc[:, IndexSlice[:, column]]
+        ret = df.boxplot(fontsize=fontsize, rot=rot,
                          grid=grid, ax=ax, figsize=figsize,
                          layout=layout, **kwds)
     return ret
```

![gh](https://user-images.githubusercontent.com/1312546/31740251-5d64c204-b416-11e7-8a2c-56e4585ad271.png)

Is anyone able to verify that that's correct, and that that fix doesn't break anything else? @ adhaamehab are you still interested in working on this issue (sorry I didn't see your comment the first time around).
In particular, I haven't explored how that change interacts with the other options to groupby boxplot. I'm not familiar with that section of the codebase.
I am also experiencing this issue in pandas 0.20.3. Is anybody working on it? @TomAugspurger the plot for the fix from 18 Oct 2017 looks good except that I think the x-axis labels should just be (for example) `a` instead of `(a, v)` since all labels contain `v`. The hack mentioned by @fuglede works but now throws numpy `FutureWarning: reshape is deprecated and will raise in a subsequent release. Please use .values.reshape(...) instead
  return getattr(obj, method)(*args, **kwds)`.
I  have same issue in pandas 0.23.4.

Commit message: Remove skip

Patch to review:
diff --git a/doc/source/whatsnew/v1.0.0.rst b/doc/source/whatsnew/v1.0.0.rst
index 7fe358d3820f2..2aa51dde8abd7 100644
--- a/doc/source/whatsnew/v1.0.0.rst
+++ b/doc/source/whatsnew/v1.0.0.rst
@@ -171,6 +171,7 @@ Plotting
 - Bug in :meth:`DataFrame.plot` producing incorrect legend markers when plotting multiple series on the same axis (:issue:`18222`)
 - Bug in :meth:`DataFrame.plot` when ``kind='box'`` and data contains datetime or timedelta data. These types are now automatically dropped (:issue:`22799`)
 - Bug in :meth:`DataFrame.plot.line` and :meth:`DataFrame.plot.area` produce wrong xlim in x-axis (:issue:`27686`, :issue:`25160`, :issue:`24784`)
+- Bug in :meth:`DataFrame.groupby.boxplot` when ``subplots=False``, a KeyError would raise (:issue:`16748`)
 
 Groupby/resample/rolling
 ^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/pandas/plotting/_matplotlib/boxplot.py b/pandas/plotting/_matplotlib/boxplot.py
index 8ff7441df5354..6ebc8a616e367 100644
--- a/pandas/plotting/_matplotlib/boxplot.py
+++ b/pandas/plotting/_matplotlib/boxplot.py
@@ -8,6 +8,7 @@
 from pandas.core.dtypes.missing import remove_na_arraylike
 
 import pandas as pd
+from pandas import Index, IndexSlice, MultiIndex
 
 from pandas.io.formats.printing import pprint_thing
 from pandas.plotting._matplotlib import converter
@@ -314,10 +315,21 @@ def plot_group(keys, values, ax):
             with plt.rc_context(rc):
                 ax = plt.gca()
         data = data._get_numeric_data()
+
+        # if columns is None, use all numeric columns of data; if data columns
+        # is MultiIndex, which means a Groupby has been applied before, select
+        # data using new grouped column names; if data columns is Index, select
+        # data simply using columns
         if columns is None:
             columns = data.columns
-        else:
-            data = data[columns]
+        elif isinstance(data.columns, MultiIndex):
+
+            # reselect columns with after-groupby multi-index columns
+            data = data.loc[:, IndexSlice[:, columns]]
+            columns = data.columns
+        elif isinstance(data.columns, Index):
+            data = data.loc[:, columns]
+            columns = data.columns
 
         result = plot_group(columns, data.values.T, ax)
         ax.grid(grid)
diff --git a/pandas/tests/plotting/test_boxplot_method.py b/pandas/tests/plotting/test_boxplot_method.py
index 5bbaff580c356..92d867d9a7b0f 100644
--- a/pandas/tests/plotting/test_boxplot_method.py
+++ b/pandas/tests/plotting/test_boxplot_method.py
@@ -412,3 +412,46 @@ def test_fontsize(self):
         self._check_ticks_props(
             df.boxplot("a", by="b", fontsize=16), xlabelsize=16, ylabelsize=16
         )
+
+    @pytest.mark.parametrize(
+        "col, expected_xticklabel",
+        [
+            ("v", ["(a, v)", "(b, v)", "(c, v)", "(d, v)", "(e, v)"]),
+            (["v"], ["(a, v)", "(b, v)", "(c, v)", "(d, v)", "(e, v)"]),
+            ("v1", ["(a, v1)", "(b, v1)", "(c, v1)", "(d, v1)", "(e, v1)"]),
+            (
+                ["v", "v1"],
+                [
+                    "(a, v)",
+                    "(a, v1)",
+                    "(b, v)",
+                    "(b, v1)",
+                    "(c, v)",
+                    "(c, v1)",
+                    "(d, v)",
+                    "(d, v1)",
+                    "(e, v)",
+                    "(e, v1)",
+                ],
+            ),
+        ],
+    )
+    def test_groupby_boxplot_subplots_false(self, col, expected_xticklabel):
+        # GH 16748
+        df = DataFrame(
+            {
+                "cat": np.random.choice(list("abcde"), 100),
+                "v": np.random.rand(100),
+                "v1": np.random.rand(100),
+            }
+        )
+        grouped = df.groupby("cat")
+
+        # check is boxplot with subplots=False works
+        axes = _check_plot_works(
+            grouped.boxplot, subplots=False, column=col, return_type="axes"
+        )
+
+        # check if xticks labels are plotted correctly
+        result_xticklabel = [x.get_text() for x in axes.get_xticklabels()]
+        assert expected_xticklabel == result_xticklabel


Code review comments:
diff_hunk: @@ -8,6 +8,7 @@
 from pandas.core.dtypes.missing import remove_na_arraylike
 
 import pandas as pd
+from pandas import Index, IndexSlice, MultiIndex
path: pandas/plotting/_matplotlib/boxplot.py
text: @user1:
What if you remove this import, and just use `pd.Index`, etc. below? Can you see if the test still fails?

@author:
wow, genius!! the issue is gone!! 👍  do you mind explaining a bit the difference? @user1 

@user1:
No idea, unfortunately :/ Just a hunch since that test deals with first-time imports, and we were importing pandas twice. That perhaps indicates an issue with how or warning is implemented though, but that's a separate issue.