BUG: Fix groupby over a CategoricalIndex in axis=1
State: closed

- [x] closes #18432
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] add whatsnew entry

Labels:
name: Bug
name: Groupby
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Categorical
description: Categorical Data Type

Comments:
@ekisslinger : Don't forget to add a `whatsnew` for `0.22.0`
Any chance this fix can be added to 0.21.1?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=h1) Report
> Merging [#18525](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/262e8ff367c9291c79c4df0c2daf4713de52abc0?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18525/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18525      +/-   ##
==========================================
+ Coverage   91.32%   91.33%   +<.01%     
==========================================
  Files         163      164       +1     
  Lines       49798    49819      +21     
==========================================
+ Hits        45479    45503      +24     
+ Misses       4319     4316       -3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.13% <100%> (+0.02%)` | :arrow_up: |
| #single | `40.8% <0%> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS5weQ==) | `92.03% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/util/\_test\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX3Rlc3RfZGVjb3JhdG9ycy5weQ==) | `95.23% <0%> (ø)` | |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.71% <0%> (+0.09%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.25% <0%> (+1.81%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=footer). Last update [262e8ff...2a11e78](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=h1) Report
> Merging [#18525](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/262e8ff367c9291c79c4df0c2daf4713de52abc0?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18525/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18525      +/-   ##
==========================================
+ Coverage   91.32%   91.33%   +<.01%     
==========================================
  Files         163      164       +1     
  Lines       49798    49819      +21     
==========================================
+ Hits        45479    45503      +24     
+ Misses       4319     4316       -3
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.13% <100%> (+0.02%)` | :arrow_up: |
| #single | `40.8% <0%> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS5weQ==) | `92.03% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/util/\_test\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX3Rlc3RfZGVjb3JhdG9ycy5weQ==) | `95.23% <0%> (ø)` | |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.71% <0%> (+0.09%)` | :arrow_up: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.25% <0%> (+1.81%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=footer). Last update [262e8ff...2a11e78](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=h1) Report
> Merging [#18525](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/7627ccaf9442f4101afda69b6077e7f035e23543?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18525/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18525      +/-   ##
==========================================
- Coverage   91.35%   91.33%   -0.02%     
==========================================
  Files         164      164              
  Lines       49802    49802              
==========================================
- Hits        45496    45487       -9     
- Misses       4306     4315       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.13% <100%> (ø)` | :arrow_up: |
| #single | `40.81% <0%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS5weQ==) | `92.04% <100%> (ø)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=footer). Last update [7627cca...19f6041](https://codecov.io/gh/pandas-dev/pandas/pull/18525?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> Any chance this fix can be added 0.21.1?

@jreback @jorisvandenbossche : Thoughts?
will look in a bit 
this would be ok for 0.21.1, pls add a whatsnew note
also add the transposed version of this in your test (which works now)
I think ``test_groupby_categorical_index()`` in pandas/tests/groupby/test_categorical.py tests the transposed version (i.e. group over axis 0). Here is the relevant code:

```
   def test_groupby_categorical_index(self):

        s = np.random.RandomState(12345)
        levels = ['foo', 'bar', 'baz', 'qux']
        codes = s.randint(0, 4, size=20)
        cats = Categorical.from_codes(codes, levels, ordered=True)
        df = DataFrame(
            np.repeat(
                np.arange(20), 4).reshape(-1, 4), columns=list('abcd'))
        df['cats'] = cats

        # with a cat index
        result = df.set_index('cats').groupby(level=0).sum()
        expected = df[list('abcd')].groupby(cats.codes).sum()
        expected.index = CategoricalIndex(
            Categorical.from_codes(
                [0, 1, 2, 3], levels, ordered=True), name='cats')
        assert_frame_equal(result, expected)
```
I updated the bug fix branch. The changes are:

1. Add whatsnew entry to 0.21.1
2. Fix error message
3. Add multi-index version to associated unit test
i made the requested changes
In case you didn't see the last message, as it is a bit hidden, I made the last change which was to use integer instead of random data and directly construct the dataframe.
thanks @ekisslinger