Support for use of Enums in MultiIndex
State: closed

- [x] closes #21298
- [x] tests added / passed: added `test_use_enum_in_multiindex` in `tests/indexes/test_multi.py`, see diff
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

The boolean flag in https://github.com/pandas-dev/pandas/blob/9f95f7dbffef7752175ca9ed918314cb6f0b9b18/pandas/core/arrays/categorical.py#L2526-L2528 incorrectly caused a `TypeError` to be raised in
https://github.com/pandas-dev/pandas/blob/9f95f7dbffef7752175ca9ed918314cb6f0b9b18/pandas/core/arrays/categorical.py#L352-L354
when an Enum was used in a MultiIndex. No tests fail when the flag is flipped back. This is fixed in this PR.

Labels:
name: Enhancement
name: MultiIndex

Comments:
CircleCI failed due to unavailability of module `enum` in test environment. I don't know how else to test this---should I simply remove the new test I added?
> CircleCI failed due to unavailability of module enum in test environment. I don't know how else to test this---should I simply remove the new test I added?

@benediamond : I believe the build that's failing is a Python 2.7 build.  I would use `pytest`'s skip functionality to skip whenever we're in Python 2 land.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=h1) Report
> Merging [#21348](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/20ae4543c1d8838f52229830bfae0cc8626801bb?src=pr&el=desc) will **decrease** coverage by `0.39%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21348/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #21348     +/-   ##
=========================================
- Coverage   92.28%   91.89%   -0.4%     
=========================================
  Files         161      153      -8     
  Lines       51500    49596   -1904     
=========================================
- Hits        47528    45576   -1952     
- Misses       3972     4020     +48
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.29% <100%> (-0.39%)` | :arrow_down: |
| #single | `41.86% <100%> (-0.45%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/arrays/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2NhdGVnb3JpY2FsLnB5) | `95.69% <100%> (+0.34%)` | :arrow_up: |
| [pandas/core/arrays/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2Jhc2UucHk=) | `83.95% <0%> (-13.41%)` | :arrow_down: |
| [pandas/util/\_doctools.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RvY3Rvb2xzLnB5) | `0% <0%> (-12.88%)` | :arrow_down: |
| [pandas/io/parquet.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnF1ZXQucHk=) | `71.79% <0%> (-12.83%)` | :arrow_down: |
| [pandas/core/reshape/util.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS91dGlsLnB5) | `90.32% <0%> (-9.68%)` | :arrow_down: |
| [pandas/util/\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RlY29yYXRvcnMucHk=) | `82.25% <0%> (-9.09%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `88.85% <0%> (-8.83%)` | :arrow_down: |
| [pandas/core/dtypes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Jhc2UucHk=) | `92.1% <0%> (-7.9%)` | :arrow_down: |
| [pandas/errors/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2Vycm9ycy9fX2luaXRfXy5weQ==) | `92.3% <0%> (-7.7%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `92.09% <0%> (-6.32%)` | :arrow_down: |
| ... and [144 more](https://codecov.io/gh/pandas-dev/pandas/pull/21348/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=footer). Last update [20ae454...06437cd](https://codecov.io/gh/pandas-dev/pandas/pull/21348?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@gfyoung I'm stumped by this `whatsnew` conflict, any thoughts?
@jreback bumping this.
for whatsnew, see: https://github.com/benediamond/pandas/blob/4cd2dff96775a828687acd7c65919ec135824cf4/doc/source/whatsnew/v0.24.0.txt#L132-L135
> can you use @td.skip_if_no('enum') here (then do the actual import inside), we have to guard against the tests being run on 2.7 w/o the compat library (by a user).

@jreback implemented here:
```python
    @td.skip_if_no('enum')
    def test_use_enum_in_multiindex(self):
        # GH 21298
        # Allow use of Enums as one of the factors in a MultiIndex.
        from enum import Enum
        MyEnum = Enum("MyEnum", "A B")
        ...
```
but yields:
```
==================================== ERRORS ====================================
_____________ ERROR collecting pandas/tests/indexes/test_multi.py ______________
pandas/tests/indexes/test_multi.py:30: in <module>
    class TestMultiIndex(Base):
pandas/tests/indexes/test_multi.py:3310: in TestMultiIndex
    @td.skip_if_no('enum')
E   NameError: name 'td' is not defined
------- generated xml file: /tmp/circle-junit.2ZpyFLd/reports/junit.xml --------
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
===================== 3 skipped, 1 error in 43.23 seconds ======================
```
not sure what's going on 😩
I think you need to `import pandas.util._test_decorators as td` at the top of `test_multi.py`
closing as stale, if you'd like to continue, pls ping.
@jreback, my understanding was that this commit was ready, but had been waiting for your approval. I found it hard to get responses from you. if you're ready to go, i'm ready to play my part. it should be almost there.
i left multiple comments 
you can update and rebase in master
Hello @benediamond! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`pandas/core/arrays/categorical.py`](https://github.com/benediamond/pandas/blob/702594b13acf3e89f6540fc77cbcd4364621be8f/pandas/core/arrays/categorical.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexes/test_multi.py`](https://github.com/benediamond/pandas/blob/702594b13acf3e89f6540fc77cbcd4364621be8f/pandas/tests/indexes/test_multi.py) !


@jreback and others: looks like this issue has _since been fixed_ by PR #22072, see also issue #15457. I have gone ahead and preserved the test I wrote in my rebase, in case you'd like to merge it in---if not no worries and this (as well as issue #21298) can be closed! (note that issue #15457 was a somewhat different problem). thanks for everyone's time and patience.
@benediamond closing per your latest comment. Ping if we need to reopen
I want to point out that when merging dataframes with enums as multi-index it throws
``RuntimeWarning: The values in the array are unorderable. Pass `sort=False` to suppress this warning.``
so I think the support for enums is not yet completed
you could open a separate issue with a reproducible example

but we have no official support for enums and would need a champion in the community