API: re-allow duplicate index level names
State: closed

One possible solution for https://github.com/pandas-dev/pandas/issues/19029

WIP (need to clean up tests and possibly re-add some ones that have been removed in https://github.com/pandas-dev/pandas/pull/18882)

Labels:
name: API Design
name: MultiIndex

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=h1) Report
> Merging [#21423](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/44c54607828f03d2b4b172838d5270e66b2a5312?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21423/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21423      +/-   ##
==========================================
+ Coverage    91.9%    91.9%   +<.01%     
==========================================
  Files         154      154              
  Lines       49558    49560       +2     
==========================================
+ Hits        45545    45548       +3     
+ Misses       4013     4012       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.27% <100%> (ø)` | :arrow_up: |
| #single | `42.02% <20%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/21423/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.78% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/multi.py](https://codecov.io/gh/pandas-dev/pandas/pull/21423/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9tdWx0aS5weQ==) | `94.96% <100%> (+0.07%)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/21423/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `93.13% <0%> (-0.04%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/21423/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.19% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=footer). Last update [44c5460...fde8231](https://codecov.io/gh/pandas-dev/pandas/pull/21423?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

+1 in general.

Will want a semi-detailed release note, potentially laying out future plans with respect to duplicate names?
> Will want a semi-detailed release note, potentially laying out future plans with respect to duplicate names?

Until we don't exactly know what those future plans are, I would rather keep it limited (only noting that duplicated level names are allowed again?)
@jreback @toobaz ready for review

I went through the tests and added back some of the tests removed in https://github.com/pandas-dev/pandas/pull/18882

I also added back `MultiIndex._reference_duplicate_name` to use it in `stack`/`unstack` (so we can there raise an informative error message). However, I would personally prefer that `MultiIndex._get_level_number` directly raises this informative warning, then the extra checking in `stack`/`unstack` can be removed. However, that is a slight change in behaviour for eg `get_level_values` (KeyError -> ValueError), so I would rather leave that for 0.24.0.
> However, I would personally prefer that MultiIndex._get_level_number directly raises this informative warning

That was also the original intent of:

https://github.com/pandas-dev/pandas/blob/a620e7255ce896c9185275d82172513268b1a719/pandas/core/indexes/multi.py#L771-L776

where already an informative ValueError is raised, but now the ValueError is catched, and then a generic KeyError is raised (this was changed 5 years ago: https://github.com/pandas-dev/pandas/commit/bdf270c77ef8c84a04fcf25f805b56b00e6725c9#diff-dc96e62847cdfe77b570cf9c6b0e8086R1536)
> Until we don't exactly know what those future plans are, I would rather keep it limited (only noting that duplicated level names are allowed again?)

Agreed.
fair enough, is there an issue to clean this up?
@jreback I am doing a PR, but ran into some issues: https://github.com/pandas-dev/pandas/issues/21677
Opened the PR to not forget his (https://github.com/pandas-dev/pandas/pull/21678), but as mentioned, we first need to discuss the other issue.