API: harmonize Index.to_flat_index to MultiIndex.to_flat_index
State: closed

- [x] closes #23670
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

After "implementing" it (that is, just unifying the method), I'm even more convinced about my [idea](https://github.com/pandas-dev/pandas/issues/23670#issue-380340023)... but @WillAyd and @h-vetinari feel free to consider this just a stimulus to resurrect (and conclude) the discussion ;-)


Labels:
name: API Design
name: MultiIndex

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=h1) Report
> Merging [#25094](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/bb43726e1f52a0ddee45fcf485690719f262870d?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25094/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25094      +/-   ##
==========================================
+ Coverage   92.37%   92.37%   +<.01%     
==========================================
  Files         166      166              
  Lines       52420    52421       +1     
==========================================
+ Hits        48423    48424       +1     
  Misses       3997     3997
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.79% <100%> (ø)` | :arrow_up: |
| #single | `42.87% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/multi.py](https://codecov.io/gh/pandas-dev/pandas/pull/25094/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9tdWx0aS5weQ==) | `95.59% <ø> (-0.01%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25094/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.55% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=footer). Last update [bb43726...9070163](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=h1) Report
> Merging [#25094](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/683c7b55f5195fdf4f524239066cbf6f1301f0e7?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25094/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25094      +/-   ##
==========================================
+ Coverage   92.34%   92.35%   +<.01%     
==========================================
  Files         166      166              
  Lines       52314    52315       +1     
==========================================
+ Hits        48311    48313       +2     
+ Misses       4003     4002       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.78% <100%> (ø)` | :arrow_up: |
| #single | `42.84% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/multi.py](https://codecov.io/gh/pandas-dev/pandas/pull/25094/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9tdWx0aS5weQ==) | `95.59% <ø> (-0.01%)` | :arrow_down: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25094/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.55% <100%> (ø)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25094/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `88.05% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=footer). Last update [683c7b5...f7935c2](https://codecov.io/gh/pandas-dev/pandas/pull/25094?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

> @toobaz: [...] but @WillAyd and @h-vetinari feel free to consider this just a stimulus to resurrect (and conclude) the discussion ;-)

A bit late to the party here, but I still don't understand the need for flattening MIs. I looked at [the original issue](https://github.com/pandas-dev/pandas/issues/19950) and the usecase there (squashing MI-columns for csv output) could IMO more idiomatically be solved by something like `MI.str.cat(', ')`

Of course, the latter current does not work yet (see also #23679), but I continue to think that tuples should not be usable in indexes. They are already necessary to "talk" to MIs, which makes their use ambiguous, plus it's incredibly hard to get right (and we don't, for large parts of the code base, including basic indexing), not least because it needs so much special casing for interacting with numpy (which treats tuples as arrays by default). See #24688 
>  I continue to think that tuples should not be usable in indexes

Will comment on #24688 in few seconds. But this PR assumes that ``to_flat_index`` exists, and tries to fix it. This is a simple PR, worth doing even if ``to_flat_index`` disappears in a couple of releases. Doesn't need all that discussion.

> len 1 MultiIndexes, while supported are not very useful. This elevates them to a first class citizen, not a good thing.

No, this PR doesn't change 1 level ``MultiIndex``es in any way.

> @toobaz pls in this issue lay out why you think this is a good think

This is quite clear in the issue: #23670

Anyway, in short, it's really simple:
1) ``Index.to_flat_index()`` doesn't make much sense per se - the index _is_ already flat
2) we support it only for compatibility with ``MultiIndex``
3) ... but in the current form, it is _not_ compatible. The output for ``MultiIndex`` with ``n`` levels is an ``Index`` with tuples of length ``n``
5) So the output for a flat ``Index``, which has 1 level, should be an ``Index`` with tuples of length 1
6) .. which is incidentally the output on a ``MultiIndex`` with 1 level. Which is good, because 1-level ``MultiIndexes`` should behave like flat ``Index``es (I couldn't care less about whether they are popular, encouraged or blamed: as long as they are allowed, we want them to behave consistently)

This is really a simple thing, even @WillAyd is [now neutral](https://github.com/pandas-dev/pandas/issues/23670#issuecomment-460010083). Please provide any reason not to merge this, or let's proceed.
Closing for now - I think it's worth continuing conversation in issue before updating any PRs on this