Repository: pandas-dev/pandas

Issue:
Add method to flatten all multi-index levels
Sometimes it is useful to flatten all levels of a multi-index. For example, when pivoting data into a wide format, the new columns are generally multi-indexed. However, when exporting to CSV, sometimes it might be desirable to have only one header row.

AFAIK, there is no dedicated method to flatten an existing multi-index. Assuming I want to combine the multi-index levels with an underscore, I would do this to get one header row:

```
df.columns = ["_".join(v) for v in df.columns.values]
````

This is not really obvious, and maybe a dedicated `MultiIndex` method should be added, e.g. `df.columns.flatten(sep="_", inplace=False)`. What do you think?


Discussion:
having a ``.to_index(sep=None)`` would be useful, the default would be equivalent of ``.tolist()``, IOW return tuples, while a ``sep`` could stringify and join. Note that this is a bit tricky because for example what do you do with a ``Timestamp``?
I like your idea of a `to_index` method. Good point, specifying a `sep` character implies that individual entries are converted to strings. How do `Timestamp` objects behave when I try to `str` them? If this is not possible, the method could raise an error in this case (and only work for string or int index objects).
Actually, is this even possible in the use case I described? I mean, I was referring to a multi-index created by `pivot_table` - can a level consist of timestamps?
```
In [29]: index=pd.MultiIndex.from_product([list('abc'), pd.date_range('20180101', periods=3)])

In [30]: index
Out[30]: 
MultiIndex(levels=[['a', 'b', 'c'], [2018-01-01 00:00:00, 2018-01-02 00:00:00, 2018-01-03 00:00:00]],
           labels=[[0, 0, 0, 1, 1, 1, 2, 2, 2], [0, 1, 2, 0, 1, 2, 0, 1, 2]])


In [32]: pd.Index(['_'.join([str(_) for _ in v]) for v in index.values])
Out[32]: 
Index(['a_2018-01-01 00:00:00', 'a_2018-01-02 00:00:00',
       'a_2018-01-03 00:00:00', 'b_2018-01-01 00:00:00',
       'b_2018-01-02 00:00:00', 'b_2018-01-03 00:00:00',
       'c_2018-01-01 00:00:00', 'c_2018-01-02 00:00:00',
       'c_2018-01-03 00:00:00'],
      dtype='object')
```
slightly OT: note that ``.wide_to_long`` basically does the inverse of this, meaning it splits columns names on regex. 
Looks good! So I guess no problem then? I didn't know about `.wide_to_long`, maybe adding a `.long_to_wide` would make sense?

Just to motivate this idea: I love how `unite` and `spread` from the `tidyr` R package do exactly what I describe here.
FWIW There is a very popular [question related to this on SO](https://stackoverflow.com/questions/14507794/python-pandas-how-to-flatten-a-hierarchical-index-in-columns), so by that metric it seems like quite a few people could benefit from a solution similar to what is suggested here.

Resolving patch:
diff --git a/pandas/core/indexes/base.py b/pandas/core/indexes/base.py
--- a/pandas/core/indexes/base.py
+++ b/pandas/core/indexes/base.py
@@ -1113,6 +1113,26 @@ def _format_attrs(self):
         """
         return format_object_attrs(self)
 
+    def to_flat_index(self):
+        """
+        Identity method.
+
+        .. versionadded:: 0.24.0
+
+        This is implemented for compatability with subclass implementations
+        when chaining.
+
+        Returns
+        -------
+        pd.Index
+            Caller.
+
+        See Also
+        --------
+        MultiIndex.to_flat_index : Subclass implementation.
+        """
+        return self
+
     def to_series(self, index=None, name=None):
         """
         Create a Series with both index and values equal to the index keys
diff --git a/pandas/core/indexes/multi.py b/pandas/core/indexes/multi.py
--- a/pandas/core/indexes/multi.py
+++ b/pandas/core/indexes/multi.py
@@ -193,6 +193,7 @@ class MultiIndex(Index):
     set_levels
     set_labels
     to_frame
+    to_flat_index
     is_lexsorted
     sortlevel
     droplevel
@@ -1246,6 +1247,34 @@ def to_hierarchical(self, n_repeat, n_shuffle=1):
                       FutureWarning, stacklevel=2)
         return MultiIndex(levels=levels, labels=labels, names=names)
 
+    def to_flat_index(self):
+        """
+        Convert a MultiIndex to an Index of Tuples containing the level values.
+
+        .. versionadded:: 0.24.0
+
+        Returns
+        -------
+        pd.Index
+            Index with the MultiIndex data represented in Tuples.
+
+        Notes
+        -----
+        This method will simply return the caller if called by anything other
+        than a MultiIndex.
+
+        Examples
+        --------
+        >>> index = pd.MultiIndex.from_product(
+        ...     [['foo', 'bar'], ['baz', 'qux']],
+        ...     names=['a', 'b'])
+        >>> index.to_flat_index()
+        Index([('foo', 'baz'), ('foo', 'qux'),
+               ('bar', 'baz'), ('bar', 'qux')],
+              dtype='object')
+        """
+        return Index(self.values, tupleize_cols=False)
+
     @property
     def is_all_dates(self):
         return False


Test patch:
diff --git a/pandas/tests/indexes/multi/test_conversion.py b/pandas/tests/indexes/multi/test_conversion.py
--- a/pandas/tests/indexes/multi/test_conversion.py
+++ b/pandas/tests/indexes/multi/test_conversion.py
@@ -170,3 +170,11 @@ def test_to_series_with_arguments(idx):
     assert s.values is not idx.values
     assert s.index is not idx
     assert s.name != idx.name
+
+
+def test_to_flat_index(idx):
+    expected = pd.Index((('foo', 'one'), ('foo', 'two'), ('bar', 'one'),
+                         ('baz', 'two'), ('qux', 'one'), ('qux', 'two')),
+                        tupleize_cols=False)
+    result = idx.to_flat_index()
+    tm.assert_index_equal(result, expected)
diff --git a/pandas/tests/indexes/test_base.py b/pandas/tests/indexes/test_base.py
--- a/pandas/tests/indexes/test_base.py
+++ b/pandas/tests/indexes/test_base.py
@@ -2266,6 +2266,14 @@ def test_tab_complete_warning(self, ip):
             with provisionalcompleter('ignore'):
                 list(ip.Completer.completions('idx.', 4))
 
+    def test_to_flat_index(self, indices):
+        # 22866
+        if isinstance(indices, MultiIndex):
+            pytest.skip("Separate expectation for MultiIndex")
+
+        result = indices.to_flat_index()
+        tm.assert_index_equal(result, indices)
+
 
 class TestMixedIntIndex(Base):
     # Mostly the tests from common.py for which the results differ