Repository: modin-project/modin
Pull request: Fixed MultiIndex.from_frame implementation
<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/CONTRIBUTING.html
if you have questions about contributing.
-->

## What do these changes do?

<!-- Please give a short brief about these changes. -->

- [x] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/CONTRIBUTING.html)
- [x] passes `flake8 modin`
- [x] passes `black --check modin`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #2575 <!-- issue must be created for each patch -->
- [x] tests added and passing


Problem statement:
modin DataFrame does not qualify as DataFrame in pd.MultiIndex.from_frame
### System information
- **OS Platform and Distribution**: Linux Ubuntu 16.04
- **Modin version**: 0.8.2
- **Python version**: 3.8.2
- **Code we can use to reproduce**:

```python
from modin import pandas as pd
df = pd.DataFrame(
     [["bar", 1], ["bar", 2], ["foo", 1], ["foo", 2]],
     columns=["first", "second"],
)
print(type(df))
pd.MultiIndex.from_frame(df)
```

### Problem
When trying to look at the indices of the DataFrame created as above the type of the DataFrame is not recognized.

So, with

```python
print(type(df))
# <class 'modin.pandas.dataframe.DataFrame'>
```

the output of the following 

```python
pd.MultiIndex.from_frame(df)
```

returns

```
TypeError: Input must be a DataFrame
```

### Source code / logs

```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/myscript.py in <module>
      4      columns=["first", "second"],
      5 )
----> 6 pd.MultiIndex.from_frame(df)

~/somepath/.venv/lib/python3.8/site-packages/pandas/core/indexes/multi.py in from_frame(cls, df, sortorder, names)
    626         """
    627         if not isinstance(df, ABCDataFrame):
--> 628             raise TypeError("Input must be a DataFrame")
    629 
    630         column_names, columns = zip(*df.items())
TypeError: Input must be a DataFrame
```

Hints from issue discussion:
Yes, we currently reuse Pandas class `MultiIndex` as is, and therefore its method `from_frame` expects a Pandas DataFrame type, not Modin. As a temporary workaround you can write `pd.MultiIndex.from_frame(df._to_pandas())` and it should work.

The fix for this can be relatively simple, we can just monkey-patch the `from_frame` classmethod.

```python
from pandas import MultiIndex

def from_modin_frame(df):
    if isinstance(df, modin.pandas.DataFrame):
        df = df._to_pandas()
    return MultiIndex.from_frame(df)

MultiIndex.from_frame = from_modin_frame
```

Commit message: FIX-#2575: Patch MultiIndex with out implementation of from_frame

Signed-off-by: Gregory Shimansky <gregory.shimansky@intel.com>

Patch to review:
diff --git a/modin/pandas/test/dataframe/test_indexing.py b/modin/pandas/test/dataframe/test_indexing.py
index d7201dea7c6..db81111de73 100644
--- a/modin/pandas/test/dataframe/test_indexing.py
+++ b/modin/pandas/test/dataframe/test_indexing.py
@@ -1325,3 +1325,17 @@ def test_index_order():
             getattr(df_modin, func)(level=0).index,
             getattr(df_pandas, func)(level=0).index,
         )
+
+
+@pytest.mark.parametrize("data", test_data_values, ids=test_data_keys)
+@pytest.mark.parametrize("sortorder", [0, 3, 5])
+def test_multiindex_from_frame(data, sortorder):
+    modin_df, pandas_df = create_test_dfs(data)
+
+    def call_from_frame(df):
+        if type(df).__module__.startswith("pandas"):
+            return pandas.MultiIndex.from_frame(df, sortorder)
+        else:
+            return pd.MultiIndex.from_frame(df, sortorder)
+
+    eval_general(modin_df, pandas_df, call_from_frame, comparator=assert_index_equal)
diff --git a/modin/pandas/utils.py b/modin/pandas/utils.py
index 1c631514c15..c8be991258b 100644
--- a/modin/pandas/utils.py
+++ b/modin/pandas/utils.py
@@ -13,6 +13,8 @@
 
 """Implement utils for pandas component."""
 
+from pandas import MultiIndex
+
 
 def from_non_pandas(df, index, columns, dtype):
     """
@@ -105,3 +107,35 @@ def is_scalar(obj):
     from .base import BasePandasDataset
 
     return not isinstance(obj, BasePandasDataset) and pandas_is_scalar(obj)
+
+
+def from_modin_frame(df, sortorder=None, names=None):
+    """
+    Make a MultiIndex from a DataFrame.
+
+    Parameters
+    ----------
+    df : DataFrame
+        DataFrame to be converted to MultiIndex.
+    sortorder : int, optional
+        Level of sortedness (must be lexicographically sorted by that
+        level).
+    names : list-like, optional
+        If no names are provided, use the column names, or tuple of column
+        names if the columns is a MultiIndex. If a sequence, overwrite
+        names with the given sequence.
+
+    Returns
+    -------
+    MultiIndex
+        The MultiIndex representation of the given DataFrame.
+    """
+    from .dataframe import DataFrame
+
+    if isinstance(df, DataFrame):
+        df = df._to_pandas()
+    return _original_pandas_MultiIndex_from_frame(df, sortorder, names)
+
+
+_original_pandas_MultiIndex_from_frame = MultiIndex.from_frame
+MultiIndex.from_frame = from_modin_frame


Code review comments:
diff_hunk: @@ -105,3 +107,35 @@ def is_scalar(obj):
     from .base import BasePandasDataset
 
     return not isinstance(obj, BasePandasDataset) and pandas_is_scalar(obj)
+
+
+def from_modin_frame(df, sortorder=None, names=None):
+    """
+    Make a MultiIndex from a DataFrame.
+
+    Parameters
+    ----------
+    df : DataFrame
+        DataFrame to be converted to MultiIndex.
+    sortorder : int, optional
+        Level of sortedness (must be lexicographically sorted by that
+        level).
+    names : list-like, optional
+        If no names are provided, use the column names, or tuple of column
+        names if the columns is a MultiIndex. If a sequence, overwrite
+        names with the given sequence.
+
+    Returns
+    -------
+    MultiIndex
+        The MultiIndex representation of the given DataFrame.
+    """
+    from .dataframe import DataFrame
+
+    if isinstance(df, DataFrame):
+        df = df._to_pandas()
+    return _original_pandas_MultiIndex_from_frame(df, sortorder, names)
path: modin/pandas/utils.py
text: @user1:
What is the need to use this additional function?

@author:
What do you mean "additional function"?

@user1:
I mean using this line `MultiIndex.from_frame(df, sortorder, names)`. I think the line 143 is extra.

@author:
That is how it I initially implemented it, but it results in an infinite recursion because `MultiIndex.from_frame` is `from_modin_frame_to_mi` now. This is a patch which makes a change that you request:

```patch
diff --git a/modin/pandas/utils.py b/modin/pandas/utils.py
index e151d7c..4c06005 100644
--- a/modin/pandas/utils.py
+++ b/modin/pandas/utils.py
@@ -137,8 +137,7 @@ def from_modin_frame_to_mi(df, sortorder=None, names=None):

         ErrorMessage.default_to_pandas("`MultiIndex.from_frame`")
         df = df._to_pandas()
-    return _original_pandas_MultiIndex_from_frame(df, sortorder, names)
+    return MultiIndex.from_frame(df, sortorder, names)


-_original_pandas_MultiIndex_from_frame = MultiIndex.from_frame
 MultiIndex.from_frame = from_modin_frame_to_mi
```
and this is how it works:
```
In [2]: import modin.pandas as pd
UserWarning: Modin Ray engine was started with 15 GB free space avaliable, if it is not enough for your application, please set environment variable MODIN_ON_RAY_PLASMA_DIR=/directory/without/space/limiting

In [3]: import pandas

In [4]:     mdf = pd.DataFrame(
   ...:         [["bar", 1, "1", 1.1], ["bar", 2, "2", 2.2], ["foo", 1, "3", 3.3], ["foo", 2, "4", 4.4]],
   ...:             columns=["first", "second", "data_str", "data_f"],
   ...:             )
UserWarning: Distributing <class 'list'> object. This may take some time.

In [5]:     pdf = pandas.DataFrame(
   ...:         [["bar", 1, "1", 1.1], ["bar", 2, "2", 2.2], ["foo", 1, "3", 3.3], ["foo", 2, "4", 4.4]],
   ...:             columns=["first", "second", "data_str", "data_f"],
   ...:             )

In [6]: pd.MultiIndex.from_frame(pdf)
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-6-01e1da826c3c> in <module>
----> 1 pd.MultiIndex.from_frame(pdf)

/localdisk/gashiman/modin/modin/pandas/utils.py in from_modin_frame_to_mi(df, sortorder, names)
    138         ErrorMessage.default_to_pandas("`MultiIndex.from_frame`")
    139         df = df._to_pandas()
--> 140     return MultiIndex.from_frame(df, sortorder, names)
    141
    142

... last 1 frames repeated, from the frame below ...

/localdisk/gashiman/modin/modin/pandas/utils.py in from_modin_frame_to_mi(df, sortorder, names)
    138         ErrorMessage.default_to_pandas("`MultiIndex.from_frame`")
    139         df = df._to_pandas()
--> 140     return MultiIndex.from_frame(df, sortorder, names)
    141
    142

RecursionError: maximum recursion depth exceeded while calling a Python object

In [7]: pd.MultiIndex.from_frame(mdf)
UserWarning: `MultiIndex.from_frame` defaulting to pandas implementation.
To request implementation, send an email to feature_requests@user2.org.
---------------------------------------------------------------------------
RecursionError                            Traceback (most recent call last)
<ipython-input-7-31daf7b77262> in <module>
----> 1 pd.MultiIndex.from_frame(mdf)

/localdisk/gashiman/modin/modin/pandas/utils.py in from_modin_frame_to_mi(df, sortorder, names)
    138         ErrorMessage.default_to_pandas("`MultiIndex.from_frame`")
    139         df = df._to_pandas()
--> 140     return MultiIndex.from_frame(df, sortorder, names)
    141
    142

... last 1 frames repeated, from the frame below ...

/localdisk/gashiman/modin/modin/pandas/utils.py in from_modin_frame_to_mi(df, sortorder, names)
    138         ErrorMessage.default_to_pandas("`MultiIndex.from_frame`")
    139         df = df._to_pandas()
--> 140     return MultiIndex.from_frame(df, sortorder, names)
    141
    142

RecursionError: maximum recursion depth exceeded while calling a Python object
```

@user1:
Oh, I see.

diff_hunk: @@ -105,3 +107,35 @@ def is_scalar(obj):
     from .base import BasePandasDataset
 
     return not isinstance(obj, BasePandasDataset) and pandas_is_scalar(obj)
+
+
+def from_modin_frame(df, sortorder=None, names=None):
+    """
+    Make a MultiIndex from a DataFrame.
path: modin/pandas/utils.py
text: @user1:
please, rename `MultiIndex` -> `pandas.MultiIndex` here and below.

diff_hunk: @@ -105,3 +107,35 @@ def is_scalar(obj):
     from .base import BasePandasDataset
 
     return not isinstance(obj, BasePandasDataset) and pandas_is_scalar(obj)
+
+
+def from_modin_frame(df, sortorder=None, names=None):
path: modin/pandas/utils.py
text: @user1:
Because `from_modin_frame` doesn't belong to any class, it is not clear what return value is. So, I would suggest renaming function to `from_modin_frame_to_mi`.