Repository: modin-project/modin

Issue:
df.groupby('index_column_name') results in a key error. In pandas it doesn't
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: I use arch btw
- **Modin version** (`modin.__version__`): 0.8.0
- **Python version**: 3.8
- **Code we can use to reproduce**: 

### Pandas (works fine):
```
import pandas as pd
from pandas import util
df= util.testing.makeMixedDataFrame()
print(df.head())
df = df.to_numpy()
df = pd.DataFrame(df)
df.columns = ["A", "B", "C", "D"]
df = df.set_index("C")
df.groupby("C")
```
### Modin (gives error):
```
import modin.pandas as pd
from pandas import util
df= util.testing.makeMixedDataFrame()
print(df.head())
df = df.to_numpy()
df = pd.DataFrame(df)
df.columns = ["A", "B", "C", "D"]
df = df.set_index("C")
df.groupby("C") #  <- Key Error! (scroll below)

```
<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->

Using groupby with index column name in Pandas does not give a key error and works fine. However in Modin this results in a key error

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->
```
KeyError                                  Traceback (most recent call last)
 in 
      7 df.columns = ["A", "B", "C", "D"]
      8 df = df.set_index("C")
----> 9 df.groupby("C")

~/anaconda3/envs/recnn/lib/python3.8/site-packages/modin/pandas/dataframe.py in groupby(self, by, axis, level, as_index, sort, group_keys, squeeze, observed)
    434                 pass
    435             else:
--> 436                 by = self.__getitem__(by)._query_compiler
    437         elif isinstance(by, Series):
    438             drop = by._parent is self

~/anaconda3/envs/recnn/lib/python3.8/site-packages/modin/pandas/base.py in __getitem__(self, key)
   3458             return self._getitem_slice(indexer)
   3459         else:
-> 3460             return self._getitem(key)
   3461 
   3462     def _getitem_slice(self, key):

~/anaconda3/envs/recnn/lib/python3.8/site-packages/modin/pandas/dataframe.py in _getitem(self, key)
   2422             # return self._getitem_multilevel(key)
   2423         else:
-> 2424             return self._getitem_column(key)
   2425 
   2426     def _getitem_column(self, key):

~/anaconda3/envs/recnn/lib/python3.8/site-packages/modin/pandas/dataframe.py in _getitem_column(self, key)
   2426     def _getitem_column(self, key):
   2427         if key not in self.keys():
-> 2428             raise KeyError("{}".format(key))
   2429         s = DataFrame(
   2430             query_compiler=self._query_compiler.getitem_column_array([key])

KeyError: 'C'
```




Discussion:
I have a code like this
```
    user_dict = {}

    def app(x):
        userid = int(x.index[0])
        if userid not in user_dict:
            user_dict[userid]= {}
        for dict_name, column_name in [('items', 'movieId'), ('ratings', 'rating')]:
            if column_name in x.columns:
                user_dict[userid][dict_name] = x[column_name].values

    
    ratings.apply(app)
```
and I would like to use userId as an index and group by it
So that when app gets called, I get an index (userId) as well as some other column to process


One stupid workaraound I've sound is to copy the column and group by it:
```
import modin.pandas as pd
from pandas import util
df= util.testing.makeMixedDataFrame()
print(df.head())
df = df.to_numpy()
df = pd.DataFrame(df)
df.columns = ["A", "B", "C", "D"]
df['C_copy'] = df['C']
df = df.set_index("C")
df.groupby("C_copy") #  works fine but requires extra memory 

```
@awarebayes Thanks for posting this issue!

We have this working for the `MultiIndex` case but it is not working for other types of `Index` objects. We will fix this it should be relatively straightforward.

For your workaround, a slightly better way (until we fix the problem) would be as follows:

```python
import modin.pandas as pd
from pandas import util
df= util.testing.makeMixedDataFrame()
print(df.head())
df = df.to_numpy()
df = pd.DataFrame(df)
df.columns = ["A", "B", "C", "D"]
df.index = df["C"]
df.groupby("C")
```
Ran into the same issue today.
Have to say I'm so surprised by trying modin and getting crashes on just a plain and simple .groupby. 
I mean, what else would you use pandas if not for groupbying and making things? :)

Resolving patch:
diff --git a/modin/data_management/functions/groupby_function.py b/modin/data_management/functions/groupby_function.py
--- a/modin/data_management/functions/groupby_function.py
+++ b/modin/data_management/functions/groupby_function.py
@@ -30,7 +30,7 @@ def caller(
             numeric_only=True,
             drop=False,
         ):
-            if not isinstance(by, type(query_compiler)):
+            if not isinstance(by, (type(query_compiler), str)):
                 by = try_cast_to_pandas(by)
                 return query_compiler.default_to_pandas(
                     lambda df: map_func(
@@ -53,38 +53,22 @@ def caller(
             as_index = groupby_args.get("as_index", True)
             observed = groupby_args.get("observed", False)
 
-            def _map(df, other):
-                def compute_map(df, other):
-                    # Set `as_index` to True to track the metadata of the grouping object
-                    # It is used to make sure that between phases we are constructing the
-                    # right index and placing columns in the correct order.
+            if isinstance(by, str):
+
+                def _map(df):
+                    # Set `as_index` to True to track the metadata of the grouping
+                    # object It is used to make sure that between phases we are
+                    # constructing the right index and placing columns in the correct
+                    # order.
                     groupby_args["as_index"] = True
                     groupby_args["observed"] = True
-                    other = other.squeeze(axis=axis ^ 1)
-                    if isinstance(other, pandas.DataFrame):
-                        df = pandas.concat(
-                            [df] + [other[[o for o in other if o not in df]]], axis=1
-                        )
-                        other = list(other.columns)
+
                     result = map_func(
-                        df.groupby(by=other, axis=axis, **groupby_args), **map_args
+                        df.groupby(by=by, axis=axis, **groupby_args), **map_args
                     )
-                    # if `other` has category dtype, then pandas will drop that
-                    # column after groupby, inserting it back to correctly process
-                    # reduce phase
-                    if (
-                        drop
-                        and not as_index
-                        and isinstance(other, pandas.Series)
-                        and isinstance(other.dtype, pandas.CategoricalDtype)
-                        and result.index.name is not None
-                        and result.index.name not in result.columns
-                    ):
-                        result.insert(
-                            loc=0, column=result.index.name, value=result.index
-                        )
-                    # The _modin_groupby_ prefix indicates that this is the first partition,
-                    # and since we may need to insert the grouping data in the reduce phase
+                    # The _modin_groupby_ prefix indicates that this is the first
+                    # partition, and since we may need to insert the grouping data in
+                    # the reduce phase
                     if (
                         not isinstance(result.index, pandas.MultiIndex)
                         and result.index.name is not None
@@ -95,12 +79,58 @@ def compute_map(df, other):
                         )
                     return result
 
-                try:
-                    return compute_map(df, other)
-                # This will happen with Arrow buffer read-only errors. We don't want to copy
-                # all the time, so this will try to fast-path the code first.
-                except ValueError:
-                    return compute_map(df.copy(), other.copy())
+            else:
+
+                def _map(df, other):
+                    def compute_map(df, other):
+                        # Set `as_index` to True to track the metadata of the grouping object
+                        # It is used to make sure that between phases we are constructing the
+                        # right index and placing columns in the correct order.
+                        groupby_args["as_index"] = True
+                        groupby_args["observed"] = True
+
+                        other = other.squeeze(axis=axis ^ 1)
+                        if isinstance(other, pandas.DataFrame):
+                            df = pandas.concat(
+                                [df] + [other[[o for o in other if o not in df]]],
+                                axis=1,
+                            )
+                            other = list(other.columns)
+                        result = map_func(
+                            df.groupby(by=other, axis=axis, **groupby_args), **map_args
+                        )
+                        # if `other` has category dtype, then pandas will drop that
+                        # column after groupby, inserting it back to correctly process
+                        # reduce phase
+                        if (
+                            drop
+                            and not as_index
+                            and isinstance(other, pandas.Series)
+                            and isinstance(other.dtype, pandas.CategoricalDtype)
+                            and result.index.name is not None
+                            and result.index.name not in result.columns
+                        ):
+                            result.insert(
+                                loc=0, column=result.index.name, value=result.index
+                            )
+                        # The _modin_groupby_ prefix indicates that this is the first partition,
+                        # and since we may need to insert the grouping data in the reduce phase
+                        if (
+                            not isinstance(result.index, pandas.MultiIndex)
+                            and result.index.name is not None
+                            and result.index.name in result.columns
+                        ):
+                            result.index.name = "{}{}".format(
+                                "_modin_groupby_", result.index.name
+                            )
+                        return result
+
+                    try:
+                        return compute_map(df, other)
+                    # This will happen with Arrow buffer read-only errors. We don't want to copy
+                    # all the time, so this will try to fast-path the code first.
+                    except ValueError:
+                        return compute_map(df.copy(), other.copy())
 
             def _reduce(df):
                 def compute_reduce(df):
@@ -147,9 +177,14 @@ def compute_reduce(df):
                     return compute_reduce(df.copy())
 
             # TODO: try to precompute `new_index` and `new_columns`
-            new_modin_frame = qc._modin_frame.groupby_reduce(
-                axis, by._modin_frame, _map, _reduce
-            )
+            if isinstance(by, str):
+                new_modin_frame = qc._modin_frame._map_reduce(
+                    axis, _map, reduce_func=_reduce, preserve_index=False
+                )
+            else:
+                new_modin_frame = qc._modin_frame.groupby_reduce(
+                    axis, by._modin_frame, _map, _reduce
+                )
             result = query_compiler.__constructor__(new_modin_frame)
             if result.index.name == "__reduced__":
                 result.index.name = None
diff --git a/modin/pandas/dataframe.py b/modin/pandas/dataframe.py
--- a/modin/pandas/dataframe.py
+++ b/modin/pandas/dataframe.py
@@ -459,6 +459,8 @@ def groupby(
             if (
                 self._query_compiler.has_multiindex(axis=axis)
                 and by in self.axes[axis].names
+                or hasattr(self.axes[axis], "name")
+                and self.axes[axis].name == by
             ):
                 # In this case we pass the string value of the name through to the
                 # partitions. This is more efficient than broadcasting the values.


Test patch:
diff --git a/modin/pandas/test/test_groupby.py b/modin/pandas/test/test_groupby.py
--- a/modin/pandas/test/test_groupby.py
+++ b/modin/pandas/test/test_groupby.py
@@ -87,11 +87,20 @@ def test_mixed_dtypes_groupby(as_index):
         ("col1",),
         (lambda x: x % 2,),
         (modin_df["col0"].copy(), pandas_df["col0"].copy()),
+        ("col3",),
     ]
 
     for by in by_values:
-        modin_groupby = modin_df.groupby(by=by[0], as_index=as_index)
-        pandas_groupby = pandas_df.groupby(by=by[-1], as_index=as_index)
+        if by_values[0] == "col3":
+            modin_groupby = modin_df.set_index(by[0]).groupby(
+                by=by[0], as_index=as_index
+            )
+            pandas_groupby = pandas_df.set_index(by[0]).groupby(
+                by=by[-1], as_index=as_index
+            )
+        else:
+            modin_groupby = modin_df.groupby(by=by[0], as_index=as_index)
+            pandas_groupby = pandas_df.groupby(by=by[-1], as_index=as_index)
 
         modin_groupby_equals_pandas(modin_groupby, pandas_groupby)
         eval_ngroups(modin_groupby, pandas_groupby)