Repository: modin-project/modin

Issue:
DataFrame.groupby by multiple columns 
<!--
General questions should be asked on the mailing list modin-dev@googlegroups.com.

Before submitting an issue, please fill out the following form.
-->

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:  macOS Mojave
- **Modin installed from (source or binary)**: pip install modin
- **Modin version**: 0.2.2
- **Python version**:3.6.6
- **Exact command to reproduce**: DataFrame.groupby([column1,columns2])

<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->
Hi guys, I've noticed an incompatibility with the regular pandas.DataFrame.groupby()
I've this DataFrame with multiple columns:

```
import pandas as pd  
df = pd.DataFrame(np.concatenate([np.ones([2,3]),2*np.ones([1,3]),3*np.ones([3,3])]),columns['a','b','c'])
for i in df.groupby(['a','b']):
    display(i[0],i[1])
```
it does the right thing returning 3 GroupBy objects, while when you tr:

```
import modin.pandas as pd  
df = pd.DataFrame(np.concatenate([np.ones([2,3]),2*np.ones([1,3]),3*np.ones([3,3])]),columns['a','b','c'])
for i in df.groupby(['a','b']):
    display(i[0],i[1])
```

it seems to not grouping in the right way and it returns the name of the column.






Discussion:
Thanks for the report!

It looks like the way we are presenting the groups when iterating over the grouped dataframe is causing this issue. https://github.com/modin-project/modin/blob/master/modin/pandas/groupby.py#L77. We use a different way of grouping for the iterator because it is kind of a special case. I will get this fixed, thanks again for the report!

Resolving patch:
diff --git a/modin/pandas/groupby.py b/modin/pandas/groupby.py
--- a/modin/pandas/groupby.py
+++ b/modin/pandas/groupby.py
@@ -6,6 +6,7 @@
 import pandas.core.groupby
 from pandas.core.dtypes.common import is_list_like
 import pandas.core.common as com
+import warnings
 
 from .utils import _inherit_docstrings
 
@@ -28,6 +29,8 @@ def __init__(
         self._index = self._query_compiler.index
         self._columns = self._query_compiler.columns
         self._by = by
+        # This tells us whether or not there are multiple columns/rows in the groupby
+        self._is_multi_by = all(obj in self._df for obj in self._by)
         self._level = level
         self._kwargs = {
             "sort": sort,
@@ -57,10 +60,32 @@ def __getattr__(self, key):
     @property
     def _index_grouped(self):
         if self._index_grouped_cache is None:
-            if self._axis == 0:
-                self._index_grouped_cache = self._index.groupby(self._by)
+            if self._is_multi_by:
+                # Because we are doing a collect (to_pandas) here and then groupby, we
+                # end up using pandas implementation. Add the warning so the user is
+                # aware.
+                warnings.warn("Defaulting to Pandas implementation", UserWarning)
+                if self._axis == 0:
+                    self._index_grouped_cache = {
+                        k: v.index
+                        for k, v in self._df._query_compiler.getitem_column_array(
+                            self._by
+                        )
+                        .to_pandas()
+                        .groupby(by=self._by)
+                    }
+                else:
+                    self._index_grouped_cache = {
+                        k: v.index
+                        for k, v in self._df._query_compiler.getitem_row_array(self._by)
+                        .to_pandas()
+                        .groupby(by=self._by)
+                    }
             else:
-                self._index_grouped_cache = self._columns.groupby(self._by)
+                if self._axis == 0:
+                    self._index_grouped_cache = self._index.groupby(self._by)
+                else:
+                    self._index_grouped_cache = self._columns.groupby(self._by)
         return self._index_grouped_cache
 
     _keys_and_values_cache = None
@@ -380,7 +405,7 @@ def _apply_agg_function(self, f, **kwargs):
         assert callable(f), "'{0}' object is not callable".format(type(f))
         from .dataframe import DataFrame
 
-        if all(obj in self._df for obj in self._by):
+        if self._is_multi_by:
             return self._default_to_pandas(f, **kwargs)
 
         new_manager = self._query_compiler.groupby_agg(


Test patch: