Repository: modin-project/modin

Issue:
groupby where by is a list of integers can cause problems based on internal index
### Describe the problem
<!-- Describe the problem clearly here. -->
`groupby` can return incorrect or truncated results when `by` is a list of integers that are all contained in the `index` in each partition.

### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->

```
In [1]: import modin.pandas as pd
Process STDOUT and STDERR is being redirected to /tmp/raylogs/.
Waiting for redis server at 127.0.0.1:49938 to respond...
Waiting for redis server at 127.0.0.1:39318 to respond...
Starting local scheduler with the following resources: {'CPU': 8, 'GPU': 0}.

======================================================================
View the web UI at http://localhost:8888/notebooks/ray_ui75359.ipynb?token=546e4e9fca85392d39d82072b5d206ef67d2753a5b3743e3
======================================================================

In [2]: import ray

In [3]: import pandas

In [4]: pandas_df = pandas.DataFrame({'col1': [0, 1, 2, 3],
                                      'col2': [4, 5, 6, 7],
                                      'col3': [3, 8, 12, 10],
                                      'col4': [17, 13, 16, 15],
                                      'col5': [-4, -5, -6, -7]})

In [5]: modin_df = pd.DataFrame(pandas_df)

In [6]: for k, v in modin_df.groupby(by=[1,2,1,2]):
    print(k)
    print(v)
```


Discussion:
This was found when testing #49.

Resolving patch:
diff --git a/modin/pandas/groupby.py b/modin/pandas/groupby.py
--- a/modin/pandas/groupby.py
+++ b/modin/pandas/groupby.py
@@ -92,9 +92,17 @@ def _grouped_partitions(self):
 
         # It is expensive to put this multiple times, so let's just put it once
         remote_by = ray.put(self._by)
+        remote_index = \
+            [ray.put(v.index) for _, v in
+             self._df._col_metadata._coord_df.copy().groupby(by='partition')] \
+            if self._axis == 0 \
+            else [ray.put(v.index) for _, v in
+                  self._df._row_metadata._coord_df.copy()
+                      .groupby(by='partition')]
 
         if len(self._index_grouped) > 1:
-            return zip(*(groupby._submit(args=(remote_by,
+            return zip(*(groupby._submit(args=(remote_index[i],
+                                               remote_by,
                                                self._axis,
                                                self._level,
                                                self._as_index,
@@ -104,7 +112,7 @@ def _grouped_partitions(self):
                                          + tuple(part.tolist()),
                                          num_return_vals=len(
                                              self._index_grouped))
-                         for part in self._partitions))
+                         for i, part in enumerate(self._partitions)))
         elif self._axis == 0:
             return [self._df._col_partitions]
         else:
@@ -554,10 +562,14 @@ def _apply_df_function(self, f, concat_axis=None):
 
 @ray.remote
 @post_task_gc
-def groupby(by, axis, level, as_index, sort, group_keys, squeeze, *df):
+def groupby(index, by, axis, level, as_index, sort, group_keys, squeeze, *df):
 
     df = pandas.concat(df, axis=axis)
 
+    if axis == 0:
+        df.columns = index
+    else:
+        df.index = index
     return [v for k, v in df.groupby(by=by,
                                      axis=axis,
                                      level=level,


Test patch: