Repository: databricks/koalas

Issue:
Add support for multiple aggregations on a single column
It'd be useful if we could perform multiple aggregations on a single column like pandas.

```python
df.groupby('a').agg({'b': ['min', 'max', 'mean']})
```


Discussion:
A simple example in pandas to perform multiple aggregations on a single column and flatten multi-level columns
```
>>> import pandas as pd

>>> df = pd.DataFrame({
...   'a': [0, 0, 0, 1, 1, 1],
...   'b': [1, 2, 3, 4, 5, 6]
... })

>>> df_multi_agg = df.groupby('a').agg({'b': ['min', 'max', 'mean']})
>>> df_multi_agg
    b
  min max mean
a
0   1   3    2
1   4   6    5

>>> df_multi_agg.columns
MultiIndex(levels=[['b'], ['min', 'max', 'mean']],
           labels=[[0, 0, 0], [0, 1, 2]])


>>> cols_flattened = df_multi_agg.columns.ravel()
>>> cols_flattened 
array([('b', 'min'), ('b', 'max'), ('b', 'mean')], dtype=object)


>>> df_multi_agg.columns = ['_'.join(c) for c in cols_flattened]
>>> df_multi_agg
   b_min  b_max  b_mean
a
0      1      3       2
1      4      6       5

```
@harupy would you be able to submit a pull request to address this?
What I came up with to address this was

```python
from databricks import koalas as ks

df = ks.DataFrame({
  'x': [1, 2, 2, 3, 3, 3],
  'y': [0, 0, 0, 0, 0, 0],
  'z': [1, 2, 3, 4, 5, 6]
})


agg_dict = {
  'y': 'count',
  'z': ['max', 'min'],
}

# convert to a list of expressions
agg_exprs = []
for col, funcs in agg_dict.items():
  if not isinstance(funcs, (list, tuple)):
    funcs = [funcs]

  for fn in funcs:
    agg_exprs.append(F.expr('{1}({0}) as {0}_{1}'.format(col, fn)))


df._sdf.groupBy('x').agg(*agg_exprs).orderBy('x').show()
```

This gives
```
+---+-------+-----+-----+
|  x|y_count|z_max|z_min|
+---+-------+-----+-----+
|  1|      1|    1|    1|
|  2|      2|    3|    2|
|  3|      3|    6|    4|
+---+-------+-----+-----+
```

But the result is not exactly the same as pandas because pandas supports multi-level columns.
For multi-level columns, I thought we can simply add the index column into `groupBy`. For instance, as what we do here:

https://github.com/databricks/koalas/blob/0033b886598621d9f9123458d189caa6f7acdcca/databricks/koalas/groupby.py#L105-L114
@HyukjinKwon
Your idea is adding the index column for aggregation types like below?

```
      y  z
x agg 
1 max 1  5
  min 2  6
2 max 3  7
  min 4  8
```
Ah, gotya, you mean https://github.com/databricks/koalas/issues/301? Yes, I think we don't currently support this.
Yes. If the multi-index column structure was supported, we could store multiple aggregation results as a nested column.
Hi @harupy,
We recently added a basic multi-index support in columns (#590).
Does that help you submit a PR?

Resolving patch:
diff --git a/databricks/koalas/groupby.py b/databricks/koalas/groupby.py
--- a/databricks/koalas/groupby.py
+++ b/databricks/koalas/groupby.py
@@ -53,7 +53,8 @@ def aggregate(self, func_or_funcs, *args, **kwargs):
         Parameters
         ----------
         func : dict
-             a dict mapping from column name (string) to aggregate functions (string).
+             a dict mapping from column name (string) to
+             aggregate functions (string or list of strings).
 
         Returns
         -------
@@ -98,26 +99,44 @@ def aggregate(self, func_or_funcs, *args, **kwargs):
         1  1  0.589
         2  3  0.705
 
+        >>> aggregated = df.groupby('A').agg({'B': ['min', 'max']})
+        >>> aggregated  # doctest: +NORMALIZE_WHITESPACE
+             B
+           min  max
+        A
+        1    1    2
+        2    3    4
+
         """
         if not isinstance(func_or_funcs, dict) or \
-                not all(isinstance(key, str) and isinstance(value, str)
+                not all(isinstance(key, str) and
+                        (isinstance(value, str) or
+                         isinstance(value, list) and all(isinstance(v, str) for v in value))
                         for key, value in func_or_funcs.items()):
             raise ValueError("aggs must be a dict mapping from column name (string) to aggregate "
-                             "functions (string).")
+                             "functions (string or list of strings).")
 
         sdf = self._kdf._sdf
         groupkeys = self._groupkeys
         groupkey_cols = [s._scol.alias('__index_level_{}__'.format(i))
                          for i, s in enumerate(groupkeys)]
+        multi_aggs = any(isinstance(v, list) for v in func_or_funcs.values())
         reordered = []
+        data_columns = []
+        column_index = []
         for key, value in func_or_funcs.items():
-            if value == "nunique":
-                reordered.append(F.expr('count(DISTINCT `{0}`) as `{0}`'.format(key)))
-            else:
-                reordered.append(F.expr('{1}(`{0}`) as `{0}`'.format(key, value)))
+            for aggfunc in [value] if isinstance(value, str) else value:
+                data_col = "('{0}', '{1}')".format(key, aggfunc) if multi_aggs else key
+                data_columns.append(data_col)
+                column_index.append((key, aggfunc))
+                if aggfunc == "nunique":
+                    reordered.append(F.expr('count(DISTINCT `{0}`) as `{1}`'.format(key, data_col)))
+                else:
+                    reordered.append(F.expr('{1}(`{0}`) as `{2}`'.format(key, aggfunc, data_col)))
         sdf = sdf.groupby(*groupkey_cols).agg(*reordered)
         internal = _InternalFrame(sdf=sdf,
-                                  data_columns=[key for key, _ in func_or_funcs.items()],
+                                  data_columns=data_columns,
+                                  column_index=column_index if multi_aggs else None,
                                   index_map=[('__index_level_{}__'.format(i), s.name)
                                              for i, s in enumerate(groupkeys)])
         return DataFrame(internal)


Test patch:
diff --git a/databricks/koalas/tests/test_groupby.py b/databricks/koalas/tests/test_groupby.py
--- a/databricks/koalas/tests/test_groupby.py
+++ b/databricks/koalas/tests/test_groupby.py
@@ -87,6 +87,9 @@ def test_aggregate(self):
         self.assert_eq(kdf.groupby('A').agg({'B': 'min', 'C': 'sum'}),
                        pdf.groupby('A').agg({'B': 'min', 'C': 'sum'}))
 
+        self.assert_eq(kdf.groupby('A').agg({'B': ['min', 'max'], 'C': 'sum'}),
+                       pdf.groupby('A').agg({'B': ['min', 'max'], 'C': 'sum'}))
+
     def test_all_any(self):
         pdf = pd.DataFrame({'A': [1, 1, 2, 2, 3, 3, 4, 4, 5, 5],
                             'B': [True, True, True, False, False, False, None, True, None, False]})