BUG: Groupby.apply wasn't allowing for functions which return lists
State: closed

- [ ] closes #31441 
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry


Labels:
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
NOTE
----
The following refers to a commit with line 507 (in revised code) changed to:
```
piece = deepcopy(piece)
```
(so, without the `if isinstance(piece, list)` part)

> i think also need to run some perf checks to make sure this isn't super expensive.

Sure, here's the output from a run I did:
```
$ python setup.py develop  [I've removed the output]

$ cd asv_bench/

$ asv continuous -f 1.1 upstream/master HEAD -b groupby.Apply
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Building 6369f768 <issue-31441> for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt......................................................
·· Installing 6369f768 <issue-31441> into conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
· Running 10 total benchmarks (2 commits * 1 environments * 5 benchmarks)
[  0.00%] · For pandas commit d81e226b <issue-31441~3> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[  5.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[  5.00%] ··· Running (groupby.Apply.time_copy_function_multi_col--).....
[ 25.00%] · For pandas commit 6369f768 <issue-31441> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 30.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 30.00%] ··· Running (groupby.Apply.time_copy_function_multi_col--).....
[ 50.00%] · For pandas commit 6369f768 <issue-31441> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 55.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 55.00%] ··· groupby.Apply.time_copy_function_multi_col                                                                                                                                               3.64±0.01s
[ 60.00%] ··· groupby.Apply.time_copy_overhead_single_col                                                                                                                                              1.44±0.01s
[ 65.00%] ··· groupby.Apply.time_scalar_function_multi_col                                                                                                                                             57.7±0.4ms
[ 70.00%] ··· groupby.Apply.time_scalar_function_single_col                                                                                                                                            17.8±0.4ms
[ 75.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                    183±0.9ms
[ 75.00%] · For pandas commit d81e226b <issue-31441~3> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 80.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 80.00%] ··· groupby.Apply.time_copy_function_multi_col                                                                                                                                                  5.07±0s
[ 85.00%] ··· groupby.Apply.time_copy_overhead_single_col                                                                                                                                                 2.02±0s
[ 90.00%] ··· groupby.Apply.time_scalar_function_multi_col                                                                                                                                               59.1±3ms
[ 95.00%] ··· groupby.Apply.time_scalar_function_single_col                                                                                                                                            18.1±0.5ms
[100.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                      185±4ms
       before           after         ratio
     [d81e226b]       [6369f768]
     <issue-31441~3>       <issue-31441>
-         5.07±0s       3.64±0.01s     0.72  groupby.Apply.time_copy_function_multi_col
-         2.02±0s       1.44±0.01s     0.71  groupby.Apply.time_copy_overhead_single_col

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

$ asv continuous -f 1.1 upstream/master HEAD -b groupby.ApplyDictReturn
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing 6369f768 <issue-31441> into conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit d81e226b <issue-31441~3> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running (groupby.ApplyDictReturn.time_groupby_apply_dict_return--).
[ 25.00%] · For pandas commit 6369f768 <issue-31441> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running (groupby.ApplyDictReturn.time_groupby_apply_dict_return--).
[ 50.00%] · For pandas commit 6369f768 <issue-31441> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                      186±2ms
[ 75.00%] · For pandas commit d81e226b <issue-31441~3> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                      187±2ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```

EDIT
----
 seems like there's some test failures though
Performance results with current commit:

```
$ asv continuous -f 1.1 upstream/master HEAD -b groupby.Apply
· Creating environments
· Discovering benchmarks
· Running 10 total benchmarks (2 commits * 1 environments * 5 benchmarks)
[  0.00%] · For pandas commit d81e226b <issue-31441~3> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[  5.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[  5.00%] ··· Running (groupby.Apply.time_copy_function_multi_col--).....
[ 25.00%] · For pandas commit 6369f768 <issue-31441> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 30.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 30.00%] ··· Running (groupby.Apply.time_copy_function_multi_col--).....
[ 50.00%] · For pandas commit 6369f768 <issue-31441> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 55.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 55.00%] ··· groupby.Apply.time_copy_function_multi_col                                                                                                                                               3.77±0.07s
[ 60.00%] ··· groupby.Apply.time_copy_overhead_single_col                                                                                                                                              1.46±0.02s
[ 65.00%] ··· groupby.Apply.time_scalar_function_multi_col                                                                                                                                             58.1±0.7ms
[ 70.00%] ··· groupby.Apply.time_scalar_function_single_col                                                                                                                                           17.7±0.08ms
[ 75.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                    181±0.3ms
[ 75.00%] · For pandas commit d81e226b <issue-31441~3> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 80.00%] ··· Setting up groupby:72                                                                                                                                                                            ok
[ 80.00%] ··· groupby.Apply.time_copy_function_multi_col                                                                                                                                                  5.14±0s
[ 85.00%] ··· groupby.Apply.time_copy_overhead_single_col                                                                                                                                              2.09±0.04s
[ 90.00%] ··· groupby.Apply.time_scalar_function_multi_col                                                                                                                                             57.4±0.8ms
[ 95.00%] ··· groupby.Apply.time_scalar_function_single_col                                                                                                                                            17.6±0.1ms
[100.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                      181±1ms
       before           after         ratio
     [d81e226b]       [6369f768]
     <issue-31441~3>       <issue-31441>
-         5.14±0s       3.77±0.07s     0.73  groupby.Apply.time_copy_function_multi_col
-      2.09±0.04s       1.46±0.02s     0.70  groupby.Apply.time_copy_overhead_single_col

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

$ asv continuous -f 1.1 upstream/master HEAD -b groupby.ApplyDictReturn
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing 6369f768 <issue-31441> into conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit d81e226b <issue-31441~3> (round 1/2):
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running (groupby.ApplyDictReturn.time_groupby_apply_dict_return--).
[ 25.00%] · For pandas commit 6369f768 <issue-31441> (round 1/2):
[ 25.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 25.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running (groupby.ApplyDictReturn.time_groupby_apply_dict_return--).
[ 50.00%] · For pandas commit 6369f768 <issue-31441> (round 2/2):
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                    181±0.2ms
[ 75.00%] · For pandas commit d81e226b <issue-31441~3> (round 2/2):
[ 75.00%] ·· Building for conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[ 75.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numba-numexpr-numpy-odfpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· groupby.ApplyDictReturn.time_groupby_apply_dict_return                                                                                                                                    182±0.8ms

BENCHMARKS NOT SIGNIFICANTLY CHANGED.
```
thanks @MarcoGorelli