Repository: pandas-dev/pandas
Pull request: REGR: groupby with as_index=False on an empty frame
- [x] closes #41998
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them

No whatsnew because this was introduced in 1.3


Problem statement:
REGR: 1.3 behavior change with groupby and asindex=False
#### Code Sample, a copy-pastable example

```python
pd.DataFrame(columns=["date", "crew_member_id"])
    .groupby(["date", "crew_member_id"], as_index=False)
    .sum()
```

#### Problem description

With 1.3rc1 this fails with `ValueError: Length of values (0) does not match length of index (1)` while 1.2 returns and empty frame.

#### Expected Output

If this is the expected behavior fine. Just wanted to ensure that it is not an accidental regression.

Hints from issue discussion:
Thanks @aberres for the report

> With 1.3rc1 this fails with `ValueError: Length of values (0) does not match length of index (1)` while 1.2 returns and empty frame.

first bad commit: [48a5853594f95b651e2490e456e5792a1cc1d56a] REF: preserve Index dtype in BlockManager._combine (#41354)

before this commit the result was

```
2021-06-14T12:25:54.6429723Z 1.3.0.dev0+1555.g3cce96f515
2021-06-14T12:25:54.6430581Z Empty DataFrame
2021-06-14T12:25:54.6430967Z Columns: [date, crew_member_id]
2021-06-14T12:25:54.6431332Z Index: []
2021-06-14T12:25:54.7091204Z Bisecting: 1 revision left to test after this (roughly 1 step)
```
which is consistent with the result from a non-empty DataFrame, unlike the result on 1.2.4 which was an empty DataFrame with no columns

cc @jbrockmendel 
Yep my ci picked that up as well. Should we label this as a regression @simonjayhawkins ?
sure. I left it in the needs triage state for another set of eyes as I only bisected the change of behavior. If we use a populated DataFrame in the code sample instead of an empty one. the result does not make much sense to me.
This one fails too:

```
df = pd.DataFrame(columns=["a", "b"]).groupby(["a"], as_index=False).sum()
```

The initial code snippet seems a bit artificial to me, mostly because I don't know what to actually expect. But this one returned
```
Empty DataFrame
Columns: [a, b]
Index: []
```

on 1.2.4 which looks sensible to me.
Labelling as a regression
ive got it on my todo list (i.e. an open browser tab) to look at this, but nothing obvious off the top of my head
The method `_wrap_agged_manager` uses `mgr.shape[1]` for the length of the index, it seems to me it should be using

    mgr.shape[1] if not mgr.shape[0] == 0 else 0

This fixes the issue in the OP and gives the output

```
Empty DataFrame
Columns: [date, crew_member_id]
Index: []
```

which seems to me in-line with the result of

```
(
    pd.DataFrame([[1, 2], [3, 4]], columns=["date", "crew_member_id"])
    .groupby(["date", "crew_member_id"], as_index=False)
    .sum()
)
```

which gives 

```
   date  crew_member_id
0     1               2
1     3               4
```

However, for the issue in https://github.com/pandas-dev/pandas/issues/41998#issuecomment-861872176, it gives

```
Empty DataFrame
Columns: [a]
Index: []
```

which to me looks wrong - as @phofl said we should also be getting the column 'b'. I don't this the column information in the `mgr` passed to `_wrap_agged_manager`, so maybe the fix needs to happen more upstream.
Ah - this path is dropping column 'b' because it is object dtype. Doing

    print(pd.DataFrame(columns=["a", "b"], dtype=int).groupby(["a"], as_index=False).sum())

gives

```
Empty DataFrame
Columns: [a, b]
Index: []
```

which is better. Running tests now, PR to follow.

Commit message: fixup

Patch to review:
diff --git a/pandas/core/groupby/generic.py b/pandas/core/groupby/generic.py
index 5cb0eac5d9074..2b9fb82744536 100644
--- a/pandas/core/groupby/generic.py
+++ b/pandas/core/groupby/generic.py
@@ -1676,7 +1676,8 @@ def _wrap_transformed_output(
 
     def _wrap_agged_manager(self, mgr: Manager2D) -> DataFrame:
         if not self.as_index:
-            index = Index(range(mgr.shape[1]))
+            rows = mgr.shape[1] if mgr.shape[0] > 0 else 0
+            index = Index(range(rows))
             mgr.set_axis(1, index)
             result = self.obj._constructor(mgr)
 
diff --git a/pandas/tests/groupby/conftest.py b/pandas/tests/groupby/conftest.py
index b69a467f91659..13163aea30570 100644
--- a/pandas/tests/groupby/conftest.py
+++ b/pandas/tests/groupby/conftest.py
@@ -12,6 +12,11 @@
 )
 
 
+@pytest.fixture(params=[True, False])
+def as_index(request):
+    return request.param
+
+
 @pytest.fixture
 def mframe():
     index = MultiIndex(
diff --git a/pandas/tests/groupby/test_groupby.py b/pandas/tests/groupby/test_groupby.py
index 89944e2a745e4..0181481b29c44 100644
--- a/pandas/tests/groupby/test_groupby.py
+++ b/pandas/tests/groupby/test_groupby.py
@@ -15,6 +15,7 @@
     Grouper,
     Index,
     MultiIndex,
+    RangeIndex,
     Series,
     Timestamp,
     date_range,
@@ -2360,19 +2361,21 @@ def test_groupby_all_nan_groups_drop():
     tm.assert_series_equal(result, expected)
 
 
-def test_groupby_empty_multi_column():
-    # GH 15106
+@pytest.mark.parametrize("numeric_only", [True, False])
+def test_groupby_empty_multi_column(as_index, numeric_only):
+    # GH 15106 & GH 41998
     df = DataFrame(data=[], columns=["A", "B", "C"])
-    gb = df.groupby(["A", "B"])
-    result = gb.sum(numeric_only=False)
-    expected = DataFrame(
-        [], columns=["C"], index=MultiIndex([[], []], [[], []], names=["A", "B"])
-    )
+    gb = df.groupby(["A", "B"], as_index=as_index)
+    result = gb.sum(numeric_only=numeric_only)
+    if as_index:
+        index = MultiIndex([[], []], [[], []], names=["A", "B"])
+        columns = ["C"] if not numeric_only else []
+    else:
+        index = RangeIndex(0)
+        columns = ["A", "B", "C"] if not numeric_only else ["A", "B"]
+    expected = DataFrame([], columns=columns, index=index)
     tm.assert_frame_equal(result, expected)
 
-    result = gb.sum(numeric_only=True)
-    tm.assert_frame_equal(result, expected[[]])
-
 
 def test_groupby_filtered_df_std():
     # GH 16174


Code review comments:
diff_hunk: @@ -1676,7 +1676,8 @@ def _wrap_transformed_output(
 
     def _wrap_agged_manager(self, mgr: Manager2D) -> DataFrame:
         if not self.as_index:
-            index = Index(range(mgr.shape[1]))
+            rows = mgr.shape[1] if mgr.shape[0] > 0 else 0
path: pandas/core/groupby/generic.py
text: @user1:
comment here about why, pointing back to the issue/pr?

@author:
Thanks - added.

@user2:
we should strive to move this code inside internal (separate PR for sure)