Repository: pandas-dev/pandas

Issue:
df.groupby(...).transform(func) breaks when func renames input df
Calling `df.groupby(...).transform(func)` breaks when `func` is capable of taking in a `pd.DataFrame` but renames its columns.

#### Code sample

```python
def demean_rename(x):
    result = x - x.mean()

    if isinstance(x, pd.Series):
        return result

    result = result.rename(columns=
        {c: '{}_demeaned'.format(c) for c in result.columns})

    return result

df = pd.DataFrame({'group': list('ababa'),
                   'value': [1, 1, 1, 2, 2]})
expected = pd.DataFrame({'value': [-1./3., -0.5, -1./3., 0.5, 2./3.]})

result = df.groupby('group').transform(demean_rename)
tm.assert_frame_equal(result, expected)
# Instead, this prints:
# E   DataFrame.iloc[:, 0] values are different (40.0 %)
# E   [left]:  [-0.33333333333333326, nan, -0.33333333333333326, nan, 0.6666666666666667]
```
#### Problem description

The current behavior gives wrong results (with nan) for everything except the first group. This happens even when `func` can be called successfully with each column and not return any nans.

This problem is present in master. I already have the fix: there is a bug in how results from the slow-path and the fast-path are joined in `df.groupby(...).transform(func)`. I will be putting up a PR very soon with the fix.


Discussion:

Resolving patch:
diff --git a/pandas/core/groupby/generic.py b/pandas/core/groupby/generic.py
--- a/pandas/core/groupby/generic.py
+++ b/pandas/core/groupby/generic.py
@@ -586,14 +586,17 @@ def _choose_path(self, fast_path, slow_path, group):
         try:
             res_fast = fast_path(group)
 
-            # compare that we get the same results
+            # verify fast path does not change columns (and names), otherwise
+            # its results cannot be joined with those of the slow path
+            if res_fast.columns != group.columns:
+                return path, res
+            # verify numerical equality with the slow path
             if res.shape == res_fast.shape:
                 res_r = res.values.ravel()
                 res_fast_r = res_fast.values.ravel()
                 mask = notna(res_r)
-            if (res_r[mask] == res_fast_r[mask]).all():
-                path = fast_path
-
+                if (res_r[mask] == res_fast_r[mask]).all():
+                    path = fast_path
         except Exception:
             pass
         return path, res


Test patch:
diff --git a/pandas/tests/groupby/test_transform.py b/pandas/tests/groupby/test_transform.py
--- a/pandas/tests/groupby/test_transform.py
+++ b/pandas/tests/groupby/test_transform.py
@@ -808,3 +808,26 @@ def test_any_all_np_func(func):
 
     res = df.groupby('key')['val'].transform(func)
     tm.assert_series_equal(res, exp)
+
+
+def test_groupby_transform_rename():
+    # https://github.com/pandas-dev/pandas/issues/23461
+    def demean_rename(x):
+        result = x - x.mean()
+
+        if isinstance(x, pd.Series):
+            return result
+
+        result = result.rename(
+            columns={c: '{}_demeaned'.format(c) for c in result.columns})
+
+        return result
+
+    df = pd.DataFrame({'group': list('ababa'),
+                       'value': [1, 1, 1, 2, 2]})
+    expected = pd.DataFrame({'value': [-1. / 3, -0.5, -1. / 3, 0.5, 2. / 3]})
+
+    result = df.groupby('group').transform(demean_rename)
+    tm.assert_frame_equal(result, expected)
+    result_single = df.groupby('group').value.transform(demean_rename)
+    tm.assert_series_equal(result_single, expected['value'])