Repository: pandas-dev/pandas
Pull request: Update compute_dict_like to get all columns
- [x] closes #55041  (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Problem statement:
BUG: groupby.agg fails when input has duplicate columns and dict input
```
df = DataFrame(
    [[1, 2, 3, 4], [1, 3, 4, 5], [2, 4, 5, 6]],
    columns=["a", "b", "c", "c"],
)
gb = df.groupby("a")
gb.agg({"c": "sum"})
```

raises `AttributeError: 'DataFrame' object has no attribute 'name'. Did you mean: 'rename'?`

I would expect it to aggregate both of the columns named "c".

Hints from issue discussion:

Commit message: Remove newline from rst

Patch to review:
diff --git a/doc/source/whatsnew/v3.0.0.rst b/doc/source/whatsnew/v3.0.0.rst
index c77348b365370..7e49b532c66b5 100644
--- a/doc/source/whatsnew/v3.0.0.rst
+++ b/doc/source/whatsnew/v3.0.0.rst
@@ -440,6 +440,7 @@ Groupby/resample/rolling
 - Bug in :meth:`.DataFrameGroupBy.quantile` when ``interpolation="nearest"`` is inconsistent with :meth:`DataFrame.quantile` (:issue:`47942`)
 - Bug in :meth:`.Resampler.interpolate` on a :class:`DataFrame` with non-uniform sampling and/or indices not aligning with the resulting resampled index would result in wrong interpolation (:issue:`21351`)
 - Bug in :meth:`DataFrame.ewm` and :meth:`Series.ewm` when passed ``times`` and aggregation functions other than mean (:issue:`51695`)
+- Bug in :meth:`DataFrameGroupBy.agg` that raises ``AttributeError`` when there is dictionary input and duplicated columns, instead of returning a DataFrame with the aggregation of all duplicate columns. (:issue:`55041`)
 - Bug in :meth:`DataFrameGroupBy.apply` that was returning a completely empty DataFrame when all return values of ``func`` were ``None`` instead of returning an empty DataFrame with the original columns and dtypes. (:issue:`57775`)
 
 
diff --git a/pandas/core/apply.py b/pandas/core/apply.py
index 832beeddcef3c..3f7fe1d4c0156 100644
--- a/pandas/core/apply.py
+++ b/pandas/core/apply.py
@@ -471,8 +471,22 @@ def compute_dict_like(
 
                 keys += [key] * len(key_data)
                 results += key_data
-        else:
+        elif is_groupby:
             # key used for column selection and output
+
+            df = obj.obj
+            results, keys = [], []
+            for key, how in func.items():
+                for index in range(df.shape[1]):
+                    col = df.iloc[:, index]
+                    if col.name != key:
+                        continue
+
+                    series = obj._gotitem(key, ndim=1, subset=col)
+                    result = getattr(series, op_name)(how, **kwargs)
+                    results.append(result)
+                    keys.append(key)
+        else:
             results = [
                 getattr(obj._gotitem(key, ndim=1), op_name)(how, **kwargs)
                 for key, how in func.items()
@@ -496,11 +510,14 @@ def wrap_results_dict_like(
         is_ndframe = [isinstance(r, ABCNDFrame) for r in result_data]
 
         if all(is_ndframe):
-            results = dict(zip(result_index, result_data))
+            results = [result for result in result_data if not result.empty]
             keys_to_use: Iterable[Hashable]
-            keys_to_use = [k for k in result_index if not results[k].empty]
+            keys_to_use = [k for k, v in zip(result_index, result_data) if not v.empty]
             # Have to check, if at least one DataFrame is not empty.
-            keys_to_use = keys_to_use if keys_to_use != [] else result_index
+            if keys_to_use == []:
+                keys_to_use = result_index
+                results = result_data
+
             if selected_obj.ndim == 2:
                 # keys are columns, so we can preserve names
                 ktu = Index(keys_to_use)
@@ -509,7 +526,7 @@ def wrap_results_dict_like(
 
             axis: AxisInt = 0 if isinstance(obj, ABCSeries) else 1
             result = concat(
-                {k: results[k] for k in keys_to_use},
+                results,
                 axis=axis,
                 keys=keys_to_use,
             )
diff --git a/pandas/tests/groupby/aggregate/test_aggregate.py b/pandas/tests/groupby/aggregate/test_aggregate.py
index 2b9df1b7079da..5fc7e98c91c11 100644
--- a/pandas/tests/groupby/aggregate/test_aggregate.py
+++ b/pandas/tests/groupby/aggregate/test_aggregate.py
@@ -1663,3 +1663,89 @@ def func(x):
     msg = "length must not be 0"
     with pytest.raises(ValueError, match=msg):
         df.groupby("A", observed=False).agg(func)
+
+
+def test_groupby_aggregation_duplicate_columns_single_dict_value():
+    # GH#55041
+    df = DataFrame(
+        [[1, 2, 3, 4], [1, 3, 4, 5], [2, 4, 5, 6]],
+        columns=["a", "b", "c", "c"],
+    )
+    gb = df.groupby("a")
+    result = gb.agg({"c": "sum"})
+
+    expected = DataFrame(
+        [[7, 9], [5, 6]], columns=["c", "c"], index=Index([1, 2], name="a")
+    )
+    tm.assert_frame_equal(result, expected)
+
+
+def test_groupby_aggregation_duplicate_columns_multiple_dict_values():
+    # GH#55041
+    df = DataFrame(
+        [[1, 2, 3, 4], [1, 3, 4, 5], [2, 4, 5, 6]],
+        columns=["a", "b", "c", "c"],
+    )
+    gb = df.groupby("a")
+    result = gb.agg({"c": ["sum", "min", "max", "min"]})
+
+    expected = DataFrame(
+        [[7, 3, 4, 3, 9, 4, 5, 4], [5, 5, 5, 5, 6, 6, 6, 6]],
+        columns=MultiIndex(
+            levels=[["c"], ["sum", "min", "max"]],
+            codes=[[0, 0, 0, 0, 0, 0, 0, 0], [0, 1, 2, 1, 0, 1, 2, 1]],
+        ),
+        index=Index([1, 2], name="a"),
+    )
+    tm.assert_frame_equal(result, expected)
+
+
+def test_groupby_aggregation_duplicate_columns_some_empty_result():
+    # GH#55041
+    df = DataFrame(
+        [
+            [1, 9843, 43, 54, 7867],
+            [2, 940, 9, -34, 44],
+            [1, -34, -546, -549358, 0],
+            [2, 244, -33, -100, 44],
+        ],
+        columns=["a", "b", "b", "c", "c"],
+    )
+    gb = df.groupby("a")
+    result = gb.agg({"b": [], "c": ["var"]})
+
+    expected = DataFrame(
+        [[1.509268e11, 30944844.5], [2.178000e03, 0.0]],
+        columns=MultiIndex(levels=[["c"], ["var"]], codes=[[0, 0], [0, 0]]),
+        index=Index([1, 2], name="a"),
+    )
+    tm.assert_frame_equal(result, expected)
+
+
+def test_groupby_aggregation_multi_index_duplicate_columns():
+    # GH#55041
+    df = DataFrame(
+        [
+            [1, -9843, 43, 54, 7867],
+            [2, 940, 9, -34, 44],
+            [1, -34, 546, -549358, 0],
+            [2, 244, -33, -100, 44],
+        ],
+        columns=MultiIndex(
+            levels=[["level1.1", "level1.2"], ["level2.1", "level2.2"]],
+            codes=[[0, 0, 0, 1, 1], [0, 1, 1, 0, 1]],
+        ),
+        index=MultiIndex(
+            levels=[["level1.1", "level1.2"], ["level2.1", "level2.2"]],
+            codes=[[0, 0, 0, 1], [0, 1, 1, 0]],
+        ),
+    )
+    gb = df.groupby(level=0)
+    result = gb.agg({("level1.1", "level2.2"): "min"})
+
+    expected = DataFrame(
+        [[-9843, 9], [244, -33]],
+        columns=MultiIndex(levels=[["level1.1"], ["level2.2"]], codes=[[0, 0], [0, 0]]),
+        index=Index(["level1.1", "level1.2"]),
+    )
+    tm.assert_frame_equal(result, expected)


Code review comments:
diff_hunk: @@ -471,8 +471,22 @@ def compute_dict_like(
 
                 keys += [key] * len(key_data)
                 results += key_data
-        else:
+        elif is_groupby:
             # key used for column selection and output
+
+            df = obj.obj
+            results, keys = [], []
+            for key, how in func.items():
+                for index in range(df.shape[1]):
+                    col = df.iloc[:, index]
+                    if col.name != key:
+                        continue
+
+                    series = obj._gotitem(key, ndim=1, subset=col)
+                    result = getattr(series, op_name)(how, **kwargs)
path: pandas/core/apply.py
text: @user1:
Can you see if the following works instead:

```
subset = selected_obj[[key]]
subobj = obj._gotitem(key, ndim=2, subset=subset)
result = getattr(subobj, op_name)(how, **kwargs)
```

@author:
Using `getattr` on DataFrame throws exception in this case
```
df = pd.DataFrame(
        [[1, 2, 3], [1, 3, 4], [2, 4, 5]],
        columns=["a", "b", "c"],
    )

    gb = df.groupby("a")
    result = gb.agg({"c": ["sum", "min", "max", "min"]})
    print(result)
```
This is due to `reconstruct_func` in `DataFrameGroupby` 
```
if not relabeling:
        if isinstance(func, list) and len(func) > len(set(func)):
            # GH 28426 will raise error if duplicated function names are used and
            # there is no reassigned name
            raise SpecificationError(
                "Function names must be unique if there is no new column names "
                "assigned"
            )
  ```

diff_hunk: @@ -471,8 +471,22 @@ def compute_dict_like(
 
                 keys += [key] * len(key_data)
                 results += key_data
-        else:
+        elif is_groupby:
             # key used for column selection and output
+
+            df = obj.obj
path: pandas/core/apply.py
text: @user1:
Use `selected_obj`, `obj.obj` contains all the columns of the input.

```
df = pd.DataFrame({'a': [1], 'b': 2, 'c': 3})
gb = df.groupby('a')[['b']]
print(gb.obj.columns)
# Index(['a', 'b', 'c'], dtype='object')  <-- wrong
print(gb._selected_obj.columns)
# Index(['b'], dtype='object')  <-- right

```

@author:
resolved, the original code (now in the else branch) uses `(obj._gotitem(key, ndim=1)` which internally uses `self.obj`, equivalent to `obj.obj`. is that ok since any column that does not exist is already check for beforehand?