Repository: dask/dask

Issue:
`cumsum` results differ from pandas for series groupbys
**What happened**:
I noticed that taking the cumulative sum of a pandas series groupby and an equivalent dask series groupby returns numerically different results.


**Minimal Complete Verifiable Example**:

```python
import pandas as pd
import numpy as np
import dask.dataframe as dd

np.random.seed(0)
size=10
npartitions=2


pdf = pd.DataFrame(
    {
        "xx": np.random.randint(0, 5, size=size),
        "x": np.random.normal(size=size),
        "y": np.random.normal(size=size),
    }
)

ddf = dd.from_pandas(pdf, npartitions=npartitions)

pdf_grouped = pdf.groupby('xx').xx
ddf_grouped = ddf.groupby('xx').xx

print(pdf_grouped.cumsum())
print(ddf_grouped.cumsum().compute())

```
```
0     4
1     0
2     3
3     6
4     9
5     1
6    12
7     2
8     8
9     0
Name: xx, dtype: int64
0    4
1    0
2    3
3    6
4    9
5    1
6    6
7    2
8    8
9    0
Name: xx, dtype: int64
```


**Anything else we need to know?**:
I get the right results when `npartitions=1.` 


**Environment**:

- Dask version: 2022.7.0+16.gedfa91e22
- Python version: 3.9.13 
- Operating System: Ubuntu 18.04.6 LTS
- Install method: conda


Discussion:
Thanks for reporting!

This only seems to affect the column we're grouping on (e.g., `ddf.groupby('xx').y.cumsum()` works fine). I'll keep looking into it.
@jrbourbeau and I looked into this. The issue seems to be in:

https://github.com/dask/dask/blob/8b95f983c232c1bd628e9cba0695d3ef229d290b/dask/dataframe/groupby.py#L1274-L1281

Consider the following example (same example as OP, only the values are hardcoded for understanding):

```python
import dask.dataframe as dd
import pandas as pd

pdf = pd.DataFrame(
    {
        "x": [4, 0, 3, 3, 3, 1, 3, 2, 4, 0],
        "a": [4, 0, 3, 3, 3, 1, 3, 2, 4, 0],
    }
)

ddf = dd.from_pandas(pdf, npartitions=2)

# CASE 1 -- cumsum on 'a' -- produces correct results
pdf_grouped = pdf.groupby("x").a.cumsum()
ddf_grouped = ddf.groupby("x").a.cumsum().compute()

# CASE 2 -- cumsum on 'x' -- produces wrong results
pdf.groupby("x").x.cumsum()
ddf.groupby("x").x.cumsum().compute()
```

The `cumpart_ext` calculated is different in both cases:

CASE 1:
```python
cumpart_ext.compute()
   a  x
0  4  4
1  0  0
2  3  3
3  6  3
4  9  3
5  1  1
6  3  3
7  2  2
8  4  4
9  0  0
```

CASE 2:
```python
cumpart_ext.compute()
   x
0  4
1  0
2  3
3  3
4  3
5  1
6  3
7  2
8  4
9  0
```

In case 2, while creating `cumpart_ext` (look at the permalink) the column 'x' is being overridden. This is because we're creating (`assign`ing) a new column with the same name as `by` (column we groupby). Does that make sense?

The solution here could be to rename the `by` column internally (like, add a suffix to it to make sure it's unique), throughout the cumulative logic.

Resolving patch:
diff --git a/dask/dataframe/groupby.py b/dask/dataframe/groupby.py
--- a/dask/dataframe/groupby.py
+++ b/dask/dataframe/groupby.py
@@ -1,6 +1,7 @@
 import collections
 import itertools as it
 import operator
+import uuid
 import warnings
 from numbers import Integral
 
@@ -1284,7 +1285,19 @@ def _cum_agg(self, token, chunk, aggregate, initial):
         """Wrapper for cumulative groupby operation"""
         meta = chunk(self._meta)
         columns = meta.name if is_series_like(meta) else meta.columns
-        by = self.by if isinstance(self.by, list) else [self.by]
+        by_cols = self.by if isinstance(self.by, list) else [self.by]
+
+        # rename "by" columns internally
+        # to fix cumulative operations on the same "by" columns
+        # ref: https://github.com/dask/dask/issues/9313
+        if columns is not None and set(columns).intersection(set(by_cols)):
+            by = []
+            for col in by_cols:
+                suffix = str(uuid.uuid4())
+                self.obj = self.obj.assign(**{col + suffix: self.obj[col]})
+                by.append(col + suffix)
+        else:
+            by = by_cols
 
         name = self._token_prefix + token
         name_part = name + "-map"


Test patch:
diff --git a/dask/dataframe/tests/test_groupby.py b/dask/dataframe/tests/test_groupby.py
--- a/dask/dataframe/tests/test_groupby.py
+++ b/dask/dataframe/tests/test_groupby.py
@@ -10,7 +10,7 @@
 import dask
 import dask.dataframe as dd
 from dask.dataframe import _compat
-from dask.dataframe._compat import PANDAS_GT_110, PANDAS_GT_150, tm
+from dask.dataframe._compat import PANDAS_GT_110, PANDAS_GT_130, PANDAS_GT_150, tm
 from dask.dataframe.backends import grouper_dispatch
 from dask.dataframe.utils import assert_dask_graph, assert_eq, assert_max_deps
 from dask.utils import M
@@ -1532,10 +1532,19 @@ def test_groupby_numeric_column():
     assert_eq(ddf.groupby(ddf.A)[0].sum(), df.groupby(df.A)[0].sum())
 
 
-@pytest.mark.parametrize("sel", ["c", "d", ["c", "d"]])
+@pytest.mark.parametrize("sel", ["a", "c", "d", ["a", "b"], ["c", "d"]])
 @pytest.mark.parametrize("key", ["a", ["a", "b"]])
 @pytest.mark.parametrize("func", ["cumsum", "cumprod", "cumcount"])
 def test_cumulative(func, key, sel):
+    if (
+        not PANDAS_GT_130
+        and not func == "cumcount"
+        and sel == ["a", "b"]
+        and key == ["a", "b"]
+    ):
+        pytest.xfail(
+            reason="cumsum and cumprod will raise DataError: No numeric types to aggregate"
+        )
     df = pd.DataFrame(
         {
             "a": [1, 2, 6, 4, 4, 6, 4, 3, 7] * 6,