TST (string dtype): fix groupby xfails with using_infer_string + update error message
State: closed

This fixes 590 xfails.

Two potential follow-ons: 1) should ArrowStringArrayNumpySemantics support sum (done in the meantime), 2) make the exception types and messages match between pyarrow and non-pyarrow cases (see update at the top)

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Groupby
name: Strings
description: String extension data type and string data

Comments:
> should ArrowStringArrayNumpySemantics support sum

Personally I am fine with not supporting that. I listed this as one of the breaking changes in https://github.com/pandas-dev/pandas/issues/59328, but we can definitely discuss whether we want to keep support for it or not (let's use that issue for it)
Updated to make the exceptions match for object/pyarrow variants.  Still needs some work, but now is a good time to bikeshed exception messages.
well if no one else wants to bikeshed, i'll start: using `f"{self.dtype} dtype does not support prod operations"` renders as "str dtype [...]" which seems weird.  maybe add ticks around `'str'`?
> well if no one else wants to bikeshed, i'll start: using `f"{self.dtype} dtype does not support prod operations"` renders as "str dtype [...]" which seems weird. maybe add ticks around `'str'`?

Yeah, to avoid that something reads a bit strange depending on the exact dtype and operation that is filled in, I would maybe do both with quotes, something like `f"dtype '{dtype}' does not support operation '{op}'"`
Updated to fix most of the remaining affected tests.  This PR changes the behavior of groupby.sum with string dtype to raise rather than cast to object and attempt to sum, which causes 11 new test failures that this PR does not yet address.  im on the fence as to whether we should implement StringArray.sum and allow groupby.sum to go unchanged.
Thanks for the updates! Added a few more comments
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Still active
I merged the sum PR, so this could be updated now, I think
Thanks @jbrockmendel!
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 e5dd89d4d74d8e2a06256023717880788f2b10ed
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #59430: TST (string dtype): fix groupby xfails with using_infer_string + update error message'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-59430-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #59430 on branch 2.3.x (TST (string dtype): fix groupby xfails with using_infer_string + update error message)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60246