TST(string dtype): Resolve xfail when grouping by nan column
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

At first glance the behavior perhaps seems odd, but I think have to go with it while we infer dtype based on the non-NA values and accept `None` as an NA value for the inferred dtype. Namely, in

     df = DataFrame({None: [1, 1, 2, 2], "b": [1, 1, 2, 3], "c": [4, 5, 6, 7]})

the columns are `[None, "b", "c"]` and so we infer this as strings with `None` being the corresponding NA-value, in this case `np.nan`. Thus we need to groupby `np.nan`.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Strings
description: String extension data type and string data

Comments:
> At first glance the behavior perhaps seems odd, but I think have to go with it while we infer dtype based on the non-NA values and accept `None` as an NA value for the inferred dtype.

I think what is being tested here took advantage of the old implementation, giving `None` a special case as an object sentinel. In the log run, I don't see us supporting that natively for strings, so the coercion to a missing value indicator makes sense (and is consistent with constructors)
Thanks @rhshadrach 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 55a6d0a613897040fec1ae11adc15f5f04728032
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60712: TST(string dtype): Resolve xfail when grouping by nan column'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60712-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60712 on branch 2.3.x (TST(string dtype): Resolve xfail when grouping by nan column)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Backport PR: #60719 
I do think that on the short term, we should maybe keep supporting `df.groupby(by=[None]).sum()` ? (and potentially deprecate it later?)

There are similar tests with pivot or stack/unstack where we also refer to a column/index label with `None`. For indexing itself, we kept label-lookup working (`df[None]` gives the column with label `NaN`).
One of the problems with doing that I think would be breaking usability between the `str` and `string` types; the latter has never supported using `None` as a special case
> I do think that on the short term, we should maybe keep supporting `df.groupby(by=[None]).sum()` ?

For all dtypes? Or just `str`? What happens with object dtype - is `None` also treated as an NA value?

I'm a bit resistant to changing groupby behavior here. This seems to me to be quite an edge case, and is long standing behavior for other dtypes. E.g.

```python
df = pd.DataFrame([[1, 1, 2], [3, 4, 5]], columns=pd.Index([1, pd.NA, 2], dtype="Int64"))
gb = df.groupby([None])
```

raises.