REGR: Groupby first/last/nth treats None as an observation
State: closed

- [x] closes #38286
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

This is a regression from 1.0.x to 1.1.x, introduced by #33462. Assuming it doesn't get into the 1.2 rc, not sure if it should go into 1.2 during the rc phase or wait until 1.3.

Had to decide on some edge-case behaviors in odd situations with missing values, see https://github.com/pandas-dev/pandas/issues/38286#issuecomment-739521117

Labels:
name: Groupby
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
@jreback This was originally changed in #33462 so that 

    pd.DataFrame({"id": ["a"], "value": [None]}, dtype=object).groupby("id").first()

returns `None` rather than `np.nan`. That behavior is retained here.
so you think that the original issues are wrong? this is deleting that test case and reverting 
> This is a regression from 1.0.x to 1.1.x

if this is the case, should go in 1.1.5.
@jreback I believe this is reverting the fix in #33462 and solving the issue there in the proper manner, along with the regression it caused. The original test case still exists, is the first parameter to `test_first_last_with_none`
I can restore the original test and add the new one for the regression if that's preferable.
yes if this also doesn't change the original issue then leave the tests

ok for 1.1.5
@jreback test restored, added to whatsnew 1.5
@jreback asv run with `asv continuous -f 1.1 upstream/master HEAD -b ^groupby` resulted in `BENCHMARKS NOT SIGNIFICANTLY CHANGED`.

Linux py39 failure is unrelated:

    FAILED pandas/tests/io/pytables/test_store.py::TestHDFStore::test_append_with_data_columns
    = 1 failed, 143282 passed, 5482 skipped, 1091 xfailed, 4 warnings in 1132.31s (0:18:52) =

thanks for the quick patch @rhshadrach 
@meeseeksdev backport 1.1.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
$ git checkout 1.1.x
$ git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
$ git cherry-pick -m1 55e3bffe40e589b83b4dbb68f59dde3cfcc9b836
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
$ git commit -am 'Backport PR #38330: REGR: Groupby first/last/nth treats None as an observation'
```

4. Push to a named branch :

```
git push YOURFORK 1.1.x:auto-backport-of-pr-38330-on-1.1.x
```

5. Create a PR against branch 1.1.x, I would have named this PR:

> "Backport PR #38330 on branch 1.1.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
We just upgraded to 1.1.5 this morning, and it works great! Thank you for the blazing fast discussion, decision, fix, merge, and backport!