BUG: groupby.sum with timedelta64 and NaT
State: closed

- [x] closes #42659
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry


Labels:
name: Bug
name: Groupby
name: Timedelta
description: Timedelta data type

Comments:
@meeseeksdev backport 1.3.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 3352625fbc8bab296b694514355ec04e7b171d8a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #44658: BUG: groupby.sum with timedelta64 and NaT'
```

4. Push to a named branch:

```
git push YOURFORK 1.3.x:auto-backport-of-pr-44658-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #44658 on branch 1.3.x (BUG: groupby.sum with timedelta64 and NaT)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -m1 3352625fbc8bab296b694514355ec04e7b171d8a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #44658: BUG: groupby.sum with timedelta64 and NaT'
```

4. Push to a named branch:

```
git push YOURFORK 1.3.x:auto-backport-of-pr-44658-on-1.3.x
```

5. Create a PR against branch 1.3.x, I would have named this PR:

> "Backport PR #44658 on branch 1.3.x (BUG: groupby.sum with timedelta64 and NaT)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
cc @jbrockmendel as suspected, needs a manual backport.
having a hard time with the cherrypick.  flake8 complains about the cython syntax at the `git cherry-pick --continue` step.  this happens frequently and ive gotten used to ignoring it with --no-verify.  but with `git cherrypick --continue` there is no `--no-verify` option.
cc @simonjayhawkins any idea how to handle the cherrypick problem?
> cc @simonjayhawkins any idea how to handle the cherrypick problem?

didn't have any.

PS. Thanks for the PR.
At first glance this appears to cause a slowdown of up to 2x for several benchmarks, but including the cython 0.29.21 results it appears the reported regressions on the 1.3.x tracker could be false positives?

https://simonjayhawkins.github.io/fantastic-dollop/#series_methods.Mode.time_mode?branch=1.3.x&p-N=100000&p-dtype='int'&commits=d3a92ff9
> At first glance this appears to cause a slowdown of up to 2x for several benchmarks

Looking at the diff, the only non-invasive optimization I see is doing `<float64_t>NPY_NAT` just once at the module level.  Hard to imagine that making a big dent.

> but including the cython 0.29.21 results it appears the reported regressions on the 1.3.x tracker could be false positives?

yah those look pretty darn noisy.  is "wait a couple days for more data" a viable option?

this is not worth holding up 1.3.x