BUG: support non-nanos Timedelta objects in Python C API
State: closed

- [x] closes #54682
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
If someone can confirm that the CI docs/mypy failures are due to my changes, let me know and I'll address. Right now I can't tell how these are related, and the same two failures appear to be failing on other recent PRs, so I'm going to pause on spending time root causing them until someone can confirm otherwise.
Catching up a bit, im skeptical that it is _possible_ to make the python C API work with non-nanosecond Timedeltas.  (same issues will apply to Timestamps).
> Catching up a bit, im skeptical that it is _possible_ to make the python C API work with non-nanosecond Timedeltas. (same issues will apply to Timestamps).

Are you saying that we _can_ make the C-API work with non-nanosecond Timedeltas, or that we _can't_ make it work?

If it's the former, I'm all ears for a broader way to do this. If it's the latter, I may be misunderstanding the issue. Some of my test cases (specifically the ones in test_timedelta_as_unit_conversion) do successfully transform a non-nanosecond unit into one that is within the bounds of the C-APIs. The C-APIs are then accessed to confirm that the Timedelta was successfully transformed (and not just zero). This was to address the concern of the original bug; I also tested it with pyarrow to the same effect. Is this different from the behavior that we're concerned about?
> Are you saying that we can make the C-API work with non-nanosecond Timedeltas, or that we can't make it work?

I believe we can _not_ make the C-API work in the general case.  Downstream libraries will need to use the python API.
I think this PR should handle the "best effort" compatibility with the C-API for now.
Forgive me for asking a potentially dumb question here, but what if we only support large, low-resolution values within the bounds of pytimedeltas, while supporting small, higher-resolution values outside the bounds of pytimedeltas (i.e., nanoseconds and smaller)? Just because we _can_ create larger low-resolution values doesn't necessarily mean we _should_ support them.

I ask this mostly because I foresee many use cases for nanosecond and sub-nanosecond timedeltas (I myself have run into them "in the wild"), but I don't foresee very many use cases for large, low-resolution timedeltas greater than the order of 2 million years. Perhaps in fields like astrophysics and geology this would be important, but at that point we might as well consider whether it's worth it to support even lower resolution timedeltas which are as long as the age of the universe.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Bumping this issue awake again: do we want to continue pressing forward on a solution to #54682 with this PR? Or do we want the conclusion to be that, since downstream libraries need to use the python API, we won't support the C-APIs for timedeltas?

Would appreciate any thoughts from @jbrockmendel and @mroeschke . If we don't want to move forward with a fix in Pandas, we should probably close #54682 and move further discussion over to the PyArrow side.
AFAICT what is being asked is impossible and this should be closed.
> AFAICT what is being asked is impossible and this should be closed.

That aligns with my interpretation + learnings from the code as well. I think we can fix this specific issue with PyArrow, but the general problem would still remain.

@mroeschke unless you have significant objections, I'll close this PR and then the issue on the Pandas side can be closed. The fix can then be continued on the PyArrow side.
I don't have objection to deem this out of scope for pandas. Even though `Timstamp/Timedelta` subclass the associated Python objects, we have no compatibility testing with C-APIs. IMO it would still be nice to have compatibility, but I will defer to @jbrockmendel in terms of feasibility.