BrandonFors/pandas-github-issues huggingface dataset:
ENH: pyarrow dont raise on division by zero
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
Could you add a whatsnew note in the `Other API changes` section?
thx @jbrockmendel
DEPR: deprecate allowing slice in DataFrame.take
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
thx @jbrockmendel
BUG: Timedelta comparisons with very large pytimedeltas overflowing
State: closed

- [x] closes #49021 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
updated+green
Thanks @jbrockmendel
STYLE: replace unwanted-patterns-strings-to-concatenate with ruff #5… #51504Harsimran
State: closed


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Closing for now (as I'm fairly certain you didn't follow the steps I outlined in the issue), so others can work on it

Feel free to open a new PR if you follow the steps
sorry i wasn't able to do the steps.
i have run them locally but its showing error can you help me in solving this
![git test](https://user-images.githubusercontent.com/73106537/220986397-a92cda20-925d-436c-8d7b-efeb9ef66d2e.JPG)


check https://beta.ruff.rs/docs/rules/explicit-string-concatenation/
DOC Fix EX01 issues in docstrings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards #37875

Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon 
Thank you :)
DOC: fix EX02 errors in docstrings
State: closed

Related to the issue #51236

This PR enables functions:
`pandas.io.formats.style.Styler.concat`
`pandas.io.formats.style.Styler.export`
`pandas.io.formats.style.Styler.set_td_classes`
`pandas.io.formats.style.Styler.use`

Labels:
name: Docs

Comments:
> Actually, let's just take this - we'll fix EX03 errors when we fix EX03 errors
> 

I just created the change for the indentation issue.

And while running `python scripts/validate_docstrings.py pandas.io.formats.style.Styler.concat`
the missing whitespace error appeared in 3 other examples for `concat`, I corrected it as well.

Should I create a new PR for this?
sure, thanks! or just include them in the next PR
TYP: Use Self for type checking (remaining locations)
State: closed

- [X] closes #51233
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Follow-up to  #51480,  #51493 & #51493. After these 4 PRs, #51233 can be closed. 


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @topper-123
PERF: Improve performance for ser + range
State: closed

- [x] closes #51502 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

@jbrockmendel The problem seems to be inside of numexpr

Labels:
name: Performance
description: Memory or execution speed performance
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations

Comments:
closes #51502?

i wish we had a RangeArray EA analogous to RangeIndex.
PERF: maybe_convert_objects convert_numeric=False
State: closed

I think this fixes a perf regression that @phofl pinged me about a few weeks/months ago, but I forget the details.

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
maybe merge? Can't remember anymore either
Explode?
Ah explode might be the culprit, yes
couple of failing tests
thx @jbrockmendel (accidentally pressed the wrong button...)

Lets see how this changes our asvs, if we see a regression fix then can backport?
works for me
Looks like it's worth backporting

https://asv-runner.github.io/asv-collection/pandas/#reshape.Explode.time_explode
@meeseeksdev backport 2.0.x
REF: combine groupby helpers
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @jbrockmendel
CI: catch warnings in ci
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

let's backport this to get rid of the warnings on the 2.0.x branch as well

Labels:
name: CI
description: Continuous Integration
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
I think https://github.com/pandas-dev/pandas/pull/51312 should cover the changes here if it is backported (which it should IMO)
Would you be ok with checking this stricter like on this branch?
I think it would be good to have stricter warning checks for all the 2.x branches
So this is good to merge?
Oh, I had meant merging https://github.com/pandas-dev/pandas/pull/51312 will cover the warnings handled here and enable the warning checking to be stricter on the 2.x branch(es)
Should I incorporate these changes in https://github.com/pandas-dev/pandas/pull/51312? Is that what you meant with these tests compared to my branch?
Yeah exactly, I'd prefer if we check that a warning is raised instead of only filtering. Sorry for the confusion
closing, will be fixed in the other pr
DOC: object_dtype_tzaware_strings.atype(dt64) raises
State: closed

- [x] closes #50140 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @jbrockmendel 
To be explicit about the implications of doing this as a breaking change: this breaks pandas<->pyarrow roundtrips, and so eg also pandas<->parquet roundtripping (admittedly, in a somewhat corner case of using a DatetimeIndex for the column labels)

So users doing pd.read_parquet(..) with a file they wrote earlier will now get an error (with released pyarrow) that they can't easily solve themselves.
DOC: Add whatsnew file for 2.1.0
State: closed


Labels:
name: Docs
name: Release

Comments:
> This looks like the format for 2.0.1 and not a minor release?

Yes, you're correct, my mistake, fixed it.
Removed the contributor command, this failed the doc build (also left it out in the other release notes initially)
> Removed the contributor command, this failed the doc build (also left it out in the other release notes initially)

Thanks for taking care of this. That was because the tag `2.0.0` doesn't exist yet, updating to `2.0.0rc0` (which is the correct one) would have also fixed the problem, but we can add it later.
PERF: Implement round on the block level
State: closed

- [ ] closes #17254 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Benchmarks are showing same speedup as OP.

```
       before           after         ratio
     [c133327c]       [1a2c35f5]
     <main>           <faster-round>
-            115M              72M     0.63  frame_methods.Round.peakmem_round_transposed
-      1.24±0.2ms       60.9±0.9μs     0.05  frame_methods.Round.time_round
-        863±70ms       61.2±0.4μs     0.00  frame_methods.Round.time_round_transposed

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.
```

Labels:
name: Performance
description: Memory or execution speed performance

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 f298507b153a166bdd2a919274e039c45e740c5d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #51498: PERF: Implement round on the block level'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-51498-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #51498 on branch 2.0.x (PERF: Implement round on the block level)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Belated: LGTM
CI: Use pytest-randomly
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
I think this test causes issues for randomly: https://github.com/pandas-dev/pandas/issues/49916
Ah right I forgot about that issue. I'll do this testing on my own repo until we got this figured out
BUG: Timestamp.round overflows
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ran the hypothesis tests with max_examples=10**6

Labels:
name: Bug
name: Timedelta
description: Timedelta data type
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
rebased+greenish
Note moved, greenish
Thanks @jbrockmendel
TYP: Use Self instead of class-bound TypeVar (generic.py/frame.py/series.py)
State: closed

Progress towards https://github.com/pandas-dev/pandas/issues/51233.

There were more updates to do than I expected, so I split it up a bit. In this PR I do `pandas/core/generic.py`, `pandas/core/frame.py`, `pandas/core/series.py` & `pandas/indexes/`.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @topper-123!
DOC: Add comment about inplace/copy proposal
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: inplace
description: Relating to inplace parameter or equivalent

Comments:
Thx, let’s merge then we‘ll get it into the rc
TST: don't fail on systems with tzdata backzone enabled
State: closed

test_dst.py specifies Africa/Kinshasa but expects a transition date (AmbiguousTimeError) that is actually for Africa/Lagos.  On systems that enable tzdata's backzone file (which Debian and Ubuntu unstable [now do](https://bugs.launchpad.net/bugs/2003797)), these are different, and the test hence fails ([log](https://salsa.debian.org/science-team/pandas/-/jobs/3961090)).

This makes the obvious fix.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Timezones
description: Timezone data dtype

Comments:
The Windows timeout seems to be something that "randomly" happens (see other recent failures).  Maybe we should increase the timeout, or if that's too expensive, run only a subset of tests there?
Thanks @rebecca-palmer 

No objections to merging, would just appreciate it if you explain the issue a little more

Is it that `Africa/Kinshasa` is erroneously raising `AmbiguousTimeError` for `1905-07-01`, and that it shouldn't be doing so, so you're replacing it with `Africa/Lagos`, which correctly raises `AmbiguousTimeError` for that date?

thanks!
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
gentle ping (would much appreciate a little explanation of why this is necessary, I couldn't follow it from the link)
Africa/Lagos has a transition, and hence the AmbiguousTimeError the test checks for, on that date.

On systems without backzone enabled, Africa/Kinshasa is a copy of Africa/Lagos (because they're the same timezone since before 1970(?)), and also does that.

However, on systems with backzone enabled, Africa/Kinshasa has its own timezone history, which doesn't have a transition on that date.  This causes the test to fail on such systems.

This fixes this by instead using Africa/Lagos in the test.
BUG: Fix NameError in tab complete warning test
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


I was running the unit tests locally and noticed this:
<img width="1049" alt="image" src="https://user-images.githubusercontent.com/122238526/219918687-6005cb1e-b3ac-4fc0-ad48-85be36014135.png">

From what I can tell, this is a problem with the test.

With this pull request, the test now passes without throwing this NameError:
<img width="1030" alt="image" src="https://user-images.githubusercontent.com/122238526/219919331-5a7f58b6-7f1e-4a0f-b9a7-4dadaa9d28f8.png">




Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
I run `pytest pandas` in the terminal:

<img width="998" alt="image" src="https://user-images.githubusercontent.com/122238526/219968285-00bc80e6-8b9f-4604-80b8-8f4e5681a372.png">

I set up my environment using mamba following the instructions:
https://pandas.pydata.org/pandas-docs/stable/development/contributing_environment.html#option-1-using-mamba-recommended
i regularly see this in my local test output
thx @smij720
REF: de-duplicate precision_from_unit
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Thanks @jbrockmendel 
Opened a pr to revert if there is no quick fix.
Is there a way to raise exceptions from a Cython function which returns a tuple? I think the issue is that
```
cpdef inline (int64_t, int) precision_from_unit(
    str unit,
    NPY_DATETIMEUNIT out_reso=NPY_DATETIMEUNIT.NPY_FR_ns,
):
```
may raise because it calls `get_conversion_factor`. Problem is, how to propagate it?
Wrapping the get_conversion_factor call in a try/except that just passed on failure (which will then raise on the next line bc `m` is not assigned) fixes the failing tests, but is pretty ugly.  For now could check for M/Y before calling get_conversion_factor.

> Problem is, how to propagate it?

This is weird.  Seems like a cython bug.
REF: de-duplicate NDFrame.take, remove Manager.take keyword
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
thx @jbrockmendel 
@jbrockmendel Looks like this could have caused a performance regression. https://asv-runner.github.io/asv-collection/pandas/#indexing.NumericSeriesIndexing.time_iloc_array?p-index_structure='unique_monotonic_inc'&p-dtype=%3Cclass%20'numpy.int64'%3E

Could not reproduce locally though, do you have an idea?
It's plausible.  This PR makes us go through Manager.take whereas the old version operated directly on ._values/.index.  In retrospect its not that surprising, though I'd have hoped for the effect to be smaller.

Restoring Series.take (the rest of this should be OK) should fix it.

If we go for something like ExtensionManager mentioned in #51471 then we'd need to have basically everything go through a Manager method, in which case we'd need to optimize Manager.take somehow.
Yep adding Series.take back would get performance back, but as you said not really a long term solution unfortunately