BrandonFors/pandas-github-issues huggingface dataset:
Enh arrow json extension
State: closed

- [x] closes #60958 
- [x] [Tests added and passed]
- [x] All [code checks passed]
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks @asharmalik19 
Thanks @mroeschke for such quick feedback
Thank you so much for your help, @asharmalik19 and @mroeschke !
CI/TST: Address TestArrowArray::test_reduce_series_numeric supporting skew
State: closed

xref https://github.com/apache/arrow/pull/45677

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Arrow
description: pyarrow functionality

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 89bc204e94cda860085865081cd18cd1d130cec1
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #61098: CI/TST: Address TestArrowArray::test_reduce_series_numeric supporting skew'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-61098-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #61098 on branch 2.3.x (CI/TST: Address TestArrowArray::test_reduce_series_numeric supporting skew)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOCS: remove outdated `PeriodDtype.from_date_offset` comment, use `DateOffset` instead of `BaseOffset` in docstrings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

- closes #60886 
- `PeriodDtype.from_date_offset` was removed in #46338
- some drive-by walruses

Labels:

Comments:
🤦 i pushed to a branch in pandas-dev instead of in my fork, i'll start this one over
Boolean indexing inconsistency
State: closed

- [x] closes #60994
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
`iloc.__setitem__` functionality should remain the same and `iloc.__getitem__` should be implemented instead
Update hist.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the PR, but the tests are all failing and this PR doesn't appear related to an open issue. Closing this PR, but happy to have a new PR with more context
ENH: Add Rolling.nunique()
State: closed

- [ ] closes #26958 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.

Hash-map (C++ `unordered_map` via Cython) based implementation of `Rolling.nunique()` with `O(N)` average time complexity.

Labels:
name: Enhancement
name: Window
description: rolling, ewma, expanding

Comments:
Thanks @snitish
DEPS: Update NumpyExtensionArray repr for NEP51
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Continuation of #54268


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
@mroeschke - since yours went into 2.1, I was thinking it'd make sense to put this in 2.3. But no objection to holding off until 3.0 either.
@mroeschke - whatsnew added.
Thanks @rhshadrach
BUG: Improve ImportError message to suggest importing dependencies directly for full error details
State: closed

- [x] closes #61030 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Hi @rhshadrach , I have implemented your suggestion from the https://github.com/pandas-dev/pandas/issues/61030. Could you please verify if this implementation meets your expectations? Thank you!

Below are example outputs before and after this change:

* before:
  ```python
  >>> import pandas as pd
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/pandas/pandas/__init__.py", line 16, in <module>
      raise ImportError(
  ImportError: Unable to import required dependencies:
  numpy: Error importing numpy: you should not try to import numpy from
          its source directory; please exit the numpy source tree, and relaunch
          your python interpreter from there.
  ```
* after change:
  ```python
  >>> import pandas as pd
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/pandas/pandas/__init__.py", line 16, in <module>
      raise ImportError(
  ImportError: Unable to import required dependencies:
  numpy: Error importing numpy: you should not try to import numpy from
          its source directory; please exit the numpy source tree, and relaunch
          your python interpreter from there.
  
  To see the full error message, try importing the missing dependencies directly.
  ```
> what do you think about adding tzdata to the _hard_dependencies on L6 here.

Yeah we should probably add that too.
> > what do you think about adding tzdata to the _hard_dependencies on L6 here.
> 
> Yeah we should probably add that too.

@mroeschke For `tzdata`, should I add the changes in this PR or create a separate issue? Thanks!
> For tzdata, should I add the changes in this PR or create a separate issue?

Please create a separate issue for this
> > For tzdata, should I add the changes in this PR or create a separate issue?
> 
> Please create a separate issue for this

I’ve already created the issue: https://github.com/pandas-dev/pandas/issues/61273. And I'd be happy to work on this after this PR merged!
Thanks @chilin0525
Bug: Fix for overflow at edge cases in `normalize()` by raising exception
State: closed

- [x] closes #60583 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
@mroeschke @MarcoGorelli tagging here to review the PR
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Fix OverflowError in lib.maybe_indices_to_slice()
State: closed

This fixes this error when slicing massive dataframes:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/frame.py", line 4093, in __getitem__
    return self._getitem_bool_array(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/frame.py", line 4155, in _getitem_bool_array
    return self._take_with_is_copy(indexer, axis=0)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/generic.py", line 4153, in _take_with_is_copy
    result = self.take(indices=indices, axis=axis)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/generic.py", line 4133, in take
    new_data = self._mgr.take(
               ^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/internals/managers.py", line 893, in take
    new_labels = self.axes[axis].take(indexer)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/lib/python3.12/site-packages/pandas/core/indexes/datetimelike.py", line 839, in take
    maybe_slice = lib.maybe_indices_to_slice(indices, len(self))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib.pyx", line 522, in pandas._libs.lib.maybe_indices_to_slice
OverflowError: value too large to convert to int

- [x] closes #59531
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature (test not added because it requires a large amount of memory to reproduce this problem)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions (left the existing type-hint as-is because it seems `int` is correct from a Python perspective)
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
This is a squash and rebase of #59537.
> Can you also update pandas/_libs/lib.pyi like in https://github.com/pandas-dev/pandas/pull/59537/files?

If I do that, the CI fails.  See the CI run for the "Sort whatsnew entries" commit for the errors.  It seems like the type hint needs to stay as `int`?
Thanks @swt2c
Doc for backend-specific arguments for df.plot()
State: closed

Updated DataFrame.plot documentation and visualization guide for back end-specific arguments.

- [x] closes #61020 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Stale

Comments:
The doc build appears to be failing the check because of my use of double backticks in the line 1813:  "Pandas provides a consistent ``plot`` API...", but this is the same use of backticks a few lines earlier and is how the documentation is treating methods. Please advise if this is incorrect.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: PeriodIndex.to_datetime inconsistent with its docstring
State: closed

- [x] closes #59371
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: Update warning in `Index.values` docstring to clarify index modification which causes segmentation fault
State: closed

Index Modification issues (#60954)

- [x] Added [docstring](pandas/core/indexes/base.py) under existing function.

This PR updates the docstring for Index.values to clearly warn users that modifying it directly is not supported and may lead to memory corruption or segmentation faults. It also recommends safe alternatives, such as using `Index.array` or `Index.to_numpy(copy=True)`.

As per @mroeschke s note, in Pandas 3.0 this operation will raise an error under Copy-on-Write mode, making it clear that modifying is disallowed.

closes https://github.com/pandas-dev/pandas/issues/60954

Labels:
name: Docs

Comments:
@mroeschke Thanks, that was helpful. I made the changes and all the checks are successful.
Let me if any there is any changes required.
Thanks @Manju080
DOC: Fix typo in Timestamp.isoformat
State: closed

repeated word

- ~~[ ] closes #xxxx (Replace xxxx with the GitHub issue number)~~
- ~~[ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~[ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~


Labels:
name: Docs

Comments:
Thanks @wjandrea
DOC: Fix syntax highlighting in overview
State: closed

Remove bogus syntax highlighting on LICENSE in overview.rst

- ~~[ ] closes #xxxx (Replace xxxx with the GitHub issue number)~~
- ~~[ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~[ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

Labels:
name: Docs

Comments:
Thanks @wjandrea
DOC: Add inference type information to Dataframe Apply
State: closed

- [#61057] (Replace xxxx with the GitHub issue number)
-~~[ ] [Tests added and passed] if fixing a bug or adding a new feature~~
- [ ] All [code checks passed]
-~~[ ] Added [type annotations] to new arguments/methods/functions.~~
-~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~


Labels:
name: Docs

Comments:
Thanks @PenguinPen
DOC: Add link description
State: closed

- ~~[ ] closes #xxxx (Replace xxxx with the GitHub issue number)~~
- ~~[ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~[ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

Labels:
name: Docs

Comments:
I initially made another small change by removing a space, but it turns out it didn't work so I undid it. Do you want me to squash or will you?
Thanks @wjandrea
BUG: Fix na_position type in IndexEngine
State: closed

- [x] closes #58924 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature  
  (I modified an existing test to cover the problematic case)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).  
  (Stubs check fails locally for seemingly unrelated reasons. Passed on GH Actions.)
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.  
  (**Not applicable**)
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
Perhaps I should not have rebased. The changes suggested above by @mroeschke are the only changes, I promise.
Thanks @avm19
DOC: Correct a typo in ecosystem.md
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @m-ahmadi
ENH: json_normalize accepts JSON with str and bytes input
State: closed

- [x] closes #61006
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.


Labels:

Comments:
Thanks for the review! I was unaware of the deprecation, but I understand the concern about API consistency. Will be closing this for now
DOC Fix Styler.to_latex to be in Writer column
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I saw that this field for the latex writer was actually in the read column.  This moves this to the right column.  Here's a link to the current doc page: https://pandas.pydata.org/docs/user_guide/io.html


Labels:
name: Docs

Comments:
Thanks @rutrum
Renamed "normalise" to "normalize"
State: closed

- [x] closes #61047 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @arthurlw