BrandonFors/pandas-github-issues huggingface dataset:
REF: Move compute to BinGrouper.result_index_and_ids
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

BaseGrouper does the compute in `result_index_and_ids` and other methods utilize this result. This refactors so BinGrouper behaves the same.

Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @rhshadrach
CLN: Use group_info less
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

My goal is to remove redundant methods in BaseGrouper / Grouping - group_info seems like a good candidate.

Labels:
name: Groupby
name: Clean

Comments:
Thanks @rhshadrach
DOC: ES01 fix for to_csv method
State: closed

All ES01 Errors resolved in the following cases:

scripts/validate_docstrings.py --format=actions --errors=ES01 pandas.DataFrame.to_csv 

- [x] xref #57440 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the effort here, but it was decided that ES01 is not going to be enforced so closing. Happy to have PRs on other doc issues!
PERF: groupby(...).__len__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Enabled by #55738. Prior to that, `ngroups` was not reliable.

```
size = 100_000
df = pd.DataFrame(
    {
        "a": np.random.randint(0, 100, size),
        "b": np.random.randint(0, 100, size),
    }
)
%timeit len(df.groupby(["a", "b"]))

# 161 ms ± 962 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)    <-- main
# 4.96 ms ± 229 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  <-- PR
```



Labels:
name: Groupby
name: Performance
description: Memory or execution speed performance

Comments:
Thanks @rhshadrach
CLN: groupby.first/last unused compat
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

It used to be that any op winding up in `_cython_agg_general` needed a fallback. This is no longer the case since `first` and `last` now handle `object` dtype in the Cython code and `axis=1` has been removed.



Labels:
name: Groupby
name: Clean

Comments:
I was wrong - EAs are still falling back to apply. Opened #57591
DOC: Add contributors for 2.2.1
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57582/
Updating timeseries.offset_aliases links to use :ref: format
State: closed

During code review on https://github.com/pandas-dev/pandas/pull/57562 it was suggested to update the links from this format

`this link <https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#offset-aliases>`__.

to this format 

:ref:`this link<timeseries.offset_aliases>`

I found several places where this same link was using this old format, so opened this PR to update them all for consistency. 

- [x] xref https://github.com/pandas-dev/pandas/pull/57562
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
@mroeschke can you take a look at this PR?  based off the suggestion in https://github.com/pandas-dev/pandas/pull/57562
pre-commit.ci autofix


/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57581/
DOC: fixing GL08 errors for pandas.IntervalIndex.left, pandas.IntervalIndex.mid and pandas.IntervalIndex.length
State: closed

All GL08 Errors resolved in the following cases:

scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.IntervalIndex.left
scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.IntervalIndex.mid
scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.IntervalIndex.length

xref #57443

Labels:
name: Docs

Comments:
I want to know that what should I do next ? I am not sure what should I fix the error, because some of it I'm not really understand. Or should I just skip it?
pre-commit.ci autofix
@datapythonista just FYI there's an effort to minimize/remove docstring reuse (https://github.com/pandas-dev/pandas/pull/57683, https://github.com/pandas-dev/pandas/issues/57578) to minimize complexity of the docstring reuse mechanisms
@j36426052 do you want to continue working in this PR? Besides addressing the comments you'll have to resolve the conflicts.
Sorry that I'm not really sure what part I need to fix. The comment part, should I reuse some comment to let it's structure more complete. I'm not sure how to minimize/remove docstring. 

Also the comflict in Numpy Dev, it show "test-data.xml not found", but I don't see some document talk about this.

Or should I just skip the above I mention. Maybe you mean other comflict?
As mroeschke said, there's no need to worry about reusing docstrings as they are trying to minimize docstring reuse.
Focus on applying the 3 suggested "general comments" first.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Interchange object data buffer has the wrong dtype / from_dataframe incorrect 
State: closed

- [ ] closes #54781 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Interchange
description: Dataframe Interchange Protocol

Comments:
Thanks @MarcoGorelli
DEPS: update numexpr to use not buggy numexpr 2.8.x anymore (update to 2.9.0) - fix critical CVE-2023-39631⁠
State: closed

update numexpr to 2.9.0 to fix CVE-2023-39631⁠ findings

HINT: come from other project (apache-superset) where panda is injecting this faulty numexpr inside the project as sub-dependency

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the PR, but pandas tries to support optional dependencies that are at most 1 year old. Additionally, pandas just species a lower pin, so a user is able to specify numexpr>=2.9.0 if they want to avoid this CVE. pandas will bump to 2.9.0 eventually, but I think it's too early to do so at this moment so closing
DOC: fix ES01 for pandas.Flags
State: closed

All ES01 Errors resolved in the following cases:

1. scripts/validate_docstrings.py --format=actions --errors=ES01 pandas.Flags

- [x] xref https://github.com/pandas-dev/pandas/issues/57440
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @jordan-d-murphy
DOC: fixing GL08 errors for pandas.DatetimeIndex.as_unit and pandas.DatetimeIndex.freq
State: closed

All GL08 Errors resolved in the following cases:

1. scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.DatetimeIndex.as_unit
2. scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.DatetimeIndex.freq

_Note: [6232a2e](https://github.com/pandas-dev/pandas/pull/57562/commits/6232a2ed2590fb8d8b43658adc009688cb30dce2) commit message was a typo, this commit is for GL08 errors_

- [x] xref https://github.com/pandas-dev/pandas/issues/57443
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
pre-commit.ci autofix


pre-commit.ci autofix



Thanks @jordan-d-murphy
DOC: fixing PR01 errors for pandas.Categorical and pandas.Categorical.__array__
State: closed

All PR01 Errors resolved in the following cases:

1. scripts/validate_docstrings.py --format=actions --errors=PR01 pandas.Categorical
2. scripts/validate_docstrings.py --format=actions --errors=PR01 pandas.Categorical.__array__

- [x] xref https://github.com/pandas-dev/pandas/issues/57438
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks for the feedback - updated this with your suggestion, can you take another look @mroeschke? 
Thanks @jordan-d-murphy
PERF: Switch conditions in RangeIndex._shallow_copy
State: closed

xref https://github.com/pandas-dev/pandas/pull/57534#issuecomment-1957832841

`is_range_indexer` lets us short circuit unlike checking the remainder

```python
In [1]: from pandas import *; import numpy as np
   ...: np.random.seed(123)
   ...: size = 1_000_000
   ...: ngroups = 1000
   ...: data = Series(np.random.randint(0, ngroups, size=size))
+ /opt/miniconda3/envs/pandas-dev/bin/ninja
[1/1] Generating write_version_file with a custom command

In [2]: %timeit data.groupby(data).groups
16.8 ms ± 183 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # PR

In [3]: %timeit data.groupby(data).groups
17.6 ms ± 114 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  # main
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
thx @mroeschke
Remove PyArrow deprecation warning
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57556/
thanks @phofl.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 4ed67ac9ef3d9fde6fb8441bc9ea33c0d786649e
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57556: Remove PyArrow deprecation warning'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57556-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57556 on branch 2.2.x (Remove PyArrow deprecation warning)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
I'll do the backport
#57558
Roadmap: remove ArrayManager
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The deprecation of ArrayManager was recently enforced; this is no longer a Roadmap item.

Labels:
name: Docs
name: Clean

Comments:
Thanks @jbrockmendel
API: avoid passing Manager to subclass __init__
State: closed

- [x] closes #57032 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Many subclasses have `__init__` methods that call `super().__init__`, which in the relevant cases with a Manager object is deprecated.  The solution in this PR is to construct the pd.DataFrame/Series and pass that to the subclass's `_constructor`.

This removes _sliced_from_mgr and _expanddim_from_mgr, as they are not really necessary, simplifying things a bit.  Also makes _from_mgr `@final`.  Can separate these simplifications into their own PR on request.  xref #56681.

For subclasses that currently have special handling for Manager objects in their `__init__`, they will need to move that special handling to the appropriate `_constructor(_foo)?_from_mgr` method.  The only package I'm aware of that does this is geopandas, so this includes a shim special-casing GeoDataFrame intended to maintain the current behavior until they have time to move that logic.  cc @jorisvandenbossche ATM the shim is only in _constructor_from_mgr, pls advise on whether something analogous is needed in constructor_sliced_from_mgr and _constructor_expanddim_from_mgr.

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
thx @jbrockmendel 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 805dbde3651fa4f7c30d5c9c247f723dceb7dfde
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57553: API: avoid passing Manager to subclass __init__'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57553-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57553 on branch 2.2.x (API: avoid passing Manager to subclass __init__)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BLD: Add pyarrow extra for pip installation
State: closed

Partial feedback from #54466

Labels:
name: Build
description: Library building on various platforms

Comments:
thx @mroeschke 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 734501e595d1e771774949232ed5f382e238db8d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57551: BLD: Add pyarrow extra for pip installation'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57551-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57551 on branch 2.2.x (BLD: Add pyarrow extra for pip installation)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DOC: Add missing docstrings for Index.empty, Index.view and Index.names
State: closed

- [ All docstrings tests passed]


Labels:
name: Docs
name: Index
description: Related to the Index class or subclasses

Comments:
pre-commit.ci autofix
/preview
> I fixed all the issues here, if someone else can have a second look to see if this can be merge. The PR is stale but it's a nice addition, so I've been addressing all the issues myself.

LGTM.
@mroeschke all docstrings seem to have already been added in main, should we close this?

I think the changes here provide more info and I also see a few differences that look correct here but are wrong on main.  I can follow up with a PR that will add some of the changes here to the current docstrings.
> all docstrings seem to have already been added in main, should we close this?

Sure we can close this and have a follow up to improve the docstrings if needed