BrandonFors/pandas-github-issues huggingface dataset:
CLN: Enforce deprecation of method and limit in pct_change methods
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: #53520

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Clean
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
Thanks @rhshadrach
CLN: Enforce deprecation of groupby with as_index=False excluding out-of-axis groupings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: #49519

Labels:
name: Groupby
name: Clean

Comments:
Thanks @rhshadrach
CI/TST: Use worksteal over loadfile for pytest-xdist
State: closed

In theory, this should help alleviate a worker that is running tests from a larger file. Let's see if it saves any time on the CI.

Labels:
name: CI
description: Continuous Integration

Comments:
I'm seeing about a ~5-7 minute test run reduction on Ubuntu/Windows jobs and ~10-15 minute test run reduction on OSX jobs. 
Going to merge since all green here
PERF: improve StringArray.isna
State: open

See https://github.com/pandas-dev/pandas/issues/57431#issuecomment-1978293416 for context.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Speed-up is around 2x for this case (extracted from one of our ASVs):

```python
dtype = "string"
N = 10**6
data = np.array([str(i) * 5 for i in range(N)], dtype=object)
na_value = pd.NA
ser = pd.Series(data, dtype=dtype)

%timeit ser.isna()
# 11.3 ms ± 47.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)    <-- main
# 5.01 ms ± 55.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)    <-- PR
```

Not entirely sure it is worth the extra code, but so it definitely gives a decent speedup for a common operation.

Labels:
name: Performance
description: Memory or execution speed performance
name: Strings
description: String extension data type and string data
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
needs rebase, otherwise LGTM
DOC: fixing GL08 errors for pandas.IntervalIndex.right and pandas.Series.dt
State: closed

Resolved GL08 errors for the following:
1. `scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.IntervalIndex.right`
2. `scripts/validate_docstrings.py --format=actions --errors=GL08 pandas.Series.dt`

xref: #57443 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
pre-commit.ci autofix
Also, since you worked on the `main` branch, you may want to clone pandas again and start the work in different branches. If you work in `main`, then any new branch you create to work in different things in parallel will contain the changes you made in `main`, and you will have unrelated things in the branches. In general you want to make sure you never commit to `main` and it's synced with the latest `main` in pandas, so when you create branches you start from the right state of pandas.

I'll close this PR, since you'll have to open new ones from the new branches, but surely happy to review your changes in those.
moved to #57751
CI: locking `pytest` version = `8.1.0`
State: closed

- [x] closes #57718

locked `pytest` to version more then `8.1.0`, as a temporary solution.

the reason: with the latest `pytest` version `8.1.0` we get an error when runninig pytest
``` 
File "/home/panda/miniforge3/envs/pandas-dev/lib/python3.10/site-packages/pluggy/_manager.py", line 342, in _verify_hook
    raise PluginValidationError(
pluggy._manager.PluginValidationError: Plugin 'pytest_cython' for hook 'pytest_collect_file'
hookimpl definition: pytest_collect_file(path, parent)
Argument(s) {'path'} are declared in the hookimpl but can not be found in the hookspec
```
I think, it’s a bug in the last version of `pytest-cython-0.2.1` (see [link](https://github.com/lgpage/pytest-cython/issues/59))
which is related to enforcing deprecation of `path` parameter in the last version of `pytest 8.1.0` (see [link](https://github.com/pytest-dev/pytest/issues/11779))

Labels:

Comments:
I think the pytest devs just yanked 8.1 from pypi and conda forge so we may not need this anymore: https://github.com/pytest-dev/pytest/issues/12069

e.g. https://github.com/pandas-dev/pandas/actions/runs/8146954729/job/22266609711#step:4:427
indeed, seems we don't need this anymore.  Thank you, @mroeschke, I will close this PR.
Thanks for being proactive on this!
TST/CI: Fix test_repr on musl for dateutil 2.9
State: closed


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
Merging to get part of the CI to green
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 c3ae17d04d3993e1c3e4c0f8824fde03b8c9beac
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57726: TST/CI: Fix test_repr on musl for dateutil 2.9'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57726-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57726 on branch 2.2.x (TST/CI: Fix test_repr on musl for dateutil 2.9)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
[pre-commit.ci] pre-commit autoupdate
State: closed

<!--pre-commit.ci start-->
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.1.13 → v0.3.0](https://github.com/astral-sh/ruff-pre-commit/compare/v0.1.13...v0.3.0)
- [github.com/jendrikseipp/vulture: v2.10 → v2.11](https://github.com/jendrikseipp/vulture/compare/v2.10...v2.11)
- [github.com/pylint-dev/pylint: v3.0.1 → v3.1.0](https://github.com/pylint-dev/pylint/compare/v3.0.1...v3.1.0)
- [github.com/PyCQA/isort: 5.12.0 → 5.13.2](https://github.com/PyCQA/isort/compare/5.12.0...5.13.2)
- [github.com/asottile/pyupgrade: v3.15.0 → v3.15.1](https://github.com/asottile/pyupgrade/compare/v3.15.0...v3.15.1)
<!--pre-commit.ci end-->

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Since ruff was bumped https://github.com/pandas-dev/pandas/pull/57766 and other other dependencies are not critical. We'll wait until next month
update from 2022 to 2024 image
State: closed

- [x] closes  #57719
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
Thanks for identifying @lopof. Looks like `default` maps to a LTS version and is avoiding the brownout period so going to merge for now. If needed we can pin to a specific image in a follow up
DOC: Remove references to `bfill`, `ffill`, `pad`, and `backfill` in `limit_direction`
State: closed

Picked up issue #57092. This removes references to `bfill`, `ffill`, `pad`, `backfill` from `limit_direction` as they aren't supported. Also references `bfill` and `ffill` in the "See Also" section. Up to date with latest commit.

- [X] closes #57092 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Docs

Comments:
Thanks @TechnoShip123
#48188 FIX: iloc works with namedtuple
State: closed

- unit test for added for test_iloc.py that verifies that all four uses of iloc given in the reproducible example now works

- [x] closes #48188
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature

Labels:

Comments:
@WillAyd could you review this pr?
Thanks for the PR, but there's already a PR open addressing this issue https://github.com/pandas-dev/pandas/pull/57004

Closing to let that user still work on that issue but happy to have your help on another `good first issue`
BUG: Line plots aligned from start #57594
State: closed

#57594 BUG: df.plot makes a shift to the right if frequency multiplies to n > 1

The cause of this bug is that by default all elements are generated to align to the end of
a period instead of at the start and the period technically does not end at the start of
the last period but rather just before the period subsequent to the last period. To fix this
problem we therefore passed a parameter stating that the elements should be aligned to
the start of the period instead of the end. Parameter "how=S" was added in pandas/plotting/_matplotlib/timeseries.py::maybe_convert_index(), 
on the function call data.index.asfreq(freq=freq_str, how = "S").  

Additional tests were added to test whether input data match data in plot and one test was merged into one the added test to reduce code duplication. 

Labels:
name: Visualization
description: plotting
name: Stale

Comments:
Thank you for working on this @NoelMT. Do you mind adding plots to this PR (as a comment or in the description) that show how things are rendered before and after your changes please? That would give context to reviewers and it'll make reviewing much easier. Thanks! 
Thanks @NoelMT for working on this. Seems like it will fix the problem with visualisation of `DataFrame` if `PeriodIndex` as index provided.
e.g. for the first scenario:
```
from pandas import *
import numpy as np

idx = period_range("01/01/2000", freq='7h', periods=4)
df = DataFrame(
    np.array([0, 1, 0, 1]),
    index=idx,
    columns=["A"],
)
df.plot()
print(df)
```

plot before:

<img width="552" alt="res_7h" src="https://github.com/pandas-dev/pandas/assets/91160475/47bccd86-a957-42e1-b021-de15e63cf85e">

plot after:

<img width="620" alt="dfplot_7h_correct" src="https://github.com/pandas-dev/pandas/assets/91160475/235ed96a-8b94-4d50-9cb4-181a387e9242">

@NoelMT could you please add a note to `doc/source/whatsnew/v3.0.0.rst` in the section
https://github.com/pandas-dev/pandas/blob/dc19148bf7197a928a129b1d1679b1445a7ea7c7/doc/source/whatsnew/v3.0.0.rst?plain=1#L356

This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST: Add test for `grouby` after `dropna` agg `nunique` and `unique`
State: closed

- [x] closes #42016 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @luke396
REF: Remove dynamic docstrings from option methods
State: closed

xref https://github.com/pandas-dev/pandas/issues/57578#issuecomment-1960937283

These methods have complex docstrings in an attempt to display all option within the docstring. This makes these docstrings hard to automatically validate and [renders the internal implementation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.describe_option.html#pandas.describe_option) in Sphinx. I think it's reasonable to link to the docs where all the options are listed instead.

Additionally did some cleanups removing `silent` from some of these methods by instead catching warnings where needed



Labels:
name: Refactor
description: Internal refactoring of code

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57710/
REF: Avoid importing xlrd
State: closed

- [x] closes #56692 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Not sure how I'd add a test here.

Labels:
name: Refactor
description: Internal refactoring of code
name: IO Excel
description: read_excel, to_excel
name: Dependencies
description: Required and optional dependencies

Comments:
Could check sys.modules for presence of xlrd?
> Could check sys.modules for presence of xlrd?

True; since that's global I think it'd have to check it in a subprocess. It seems to me a bit of an odd thing to test - unless we think we should have more testing that we don't import an optional dependency if it's not used more generally. Are we okay foregoing a test here?
Fine by me

On Mon, Mar 4, 2024 at 1:55 PM Richard Shadrach ***@***.***>
wrote:

> Could check sys.modules for presence of xlrd?
>
> True; since that's global I think it'd have to check it in a subprocess.
> It seems to me a bit of an odd thing to test - unless we think we should
> have more testing that we don't import an optional dependency if it's not
> used more generally. Are we okay foregoing a test here?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/57708#issuecomment-1977529179>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AB5UM6HDBJRPX334VW46JMLYWTUTPAVCNFSM6AAAAABEDK5ATGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGUZDSMJXHE>
> .
> You are receiving this because you commented.Message ID:
> ***@***.***>
>

Thanks @rhshadrach
Cython guard against [c|m|re]alloc failures
State: closed

Not sure if these are testable but in the case of the system being exhausted for resources these should explicitly throw a MemoryError instead of allow undefined behavior

@MarcoGorelli don't know how difficult it would be but would be cool to have the cython linter catch uses of any low-level allocation function and ensure that a guard against NULL comes thereafter. The only exception I could think of to that rule is if you are immediately returning the result of an malloc and expect the caller to guard against NULL, but we don't have any instances of that in our code base (may also not matter for Cython)

Labels:
name: Clean

Comments:
Thanks @WillAyd
Nanosecond fixed precision of DatetimeIndex with time zone information, Fix #53473
State: closed

The expected behaviour is now correct from issue #53473 The out->picoseond value wasn't set which ended with a Json error regarding the ns calculations.

- [x] closes #53473
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: Stale

Comments:
Hi @WillAyd could you please check the new fixes, we would appreciate that :)
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DEPR: depreecate uppercase units 'MIN', 'MS', 'US', 'NS' in Timedelta and 'to_timedelta'
State: closed

Deprecated uppercase strings `“MIN”, “MS”, “US”, “NS”` denoting units from the class `Timedelta` and the method `to_timedelta`.

The reason: for period/offsets these strings are already deprecated. Instead of the uppercase strings we use lowercase: `“min”, “ms”, “us”, “ns”`.

Labels:
name: Timedelta
description: Timedelta data type
name: Deprecate
description: Functionality to remove in pandas
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
closed via #59051
DEPR: remove deprecated freqs/abbrevs 'A', 'A-DEC', 'A-JAN', etc.
State: closed

xref #55252

remove deprecated string 'A', 'A-DEC', etc. denoting timedelta abbreviations and timeseries frequencies


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @MarcoGorelli for reviewing this PR.
Thanks @natmokval
ENH: Adding `pipe` operations to `pandas.api.typing.Rolling` instances
State: closed

- [ ] closes #57076 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Fix empty MultiIndex DataFrame xlsx export
State: closed

Fix Excel xlsx export of empty (zero rows) DataFrame with MultiIndex on both axes.

- [x] closes #57696
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: IO Excel
description: read_excel, to_excel
name: MultiIndex

Comments:
Thanks @speleo3 - nice fix!
Thanks a lot for fixing this. I'm running into this exact issue at work. Since 3.0.0 hasn't been released yet, is there a workaround in the meantime?
My attempt:

```py
if df.empty and not df.columns:
    df.columns = []
```