BrandonFors/pandas-github-issues huggingface dataset:
ENH: Support pipe() method in Rolling and Expanding
State: closed

- [ ] closes #57076
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.

Support `pipe()` method for chained computation on `Rolling` and `Expanding` objects, similar to `Dataframe.pipe()`, `Groupby.pipe()` etc.

Labels:
name: Window
description: rolling, ewma, expanding

Comments:
Thanks @snitish
ENH: Expose NoDefault in pandas.api.extensions
State: closed

- [ ] closes #60488
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.

From the original issue -
```
No type is provided for the pandas.api.extensions.no_default sentinel.

This is a problem when trying to type annotate functions or methods that forward arguments into Pandas functions that accept the no_default sentinel.
```

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @snitish
BUG: Fix ValueError in DataFrame/Series regex replace for all-NA values
State: closed

- [x] closes #60688
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.

In case of all-NA values, `pandas.core.array_algos.replace.compare_or_regex_search` throws a `ValueError` as it calls `np.vectorize` without specifying `otypes`. Specifying `otypes` lets `np.vectorize` work with empty arrays.


Labels:
name: replace
description: replace method

Comments:
Thanks @snitish
PERF: Fix ASV CSV benchmarks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

`self.data` is just calling `.seek(0)` on the StringIO object. The current implementation doesn't fail on our CI because we only run each benchmark once.

Labels:
name: Performance
description: Memory or execution speed performance
name: IO CSV
description: read_csv, to_csv

Comments:
Merging as this is blocking some work I'm doing on ASVs.
Remove editable installs from CI
State: closed

The meson-python team does not suggest using editable installs in a CI environment. See also https://github.com/mesonbuild/meson-python/issues/716#issuecomment-2580286829

Labels:
name: Stale

Comments:
Not sure if this should go 2.3 or 3.0 - probably the former?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Not actively working on this - let's clear the queue
TST(string dtype): Resolve xfails in test_to_csv
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Current behavior of these tests is correct, for most we start with `object` dtype and then roundtripping through `to_csv` we end up with a StringDtype. Modifying tests so the input starts as StringDtype.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO CSV
description: read_csv, to_csv
name: Strings
description: String extension data type and string data

Comments:
Awesome thanks @rhshadrach 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 4693d1a000f34641f8215caea6872b3357f4db76
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60683: TST(string dtype): Resolve xfails in test_to_csv'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60683-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60683 on branch 2.3.x (TST(string dtype): Resolve xfails in test_to_csv)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
backport PR is https://github.com/pandas-dev/pandas/pull/60687
Add meson-format to pre-commit
State: closed

Meson 1.6.0 introduced a new tool for formatting meson configuration files

This will need https://github.com/pandas-dev/pandas/pull/60681 before  it actually runs in CI, but the changes here include what happens on a local run

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @WillAyd
Relax meson pins in CI and Fix Missed Warnings
State: closed

Back in https://github.com/pandas-dev/pandas/pull/60089 we relaxed the meson pin in our pyproject, but all of our CI still hard codes the minimum version. This updates it to treat the meson dependencies the same way as Cython, i.e. newer versions are used in CI

Labels:
name: Build
description: Library building on various platforms
name: Stale

Comments:
Ah so I guess the pinned versions here were suppressing compiler errors in some fashion. Looks like https://github.com/pandas-dev/pandas/pull/60674 will need to happen before this
I believe the remaining issues appeared with meson-python 0.16.0. Reported upstream in https://github.com/mesonbuild/meson-python/issues/716

May need to put an upper limit on the 0.15.0 release for now
The pyarrow failure here is unrelated
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Not actively working on this - let's clear the queue
Implement String Accumulations with nanoarrow
State: closed

This is an addon to the great work that @rhshadrach is doing in https://github.com/pandas-dev/pandas/pull/60633

Here is performance on this branch:

```python
In [1]: import pandas as pd

In [2]: import pyarrow as pa

In [3]: ser = pd.Series(["foo", "bar", "baz"] * 10000, dtype=pd.ArrowDtype(pa.string()))

In [4]: %timeit ser.cumsum()
504 ms ± 29.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [5]: %timeit ser.cummin()
1.23 ms ± 12.6 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [6]: %timeit ser.cummax()
1.23 ms ± 36.5 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
```

versus without nanoarrow:

```python
In [1]: import pandas as pd

In [2]: import pyarrow as pa

In [3]: ser = pd.Series(["foo", "bar", "baz"] * 10000, dtype=pd.ArrowDtype(pa.string()))

In [4]: %timeit ser.cumsum()
1.72 s ± 47.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [5]: %timeit ser.cummin()
1.49 ms ± 81.2 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [6]: %timeit ser.cummax()
1.58 ms ± 42.9 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
```

While they all show some improvement, cummin / cummax don't show as much. I believe this has to do with the fact that the implementations still have to access the Python runtime within a tight loop to perform comparisons. If we cared to optimize further, we could look at utf8proc

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Not actively working on this - let's clear the queue
Update docs for generating unoptimized debug builds
State: closed

Not sure what changes were made recently to the compilers package but I've noticed without undef'ing `_FORTIFY_SOURCES` that the compiler throws a lot of errors about having that defined with optimizations turned off. Looks like we were also missing flags for C++

I forget where the original conversation was about this, but I'm assuming these flags are never going away with the compilers package. @rgommers does NumPy still use that package or have you moved on to a possibly better solution?

Labels:
name: Docs
name: Stale

Comments:
The `compilers` package is still relevant, and we're using it in NumPy, SciPy and various other places. There is no good alternative yet unfortunately. 

I haven't seen this problem yet. Could you add the errors you're seeing and the version/platform of `compilers` where you started seeing this in a comment or PR description?
Ah OK - I think I am starting to see the warning as I'm experimenting with some C++ extensions, and the warning seems to come from C++ not C

```sh
  /home/willayd/miniforge3/envs/pandas-dev/x86_64-conda-linux-gnu/sysroot/usr/include/features.h:330:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
    330 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
```

I'll have to do some more research to pinpoint the versions and exact conditions under which this appears, but I'm seeing it on ubuntu 24.04. Looks like the compilers package installs `gcc=13.3.0=h9576a4e_1`
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Not actively working on this - let's clear the queue
TST(string dtype): Resolve replace xfails
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: replace
description: replace method

Comments:
Thanks @rhshadrach
Added improvements to the cheat sheet based on issue #40680
State: closed

- [x] closes #40680
- [ ] Tests added and passed (N/A – documentation-only change)
- [x] All code checks passed
- [ ] Added type annotations (N/A – documentation-only change)





Labels:
name: Docs
name: Stale

Comments:
cc @Dr-Irv 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Closing as stale. @deveshbervar if you'd like to continue, address the comments above and we'd be happy to reopen!
> Closing as stale. @deveshbervar if you'd like to continue, address the comments above and we'd be happy to reopen!

Hi @rhshadrach ,
Thank you for your feedback! I’d like to continue working on this and address the comments you’ve provided. Could you please reopen this issue so I can proceed with the necessary changes? I’ll make sure to update it once it’s reopened.
Thank you for your time and guidance!
DEPS: Bump optional dependencies
State: closed

These should have been out for at least ~1 year once pandas 3.0 comes out

Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
Looks good - maybe worth holding off until after 2.3 gets released? 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Doesn't seem like the WIndows ADBC issue is going to be fixed anytime soon (at least based on the issue activity). Should we move forward and merge this with the working versions?
Replace flake8-rst with flake8-rst-docstrings
State: closed

Fixes #46865

Replace `flake8-rst` with `flake8-rst-docstrings` for running flake8 on code snippets in rst files.

* **pyproject.toml**
  - Remove `flake8` from the `tool.ruff.lint` section.
  - Add `flake8-rst-docstrings` to the `tool.ruff.lint` section.
  - Update the `tool.ruff.lint` section to use `flake8-rst-docstrings` instead of `flake8-rst`.

* **.pre-commit-config.yaml**
  - Remove the `flake8-rst` hook from the `hooks` section.
  - Add a new hook for `flake8-rst-docstrings` in the `hooks` section.
  - Configure the new `flake8-rst-docstrings` hook with appropriate `additional_dependencies`.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/pandas-dev/pandas/pull/60654?shareId=89ddda75-4f5f-4ddf-87d1-3b3afb809a56).

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST (string dtype): clean-up interpolate tests
State: open

Small follow-up on https://github.com/pandas-dev/pandas/pull/60637: move the code checking for the error for non-numeric data to a separate test, so we can actually still test the "basic" behaviour properly in the case there are only numeric columns. And remove the xfail for the non-range index case.



Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Strings
description: String extension data type and string data
name: Stale

Comments:
Milestone should be 2.3 right?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Backport PR #60454 on branch 2.3.x (String dtype: coerce missing values in indexers for string dtype Index)
State: closed

Backport PR #60454: String dtype: coerce missing values in indexers for string dtype Index

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Strings
description: String extension data type and string data

Comments:
Will fixup tomorrow
BUG/TST (string dtype): raise proper TypeError in interpolate
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Strings
description: String extension data type and string data

Comments:
Thanks @rhshadrach 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 5e50d3f3d2b0ee65f0d5bfda0c6da47ffd39dcfe
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60637: BUG/TST (string dtype): raise proper TypeError in interpolate'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60637-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60637 on branch 2.3.x (BUG/TST (string dtype): raise proper TypeError in interpolate)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60652
TST(string dtype): Remove xfail for combine_first
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This was added in #55409 and no longer raises a warning. I cannot find any warning that matches `empty entries`, I'm guessing the warning itself has been removed.

Edit: found it, the warning was from https://github.com/pandas-dev/pandas/pull/52532.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data

Comments:
(the xfail doesn't exist on the 2.3.x branch, so not marking for backport)
ENH: Implement cum* methods for PyArrow strings
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Implements `cumsum`, `cummin`, and `cummax` for PyArrow-backed strings. I don't see a way to do this without passing through NumPy - if there are other ideas for approaching happy to give those a shot.

cc @WillAyd @jorisvandenbossche 

Labels:
name: Enhancement
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
> I don't see a way to do this without passing through NumPy - if there are other ideas for approaching happy to give those a shot.

The only other way I think that would be reasonable to implement this in would be to use nanoarrow, but that's a larger implementation. I think this is fine for now - just not very performant but that can always be improved later
@jorisvandenbossche - friendly ping.
Thanks @rhshadrach 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b5d4e89d378e69a87b1b9ac7f3d6fa6867840fff
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60633: ENH: Implement cum* methods for PyArrow strings'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60633-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60633 on branch 2.3.x (ENH: Implement cum* methods for PyArrow strings)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60753