BrandonFors/pandas-github-issues huggingface dataset:
DEPR: is_categorical_dtype
State: closed

- [x] closes #35793 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Categorical
description: Categorical Data Type
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
PERF: ArrowExtensionArray.to_numpy
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

There are a few different cases/paths in `ArrowExtensionArray.to_numpy` where copying can be avoided and more efficient fillna logic can be used:

```
import pandas as pd
import numpy as np

N = 10_000_000
data = np.random.randn(N)


# -----------------------------------------------------------------------------------
# case 1: no nulls

arr = pd.array(data, dtype="float64[pyarrow]")

%timeit arr.to_numpy("float64")

# 16 ms ± 620 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)        -> main
# 19.8 µs ± 711 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  -> PR


# -----------------------------------------------------------------------------------
# case 2: contains nulls

arr = pd.array(data, dtype="float64[pyarrow]")
arr[::1000] = None

%timeit arr.to_numpy("float64", na_value=np.nan)

# 75.3 ms ± 3.48 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  -> main
# 17 ms ± 204 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)    -> PR


# -----------------------------------------------------------------------------------
# case 3: null pyarrow type

arr = pd.array([None] * N, dtype="null[pyarrow]")

%timeit arr.to_numpy("float64", na_value=np.nan)

# 324 ms ± 14.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)     -> main
# 8.58 ms ± 214 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  -> PR
```



Labels:
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley
Add new test for None values on string columns on a Dataframe
State: closed

- [x]  #32218
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Although this issue seems to be gone, i noticed in the comments that a test for this would be appreciated, so this basically checks that Dataframe behaves as it should.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Thanks @GrammatikakisDimitris
DOC: Fix permission denied error in docbuild
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @phofl
WARN: std and var showing RuntimeWarning for ea dtype with one element
State: closed

- [x] closes #52515 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @phofl 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 4d6ca9d74722ab6c74f3e8541110342fcc0a1f35
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #52518: WARN: std and var showing RuntimeWarning for ea dtype with one element'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-52518-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #52518 on branch 2.0.x (WARN: std and var showing RuntimeWarning for ea dtype with one element)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
REF: Introduce enum class InferredType
State: closed

- [x] closes #29868
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the PR. Just noting it would be easier to review this PR if the tests were passing
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
REGR: fix Series construction from dict for subclasses
State: closed

- [ ] closes #52445
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Subclassing
description: Subclassing pandas objects

Comments:
thx @jorisvandenbossche
TST: Test for setting indexed Series of type string[pyarrow] using dataframe loc
State: closed

- [x] closes #52319
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Arrow
description: pyarrow functionality

Comments:
Thanks @NumanIjaz
CI: Uprade mypy min version
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @phofl
ENH: Add more dt property/method support for ArrowDtype(timestamp)
State: closed

- [x] closes #52388 (Replace xxxx with the GitHub issue number)
- [x] closes #51718
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Arrow
description: pyarrow functionality

Comments:
@phofl do you prefer backporting this too?
No real opinion here, I think the string equivalent is used more widely so was better to back port the split pr
Thanks for all the reviews @rok! 
Going to merge this in. Can address any follow ups if needed
PERF: Series.corr/Series.cov for EA dtypes
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

```
import pandas as pd
import numpy as np

N = 1_000_000

ser1 = pd.Series(np.random.randn(N), dtype="Float64")
ser2 = pd.Series(np.random.randn(N), dtype="Float64")

%timeit ser1.corr(ser2)

# 849 ms ± 56.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)    -> main
# 33.7 ms ± 320 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  -> PR

%timeit ser1.cov(ser2)

# 739 ms ± 15.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)    -> main
# 32.9 ms ± 713 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  -> PR
```


Also, aligns the error raised for non-numeric data with that of `DataFrame.corr` and `DataFrame.cov`.


Labels:
name: Performance
description: Memory or execution speed performance
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Thanks @lukemanley
ENH: Implement str.r/split for ArrowDtype
State: closed

- [x] closes #52401 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality

Comments:
Sure moved to 2.0.1
Going to merge. But happy to follow up
BUG: rank incorrect for Float64 with small values
State: closed

- [x] closes #52471 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @phofl
DOC: Fix missing `read_parquet` from list of fns with new keyword arg `dtype_backend`
State: closed

Doc fix for whats new v2.0.0.rst: the `read_parquet` function was missing from the first list of functions with the new `dtype_backend` keyword parameter.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: IO Parquet
description: parquet, feather

Comments:
Thanks @alexmalins
STYLE improve `validate-docstrings` ergonomics
State: closed

Before opening an issue for EX03 errors, I just wanted to improve the usability of `validate_docstrings`

Currently it's kind of hard to tell which lines need to change, especially if there's a long docstring.

Now, here's what it shows for `python scripts/validate_docstrings.py pandas.Series.describe`:
```python
################################################################################
################################## Validation ##################################
################################################################################

3 Errors found for `pandas.Series.describe`:
        flake8 error: line 11, col 54: E231 missing whitespace after ','
        flake8 error: line 11, col 58: E231 missing whitespace after ','
        flake8 error: line 14, col 19: E124 closing bracket does not match visual indentation
```

From trying it out locally, this makes it much easier to tell what needs changing

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @MarcoGorelli 
Arrived late to the party
WEB: Add coiled as sponsor and remove dfine
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Please wait with merging for a day

Rendered the website locally and looks fine


Labels:
name: Web
description: pandas website

Comments:
Would merge this tomorrow if no objections
Add `np.intc` to `_factorizers` in `pd.merge` 
State: closed

- [x] closes #52451 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
Did this fail on main as well? Couldn't reproduce on main yesterday
Did not test on main. I don't have a Windows computer before next week. 

Did you test on Windows? Because I can see on Linux `np.intc == np.int32`.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
I accidentally force-pushed the main into this branch, which was why this PR was closed.  I will open a new PR with the changes, though I will change `Int32Factorizer` to `Int64Factorizer`.
REF: avoid overriding some NDFrame methods
State: closed

The only change in most places is that the docstrings say "Series/DataFrame" instead of just "Series" for the Series method and "DataFrame" in the DataFrame method. 

In fillna the axis keyword says "{0 or 'index'} for Series, {0 or 'index', 1 or 'columns'} for DataFrame" instead of being more specific in each subclass.  Same for fill_axis and broadcast_axis (both of which are deprecated) in align

Labels:
name: Refactor
description: Internal refactoring of code
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @jbrockmendel
BUG: to_timedelta/datetime with numeric ArrowExtensionArray
State: closed

- [ ] closes #52425 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Arrow
description: pyarrow functionality

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b122f804c8bb398b3510967daaddd3bffe2fa176
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #52473: BUG: to_timedelta/datetime with numeric ArrowExtensionArray'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-52473-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #52473 on branch 2.0.x (BUG: to_timedelta/datetime with numeric ArrowExtensionArray)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DEPR: Categorical fastpath
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref #20110

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Categorical
description: Categorical Data Type
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel