BrandonFors/pandas-github-issues huggingface dataset:
PERF: ArrowExtensionArray.to_numpy(dtype=object)
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

redo of #51227 which had to be reverted. 

cc @mroeschke @phofl 

Labels:
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley
ENH: Index set operations with sort=True
State: closed

- [x] closes #25151 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
Thanks @jbrockmendel
PERF: remove categories
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.

cc @phofl, per https://github.com/pandas-dev/pandas/pull/50857#issuecomment-1426898195

I dont think we should revert #50857 as that fixed a few bugs. I think this gets most of the perf back though.

No whatsnew as this was a slowdown on main only. 

```
from asv_bench.benchmarks.categoricals import RemoveCategories

b = RemoveCategories()
b.setup()

%timeit b.time_remove_categories()

# 66.4 ms ± 1.92 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  <- main
# 45.2 ms ± 1.57 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
thx @lukemanley for following up so quickly
looks like we got it back to the original performance, thx
add typehint for None pct_change
State: closed

- [x ] closes #51269 
- [x ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints)

I've added a typehint for None. It looks like supplying `None` as an argument to `fill_method` is tested for dataframes under `pandas/tests/series/methods`.

Let me know if another test should be written, e.g for dataframes.

Labels:
name: Docs
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> It looks like supplying None as an argument to fill_method is tested for dataframes under pandas/tests/series/methods.

you're right, thanks

agree that it would be good to document what passing `None` does
Would you like me to write docs on this branch / PR, or should that be done on another?
PERF: Fix performance regression in get_loc of IntervalIndex
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


caused in https://github.com/pandas-dev/pandas/commit/184e1674

No need to call validate here

https://asv-runner.github.io/asv-collection/pandas/#indexing.IntervalIndexing.time_loc_list

Labels:
name: Performance
description: Memory or execution speed performance
name: Interval
description: Interval data type

Comments:
pandas/core/arrays/interval.py:729: error: Argument 2 to "_simple_new" of "IntervalArray" has incompatible type "Union[Period, Timestamp, Timedelta, NaTType, DatetimeArray, TimedeltaArray, ndarray[Any, Any]]"; expected "Union[Union[DatetimeArray, TimedeltaArray], ndarray[Any, Any]]"  [arg-type]
ENH: support addition with pyarrow string dtypes
State: closed

@mroeschke it wouldn't surprise me if you say you prefer to explicitly _not_ support addition this way, but this seems like a pretty big use case, and the _libs.ops functions are a pretty great fit.

Labels:
name: Arrow
description: pyarrow functionality

Comments:
```
=================================== FAILURES ===================================
________________________________ test_str_join _________________________________
[gw1] linux -- Python 3.8.16 /home/runner/micromamba/envs/test/bin/python3.8

    @pytest.mark.xfail(
        reason="TODO: StringMethods._validate should support Arrow list types",
        raises=AttributeError,
    )
    def test_str_join():
        ser = pd.Series(ArrowExtensionArray(pa.array([list("abc"), list("123"), None])))
>       result = ser.str.join("=")
```
raises should change to a `NotImplementedError` it appears (which is better IMO)
Nevermind it looks like this was fixed in https://github.com/pandas-dev/pandas/pull/51441
```
pandas/core/arrays/arrow/array.py:472: error: List item 0 has incompatible type "bytes"; expected "str"  [list-item]
```
greenish
Thanks @jbrockmendel
ENH: Add lazy copy to where
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
It seems there is an actual relevant test failure with CoW:

FAILED pandas/tests/frame/indexing/test_where.py::test_where_copies_with_noop[Series] - AssertionError: Series are different
Will rebase after the other pr is merged

#51403
thanks @phofl
BUG: DatetimeArray+DateOffset result unit
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
ENH: support td64/dt64 in GroupBy.std
State: closed

- [x] closes #48481 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Timedelta
description: Timedelta data type
name: Reduction Operations
description: sum, mean, min, max, etc.
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Thanks @jbrockmendel
BUG : Add Deprecation FutureWarning for parse function call
State: closed

- [ ] closes #50953 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.




Labels:

Comments:
@rhshadrach Can you please help me with this Doc string validation error?

Error: /home/runner/work/pandas/pandas/pandas/io/excel/_base.py:1530:GL09:pandas.ExcelFile.parse:Deprecation warning should precede extended summary

Where should I put the deprecation message in the Doc string?
I think the linter is thinking you are deprecating the method; even though it's somewhat of a false positive, it does seem better to have the deprecation warning above the extended summary, i.e. 4 lines above where it is now.
DOC: clarify "inplace"-ness of DataFrame.setitem
State: closed

- [x] closes #51300 
- [NA] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [NA] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [NA] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
thx @loicdiridollou 


lets backport, this is nice to have for CoW
ENH: Avoid copy when possible in merge
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Technically this would change the behavior for the non CoW case with copy=False as well

Labels:
name: Copy / view semantics

Comments:
Failure in the CoW build is the unrelated test_file_descriptor_leak
BLD: Fix regex in arm64 wheel builders
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
They are still building here, probably still buggy?
> They are still building here, probably still buggy?

They still go, but exit early. There's no way to skip them conditionally once the job has started.
Ah so there is no way to skip them completely like gha?
> Ah so there is no way to skip them completely like gha?

Unfortunately not. The only thing that I can skip on (before the job starts) is branches.
Looks like this works so I'm just going to put it in. 
Thx
BUG: non-64-bit numeric dtypes should raise in IntervalDtype constructor
State: closed

- [x] closes #45412
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.



Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Regression
description: Functionality that used to work in a prior pandas version
name: Interval
description: Interval data type
name: Stale

Comments:
Ping. @jbrockmendel, could you take a look?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Looks like this has gone stale. Feel free to reopen if/when you're interested in picking this back up
BUG: to_datetime with both origin and unit
State: closed

- [x] closes #42624 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
ENH: EA._hash_pandas_object
State: closed

- [x] closes #51108 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: hashing
description: hash_pandas_object

Comments:
@MarcoGorelli any thoughts here?
I'll take a look this week, thanks for the ping
@phofl thoughts?
Also, does it need adding here

https://github.com/pandas-dev/pandas/blob/089aa0cae6e66814a98749aba6927c9267182457/pandas/core/arrays/base.py#L117-L146  

?
> Also, does it need adding here

Yes, will update
the docs job is failing , showing

```
2023-02-27T22:46:43.2184483Z /home/runner/work/pandas/pandas/pandas/core/arrays/base.py:docstring of pandas.core.arrays.base.ExtensionArray:116: WARNING: autosummary: stub file not found 'pandas.api.extensions.ExtensionArray._hash_pandas_object'. Check your autosummary_generate setting.
```

seems related?
its complaining about No Examples section found?  how do i tell it to ignore that?
the ignore list is in code_checks.sh
thx @jbrockmendel
ENH: Use MaskedEngine for numeric pyarrow dtypes
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

```
idx = pd.Index(list(range(1_000_000)), dtype="int64[pyarrow]")
idxer = list(range(1, 500_000, 10))

pr 
%timeit idx.get_indexer(idxer)
4.85 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

main
%timeit idx.get_indexer(idxer)
16.9 ms ± 116 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

```

cc @mroeschke Is this roughly what you had in mind?

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
Nice! Yeah this is what I had in mind. 
Added a whatsnew.

We should probably figure out a better way of converting to numpy when ``na_value`` is specified and compatible with the associated NumPy dtype
@mroeschke do we want to get this in before the rc?
CI/TST: Enable -W error:::pandas in pyproject.toml
State: closed

- [ ] closes #48553 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
needs a rebase
Rebased. Added back the `ResourceWarning` check I removed in https://github.com/pandas-dev/pandas/pull/51443. Appears `tm.assert_produces_warning` still catches these
thx @mroeschke
REF: avoid try/except in _gotitem
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Resample
description: resample method
name: Window
description: rolling, ewma, expanding

Comments:
Thanks @jbrockmendel
DOC: fix Series.name docstring, closes #42550
State: closed

- [x] closes #42550 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @jbrockmendel
API: ArrowDtype.type
State: closed

cc @mroeschke we discussed this one of the pyarrow testing PRs.

Still has some failing `pandas/tests/extension/test_arrow.py::TestBaseInterface::test_contains` tests where eyeballs would be welcome.

ATM this returns Timestamp/Timedelta when the pyarrow unit is "ns" and pydatetime/pydatetime otherwise.  We could reasonably update that to be always-Timestamp/Timedelta.

Labels:
name: Arrow
description: pyarrow functionality

Comments:
So this is breaking the `float[pyarrow]` test_contains test because `EA.__contains__` does

```
        if is_scalar(item) and isna(item):
            if not self._can_hold_na:
                return False
            elif item is self.dtype.na_value or isinstance(item, self.dtype.type):
                return self._hasna
```

In the relevant case item is np.nan.  This PR makes it so that the `isinstance(item, self.dtype.type)` check is True, the pyarrow (and nullable) dont want to treat np.nan as isna.  One option is to override `ArrowArray.__contains__`.  More generally, I think we're going to need something like EADtype._is_valid_na_for that `missing.is_valid_na_for_dtype` can dispatch to.
Thanks @jbrockmendel
BUG: idxmax and idxmin raising for all na ea series
State: closed

- [x] closes #51276 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Not sure if NA or NaN makes more sense. I think we have a couple of ops where we return nan even for ea dtypes when the result is a scalar

Labels:
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
xref #33941
why closing?  i was looking forward to getting #33941 resolved (i think it has implications for fixing some of our groupby xfails)
Did not look like there was consensus and we had tests covering the changed behavior. But can revisit if you think we should change it
Let's try to discuss the desired behavior here on the next dev call?  I think this is an area where we need to Just Make A Decision.