BrandonFors/pandas-github-issues huggingface dataset:
BUG: mask/where raising for mixed dtype frame and no other
State: closed

- [x] closes #51685 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jbrockmendel @mroeschke 

Based on the commits causing the error, I don't think this was ever intended to raise?

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Thanks @phofl
BUG: ArrowExtensionArray logical ops raising KeyError
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

fixes the following:

```
import pandas as pd

ser = pd.Series([True, False, True], dtype="boolean[pyarrow]")

ser & True  # KeyError: 'and_'
ser | True  # KeyError: 'or_'

ser & ser   # KeyError: 'and_'
ser | ser   # KeyError: 'or_'
```

Labels:
name: Bug
name: Arrow
description: pyarrow functionality

Comments:
cc @mroeschke 
Sounds good to me
@lukemanley could you move the whatsnew?
> @lukemanley could you move the whatsnew?

moved to 2.0, thanks
thx @lukemanley
STYLE start enabling TCH
State: closed

This one should actually make a difference to performance, https://github.com/snok/flake8-type-checking#examples

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
xref https://github.com/pandas-dev/pandas/pull/47968 (moved non-`pandas._typing` imports into the `TYPE_CHECKING` block)
(Unrelated: ruff implements more and more of flake8-pyi, could potentially avoid flake8-pyi in a few weeks/months)
> xref https://github.com/pandas-dev/pandas/pull/47968 (moved non-pandas._typing imports into the TYPE_CHECKING block)

yeah this seems like the same thing

agree on keeping `_typing` excluded, but we can start enabling it in other places - merging then, thanks for your review!
> agree on keeping `_typing` excluded, but we can start enabling it in other places - merging then, thanks for your review!

Personally, I would be fine moving even `pandas._typing` in the `TYPE_CHECKING` block. It might not be strictly needed but otherwise, we would need to add too many exceptions to enable TCH.
BUG: concat sorting when sort=False for all dti
State: closed

- [ ] closes #51210 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

lets see if this passes everything

Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
Looks fine, haven't looked closely.  Bug me about it next week.
This one isn't urgent
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Feel free to reopen if/when you have the time to circle back
Updates Hamilton references
State: closed

This updates a reference on the ecosystem page. We recently moved the repository and updated what is used for documentation.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
thanks!
BUG: Prevent erroring out when comparing unordered categories with different permutations
State: closed

- [x] closes #51543
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Previously the hash of the Categorical `dtype` was used for comparing to see if the two `Categorical` instances could be considered equivalent. Even if the categories are the same and the only difference is the order this doesn't always result in the same hash, for example when the categories consist of a combination of string and numeric values.

Labels:
name: Stale

Comments:
Hello, @ParthivNaresh , how are things? Looks like you almost finished this, wondering if everything is okay.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Use union to combine indexes, matching behaivor when two indexes have different index types
State: closed

- [ ] closes #51666
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
This would need tests
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: ArrowDtype().type raising for fixed size pa binary
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

should backport I guess

Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks @phofl
ENH: Support na_position for sort_index and sortlevel
State: closed

- [x] closes #51612 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think the special casing in sort level was done for performance reason. Avoiding the use of categorical solves this. Both branches were equally fast 

```
%timeit midx.sortlevel(level=[0, 1], ascending=[True, True])
58.8 ms ± 674 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
%timeit midx.sortlevel(level=[0, 1], ascending=True)
60.7 ms ± 278 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
```

Labels:
name: MultiIndex

Comments:
Thanks @phofl
ENH: Add CoW mechanism to replace_regex
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This was actually buggy as well

Labels:
name: replace
description: replace method
name: Copy / view semantics

Comments:
thanks @phofl
WEB: Added stying for pre element within the codeblock divs
State: closed

- [x] closes ##51657 (Replace xxxx with the GitHub issue number)
- ENH: Enhancement, adding styling (padding and border) for code blocks


Labels:
name: Web
description: pandas website

Comments:
I was reviewing the stylesheets. Would you rather have me add the new styling to codehilite.css since it's effecting the codehilite class?

I ended up moving the css code to codehilite.css 
> I ended up moving the css code to codehilite.css

Great, probably better. I'll someone else from the core dev team have a second look and merge, otherwise I'll merge myself in couple of days. But this should be good to go.
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/51665/
> Website preview of this PR available at: https://pandas.pydata.org/preview/51665/

An example of where the changes can be seen is here: https://pandas.pydata.org/preview/51665/pdeps/0004-consistent-to-datetime-parsing.html
Thanks @dsamuel27, very nice!
ENH: Improve replace lazy copy handling
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: replace
description: replace method
name: Copy / view semantics

Comments:
Can you consolidate the inplace replace with no refs and inplace replace with refs tests by parametrizing?

Would make it a little easier to see implemented so far/missing.

done
WEB: Sort and fix link to under discussions PDEPs
State: closed

The links to the under discussion PDEPs in https://pandas.pydata.org/about/roadmap.html point to the API page, not the html page, fixing. Also, sorting the list by title.

Labels:
name: Web
description: pandas website

Comments:
> I guess sorting by title is better than the current status, but wouldn't this place PDEP-10 before PDEP-9?

Yes, you're correct. I realized when I finished implementing, but I didn't want to implement a more complex sorting using regex here, since this was originally to fix the links, and also we can consider other options like using `PDEP-0001`... (we're doing that with the file names). I'd personally merge this first, so the links are working, but happy to have the discussion here too.
thx @datapythonista
POC: implement __array_function__
State: closed

Some day I hope to merge something like this, but for now it is mostly a demonstration for the numpy devs (cc @seberg not sure who else to ping) for how difficult it is to implement `__array_function__` correctly.  The "ask" is for something like `_array_func_fallback` to be implemented upstream (xref similar request https://github.com/numpy/numpy/issues/20044)


Labels:
name: Mothballed
description: Temporarily-closed PR the author plans to return to

Comments:
Yes, one of the things that uarray was right to try to address.  It might be good to have a clean way to traverse/replace dispatched arguments.

Alternatively, a `func._array_implementation()`, which does that same thing (calling `asarray()`) but maybe it doesn't need to be public API.  Since NumPy the numpy implementation tries to be subclass/arbitrary object friendly too often that it is a bit hard to say that you just want the array operation to happen.

Will ping @shoyer again, overall I agree that it would be nice to have something.  Hopefully without too much complexity.
Mothballing.  Discussion in https://github.com/numpy/numpy/issues/20044 makes me pessimistic this is feasible.
ENH: Add na_position support to midx.sort_values
State: closed

- [ ] xref #51612 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

First step, to close the issue we have to add the keyword to sort level as well

Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: MultiIndex

Comments:
Thanks @phofl
EHN: allow for to_sql `multi` method with oracle backend
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Any chance that we can add tests for this? 
I'd gladly write tests, but I have no idea how. I can try mocking a SQLAlchemy db connection with an oracle back-end and then checking if the generated SQL is valid. Would that be OK? I presume the test would go somewhere in [here](https://github.com/pandas-dev/pandas/blob/main/pandas/tests/io/test_sql.py)?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
@mroeschke I tried my best to address your question and merged in the main. Could we reopen, please?
Thanks @jacadzaca
ENH: add na_action to Categorical.map & CategoricalIndex.map
State: closed

- [x] closes #44279
- [x] closes #22527
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.



Labels:
name: Categorical
description: Categorical Data Type

Comments:
I've updated again. I noticed that the implicit default for `na_action` is `"ignore"` for `Categorical.map`, while it's `None` generally. This is a bit of an issue because e.g. `Series.map` has a default of `None` and that can't be changed, so there's some mismatch when calling the `Categorical` directly compared to doing it through `Series.map`.

So I've set the default value of `na_action="ignore"` in `Categorical.map` as being deprecated and set to be changed in the next version.
I’ve simplified the implementation.
Ping.
Thanks @topper-123
API: ArrowExtensionArray._cmp_method to return pyarrow.bool_ type
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.

Per https://github.com/pandas-dev/pandas/issues/51462#issuecomment-1442527928, updating `ArrowExtensionArray._cmp_method` to return `boolean[pyarrow]` dtype instead of `boolean`. 

Labels:
name: Arrow
description: pyarrow functionality

Comments:
> the min versions failure looks legit?

yep. just pushed a fix. I think the failure is related to the `deepcopy` you added in https://github.com/pandas-dev/pandas/pull/50984

It looks like that test combined with the deepcopy was hitting this bug in older versions of numpy: https://github.com/numpy/numpy/issues/21883

I think this will work now. What is the reason to use deepcopy instead of `ndarray.copy` there? It can be very slow for an object dtype array:

```
import numpy as np
from copy import deepcopy

arr = np.arange(1_000_000, dtype=object)

%timeit arr.copy()
%timeit deepcopy(arr)

# 9.83 ms ± 757 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
# 298 ms ± 4.82 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)    
```
@mroeschke argued back then that this would cover eas without a copy method, but if this is slow, then I'd say we can switch to copy
Fine to switch to `copy()`
Thanks @lukemanley 
@mroeschke thoughts on back porting? This would fix #51980
Sure, +1 to backport
@meeseeksdev backport 2.0.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b3d18af7d7ae2c454b9dcef0f2c0c2ae89bea047
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #51643: API: ArrowExtensionArray._cmp_method to return pyarrow.bool_ type'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-51643-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #51643 on branch 2.0.x (API: ArrowExtensionArray._cmp_method to return pyarrow.bool_ type)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
COMPAT: Update code for latest Cython 3 beta
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
OK, this is ready, but needs a Cython bump to get the cpow directive added in 0.29.33 (best done separately).

EDIT: All done.

Going to put this in for now to get rid of red CI.

If there's any further comments, I can take care of it in a followup.