BrandonFors/pandas-github-issues huggingface dataset:
Remove enter on begin of warnign message
State: closed

Closes #57082

With current code, it is not possible to filter pandas warning about PyArrow that leads to crash of test suite with the rule to treat warning as error. 

It looks like the source of the problem is that `warnigns` library compile pattern with only IgnoreCase flag.

https://github.com/python/cpython/blob/d1b031cc58516e1aba823fd613528417a996f50d/Lib/warnings.py#L160

And then is using `match` instead of `search` so it matches to initial line. 

With this change, it this warning will be possible to ignore this using:

```
"ignore:Pyarrow will become a required dependency of pandas in the next major release of pandas.*:DeprecationWarning"
```

In napari we meet it here: https://github.com/napari/napari/pull/6609

cPython issue about problem with empty line https://github.com/python/cpython/issues/114426

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Warnings
description: Warnings that appear or should be added to pandas
name: Arrow
description: pyarrow functionality

Comments:
Numpy failure looks unrelated 
Let's hold on this for now, since we might just revert the deprecation entirely for 2.2.1.
Please consider my comment for the PR, thanks.
https://github.com/pandas-dev/pandas/issues/57082#issuecomment-1935291850

Thanks for the start here but the warning ended up getting removed in https://github.com/pandas-dev/pandas/pull/57556 so closing
Dims deprecate sizes test fix
State: closed

- [.] closes #57000 
- [.] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [.] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [.] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [.] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Looks like issue is already resolved.
PERF: StringEngine for string dtype indexing ops
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.0.rst` file if fixing a bug or adding a new feature.

Seems to be about 2x faster than `ObjectEngine` for indexing ops that use the hashmap:

```
import pandas as pd

N = 100_000
dtype = "string[pyarrow_numpy]"

strings = [f"i-{i}" for i in range(N)]

idx1 = pd.Index(strings[10:], dtype=dtype)
idx2 = pd.Index(strings[:-10], dtype=dtype)

%timeit idx1.get_indexer_for(idx2)

# 52.6 ms ± 834 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  -> main
# 25 ms ± 790 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)    -> PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Strings
description: String extension data type and string data
name: Index
description: Related to the Index class or subclasses

Comments:
Thanks @lukemanley
DEP: Loosely pin Cython to 3.0
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
Thanks @rhshadrach 
Ah sorry should have looked at the failing check more closely, could you fix the pinning in the `requirements-dev.txt`? https://github.com/pandas-dev/pandas/actions/runs/7731187330/job/21078258557
Yea - just noticed it. On it.
PERF: MultiIndex.equals for equal length indexes
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.0.rst` file if fixing a bug or adding a new feature.

Avoids densifying the levels of the multiindex to check for equality.

```
from asv_bench.benchmarks.multiindex_object import Equals

b = Equals()
b.setup()
%timeit b.time_equals_deepcopy()

# 56.4 ms ± 1.03 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)   -> main
# 3.77 ms ± 58.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)  -> PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: MultiIndex

Comments:
Thanks @lukemanley
DOC: fix EX03 errors in docstrings - pandas.errors: SettingWithCopyWarning, SpecificationError, UndefinedVariableError
State: closed

All EX03 Errors resolved in the following cases:

1. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.SettingWithCopyWarning
2. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.SpecificationError
3. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.UndefinedVariableError

OUTPUT:

1. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.SettingWithCopyWarning
```
################################################################################
################################## Validation ##################################
################################################################################

1 Errors found for `pandas.errors.SettingWithCopyWarning`:
	SA01	See Also section not found
```

2. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.SpecificationError
```
################################################################################
################################## Validation ##################################
################################################################################

1 Errors found for `pandas.errors.SpecificationError`:
	SA01	See Also section not found
```

3. scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.errors.UndefinedVariableError
```
################################################################################
################################## Validation ##################################
################################################################################

2 Errors found for `pandas.errors.UndefinedVariableError`:
	PR01	Parameters {'name', 'is_local'} not documented
	SA01	See Also section not found
```

- [ ] xref https://github.com/pandas-dev/pandas/issues/56804
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @jordan-d-murphy
fix unsigned // negative integral value for pyarrow types
State: closed

>>> a = pd.Series([7], dtype="uint8[pyarrow]")
>>> b = pd.Series([-4], dtype="int8[pyarrow]")
>>> a // b
pyarrow.lib.ArrowInvalid: Integer value -2 not in range: 0 to 255
>>>

But it works with numpy types:

>>> a = pd.Series([7], dtype="uint8")
>>> b = pd.Series([-4], dtype="int8")
>>> a // b
0   -2
dtype: int16

Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks for the PR but I am -1 to doing this. The pyarrow result makes sense given it follows usual C implicit type promotion rules but with added safety

I don't think we should try to make other libraries adhere to the NumPy type promotion rules
WEB: Fix typo in dropdown page
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Web
description: pandas website

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 21b3906a39eca5ba8220d5e34ca1f204a68a5adb
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #56986: WEB: Fix typo in dropdown page'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-56986-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #56986 on branch 2.2.x (WEB: Fix typo in dropdown page)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@meeseeksdev backport 2.2.x
There is no need to backport anything under `web/` including the `versions.json`file. Only the version in `main` is used.
> There is no need to backport anything under `web/` including the `versions.json`file. Only the version in `main` is used.

Gotcha, thanks for clarifying.
raise error on unsafe decimal parse with pyarrow types
State: closed

- [ ] closes #56984(Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Arrow
description: pyarrow functionality

Comments:
Thanks @rohanjain101
DOC: Add release notes for 2.2.1
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @lithomas1 

Labels:
name: Docs

Comments:
thanks @rhshadrach.
ENH: Adds ability to store tooltips as title attribute through pandas styler
State: closed

- [x] closes #56605 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Styler
description: conditional formatting using DataFrame.style

Comments:
@attack68 
When I run ruff locally, on the files I changed, there are no changes made.


edit: Too fast to comment.
Getting back to this tonight, will address all comments.
Do I have to worry about that Numpy failure?
No I dont think so it looks unrelated. And if on the off chance it is not it is probably worth fixing in a follow up PR.
If I merge main one more time, am I good?
Tests are passing and dangling 'comma' changes have gone. I think this is ready now.
Thanks @Delengowski!
WEB: Add version 2.2 to the dropdown
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Web
description: pandas website

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/56980/
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/56980/
self merging for now.

I think the docs build the dropdown from the versions.json on the website - so this has to be merged in first for us to see the results.
Idx get level values list
State: closed

- [x] will close #56917 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Note: if this PR looks fine, I can remove the old call `_get_level_values` and replace `_get_level_values(level) for level in levels` by `self._get_level_values(levels)`, which should enhance readability

Labels:

Comments:
pre-commit.ci autofix
pre-commit.ci autofix
Thanks for the PR, but we usually require discussion and support from core team members before proceeding with a PR. Going to close to now pending acceptance from the core development team but happy to reopen if there's acceptance.
Added a test that breaks over Series.rank with nullable ints
State: closed



- [ ] will eventually close #56976
- [ ] [Tests added and passed] Added a test for `pandas.Series.rank`

I'll have to check the rest as soon as the Issue is closed
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST: test for warnings using pytest.warns()
State: closed

Fix tests that used `pytest.raises()` to check for warnings, to use `pytest.warns()` instead.  This is more correct, and it fixes running the test suite without `-Werror`.  This is how Gentoo runs it, to avoid sudden test failures due to new deprecation warnings from upgraded dependencies.

- [ ] closes #56716 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
pre-commit.ci autofix
Thanks @mgorny 
Thanks! And sorry for not running pre-commit in the morning, as I've updated the PR just before going out.
Fix ci not raises on error
State: closed

I fixed the bug that was fixed in https://github.com/pandas-dev/pandas/pull/56855 again, also added a comment to make it more visible

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: CI
description: Continuous Integration

Comments:
Thanks @tqa236, very nice. I merged the PR that was again introducing the backslash. Great idea to have the comment.
BUG: DataFrame.drop_duplicates method fails when a column with a list…
State: closed

resolved #56784 in the file pandas/core/frame.py

Labels:
name: duplicated
description: duplicated, drop_duplicates
name: Stale

Comments:
Thank you so much for your suggestions @rhshadrach I'll make changes accordingly.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
This PR has gone stale. @manlattan - if you are interested in continuing, just comment here and we're happy to reopen!
DOC: fixed Ex03 errors in docstrings: pandas.io.formats.style.Styler.to_latex and pandas.read_parquet
State: closed

**PR SUMMARY**

Checked if validation docstrings passes for:

- [x] pandas.io.formats.style.Styler.to_latex
- [x] pandas.read_parquet

OUTPUT:
1. python scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.io.formats.style.Styler.to_latex
```
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.io.formats.style.Styler.to_latex" correct. :)
```

2. python scripts/validate_docstrings.py --format=actions --errors=EX03 pandas.read_parquet
```
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.read_parquet" correct. :)
```

**PR CHECKLIST**

- [ ] xref #56804 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @tiffanyxiao