BrandonFors/pandas-github-issues huggingface dataset:
CI: fix ci (calamine typing)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

new calamine release is typed 🥳 

Labels:
name: CI
description: Continuous Integration
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
self-merging to fix ci
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 ab2980ba52ae24db72765b3c4ae1d33ac8fad3f5
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57689: CI: fix ci (calamine typing)'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57689-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57689 on branch 2.2.x (CI: fix ci (calamine typing))"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BUG: datetime parsing fails on a leapday
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

No idea what would be a better fix. `dateutil_parse` will substitute parts of the `dt_str` into `default` and then check if it's a valid date.

Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
There is another PR also addressing this with a similar approach: #57674
Thanks - I quickly checked but missed that one. That approach is better.
CLN: Enforce deprecation of axis=None in DataFrame reductions
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: #52042

Labels:
name: Clean
name: DataFrame
description: DataFrame data structure
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Looks good. Just an ASV needs fixing
Thanks @mroeschke - removed the failing benchmark: with `axis=None` it was now computing `std` over invalid dtypes.
Thanks @rhshadrach
BUG: Subtraction fails with matching index of different name and type (pandas-dev#57524)
State: closed

- [x] closes #57524
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: pd.unique(Index) now returns Index as Index.unique
State: closed

Some old tests are adjusted for consistency.

- [x] closes #57043(Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Index
description: Related to the Index class or subclasses

Comments:
Looks like CI failures are not related. Could you please review this PR? @rhshadrach 
Thank you for the review. I push a new commit to address the requests. @rhshadrach 
Thanks @yuanx749
DOC: Extended the documentation for `DataFrame.sort_values()`
State: closed

The current docstring for [`DataFrame.sort_values()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.sort_values.html) is lacking clarity, and also contains links that are not converted to hyperlinks. I extended and rephrased part of the documentation for this method. In particular:

- Added further explanation to compare the single-column vs. multi-column sorting,
- Extended explanation for customized sorting via `key` argument,
- Provided a simplified version of `natsort` example, with added explanation.

Labels:
name: Docs
name: Sorting
description: e.g. sort_index, sort_values

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57678/
The rendered version of the changes here is available [here](https://pandas.pydata.org/preview/pandas-dev/pandas/57678/docs/reference/api/pandas.DataFrame.sort_values.html), in case it's useful.
Thanks @datapythonista for reviewing the update!

Would be happy to fix the issue. Please take a look.

Also thank you for the guidance!
It is indeed my first contribution to a large project, so I have a lot to learn 😄 
@datapythonista could you please let me know if any other modification is needed for this PR?

Thanks :)
Thanks @aallahyar
CI: avoid `guess_datetime_format` failure on 29th of Feburary
State: closed

- [ ] closes #57672 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
Looks reasonable.  I guess it would be a PITA to write a test?
> Looks reasonable. I guess it would be a PITA to write a test?

given that the function calls `datetime.now` from Cython - yes, probably
Looks good. Could you double check this typing issue? https://github.com/pandas-dev/pandas/actions/runs/8108693644/job/22162385490?pr=57674
thanks - that one should be addressed in https://github.com/pandas-dev/pandas/pull/57689
Thanks @MarcoGorelli
CLN: Enforce deprecation of pinning name in SeriesGroupBy.agg
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: #51703

Labels:
name: Groupby
name: Clean
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
@datapythonista - if you tag enforcements of deprecations as "Deprecate", our CI will add it to the list of deprecations to enforce in pandas 4.0. I think most are sticking to "Clean" for the enforcement.
Ah, thanks for the heads up, I forgot the bot.
Thanks @rhshadrach
CLN: More numpy 2 stuff
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: Clean

Comments:
I merged main thinking the CI failures would be gone by now but looks like they are still around. Happy to merge as is given I am fairly confident the failures are unrelated, but will leave it up to you
Merging then, since numpy is blocked on this.

Thanks for the review!
ENH: Report how far off the formatting is
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Not sure this warrants a ~~test change or~~ `whatsnew` entry but I can add it if requested -- just takes the error message and makes it more informative:
```
E   UserWarning: Cell contents too long (54416), truncated to 32767 characters
```

Labels:
name: Enhancement
name: IO Excel
description: read_excel, to_excel
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
Thanks @larsoner!
BUG: interchange protocol with nullable datatypes a non-null validity
State: closed

- [ ] closes #57643 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This deals with pandas nullable dtypes

pyarrow nullables ones will require changes to the PandasBuffer class, so I'll handle those separately

Labels:
name: Interchange
description: Dataframe Interchange Protocol

Comments:
Thanks @MarcoGorelli 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 03717bcc5ae762d8a0ab8d259ca000af66e8ba82
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57665: BUG: interchange protocol with nullable datatypes a non-null validity'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57665-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57665 on branch 2.2.x (BUG: interchange protocol with nullable datatypes a non-null validity)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
FIX: fix default argument of duplicated
State: closed

`df.duplicated(keep=False)`

Should have default `keep` parameter as `False`

It's proper result for `df.duplicated()` to show all duplicates.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Requirements are not satisfied.
Fixing bug_ 57596
State: closed

- [x] closes #57596 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v2.2.1.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
pre-commit.ci autofix
please always add tests and make sure the new tests pass before opening prs, this won't have any effect
CLN: Enforce deprecation of using alias for builtin/NumPy funcs (#57444)
State: closed

* CLN: Enforce deprecation of using alias for builtin/NumPy funcs

* GH# and whatsnew

* Fixup docs

* More tests

* Restore docstring

* Test fixes

* Test fixups

* Test fixes

* Test fixup

* Test fixes

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
This PR contents doesn't seem to match what's described here so closing
CLN: unify error message when parsing datetimes with mixed time zones with utc=False
State: closed

xref #57275, #55793

unified ValueError message when parsing datetimes with mixed time zones with utc=False
so far we get two different ValueError messages:
```
import pandas as pd
from datetime import datetime
import pytz

str_my ="2020-10-26 00:00:00+06:00"
ts = pd.Timestamp("2018-01-01", tz="US/Pacific")
dt = datetime(2020, 1, 1, 18, tzinfo=pytz.timezone("Australia/Melbourne"))

dt1 = pd.to_datetime([ts, str_my])
# ValueError: Mixed timezones detected. pass utc=True in to_datetime or tz='UTC' in DatetimeIndex to convert to a common timezone.
dt2 = pd.to_datetime([str_my, ts])
# ValueError: cannot parse datetimes with mixed time zones unless `utc=True`

dt1 = pd.to_datetime([dt, str_my], utc=False)
# ValueError: Mixed timezones detected. pass utc=True in to_datetime or tz='UTC' in DatetimeIndex to convert to a common timezone.
dt2 = pd.to_datetime([str_my, dt], utc=False)
# ValueError: cannot parse datetimes with mixed time zones unless `utc=True`
```

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Clean

Comments:
Thanks @natmokval
TST: fix test_complibs on mac
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: OS X
description: Related to Mac OS & hardware issues (M1)

Comments:
Thanks @jbrockmendel
DOC: RT03 fix for read_sql_query, read_feather
State: closed

- [x] xref #57416  (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @YashpalAhlawat
PERF: Return RangeIndex from RangeIndex.reindex when possible
State: closed

Discovered in https://github.com/pandas-dev/pandas/pull/57441

Labels:
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
I think something went wrong with git here, feels like this have unrelated changes, right?
As of 1a122350fd9acab946099d3a3e990b8a2af9b661, I think the diff against main should be minimal to the changes now. (In 370d20cb0ca46009c439e1dc9d6e805196df5271 I merged upstream incorrectly)
BUG: Change output returned by _LocIndexer._convert_to_indexer to return key with greater resolution value
State: closed

Resolution info:

**Resolution.RESO_SEC - 3 for seconds**
**Resolution.RESO_MIN - 4 for minute**

Observation:
With the previous code, when the current entry (**resolution: 4, entry: '2024-02-24 10:08'**) has a greater resolution value than the smallest amongst previous entries (in this case **resolution: 3, entry: '2024-02-24 10:2:30'**), it is shown that the key is said to exist within the list of keys within the Series even though it shouldn't. This does not happen when a smaller or equal resolution value entry is inserted into the Series.

![Screenshot from 2024-02-26 13-14-10](https://github.com/pandas-dev/pandas/assets/122815453/0ec3fc4d-4831-47ac-9c19-c8b8fdbbaea9)

Thought process:
With the previous code, if the current entry's resolution value is greater than the smallest resolution value amongst the previous entries, the *_LocIndexer._convert_to_indexer* method will only return an empty list (output of the method *labels.get_loc(key)*), which might be causing entries with larger resolution values not being inserted into the *Series* correctly.

Potential solution:
I have changed the method *_LocIndexer._convert_to_indexer* to return the key ('2024-02-24 10:08') instead of an empty list when the resolution value is greater than the smallest resolution value.


- [x] closes #57596 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.1.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Create median_absolute_deviation
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Needs Discussion
description: Requires discussion from core team before further action

Comments:
Thanks for the contribution @VasudevanS1906. I'm going to close this PR. You can surely open an issue to have a discussion on whether this is a function that we want to add to pandas or not.

In the case that people agree to add this, you'll have to implement this as all other methods, add tests, add documentation...