BrandonFors/pandas-github-issues huggingface dataset:
DEPR: Make FutureWarning into DeprecationWarning for groupby.apply
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: https://github.com/pandas-dev/pandas/pull/54950#issuecomment-1896155898

cc @jorisvandenbossche @phofl 

In the event we don't release a 2.3, I am okay waiting to enforce this deprecation until 4.0. The only downside I see of doing so is having users keep the shim `include_groups` for longer.

Labels:
name: Groupby
name: Deprecate
description: Functionality to remove in pandas
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
We should get this in for 2.2 right?
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 24f61394e3b88d296971ac43d5937f6c11afca42
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #56952: DEPR: Make FutureWarning into DeprecationWarning for groupby.apply'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-56952-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #56952 on branch 2.2.x (DEPR: Make FutureWarning into DeprecationWarning for groupby.apply)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
thanks @rhshadrach.
CI: avoid FutureWarnings in to_xarray tests
State: closed

FutureWarning: The return type of `Dataset.dims` will be changed to return a set of dimension names in future, in order to be more consistent with `DataArray.dims`. To access a mapping from dimension names to lengths, please use `Dataset.sizes`.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: CI
description: Continuous Integration

Comments:
thanks @lukemanley
BUG: DataFrame.join inconsistently setting result index name
State: closed

- [x] closes #55815
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @lukemanley 
I discovered that there's a `get_op_result_name`. Looks like it's mostly used numerical ops/combine as of now. Maybe we should be using that here
STY: Enable B904
State: closed

I think most of the fixes should be straightforward. For a few complex cases, I think it's reasonable to `noqa` them to allow us to enable a global check to the rest of the codebase.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
All tests passed after I ... added more info to debug. Not sure if I'm just unlucky with the flaky test or if there's a subtle error the test suite can't capture. 

The failed test in the previous commit is `test_dti_constructor_with_non_nano_now_today` because the diff surpasses the tolerance
DOC: Remove inconsistency with timestamp data in describe() method docs
State: closed

- [x] closes #56918
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @alpakpinar
BUG: Improve MultiIndex label rename checks, docs and tests
State: closed

- [x] closes #55169 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Labels:
name: Bug
name: MultiIndex
name: Stale

Comments:
Actually, I think I need to write an additional test for when `errors="raise"` and a rename is successful against a `MultiIndex`. That fails without this patch because of the inconsistency between the check and transform logic 
Hi @phofl , @mroeschke,
Would you be able to help review this PR?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Hi @mroeschke,
I've updated my branch to resolve merge conflicts. The new tests are still passing on my machine.
Could you kindly reopen this PR?
It appears I cannot reopen this PR as there was a force push. Would be happy to have a new PR with your changes
TST: Add test groupby transform no column
State: closed

- [X] closes #37093 
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Needs Tests
description: Unit test(s) needed to prevent regressions
name: Stale
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
> Thanks for the PR!

Thanks for your review and appreciate you updating the PR title, I will make a note of the format for next time.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Document some more categorical methods
State: closed

- [x] closes #56927
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Opening as a draft to make sure it renders


Labels:
name: Docs
name: Categorical
description: Categorical Data Type

Comments:
I've added references to these methods in `doc/source/reference/arrays.rst` so that pages get generated for the methods. Tbh I didn't know that there was a separate location for documenting the class here.

I don't know how much needs writing about the methods here since they all have docstrings.
Thanks @ivirshup
BUG: fix `groupby.quantile` inconsistency when interpolation='nearest'
State: closed

I removed the xfail mark in previous tests. Do we need to add more tests?
Under the hood, `np.quantile` uses `np.round` for nearest interpolation, which rounds to the even idx when exactly halfway.

- [x] closes #47942 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: quantile
description: quantile method

Comments:
Thanks @yuanx749
REGR: DatetimeTZDtype __from_arrow__ interprets UTC values as wall time
State: closed

- [ ] closes #56775  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Arrow
description: pyarrow functionality

Comments:
Agreed that the comment can be further tackled in a follow-up, so let's get this in.
Thanks @lithomas1!
ENH: Allow performance warnings to be disabled
State: closed

- [x] closes #56920 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

~This still needs tests - I plan to add tests for disabling the warnings wherever we currently hit a PerformanceWarning in the test suite. But I'd like to get feedback on the linked issue first.~

Tests added.

Labels:
name: Enhancement
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
For the test fixture, I wanted to be able to pass it directly to `tm.assert_produces_warning` instead of having the fixture return True/False and then have lines like

    warn = PerformanceWarning if uses_performance_warning else False
    with tm.assert_produces_warning(warn):
        ...
@mroeschke - should be good to merge.
Thanks @rhshadrach 
Late feedback, but was thinking about this in the context of https://github.com/pandas-dev/pandas/issues/58581, but it would make sense to change the default to something like `None`? This default would mean that users see those performance warnings that we decided to turn on by default. More specifically, that would allow us to add new performance warnings that are _not_ shown by default (in contrast to existing ones), while still giving users a way to turn them on.


That behavior sounds like an improvement to me, but I'd still prefer #55385 over it. Maybe discuss this alternative there? Would be good to have a list of performance warnings we'd want on by default (I'd guess we'd start with whatever warnings we have now, but that might not be the same as want we want to end up with).
PERF: DataFrame.join with how=left|right and sort=True 
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.3.0.rst` file if fixing a bug or adding a new feature.

```
import pandas as pd

data = [f"i-{i}" for i in range(200_000)]
idx1 = pd.Index(data, dtype="string[pyarrow_numpy]")
idx2 = pd.Index(data[1:], dtype="string[pyarrow_numpy]")

%timeit idx1.join(idx2, how="left", sort=True)

# 330 ms ± 6.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)     -> main
# 182 ms ± 2.68 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  -> PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @lukemanley
read_csv: adding "round_trip" to the "float_precision" accepted literal types
State: closed



- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Will update the checklit above.

Labels:
name: IO CSV
description: read_csv, to_csv
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @raffienficiaud
Autofix ruf010
State: closed

As now `ruff` has an autofix for `RUF010`, I enable the rule.

I separate the PR into 4 commits with the 4 actions I performed, you can review them one by one for a better diff.
- Autofix `RUF010`
- `ruff format`
- Final manual format
- Remove old "unwanted pattern" rule in `pre-commit`

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Is this rule conflicted with `pandas` style guide? If that's the case, we can move it to "Intentionally disabled" rather than "TODO: Enable gradually"
I would be okay enabling `RUF010` and having it apply everywhere
@mroeschke Thanks, I removed the old rule in the last commit. `pre-commit` is green again now.
Thanks @tqa236
DEPR: lowercase freqs 'ye', 'qe', etc. raise a ValueError
State: closed

xref #56346, #56847, #9586

Using lowercase frequencies such as ‘y’, ‘ye’, ‘q’, ‘qe’, etc. raise a`ValueError` on main, though it worked before. Using those was never documented, but if people are use these lowercase frequencies, we should keep them work.

For example:
```
>>> pd.period_range("2020-01-01", "2020-08-01", freq="y")
ValueError: Invalid frequency: y, failed to parse with error message: ValueError("Invalid frequency: Y, failed to parse with error message: KeyError('Y')")
```
```
>>> pd.date_range("1/1/2000", periods=4, freq="qe")
ValueError: Invalid frequency: qe, failed to parse with error message: ValueError("Invalid frequency: qe, failed to parse with error message: KeyError('qe')")
```
The definition of `to_offset` is corrected, so now we can see the `FutureWarning`.
```
>>> pd.period_range("2020-01-01", "2020-08-01", freq="y")
<stdin>:1: FutureWarning: 'y' is deprecated and will be removed in a future version, please use 'Y' instead.
PeriodIndex(['2020'], dtype='period[Y-DEC]')
```

```
>>> pd.date_range("1/1/2000", periods=4, freq="qe")
<stdin>:1: FutureWarning: 'qe' is deprecated and will be removed in a future version, please use 'QE' instead.
DatetimeIndex(['2000-03-31', '2000-06-30', '2000-09-30', '2000-12-31'], dtype='datetime64[ns]', freq='QE-DEC')
```

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).



Labels:
name: Frequency
description: DateOffsets
name: Deprecate
description: Functionality to remove in pandas

Comments:
@MarcoGorelli, could you please take a look at this PR?
Thank you for reviewing this PR!
@MarcoGorelli is this scheduled for removal in 4.0?

you might need to manually move the deprecation bullet from the 4.0 tracker issue to the 3.0 tracker issue if it isn't.
thanks, will do
DOC: fix imprecision in the type description of `pd.concat` parameter
State: closed

- [x] closes #56845
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @kodimatrix
WEB: Add page about benchmarks
State: closed

CC: @DeaMariaLeon @rhshadrach @lithomas1 

This is not yet ready, the conbench url provided is not correctly rendering the css, and I need to add all the information about the OVH server, but opening early in case you have any feedback.


Labels:
name: Benchmark
description: Performance (ASV) benchmarks
name: Web
description: pandas website

Comments:
Hm, the ASV link for the OVH results seem to be dead add clicking on any of the times for the runs for the conbench results also gets me a 404.

Is this expected?
The links on the conbench runs don't work either. 
Plus if the menu is not available, one cannot give feedback.  

The server where I setup conbench is here and it works: http://57.128.112.95:5000/
Adding it to pandas' website caused this problem with css - I'm working on it. 
I think I found the reason, but to fix it I will have to take the server down for a bit. 




Thanks for the heads up @lithomas1. I'm now synching the asv website to /benchmarks/asv/.  Hopefully we'll have the conbench site rendering with the css/js soon.

Any other feedback very welcome. 
@lithomas1 @rhshadrach any feedback here before I merge? There will surely be follow ups on this, but I think for now it adds value as it is.
Just to be clear for other people who may read this, regarding:

>the conbench url provided is not correctly rendering the css

Actually, conbench does not currently support this kind of URL. It might in the future. Details about this are here: https://github.com/conbench/conbench/issues/1575#:~:text=We%20did%20not%20build%20(test%2C%20qualify)%20for%20deploying%20the%20Conbench%20web%20application%20at%20a%20non%2Droot%20path%20yet.


This LGTM.

Thanks for writing this up.
Thanks @datapythonista
DEPR: freq ''2BQ-SEP" for to_period should raise an error
State: closed

- [x] closes #56899

corrected the definition to_period to raise an errror for `freq = '2BQ-SEP'` and add tests

No whatsnew note as this is only on main yet.

Labels:
name: Frequency
description: DateOffsets
name: Period
description: Period data type

Comments:
@MarcoGorelli, could you please take a look at this PR?