BrandonFors/pandas-github-issues huggingface dataset:
ES01 Extended Summaries for Categorical
State: closed

- [x] xref #57440 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the effort here, but it was decided that ES01 is not going to be enforced so closing. Happy to have PRs on other doc issues!
BUG: dt64 + DateOffset with milliseconds
State: closed

- [ ] closes #57529 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Frequency
description: DateOffsets

Comments:
> lgtm. nanoseconds is implicitly handled right?

Yeah I believe so a few lines above the last `elif "milliseconds" in kwds:` added check
PERF: Add short circuiting to RangeIndex._shallow_copy
State: closed

xref https://github.com/pandas-dev/pandas/pull/57445#issuecomment-1953881838

There needs to be More Work done to attempt to return a RangeIndex, but this should help minimize that Work

Labels:
name: Performance
description: Memory or execution speed performance
name: Index
description: Related to the Index class or subclasses

Comments:
thx
Still a 10% slowdown compared to before, could you check if we can close the gap further?
TYP: Remove None from copy deep values
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @phofl
TYP: fix mypy failures on main in pandas/core/generic.py
State: closed

fixed `mypy` failure on main in `pandas/core/generic.py`:
```
pandas/core/generic.py:6463: error: Argument "deep" to "copy" of "BaseBlockManager" has incompatible type "bool | None"; expected "bool | Literal['all']"  [arg-type]
```

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @natmokval 
cc @phofl for the copy typing issue
CoW: Clean up some copy usage
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Thanks @phofl
DOC: Fix xarray example
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @phofl 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 33a1cd7163ce712e5a38fdb5d2e04de203b3ddf9
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #57510: DOC: Fix xarray example'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-57510-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #57510 on branch 2.2.x (DOC: Fix xarray example)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BUG: Tick division is incorrect
State: closed

- [ ] closes #57264 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Frequency
description: DateOffsets
name: Stale

Comments:
did you look at Tick._next_higher_resolution?  it is what we use in `Tick.__mul__` when multiplying by a float, and i suspect that logic can be adapted pretty cleanly
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Looks like this PR has gone stale and needs a rebase. Closing to clear the queue but feel free to reopen when you have time to circle back
DOC: Fix PR01 errors in multiple files (#57438)
State: closed

Fixed PR01 errors (#57438) and added parameter descriptions for:
- `pandas.Grouper`
- `pandas.core.groupby.DataFrameGroupBy.cummax`
- `pandas.core.groupby.DataFrameGroupBy.cummin`
- `pandas.core.groupby.DataFrameGroupBy.cumprod`
- `pandas.core.groupby.DataFrameGroupBy.cumsum`
- `pandas.core.groupby.DataFrameGroupBy.filter`
- `pandas.core.groupby.DataFrameGroupBy.pct_change`
- `pandas.core.groupby.DataFrameGroupBy.rolling`
- `pandas.core.groupby.SeriesGroupBy.nunique`
- `pandas.core.groupby.SeriesGroupBy.cummax`
- `pandas.core.groupby.SeriesGroupBy.cummin`
- `pandas.core.groupby.SeriesGroupBy.cumprod`
- `pandas.core.groupby.SeriesGroupBy.cumsum`
- `pandas.core.groupby.SeriesGroupBy.cumprod`

Fixed functions also removed from code_checks.sh


Labels:
name: Docs

Comments:
Thanks @Deen-dot
DOC: Add whatsnew line for #57485
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: https://github.com/pandas-dev/pandas/pull/57485#issuecomment-1951354802

cc @phofl

Labels:
name: Docs

Comments:
thx @rhshadrach 

no need to back port in case anyone else reads this, the actual pr was merged directly to 2.2.x
Backport PR #57474 on branch 2.2.x (REGR: DataFrame.transpose resulting in not contiguous data on nullable EAs)
State: closed

…uous data on nullable EAs

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
resolve conflicts 
thx @rhshadrach
Remove dtype inference on object dtype indexes for bool
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

lets see which tests break, the subset I am normally running is green...

Labels:
name: Stale

Comments:
Some windows tests
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Looks like the CI isn't running on this PR. I think we would need a new PR here so closing this one
CLN: Remove is_foo_dtype checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: Stale

Comments:
This looks like a very nice clean up. I'm missing context, but seems it's just missing fixing the conflict and updating or removing the test were those are still tested, right? Happy to see this finished and merged if that's the case.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Looks like this PR has gone stale and some tests are still failing here. Closing to clear the queue but feel free to reopen if you have time to circle back
DOC: Add a few deprecation notes
State: closed

- [ ] closes #57382 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
pre-commit.ci autofix
thanks @phofl
REGR: astype introducing decimals when casting from int with na to string
State: closed

…

- [ ] closes #57418 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Thanks @phofl
REGR: query raising for all NaT in object column
State: closed

- [ ] closes #57068 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: expressions
description: pd.eval, query

Comments:
Thanks @phofl
CoW: Track references in unstack if there is no copy
State: closed

- [ ] xref #56633
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jorisvandenbossche @rhshadrach 

@rhshadrach was correct, we can get there without making a copy. it's a special case, but nevertheless


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Copy / view semantics

Comments:
might be worth back porting, but not necessary
this one should be good to merge
Thanks @phofl
REGR: DataFrame.update emits spurious warning about downcasting
State: closed

- [x] closes #57124 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is meant as a stopgap - I plan to finish up #55634 for 3.0 which will avoid any upcasting/downcasting altogether. However I do no like the idea of that PR being put in a patch version since it is an entirely new approach to `DataFrame.update`.

I think this can't go in main because the warning and downcasting behavior has already been removed there.

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Regression
description: Functionality that used to work in a prior pandas version
name: DataFrame
description: DataFrame data structure
name: Warnings
description: Warnings that appear or should be added to pandas
name: Conditionals
description: E.g. where, mask, case_when

Comments:
thx @rhshadrach 

Can you put up a pr with a whatsnew that adds this to main? Only the whatsnew change
CLN remove dataframe api consortium entrypoint
State: closed

Discussed privately

In short:
- I'm no longer involved in the project
- nobody in pandas has much interest in leading the effort
- pandas devs are already carrying out enough (mostly unpaid) maintenance, let's not add to it

The Consortium is more than welcome to work on this, but outside of pandas

Labels:
name: Clean

Comments:
Just curious why you didn't update the `What's New` section to match the way it was done when adding the entrypoint: https://github.com/pandas-dev/pandas/commit/809f3710807efedc684c8f0ed4adef8d01af3171#diff-cc8593fd6d3036f46dbd73bcd245aef9a5e737cd22ca67e9567e686872bf093e. It took me a while to figure out where it was deleted.

@rgommers are you aware of this change?
Hey Anatoly,

I can't remember exactly, I probably just forgot
> @rgommers are you aware of this change?

I don't think I was, thanks for pointing it out. 

This change was breaking it looks like (or you wouldn't have noticed in Modin I think), and the maintenance burden was minimal (xref gh-54383 comments), so I wouldn't have done it this way. But ultimately decisions like this are up to the maintainers alone, so it is what it is.
DEPR: read_csv keywords: keep_date_col, delim_whitespace
State: closed

- [X] closes #55569
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
@rhshadrach @lithomas1 I was wondering if you would be willing to take a look at the PR and let me know if I'm on the right track?
PERF: Improve performance for fillna with datetime
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Performance
description: Memory or execution speed performance

Comments:
Thanks @phofl
PERF: Improve performance for astype is view
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This came up in asvs, it wasn't much but we can do better

Labels:
name: Performance
description: Memory or execution speed performance
name: Copy / view semantics

Comments:
(there is a mypy error ...)
Thanks @phofl
CoW: Remove remaining cow occurrences from tests
State: closed

This should be all of them

Labels:
name: Copy / view semantics

Comments:
For the skipped fastparquet tests, was going to open an issue on their side, but commented on an open one (they already saw the same failures on their CI in the meantime, https://github.com/dask/fastparquet/issues/919)
Tbh I don’t really care. Not objecting to you investing time, but won’t bother with this

we deprecated this in dask/dask already and I would support a similar thing jere
REGR: Index.map adding back tz to tz-agnostic result
State: closed

- [x] closes #57192 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

It seems to me we should skip casting back to the caller's dtype when the caller dtype is datetime, since these are reliably represented as objects (unlike ints, which loses the size). But datetimes/tzs are definitely not in my wheelhouse.

cc @jbrockmendel 

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: Apply
description: Apply, Aggregate, Transform, Map
name: Index
description: Related to the Index class or subclasses

Comments:
I'm pretty sure the problem is in DatetimeArray._from_scalars, where we need to check that dont have tzawareness-mismatch.  instead of 

```
if lib.infer_dtype(scalars, skipna=True) not in ["datetime", "datetime64"]:
    raise ValueError
```

something like

```
if isinstance(dtype, DatetimeTZDtype) and not lib.is_datetime_with_singletz_array(scalars);
     raise ValueError
elif isinstance(dtype, np.dtype) and not lib.is_naive_datetime_array(scalars):
    raise ValueError
```

is_naive_datetime_array doesn't actually exist but seems like it should.
Thanks @jbrockmendel - this is much better. One thing I'm running into is that `is_datetime_with_singletz_array` doesn't check if any of the values are actually datetimes (will always return True when there are no datetimes), so I think this still needs to have the line with `lib.infer_dtype(scalars, skipna=True)` or maybe use `lib.is_datetime_or_datetime64_array`? Any recommendations?
I'm trying to get rid of uses of lib.infer_dtype as it is very rarely the right tool to reach for.

Seems like `is_datetime_with_singletz_array(values) and not isna(values).all()` is the simplest way to get what you've described.  Could try to combine those into a single function i guess but i wouldn't bother at this point.