BrandonFors/pandas-github-issues huggingface dataset:
CLN: Remove MultiIndex._get_grouper_for_level
State: closed

- [x] closes #49452 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: MultiIndex
name: Clean
name: Index
description: Related to the Index class or subclasses

Comments:
@rhshadrach Cleaner branch and passes tests locally (hopefully CI will work).  Is there a reason though to even keep `_get_grouper_for_level` around or use it in the Grouping class?  Around L483 in gouper.py I think I could just set `self.grouping_vector = index_level.map(mapper)` and achieve the same outcome.  Nothing else uses `_get_grouper_for_level()` so is there a need to keep it around at all?
@codamuse can you address the linting failures and merge main
> @codamuse can you address the linting failures and merge main

@jbrockmendel updated.  Do you think `_get_grouper_for_level()` makes sense to keep around?  I don't think the name is even reflective of what that function does anymore (no levels to get a grouper and `level` isn't even an accepted argument).   Since it's only used in grouper.py L483 would it be cleaner to simplify and change L483 to:

`self.grouping_vector = index_level.map(mapper)`

while dropping `_get_grouper_for_level()` or deprecating it some other way just in case?  

None of the above reflected in this PR, just wondering why not do it that way.

> [...] would it be cleaner to simplify and change [...]

seems reasonable.
LGTM pending green.  Love to see more simplifications in grouper.py
Thanks @codamuse - very nice!
BUG: Index.to_numpy returned a referenece to the index' data
State: closed

- [X] closes #49663
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

`Index.to_numpy` returned a reference instead of a copy of the underlying data, meaning that is was possible to alter the index' supposedly immutable data:

```ipython
>>> import pandas as pd
>>> idx = pd.Index(["a", "b"])
>>> res = idx.to_numpy()
>>> res[0] = "x"
>>> idx
Index(['x', 'b'], dtype='object')  # main, ups
Index(['a', 'b'], dtype='object')  # this PR
```

Labels:
name: Index
description: Related to the Index class or subclasses

Comments:
closing temporarily
STYLE: Fix redefined-outer-name linting issue
State: closed

Part of the fix for #49656
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Looks pretty good, you'll probably also need to change

https://github.com/pandas-dev/pandas/blob/fd8e3e773887c0ba9721406b3034494fff2c2567/pandas/tests/plotting/test_converter.py#L218

to `converter.mdates` (and elsewhere in that file)
Done, thanks for the quick review!
🤔 not sure why one CI job has failed - I've restarted it as it looks unrelated
Thanks, all green now! 😄
STYLE: fix pylint consider-merging-isinstance warnings
State: closed

Related to https://github.com/pandas-dev/pandas/issues/48855
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @Moisan
REF: extract method from _iterparse_nodes
State: closed

- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

While looking through the code I saw this function and thought it seemed too big and too complex (too many conditionals) for a single function. So I'm proposing refactoring it so it would be easier to understand what it does without having to look line by line.


Labels:

Comments:
@eduardostdasilva thanks for the PR, but it seems like there isn't consensus on moving forward with it.  If you think we should reconsider, please open an issue to discuss the matter.  Closing.
BUG: rendering dt64tz values with non-pytz
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Timezones
description: Timezone data dtype

Comments:
Thanks @jbrockmendel 
This patch may have induced a potential regression. Please check the links below. If any ASVs are parameterized, the combinations of parameters that a regression has been detected appear as subbullets. This is a partially automated message.

 - https://asv-runner.github.io/asv-collection/pandas/#tslibs.tslib.TimeIntsToPydatetime.time_ints_to_pydatetime
   - box='datetime'; size=100; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='datetime'; size=100; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='datetime'; size=10000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='datetime'; size=10000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='datetime'; size=1000000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='datetime'; size=1000000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='time'; size=100; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='time'; size=100; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='time'; size=10000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='time'; size=10000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='time'; size=1000000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='time'; size=1000000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='timestamp'; size=100; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='timestamp'; size=100; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='timestamp'; size=10000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='timestamp'; size=10000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
   - box='timestamp'; size=1000000; tz=<DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>
   - box='timestamp'; size=1000000; tz=tzfile('/usr/share/zoneinfo/Asia/Tokyo')
DEPR: remove Int64Index, UInt64Index, Float64Index from tests.indexes.numeric
State: closed

Progress towards #42717. Other locations will follow.

There are tests in here that probably should be parametrized when `Index` will be able to return non-64 bit dtypes. I'll do that in followups, after `Index` returns a `NumericIndex`.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Deprecate
description: Functionality to remove in pandas

Comments:
updated.
Thanks @topper-123
BUG: df.explode mulitcol with Nan+emptylist
State: closed

- [x] closes #46084
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @debnathshoham
DEPR: remove Int|Uint|Float64Index from conftest & _testing
State: closed

Progress towards #42717.

In particular cleans up conftest.py::indices_dict to not use old style indexes and cleans up the `tm.make*Index` functions to not return old style numeric indexes.

I've kept `tm.MakeIntIndex`etc. and just made them tighter versions of `tm.makeNumericIndex` though if they should be deleted that is also fine for me.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @topper-123
API: default to stdlib timezone objects for fixed-offsets
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

addresses part of #34916 but does not address non-fixed-offset tzs

Labels:
name: API Design
name: Timezones
description: Timezone data dtype
name: Mothballed
description: Temporarily-closed PR the author plans to return to

Comments:
This made me realize that `ZoneInfo` doesn't appear to have a concept of a "fixed-offset" timezone.

Are you thinking of still fully pursuing the "default to `datetime.timezone`" for 2.0? `datetime.timezone` objects don't contain the tz data info that `ZoneInfo` has.

IMO in an ideal world it would be nice to default to `zoneinfo.ZoneInfo` and then use `datetime.timezone` for fixed timezones, but ZoneInfo is only available in >3.9 and we have to support 3.8 until April 2023
> IMO in an ideal world it would be nice to default to zoneinfo.ZoneInfo and then use datetime.timezone for fixed timezones, but ZoneInfo is only available in >3.9 and we have to support 3.8 until April 2023

Agreed.  One option would be to make the changeover in 2.0 for all-but-py38
Mothballing until we drop py38 so we can do non-fixed tzinfos at the same time.  in practice this means 3.0
> Do we have a test that asserts that Timestamp.utcnow().tz. or Timestamp(..., tz="UTC").tz returns datetime.timezone.utc?

no, will update
Thanks @jbrockmendel
STYLE: Fixed redefined-outer-name linting issue
State: closed

Fixed `redefined-outer-name` linting issue in the following files (Towards #49656)

* `pandas/core/indexes/datetimes.py`
* `pandas/io/formats/xml.py`
* `pandas/io/json/_table_schema.py`

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
@MarcoGorelli Addressed the requested changes in `xml.py` and `datetimes.py`. Could you please review and let me know of any required changes?
Hey @aditya-anulekh. LGTM, but needs a rebase.
DEPR: remove Int64Index, UInt64Index, Float64Index from public namespace
State: closed

Progress towards #42717. In a followup they will be removed internally.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @topper-123 (very much for tackling this tricky deprecation :) )
Yeah, it has been surprisingly difficult; not having a Windows machine has especially been difficult when debugging int32/int64 issues. Still, I think I got it figured out now and I'm sure tomorrow I'll get #49560 to pass and then the rest is easy.
STYLE: fix pylints redefined-outer-name warnings
State: closed

Fixed pylint redefined-outer-name warnings for following 2 files

**1. pandas\core\frame.py
2. pandas\core\dtypes\inference.py**

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
should be fine thanks, I'll approve / merge on green
STYLE: fix pylint redefined-oute-name warnings (pandas-dev#49656)
State: closed

Fixed warnings for the following files :-
- pandas/core/internals/concat.py
- pandas/core/reshape/merge.py
- pandas/io/json/_json.py

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
you'll need to run pre-commit checks (see the contributing guide for how to do that), but apart from that looks good
closing as stale, thanks anyway for your PR!
DEPR: Enforce deprecation of dropping columns when numeric_only=False in groupby / resample
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Prior to calling the cython code, cummin/cummax/cumsum/cumprod raise NotImplementedError on object dtypes. The Series path intercepts this and raises a TypeError instead. I've implemented the same here ~except for DataFrame, we can't readily tell which dtype(s) would fail, so it raises a somewhat vague "at least one dtype" message~. 

Labels:
name: Groupby
name: Resample
description: resample method
name: Deprecate
description: Functionality to remove in pandas
name: Nuisance Columns
description: Identifying/Dropping nuisance columns in reductions, groupby.add, DataFrame.apply

Comments:
Thanks @rhshadrach
TYP: MergeHow + JoinHow
State: closed

- [x] closes #48503 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thx @debnathshoham
STYLE: fix pylint redefined-outer-name warnings (#49656)
State: closed

Fixed warnings for the following files :-
- **pandas/core/arrays/datetimelike.py**
- **pandas/core/base.py**
- **pandas/core/computation/pytables.py**

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
I have amended the commit in the base branch. Is that usually the workflow, or do people prefer separate commits for iterations?
ENH: semi joins
State: closed

- [x] closes #42784
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
@debnathshoham can you merge main, fix failing ci and move release note to 2.1
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
API: Float64Index.astype(datetimelike)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: API - Consistency
description: Internal Consistency of API/Behavior
name: Astype

Comments:
rebased + green
Thanks @jbrockmendel
REF: remove no-longer-needed ignore_failures
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean

Comments:
thx @jbrockmendel
REF: share astype exception messages
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Astype

Comments:
Thanks @jbrockmendel
STYLE enable pylint: use-implicit-booleaness-not-comparison
State: closed

Issue #48855. This PR enables pylint type "C" warning: `use-implicit-booleaness-not-comparison`
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Note:  Pylint at the moment raises the following error on clean main branch
```
************* Module pandas._libs.tslibs
pandas/_libs/tslibs/__init__.py:38:0: W0406: Module import itself (import-self)
```

Labels:

Comments:
Thanks @natmokval for working on this one

Are we sure this one's safe? E.g.
```python
a: list | None

a = None

if a == []:
    print('here')
```
doesn't print anything, but changing it to
```python
a: list | None

a = None

if not a:
    print('here')
```
prints `'here'`

Perhaps we should keep this one turned off, just like `use-implicit-booleaness-not-len`?
Thanks, @MarcoGorelli,  you are right. I checked the behavior only in a subset of the cases. It is cleaner to keep this check turned off.
cool, thanks - closing for now then, and sorry that you already did work on this!