BrandonFors/pandas-github-issues huggingface dataset:
BUG: Bug fix for GH48567
State: closed

Groupby for Series fails if an entry of the index of the Series is equal to the name of the index. Analyzing the bug, I think the following small change would solve the issue:

diff --git a/pandas/core/groupby/grouper.py b/pandas/core/groupby/grouper.py index 5fc713d84e..9281040a79 100644
--- a/pandas/core/groupby/grouper.py
+++ b/pandas/core/groupby/grouper.py
@@ -875,7 +875,7 @@ def get_grouper(
             exclusions.add(gpr.name)

         elif is_in_axis(gpr):  # df.groupby('name')
-            if gpr in obj:
+            if not isinstance(obj, Series) and gpr in obj:
                 if validate:
                     obj._check_label_or_level_ambiguity(gpr, axis=axis)
                 in_axis, name, gpr = True, gpr, obj[gpr]

From my point of view "gpr in obj" does not make sense for series at this point. I think the if-statement is written for dataframes to look if gbr is one of the columns. Skipping the if-statement for series gives the correct results for the examples above. The testsuite for series and groupby runs without errors after the change. I added a test to the test suite at the end of pandas/tests/groupby/test_groupby.py and a comment in the BUG section to doc/source/whatsnew/v1.6.0.rst.

- [x] closes #48567 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby

Comments:
Thanks @JanVHII
DOC: Add docs for DataFrameGroupBy.fillna and SeriesGroupBy.fillna
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Docs
name: Groupby
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
Thanks @rhshadrach
SAS7BDAT parser: Drop unused instance variables
State: closed

I'm suggesting this as an alternative to https://github.com/pandas-dev/pandas/pull/48004.

The `_page_count` attribute is read incorrectly (see https://github.com/pandas-dev/pandas/pull/48004) but it's also unused. I'm removing it here, along with some other unused things.

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean
name: IO SAS
description: SAS: read_sas

Comments:
Thanks @jonashaag
TYP: type all arguments with int default values
State: closed

This enables the last missing check from autotyping: adding type annotations for int default values.

Some of the `axis` might need some more scrutiny whether they allow only ints or also "index" and "columns". Wouldn't mind postponing that when tightening `Axis` to `Literal[0, 1, "index", "columns", "rows"]`.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @twoertwein 

> Some of the axis might need some more scrutiny whether they allow only ints or also "index" and "columns".

Yeah definitely. Is there an open issue about this in the issue tracker? IMO public API `axis` args should be able to accept  `Literal[0, 1, "index", "columns", "rows"]` but immediately converted to `0/1`
> MO public API `axis` args should be able to accept `Literal[0, 1, "index", "columns", "rows"]` but immediately converted to `0/1`

That is also my assumption/expectation :) however, I think that some of the `self.axis` might also be "index/columns". hopefully can figure it out with mypy+doc-string+some runtime testing.
REGR: groupby.size with axis=1 doesn't return a Series
State: closed

- [x] closes #48738 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 3622f15b5056b8f038d3b7adeb82eecfe980beca
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am "Backport PR #48760: REGR: groupby.size with axis=1 doesn't return a Series"
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48760-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48760 on branch 1.5.x (REGR: groupby.size with axis=1 doesn't return a Series)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Thanks for the quick fix @rhshadrach
pylint pre-commit
State: closed

- [ ] closes #48737
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Added pylint to CI. Disabled most messages due to the sheer amount of warnings and changes that would need to be fixed in other PR.

@MarcoGorelli 

Labels:
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
I guess we can organize a sprint for beginners if there are many easy things to fix to make pylint happy.
I have updated the request with the changes you have suggested, I'll start working on the lint errors.
Have fixed on raise function, however, pylint passes locally rather than on CI, due to the inline disables I have added in some locations.
looks like you just need to run the other checks as well, `pre-commit run --all-files` 
Sorry for the probably silly question, but seem like we're adding pylint, but we aren't fixing any error because of it, only adding marks to disable to validation in some places. Are we planning to fix the checks being disabled in this PR later, or how is this useful?
Yeah I think they could gradually enabled in sprints - this is at least gives us something start with
BLD: Install anaconda client in a separate step
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The uploading wasn't working properly, since conda is unfortunately not added to PATH on Github Actions, it is on Azure Pipelines on the other repo. 

I tested on my repo, and it should also work here.

Labels:
name: Build
description: Library building on various platforms

Comments:
Going to put this now, so I can see how it runs tonight.
WARN: Catch warning in tests
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


cc @mroeschke 

Labels:
name: Visualization
description: plotting
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
I'd prefer keeping it as one test, this makes comparing easier.
Thanks @phofl
PERF: MultiIndex.union
State: closed


- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v1.6.0.rst` file if fixing a bug or adding a new feature.


cc @phofl - following your improvements to MultiIndex.difference, it looks like we can replace fast_unique_multiple.

```
     [57630237]       [7d8e3cf8]
     <main>           <multiindex-union>
-      80.0±0.9ms       61.7±0.6ms     0.77  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'union')
-        81.7±1ms         61.1±1ms     0.75  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'union')
-      55.6±0.4ms       38.3±0.3ms     0.69  multiindex_object.SetOperations.time_operation('non_monotonic', 'ea_int', 'union')
-        53.8±1ms       36.6±0.8ms     0.68  multiindex_object.SetOperations.time_operation('monotonic', 'ea_int', 'union')
-      34.1±0.4ms       14.2±0.6ms     0.42  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'union')
-        33.9±2ms       13.8±0.6ms     0.41  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'union')
-        75.8±4ms       14.9±0.4ms     0.20  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'union')
-        75.4±3ms       14.3±0.6ms     0.19  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'union')
- ```

Labels:
name: Performance
description: Memory or execution speed performance
name: MultiIndex

Comments:
thx @lukemanley
API: Change DTA/TDA add/sub to match numpy
State: closed

Sits on top of #48743

Labels:
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Timedelta
description: Timedelta data type
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
@mroeschke nothing really time-sensitive here, but this is a blocker for constructor stuff
Added test_addition_doesnt_downcast_reso
> Added test_addition_doesnt_downcast_reso

Thanks. Just merged https://github.com/pandas-dev/pandas/pull/48743 which I release was supposed to be the Timestamp/Timedelta change.

Could you add a similar dont downcast test for DTA/TDA?
test added + green
Thanks @jbrockmendel
API: Change Timestamp/Timedelta arithmetic to match numpy
State: closed

cc @mroeschke as discussed.  Will do analogous DTA/TDA cases in followup.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Timedelta
description: Timedelta data type
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Thanks @jbrockmendel
rename 1.6 doc to 2.0
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
> Didn’t we want to wait till 1.5.1 is out to avoid surprises?

Fine by me.
DEPR: enforce Series/DataFrame awareness-mismatch deprecations
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timezones
description: Timezone data dtype
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
BUG: AttributeError: 'function' object has no attribute 'currentframe' 
State: closed

- [x] closes #48733  (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Note how `pylint` would've caught this:
```
$ pylint pandas/io/sql.py | grep 'inspect'
pandas/io/sql.py:1648:12: W0621: Redefining name 'inspect' from outer scope (line 15) (redefined-outer-name)
pandas/io/sql.py:1648:12: C0415: Import outside toplevel (sqlalchemy.inspect) (import-outside-toplevel)
pandas/io/sql.py:1663:48: E1101: Function 'inspect' has no 'currentframe' member (no-member)
pandas/io/sql.py:1760:8: W0621: Redefining name 'inspect' from outer scope (line 15) (redefined-outer-name)
pandas/io/sql.py:1760:8: C0415: Import outside toplevel (sqlalchemy.inspect) (import-outside-toplevel)
```

We should add that to CI

Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 e0608cb42e98f165fd99b595b2c9d0bfeb208b8f
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am "Backport PR #48736:  BUG: AttributeError: 'function' object has no attribute 'currentframe' "
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48736-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48736 on branch 1.5.x ( BUG: AttributeError: 'function' object has no attribute 'currentframe' )"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Thanks @MarcoGorelli. Could improve the test as @jorisvandenbossche mentioned but agreed that the existing test is good enough
DEPR: positional arguments in favor of keyword arguments
State: closed

- [x] closes #48395 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Deprecate
description: Functionality to remove in pandas
name: Stale

Comments:
This is currently just changing, not a deprecation. 
Just noting logistically I would prefer removing all prior deprecations before adding new ones for 2.0
> Just noting logistically I would prefer removing all prior deprecations before adding new ones for 2.0

Agreed.  Some of the 2.0 deprecations are missing from #30228, so we're gonna end up just grepping for FutureWarning.  Adding new ones will cause trouble.

This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Needs a rebase, we will start merging deprecations
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
REGR: Raise on invalid colormap for scatter plot
State: closed

- [x] closes #48726 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Visualization
description: plotting
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
Thanks @phofl
PERF: MultiIndex.size
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v1.6.0.rst` file if fixing a bug or adding a new feature.


Improvement comes from avoiding MultiIndex._values

```
import pandas as pd
import numpy as np

mi = pd.MultiIndex.from_product(
    [
        pd.date_range('1970-01-01', periods=1000),
        np.arange(1000),
    ]
)

%timeit mi.copy().size
```

```
111 ms ± 9.11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)       <- main
17.7 µs ± 1.1 µs per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  <- PR
```

At least one existing asv shows improvement (there may be others):

```
 before           after         ratio
     [cda0f6be]       [8395c77a]
     <main>           <multiindex-size>
-      25.1±0.4ms         18.3±1ms     0.73  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'symmetric_difference')
-      24.8±0.4ms         16.7±2ms     0.68  multiindex_object.SetOperations.time_operation('monotonic', 'ea_int', 'symmetric_difference')
-      24.4±0.8ms       15.8±0.3ms     0.65  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'symmetric_difference')
-      25.3±0.4ms         16.4±1ms     0.65  multiindex_object.SetOperations.time_operation('non_monotonic', 'ea_int', 'symmetric_difference')
-      23.9±0.4ms       13.5±0.2ms     0.57  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'symmetric_difference')
-      23.0±0.6ms         12.1±2ms     0.53  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'symmetric_difference')
-        25.1±1ms       13.0±0.6ms     0.52  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'symmetric_difference')
-      23.0±0.4ms       11.2±0.6ms     0.49  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'symmetric_difference')
```


Labels:
name: Performance
description: Memory or execution speed performance
name: MultiIndex

Comments:
Thanks @lukemanley