BrandonFors/pandas-github-issues huggingface dataset:
DOC towards complying with EX01 specs
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards #37875

Added functions to ignore on code_checks.sh, they are failing EX02 tests. 
More functions were failing EX02, but added line to format.py to eliminate display errors.

@MarcoGorelli


Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
I wonder why that function is not showing up in the errors locally. 

Anyhow, best motivational phrase ever:
> It's really good that you're doing this, you're helping unearth actual issues!

Thanks @MarcoGorelli. :)
PERF: improve performance of infer_dtype
State: closed

Improves performance of `api.types.infer_dtype`:

```python
>>> import numpy as np
>>> import pandas as pd
>>> from pandas.api.types import infer_dtype
>>>
>>> base_arr = np.arange(10_000_000)
>>> x = np.array(base_arr)
>>> %timeit infer_dtype(x)
596 ns ± 128 ns per loop  # main
146 ns ± 8.73 ns per loop  # this PR
>>> x = pd.Series(base_arr)
>>> %timeit infer_dtype(x)
6.83 µs ± 33.7 ns per loop  # main
583 ns ± 5.4 ns per loop  # this PR
>>> x = pd.Series(base_arr, dtype="Int32")
>>> %timeit infer_dtype(x)
3.92 µs ± 9.9 ns per loop  # main
816 ns ± 4.75 ns per loop  # this PR
>>> x = pd.array(base_arr)
>>> %timeit infer_dtype(x)
310 ns ± 0.704 ns per loop  # main
582 ns ± 4.73 ns per loop  # this PR , a bit slower
```


Labels:
name: Performance
description: Memory or execution speed performance
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
I've made a new version without ABCs.
Rerunning the tests gives these result:

```python
>>> import numpy as np
>>> import pandas as pd
>>> from pandas.api.types import infer_dtype
>>>
>>> base_arr = np.arange(10_000_000)
>>> x = np.array(base_arr)
>>> %timeit infer_dtype(x)
596 ns ± 128 ns per loop  # main
146 ns ± 8.73 ns per loop  # this PR, v1
140 ns ± 0.0592 ns per loop  # this PR, v2
>>> x = pd.Series(base_arr)
>>> %timeit infer_dtype(x)
6.83 µs ± 33.7 ns per loop  # main
583 ns ± 5.4 ns per loop  # this PR, v1
725 ns ± 5.24 ns per loop  # this PR, v2
>>> x = pd.Series(base_arr, dtype="Int32")
>>> %timeit infer_dtype(x)
3.92 µs ± 9.9 ns per loop  # main
816 ns ± 4.75 ns per loop  # this PR, v1
765 ns ± 4.81 ns per loop  # This PR, v2
>>> x = pd.array(base_arr)
>>> %timeit infer_dtype(x)
310 ns ± 0.704 ns per loop  # main
582 ns ± 4.73 ns per loop  # this PR, v1
426 ns ± 2.96 ns per loop # this PR, v2
```

Gentle ping.
Thanks @topper-123
TYP: added missing __init__ in pandas/core/methods
State: closed

See https://github.com/microsoft/pyright/issues/4544#issuecomment-1407513778


Ruff's INP001 could have caught this. Could enable it if the following are ignored:

> ci/fix_wheels.py:1:1: INP001 File `ci/fix_wheels.py` is part of an implicit namespace package. Add an `__init__.py`.
> ci/test_wheels.py:1:1: INP001 File `ci/test_wheels.py` is part of an implicit namespace package. Add an `__init__.py`.
> doc/make.py:1:1: INP001 File `doc/make.py` is part of an implicit namespace package. Add an `__init__.py`.
> doc/scripts/eval_performance.py:1:1: INP001 File `doc/scripts/eval_performance.py` is part of an implicit namespace package. Add an `__init__.py`.
> doc/source/conf.py:1:1: INP001 File `doc/source/conf.py` is part of an implicit namespace package. Add an `__init__.py`.
> setup.py:1:1: INP001 File `setup.py` is part of an implicit namespace package. Add an `__init__.py`.
> typings/numba.pyi:1:1: INP001 File `typings/numba.pyi` is part of an implicit namespace package. Add an `__init__.py`.
> web/pandas_web.py:1:1: INP001 File `web/pandas_web.py` is part of an implicit namespace package. Add an `__init__.py`.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
There are some new errors detected - will try to address or ignore them.
The second commit fixes type issues uncovered by a newer pyright version. The third commit uses Sequence[Hashable] more consistently.
Looks good just a merge conflict
> Looks good just a merge conflict

Rebased and green
Thanks @twoertwein
Enable Pylint warning `consider-using-ternary`
State: closed

- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Enables the Pylint type-W warning `consider-using-ternary`.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @uzzell
DEPR: Remove NumericIndex.__new__ + related methods
State: closed

Removes methods `__new__`, `_ensure_array`, `_validate_dtype`, `_ensure_dtype` & `_assert_safe_casting` from `NumericIndex`. Also adds needed contents to `Index` and other locations as needed.

Some scaffolding is added to `NumericIndex.__new__`, but that will be removed in the next PR.

xref #42717.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @topper-123
ENH, TST: Add FutureWarning to read_table
State: closed

- [x] closes #51017


Labels:
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
you'll also need to run the code checks, see here for how to run them https://pandas.pydata.org/docs/dev/development/contributing_codebase.html#pre-commit (and of course do reach out if something doesn't work 🙏 )
Confusion is high. Here are the errors I get from running pre-commit: https://gist.github.com/kathleenhang/edf62b27a5510f96ba4c475f98869c12

Things I've tried:
- Updating from MacOS Monterey -> MacOS Ventura
- Changing Python version in pyproject.toml
- Downgrading to Python 3.9.2 in Dockerfile and rebuilding container
- https://bobbyhadz.com/blog/python-preparing-metadata-pyproject-toml-did-not-run-successfully

Edit: Figured it out! Had to run `pre-commit autoupdate`
Looks like it may have been an Apple M1 issue. This article was helpful: https://stackoverflow.com/questions/71291254/pre-commit-giving-error-with-mirrors-mypy-how-do-i-fix-it

I ran pre-commit in VSCode which passed all code checks for all modified files. However, it fails the Github code checks. 

I also got this complaint when trying to commit my changes: `[ERROR] Your pre-commit configuration is unstaged.` It wouldn't let me commit without adding the pre-commit configuration file to staging. 

I wasn't sure how to override the complaint. If I revert my pre-commit configuration file, it causes pre-commit to break again with the same error.
Thanks for sticking with this! We're getting close

Having done `pre-commit autoupdate`, CI is now getting extra warnings from the updated versions of the tools you've updated

Was it only `isort` which wasn't working on your Apple? If so, perhaps you could just bump the version of `isort` in `.pre-commit-config.yaml`, and keep the others as they are (and we can fix them in a separate PR)

It looks like the `black` formatter isn't passing though - did you stage and commit it after you'd ran `pre-commit`? If not, you can just run
```
pre-commit run --files pandas/tests/io/parser/test_parse_dates.py
```
and then `git add`, `git commit`, `git push`
Correct, it was only `isort` which was not working. I was on the latest isort 5.12.0. I'm not sure why the `black` formatter failed, but I reset my staging area and added the files back to the staging area which seems to pass black.

Okay, I've reverted back to the original pre-commit and did as you mentioned. It has passed all checks on my end.
@MarcoGorelli Thank you!

![plankton-slow-clap](https://user-images.githubusercontent.com/2395780/215495526-36bca8b8-09b9-47b5-93b6-097c1da9c123.gif)

We have to deploy the isort fix on 1.5.x as well
you're right, thanks - I should've made a separate PR just for that so we could just have backported it

I've opened https://github.com/pandas-dev/pandas/pull/51069
Thx
BUG: Series.take needs to validate axis
State: closed

- [X] closes #51022 (Replace xxxx with the GitHub issue number)
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Stale

Comments:
the linter is complaining about something.  take a look
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DEPR: Remove *args and **kwargs on resample methods
State: closed

- [ ] closes #50977 (Replace xxxx with the GitHub issue number)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Resample
description: resample method
name: Deprecate
description: Functionality to remove in pandas

Comments:
We want to issue a deprecation warning first before removing them
Sorry misunderstood. Do we want something like the warnings that were removed in this https://github.com/pandas-dev/pandas/pull/50338?
Yep the FutureWarning that was removed there
Do I have to add tests for the warnings?
Yep, something with tm.assert_produces_warning
What exactly? 
> What exactly? 

@phofl 
Your diff in v2.0.0 is huge, that is odd
Hmm it is. I may of accidentally deleted something and added it back. If I'm not getting any merge conflicts on my end then it should be okay I guess. 
No not really, you have to revert the unnecessary changes
Okay will look into it. 
@phofl latest commit looks okay to me (i.e one before the merge)
diff is still showing 1k lines, e.g. https://github.com/pandas-dev/pandas/pull/51042/files
@phofl done (I hope)
thx @kostyafarber
BUG: GroupBy.sum,prod,cumsum,cumprod with PeriodDtype
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Period
description: Period data type

Comments:
Thanks @jbrockmendel
DOC: Highlight dtype_backend with note
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
  
cc @mroeschke 

Labels:
name: Docs

Comments:
<img width="601" alt="image" src="https://user-images.githubusercontent.com/61934744/215235770-71bad658-e02a-4d71-aa16-9f52cf6a3038.png">

Thanks @phofl
TYP: Adjust typing for Blockplacement.__getitem__
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jbrockmendel 

I might be missing something, but an array should be valid as well

Labels:
name: Internals
description: Related to non-user accessible pandas implementation
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
thanks @phofl
BUG: GroupBy.min/max with unordered Categorical and no groups
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Have figured out fixes for all the xfailed cases in test_groupby_empty, splitting them into specific bugfixes where appropriate.

Labels:
name: Groupby
name: Categorical
description: Categorical Data Type

Comments:
Thanks @jbrockmendel
API: Avoid returning same object for various methods
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jorisvandenbossche 

Labels:
name: Copy / view semantics

Comments:
We should note this as an API change, I think (which it strictly speaking is), but personally I think that would be a good change for 2.0. It would be nice to have the general rule that a method that _typically_ returns a new series/dataframe never returns `self`, even if no changes were needed.
Added a note
Thanks!
ENH: Avoid copying unnecessary columns in setitem by splitting blocks for CoW
State: closed

xref https://github.com/pandas-dev/pandas/issues/48998

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
I think this is ready, will look into the single block case separately. The split allows us to only copy columns that are modified
DOC: Fix intendation of compression notes
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
<img width="684" alt="image" src="https://user-images.githubusercontent.com/61934744/215222203-9b98adf7-8cc2-490a-8d3a-cd28ff3272bf.png">
DOC: fix broken links
State: closed

fix broken link to the article and another one in the "See also" section

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
thx @partev
DOC: Remove inplace from user guide
State: closed

This starts with removing inplace from the user guide


Labels:
name: Docs

Comments:
Just a thought: do we want to add any note/guidance somewhere for folks who might be looking specifically for inplace?
Are these errors expected? (in: `advanced.rst`, local build)
<img width="858" alt="Screenshot of the amended example showing AttributeError" src="https://user-images.githubusercontent.com/6564007/215159209-c5edad20-c549-4aa3-b5b1-6db87fd1343f.png">

Everything else generated without errors.
> Just a thought: do we want to add any note/guidance somewhere for folks who might be looking specifically for inplace?

Was thinking of writing up the PDEP and then linking the discussion in the whatsnew. There could be a better way though

Thx for catching, did not pay enough attention when modifying the examples. Fixed now
> > Just a thought: do we want to add any note/guidance somewhere for folks who might be looking specifically for inplace?
> 
> Was thinking of writing up the PDEP and then linking the discussion in the whatsnew. There could be a better way though
> 
> Thx for catching, did not pay enough attention when modifying the examples. Fixed now

@phofl Can you DM me on slack? I was also thinking about writing up a PDEP (didn't get too far though).
Everyone ok with going ahead and merging?
Thanks @phofl
Remove broken link to NumPy's `HOWTO_DOCUMENT.rst.txt`
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This link is broken (for 4+ years). The "fixed" link is https://github.com/numpy/numpy/blob/main/doc/HOWTO_DOCUMENT.rst which simply points to the same numpydoc page that pandas does here.

The "original" document I think is here: 
https://github.com/numpy/numpy/blob/254c50af3a52c8b1444e46857e9ec59fcb212a41/doc/HOWTO_DOCUMENT.rst.txt

which may have evolved to this:
https://numpy.org/doc/stable/dev/howto-docs.html

I think it's easiest/clearest to simply delete this link, but I can update the link or keep a comment in parentheses if desired.

Labels:
name: Docs

Comments:
thx @eriknw
ENH: Add ignore_index to dropna
State: closed

- [x] closes #31725 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement

Comments:
Thanks @phofl
BUG: to_coo raising Systemerror for EA
State: closed

- [x] closes #50996 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Sparse
description: Sparse Data Type
name: MultiIndex
name: Categorical
description: Categorical Data Type
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Thanks @phofl