BrandonFors/pandas-github-issues huggingface dataset:
Update on_bad_lines logic in python_parser.py
State: closed

On_bad_lines only considers a bad line as "a line with too many fields" and silently pads NaNs for rows with missing fields.  Either consider all rows with inconsistent fields as a bad line, as I am proposing here (tested only with our framework, and we log those error details as part of file validation), or also have a param like "allow_jagged_rows = true/false" so users can decide whether to pad or use this logic proposed.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Please open an issue to facilitate a discussion first before proposing a PR
Add Python 3.12 classifier to pyproject.toml (against 2.1.x)
State: closed

Looking at https://pypi.org/project/pandas/, we were unsure if pandas supports Python 3.12 yet.
Based on multiple PRs such as https://github.com/pandas-dev/pandas/pull/56333, we assume 2.1.x also has support for Python 3.12.

(the checkboxes below should not apply for this minor doc fix?) 

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the PR but unfortunately the 2.1.x branch is no longer accepting changes and not adding this classifier was an oversight. This will be correctly documented for version 2.2 which will be released this week. Closing
Support large strings in interchange protocol
State: closed

- [ ] closes #56702 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @MarcoGorelli 

Labels:
name: Interchange
description: Dataframe Interchange Protocol

Comments:
That may also fix https://github.com/pandas-dev/pandas/issues/56754
@MarcoGorelli Does your commit fix the issue you detected earlier? If yes, we could backport to 2.2
Thanks to you :)
BUG: to_stata not handling ea dtypes correctly
State: closed

- [ ] closes #54671 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Stata
description: read_stata, to_stata
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Arrow
description: pyarrow functionality

Comments:
Thanks @phofl
BUG: replace matching Floats with bools for ea dtypes
State: closed

- [ ] closes #55398 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: replace
description: replace method
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
Update check_test_naming.py
State: closed

Changes made:

1. Removed unnecessary import of `annotations` from `__future__`.
2. Simplified the `find_names` function to handle both `ast.Name` and `ast.Attribute`.
3. Combined the `_is_fixture` and `_is_register_dtype` functions into `is_fixture` and `is_register_dtype`.
4. Replaced `any()` with a generator expression to check for conditions in a more concise way.
5. Changed function and variable names to follow snake_case naming convention.
6. Made the code more readable and removed redundant code.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Please make yourself familiar with public vs private, annotations is there on purpose, please read about our pre-commit checks in the contributor guide
BUG: Series.round raising for nullable bool dtype
State: closed

- [ ] closes #55936 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
BUG: IntervalIndex.from_tuples raising with masked subtype
State: closed

- [ ] closes #56765 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Interval
description: Interval data type
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Looks good just needs a whatsnew in 2.2
There is already a whatsnew, so will merge
Remove warning about unused groups
State: closed

This warning is annoying and serves to discourage use of named capturing groups (which are more maintainable) because users must either switch to extracting the groups (not always necessary) or replace their named groups with "(?:" (unnamed groups are harder to maintain because it's less clear what is their objective within a greater regex pattern). 

If users need to specialize their regex patterns to each command, then they need to maintain multiple copies, some with unnamed groups, some without, just to silence some warning, also, if they remove the groups, then later on when they want to use them, they might have to figure out how to replace groups they removed just to silence a warning, and be frustrated.

If we remove this unnecessary warning, then we no longer discourage users who use named capturing groups, thus facilitating readability of the patterns, and portability to other contexts, such as debuggers or the "extract" method mentioned in the removed warning.

TL;DR: This warning doesn't need to exist, so I'm removing it. If there are tests which expect it, then they could also be removed. Code to silence the warning should still work, but could also be removed.

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Please open issues to facilitate a discussion before you open a PR
BUG: fix subclass metadata preservation in groupby column selection
State: closed

This is a small regression that was introduced with https://github.com/pandas-dev/pandas/pull/51090 (in pandas 2.0): before that PR, the column subselection in the groupby code was done with a standard `__getitem__` call, while the PR introduced an optimized helper `_getitem_nocopy` to avoid the copy that a standard `__getitem__` does. However, that new helper forgot to add a `__finalize__` call after `_constructor` from a manager.

The reason we noticed this in geopandas (https://github.com/geopandas/geopandas/pull/3130) was to update our tests for the `apply` deprecation to include group columns (for which one alternative is to specifically select the non-group columns, but so that didn't work with recent pandas versions)

Labels:
name: Bug
name: Groupby
name: Regression
description: Functionality that used to work in a prior pandas version
name: Subclassing
description: Subclassing pandas objects

Comments:
thx @jorisvandenbossche
TYP: Persist typing information for pipe args and kwargs
State: closed

- [x] closes #56359
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Porting work done in:

- https://github.com/pandas-dev/pandas-stubs/pull/823

over to this repository.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Still working on this, but sharing as an early draft for feedback and transparency. 
Thank you @paw-lu!
TST: assert that informative error is raised when offset not supported as a period frequency is passed to DataFrame.asfreq
State: closed

- [x] closes #56718 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature (python -m pytest pandas/tests/frame/methods/test_asfreq.py)
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] N/A: Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] N/A: Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
@phofl anything I should do now to move this forward?
this has been addressed by #56945

really sorry that you spent time on this one, and I hope you got something out of the experience anyway - i hate to do this, but i'll close then
reopening, as:
- the tests from this PR are different to those in the linked one, so they would still be good to add
- I just didn't handle this well, I wasn't aware of this PR when I'd started reviewing the other one

will add a commit soon-ish (early next week) to just keep the tests, then we can merge it in

I didn't get this right as reviewer, I need to own up to this
ENH: Implement interpolation for arrow and masked dtypes
State: closed

- [ ] closes #56267 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Arrow
description: pyarrow functionality

Comments:
Thanks @phofl 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 5fc2ed2703a1370207f4ebad834e665b6c2ad42f
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #56757: ENH: Implement interpolation for arrow and masked dtypes'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-56757-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #56757 on branch 2.2.x (ENH: Implement interpolation for arrow and masked dtypes)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
ENH: add NDArrayBackedExtensionArray to public API
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Reviving https://github.com/pandas-dev/pandas/pull/45544 as I'd like to use it for an UncertaintyArray.

Labels:
name: API Design
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Stale

Comments:
Can we make this "public" with some kind of documentation that we may adjust it in non-user-facing ways more aggressively than we do with regular APIs?
> Can we make this "public" with some kind of documentation that we may adjust it in non-user-facing ways more aggressively than we do with regular APIs?

added a note to the class docstring
I'm unsure what to do about this error as `to_numpy` is an inherited method:
```
Error: /home/runner/work/pandas/pandas/pandas/core/arrays/base.py:542:EX01:pandas.api.extensions.NDArrayBackedExtensionArray.to_numpy:No examples section found
```

Also not sure what to do about 
```
Error: /home/runner/work/pandas/pandas/pandas/core/arrays/datetimes.py:179:EX03:pandas.arrays.DatetimeArray:flake8 error: line 2, col 4: E121 continuation line under-indented for hanging indent
```
as I haven't changed that file.
Given that the Pandas team was finalizing the 2.2 release, you may have hit the docs system at a bad time.  I've had that happen to me.  Might make sense to try merging with latest and resubmitting.
I tried jumping in to see if I could fix the missing toctree problem, but two things defeated me:

1. In the normal pandas build process (running `doc/make.py --warnings-are-errors`) a child process segfaulted on Mac OS 14.1.2 in a Python 3.11.4 enviroment
2. When I tried running `doc/make.py --warnings-are-errors --single reference/api/pandas.api.extensions.NDArrayBackedExtensionArray.argmax.rst` (a command that failed with the toctree warning), that single file built successfully.

I'm trying to see if I can isolate a reproducer for (1) above.
cc @jreback @jorisvandenbossche from original PR
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Create function program
State: closed

- [x] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Please don't open PRs like this, always explain what you are doing and if necessary open issues first
CI: Allow no argument in code_checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

#56408 introduced `set -u`, meaning the script will fail when using any unset variable. This includes `$1`.

Labels:
name: Bug
name: CI
description: Continuous Integration

Comments:
Thanks @rhshadrach
DEPR: by_row="compat" in DataFrame.apply and Series.apply
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Followup to #53400. Fell off my radar that this needed to be deprecated. Will only be able to specify `by_row=False` in 3.0, and then the argument will be removed in 4.0.

cc @topper-123 

Labels:
name: Deprecate
description: Functionality to remove in pandas
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
:facepalm: I forgot there is opposition with enforcing this deprecation in #52140.
DOC: Modified docstring of DataFrame.to_dict() to make the usage of orient='records' more clear
State: closed

Modified documentation for parameter `index` as is mentioned in #56483.
However, I don't think this should be the final solution to this issue. The current behavior remains counter-intuitive and will confuse users who haven't checked the doc.

closes #56483

Labels:
name: Docs

Comments:
Thanks @Huanghz2001
TYP: mostly DataFrame return overloads
State: closed

- `DataFrame.any/bool` return `bool` when `axis=None` and `Series` otherwise
- `DataFrame.sum/prod` currently always returns a `Series` (but will return anything when `axis=None` in the future)
- `DataFrame.mean/median/var/std/skew/kurt` return `Series` when `axis` is not `None` and potentially anything when `axis=None`

I assume it is fine to directly use keyword-only arguments in NDFrame as it is private(?), and Series+DataFrame overwrite the methods in question.

Labels:
name: DataFrame
description: DataFrame data structure
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
The keyword-only changes do not seem to be needed by mypy/pyright: can revert those changes (DataFrame.any is already keyword-only, but DataFrame.all/min//sum/... aren't)
> The keyword-only changes do not seem to be needed by mypy/pyright: can revert those changes (DataFrame.any is already keyword-only, but DataFrame.all/min//sum/... aren't)

xref #57087 (I'm also in favor of keyword-only)
@mroeschke Let me know if I should try to outsource some of the changes to a separate PR to make it easier to review. The keyword-only changes caused many changes: typing overloads and fixing positional usage in the tests and docs.
Yeah could you break out the keyword-only deprecation into it's own PR separate from the misc typing changes?
> Yeah could you break out the keyword-only deprecation into it's own PR separate from the misc typing changes?

I removed the keyword-deprecation decorators
Thanks @twoertwein