BrandonFors/pandas-github-issues huggingface dataset:
DOC: Clarify nrows behavior in read_csv
State: closed

- [x] closes #59078
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Here’s the updated summary with a note section included:

Summary:

This pull request updates the `nrows` parameter documentation for `read_csv` to address ambiguity in the previous text. The original documentation was vague and did not clearly specify that `nrows` counts only the data rows returned, excluding the header, and skipped rows. The revised documentation provides a clearer definition of `nrows` and includes practical examples to better illustrate its usage, improving the overall clarity of the documentation.

Note:

Thanks to the suggestions from a previous PR made by the issuer and insights from Stack Overflow (https://stackoverflow.com/questions/23853553/python-pandas-how-to-read-only-first-n-rows-of-csv-files-in), the updated documentation hopefully clarifies that `nrows` excludes rows skipped with `skiprows`, and the header row. Please let me know if any examples should be added, revised, or removed to further enhance clarity. I understand that the example with `skiprows` might seem obvious, but it was included to address potential confusion.


Labels:
name: Docs

Comments:
Thanks @johnyu013
REF (string): de-duplicate str_map_nan_semantics
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The differences in the existing implementations are cosmetic.

Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jbrockmendel
REF: remove unused variable and checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
@jbrockmendel should this be backported along with your previous PRs touching `_str_map`? I assume this one is not critical because just a small cleanup, but for avoiding conflicts if you make further changes to those methods, better to backport as well? (if so, then this is a small reminder to please include "string dtype" in your PR title to make it a bit easier to find the commits/PRs, although we should maybe start using a 2.3 milestone to keep track of this as well)
This doesnt need backport, no.
Thanks @jbrockmendel
DOC: Fix docstrings for Timestamp: normalize, replace, to_numpy, to_period
State: closed

This PR fixes docstrings for some methods listed in https://github.com/pandas-dev/pandas/issues/59458

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
@natmokval I can't edit the reviewers for this PR. Please have a look.
> You'll need to fix the NaT docstrings as well

Thanks @mroeschke! Fixed.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Fix obj arguments in assertions
State: closed

- [x] closes #59440
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Hello @rhshadrach, thanks for your review!
I left the PR in the draft state as I feel like only the first line changed is actually tested, the two other changes didn't break any assert so I wanted to add more tests/asserts (but I didn't have time to tackle #59459 yet).
That said, I let you all decide if more work is needed here 🙂 
Thanks @serl
ENH: Add return_dict parameter to DataFrame.info() method to return info as a dictionary #59387
State: closed

- [x] closes #59387 
- [x] Tests added and passed
- [x] All code checks passed
- [x] Added type annotations
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
pre-commit.ci autofix
pre-commit.ci autofix
Thanks for the PR but based on the discussion on the issue there wasn't much interest from the core team for adding this in pandas so closing
Use bash as default shell in Dockerfile
State: closed

GitPod's VSCode in browser (and probably others?) defaults to use `$SHELL` in integrated terminal, which is `sh`.
This change is just confort to have it open `bash` without having to change the settings or open the right one manually.

Note that the `.devcontainer` setting should not be necessary anymore, as the algorithm to chose the right shell should be the same for GitPod and VSCode remote.

From the [relevant VSCode doc](https://code.visualstudio.com/docs/terminal/profiles):
> The default terminal profile shell defaults to `$SHELL` on Linux and macOS and PowerShell on Windows. VS Code will automatically detect most standard shells that can then be configured as the default.


**EDIT**
Added a couple of bonus points:
- `--install-hooks` in pre-commit. This will configure everything immediately. Otherwise, the first commit will be very slow (pre-commit creates the environments on-demand, this will force the creation at the first start)
- `bash-completion`. So to <TAB> everything (git completion works out of the box, for example)

Labels:

Comments:
Thanks @serl these are nice changes. I don't think very many core developers are using this at the moment, so if you run into issues feel free to keep pushing up PRs. Would be great to improve
Hello @WillAyd, thanks for the fast reply!

Yes I think there'll be more to do (looking at https://github.com/pandas-dev/pandas/issues/59459). I'm not really familiar with the project yet, so if you have advices on that issue, I'd be grateful (otherwise I'll crawl my way in 🦝)
Thanks @serl . I also do not use Gitpod so am not up to speed on what may or may not be working with it. If you use it and run into problems, I would just continue to create an issue for the buggy behavior (or find one, if it already exists) and can get directed feedback from there
ENH: Add an option to prevent stripping extra whitespaces in pd.read_html
State: closed

- [X] closes #24766
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Inspired from https://github.com/pandas-dev/pandas/pull/46075

Labels:
name: Enhancement
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Updated the PR
> This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.

Remove stale
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: clarify the documentation for DataFrame.to_timestamp/Series.to_timestamp
State: closed

clarify the documentation for `DataFrame.to_timestamp` and `Series.to_timestamp`



Labels:
name: Docs
name: Stale
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
REF (string dtype): de-duplicate _str_map (2)
State: closed

Follows #59443.

For the non-arrow version this is logic-neutral.  For the arrow version the addition of L377-379 is a small change.  It was introduced in #55736 for the non-arrow version and there is no indication the divergence was intentional.

Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jbrockmendel
String dtype: fix alignment sorting in case of python storage
State: closed

Follow-up on the implementation of the object-dtype version of the NaN string dtype in https://github.com/pandas-dev/pandas/pull/58451. From expanding test coverage for this dtype in https://github.com/pandas-dev/pandas/pull/59437, there were a lot of alignment failures:

```
In [1]: pd.options.future.infer_string = True

In [2]: a = Series([1, 1, 1, np.nan], index=["a", "b", "c", "d"])
   ...: b = Series([2, np.nan, 1, np.nan], index=["a", "b", "d", "e"])

In [3]: b.align(a)
Out[3]: 
(a    2.0
 b    NaN
 d    1.0
 e    NaN
 c    NaN
 dtype: float64,
 a    1.0
 b    1.0
 d    NaN
 e    NaN
 c    1.0
 dtype: float64)
```

The above result is not sorted, while normally alignment does sort by default. 

The underlying issue is that `idx1.union(idx2, sort=True)` is not sorting for string dtype (will open a separate issue for that), but the reason we were getting to that code path is because we didn't correctly indicate we can use the faster libjoin for this dtype.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
DOC: Add SA01 for pandas.api.types.is_timedelta64_ns_dtype, pandas.api.types.is_period_dtype and pandas.api.types.is_numeric_dtype
State: closed

xref https://github.com/pandas-dev/pandas/issues/58577
fixes  `-i "pandas.api.types.is_timedelta64_ns_dtype SA01" \, -i pandas.api.types.is_period_dtype SA01 and         -i "pandas.api.types.is_numeric_dtype SA01" \`

Labels:
name: Docs

Comments:
Thanks @rgemawat2000
COMPAT: Fix numpy 2.1 timedelta * DateOffset
State: closed


Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Timedelta
description: Timedelta data type
name: Frequency
description: DateOffsets
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Not sure if the change upstream in numpy is expected, but IMO the checks added are more explicit so I'd say this additional check is fine. Merging to get the CI to green
removed link to numpy's resources tutorial
State: closed

- [ ] closes #59386
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @fbourgey
TST (string dtype): add test build with future strings enabled without pyarrow
State: closed

~On top of the commits of https://github.com/pandas-dev/pandas/pull/58451 and https://github.com/pandas-dev/pandas/pull/59388 for now.~  

Follow-up on https://github.com/pandas-dev/pandas/pull/58451 which implemented the `StringDtype("python", na_value=np.nan)` variant, this PR is adding a custom CI test build for it (future option enabled but without pyarrow installed, such that it uses that new dtype by default)

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: CI
description: Continuous Integration
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
DOC: Update the usage of rtol and atol arguments
State: closed

 Update the usage of `rtol` and `atol` arguments in `pd.testing.assert_frame_equal`

- [x] closes #59344
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Docs

Comments:
Thanks @tan-i-ham
TST (string dtype): un-xfail string tests specific to object dtype
State: closed

xref https://github.com/pandas-dev/pandas/issues/54792


Labels:
name: Strings
description: String extension data type and string data

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Still active
Thanks @jbrockmendel 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 dbeeb1f05bca199b3c1aed979e6ae72074a82243
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #59433: TST (string dtype): un-xfail string tests specific to object dtype'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-59433-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #59433 on branch 2.3.x (TST (string dtype): un-xfail string tests specific to object dtype)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60180
BUG Fix for Add numeric_only to function signature of DataFrameGroupBy.cumprod and `DataFrameGroupBy.cumsum 
State: closed

- [x] closes #53071  
- [x] Tests added and passed
- [x] All code checks passed.
- [x] Added type annotations to new arguments.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby

Comments:
Hello! I have made changes to include 'numeric_only' flag in the function parameters as described in the issue description scope. Please let me know if we need to also have 'skipna' in the function parameters and remove *args and **kwargs entirely. Thank you!
Thanks @maushumee
DOC: Improve DataFrameGroupBy.std ddof doc
State: closed

- [x] closes #59405 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Modified ddof parameter based on `Dataframe.std` ddof doc
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.std.html#pandas.DataFrame.std

<img width="762" alt="image" src="https://github.com/user-attachments/assets/5a65bb3b-7aa2-4bbb-9f7d-58b22a42bf14">



Labels:
name: Docs

Comments:
@rhshadrach - Saw a couple of DataFrameGroupBy functions with the similar "Degrees of Freedom" to the current .std(). Should I change it as well to copy the doc on DataFrame (ie DataFrame.var)

https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.sem.html
https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.var.html
Thanks @KevsterAmp 
> @rhshadrach - Saw a couple of DataFrameGroupBy functions with the similar "Degrees of Freedom" to the current .std(). Should I change it as well to copy the doc on DataFrame (ie DataFrame.var)
> 
> https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.sem.html https://pandas.pydata.org/docs/reference/api/pandas.core.groupby.DataFrameGroupBy.var.html

Yep!
CI: Install libegl explicitly for pytest-qt on ubuntu
State: closed

Seems like ubuntu CI images don't include this anymore e.g. https://github.com/pandas-dev/pandas/actions/runs/10253907588/job/28367579964

Labels:
name: CI
description: Continuous Integration

Comments:
This allows the tests to run again so merging
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 81a12ddd270fc3bbed4e6c49098d3a5f37dfef77
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #59423: CI: Install libegl explicitly for pytest-qt on ubuntu'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-59423-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #59423 on branch 2.2.x (CI: Install libegl explicitly for pytest-qt on ubuntu)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
[pre-commit.ci] pre-commit autoupdate
State: closed

<!--pre-commit.ci start-->
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.5.0 → v0.5.6](https://github.com/astral-sh/ruff-pre-commit/compare/v0.5.0...v0.5.6)
- [github.com/asottile/pyupgrade: v3.16.0 → v3.17.0](https://github.com/asottile/pyupgrade/compare/v3.16.0...v3.17.0)
<!--pre-commit.ci end-->

Labels:

Comments:
@mroeschke - locally I'm seeing pyright 1.1.374 not report any errors regarding this change, but pinned to 1.1.352 gives ` error: "Generator" expects 3 type arguments, but 1 given`. However I get other errors/warnings on 1.1.374 too.
OK given the inconsistency in pyright with maybe another package, let's just skip this pre-commit update for now. We can revisit this in next month's update
DOC: Remove outdated performance metrics from 'Accelerated operations…
State: closed

This PR addresses the issue https://github.com/pandas-dev/pandas/issues/59130 by removing outdated performance metrics from the 'Accelerated operations' section in the documentation. The change ensures that the documentation remains relevant and up-to-date, reflecting the current state of pandas performance. Additionally, this PR serves as a small experiment to demonstrate Devin's capabilities in solving real-life issues from well-known repositories, showcasing the potential for AI-assisted development in open-source projects.

Labels:
name: Docs

Comments:
Thanks @anuk909