BrandonFors/pandas-github-issues huggingface dataset:
TST: Remove test_apply_mutate.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Thanks @rhshadrach
DOC: Fix numpydoc section underlines
State: closed

e.g

```bash
/home/runner/micromamba/envs/test/lib/python3.10/site-packages/numpydoc/docscrape.py:203: UserWarning: potentially wrong underline length... 
Returns 
-------- in 

Return utc offset.... in the docstring of utcoffset
  while not self._is_at_section() and not self._doc.eof():
/home/runner/micromamba/envs/test/lib/python3.10/site-packages/numpydoc/docscrape.py:203: UserWarning: potentially wrong underline length... 
Examples 
------------ in 

Check whether all characters in each string are titlecase.... in the docstring of istitle in /home/runner/work/pandas/pandas/pandas/core/strings/accessor.py.
  while not self._is_at_section() and not self._doc.eof():
```

Labels:
name: Docs

Comments:
Doc checks are passing so merging
DOC: Make warning on query/eval consistent
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Ref: #60602

Labels:
name: Docs
name: expressions
description: pd.eval, query

Comments:
Thanks @rhshadrach
TST(string dtype): Resolve HDF5 xfails in test_put.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

It would be more performant to implement a way of writing the string dtypes without converting to NumPy, but this approach gets us correct behavior for now.

Labels:
name: IO HDF5
description: read_hdf, HDFStore
name: Strings
description: String extension data type and string data

Comments:
Closing in favor of #60663
Update timestamps.pyx
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Proposing a change to fix this issue: https://github.com/pandas-dev/pandas/issues/60583. My understanding is that the bug is due to the code logic described in timestamps.pyx. 

The first option that I considered was modifying normalize in timestamps.pyx. The second option that I considered was modifying normalize_i8_stamp() in timestamps.pyx, and also updating normalize_i8_stamp() in timestamps.pxd to update the normalize_i8_stamp() declaration to match the new signature.

datetimes.py would not be modified because my understanding is that this overflow issue is not present in datetimes. 


Labels:
name: Bug
name: Stale
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
[WIP] `df.apply`: add support for `engine='bodo'`
State: closed

Adds support for specifying engine="bodo" when executing UDF's on DataFrames via `df.apply`.

- [x] closes #60668
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Needs Discussion
description: Requires discussion from core team before further action

Comments:
Is there an issue open to discuss this? Adding another engine internally is a non-trivial amount of work and maintenance, so I'm not sure we would even add this without discussion up front.
> Is there an issue open to discuss this? Adding another engine internally is a non-trivial amount of work and maintenance, so I'm not sure we would even add this without discussion up front.

@WillAyd Hi Will, thank you for the feedback, I would be happy to open an issue and start a conversation. The intention of this draft was mainly experimentation.
@datapythonista  @WillAyd, I believe I have addressed your feedback. Please let me know what are the next steps. I see there are some failures in CI that seem to unrelated to the engine tests. Would be happy to take a closer look into these or work through them together.
@MarcoGorelli Responded to your comment under the issue as well. We would be more than happy to explore this direction.
I think this has been superseded by #61032. Let me know @scott-routledge2 if anyfeedback.
Fix59933
State: closed

- [x] closes #59933 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
This PR revived from https://github.com/pandas-dev/pandas/pull/60052
Instead of opening a new PR, can you continue in the previous one. It is important to keep the history/discussion consolidated.
TST/CLN: Improve some groupby.apply tests
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Test full results; use parametrization

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Clean

Comments:
Thanks @rhshadrach
TST/CLN: Remove groupby tests with mutation
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We no longer support UDFs that mutate the data

https://pandas.pydata.org/pandas-docs/dev/user_guide/gotchas.html#mutating-with-user-defined-function-udf-methods

so I don't think we need to test for this anymore. Plan to move the remaining tests in this file into apply in a followup.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Clean

Comments:
Thanks @rhshadrach
TST(string dtype): Resolve some HDF5 xfails
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The behavior here is that if you write out strings using object dtype, then you still get the inferred string dtype when reading back in. That is, we (likely) do not round-trip dtypes if the value of `future.infer_string` is different when reading/writing.

Labels:
name: IO HDF5
description: read_hdf, HDFStore
name: Strings
description: String extension data type and string data

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 2edc7c9ad9a8b2e1f8df981def5b5b0c434d9ab0
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60615: TST(string dtype): Resolve some HDF5 xfails'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60615-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60615 on branch 2.3.x (TST(string dtype): Resolve some HDF5 xfails)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Thanks @rhshadrach 
Opened manual backport: #60626
TST(string dtype): Resolve to_latex xfail
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

`map_index` expects a return of `None` in certain cases. When using the string dtype, this gets stored in the resulting DataFrame as NaN which is truthy. Adding the explicit NA check resolves.

Labels:
name: Strings
description: String extension data type and string data
name: IO LaTeX
description: to_latex

Comments:
Thanks @rhshadrach
ENH: Make print_thing respect display.precision for Real numbers
State: closed

- [x] closes #60503

- [x] Tests added and passed:
  - Added test_pprint_thing_real_precision to test precision formatting
  - Verified with multiple display.precision values (2, 3, 4)
  - Tested with various float numbers including np.pi

- [x] All code checks passed

- [x] Added type annotations for clarity in modified functions

- [x] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file:
      - Enhanced print_thing to respect display.precision setting for Real numbers

This PR modifies the print_thing function to respect display.precision settings when formatting Real numbers. For example:

```python
with pd.option_context('display.precision', 2):
    df = pd.DataFrame({"value": [np.pi]})
    print(df)  # Shows 3.14

with pd.option_context('display.precision', 4):
    print(df)  # Shows 3.1416

Labels:
name: Enhancement
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Stale

Comments:
@aydinomer00 - looks like the CI is failing. Let me know if you'd like assistance.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Closing as stale. @aydinomer00 - if you'd like to continue, merge main and address the comments above, and we'd be more than happy to reopen!
DOC: Adding a Persian version of Pandas Cheat Sheet
State: closed

I have translated the Pandas cheat sheet for the Persian-speaking community around the world, and it is ready to be added to the Pandas project documentation.

Labels:
name: Docs

Comments:
> Actually, I just realized that you should update https://github.com/pandas-dev/pandas/blob/main/doc/cheatsheet/README.md as well. Can you include that?

No problem, I'll do it.
@Dr-Irv I added the Pandas_Cheat_Sheet_FA to the README.md as per your request.
DOC: Add Hugging Face Hub access
State: closed

Hi team ! We recently made most of the 200k+ datasets from the community on Hugging Face loadable in pandas :)
And we even show the code snippet to load a dataset on each dataset page (see images below)

In this PR I added documentation on accessing Hugging Face Datasets using `fsspec`. It uses `hf://` paths and the implementation is part of the `huggingface-hub` library. Let me know what you think !


1. "Use this dataset" button on a dataset page:
<img width="1290" alt="image" src="https://github.com/user-attachments/assets/6ab57ddf-c42d-4892-8572-6bd0e772e7ce" />

2. Modal showing [pandas code snippet](https://huggingface.co/datasets/google/FACTS-grounding-public?library=pandas):
<img width="1294" alt="image" src="https://github.com/user-attachments/assets/c08ca12f-3b98-4748-8569-8d71892234c3" />

closes https://github.com/pandas-dev/pandas/issues/46000


Labels:
name: Docs

Comments:
Added a section in ecosystem.md :) And I removed the pip extra, but I think the HF fsspec implementation should be an option at the same level as S3 or GCS no ?

I kept HF as an option in the docs on fsspec support in `io.rst` as well, let me know if it's fine.
Thanks @lhoestq
BUILD: Removed tests folder in package distribution to decrease package size
State: closed

- [x] closes #60606
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Thanks for the interest but it is unfortunately more complicated than what is proposed here. There is still a (long) open PR to implement what you are after that has a lot of feedback filed already. If you are interested in seeing this through, I would suggest pinging there to see if you can take it over and then addressing the outstanding comments

https://github.com/pandas-dev/pandas/pull/53007
DOC :  Add Class Diagram (Inheritance) for the groupby and dtypes Directory
State: closed

Highlights inheritance
This pull request introduces a comprehensive class diagram for the groupby and dtypes directory in pandas, providing a high-level overview of the module's architecture. helping developers better understand the structure and functionality of the groupby and dtypes module.


Labels:

Comments:
Thanks for the PR, but we don't currently support maintaining class diagrams in the project as the internals are subject to change which would make these outdated quickly.

Going to close, but happy to have contributions for open issues labeled `good first issue`
DOC: Disable parallel Sphinx builds to avoid EOFError
State: closed

- [x] closes #51536
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
DOC: Disable parallel Sphinx builds to avoid EOFError during documentation generation (Fixes #51536)

Những điều này thực sự tôi không hề biết đến cho đến những ngày ngần đây.
6năm trước sau khi 1 sự cố sảy ra gmail này cũng đã k được sủ dụng cho đến
gần đây. Vì cảm xúc bản thân. Vì hoàn cảnh đang gặp phải. Vì k hiểu biết và
tiếp nhân giao thức. Xin lỗi

Vào CN, 22 thg 12, 2024 lúc 08:02 Vineeth Sai ***@***.***> đã
viết:

> Closed #60597 <https://github.com/pandas-dev/pandas/pull/60597>.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/60597#event-15740586752>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKUN6KAGHG5PNA4AFGEMC5D2GYFSFAVCNFSM6AAAAABUBCYDUSVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJVG42DANJYGY3TKMQ>
> .
> You are receiving this because you are subscribed to this thread.Message
> ID: ***@***.***>
>
DOC: Fix "kwargs" description for .assign()
State: closed

"kwargs" isn't a dict; the keyword arguments are *converted* to a dict.
Secondly, keyword arguments are strings by definition.

- ~~[ ] closes #xxxx (Replace xxxx with the GitHub issue number)~~
- ~~[ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~[ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

Labels:
name: Docs

Comments:
Thanks @wjandrea
CI: Install nightly numpy on free threading build to avoid numpy 2.2.0 segfaults
State: closed

e.g. https://github.com/pandas-dev/pandas/actions/runs/12360467132/job/34495506079

Looks like the segfaults were fixed in https://github.com/numpy/numpy/pull/27955

Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies

Comments:
Merging since the free threading build is green
ENH: Add first and last aggregations to Rolling and Expanding
State: closed

- [ ] closes #33155 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Window
description: rolling, ewma, expanding

Comments:
@mroeschke would `take` be able to handle NAs efficiently though? i.e. perform the aggregation in `O(N)` time as opposed to `O(NW)` (W = window)
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
> This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.

bumping this for visibility.
Some quick benchmarks I ran:

```
In [5]: ser = pd.Series(np.random.randn(10000))

In [6]: %timeit ser.rolling(5).apply(lambda x: x[~x.isna()].iloc[0]) # using custom lambda
831 ms ± 4.21 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [7]: %timeit ser.rolling(5).first()
150 μs ± 881 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

In [8]: %timeit ser.rolling(5).last()
144 μs ± 1.14 μs per loop (mean ± std. dev. of 7 runs, 10,000 loops each)

```
Sorry for the delay @snitish. Thanks for sticking with this 
> Sorry for the delay @snitish. Thanks for sticking with this

No problem at all. Thanks for approving!