BrandonFors/pandas-github-issues huggingface dataset:
decorate `Series` with `@set_module('pandas')`
State: closed

- [ ] Part of https://github.com/pandas-dev/pandas/issues/55178
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR add @set_module decorator around class pd.Series (similar to https://github.com/pandas-dev/pandas/pull/55171)


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Sprints
description: Sprint Pull Requests

Comments:
closing in favor of #60263
Update mypy version from 1.11.2 to 1.13.0
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking
name: Sprints
description: Sprint Pull Requests

Comments:
Man Group London Hackathon 2024

@simonjayhawkins 

@ofsouzap
@nvasilev
@shengjie2013
@pyfra
@YasithDSL
@dl-lim
[backport 2.3.x] TST (string dtype): resolve xfails in pandas/tests/copy_view (#60245)
State: closed

(cherry picked from commit 084b1999cffde35bf9e49e5e5b8a5a0482bf927d)

Backport of https://github.com/pandas-dev/pandas/pull/60245

Labels:

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
[backport 2.3.x] BUG (string dtype): correctly enable idxmin/max for python-storage strings (#60242)
State: closed

(cherry picked from commit 754d09163ae08f2b87daa41f2263556dbb809616)

Backport of https://github.com/pandas-dev/pandas/pull/60242

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data

Comments:
Thanks @jorisvandenbossche
String dtype: enable in SQL IO + resolve all xfails
State: closed

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: Strings
description: String extension data type and string data

Comments:
I think the issue with the non-pyarrow future string build has to do with the tests running in parallel. @mroeschke maybe I am misunderstanding, but isn't each module supposed to run all of its tests on one worker? Wonder why this build in particular is the only one that seems to fail
> but isn't each module supposed to run all of its tests on one worker?

I changed the execution method to worksteal a few months ago (test ran much faster that way).

I suspect that possibly not all the fixtures correctly create and clean up the tables they create, and since the ADBC and sqlalchemy connections reference the same table names on the same dbs, the `importorskip("pyarrow")`s added may have disrupted the table states on the db.

(In an ideal world, teach test should work with a unique table name which is cleaned up appropriately upon failure/success)
Ah OK interesting...I wonder how that hasn't affected the main CI jobs to date...

> I suspect that possibly not all the fixtures correctly create and clean up the tables they create

They do as long as they only operate within a single thread, but with multiple threads there is nothing that prevents the tests from getting a race condition, since a lot of the test table names are hard-coded (although we probably could change them to a uuid)

I'm guessing the fix for now would be to add a pytestmarker for single_cpu to the test_sql.py module (?)
Hmm one more failing test now with `test_insertion_method_on_conflict_do_nothing` (not directly sure how that is caused by code change, because the wrong dtype is for the float column)
Thanks @WillAyd!
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 ba4d1cfdda14bf521ff91d6ad432b21095c417fd
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60255: String dtype: enable in SQL IO + resolve all xfails'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60255-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60255 on branch 2.3.x (String dtype: enable in SQL IO + resolve all xfails)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60315
TST: Add test for visibility of x label and xtick labels for `plot.hexbin`
State: closed

- [x] closes #44050 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

This PR adds a unit test to `pandas/tests/plotting/frame/test_frame.py` to verify that hexbin plots correctly display x labels and xtick labels. Adds a test for the solved issue described in GH https://github.com/pandas-dev/pandas/issues/44050.



Labels:
name: Visualization
description: plotting
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
Thanks @jasonmokk
DOC: fix SA01,ES01 for pandas.core.groupby.DataFrameGroupBy.groups, pandas.core.groupby.SeriesGroupBy.groups and pandas.core.resample.Resampler.groups
State: closed

fixes

```
pandas.core.groupby.DataFrameGroupBy.groups SA01,ES01
```

Labels:
name: Docs
name: Groupby

Comments:
Thanks @tuhinsharma121!
BUG: remove single quotes in index names when printing
State: closed

- [x] closes #60190 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) 
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) (N/A – no new classes/functions added)
- [x] Modified `__str__` in `FrozenList ` (from `pandas/core/indexes/frozen.py`) to escape single quotes and updated `pprint_thing `in `pandas/io/formats/printing.py` to replace single quotes with an empty string.


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string

Comments:
> Thanks for the PR! Whenever making changes to behavior, please add tests.

> Tests have been added.
Thanks @thedataninja1786!
np!
DOC: Additions/updates documentation for df.interpolate() methods #60227
State: closed

- [x] closes #60227 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


### This PR improves the documentation for the `DataFrame.interpolate()` method and closes #60227

The focus is on providing more precise descriptions and examples for the following `method` options:

- **`method="index"`**: Emphasized that this method uses the actual numerical values of the index, which is particularly useful for non-sequential or time-based indices.
- **`method="time"`**: Expanded on the description to specify that this method interpolates based on the time intervals between observations, suitable for data with a time-based index.
- **`method="values"`**: Clarified that this method performs interpolation based on the actual numerical values within the DataFrame, treating them as equally spaced along the index.


### Changes Made

- Enhanced the docstring descriptions for `values`, `index`, and `time` methods in `DataFrame.interpolate()`.
- Added a practical example using a DataFrame with a non-sequential index to demonstrate the difference between `method="linear"` and `method="index"`.


Labels:
name: Docs
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments:
@rhshadrach  Could you review this PR to improve documentation for df.interpolate() methods as part of issue #60227?”
@rhshadrach  I've made the requested changes to the documentation. Could you please review it again? Thanks!

Thanks @anishkarki
[backport 2.3.x] TST (string dtype): fix groupby xfails with using_infer_string + update error message (#59430)
State: closed

(cherry picked from commit e5dd89d4d74d8e2a06256023717880788f2b10ed)

Backport of https://github.com/pandas-dev/pandas/pull/59430

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Strings
description: String extension data type and string data

Comments:
Looks like some of the error messages on this branch need updating too
TST (string dtype): resolve xfails in pandas/tests/copy_view
State: closed

One remaining xfail is due to Index not preserving dtype through `join`, which sounds like an actual bug.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 084b1999cffde35bf9e49e5e5b8a5a0482bf927d
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60245: TST (string dtype): resolve xfails in pandas/tests/copy_view'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60245-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60245 on branch 2.3.x (TST (string dtype): resolve xfails in pandas/tests/copy_view)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60257
BUG (string dtype): correctly enable idxmin/max for python-storage strings
State: closed

The `argmin`/`argmax` methods already work fine on the EAs, but it was not yet properly wired up in `EA._reduce` for the python storage variant.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 754d09163ae08f2b87daa41f2263556dbb809616
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60242: BUG (string dtype): correctly enable idxmin/max for python-storage strings'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60242-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60242 on branch 2.3.x (BUG (string dtype): correctly enable idxmin/max for python-storage strings)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60256
Fix Bug: last column missing formatting in Latex (#52218)
State: closed

Still need to write the tests, bug getting this fix started.

- [x] closes #52218 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Styler
description: conditional formatting using DataFrame.style

Comments:
When you add tests can you test the following:

- A DataFrame with two value columns and an Index. Both values are styled and should have affected output in LaTeX.

The following scenarios:

- The index is a single `Index` and it is hidden,
- The index is a 3 column `MultiIndex` and two levels are hidden.
- The index is a 3 column `MultiIndex` and all levels are hidden.

Thanks.
Sorry for the noise, was trying to change it into a feature branch like the the contribution guide mentions :-)

Will re-submit a PR with the included tests and from a feature branch.

https://pandas.pydata.org/docs/dev/development/contributing.html#contributing-forking
ENH (string dtype): convert string_view columns to future string dtype instead of object dtype in Parquet/Feather IO
State: closed

This is a follow-up on https://github.com/pandas-dev/pandas/pull/60222, which allows passing string_view data to the string dtype constructor, but in this PR ensuring we also use this capability when reading Parquet (or Feather, ORC) files that might use that type.

PyArrow does not yet support writing string_view to Parquet, so we can't test it yet with Parquet, only with Feather.



Labels:
name: Strings
description: String extension data type and string data
name: IO Parquet
description: parquet, feather
name: Arrow
description: pyarrow functionality

Comments:
Thanks @jorisvandenbossche 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 f307a0a3615d93c2177f6581133bdb541e12a93c
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60235: ENH (string dtype): convert string_view columns to future string dtype instead of object dtype in Parquet/Feather IO'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60235-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60235 on branch 2.3.x (ENH (string dtype): convert string_view columns to future string dtype instead of object dtype in Parquet/Feather IO)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60291
TST (string dtype): resolve xfails in pandas/tests/series
State: closed

This should resolve all remaining string-dtype related xfails / TODOs in `pandas/tests/series`

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data

Comments:
Thanks @jorisvandenbossche 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 3f7bc81ae6839803ecc0da073fe83e9194759550
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60233: TST (string dtype): resolve xfails in pandas/tests/series'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60233-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60233 on branch 2.3.x (TST (string dtype): resolve xfails in pandas/tests/series)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60240
BUG: Fix to_excel storing decimals as strings instead of numbers (issue #49598)
State: closed

- [x] closes #49598
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is the same bug fix code as PR #50174, which went stale and had no tests. The author of that PR is no longer responding. This PR adds tests and release notes.

Labels:
name: IO Excel
description: read_excel, to_excel

Comments:
Thanks @michiel-de-muynck 
Hi, thanks for your work! 
When I export a dataframe to my PostgreSQL database with `to_sql(...)` using `if_exists='replace'`, `decimal.Decimal` values are stored as `TEXT` instead of `DECIMAL`. Does this patch also fix this issue?
Hi Lorenzo, this only affects Excel, not PostgreSQL.
@mroeschke this is milestoned 3.0 but the whatsnew entry is in 2.3. does that need to be addressed (or backported)?
Oops the whatsnew entry should be moved to `v3.0.0.rst`. Could you open another PR to do that @michiel-de-muynck?
ENH (string dtype): accept string_view in addition to string/large_string for ArrowStringArray input
State: closed

Looking into support for string dtype on the pyarrow side, and noticed we do not yet support string_view being converted to string dtype for the default pyarrow `to_pandas()` behaviour.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality

Comments:
Thanks @jorisvandenbossche
BUG (string dtype): fix qualifier in memory usage info
State: closed

Ensuring the `+` qualifier in the memory usage is correct: we don't need it for the pyarrow-based string dtype, but still need it if it is object-dtype based.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data

Comments:
Thanks @jorisvandenbossche 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 0937c95777d44462d67fd5b299d4563984e78332
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60221: BUG (string dtype): fix qualifier in memory usage info'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60221-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60221 on branch 2.3.x (BUG (string dtype): fix qualifier in memory usage info)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60231