BrandonFors/pandas-github-issues huggingface dataset:
Issue 60723 resolved
State: closed

- [x] closes #60723 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `core/reshape/concat.py` file if fixing a bug or adding a new feature.
I am attaching test case:
![image](https://github.com/user-attachments/assets/a67a7794-4062-40f6-8cab-93a73db3909d)
![image](https://github.com/user-attachments/assets/dabe284d-2a90-44cf-8d7d-ec85d465e093)

Adjusted the condition to set name = 0 only when name is explicitly None.
Previously, the name attribute was being overwritten unconditionally when ignore_index=True, even if a valid name existed.



Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
REGR: from_records not initializing subclasses properly
State: closed

- [ ] closes #57008 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

follow up from #58145

Are we back porting regression fixes or just into 3.0?

Labels:
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
> Are we back porting regression fixes or just into 3.0?

I think these should just go into 3.0 as 2.3 will just be new string functionality


Sounds good, I’ll move the whatsnew
Moved
Thanks @phofl
CI: Test Github Actions Arm64 Runners
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
Looks like we're too early :(

some issues with micromamba it seems.
> some issues with micromamba it seems.

Might be worth raising an issue in https://github.com/mamba-org/setup-micromamba/?
issue could be https://github.com/actions/partner-runner-images/issues/25?
This should be mostly working (failures don't seem like my fault?)
Thanks @lithomas1. We're on our way to removing the CircleCI builds! 
@meeseeksdev backport 2.3.x
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b98336653128790661d4c66d398f3e44d481dd3b
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60722: CI: Test Github Actions Arm64 Runners'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60722-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60722 on branch 2.3.x (CI: Test Github Actions Arm64 Runners)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Let's backport this - I think I was having issues with build versioning last release on the Circle jobs.

Consolidating under GHA should provide better observability/debugability into failures in general.
Only backporting the wheel change could be sufficient then?
> Only backporting the wheel change could be sufficient then?

Yes, but the GHA jobs have better coverage so we might as well backport that too.
Manual backport -> https://github.com/pandas-dev/pandas/pull/60780
DOC: Whatsnew for sorting mode result
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Entry from #60702

Labels:
name: Docs

Comments:
Thanks @rhshadrach
Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0
State: closed

With the upcoming pyarrow 19.0, it should start returning the future default string dtype itself, which will require some (test) changes on our side. 

Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions
name: IO Parquet
description: parquet, feather
name: Arrow
description: pyarrow functionality

Comments:
Thanks for tackling this. Maybe we should also have a note in our string guide about preferring PyArrow > 19 to ensure better type preservation?
Merging to get the CI to green. Happy to have the doc note about Pyarrow interop in a follow up PR
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 5efac8250787414ec580f0472e2b563032ec7d53
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60716: Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60716-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60716 on branch 2.3.x (Update PyArrow conversion and arrow/parquet tests for pyarrow 19.0)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
@mroeschke thanks for finishing this!

Manual backport -> https://github.com/pandas-dev/pandas/pull/60755
TST(string dtype): Resolve xfail in test_base.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Copy / view semantics

Comments:
Thanks @rhshadrach 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 8bc8c0a6119b053e520f5018dc1350863f7277e4
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60713: TST(string dtype): Resolve xfail in test_base.py'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60713-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60713 on branch 2.3.x (TST(string dtype): Resolve xfail in test_base.py)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Backport PR: https://github.com/pandas-dev/pandas/pull/60742
TST(string dtype): Resolve xfail in groupby.test_size
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

groupby does inference on the group labels across the board.

```python
df = pd.DataFrame({"a": [1, 1, 2], "b": [3, 4, 5]}, dtype="object")
gb = df.groupby("a")
result = gb.sum()
print(result.index.dtype)
# int64
```

While I agree long-term I'd prefer to preserve object dytpe, I do not think we should be changing this at this point.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Groupby
name: Strings
description: String extension data type and string data

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 354b61f88bc0523d4bb9f3cfe1d6c12f9a3d6567
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60711: TST(string dtype): Resolve xfail in groupby.test_size'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60711-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60711 on branch 2.3.x (TST(string dtype): Resolve xfail in groupby.test_size)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport in https://github.com/pandas-dev/pandas/pull/60782
API(str dtype): Raise on StringDtype for unary op +
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Python strings, NumPy string arrays, object dtype, and StringDtype backed by Python all raise for this op.

Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Strings
description: String extension data type and string data
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 1bb264c443f6be64ac28ff9afc0341eed0bcc455
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60710: API(str dtype): Raise on StringDtype for unary op +'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60710-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60710 on branch 2.3.x (API(str dtype): Raise on StringDtype for unary op +)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Manual backport -> https://github.com/pandas-dev/pandas/pull/60763
DOC: Fix IPython prompts in io.rst
State: closed

- ~~[ ] closes #xxxx (Replace xxxx with the GitHub issue number)~~
- ~~[ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~[ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~[ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

Labels:
name: Docs

Comments:
Thanks @wjandrea
TST(string dtype): Resolve xfails in stack_unstack
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Strings
description: String extension data type and string data

Comments:
Thanks @rhshadrach
ENH: Enable .mode to sort with NA values
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We sort values in groupby with NAs using `safe_sort`, we can do the same for mode. This resolves an inconsistency between Python and PyArrow backed strings as PyArrow's mode can sort the NA values.

The other option here is to not sort the result in all cases. This would be my preference as users can sort the result if they like but they cannot unsort it. But it would be a breaking change.

cc @WillAyd @jorisvandenbossche @mroeschke 

Labels:
name: Enhancement
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Needs Discussion
description: Requires discussion from core team before further action

Comments:
lgtm as well. I see marked for 2.3 but should this not wait until 3? 
This is fixing an inconsistency between str dtypes, which is why I think we'd want it in 2.3. I am comfortable calling this inconsistency a bug. Do you think it is too breaking of a change?
Ah OK - that makes sense for 2.3 then. Can you take this out of draft mode?
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.3.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 1708e9020c418e91fae430cf6a7a6ec09c466429
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #60702: ENH: Enable .mode to sort with NA values'
```

4. Push to a named branch:

```
git push YOURFORK 2.3.x:auto-backport-of-pr-60702-on-2.3.x
```

5. Create a PR against branch 2.3.x, I would have named this PR:

> "Backport PR #60702 on branch 2.3.x (ENH: Enable .mode to sort with NA values)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
I put it in draft mode because it wasn't ready to merge - needs a what's new. No worries at all here, can do a followup, but just as a general ways of working can I request not having others take my PRs out of draft.
Sure no problem
Manual backport -> https://github.com/pandas-dev/pandas/pull/60754
TST(str dtype): Resolve xfail in test_value_counts.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Strings
description: String extension data type and string data

Comments:
Thanks @rhshadrach
TST(string dtype): Resolve xfail in test_find_replace.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We never hit the else branch here.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data

Comments:
Thanks @rhshadrach
ENH: Format `decimal.Decimal` as full precision strings in `.to_json(...)`
State: closed

Encodes `decimal.Decimal` as full precision strings in `.to_json(...)`. The old behaviour is to convert `decimal.Decimal` to floats potentially loosing precision.

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
> Thanks for working on this

Thanks for taking your time to review this. I'm planing to open an issue discussing if this should be hidden behind a flag, such that the default behaviour remains unchanged. This draft is just a proof of concept.

The decimal module has no C API, unfortunately. I found a recent issue in the cpython repo discussing an implementation but it hasn't happened yet. As far as I recall the maintainers of cpython thought this would bloat the C API without any real gain.

I do agree on your comments and will update the patch. It should be noted that most of the implementation is copy/pasted from here. Maybe this has some bugs as-well?
https://github.com/pandas-dev/pandas/blob/7415aca37159a99f8f99d93a1908070ddf36178c/pandas/_libs/src/vendored/ujson/python/objToJSON.c#L352-L374
> It should be noted that most of the implementation is copy/pasted from here. Maybe this has some bugs as-well?

That's certainly possible. Ironically I am probably also the one responsible for that. Without looking at the blame...I'll just say the less I know, the better.

Fixing that is out of scope for this PR, but I would prefer to not copy/paste those issues over. Thanks!
@Tolker-KU generally this looks pretty good. Can you add a whatsnew note to our 3.0 release for this?
@mroeschke mind taking a look?
Thanks @Tolker-KU