BrandonFors/pandas-github-issues huggingface dataset:
BUG: groupby.var with ArrowDtype(pa.decimal128)
State: closed

- [x] closes #54627 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


I don't think modifying `_arrow_dtype_mapping` is completely correct because I'm not sure we want to do this conversion all the time.


Labels:
name: Groupby
name: Arrow
description: pyarrow functionality

Comments:
Looks like the solution of converting to masked is not generic enough depending on the agg function. Probably needs a more native solution
BUG: Fix error in printing timezone series
State: closed

- [x] closes #54623
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Timezones
description: Timezone data dtype
name: Sprints
description: Sprint Pull Requests

Comments:
@jorisvandenbossche 
Done
Thanks @AdrianDAlessandro
add the words 'of very good level'
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
If interested in contributing feel free to tackle issues any of these issues that are not assigned https://github.com/pandas-dev/pandas/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22
made path_or_buffer rely on position
State: closed

- [x] part of  #54616
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: Stale
name: Sprints
description: Sprint Pull Requests

Comments:
Thanks for the PR but https://github.com/pandas-dev/pandas/issues/54616 is still under discussion and not ready for a PR yet so closing for now. Feel free to tackle any other unassigned issues labeled `good first issue`
@mroeschke for context, this was done together at a sprint. I agree it certainly still requires discussion (and for that I opened https://github.com/pandas-dev/pandas/issues/54616), but we felt it could already be useful to start with a draft PR.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. Let's still take the discussion to the issue before moving forward here so closing for now
BUG: Prevent OutOfBoundsDatetime error for constructing tz-aware series from list
State: closed

- [x] closes #54442
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Note that this PR only fixes the case where the units are explicitly specified and it is a timezone-aware datetime. The more general case(s) appears to be more complicated.

Labels:
name: Timezones
description: Timezone data dtype
name: Stale
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
@jorisvandenbossche 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
does this need a whatsnew for the bugfix?
@AdrianDAlessandro can you address comments and merge main
Yes, sorry I haven't been able to find time to continue working on this. I'm not convinced it's fully finished because the original bug is only for a timezone-aware series, which this fixes, but the bug also happens with the more generic case of all Datetime Series.

This will require further changes to the Cython code in `tslib.pyx`, specifically the `array_to_datetime` function. But I've never really worked with Python outside of this PR.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
> Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

Done
request for a test for .astype, otherwise LGTM
DOC: Add information on fetching tags for contributors building pandas.
State: closed

- [N/A] closes #xxxx (Replace xxxx with the GitHub issue number)
- [N/A] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [N/A] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [N/A] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [N/A] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @stefmolin
DOC: Convert docstring to numpydoc.
State: closed

- [N/A] closes #xxxx (Replace xxxx with the GitHub issue number) https://github.com/jorisvandenbossche/euroscipy-2023-pandas-sprint/issues/7
- [N/A] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [N/A] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [N/A] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @stefmolin
DEPR: deprecated nonkeyword arguments in to_json
State: closed

- [x] xref #54229 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO JSON
description: read_json, to_json, json_normalize
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @rsm-23
DEPR: deprecated nonkeyword arguments in to_html
State: closed

- [x] xref #54229 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO HTML
description: read_html, to_html, Styler.apply, Styler.applymap
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @rsm-23
BUG: merge not always following documented sort behavior
State: closed

- [x] closes #18776
- [x] closes #24730
- [x] closes #33554 
- [x] closes #40608
- [x] closes #48021
- [x] closes #53157
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.

When merging dataframes there are a number of different code paths hit depending on arguments passed (e.g. how, sort, on index vs columns) as well as left/right characteristics (e.g. unique, monotonic)

The resulting sort behavior is not always consistent and does not always align with documented behavior. 

The docs state:

```
sort: bool, default False
    Sort the join keys lexicographically in the result DataFrame. If False, the order 
    of the join keys depends on the join type (how keyword).

...

left: preserve the order of the left keys
right: preserve the order of the right keys
outer: sort keys lexicographically
inner: preserve the order of the left keys
```

This MR aims to fix the sort behavior for cases where it does not follow documented behavior and add tests to validate sort behavior across a wide range of arguments. 

NOTE: a few existing tests that relied on incorrect sort behavior were updated.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Nice! Thanks @lukemanley
Fixes NonExistentTimeError when resampling weekly 
State: closed

- [ ] closes #53666 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:

Comments:
@MarcoGorelli thanks for reviewing.
Do you have any recommended steps to get started?

@mroeschke Please could you help my mentee with advice/review here? Thanks
Sorry for the delay here. I don't think anyone on the core team has bandwidth right now to suggest a quick solution to this complex issue right now. As alluded to in https://github.com/pandas-dev/pandas/pull/54609#pullrequestreview-1583975480, this may require a larger refactor.

Going to close for now, but happy to have contribution on other issues
DEPR: deprecated nonkeyword arguments in to_markdown
State: closed

- [x] xref #54229 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @rsm-23
DEPR: deprecated nonkeyword arguments in to_stata
State: closed

- [ ] xref #54229 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
@mroeschke the definition of this method already takes care of not allowing any other parameters except `path`.

1.  Do we still need to add the decorator for this method? 
2. Even if we add it, there's no way we can write a test for this method.
Yeah this method already enforces nonkeyword arguments except path so I think we can close this
DEPR: Nonkeyword arguments in to_latex
State: closed

- [x] xref #54229 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @rsm-23
DEPR: Nonkeyword arguments in to_pickle
State: closed

- [x] xref #54229 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas
name: IO Pickle
description: read_pickle, to_pickle

Comments:
Thanks @rsm-23
DEPR: deprecated nonkeyword arguments for to_string
State: closed

- [x] xref #54229  (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
@mroeschke I believe this also requires a test case. Can you please point me to the right file?
`pandas/tests/io/formats/test_to_string.py`
Thanks @rsm-23
Implement any and all for pyarrow numpy strings
State: closed

This should work if we want to follow NumPy semantics as close as possible. I think this should work in general, but that's a discussion for another day

Labels:
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality

Comments:
> I think this should work in general, but that's a discussion for another day

+1

(Also I recently realized that `Index(["A", "B"]).any_or_all()` is different from `Series(["A", "B"]).any_or_all()` and we should deprecate to get matching behavior)
rebased as well
Is the goal to get this in for 2.1 or is this fine for a later release?
Ideally we get this in, but not a deal breaker.

cc @jorisvandenbossche for a final review 
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.1.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 23b546f96f49611541585c3914301ba874ced308
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #54591: Implement any and all for pyarrow numpy strings'
```

4. Push to a named branch:

```
git push YOURFORK 2.1.x:auto-backport-of-pr-54591-on-2.1.x
```

5. Create a PR against branch 2.1.x, I would have named this PR:

> "Backport PR #54591 on branch 2.1.x (Implement any and all for pyarrow numpy strings)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
I'll take care of the back-Ort
> I think this should work in general, but that's a discussion for another day

Another day has come!
Use NaN as na_value for new pyarrow_numpy StringDtype
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Arrow
description: pyarrow functionality

Comments:
Looking just at the single commit https://github.com/pandas-dev/pandas/pull/54585/commits/d862ecaf3aa05082d36512f994d99106cc7c10b1, that looks good to me
cc @jbrockmendel 
rebased, the diff is more manageable now

cc @jorisvandenbossche @jbrockmendel 
merging this
@meeseeksdev backport to 2.1.x
ExtensionArray.Round method and tests
State: open

- [x] closes #49387
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This adds abstract round method in ExtensionOpsMixin to tell EA authors to implement round, and provides a round test.


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Stale

Comments:
Any pointers on how to get mypy to accept two function signitures for round? Datetimelikes get overridden:
```
    def round(self, decimals: int = 0, *args, **kwargs) -> Self:
```
gets overridden by
```
    def round(
        self,
        freq,
        ambiguous: TimeAmbiguous = "raise",
        nonexistent: TimeNonexistent = "raise",
    ) -> Self:
```
If the typing overload decorator doesn't work I think in this case it's OK to ignore the mypy failures since it's a known differnece.

https://docs.python.org/3/library/typing.html#overload
this is ready for review again
Let's do this!
poke @jbrockmendel 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Couple small comments, otherwise LGTM.  Thanks for your patience.
REF: ujson cleanups
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Cool nice work - merge on green
Thanks @jbrockmendel 
This breaks user code that relied on `pandas.io.json.loads`, I'm surprised this went in without a deprecation period or a mention in pandas 2.2's changelog. Is this not considered public API ? (the lack of any trailing underscore suggests it's not private)
`pandas.io.json.loads` is not a public API. The public JSON methods are listed in https://pandas.pydata.org/docs/reference/io.html#json and the more general policy on what's public is stated in https://pandas.pydata.org/docs/reference/index.html#api-reference
ENH: Reflect changes from `numpy` namespace refactor Part 3
State: closed

Hi!
Due to NumPy's main namespace being changed in https://github.com/numpy/numpy/pull/24376, here I update NumPy's imports/aliases: e.g. `np.NaN` and `np.infty` won't be available anymore. Also, a canonical name for NumPy's `float` and `complex` should be used, which are `np.float64` and `np.complex128`.

`sctypes` will be removed from the top namespace, but its destination is not decided yet - therefore I import it from internal `np.core` for now. 

Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Thanks @mtsokol