BrandonFors/pandas-github-issues huggingface dataset:
TST (string dtype): remove usage of arrow_string_storage fixture
State: closed

This removes one other use of `"pyarrow_numpy"` in the tests.

This essentially reverts https://github.com/pandas-dev/pandas/pull/54535, because we now again can check `.storage == "pyarrow" for both dtype variants that use pyarrow as storage. 

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
CI: Use Miniforge over Mambaforge
State: closed

Mambaforge is being sunsetted https://conda-forge.org/news/2024/07/29/sunsetting-mambaforge/

Labels:
name: Build
description: Library building on various platforms
name: CI
description: Continuous Integration

Comments:
Looks like the swap to miniforge worked so merging
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.2.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 642d2446060afb11f9860c79a7339eb6ec96fea7
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #59357: CI: Use Miniforge over Mambaforge'
```

4. Push to a named branch:

```
git push YOURFORK 2.2.x:auto-backport-of-pr-59357-on-2.2.x
```

5. Create a PR against branch 2.2.x, I would have named this PR:

> "Backport PR #59357 on branch 2.2.x (CI: Use Miniforge over Mambaforge)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Actually we probably won't need this for 2.2.x since we'll be on 3.0 by the time mambaforge is fully deprecated so not backporting
Update resample.py
State: closed

Update to solve :-

BUG: groupby then resample on column gives incorrect results if the index is out of order #59350

- [x] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Resample
description: resample method

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST (string dtype): follow-up on GH-59329 fixing new xfails
State: closed

Follow-up on https://github.com/pandas-dev/pandas/pull/59329, which caused some failures on main after being merged (see https://github.com/pandas-dev/pandas/pull/59329#issuecomment-2258200402)


xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
The build (for future strings) is green again. I think it would be good to merge this rather quickly given it's fixing failures on main (will do in a few hours if no-one beats me to it).
Thanks @jorisvandenbossche
DEPR: keep_default_dates and convert_dates in read_json
State: closed

Issue #59161

(NOT TOTALLY DONE THIS IS A DRAFT PULL REQUEST ... THE DESCRIPTION WILL BE DONE IN PROFESSIONAL LANGUAGE IF I CAN GET TO THE ACTUAL PULL REQUEST)

New contributor and self-taught non-professional programmer who loves pandas and fixed a few tests here in 2022.

The part not finished yet = refactoring `dtype` conversion to use `infer_dtype` instead of `astype` guessing ... reading the directions closely enough to get the development environment built correctly took a while and then I've made quite a few changes and am a little bit confused about the aforementioned part and was hoping to get the other changes checked in order to make sure that I am on the correct path.

I tried to delete instances of `keep_default_dates` and `convert_dates` in parameters, calls, and corresponding tests of `read_json` and in related functions, so that it will fall back on using `dtype` conversions from the ujson parser, while keeping the functions and tests intact whenever they are not totally dependent on `keep_default_dates` and `convert_dates`.

`convert_dates` also exists in a file for `sas7bdat` and in parts of the code involving `Stata`, but not as part of the `read_json` function in such cases, so I am pretty sure that these are separate but I wanted to check.


Labels:

Comments:
Apologies for the lack of review, but one quick feedback item: when deprecating something, we initially need to keep the functionality working (but with raising a warning when the user specifies the keyword), and the actual code only gets removed in a later stage (in the next major release).
No problem, thank you for the message. I unassigned myself on the issue and wrote a message explaining it, it might be a little bit too high for my current level and I don't want to keep it blocked. I will check out old pull requests for deprecations following your feedback.
Thanks for the PR but it appears to have gone stale. Going to close but please let us know if you'd like to continue here
TST (string dtype): change any_string_dtype fixture to use actual dtype instances
State: closed

~(for now on top of https://github.com/pandas-dev/pandas/pull/59330, so only look at second commit)~

This will make it easier to add the object-dtype based variant of the dtype with NaN sentinel

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
OK, this seems to be passing now. And it's also fixing some additional failures on main (a newly added test that should have had an xfail)
Thanks @jorisvandenbossche
TST: check visibility of x label and xtick labels for plot.hexbin
State: closed

- [x] closes #44050

added a test for checking visibility of xlabel and xtick labels

Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the PR but it looks to have gone stale. Going to close to clear the PR queue but feel free to ping if you want to continue
DOC: add SA01 for pandas.Timestamp.isoweekday
State: closed

* [x]  xref [DOC: Enforce Numpy Docstring Validation | pandas.Timestamp #58505](https://github.com/pandas-dev/pandas/issues/58505)

fixes

```
 pandas.Timestamp.isoweekday
```

Labels:
name: Docs

Comments:
Thanks @CaesarTY
BUG: Added test for setitem using loc not aligning on index
State: closed

- [x] closes #56024
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @DipanshiB
Doc: Fix docstring SA01 error for pandas.core.groupby median and pandas.core.resample median
State: closed

This PR fixes SA01 error for pandas.core.groupby.DataFrameGroupBy.median and pandas.core.groupby.SeriesGroupBy.median referenced here https://github.com/pandas-dev/pandas/issues/58065

Output of docstring Validation:
################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.core.groupby.DataFrameGroupBy.median" correct. :)

################################################################################
################################## Validation ##################################
################################################################################

Docstring for "pandas.core.groupby.SeriesGroupBy.median" correct. :)

Labels:
name: Docs

Comments:
Thanks @sukriti1
TST:  Add test for `numpy.maximum` with `Timestamp` and `Series` of `datetime64`
State: closed

- [x] closes #50864 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)

Added new test to make sure `numpy.maximum` on `Timestamp` and `Series` of `datetime64` works correctly and does not fail with `TypeError`.

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
CI failures here may get fixed after https://github.com/pandas-dev/pandas/pull/59329
Thanks @Tirthraj93
ENH: Allow to plot weighted KDEs.
State: closed

- [ ] closes #59087 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Visualization
description: plotting

Comments:
Thanks @fbourgey
BUG: Fix for _validate_setitem_value fails to raise for PandasArray 
State: closed

- [x] closes #51044 
- [x] Test cases added and passed
- [x] All code checks passed
- [x] Added type annotations to new methods i.e. _validate_setitem_value in pandas/core/arrays/numpy_.py.
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file for the bug fix.


Labels:

Comments:
Hi @mroeschke, @jbrockmendel! I am a new contributor to pandas community and would love to get the feedback on the changes here. Thanks! 
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: Fix 57735
State: closed

- [x] closes #57735 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

#52063 seems to have introduced the regression reported in #57735.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Thanks @matiaslindgren
DOC: Added extra sentences to clarify series.GroupBy snippets in examples
State: closed

- [ ] closes #59309 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Docs

Comments:
can you check your pr again please? something's gone wrong
> can you check your pr again please? something's gone wrongH

Hello marco, i am trying to understand what went wrong in this pr, and am in the middle of my research. I was about to ask for guidance on the issue tracker soon if I could not figure it out.

However, I will ask for guidance now itself as I have not made progress.

Timeline:

First I made changes to the documentation as specified in the issue, however i noticed that the tests kept falling over with errors related to numpy and dataframe mismatches.  Look at the commit [here](https://github.com/pandas-dev/pandas/pull/59331/commits/da61c6b112d929234db43810bc7ed2a94b08f263)

To verify that the error is related to my changes, I deleted all the changes I made and synced it up with main branch and it passed. Commit [here](https://github.com/pandas-dev/pandas/pull/59331/commits/144e862ade7e09d35e13f10413cb0f1c9ef386ed)


I believed that I wrote the documentation changes incorrectly, however, I tested if the tests fail even with a small change and the tests still fell over. Commit [Here
](https://github.com/pandas-dev/pandas/pull/59331/commits/e3c9efa39f0a0203c0ef7932a1eb9cbf2bfbf462)

My conclusion now is that it is just not happy if I make even the smallest changes to the documentation and thus I need to make changes somewhere else to give me permissions to allow me to make changes to the main area of documentation. I was trying to find instructions on it but haven't been able to find anything. 

Could you provide guidance as to where I am going wrong? Or should I go on Stack Overflow instead?


check https://github.com/pandas-dev/pandas/pull/59331/files and see if it looks right
> however, I tested if the tests fail even with a small change and the tests still fell over

there may have been some test failures unrelated to your, you shouldn't worry about them
> > however, I tested if the tests fail even with a small change and the tests still fell over
> 
> there may have been some test failures unrelated to your, you shouldn't worry about them

Okay I have pushed all the changes I think should be made according to the suggestions made in the issue.
Thanks @ApoorvApoorv 
> Thanks @ApoorvApoorv

You're welcome! Just one question, is it okay to delete the branch now? I could not find guidance on whether I should or not?
Yes it's OK to delete the branch
String dtype: rename the storage options and add `na_value` keyword in `StringDtype()`
State: closed

xref https://github.com/pandas-dev/pandas/issues/54792

Rename the storage options and add `na_value` keyword, i.e. from the PDEP: from current `storage="pyarrow_numpy"` to `storage="pyarrow", na_value=np.nan`

This does not yet deprecate the `"pyarrow_numpy"` string option or `"string[pyarrow_numpy]"` alias, as our tests are using that a lot (I would leave that for a follow-up PR to first update all tests and then deprecate it).



Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
@mroeschke over to you
```bash
pandas/core/arrays/string_.py:123: error: Incompatible types in assignment (expression has type "tuple[str, str]", base class "StorageExtensionDtype" defined the type as "tuple[str]")  [assignment]
```
Thanks @jorisvandenbossche
TST (string dtype): xfail all currently failing tests with future.infer_string
State: closed

~On top of https://github.com/pandas-dev/pandas/pull/59320 and https://github.com/pandas-dev/pandas/pull/59323~ (both merged now)

https://github.com/pandas-dev/pandas/pull/58459 added a CI build with `future.infer_string` enabled, but did not yet surface any failures in a red status (still many tests failing).

I was thinking to add xfails to all currently failing tests, then we can actually enable the CI build properly (now the failure is just suppressed), which might be easier for follow-up PRs to actually tackle those failures part by part.

I mostly blindly added `@pytest.mark.xfail(using_string_dtype(), reason="TODO(infer_string)")` to any test that is currently still failing on the build that enables the future string dtype, i.e. without looking into why it is failing or whether it is an easy fix or not. That can be done in chunks after this PR (essentially after this, we have to go through any of those xfails and fix the code or fix the test to get it passing properly).

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
@mroeschke thanks for the review! (and removed the xlsx file)
Thanks @jorisvandenbossche 
This seems to have caused a few failures on main that for some reason didn't show up in this PR .. 
Not directly sure how to explain that, but looking into them.
Ah, I suppose it was because of the merging of https://github.com/pandas-dev/pandas/pull/59330 before merging this PR (and we should have re-run this PR again after that was merged)
BUG: fix for _validate_setitem_value fails to raise for PandasArray and fix tests
State: closed

- [ x] closes #51044. Added _validate_setitem_value method to NumpyExtensionArray class
- [ x] All tests passed. Modified 2 tests as per new expected behavior.
- [ x] All code checks passed(https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ x] Added type annotations(https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to methods.
- [ x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Closing this PR. Will open a new draft one and continue with the bug fix.
BUG: Missing value code not recognised for Stata format version 105 a…
State: closed

…nd earlier

- [x] closes #58149 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is an initial attempt at fixing this bug, although a more efficient implementation probably exists.

As Pandas does not write in any of these version formats I have simply replaced instances of the old missing code with the current one for these versions before carrying on as before.

I believe that the range of valid values for float/double was also changed when the missing code was changed, however as this was widened any existing files written in these formats should still be within the current range and therefore will be read correctly. If an older format version is created with values outside the documented range then the current version of Stata (18) reads them as valid values, rather than converting them to missing, so I think the behaviour here is consistent with that.

Labels:
name: IO Stata
description: read_stata, to_stata

Comments:
Thanks @cmjcharlton
TST (string dtype): clean-up xpasssing tests with future string dtype
State: closed

Follow-up PR on https://github.com/pandas-dev/pandas/pull/59320, doing some superficial clean-up of the tests (largely removing some xfails that were actually passing in the meantime)

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
The failures with python-dev / numpy-dev seem unrelated.
DOC: warn about when to not use the interchange protocol
State: closed

examples of critical issues:
- #57664 
- #57643 

my major issue with the interchange protocol is that the consortium insisted on upstreaming it to pandas - too late for that now anyway

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Interchange
description: Dataframe Interchange Protocol

Comments:
Thanks @MarcoGorelli 
Maybe we should just start promoting the Arrow C data interface alongside the Arrow PyCapsule interface? IIRC one of the only structural things that it seemed like wasn't being solved there was for GPU-use, but I believe that has since been solved
sounds good to me!
REF (string dtype): rename using_pyarrow_string_dtype to using_string_dtype
State: closed

xref https://github.com/pandas-dev/pandas/issues/54792

This is a plain search-replace of `using_pyarrow_string_dtype` with `using_string_dtype` plus some automated formatting because of the change.

Given the future string dtype will not be strictly backed by pyarrow, this general config helper to check if the future option is enabled can be better renamed (we might need a more specific check if we are using the pyarrow variant in certain cases later, but that can be added when needed), and given there are coming more PRs working on this, better to do this rename first.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
DOC: Typo in docs for na_values parameter in pandas.read_csv function #59314 
State: closed

Fixed the documentation of `read_csv` to list the correct default list of `na_values`.
- [x] closes #59314
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: IO CSV
description: read_csv, to_csv

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/59318/
I just checked the `quoting` format breaks. [ref](https://pandas.pydata.org/preview/pandas-dev/pandas/59318/docs/reference/api/pandas.read_csv.html#pandas.read_csv).  I will fix it, so please don't merge right now.
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/59318/
Thanks @kushagr96