BrandonFors/pandas-github-issues huggingface dataset:
REF (string): move ArrowStringArrayNumpySemantics methods to base class
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jbrockmendel
Fix valerror dataframe array
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: metadata
description: _metadata, .attrs

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: Fix docstrings for Timestamp: unit, utcoffset, utctimetuple
State: closed

Part of https://github.com/pandas-dev/pandas/issues/59458

Addresses:

pandas.Timestamp.unit having SA01
pandas.Timestamp.utcoffset having SA01
pandas.Timestamp.utctimetuple having SA01

Changes:
Added See Also section for all 3.
Added Returns section for all 3.
Added extended section for all 3
Removed all 3 from code_checks.sh

Labels:
name: Docs

Comments:
Thanks @shreyas-dev
Issue #59098: fix bug for default level behavior in xs function
State: closed

- [ ] closes #59098 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Fix issue #47101
State: closed

Bisecting two years ago ( https://github.com/pandas-dev/pandas/issues/47101#issuecomment-1139606268 ) shows this regression was introduced in b2d54d9 in 2021. Somehow this hasn't been patched since then.

PR #48313 was supposed to address this, but the PR was closed and never merged and the bug has persisted.

Patch takes two tracks -- it first replaces the `arr == x` with [`np.equals()`](https://numpy.org/doc/1.26/reference/generated/numpy.equal.html) which will handle broadcasting in most cases by itself. For the remainder of cases, an explicit type check is done to return an array for the mask.

- [x] closes #47101
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
I honestly don't understand what failed here

> ```
> ruff-format.............................................................................................Failed
> - hook id: ruff-format
> - files were modified by this hook
> ```
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
> Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

Reopened as https://github.com/pandas-dev/pandas/pull/59766 @mroeschke
DOC: add pandas-gbq and related db-dtypes packages to ecosystem
State: closed

As of pandas 2.2.0, pandas-gbq functionality is fully separated, so I think now would be a good time to introduce it as part of the "ecosystem".

db-dtypes was built for use from pandas-gbq but should be applicable more generally.

- [N/A] closes #xxxx (Replace xxxx with the GitHub issue number)
- [N/A] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [N/A] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [N/A] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [N/A] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @tswast
STY: Add Yields section to option_context
State: closed

Currently failing on main

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Seems to be triggered by an update of numpydoc from 1.7 to 1.8. 
It doesn't seem that informative to have both a returns and yields section, but if it gets CI green again, +1 :)
> It doesn't seem that informative to have both a returns and yields section

Agreed, but just following the rules :)
BUG: Fix out-of-bounds access in safe_sort with an empty array and non-empty codes
State: closed

`safe_sort` could handle out-of-bounds references in `codes` so long as the array was not empty.
By masking all out-of-bounds values to -1, we can handle both empty and non-empty array cases.
Previously we masked out-of-bounds values to 0 and then fixed them up afterwards, but this results in accessing the 0th element which doesn't exist in the empty array case.

- [x] closes #59421
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
> Could you also add a unit test for your original example in [#59421 (comment)](https://github.com/pandas-dev/pandas/issues/59421#issue-2449172337)?

Done. I also verified that with bounds checking enabled on `take_1d_*`, this additional test failed at HEAD and passes with the changes in this PR.
Thanks @jsspencer
String dtype: honor mode.string_storage option (and change default to None)
State: closed

At the moment, the `pd.options.mode.string_storage` option defaults to `"python"`, and the `StringDtype()` constructor just ignores it when constructing the future default NaN-variant (i.e. the option is only used for creating the NA-variant), but uses the logic of "use `"pyarrow"` if installed otherwise `"python"`".

Given this option exists, I think it makes sense to let it always be honored, so that a user can actually control this through the option (and also for testing this makes things a bit easier to check the behaviour in case pyarrow is not installed without uninstalling/installing it constantly). 

In order to honor the option, I changed the default to `None` to mean this "use `"pyarrow"` if installed otherwise `"python"`" logic as default if constructing the NaN-variant. 
When constructing the NA-variant, for now `None` still gets translated to `"python"`. I think we should also change the default for this dtype to use pyarrow if installed, but that's a change only for 3.0, I think (not to backport for 2.3), so therefore didn't include that in this PR.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Should I use `"auto"` instead of `None` for the default? (for some other options like the parquet engine we use `"auto"` for the default that depends on what is installed)
> Should I use `"auto"` instead of `None` for the default? (for some other options like the parquet engine we use `"auto"` for the default that depends on what is installed)

Yea I prefer auto for string options like this
Thanks @jorisvandenbossche
String dtype: still return nullable NA-variant in object inference (`maybe_converts_object`) if requested
State: closed

The `lib.maybe_converts_object()` function (which is used for object dtype inference in various places) has a `convert_to_nullable_dtype=Flase/True` keyword to control whether to return dtypes using NA or not.  Currently we were giving the future `infer_string` option priority. But also when the future dtype is enabled, we should still return an NA-variant of the dtype if explicitly asked for. So this PR switches the order of those two checks to let the `convert_to_nullable_dtype` keyword take precedence over the option.

This came up in SQL tests, but actually this also impacts the `pd.array(..)` behaviour. Currently on main, `pd.array(["some", "strings"])` will infer the nullable `StringDtype()`, consistent with the other dtypes that have such a nullable variant (i.e. returning our Int64 instead of np.int64 for integers, etc). 
This is tested behaviour, but we were still xfailing those tests when `future.infer_string` was enabled. With the change in this PR, also in the future `pd.array(..)` will infer the NA-variant.



(sidenote, we should update this simple `convert_to_nullable_dtype` True/False flag to something that allows passing whether we want a numpy_nullable or pyarrow array, but that's for a separate issue) 

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
> (sidenote, we should update this simple `convert_to_nullable_dtype` True/False flag to something that allows passing whether we want a numpy_nullable or pyarrow array, but that's for a separate issue)

I think generally should just stick with `dtype_backend` throughout right? Not sure where we go from one to the other but that seems like a lossy move
Thanks @jorisvandenbossche
DOC: fix PR02 for pandas.tseries.offsets.*
State: closed

Related to https://github.com/pandas-dev/pandas/issues/58536
- Fixes most of the PR02 checks in `pandas.tseries.offsets.*` 

Full list of code checks addressed:
```
pandas.tseries.offsets.BQuarterBegin PR02
pandas.tseries.offsets.BYearEnd PR02
pandas.tseries.offsets.DateOffset PR02
pandas.tseries.offsets.Easter PR02
pandas.tseries.offsets.FY5253 PR02
pandas.tseries.offsets.FY5253Quarter PR02
pandas.tseries.offsets.Hour PR02
pandas.tseries.offsets.LastWeekOfMonth PR02
pandas.tseries.offsets.Micro PR02
pandas.tseries.offsets.Milli PR02
pandas.tseries.offsets.Minute PR02
pandas.tseries.offsets.MonthBegin PR02
pandas.tseries.offsets.Nano PR02
pandas.tseries.offsets.QuarterBegin PR02
pandas.tseries.offsets.Second PR02
pandas.tseries.offsets.SemiMonthBegin PR02
pandas.tseries.offsets.Week PR02
pandas.tseries.offsets.WeekOfMonth PR02
```


Labels:
name: Docs

Comments:
Thanks @abhinav-thimma
DOC: Fix docstrings for Timestamp: tz_localize, tzname, utcfromtimestamp
State: closed

Part of #59458 

Addresses:

pandas.Timestamp.tz_localize SA01
pandas.Timestamp.tzname SA01
pandas.Timestamp.utcfromtimestamp PR01,SA01

Changes:

Added See Also section to tz_localize
Added See Also and Extended sections to tzname
Added See Also, Extended and Parameters section to utcfromtimestamp
Removed all 3 from code_checks.sh

Labels:
name: Docs

Comments:
Thanks @MichaelMoyles
use x.com and twitter's new X logo
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Web
description: pandas website

Comments:
Thanks @partev
TST (string dtype): clean up construction of expected string arrays
State: closed

~(depends on https://github.com/pandas-dev/pandas/pull/59470~, merged and rebased on top of it)

We have various places in the tests where we very explicitly construct the "raw" arrays for the different variants of the string dtype using their class constructors. 

I think we can simplify this by using our normal `pd.Series`/`pd.array` constructors with specifying the exact dtype (which will then rely on `StringDtype.construct_array_type()._from_sequence(..)` to construct the underling string array). This should give equivalent tests for what it is testing (and the constructors itself are tested elsewhere explicitly).

Also removes some xfails in those tests by fixing the logic of the expected string dtype (i.e. don't change the expected dtype when `future.infer_string` is enabled, see comment below at https://github.com/pandas-dev/pandas/pull/59481#pullrequestreview-2231818481)

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Sorry, I missed that with the last merge of main there were actually relevant failures (maybe some interaction with one of my other merged PRs). Quick fixup -> https://github.com/pandas-dev/pandas/pull/59509
DOC: Fix docstrings for Timestamp: unit, utcoffset, utctimetuple
State: closed

Part of https://github.com/pandas-dev/pandas/issues/59458

Addresses:

pandas.Timestamp.unit having SA01
pandas.Timestamp.utcoffset having SA01
pandas.Timestamp.utctimetuple having SA01


Changes:
Added See Also section for all 3.
Added Returns section for all 3.
Added extended section for all 3 
Removed all 3 from code_checks.sh

Labels:
name: Docs
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Will raise another PR
BUG (string dtype): convert dictionary input to materialized string array in ArrowStringArray constructor
State: closed

https://github.com/pandas-dev/pandas/pull/54074 allowed to pass dictionary encoded string data in the `ArrowStringArray` constructor, because such values can be passed when reading partitioned datasets (it was fixing https://github.com/pandas-dev/pandas/issues/53951).

However, when you actually have a column with string dtype but backed by a dictionary encoded pyarrow array, our StringArray implemention is not set up for that, so once you call some string specific functionality, you can run into errors. Example:

```
In [2]: arr = pd.core.arrays.ArrowStringArray(pa.array(["a", "b", None, "a"], pa.large_string()).dictionary_encode())

In [3]: arr
Out[3]: 
<ArrowStringArray>
['a', 'b', <NA>, 'a']
Length: 4, dtype: string

In [4]: pd.Series(arr)
Out[4]: 
0       a
1       b
2    <NA>
3       a
dtype: string

In [5]: pd.Series(arr).str.len()
...
ArrowNotImplementedError: Function 'utf8_length' has no kernel matching input types (dictionary<values=large_string, indices=int32, ordered=0>)
```

Maybe we could at some point properly support storing dict encoded string values under the hood. But until then, I think it is better to materialize the dictionary encoded input to a plain pyarrow string array, which is what this PR is doing.

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
added attribute section to Timestamp docstring; added definitions of …
State: closed

…minute, month, second, and nanosecond for Timestamp attributes

Part of https://github.com/pandas-dev/pandas/issues/59458

Addresses:

pandas.Timestamp.month having GL08
pandas.Timestamp.minute having GL08
pandas.Timestamp.second having GL08
pandas.Timestamp.nanosecond having GL08

Changes:

Added Attributes section to the pandas.Timestamp docstring
Removed all from code_checks.sh


Labels:
name: Docs

Comments:
Sorry this is causing so much trouble. Could someone point me in the right direction here? I thought that an attribute section was needed, but that failed the Docstring check. I saw where, in another PR for this parent issue, someone added docstrings for items flagged as GL08 in the "See Also" section. So, I did that, but it's still not passing. 
the issue is about adding a "see also" section to those classes, not about adding those classes to another class' "see also"
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: Enforce Numpy Docstring Validation for freqstr, nanos, and rule_code methods
State: closed

Dosctring Validation for freqstr, nanos, and rule_code methods of pandas.tseries.offsets Week, WeekOfMonth, YearBegin, and YearEnd classes.

- [x] fixes #58536
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added/modified dosctrings for freqstr, nanos, and rule_code methods of pandas.tseries.offsets Week, WeekOfMonth, YearBegin, and YearEnd classes in the pandas/_libs/tslibs/offsets.pyx file.
- [x] Deleted the relevant names from code_checks.sh
- [x] Tested doctstring validation for all the above methods.


Labels:
name: Docs
name: Timezones
description: Timezone data dtype

Comments:
Thanks @Siniade
DOC: Enforce Numpy Docstring Validation for pandas.Series.str methods
State: closed

As a part of: https://github.com/pandas-dev/pandas/issues/58504
Doctrings for pandas.Series.str functions:    
    pandas.Series.str.capitalize
    pandas.Series.str.casefold
    pandas.Series.str.center
    pandas.Series.str.decode
    pandas.Series.str.encode

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Strings
description: String extension data type and string data

Comments:
Thanks @awojno-bloomberg
DOC: Add missing single quote to `Timestamp.tz_localize`
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
@MarcoGorelli good catch! Sorry about that. Found a couple more cases of that same line.
BUG: Handle nonexistent end dates when resampling
State: closed

- [x] closes #58380
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Timezones
description: Timezone data dtype
name: Resample
description: resample method

Comments:
Thanks @rob-sil!
String dtype: fix convert_dtypes() to convert NaN-string to NA-string
State: closed

At the moment, when having a DataFrame/Series with the new (future) default string dtype, and calling `convert_dtypes()`, it would only convert object dtype with strings to the nullable (NA) string dtype, but not columns that already are the new string dtype. Similarly like default int64 or float64 columns are converted to Int64/Float64, also str should be converted to string I think.

xref https://github.com/pandas-dev/pandas/issues/54792

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
DOC: Cheatsheet update 3rd page now with frequently used display options
State: closed

- Added an entry in the latest `doc/cheatsheet/Pandas_Cheat_Sheet.pdf` and `doc/cheatsheet/Pandas_Cheat_Sheet.pptx` files that includes a third page with frequently used display options.


Labels:
name: Docs

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: fix docstring validation errors for pandas.Timestamp as part of Issue #59458
State: closed

Part of #59458 

Addresses:
- pandas.Timestamp.month_name SA01
- pandas.Timestamp.quarter SA01

Added See Also sections to both of these functions 

Removed these functions from the list in ci/code_checks.sh


Labels:
name: Docs

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.