BrandonFors/pandas-github-issues huggingface dataset:
REF (string): avoid copy in StringArray factorize
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Initial motivation was to avoid the ugly overrides of _from_backing_data.  Turns out that override is needed for NumpyExtensionArray bc of the masking currently done in NumpyEA._values_for_factorize.  That masking is in turn necessary because of incorrect comparisons in the hashtable cython code.

<b>update 2024-08022</b> adding "(string)" to the title since this now fixes an infer-string xfail.

Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jbrockmendel
Starting month ignored by resample
State: closed

- [x] closes #29576
- [ ] Tests added and passed
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Resample
description: resample method
name: Stale

Comments:
I can see that other tests fail, so my solution doesn't seem to be correct. I'll look deeper into why the issue occurs
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: allow None as name in multi-index during join
State: closed

- [x] closes #58721
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Another possible fix is to throw if `MultiIndex.names` contains `None`.
The exception message could say something like "empty index names must be empty strings, not None".

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex

Comments:
Automatic codeowner review requests were unintended, sorry. I accidentally pushed commits from my fork to this branch and that touched files unrelated to this PR. Force-pushed them away.
pre-commit.ci autofix
Thanks @matiaslindgren
CI: Fix ci for numpy 2 failures
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Compat
description: pandas objects compatability with Numpy or Python functions

Comments:
Thanks @phofl
DOC: Fix Numpy Docstring validation errors in pandas.api.extensions.ExtensionArray
State: closed

- Related to: https://github.com/pandas-dev/pandas/issues/58539

Fixes docstring issues listed below:
```
   -i "pandas.api.extensions.ExtensionArray.isna SA01" \
   -i "pandas.api.extensions.ExtensionArray.nbytes SA01" \
   -i "pandas.api.extensions.ExtensionArray.ndim SA01" \
   -i "pandas.api.extensions.ExtensionArray.ravel RT03,SA01" \
   -i "pandas.api.extensions.ExtensionArray.take RT03" \
```


Labels:
name: Docs

Comments:
Thanks @abhinav-thimma
DOC: Fixed bug number in what's new v3.0.0
State: closed

- [x] closes #59532
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @matsidzi
Update maybe_indices_to_slice to work with massive dataframes
State: closed

- [X] closes #59531 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

As per conversation with WillAyd revised patch as intp_t is the proper size for an indexer.

Labels:

Comments:
Hopefully fixed on [PR59537](https://github.com/pandas-dev/pandas/pull/59537)   If I work out how to delete this one I will!
Patch - updates maybe_indices_to_slice to use intp_t
State: closed

- [ ] closes ##59531 (https://github.com/pandas-dev/pandas/issues/59531)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Internals
description: Related to non-user accessible pandas implementation
name: Stale

Comments:
Happy to give it a try.  I'll let you know.
Do you mean intp_t ?  If so, changing max_len to intp_t and np.intp respectively in lib.pyx and lib.pyi (to match the array definitions) also produces expected results:

```
>>> import pandas as pd
>>> print (pd.__version__)
3.0.0.dev0+1351.g523afa840a.dirty
>>> test = pd.read_pickle("summary.pkl")
>>> summary = test[test['count'] > 0]
>>> print (len(test))
4261028590
>>> print (len(summary))
1621743
```

Very minor changes but happy to submit another PR if that works for you.
Yes that's correct. You can just update this PR - no need to create another
Got there.... Many thanks for your help! :)
@benjamindonnachie great I think this change looks pretty good! Let's see if CI is green.

You will also want to add a whatsnew entry, which can be added to pandas/doc/source/whatsnew/v3.0.0.rst`  You can add it to the section on Bug Fixes > Indexing (just follow the same pattern you see for other notes already there)
Whatsnew/v3.0.0.rst updated.  Fingers crossed! :)
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
@mroeschke  I'm not sure what else you need given the minor nature of this patch.
It appeared the code checks were not all passing https://github.com/pandas-dev/pandas/actions/runs/10938423968/job/30366432851
@benjamindonnachie any plans to revive this PR?  Otherwise, I will - I'm affected by this bug as well.
Opened #61080 as a replacement, but left @benjamindonnachie as the original author.
DOCS: remove pandas-coverage app
State: closed

I'd originally made this to help with Cython file, but since Cython coverage isn't tracked any more (and https://github.com/pandas-dev/pandas/pull/54453 was closed) it's not really very useful any more

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
merging as this is too trivial to be worth spending ci minutes on
Backport #59144 on 2.2.x / 2.3.x (remove ops div class to solve #2137)
State: closed

Backport #59144

* remove core.computation.ops.Div resolves #21374 #58748

* need to preserve order

* updating tests

* ~~update whatsnew~~ // no whatsnew for 2.2.x and 2.3 yet

* solve mypy issue

* fixing pytests

* better than cast

* adding specific test

* ~~Update pandas/tests/frame/test_query_eval.py~~ // Not backported, fails on 2.2

* Update pandas/tests/computation/test_eval.py


---------

- [x] closes #58548
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.  -- *no one found for 2.x*


Labels:
name: Bug
name: expressions
description: pd.eval, query

Comments:
@WillAyd - 2.3.x should only have backports related to PDEP-14, is that right?
If that's your policy, be advised that the current stable release 2.2 is not and minor release updates of it will never fully work with Numpy 2
There's a few things not required for PDEP-14 that have made their way in just to get the test suite green, but mostly it is for PDEP-14.

We actually haven't started backporting anything yet to 2.3 - the first PR to do so an get that branch up to speed is https://github.com/pandas-dev/pandas/pull/59513

I'll take a look at this particular change as part of that
Thanks @WillAyd - I wouldn't push for this PR to be backported to 2.3, but no objection either.
I've actually been wondering about a few of the errors on 2.3 (see https://github.com/pandas-dev/pandas/pull/59513#issuecomment-2305587191) and this appears to address some of them, so I _think_ it will be required (thanks @bnavigator for investigating!)
cc @lithomas1 
I _think_ this fixes the failures, so going to merge on red.

thanks @bnavigator 
AFAICT most of this has also been used in #59513 by @WillAyd, so now that this one has been merged into `2.2.x`, his PR into `2.3.x` will probably need adjustments as well. Plus `2.3.x` probably needs a forward
BUG: Series[python_str].__setitem__(series[pyarrow_string] #52987
State: closed

- [ ] closes #52987 
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Bug
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality

Comments:
pre-commit.ci autofix
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Attempt unit us to_datetime
State: closed

#58989

This PR is incomplete - getting feedback from maintainers

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Given the comment in https://github.com/pandas-dev/pandas/issues/58989#issuecomment-2294173834 going to close
API (string): return str dtype for .dt methods, DatetimeIndex methods
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jbrockmendel
Fix docstrings for pandas.Period.year SA01
State: closed

Part of #59458

Addresses:
pandas.Period.year having SA01

Changes:
Added See Also section .
Added Returns section .
Added extended section .
Added notes section .
Added examples section .
Removed "pandas.Period.year SA01"  from code_checks.sh

Labels:
name: Docs

Comments:
Thanks @uditbaliyan
TST: Added test for date_range for bug GH#57456
State: closed

- [x] closes #57456
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Added test for `date_range` for bug #57456. The bug #57456 is no longer reproduced on the `main` branch of `pandas`, because it is fixed after PR #56831.

Labels:
name: Needs Tests
description: Unit test(s) needed to prevent regressions
name: datetime.date
description: stdlib datetime.date support

Comments:
Thanks @matsidzi
Fix docstrings for pandas.Period.month
State: closed

Part of https://github.com/pandas-dev/pandas/issues/59458

Addresses:
pandas.Period.month having SA01
pandas.Period.asfreq SA01


Changes:
Added See Also section .
Added Returns section .
Added extended section .
Added notes section .
Added examples section .
Removed  "pandas.Period.month SA01" and "pandas.Period.asfreq SA01" from code_checks.sh

Labels:
name: Docs
name: Period
description: Period data type

Comments:
thnx, please check ci failure
Thanks @uditbaliyan
REF (string): remove _str_na_value
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
```bash
mypy.....................................................................Failed
- hook id: mypy
- duration: 79.93s
- exit code: 1

pandas/core/strings/object_array.py:67: error: "ObjectStringArrayMixin" has no attribute "dtype"  [attr-defined]
pandas/core/strings/object_array.py:273: error: "ObjectStringArrayMixin" has no attribute "dtype"  [attr-defined]
pandas/core/strings/object_array.py:467: error: "ObjectStringArrayMixin" has no attribute "dtype"  [attr-defined]
Found 3 errors in 1 file (checked 1441 source files)
```
Thanks @jbrockmendel
REF (string): Move StringArrayNumpySemantics methods to base class
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Strings
description: String extension data type and string data
name: backported

Comments:
```bash
mypy.....................................................................Failed
- hook id: mypy
- duration: 80.69s
- exit code: 1

pandas/core/arrays/string_.py:841: error: Incompatible types in assignment (expression has type "BooleanArray", variable has type "ndarray[Any, dtype[Any]]")  [assignment]
pandas/core/arrays/string_.py:844: error: "ndarray[Any, dtype[Any]]" has no attribute "to_numpy"  [attr-defined]
pandas/core/arrays/string_.py:846: error: "ndarray[Any, dtype[Any]]" has no attribute "to_numpy"  [attr-defined]
Found 3 errors in 1 file (checked 1441 source files)
```
Thanks @jbrockmendel
Changes to fix docstring issue with pandas.Timestamp.max and min
State: closed

Context in https://github.com/pandas-dev/pandas/issues/59458

Addressed:
- pandas.Timestamp.max
- pandas.Timestamp.min

Changes done:

- Added docstring for all the above
- Added the below in all the segments:
  - Returns 
  - Summary
  - Extended Summary
  - Example
  - See also


Labels:
name: Docs
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST (string dtype): fix IO dtype_backend tests for storage of str dtype of columns' Index
State: closed

Quick follow-up on https://github.com/pandas-dev/pandas/pull/59481 (I missed that with the last merge of main there was actually one relevant failure, maybe some interaction with one of my other merged PRs)

This was caused because of the merge of https://github.com/pandas-dev/pandas/pull/59488, that changed to honor the `mode.string_storage` option for the future default string dtype. That meant that DataFrames with string column names that were created in a `with pd.option_context("mode.string_storage", string_storage):` context would now also follow the specified storage for the column names.

Labels:
name: Strings
description: String extension data type and string data
name: backported

Comments:
Thanks @jorisvandenbossche
fix: type error when exporting file as bytes
State: closed

- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
![image](https://github.com/user-attachments/assets/50cb4a49-df23-4673-8d76-9db5dca7cc0a)
[official doc url](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#writing-excel-files)
"Writing Excel files to memory" part



Labels:
name: IO Excel
description: read_excel, to_excel
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks for the PR! It seems to me that `WriteExcelBuffer` was should have `io.BytesIO` as a subtype, is this not the case?
No, while it seems to be a subtype, it doesn't actually work.
before
![image](https://github.com/user-attachments/assets/d1b3f11a-0545-4714-a519-c02e2d3e4d76)
![image](https://github.com/user-attachments/assets/9e30736f-6338-486f-b97d-1c29bd5b3e22)
I tried. It's the best solution I've got. line 1242
after
![image](https://github.com/user-attachments/assets/5886ce2c-898a-4e23-9e1c-3d4c7b87d4cf)
![image](https://github.com/user-attachments/assets/0d08fb86-c807-4fbf-a6a9-604491395b77)


> No, while it seems to be a subtype, it doesn't actually work.

I'm against adding unnecessary type-hints to make IDEs happy. If it is a subtype, then the IDE should be fine with it and this is an issue with the IDE, not pandas.
Okay, I'll double-check if it's an IDE problem.
BUG (string): ArrowEA comparisons with mismatched types
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.



Labels:
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Arrow
description: pyarrow functionality
name: backported

Comments:
Thanks @jbrockmendel