BrandonFors/pandas-github-issues huggingface dataset:
BUG: Avoid RangeIndex conversion in read_csv if dtype is specified
State: closed

- [x] closes #59077 (Replace xxxx with the GitHub issue number)

No whatsnew needed since a it's a bug on main

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO CSV
description: read_csv, to_csv

Comments:
Going to merge since this fixes the bug on main. Happy to follow up if needed
DOC: Clarify row delete comparison to SQL
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @yuriishizawa
BUG: Integer values at the top end of the supported range incorrectly…
State: closed

… interpreted as missing for format versions 111 and prior

- [x] closes #58130 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

There are several ways that this could be solved. As Pandas doesn't support writing data in the applicable format versions I have taken the minimal approach here of expanding the integer limits to their documented maximums for these older versions, and recoding the old missing value for (.) to its current equivalent.

Labels:
name: IO Stata
description: read_stata, to_stata

Comments:
Thanks @cmjcharlton
DOC: Add SA01 for `pandas.api.types.is_string_dtype`
State: closed

xref https://github.com/pandas-dev/pandas/issues/58577
fixes  `-i "pandas.api.types.is_string_dtype SA01"` in [`ci/code_checks.sh`](https://github.com/pandas-dev/pandas/blob/c468028f5c2398c04d355cef7a8b6a3952620de2/ci/code_checks.sh#L657)

Labels:
name: Docs

Comments:
Thanks @tan-i-ham
DOC: pyarrow.rst: Add missing IO readers
State: closed

# Changes

* Add `read_parquet` and `read_table` (experimental) to the list of IO readers that support the PyArrow engine.
* Alphabetize list to improve readability.


- [x] (N/A) closes #xxxx
- [x] (N/A) [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] (N/A) All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] (N/A) Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] (N/A) Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @mattayes 
Thanks for the quick review/merge @mroeschke!
DOC: Add Bodo to out-of-core projects in ecosystem
State: closed

Sinnce bodo supports
- [x] closes #59301 
- [x] ~~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature~~
- [x] ~~All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).~~
- [x] ~~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- [x] ~~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~


Labels:
name: Docs

Comments:
Thanks @ritwika314
DOC: add note to docstring about `dtype` keyword when creating Series from abother Series
State: closed

When creating Series from another Series `dtype` keyword is not not used. Add note on `dtype` keyword to Series and DataFrame constructor. See GH #59060.

- [x] closes #59060 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Series
description: Series data structure
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
Undo the previous changes and add a note to the `dtype` keyword on `pd.Series` and `pd.DataFrame` that says that the keyword is ignored if `data` is Series and DataFrame correspondingly.
Thanks @taranarmo 
I was going to comment that I find this a bit surprising behaviour, and so whether we shouldn't rather consider this as a bug instead of documenting it. 

But from a quick check, it seems that the `dtype` argument in fact is _not_ ignored, at least not in all cases (https://github.com/pandas-dev/pandas/issues/59060 clearly shows there are also cases where it is indeed ignored):

```
In [18]: pd.Series(pd.Series([1.0], dtype=float), dtype=int)
Out[18]: 
0    1
dtype: int64
```

Here it does cast the float series to an int64 dtype, so the `dtype` keyword was not ignored.

Will also comment on the issue, but so it seems the simple "If ``data`` is Series then is ignored." message added in this PR is not fully correct.
IIUC I agree with Joris. The solution is not to change the doc, but to make the behavior consistent. The dtype kwarg should never be silently ignored.
BUG: `query` on columns with characters like # in its name
State: closed

- [x] closes #59285
- [x] closes #49633
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

---
This PR first added parsing.py: https://github.com/pandas-dev/pandas/pull/28215

---
The issue is that none of the libraries `tokenize`, `ast`, or `numexpr` handle # the way we want it to.

`tokenize` is used here: https://github.com/pandas-dev/pandas/blob/18a3eec55523513c5e08fe014646c044cc825fa4/pandas/core/computation/expr.py#L169 https://github.com/pandas-dev/pandas/blob/18a3eec55523513c5e08fe014646c044cc825fa4/pandas/core/computation/parsing.py#L186
`ast` is used here: https://github.com/pandas-dev/pandas/blob/18a3eec55523513c5e08fe014646c044cc825fa4/pandas/core/computation/expr.py#L407
`numexpr` is used here: https://github.com/pandas-dev/pandas/blob/18a3eec55523513c5e08fe014646c044cc825fa4/pandas/core/computation/engines.py#L129


Labels:
name: Bug
name: expressions
description: pd.eval, query

Comments:
@aram-cinnamon for the pre-commit issue you can see if you find anything of note in https://github.com/pre-commit/pygrep-hooks/issues, but I am doubtful that there is any real solve. You may just need to add  `exclude: ^pandas/tests/frame/test_query_eval.py` to the `rst-inline-touching-normal` hook in the .pre-commit-config.yaml file
Thanks so much Will!!
Hm tests still failing. Will look into it soon
Hmm the failure on the `Future Infer Strings` job is kind of tricky, and tangential to what you are doing. For now I would just add the `@pytest.mark.xfail(using_string_dtype(), reason="TODO(infer_string)")` decorator that you see on some other tests
Thanks @aram-cinnamon - very impressive!
Great work!  Had been hoping to see this in the next patch release which was `2.2.3` (released 20-sep-2024), but didn't make it.  Any thoughts when this might make its way into an official release?
It will be part of the 3.0 release
allow set_index to set a multiindex from a tuple, not just a list
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Given the discussion in https://github.com/pandas-dev/pandas/issues/59294, we'll probably not move forward with this so closing
Add fixture docstring for series indexing
State: closed

- [x] closes #19159 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @sliuos
DOCS: Add docstrings to fixtures in /io/json/test_json_table_schema_ext_dtype.py
State: closed

Add docstrings to fixtures in `/io/json/test_json_table_schema_ext_dtype.py` file.

Partially addresses https://github.com/pandas-dev/pandas/issues/19159


Labels:
name: Docs

Comments:
Thanks @ivonastojanovic
BUG: Hash and compare tuple subclasses as builtin tuples
State: closed

- [x] closes #57922
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Relaxing the `PyTuple_CheckExact` in [khash_python.h](https://github.com/pandas-dev/pandas/blob/080add14720218a20ca1dbde5e59d82d05534c5a/pandas/_libs/include/pandas/vendored/klib/khash_python.h#L210-L211) to `PyTuple_Check` would allow the usage of types created with `collections.namedtuple` as index keys everywhere `tuple` is expected.

It is worth noting this would change the behaviour of all client code expecting custom `__eq__` and `__hash__` functions in `tuple` subclasses to be respected.

This also removes the non-deterministic behaviour in cases where `collections.namedtuple` types and `tuple`s are used interchangeably.
I traced the intermittent `KeyError`s to [this part](https://github.com/pandas-dev/pandas/blob/080add14720218a20ca1dbde5e59d82d05534c5a/pandas/_libs/hashtable_class_helper.pxi.in#L1352-L1356) in `PyObjectHashTable`.
I did not verify this, but I suspect the [custom tuple hasher](https://github.com/pandas-dev/pandas/blob/080add14720218a20ca1dbde5e59d82d05534c5a/pandas/_libs/include/pandas/vendored/klib/khash_python.h#L274) leads to a scenario where builtin `tuple`s are hashed using the custom hasher, while all `tuple` subclasses are hashed with their own `__hash__`, resulting in different hash values.
For small hash tables, the resulting index might point to the correct bucket just by coincidence.

A similar issue, but for different types, was reported in https://github.com/pandas-dev/pandas/issues/39585.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Nested Data
description: Data where the values are collections (lists, sets, dicts, objects, etc.).

Comments:
Thanks @matiaslindgren
BUG: Fix precision loss in read_json
State: closed

- [x] closes #20608 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
pre-commit.ci autofix
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Yes please reopen this PR, I have merged the main branch.
Hi @rhshadrach , is there anything else that needs to be changed in this PR
Thanks, I have updated the whatsnew 
Thanks @fawazahmed0!
DOC: Using np.histogram_bin_edges with pd.cut
State: closed

- [x] closes #59165 
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

As per reviews on #https://github.com/pandas-dev/pandas/pull/59241, adding documentation support for using np.histogram_bin_edges with pd.cut to allow bins calculation with strings. 

Labels:
name: Docs

Comments:
@asishm Addressed your comments! 
Thanks @chaarvii
TST: Added test for non-nano DTI intersection and updated whatsnew
State: closed

- [x] closes #59271 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v3.0.0.rst` file if fixing a bug or adding a new feature.

Bug was fixed in #59037 but only references fixing `DTI.union`. Modified whatsnew as well

Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @asishm
DOC:  add See Also section to series.to_dict
State: closed

This PR fixes a numpy docstring validation error described in https://github.com/pandas-dev/pandas/issues/58065 for pandas.Series.to_dict

Docstring validation results
```
################################################################################
################################## Validation ##################################
################################################################################

1 Errors found for `pandas.Series.to_dict`:
	ES01	No extended summary found
```

Labels:
name: Docs

Comments:
Thanks @ktseng4096
DOC: Add fixtures for testing DropDuplicates for datetimelike dataframes
State: closed

Add docstrings to fixtures in `pandas/tests/indexes/datetimelike_/test_drop_duplicates.py` file.
Partially addressed #19159.

- [ ] Closes #xxx
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @kushagr96
Removed SA01 error for pandas.Series.to_frame. Added See Also section…
State: closed

… to pandas.Series.to_frame. Modified ci/codecheks.sh and pandas/core/series.py.

- [ ] closes #58065 
- [ ] Removed SA01 error for pandas.Series.to_frame.
- [ ] Removed pandas.Series.py from ci/code_checks.sh
- [ ] Added See Also section to pandas/Series.py (pandas.Series.to_frame)


Labels:
name: Docs

Comments:
Fixed SA01 error for pandas.Series.to_frame
Sorry about butting in but I figure it was better to comment than put up another PR. I had started looking into this one as well and thought the following would be good to include

```
DataFrame.squeeze: Create a series from a single-column DataFrame.
```
Thanks @Aditya060 
> Sorry about butting in but I figure it was better to comment than put up another PR. I had started looking into this one as well and thought the following would be good to include
> 
> ```
> DataFrame.squeeze: Create a series from a single-column DataFrame.
> ```
Hi, do you mean to include this in the See Also section of series.to_frame?

> > Sorry about butting in but I figure it was better to comment than put up another PR. I had started looking into this one as well and thought the following would be good to include
> > ```
> > DataFrame.squeeze: Create a series from a single-column DataFrame.
> > ```
> 
> Hi, do you mean to include this in the See Also section of series.to_frame?

Yea I thought including the inverse of the function would be helpful to some people.
BUG: pandas-dev#58594
State: closed

- [X] closes #58594 
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
No errors raised for all cases noted in https://github.com/pandas-dev/pandas/issues/58594.

```
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> empty_df = pd.DataFrame([])  # works
>>> df = pd.DataFrame([], index=empty_df.index)  # works
>>> df = pd.DataFrame([], index=[])  # works
>>> df = pd.DataFrame.from_records([])  # works
>>> df = pd.DataFrame.from_records([], index=empty_df.index)  # works now!
>>> df = pd.DataFrame.from_records([], index=[])  # works now!
```
Thanks @kirill-bash
CI/DOC: Fix CI failure for Status: Draft PDEPs and don't show them on the roadmap webpage
State: closed

- [x] closes #59253

This solution will treat any Draft PR as a `Status: Draft` PDEP (and therefore not show it on the roadmap page)
Other solutions will likely complicate the code too much.

Labels:
name: Web
description: pandas website

Comments:
Makes sense. Thanks @Aloqeely
Remove extra space in ``resample`` freq deprecation
State: closed

While running `dask` tests against the nightly version of `pandas` I noticed there's an extra space in the resample frequency deprecation warning (note the extra space before "instead"):

```
FutureWarning: 'd' is deprecated and will be removed in a future version, please use 'D'  instead.
```

This PR just removes the extra space. 

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Warnings
description: Warnings that appear or should be added to pandas

Comments:
Thanks @jrbourbeau
TST: unskip EA reduction tests
State: closed

```
> pytest -k reduce pandas/tests/extension/

# main
3055 passed, 886 skipped, 143 xfailed

# PR
3216 passed, 782 skipped, 86 xfailed 
```

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @lukemanley
BUG: `pandas.tseries.frequencies.to_offset()` raises `ValueError` when parsing a `LastWeekOfMonth` frequency string
State: closed

- [x] closes #59218
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Frequency
description: DateOffsets

Comments:
thanks @aram-cinnamon , could you add a whatsnew note please?
> thanks @aram-cinnamon , could you add a whatsnew note please?

I can only imagine how often people forget, how often you have to remind them, and how patient you have to be :joy:
BUG: Fix 58807
State: closed

- [x] closes #58807 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
Thanks @matiaslindgren