BrandonFors/pandas-github-issues huggingface dataset:
BUG: Timestamp unit with time and not date
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
@MarcoGorelli this was the root cause of something you were looking at a few weeks ago, but i dont remember what the motivating issue was
CLN: Replace confusing brackets with backticks
State: closed

There are some errors which read like:

> only list-like objects are allowed to be passed to isin(), you
> passed a [str]

(note how the type `str` is enclosed in square brackets)

This is potentially confusing, since some languages use notation like `[TYPE]` to denote a list of TYPE objects. So the user could read the error message saying they can't use `[str]`, but intepret `[str]` to mean "a list of str objects".

This commit removes the possibility of confusion by using backticks instead of square brackets.

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @beyarkay 
Thanks for the review!
DOC: Update README.md to use URL in documentation link.
State: closed

- [x] closes #54089 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @fgo
DOC: Try out sphinx_remove_toctrees
State: closed

- [ ] closes #54045 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/54088/
Thanks for the suggestion @jorisvandenbossche. Looks like the docbuild went from ~1 hour 14 mins to 8 mins! 

It appears the difference is that `Section Navigation` no longer has the collapsible page toctree (as expected), but I think that's okay as it feels redundant with the main content on the page  
Not sure if I like this, I am regularly browsing the submenu
Can we maybe have them generate on releases only?
> Can we maybe have them generate on releases only?

Since the dev docs are released on every push to main, I don't think it's worth the added complexity to only generate them sometimes


Will leave this open for a little bit, but if there is no further support for this I guess I'll close
BUG: merge cross with Series
State: closed

- [x] closes #54055 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Thanks @jbrockmendel
TYP: update mypy and small pyi fixes from ruff
State: closed

There is a much newer version of pyright but I haven't yet managed to get it to pass.

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
> This looks good to me. Will let others approve/merge.

Would still need to decide whether we prefer to disable the ruff/pylint rule or add ignore comments for pyright. Currently, I disabled the rule for ruff but not yet pylint.
I would slightly prefer enabling the pylint/ruff rules and adding pyright ignores, if there are not too many ignores needed
> I would slightly prefer enabling the pylint/ruff rules and adding pyright ignores, if there are not too many ignores needed

Using the pyright comments now
Thanks @twoertwein
DOC: Update merge copy parameter code and doc
State: closed

- [ ] closes #4599 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
I have checked merge copy related codes, found that following codes will decide if copy:
```
pandas/pandas/core/internals/concat.py:229
        if needs_copy and not indexers:
            mgr = mgr.copy()
```
The needs_copy parameter is defined by user input, CoW status and concat axis.The indexers parameter is defined by following codes:
```
pandas/pandas/core/reshape/concat.py:666
            for obj in self.objs:
                indexers = {}
                for ax, new_labels in enumerate(self.new_axes):
                    # ::-1 to convert BlockManager ax to DataFrame ax
                    if ax == self.bm_axis:
                        # Suppress reindexing on concat axis
                        continue

                    # 1-ax to convert BlockManager axis to DataFrame axis
                    obj_labels = obj.axes[1 - ax]
                    if not new_labels.equals(obj_labels):
                        indexers[ax] = obj_labels.get_indexer(new_labels)

                mgrs_indexers.append((obj._mgr, indexers))
```
I don't know why new_labels can be not equal to obj_labels. They are both from objs, although there are some operations in self.new_axes function. I have tried several examples, but can't get an indexers value which is not None.

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: checking for value type when parquet is partitioned
State: closed

- [ ] closes #53951 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x]  Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Instead of checking pyarrow array's type to be string, it could be a dictionary of values with string type when partitioned. Performed checks for both cases.


Labels:
name: IO Parquet
description: parquet, feather
name: Arrow
description: pyarrow functionality

Comments:
Thanks @SanjithChockan
TST: fix test broken on main
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @jbrockmendel
Revert "BUG: DataFrame.stack sometimes sorting the resulting index"
State: closed

Closes #53969

Reverts pandas-dev/pandas#53825


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Regression
description: Functionality that used to work in a prior pandas version

Comments:
I plan on merging this tomorrow to start making progress on #53921
Simplify pandas footer 
State: closed

- [X] closes #51536  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Web
description: pandas website
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: constant check in series
State: closed


- [X] closes #54033
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @sbrugman
DOC: Add security policy
State: closed

- [ ] closes #8545 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Thanks @liang3zy22
BLD: Shrink sdist/wheel sizes
State: closed

- [ ] closes #53224 (Replace xxxx with the GitHub issue number)
- [ ] closes #50302 (this should also fix that)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Build
description: Library building on various platforms

Comments:
Still half a meg bigger on all platforms, sadly, but basically the same now.

I am building from the sdist again, though. Let's hope the Github bug doesn't strike again.
(may have to build a sdist in every build that happens)
Nice thanks @lithomas1
CI: Use PYTEST_WORKERS=0 instead of 1 for single cpu runtime
State: closed

It appears `-n 1` with pytest-xdist will spawn a worker and run a test on the single worker while `-n 0` avoids that extra work of spawning a worker and still runs the tests in a "single cpu" runtime

Labels:
name: CI
description: Continuous Integration

Comments:
Looks like this shaves off significant time off the windows builds which is nice. Going to merge but happy to follow up.
STY: Use ruff instead of flake8 to lint docstring examples
State: closed


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
`ruff` assumes you're running `black`, which we can't run easily on docstrings because of all the docstring sharing (😢 ), so it doesn't flag some things because it assumes `black` will already have formatted them for you

do we still want to do this?
Ah I didn't know black was a prereq for ruff. ruff was catching errors when using our `pyproject.toml` configuration, but I realized the docstring examples were not formatted as rigorously. The main motivation for this change was the reduce the time it takes for our `docstring validation` check (12 mins vs 4 mins). 
wow that's really noticeable

OK sure let's do this
from the CI logs

> E           assert "flake8 error: line 1, col 33: E231 missing whitespace after ','" in 'No summary found (a short summary in a single line should be present at the beginning of the docstring) No extended summary found See Also section not found'

so this is the kind of thing which ruff would no longer be catching
Ah darn. I guess this is not entirely equivalent then so closing
DOC: Fix typos in groupby user guide
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

-----

Very trivial changes! The capitalisation of UDF matches that used elsewhere in the docs.

Labels:
name: Docs

Comments:
Nice catches, thanks @penelopeysm
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

We need to preview here as html documents for `Styler` can't be done locally.  

Labels:
name: Docs

Comments:
/preview
No preview found for PR #54039. Did the docs build complete?
looks like the docs build is failing
```
/home/runner/work/pandas/pandas/pandas/io/formats/style.py:docstring of pandas.io.formats.style.Styler:134: WARNING: undefined label: 'style.html'
/home/runner/work/pandas/pandas/pandas/io/formats/style.py:docstring of pandas.io.formats.style.Styler.from_custom_template:48: WARNING: undefined label: 'table visualization for more examples <style.html.subclassing'
```
Is this my fault - I wouldn't think so but I can't be sure.
```/home/runner/micromamba/envs/test/lib/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~```
that's unrelated 😄 I think this one is more related
```
/home/runner/work/pandas/pandas/pandas/io/formats/style.py:docstring of pandas.io.formats.style.Styler:133: WARNING: Mismatch: both interpreted text role prefix and reference suffix.
/home/runner/work/pandas/pandas/pandas/io/formats/style.py:docstring of pandas.io.formats.style.Styler.from_custom_template:48: WARNING: Mismatch: both interpreted text role prefix and reference suffix.
```
Could you merge in main once more here?
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/54039/
BUG: Fix failing hash function for certain non-ns resolution `Timedelta`s
State: closed

I noticed that hashing certain large `pd.Timedelta`s resultet in an `OutOfBoundsTimedelta` exception, which shouldn't happen.

In the hash implementation there is seems to be an attempt at catching this error, as the offending line is wrapped in a `try` clause, but, the `except` clause catches an `OverflowError` instead. The call that raises `OutOfBoundsTimedelta` does so when itself catches an `OverflowError`, so my interpretation is that the intention was for the hash implementation to catch the `OutOfBoundsTimedelta`. My changes reflect this. I've also added a test reproducing the issue.

- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @harahu
TST: Add test for Timedelta hash invariance
State: closed

Adding an xfailing test to reproduce and document #44504.

Ideally the test could cover an even wider set of comparisons, but I think this is a good start.

- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Timedelta
description: Timedelta data type

Comments:
Thanks @harahu
Implement Pydantic V2 protocol
State: closed

- [X] closes #53999
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

1. What's the best file to add a test for this change? 😅
2. Should I implement the one for `pd.DataFrame` as well on this PR?

Labels:
name: Stale

Comments:
thanks for your PR - could you show an example of how users would use this?

As in:
- how they can use this now
- how this PR changes/improves that

As for tests, maybe `test_downstream.py`?
> If [#53999 (comment)](https://github.com/pandas-dev/pandas/issues/53999#issuecomment-1625289148) is true, then adding the pydantic protocol in pandas directly isn't a good idea if the protocol could differ on the use case

There was a miscommunication. This is what I meant: https://github.com/pandas-dev/pandas/issues/53999#issuecomment-1627346045 
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
closing as it looks like this is no longer active

but I'm interested in the idea - if explain it a bit more clearly, then am open to discussing this further
It's active, and this is on my TODO list. 🤷‍♂️
DOC: add backticks to docstrings
State: closed

- [ ] Issue DOC: Inconsistent use of code-style formatting (backticks) in docstrings #53674
- [ ] Issue originally referenced only read_csv and read_excel but open to 
all docstrings to update this formatting for consistency

This PR doesn't address all backsticks needed in pandas/io/xml.py but want to get the ball rolling to see if I'm headed in the right direction. 

Labels:
name: Docs

Comments:
Thanks @wcgonzal 
Thank you, will  continue working on this.
BUG: fixes #53935 Categorical order lost after call to remove_categories 
State: closed

- [x] closes https://github.com/pandas-dev/pandas/issues/53935
- [x]  [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
 - [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
 - [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
 - [x] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

I simply put an if/else in the `remove_categories` method.  If `ordered=True` it will pass `sort=False` into the `difference1 method.



Labels:
name: Categorical
description: Categorical Data Type

Comments:
Thanks @paulreece 
Great working with you, @mroeschke!