BrandonFors/pandas-github-issues huggingface dataset:
FIX: Add pre-commit Blacken-docs hooks
State: closed

- [x] Part of  #46865 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This PR adds the configuration for the `blacken-docs` hook to the project's `.pre-commit-config.yaml` file. The `blacken-docs` hook applies the Black code formatter to reStructuredText (reST) files.

To accommodate potential parsing errors encountered with certain code blocks, the `--skip-errors` argument is included in the hook configuration. This argument instructs `blacken-docs` to skip errors and continue processing other files and blocks.

By including the `blacken-docs` hook with error skipping, we can ensure consistent formatting of reST files while allowing exceptions for code blocks that may contain unsupported syntax.

Please review this PR and provide any feedback or suggestions for improvement.

@noatamir @phofl SciPy Sprint 2023



Labels:
name: Stale
name: Sprints
description: Sprint Pull Requests

Comments:
Thanks for the PR! By putting `partially closes #...` in the OP, I believe merging this will automatically close the corresponding issue. This is because GitHub does not pickup on the "partially". Instead, I recommend "Part of #..."
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Update the double quotes for the NA value to single quotes
State: closed

- [x] closes #54107 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs
name: Sprints
description: Sprint Pull Requests

Comments:
Hi @Tianye-Song, thanks for taking this on! Just a note for when you make this update that there are other functions that use this parameter whose docstrings may also need to be updated. 

Also, I noticed the title you have here is about adding single quotes instead of double quotes; however, I think the idea was to add double back-ticks to add code style formatting so that each is clearly a string literal. That may be what you meant but just wanted to confirm. Thanks again!
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/54136/
thx @Tianye-Song
BUG: groupby shift fill_value, freq followup
State: closed

- [ ] closes #54093 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Follow up to https://github.com/pandas-dev/pandas/pull/53832


Labels:
name: Groupby
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
Green now, `_reindex_with_indexers` needed to be passed a fill_value of None as opposed to no_default
Thanks @mroeschke
BUG: DataFrame repr with ArrowDtype with extension
State: closed

- [x] closes #54062 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


I _think_ it makes sense for the python `.type` of a pyarrow ExtensionType to be its storage's `.type`.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: DataFrame
description: DataFrame data structure
name: Arrow
description: pyarrow functionality

Comments:
Might also close #54063?
> Might also close #54063?

I'll leave that open for now. I interpreted that issue as maybe there's a fix that could be backported to 2.0.4 (but I am not sure that there is)
BUG: groupby.count maintains masked and arrow dtypes
State: closed

- [x] closes #53831 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays
name: Arrow
description: pyarrow functionality

Comments:
Looks like tests are passing here so merging. Happy to follow up if needed
Add ASCII art script
State: closed

- [x] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
has this been discussed somewhere?  why would we want this?  
Thanks but no thanks. If interested in contributing, feel free to tackle issues labeled `good first issue`
thank you, i appreciate it
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

It's not possible to make the html file for each method before opening the PR:
/preview needed. 

Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon
DOC: Fixing EX01 - Added example
State: closed

- [ x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

~~Modified `validate_docstrings.py`: it was raising for "importing pandas". If I didn't import pandas_gbq on the docstring, the error was for undeclared name.~~

~~Not sure if I have to add a test for that case or if the fix is good enough.~~


Labels:
name: Docs

Comments:
Looks good. Could you merge main once more?
It looks green if someone very kind wants to merge, it would be great. 

Thanks @DeaMariaLeon
DOC: Update to Documentation - Grammatical Updates and Typo Corrections
State: closed

Typo correction and grammatical updates on the "Pandas Documentation" page (https://pandas.pydata.org/docs/). 

Title: 
pandas documentation -> Pandas Documentation
Capitalization of title.

The text below "Useful links":
The word "pandas" is not capitalized. Since it is the first word of a sentence, it must be capitalized. 
Grammatical update to the whole sentence. 

No links were changed.

Before changes:
![before](https://github.com/pandas-dev/pandas/assets/88173271/2cab605e-4e5d-4cbc-b8fb-5cac049d4f90)

After Changes:
![after](https://github.com/pandas-dev/pandas/assets/88173271/80e08d14-cce2-4000-9ec3-6555b66f6063)

Labels:

Comments:
Thanks but the lowercasing is intentional. See https://pandas.pydata.org/about/citing.html
Would it be possible for me to send another PR without any capitalized words? 
IMO the sentence you edited is fine as is, but happy to have contributions towards issues labeled `good first issue`
CoW: Copy less in replace implementation with listlikes
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is the best version I could come up with



Labels:
name: Copy / view semantics

Comments:
> This is the best version I could come up with

General concern: do we think this is worth the added complexity? (and the replace code is already _really_ complex ..)

The case it is trying to solve is that if you do an inplace replace with a list like, this could be done actually inplace (if there are no other refs to start with), while now it always makes a copy?
This is generally for listlike replace ops, not only inplace. The current implementation copies for every pass, which is too slow imo
> The current implementation copies for every pass, which is too slow imo

And does that only happen under CoW, or also in the default?  

Both as far as I know, but we should fix this anyway
Also agreed to merge
ENH: .shift optionally takes multiple periods
State: closed

Taking over from https://github.com/pandas-dev/pandas/pull/44660/files

- [x] closes #44424
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Transformations
description: e.g. cumsum, diff, rank

Comments:
I think the pyarrow failure is unrelated? Otherwise green.
Ok now it's all green.
All green!
@rhshadrach @mroeschke I originally wanted this functionality to build simple Almon distributed lags models (or the [corresponding thing from cognitive neuroscience](https://mne.tools/dev/auto_examples/stats/linear_regression_raw.html)), would there be a room for a Pandas docs example for this? Basically showing how an impulse response function can be recovered via regression of a multi-lagged feature.

E: maybe this PR isn't the space for this, I can also move to an issue
Done - @rhshadrach everything ok from your end?
Thanks @jona-sassenhagen! Great work!
Thanks @rhshadrach !

May I re-raise this: https://github.com/pandas-dev/pandas/pull/54115#issuecomment-1649676735
> May I re-raise this: [#54115 (comment)](https://github.com/pandas-dev/pandas/pull/54115#issuecomment-1649676735)

I recommend making an issue. There a few places that come to my mind:

 - An example in the User Guide
 - [The cookbook](https://pandas.pydata.org/docs/user_guide/cookbook.html)
 - [The ecosystem](https://pandas.pydata.org/community/ecosystem.html)

A user guide example would need to be (a) self-contained and (b) not too long to be appropriate; the cookbook should be for generic operations (no domain context); and the ecosystem for packages. I'm not sure exactly what you're thinking of, but it sounds like it may not fit in any of these. But we can discuss further in an issue.
Thanks, I will do that.
CoW: Avoid unnecessary writeable=False ops
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Still planning on working on this?
Closing to clear the queue, but feel free to reopen when ready
CI: Fix pyarrow nightly build
State: closed

EDIT: Looks like it was a stale cache issue. After deleting the cache (which the weekly job should help going forward), looks like pyarrow dev has an error

Labels:
name: CI
description: Continuous Integration
name: Arrow
description: pyarrow functionality

Comments:
Tracking the change in double to temporal casting in https://github.com/apache/arrow/issues/36677
Tests seem to be passing and the casting workarounds seem to be the way forward with https://github.com/apache/arrow/issues/36677 so merging.
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 db27c36272fc3d7ce76cfecbeaa1eac704c2644a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #54110: CI: Fix pyarrow nightly build'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-54110-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #54110 on branch 2.0.x (CI: Fix pyarrow nightly build)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
ENH: support argmin/max, idxmin/max with object dtype
State: closed

- [x] closes #4279
- [x] closes #18021
- [x] closes #40685
- [x] closes #43697
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Agreed upon in yesterday's call.

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Thanks @jbrockmendel
DOC: Organize README.md badges
State: closed

Inspired by https://github.com/pypa/hatch/blob/master/README.md

Labels:
name: Docs

Comments:
Thanks! Is there something weird happening with the line breaks maybe maybe? I see this when I check your branch:
<img width="1038" alt="Screenshot of the rendered table with badges. There's a weird line break" src="https://github.com/pandas-dev/pandas/assets/6564007/57b19b96-3778-4968-b8dd-b655e711234b">

> Thanks! Is there something weird happening with the line breaks maybe maybe? I see this when I check your branch: 

Ah I didn't know how to preview the markdown. Thanks, I was able to fix the line break issue


thx @mroeschke
Update multiple instances of `dtype_backend` parameter description to make them all consistent
State: closed

- [x] closes #53878
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Note that this supersedes PR #53881.

Labels:
name: Docs

Comments:
/preview
Website preview of this PR available at: https://pandas.pydata.org/preview/54104/
TST: Add test for concat multiindex with category
State: closed

https://github.com/pandas-dev/pandas/pull/53697#issuecomment-1633340930

cc @phofl


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: MultiIndex
name: Categorical
description: Categorical Data Type

Comments:
Thanks @lukemanley 
Thx
BUG: Timestamp(pd.NA) raising TypeError
State: closed

- [x] closes #45481
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Is it obvious we want to allow this?  If we consider pd.NA as meaningfully distinct from pd.NaT, then silently converting would be wrong
I would've thought so - the docs say

> For datetime64[ns] types, NaT represents missing values

and `pd.NA` is what signals missing values. Will check what's done in other cases, not really sure
> I would've thought so - the docs say [...] and pd.NA is what signals missing values. Will check what's done in other cases, not really sure

The meaning of "missing" has changed since that doc was written.  In particular NaT has nan-like semantics.
> Is it obvious we want to allow this? If we consider pd.NA as meaningfully distinct from pd.NaT, then silently converting would be wrong


I dont have a strong opinion but I'll note:

1. Other na-likes already get converted to NaT (e.g. None -> NaT, NaN -> NaT)

2. The current behavior does seem inconsistent:

```
pd.to_timedelta(pd.NA)  # -> NaT
pd.to_datetime(pd.NA)   # -> NaT
pd.Timedelta(pd.NA)     # -> NaT
pd.Timestamp(pd.NA)     # -> TypeError
```

> The current behavior does seem inconsistent:

fair enough.  no objection here.
This _might_ cause issue if we every introduced a nullable numpy datetime/timedelta types with `pd.NA` as the missing value, but that's not planned right?
> This might cause issue if we every introduced a nullable numpy datetime/timedelta types with pd.NA as the missing value, but that's not planned right?

Does the numpy-nullable part matter here?  timestamp[pyarrow] already behaves that way


> Does the numpy-nullable part matter here? timestamp[pyarrow] already behaves that way

Sure, I was just confirming that there wasn't appetite to have BaseMaskedArray support datetime/timedelta types for example 
Thanks @lukemanley
REF: rename PandasArray->NumpyExtensionArray
State: closed

- [x] closes #53694 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I'm open to suggestions on names.

Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
Thanks @jbrockmendel
DOC: point out the limitation of precision while doing serialization
State: closed

- [x] closes #38437

Updated docstring for `to_json`, pointed out that the maximal possible value for `double_precision` is equal 15.


Labels:
name: Docs
name: IO JSON
description: read_json, to_json, json_normalize

Comments:
Hi @phofl. I made a small documentation improvement: pointed out in `to_json` docstring, that the maximal possible value for the `double_precision` is `15`. Could you please take a look at my pr?
BUG: groupby.resample(kind="period") raising AttributeError
State: closed

- [x] closes #24103
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Resample
description: resample method

Comments:
Thanks @lukemanley
DEV: Improves Error msg when constructing a Timedelta
State: closed

- [x] closes #54059 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Looks good to go @mroeschke / @MarcoGorelli  
Thanks @mcgeestocks