BrandonFors/pandas-github-issues huggingface dataset:
REF: separate Block.pad_or_backfill from interpolate
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Thanks @jbrockmendel
TST: add the parameter MonthEnd(2) to 2 tests with freq= 2M
State: closed

xref #52064

added parametrization to 2 tests in which we pass `freq = '2M'`  to check if they work correctly not only with str, but with offsets (`MonthEnd(2)`) as well.

Labels:
name: Frequency
description: DateOffsets
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
@MarcoGorelli, could you please take a look at this pr?
Revert addition of 'None' to default na_values of the parser
State: closed

Running the tests for https://github.com/pandas-dev/pandas/issues/52493

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO CSV
description: read_csv, to_csv
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
It appears from https://github.com/pandas-dev/pandas/issues/52493#issuecomment-1699205364 that we're moving forward with this behavior, so going to close for now
REF: use CoW helpers more
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Copy / view semantics

Comments:
@phofl in Block._maybe_copy there is a path where we can pass deep=False to Block.copy.  i think we could simplify things if instead we just returned `self` there.  would that be OK?
I’ll have to look into this more, there are scenarios where this could be problematic. This ties a bit into my todo list that’s related to replace. Might get to this over the weekend or next week 
> there are scenarios where this could be problematic

Yah, just remembered there are a couple places where we set `block.values` post-`__init__`.  Refactoring to avoid those might make that safer.  I don't think this PR should be affected though.
i think the helper here should be OK?
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875


Labels:
name: Docs

Comments:
```
Error: /home/runner/work/pandas/pandas/pandas/core/resample.py:1106:RT01:pandas.core.resample.Resampler.max:No Returns section found
Error: /home/runner/work/pandas/pandas/pandas/core/resample.py:1076:RT01:pandas.core.resample.Resampler.min:No Returns section found
```
I hope it's good now. Sorry.
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

Note: the examples for `pandas.HDFStore.groups` and `pandas.HDFStore.walk` were done [here](https://github.com/pandas-dev/pandas/pull/53889).


Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon
ENH: Support scalar condition in `Series.where` and `DataFrame.where`
State: closed

- [x] closes #53903
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
    - added/updated tests in `series/indexing/test_where.py`, `series/indexing/test_where.py`, `series/indexing/test_mask.py`
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
  - Regarding documentation, I am not sure anything needs to be updated since both Series.where and DataFrame.where specify array-like as a valid input type, and scalars are considered array-like.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.1.0.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
@jbrockmendel Any other objections?
> Any other objections?

Not about the implementation, I'm just not sure I want to support this.  In general we're trying to move towards being stricter in signatures.  But not strong enough an opinion to cause a stink here.
Sounds like this feature needs more discussion and buy in in the original issue so going to close for now awaiting that decision. Can reopen once that has been done
DOC: fix asv test results link in contributors doc
State: closed

- [x] Closes #53851
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)


Labels:
name: Docs

Comments:
thx @Charlie-XIAO
CI: linting check to ensure `lib.NoDefault` is only used for typing
State: closed

- [x] See https://github.com/pandas-dev/pandas/pull/53877#issuecomment-1609917156
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions

In this PR, I added a pre-commit hook to check that `pandas._libs.lib.NoDefault` is used only for typing. This is invoked by #53877. Before that PR, this new hook correctly catches all misused `NoDefault`:

<details>
<p></p>

```
(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Failed   
- hook id: unwanted-patterns-nodefault-not-used-for-typing
- exit code: 1

pandas/tests/reshape/test_pivot_multilevel.py:38:NoDefault is not used for typing       
pandas/tests/reshape/test_pivot_multilevel.py:75:NoDefault is not used for typing       
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/reshape/pivot.py:514:NoDefault is not used for typing
pandas/core/reshape/pivot.py:515:NoDefault is not used for typing
pandas/core/reshape/pivot.py:529:NoDefault is not used for typing
pandas/core/reshape/pivot.py:525:NoDefault is not used for typing
pandas/core/reshape/pivot.py:530:NoDefault is not used for typing
pandas/core/reshape/pivot.py:535:NoDefault is not used for typing
pandas/core/reshape/pivot.py:542:NoDefault is not used for typing
pandas/core/reshape/pivot.py:572:NoDefault is not used for typing
```

</details>

After that PR, the check passes normally:

<details>
<p></p>

```
(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Passed
```

</details>

In fact I don't work with ASTs very often so not sure if this check covers all possibilties. Please let me know if any modification is needed. (The runtime is a bit long but seems acceptable, approximately 10s to check all files.)


Labels:
name: Code Style
description: Code style, linting, code_checks
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
nice idea

I think you're repeatedly walking up and down the same paths - do you want to check `scripts/no_bool_in_generic.py`? that one checks for annotations, maybe there's some logic there that can be reused?
Improved from approx. 10s for all files to approx. 4s @MarcoGorelli Thanks for your suggestion!
Well done!

Would appreciate it if we could add a tiny little test case to scripts/tests/test_validate_unwanted_patterns.py too

Other than that, looks good!
Didn't notice that there are tests for linting scripts, thanks for the reminder @MarcoGorelli. Added some simple tests now.
Nice thanks @Charlie-XIAO
DOC: Update release date for 2.0.3
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Docs

Comments:
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 b01dc6205f7e9a582181956b0854c5feaf4282f1
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #53897: DOC: Update release date for 2.0.3'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-53897-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #53897 on branch 2.0.x (DOC: Update release date for 2.0.3)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
BUG: read_parquet from public parquet file with AWS credentials in environment gives OSError
State: closed

- [ ] [Tests added and passed](https://github.com/pandas-dev/pandas/issues/53701 if fixing a bug or adding a new feature

Added the retry mechanism found in common.py([here](https://github.com/pandas-dev/pandas/blob/0bc16da1e53e9a4d637cf81c770e6517da65a029/pandas/io/common.py#L395-L397)) for file handling when read_parquet gets invoked

Fixes the bug when invalid credentials are put in to access public files


Labels:
name: Stale

Comments:
> Instead of copy pasting ideally this should reuse the same code

How would you recommend going about this then? Should I create a new method that contains this piece of code and import it into functions that use this?
@mroeschke I've added a helper method in the PyArrowImpl to reuse code to make sure the read_parquet goes through a second round with anon=True in storage options.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: join changes index type when doing join on empty dataframes
State: closed

- [X] closes #52777
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
Looks good, but since 2.0.3 going out today, lets wait for this to go into 2.0.4
whatsnew for v2.0.4 isn't in the repo yet. Should I just move this to v2.1.0?
Yeah let's just have this fix for 2.1
👍. I've moved it.
Thanks again @topper-123
TST: add test with freq=2M for period_array from datetime64
State: closed

xref #52064

added a test in `pandas/tests/arrays/period/test_constructors.py`, which covers a case `freq='2M'` for period_array from datetime64. The reason: existing tests were all for `"freq='M'"`,  rather then for `"freq='2M'"`.

Labels:
name: Period
description: Period data type
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
@MarcoGorelli, could you please take a look at this pr? Seems like ci failures aren't related to my changes.
DOC: Fixing EX01 - Added examples
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Towards https://github.com/pandas-dev/pandas/issues/37875

Note: I'll remove them from code_checks tomorrow to avoid merge conflict with a PR being tested now.

Labels:
name: Docs

Comments:
Thanks @DeaMariaLeon
REF: de-duplicate methods, remove unused kwd
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Thanks @jbrockmendel
REF: helpers to de-duplicate CoW checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Can do a little bit more along these lines, but am splitting it into smaller chunks bc the refactored helpers are not perfect cut/paste.

Labels:
name: Refactor
description: Internal refactoring of code
name: Copy / view semantics

Comments:
Thanks @jbrockmendel 
Thx for cleaning this up!
CLN: use `lib.no_default` instead of `lib.NoDefault` in `.pivot`
State: closed

- [x] Closes #53826
- [x] *No additional tests needed, it suffices to require all existing tests to pass*
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
- [x] *I assume that this does not need a changelog entry*


Labels:
name: Clean

Comments:
Nice one thanks @Charlie-XIAO (if you can think of a linting check to ensure `lib.NoDefault` is only used for typing, that would be a bonus)
REF: de-duplicate _gotitem helper
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
```
mypy.......................................................................Failed
- hook id: mypy
- duration: 225.77s
- exit code: 1

pandas/core/resample.py:1388: error: "_GroupByMixin" has no attribute "_infer_selection"; maybe "_selection"?  [attr-defined]
Found 1 error in 1 file (checked 1418 source files)
```
Thanks @jbrockmendel
CLN: unused args in pad_or_backfill
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean

Comments:
Thanks @jbrockmendel
BUG: merge with mismatched index dtypes failing to raise
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @jbrockmendel 
just a question: I believe this change alters edge case behavior of running pd.merge_asof(df, pd.DataFrame(), left_index=True, right_index=True) for a df with a datetime index.

pandas version 2.0.3 returns df for the above function call (instead of raising).

wondering whether this edge behavior ought to be preserved or if there is a better work around to potentially merging a non-empty with and empty df.
TST: increase coverage of asfreq
State: closed

In order to  increase coverage of `asfreq` a test is added, in which we pass `freq = '2M'` to `asfreq`. There's no test which has a number with the frequency yet.

Labels:
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
@MarcoGorelli, could you please take a look at this pr? Seems like ci failure isn't related to my changes.
just to clarify for anyone passing by - the issue is that existing tests were all like "freq='M'", rather than "freq='2M'". adding such a test increases our confidence in https://github.com/pandas-dev/pandas/pull/52064
REF: share methods AM/BM
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Internals
description: Related to non-user accessible pandas implementation

Comments:
Thanks @jbrockmendel
BUG: to_sql fails for pyarrow date dtype (#53854)
State: closed

- [x] closes #53854
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO SQL
description: to_sql, read_sql, read_sql_query
name: Arrow
description: pyarrow functionality

Comments:
Thanks @NMAC427