BrandonFors/pandas-github-issues huggingface dataset:
Fix groupby-resample KeyError when resampling on Index and giving explicit list of columns.
State: closed

- [x] closes #50840 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Resample
description: resample method

Comments:
Related #50841 
This PR is for the main branch, #50841 is for a fix to the 1.5.x branch. 
@mroeschke, here are  the same changes, but on the main branch.
Let me know if there is anything else you need from me.
There is one failing check, but from the error log I think this has nothing to do with my changes. Can anyone comment on why Ubuntu / Numpy Dev is failing? [link](https://github.com/pandas-dev/pandas/actions/runs/3964425510/jobs/6793275375)
Sure, I'll just revert the last commit.
@mroeschke , I was typing up that question as I didn't see `unittest` used anywhere else in the project, but you were super quick on the review :)
Is there anything else I should do for this PR or can I consider my portion of the work done?
Thanks @czroth
TST: Add test for to_dict converting masked to python types
State: closed

- [x] closes #34665 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
DOC: improve Timestamps docstrings
State: closed

- [x] xref #27977
- ~~[Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development~~/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ~~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~~
- ~~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~~

Add _Returns_ and _See Also_ sections:

- is_year_start
- is_year_end
- is_leap_year
- day_of_week
- day_of_year
- quarter
- week
- days_in_month

Labels:
name: Docs
name: Timestamp
description: pd.Timestamp and associated methods

Comments:
Thanks @abonte
CLN: remove redundant latex options
State: closed

after #48970


Labels:
name: IO LaTeX
description: to_latex
name: Styler
description: conditional formatting using DataFrame.style

Comments:
There is one test here that checks the default for ``escape`` is True. Since that has specifically changed here, I wonder if that test should be removed or changed to test that the default for ``escape`` is now False.
test failure seems unrelated, all other builds seem to pass.
would be good if this can be reviewed for 2.0. to supplement the 2.0 to_latex changes.
Thanks @attack68
BUG: to_datetime with Y or M unit not matching Timestamp
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug

Comments:
Thanks @jbrockmendel
REF: simplify array_to_datetime
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
Thanks @jbrockmendel
BUG: Fix agg ingore arg/kwargs when given list like func
State: closed

- [x] closes #50624  (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Previous [Pull Request #50725](https://github.com/pandas-dev/pandas/pull/50725), for multiple elements-func list this time.

I would appreciate any feedback or suggestions on this draft pull request to help improve it. Thank you in advance for your help and cooperation.

Labels:
name: Apply
description: Apply, Aggregate, Transform, Map

Comments:
cc @rhshadrach, I've reworked the implementation, any suggestions for improvement?
Thanks @luke396!
> Thanks @luke396!

This really couldn't have been done if it wasn't for your great help.
STYLE: remove absolufy imports, use ruff
State: closed

- [ ] closes #50812 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Checks are failing, looks like this didn't work - did you manage to run it locally? 
Locally I run with this command and it failed
`pre-commit run ruff --all-files`
Exactly, could you fix up the errors please? 
looks like this has gone stale - closing for now so others can work on it, but thanks for your PR! if you want to continue, please open a new pull request
BUG/PERF: Series(category).replace
State: closed

- [x] closes #50872
- [x] closes #46884
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

Redo of #49404 with fix that was reverted in #50848

Labels:
name: Bug
name: Performance
description: Memory or execution speed performance
name: Categorical
description: Categorical Data Type

Comments:
Thanks for resubmitting @lukemanley 
Could this have caused a performance regression? https://asv-runner.github.io/asv-collection/pandas/#categoricals.RemoveCategories.time_remove_categories
REF: Use * syntax to make reindex kwargs keyword only
State: closed

After enforcing #17966, this makes `DataFrame.reindex` effectively keyword only except for `labels`, so removed ad-hoc logic to enforce that. `Series.reindex` also has similar ad-hoc logic which can be removed.

Also did some docstring cleanup for both functions.

Labels:
name: Refactor
description: Internal refactoring of code

Comments:
docstrings left unchanged?
> docstrings left unchanged?

Yea but slightly improved. Went from

```
keywords for axesarray-like, optional

    New labels / index to conform to, should be specified using keywords. Preferably an Index object to avoid duplicating data.
```

to more specifically what is used e.g. `DataFrame.reindex`

```
labels : array-like, optional
    New labels / index to conform the axis specified by 'axis'.
index : array-like, optional
    New labels for the index. Preferably an Index object to avoid
    duplicating data.
columns : array-like, optional
    New labels for the columns. Preferably an Index object to avoid
    duplicating data.
axis : int or str, optional
    Axis to target. Can be either the axis name ('index', 'columns')
    or number (0, 1).
```
Going to merge, but can follow up if needed
REF: consolidate cast_from_unit checks
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
REF: tighter typing in parsing.pyx
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code

Comments:
rebased + greenish
Thanks @jbrockmendel
ENH: Add ea support to get_dummies
State: closed

- [x] closes #32430 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
```
pyright 1.1.276
/home/runner/work/pandas/pandas/pandas/core/reshape/encoding.py
  /home/runner/work/pandas/pandas/pandas/core/reshape/encoding.py:245:13 - error: Expression of type "DtypeObj" cannot be assigned to declared type "str | dtype | Type[str] | Type[complex] | Type[bool] | Type[object] | None"
    Type "DtypeObj" cannot be assigned to type "str | dtype | Type[str] | Type[complex] | Type[bool] | Type[object] | None"
      Type "ExtensionDtype" cannot be assigned to type "str | dtype | Type[str] | Type[complex] | Type[bool] | Type[object] | None"
        "ExtensionDtype" is incompatible with "str"
        "ExtensionDtype" is incompatible with "dtype"
        Type "ExtensionDtype" cannot be assigned to type "Type[str]"
        Type "ExtensionDtype" cannot be assigned to type "Type[complex]"
        Type "ExtensionDtype" cannot be assigned to type "Type[bool]"
        Type "ExtensionDtype" cannot be assigned to type "Type[object]"
    ... (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations 
Completed in 18.065sec

mypy.....................................................................Failed
- hook id: mypy
- duration: 216.68s
- exit code: 1

pandas/core/reshape/encoding.py:245: error: Incompatible types in assignment (expression has type "Union[dtype[Any], ExtensionDtype]", variable has type "Optional[Union[str, dtype[Any], Type[str], Type[complex], Type[bool], Type[object]]]")  [assignment]
```
Thx, fixed
Thanks @phofl
BUG: read_csv overflowing for ea int with nulls
State: closed

- [x] closes #32134 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO CSV
description: read_csv, to_csv
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
ENH: Add ignore_index to Series.drop_duplicates
State: closed

- [x] closes #48304 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: duplicated
description: duplicated, drop_duplicates

Comments:
Thanks @phofl
adding test to check if rows are skipped when skip_blank_lines is set…
State: closed

… to true in read_csv

xref #22693
test added with xfail decorator
@noatamir @phofl

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
CI is currently failing, we have to merge main after this is fixed
@phofl Could you please merge this if there's no problem in the code? 
Thanks for the comments and approval @phofl!
Pre-commit is failing, could you check? Otherwise lgtm 
Not sure why 'Ubuntu / Data Manager (pull_request)' fails. Under details, I see 'The runner has received a shutdown signal...' in GHA. GH 45651". Anything I can do about it?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
Fix groupby-resample KeyError when resampling on Index and giving explicit list of columns. (1.5.x)
State: closed

- [x] closes #50840
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Pandas Devs:
I updated the 1.5.x branch with the fix. Please reply with instructions on if I should add the fix and test code to the main branch as well, or if you would like to do that. This is my first contribution to Pandas, so I'm unfamiliar with the workflow at this point.
PRs should generally always target the main branch. This bug fix should also target 2.0 and not 1.5.x as this doesn't appear like a regression per se
@mroeschke, I'll leave this PR open in case you also want to have the changes made to the 1.5.x branch. I'll also open a new PR (linked with this one) where I've applied the fix to the main branch and added to the whatsnew for v2.0
We have a backporting bot that would take care of this (and this fix doesn't appear eligible for it) so will close this one
PERF: Avoid re-computing mask in nanmedian
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Performance
description: Memory or execution speed performance
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @phofl
WEB/CI: Use GITHUB_TOKEN when making Github API calls
State: closed

Follow up to https://github.com/pandas-dev/pandas/pull/50811, might as well still make authenticated API calls to Github

Labels:
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
Wouldn't you have to expose the token using a ``with:`` block as well?
(e.g. https://github.com/pandas-dev/pandas/blob/3d8993146ccb6054ecad6b7e97ee39e97c6d84e5/.github/workflows/ubuntu.yml#L177-L178 which is setting the environment-file env var)

I doubt that the GITHUB_TOKEN would be directly accessible without doing it (IIRC secrets are encrypted).
It's already here https://github.com/pandas-dev/pandas/blob/3d8993146ccb6054ecad6b7e97ee39e97c6d84e5/.github/workflows/docbuild-and-upload.yml#L18 (it wasn't removed in https://github.com/pandas-dev/pandas/pull/50811)
Ah, makes sense. Sorry for the noise.
Sorry to remove that, I assumed it was to fix the quota problem, and that it didn't work. What is this doing? Is it increasing the quota, or something else?
Yeah I think authenticating with the GIthub Token bumps up our requests/hour (but is still limited) https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limits-for-requests-from-github-actions
Did you check if the token is working as expected? I see github send some headers with the limits: https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limit-headers

We should have quota for more than 20 web builds per hour with the 1,000 limit (not considering we were caching most of them in theory), and it felt that the CI was failing pretty quickly anyway. Since secrets aren't available for PRs, only for commits, I guess this may be indeed helping the commit builds, the ones that are deployed and really matter. Not sure if the failures I saw were mainly PRs.
BUG: merge_asof with non-nano
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
ENH: Improve performance for idxmin and idxmax
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


This avoids re-computing the mask in the nargminmax algorithm

Labels:
name: Performance
description: Memory or execution speed performance
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Should have had a closer look at the actual implementation, this is not necessary
ENH: Get rid of float cast in masked reduction ops
State: closed

- [x] closes #30436 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Shouldn't be user visible

Labels:
name: Enhancement
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Yep, but this might allow optimisations later on. Also, this fixes a couple of mask related bugs in the nan... functions
Thanks @phofl