BrandonFors/pandas-github-issues huggingface dataset:
BUG: parsing nanoseconds incorrect resolution
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/v1.5.0.rst` file if fixing a bug or adding a new feature.

This is broken off from a branch that fixes a couple of higher-level bugs.


Labels:
name: Bug
name: Datetime
description: Datetime data dtype

Comments:
hmm lots of failures.....
> hmm lots of failures.....

messed up fixing a merge conflict, now fixed+green
DOC: building page for nested methods doesn't work
State: closed

closes #46804 

Like this
```
 python make.py clean && python make.py --single pandas.Series.str.rsplit
```
and
```
 python make.py clean && python make.py --single pandas.Series.value_counts
```
and
```
 python make.py clean && python make.py --single pandas.Series
```
have their docs built fine

cc @jorisvandenbossche 

I didn't find any tests for building docs - are they just tested manually?

Labels:
name: Docs

Comments:
So this works locally for you? (Doc build passed which is promising)
Yup, worked for a contributor too 

I'll see if I can put a quick check for this together
> Doc build passed which is promising

Building with `--single` wasn't previous tested though, so it would've passed anyway :smile:  Have added a quick check for it to the doc build job
Very nice! Thanks @MarcoGorelli 
@MarcoGorelli - running the code_checks locally, I now get some docs opening up in my browser:

https://github.com/pandas-dev/pandas/blob/c5610d22eda24b7cd8987972d4c2fe9b1a661780/doc/make.py#L236-L242

I don't think this was intentional, right? No much of an issue, more of an oddity. Could we add an argument, e.g. ``--open-browser`` that when unspecified gives current behavior and when specified overrides? Then we could do ``open-browser=False`` in the code checks.
Hey @rhshadrach 

> I don't think this was intentional, right?

that's right

how about
```
    argparser.add_argument(
        "--no-browser", help="Don't open browser", action="store_true"
    )
```
which, if passed, won't open the browser, and if not passed, retains the current behaviour?
ENH: adding datetime default output formatting in Styler
State: closed

- [x] partly addresses #46755
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Will add any further necessaries if this PR is considered worthwhile.

Labels:
name: Stale

Comments:
Hmm I'd be +0 for this inclusion. It's not entirely clear to me how this differs from just calling `dt.strftime` beforehand
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Closing as doesn't seem to have much traction.. might revisit later..
BUG: `DataFrameGroupBy.value_counts` includes non-observed categories of non-grouping columns
State: closed

- [X] closes #46357
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff
name: Categorical
description: Categorical Data Type

Comments:
This PR has not been reviewed for 1 month. @rhshadrach as you were involved in the issue description, would you like to have a look, or should I wait/ping for the review from a specific maintainer ?
@jreback @rhshadrach do you have any other comments to add on this PR or should we consider merging it (after rebase) ?
Unsuccessful job seems to be related to the flakyness of https://github.com/pandas-dev/pandas/pull/45651
@jreback @rhshadrach `DataFrameGroupBy.value_counts` was added in pandas-1.4.0

Firstly. do we need the extensive release note. These are normally reserved for bug fixes that have a significant change in behavior that might be considered a breaking change by users.

Also because `DataFrameGroupBy.value_counts` is a new feature we could consider backporting to 1.4.x.

no backport in this
and i think does need a big note as this is a significant change that's not obvious
@rhshadrach I fixed the whatsnew
Job failure is due to the unavailability of Github Actions
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Apologies @LucasG0 - this fell off my radar. Can you resolve conflicts.

@simonjayhawkins - are you good here?
No worries @rhshadrach, conflicts resolved.
@simonjayhawkins - friendly ping
> @simonjayhawkins - friendly ping

My comments were related to whether this should have been a targeted backport or not.

But I'm not going to block.
@jreback - friendly ping
thanks @LucasG0 this is very nice! and thanks @rhshadrach for all the work on this as well.
Change `mylen` function to account for NaN empty lists when exploding on multiple columns
State: closed

- [x] closes #46084 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
Thanks for the pull request. Doesn't this fix relate to an existing issue? Can you demonstrate the behavior before/after fix?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: .fillna(-1) does not work in pd.eval
State: closed

Adding ```self.value``` resolves the issue #46471

- [x] closes #46471
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in main, please address the review and we can reopen.
BUG FIX: groupby.last() no longer changes np arrays with pd.NA to dtype Int64
State: closed

- [ X] closes #46409  
- [X ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Hello @sydneyeh! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-04-16 16:17:43 UTC
@github-actions pre-commit
TYP: CallableDynamicDoc
State: closed

The more tricky parts of #46698.

Adding type annotations to `CallableDynamicDoc` seems to uncover multiple missing type annotations. I'm not very confident in some of the new pyi files (types for cython/c functions).

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Hello @twoertwein! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-05-25 23:36:05 UTC
@jbrockmendel Could you have a look at `json.pyi` and `indexing.py`?
Thanks @twoertwein
To excel storage options fix
State: closed

- [ ] closes #46381  
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Duplicate Report
description: Duplicate issue or pull request
name: IO Excel
description: read_excel, to_excel
name: Styler
description: conditional formatting using DataFrame.style

Comments:
Hello @ben-feldstein! Thanks for opening this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



* In the file [`pandas/tests/io/formats/test_to_excel.py`](https://github.com/pandas-dev/pandas/blob/c488b3a7f856395d6b52e085bf5dfe34b5188184/pandas/tests/io/formats/test_to_excel.py):

> [Line 346:1](https://github.com/pandas-dev/pandas/blob/c488b3a7f856395d6b52e085bf5dfe34b5188184/pandas/tests/io/formats/test_to_excel.py#L346): [E303](https://duckduckgo.com/?q=pep8%20E303) too many blank lines (3)
> [Line 353:5](https://github.com/pandas-dev/pandas/blob/c488b3a7f856395d6b52e085bf5dfe34b5188184/pandas/tests/io/formats/test_to_excel.py#L353): [E722](https://duckduckgo.com/?q=pep8%20E722) do not use bare 'except'
> [Line 354:60](https://github.com/pandas-dev/pandas/blob/c488b3a7f856395d6b52e085bf5dfe34b5188184/pandas/tests/io/formats/test_to_excel.py#L354): [W292](https://duckduckgo.com/?q=pep8%20W292) no newline at end of file


duplicate of #46491
ENH: Added Index param to to_dict like in to_json #46398
State: closed

- [X] closes #46398 
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/v1.5.0.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
Hello, it's my first time contributing! I'm not sure why pre-commit.ci is timing out. I ran pre-commit locally and all the checks seemed to pass fine. Any suggestions on what might be going wrong?
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in contributing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST: Added test for frequency retention in categorical constructor
State: closed

- [ ] xref #33830 (Replace xxxx with the Github issue number)
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
@bgollop contributed
Thanks @andjhall (failure unrelated)
BUG: fixed buggy behavior in pivot_table with margins=True and numeric columns
State: closed

- [X] closes #26568(Replace xxxx with the Github issue number)
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Found a error in the way reorder_levels was being called in _generate_marginal_values in pivot.py. Added a change that would modify the representation of the new order to use keys instead of position in cases where valid key was also a valid position. Included a test from original bug report that exposed the buggy behavior.

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Stale

Comments:
Hello @andjhall! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-04-14 19:46:41 UTC
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in contributing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DEBUG verbose pre-commit
State: closed

pre-commit-ci timed out, so just checking hooks' timings with `verbose: true`

Labels:

Comments:
Is it possible to make the the pre-commit.ci check xfail-able while it's being debugged? 
Hey @mroeschke - I don't think so, I've suggested to @jreback that we simply uninstall it for now as it times out at 180s and `flake8` and `black` alone go over that limit

Closing this for now anyway as the pre-commit/action, although deprecated, seems to be working fine
@MarcoGorelli ok i should install?
yeah I'd say let's uninstall, thanks
ok it's suspended
ENH: Create ArrowDtype
State: closed

- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Not activated yet. Opening to get any feedback about the approach cc @jbrockmendel 


Labels:
name: Arrow
description: pyarrow functionality

Comments:
discussed on the March call, id like to migrate from BaseMaskedDtype/NumericDtype etc to just MaskedDtype.  Similarly id suggest here just ArrowDtype.
By "just ArrowDtype" you mean have `ArrowDtype` inherit directly from `ExtensionDtype`?
> By "just ArrowDtype" you mean have ArrowDtype inherit directly from ExtensionDtype?

Right, without all the subclasses
> * [x]  All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
> 
> Not activated yet. Opening to get any feedback about the approach cc @jbrockmendel
> 
> 1. `BaseArrowDtype` inherits from `BaseMaskedDtype`
> 2. `NumericArrowDtype` mirrors `NumericDtype`

can we rename NumericDtype -> NumericMaskedDtype (alt maybe NumericNumpyDtype and rename BaseMaskedDtype)
> > * [x]   All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
> > 
> > Not activated yet. Opening to get any feedback about the approach cc @jbrockmendel
> > 
> > 1. `BaseArrowDtype` inherits from `BaseMaskedDtype`
> > 2. `NumericArrowDtype` mirrors `NumericDtype`
> 
> can we rename NumericDtype -> NumericMaskedDtype (alt maybe NumericNumpyDtype and rename BaseMaskedDtype)

This PR will just add the base ArrowDtype for now
BUG: Prevent segfault in roll_weighted_var
State: closed

- closes #46760
- edit: I added a test to `test_win_type.py`
- I added an entry in the latest `doc/source/whatsnew/v1.5.0.rst` file because this fixes a bug

Labels:
name: Window
description: rolling, ewma, expanding

Comments:
Ok, thanks.

I'm unsure how to verify that my test I added to `test_win_type.py` is working. I'm unfamiliar with implementing tests in projects like pandas, forgive me while I'm learning :) and thanks again for your help.
Awesome, thanks @joseortiz3 (the failure was unrelated)
BUG: date_range not excluding dates correctly with periods and inclusive not 'both'
State: closed

- [x] closes #46331 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please resubmit this pull request such that it will be put on the top of the pull request queue; it will help with visibility for review.
CI: Fail Numpy Dev build on DeprecationWarnings from numpy only
State: closed

- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

I believe the intent of `-W error` was to catch DeprecationWarnings from numpy primarily. Currently builds are failing because of a warning from distutils which idk if we can do anything about.

Labels:
name: CI
description: Continuous Integration

Comments:
backport?
> backport?

Yup probably a good idea
@meeseeksdev backport 1.4.x
DOC: add examples to groupby.first
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Add a few more examples to `groupby.first`. I also removed the mention of `numeric_only = None` since the code doesn't check for None any case, nor does the type allow for it.

Labels:
name: Docs

Comments:
Hello @Moisan! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-04-24 14:36:17 UTC
doesn't this fix the same as #46792 ? though this fixes for groupby i think that is fixed as well.
https://github.com/pandas-dev/pandas/pull/46792 covers unused arguments in `Resampler.first` however `GroupBy.first` doc doesn't include unused arguments so I don't think that PR is related.
@Moisan could you merge in the main branch one more time? After that LGTM
thanks @Moisan
Use np.dtype with explicit type argument rather than just using np.dtype with implicit Any type argument.
State: closed

Existing type annotation used np.dtype assuming type argument "Any" will be used on type checking. but some type checker such as pyright with the highest type checking mode will complain about using generic type with implicit Any type argument.

so, I went through all usages of np.dtype in type annotation and added type argument explicitly. Probably there will be more case of using generic type in type annotation without explicit type arguments, but I only audited np.dtype case.

related issue in pylance - https://github.com/microsoft/pylance-release/issues/1968


Labels:

Comments:
Hello @heejaechang! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





























There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-04-12 20:54:53 UTC
it looks like [np.generic] throws on runtime if version < 3.9. it looks like we can't have it in pandas until people move to >= 3.9
> it looks like [np.generic] throws on runtime if version < 3.9. it looks like we can't have it in pandas until people move to >= 3.9

You could try defining a new type `NpGenericDtype` in `_typing.py` that is `np.dtype` if python <= 3.8, and `np.dtype[np.generic]` if python >= 3.9, then replace all the `np.dtype` references in the code to use `NpGenericDtype`, and then you'd have something that would possibly work with python 3.8 and 3.9.

So in `_typing.pyi`, you'd do:
```python
from pandas.compat import PY39
```
then
```python
if not PY39:
    NpGenericDtype = np.dtype
else:
    NpGenericDtype = np.dtype[np.generic]
```

I think that will work.