BrandonFors/pandas-github-issues huggingface dataset:
CLN/TST: test_series_align_multiindex_containing_nan
State: closed

- [x] closes #49443
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
Thanks @lukemanley 
Thx as well, this was really hard to understand before
BUG: Series.isnull() respect use_inf_as_na
State: closed

- [x] closes #50495 
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
@phofl 

> Please always add tests

It seems that I misunderstood the checkbox (it's my first time contributing to this repo). I thought it meant that my code was passing tests which were already there but I have now realized that it is referring to any new tests that I have added (which I haven't since there was already a test for it). Is that making sense or am I still wrong?
Yes correct, every fix should include a test that did not pass before 
any update on this?
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC: Add ignore_functions option to validate_docstrings.py
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Following @MarcoGorelli [comment](https://github.com/pandas-dev/pandas/issues/49968#issuecomment-1333467996), this PR adds an option to `validate_docstrings.py` to ignore functions. This allows to ignore the remaining `RT02` errors, described in https://github.com/pandas-dev/pandas/issues/49968,  in `ci/code_checks.sh`. This will also be useful for other documentation-related errors that required multiple fixes.


Labels:
name: Docs
name: CI
description: Continuous Integration
name: Code Style
description: Code style, linting, code_checks

Comments:
thanks for working on this! i'll take a look
> I like that it allows to validate more things, and avoid making mistakes in docstrings already passing RT02, but at the same time this can start to become quite complex, if for each docstring error we need to keep a separate call and the list of docstrings not passing it.

True, but this should hopefully just be temporary - once the errors are fixed, then this separate call can be removed
> True, but this should hopefully just be temporary - once the errors are fixed, then this separate call can be removed

I think we started fixing docstrings 5 years ago ;) But sure, seems fine to me if that's the plan/
Thanks for reviews! I did a small refactor to make the testing easier. I also rebased on `main`.
> I think we started fixing docstrings 5 years ago ;)

EX02 is now enabled 🎉 

I've come to believe that the only way to get validation issues fixed is to get something running in CI as quickly as possible (with an exclusions list if necessary). So thanks again @Moisan for this script, looks like it ended up delivering!

Let's see if EX01 follows a similar fate 🤞 
Excellent work. Thanks @MarcoGorelli for leading this!
BUG: DataFrameGroupBy.value_counts fails with a TimeGrouper
State: closed

- [x] closes #50486 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Unlike other Groupings, BinGrouper was creating its Grouping based on the labels that were to be in the result. Thus the `Grouping.grouping_vector` length did not in general match the input object, which then fails when combing with other Groupings that do match the input object.

In order to include empty groups, e.g. the 2nd group below

```
df = pd.DataFrame(
    {
        "datetime": to_datetime(["2022-01-01", "2022-01-03"]),
        "values": [2, 3],
    }
)
gb = df.groupby(pd.Grouper(freq="1D", key="datetime"))
print(gb.sum())
#             values
# datetime          
# 2022-01-01       2
# 2022-01-02       0
# 2022-01-03       3
```

added a `uniques` argument to Grouping and then rely on Categorical to encode the groups including those that are missing.

Labels:
name: Groupby
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
Thanks @rhshadrach
BUG: to_numpy not respecting na_value before converting to array
State: closed

- [x] closes #48951 (Replace xxxx with the GitHub issue number)
- [x] closes #24345 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
ENH: Add use_nullable_dtypes to to_numeric
State: closed

- [x] closes #37294 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think to_numeric should be able to return nullable dtypes as well, similar to our I/O functions

Labels:
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thx, this found a couple of bugs
Added a test and found a problem for BooleanArray
Merging for now, happy to address additional comments in a follow up
ENH: Add pandas nullable support to read_orc
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
Thanks @phofl
ENH: Add use_nullable_dtypes to read_clipboard
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: IO Data
description: IO issues that don't fit into a more specific label

Comments:
LGTM pending CI
cc @mroeschke Would you mind having a quick look?
Thanks @phofl 
Thanks
ENH: Add lazy copy to concat and round
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Would merge tomorrow or so if no objections
Thanks @phofl !
For future reference: the usage of internal APIs outside the internals (in reshape/concat.py) is being removed again in https://github.com/pandas-dev/pandas/pull/50777
Thx
ENH: Add use_nullable_dtypes to read_xml
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: IO XML
description: read_xml, to_xml

Comments:
Thanks @phofl
API / CoW: constructing DataFrame from DataFrame creates lazy copy
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @jorisvandenbossche 

Labels:
name: Copy / view semantics

Comments:
#51239
DOC: Add `pip install odfpy` for io.rst
State: closed

Fixes #50487

- [x] closes #50487

Perhaps because of my misunderstanding, adding 'pip install' here may not be consistent with the previous docs style. Because there is no direct instruction in the previous docs how to install the engine manually.


Labels:
name: Docs
name: IO Excel
description: read_excel, to_excel

Comments:
thx @luke396
CLN avoid some upcasting when its not the purpose of the test
State: closed

Generally, tests follow the pattern:
- create input
- do something to it
- check that it matches expected output

There are a few places when, whilst creating the test input, some column is created of some dtype (e.g. 'int'), and then some incompatible type is set (e.g. a string `'foo'`), thus upcasting the column to dtype object. After that, this input is put through the function which the test is meant to test.

Given that the upcasting isn't the purpose of these tests, it'd be cleaner to just create the input right away with object dtype.

The "avoid upcasting to object" part of #50424 does seem uncontroversial, so this would help reduce the diff when a set of PRs to address that will be raised

Labels:
name: Clean

Comments:
good call, thanks
TST: Test cow for set_flags
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

No changes needed here, just adding a test to ensure behavior does not change


Labels:
name: Copy / view semantics

Comments:
Merged

> Side note: (not this PR, but generally enabling CoW) in case of set_flags, this is actually a change in behaviour, since its behaviour was copy=False by default (while almost all other methods have a copy=True default)

This is something I wanted to discuss tomorrow as well
There is a linting failure
Thx, shouldn't edit files through the browser
TST: GH50347 create test for groupby sum of boolean values.
State: closed

- [x] closes #50347
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Does not apply:
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby

Comments:
thx @Daquisu
CI/WEB: Adding cache for maintainers' github info
State: closed

I've seen some CI failures recently caused by hitting github API quota when building the website. The website requests from github the info of maintainers to show names and photos in https://pandas.pydata.org/about/team.html (also the releases in the home, but that's a single request not affecting the quota much).

Feels like in the past every github worker was independent and had its own quota, but now seems like running jobs too quickly hits the quota. In this PR I add a cache to reuse the maintainers info for one hour (the time it takes the quota to reset), which should avoid those failures.

Labels:
name: CI
description: Continuous Integration
name: Web
description: pandas website

Comments:
Thanks for the review. The changes to the script work well, tried locally.
I guess we should backport? Seeing the same failures on 1.5.x
@meeseeksdev please backport to 1.5.x
Yep, good point, I didn't think about it.
DOC: Clean dubious docstring
State: closed

Fixes #50344

- [x] closes #50344 



Labels:
name: Docs
name: Clean

Comments:
Thanks @luke396 this looks nice. I think in general we don't document `__init__` but the class itself. Do you mind moving what's in the `__init__` docstring to the class docstring? I think that will be even better and more consistent. Thanks! 
> Thanks @luke396 this looks nice. I think in general we don't document `__init__` but the class itself. Do you mind moving what's in the `__init__` docstring to the class docstring? I think that will be even better and more consistent. Thanks!

I change it, what about this time? 😃 
> Looks great. Just added couple of small things, you can commit them from the github UI

Thanks for the review!
thx @luke396
DEP: Bump pytest, xdist, hypothesis
State: closed

All these libraries were released ~1 year ago.


Labels:
name: Build
description: Library building on various platforms
name: Dependencies
description: Required and optional dependencies

Comments:
Check minimum version of dependencies are aligned.......................................................Failed
- hook id: validate-min-versions-in-sync
- duration: 0.06s
- exit code: 1

The follow minimum version differences were found between  /home/runner/work/pandas/pandas/ci/deps/actions-38-minimum_versions.yaml, /home/runner/work/pandas/pandas/pandas/compat/_optional.py AND /home/runner/work/pandas/pandas/pyproject.toml. Please ensure these are aligned: 

hypothesis
/home/runner/work/pandas/pandas/ci/deps/actions-38-minimum_versions.yaml: 6.34.2
/home/runner/work/pandas/pandas/pandas/compat/_optional.py: 6.34.2
/home/runner/work/pandas/pandas/pyproject.toml: Not specified

Fixed and all green
thanks @mroeschke
DEPR: Remove int64 uint64 float64 index tests
State: closed

This PR updates various tests that currently use Int64Index/UInt64Index/Float64Index to instead use `Index` with the relevant dtype (parametrized dtypes where relevant). Sections of the tests suite are changed in separate commits, so it will be easier to separate them out, if desired. 

This PR's first 3 commits correspond to #50052, which again build on #49560, which builds on #50195. Those 3 PRs are not part of this PR, but should ideally be treated as precursors for this PR, if possible.

Labels:

Comments:
needs rebase
Yeah, this builds on top of #49560. If I can get that PR in, I will update this (and proceed with other remaining Int/Uint/Float64Index removal.
Superseded by various other PRs.
ENH: Add lazy copy for take and between_time
State: closed

- [ ] xref #49473 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Copy / view semantics

Comments:
Switched to array_equal_fast here
BUG: loc.setitem coercing rhs df dtypes
State: closed

- [x] closes #50467 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
CI fail is unrelated
thanks @phofl
DOC: Add Copy on write whatsnew
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @mroeschke 

As discussed in the other pr. This is for 1.5, will add another pr for the lazy copy methods.
Do you think it makes sense linking https://phofl.github.io/cow-introduction.html instead of explaining everything in here?


Labels:
name: Docs
name: Copy / view semantics

Comments:
>  Do you think it makes sense linking https://phofl.github.io/cow-introduction.html instead of explaining everything in here?

Sure, would be good to link to your article to learn more.
Can you link to the other PR you mention in the description please? I'm missing the context.

Ok by me to add a link from the what's new to the blog post.
Thanks, added the initial pr that implemented it and added a link to the blog post. Happy to remove it, if any objections though.
Thanks @phofl
CI try saving .coverage
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

just checking how long this takes / if it's feasible to do this in CI

Labels:

Comments:
probably too heavy for CI, I'll do this separately once a week
ERR: "day out of range" doesn't show position of error
State: closed

- [ ] closes #50462 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Haven't added a whatsnew note, as this can be considered a follow-up to #50366

Labels:
name: Datetime
description: Datetime data dtype
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @MarcoGorelli