BrandonFors/pandas-github-issues huggingface dataset:
added test for issue #31104
State: closed

- [ ] closes #31104 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Looks like the tests are failing. This test should also go in a file with similar tests
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
DOC fix EX02 algorithms.factorize docstring
State: closed

- [x] Part of #51236
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards #51236.

Fix EX02 docstring errors for :

`pandas.factorize`
`pandas.Index.factorize`
`pandas.Series.factorize`


Labels:
name: Docs

Comments:
thanks! can you remove them from the excludes list?
Done. Thanks for the review @MarcoGorelli 
The Docstring validation should be good now.
BUG: bug in Index._should_fallback_to_positional
State: closed

- [x] closes #51053
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Fixes #51053.

Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @topper-123
API / CoW: constructing DataFrame from DataFrame/BlockManager creates lazy copy
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Checking if everything passes

Labels:
name: Copy / view semantics

Comments:
This replaces https://github.com/pandas-dev/pandas/pull/50499?
If ci passes everywhere, e.g. I did not miss anything, yes. Wanted to keep the other pr open until a run is through here
Looks good I think
I think this is ready as well except the doc comment
(I changed the ENH -> API in the title, as for CoW enabled, this is actually a behaviour change (copy=False is the default), not just avoiding a copy where we currently copy)
@jorisvandenbossche are you ok here?
Merging, we can still address the mgr case before 2.0 is out if necessary
Yeah, for the manager case, it might be good to check some benchmarks for this. Although in practice I suppose it will only show up when creating many small DataFrames in some operation, so not sure what a typical use case would be where this would be a significant issue.
TST: categorical alignment
State: closed

- [x] closes #13365
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.

Already fixed on main, just adding a test

Labels:
name: Categorical
description: Categorical Data Type

Comments:
Thanks @lukemanley
DOC Correcting EX02 errors
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards https://github.com/pandas-dev/pandas/issues/37875 and https://github.com/pandas-dev/pandas/issues/27977

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @MarcoGorelli ... yes, it's nice that the list is a bit shorter and that cats have fun. :)
BUG: groupby.agg with numba and as_index=False
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: numba
description: numba-accelerated operations

Comments:
whatsnew added + green
Thanks @jbrockmendel
PERF: ArrowExtensionArray.to_numpy(dtype=object)
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

Perf improvement for `ArrowExtensionArray.to_numpy(dtype=object)`. 

(`dtype=object` is not uncommon, see discussion in #22791) 

Some examples where it has an impact:

```
import pandas as pd
import numpy as np

data = np.random.randn(1_000_000, 10)
df = pd.DataFrame(data, dtype="float64[pyarrow]")

%timeit df.sum(axis=1)

# 4.04 s ± 53.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <- main
# 796 ms ± 22.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <- PR

%timeit df.clip(0.0, 0.1)

# 4.82 s ± 159 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   <- main
# 1.51 s ± 10.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  <- PR
```

Labels:
name: Performance
description: Memory or execution speed performance
name: Arrow
description: pyarrow functionality

Comments:
dumb question: why isn't `self._data.to_numpy()` good enough here?
> dumb question: why isn't `self._data.to_numpy()` good enough here?

I believe there is a desire to maintain the raw dtype (even if wrapped in an object array) and use `pd.NA` by default. 

For instance, this converts int to float when nulls are present:

```
In [1]: import pandas as pd

In [2]: arr = pd.array([1, None], dtype="int64[pyarrow]")

In [3]: arr._data.to_numpy()
Out[3]: array([ 1., nan])
```

I'm not sure its required, but the current implementation aligns mostly with `BaseMaskedArray`:

```
In [4]: arr = pd.array([1, None], dtype="Int64")

In [5]: arr.to_numpy()
Out[5]: array([1, <NA>], dtype=object)

In [6]: arr = pd.array([1, None], dtype="int64[pyarrow]")

In [7]: arr.to_numpy()
Out[7]: array([1, <NA>], dtype=object)
```


Thanks @lukemanley 
Not totally sure, but looks like this is failing the ci on main. Any idea why?
Ah darn. Should we revert this in the meantime?
Probably a good idea, if we don't know how to fix
Yeah locally reverting this change fixes the test failure
REF: simplify aggregate_frame, column-name-pinning
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @jbrockmendel
ENH: add unit, as_unit to dt accessor
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
rebased+green
Thanks @jbrockmendel
REF: remove aggregate_item_by_item
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @jbrockmendel
DEP: Deprecated pad/backfill for Series/DataFrame
State: closed

- [x] xref #33396 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

We should make this consistent again with GroupBy etc.

Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @phofl (if you could update the deprecation tracker that would be great)
updated
REF: de-duplicate BusinessDay apply, apply_array
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Frequency
description: DateOffsets

Comments:
Thanks @jbrockmendel
CI/TST: Mark test_basic_series_frame_alignment for python engine as flaky
State: closed

Seems to also be flaky with the python engine: https://github.com/pandas-dev/pandas/actions/runs/4109864025/jobs/7092127908

Labels:
name: CI
description: Continuous Integration
name: Unreliable Test
description: Unit tests that occasionally fail

Comments:
Bummer.  Does pytest have a marker to mark something as flaky rather than non-strict xfail?  Im thinking something like "if it fails re-run up to N times and pass if it passes any time"
Looks like there is a plugin 

https://github.com/pytest-dev/pytest-rerunfailures


No idea if this is maintained well enough though
I can explore that plugin in a follow up PR. I think we have a few tests that are flaky like this
Sounds good! Thx
thx @mroeschke
Removed **kwds from parse function in read_excel
State: closed

- [ ] closes #50953 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


The code works fine locally.

![image](https://user-images.githubusercontent.com/12509045/217342062-661ee8ea-73c6-4aae-9b21-baa3287ebcd7.png)

 Checking CI/CD pipeline now.

Labels:
name: Bug
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO Excel
description: read_excel, to_excel

Comments:
In the comment below, I'm assuming that all valid kwargs that can be passed to `parse` are still available, but haven't confirmed that yet.

> this is an API change, not sure if we can do this without a deprecation cycle

Part of this change is raising if a user passes invalid keyword args. I think we can regard this as a bugfix. I guess this change might also mess up someone who is using positional arguments - I'd have to guess that is quite rare but I suppose that aspect is an API change. Would you agree?
No problem with making the keywords explicit, but would prefer keeping the order, also would rather deprecate the kwargs instead of removing outright, like we did in other places with unused kwargs
I think the way forward is to emit a FutureWarning if invalid kwargs are in present. I assume @pacificdragon added the keyword arguments in their current locations to agree with the ordering of other methods, but I haven't checked this. We currently can't add new arguments unless they are at the end to retain backwards compatibility. For this, I think there are two ways forward:

 a. Add keywords to the tail end of the signature
 b. Deprecate specifying any argument other than `sheet_name` by position

If we go with (a), we'll still have different ordering of arguments which isn't great. With (b), we can add arguments in any order after the deprecation is enforced. So I'd lean toward (b) here unless it's preferred that these arguments are allowed to be positional.
b sounds good, also +1 on adding a FutureWarning and removing in 3.0
I would actually be OK with doing this without a deprecation cycle. I think it is a bug for a function to silently accept arguments it does nothing with. We also have a precedent for doing this in the excel space like https://github.com/pandas-dev/pandas/pull/34464
Thanks, Everyone for taking action on this. @rhshadrach Created PR for the same [51331](https://github.com/pandas-dev/pandas/pull/51331)
DOC Correcting EX02 errors
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Towards https://github.com/pandas-dev/pandas/issues/37875 and https://github.com/pandas-dev/pandas/issues/27977

Labels:
name: Docs
name: Code Style
description: Code style, linting, code_checks

Comments:
:D ... Thanks @MarcoGorelli!
REF: unused group_keys, indexer from BaseGrouper
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Refactor
description: Internal refactoring of code
name: Groupby

Comments:
Thanks @jbrockmendel
TST: suppress deprecation messages in doc tests for Index.is_foo
State: closed

There were a lot of deprecation messages shown when running `ci/code_checks.sh docstrings`.

This suppresses those messages.

Example of messages that are now suppressed:

```
<doctest pandas.Index.is_boolean[1]>:1: FutureWarning: Index.is_boolean is deprecated. Use pandas.api.types.is_bool_type instead.
  idx.is_boolean()
<doctest pandas.Index.is_categorical[1]>:1: FutureWarning: CategoricalIndex.is_categorical is deprecated.Use pandas.api.types.is_categorical_dtype instead
  idx.is_categorical()
<doctest pandas.Index.is_categorical[3]>:1: FutureWarning: Index.is_categorical is deprecated.Use pandas.api.types.is_categorical_dtype instead
  idx.is_categorical()
<doctest pandas.Index.is_categorical[6]>:1: FutureWarning: RangeIndex.is_categorical is deprecated.Use pandas.api.types.is_categorical_dtype instead
  s.index.is_categorical()
```

xref #50042.

Labels:
name: Docs

Comments:
Does this fix all the warnings when validating docstrings? https://github.com/pandas-dev/pandas/issues/44642

If so, we might want to errors on warnings
You’re talking about making `ci/code_checks.sh docstrings`fail when a warning is raised? This diesn’t do that, this is only about the ones that were caused directly by #50042. 

Thanks @topper-123
ENH: Allow ArrowDtype(pa.string()) to be compatable with str accessor
State: closed

Reboot of #50325

Notable change was that `pandas.core.strings.BaseStringArrayMethods` in the `__init__.py` was causing a circular import when used in `ArrowExtensionArray`, so needed to remove the import into `__init__.py.` This breaks one of our dask tests but it has been fixed downstream https://github.com/dask/dask/pull/9907

Labels:
name: Strings
description: String extension data type and string data
name: Arrow
description: pyarrow functionality

Comments:
Looks like dask-related doctests are failing.  do we need to pin a version somewhere?
I think the docbuild will fail until dask releases a new version.

@jrbourbeau typically how often does dask release a new version?
> @jrbourbeau typically how often does dask release a new version?

We release every two weeks. In fact, the latest release was pushed out a couple of hours ago 
Oh perfect looks like this PR picked up the dask update and the tests are passing now!
Any other comments here? (Failures are unrelated)
Nice! thx
API: dont infer dtype for object-dtype groupby reductions
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: API - Consistency
description: Internal Consistency of API/Behavior

Comments:
needs a rebase
Thanks @jbrockmendel
TYP: Upgrade mypy to 1.0
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @phofl
DOC: Minor doc cleanups
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The MultiIndex methods have dedicated docstrings, so should render docs for them

Labels:
name: Docs

Comments:
Thanks @phofl