BrandonFors/pandas-github-issues huggingface dataset:
Standard library imports style check
State: closed

- [X] closes #49647
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
you can push to the same branch, no need to open a new pull request
@MarcoGorelli I closed the previous pull request and merged the two sets of changes into this PR.
OK - in the future a single PR is fine, you can just add extra commits to it
there's a few pylint failures: https://github.com/pandas-dev/pandas/actions/runs/3791678410/jobs/6447285304

I'm thinking it might not be worth introducing this lint check, sorry - if you want to clean up the import changes in the tests (such that pylint still passes), then we could take that
@MarcoGorelli please look at this PR.  I have two code tests failing that I can't seem to fix without making it worse.  Code Checks/Docstring: I am not sure how to get around the duplication between xml.etree... (line 13) and lxml.etree... (lines 517/590).
Requested changes have been made and pushed.
should be good, thanks, just pending green
CI try using ubuntu20.04
State: closed

trying to fix broken CI builds

closes https://github.com/pandas-dev/pandas/issues/50112

Labels:
name: CI
description: Continuous Integration

Comments:
BTW, I might be considering making a pre-commit hook to ban uses of latest versions of images from being used. 

There's not really much benefit to doing this, and we already pin when building wheels (I've been burned by this in the past, when Visual Studio 2017 was upgraded to Visual Studio 2019). 
Thanks - jobs are passing now, so let's merge to unblock all other PRs

> BTW, I might be considering making a pre-commit hook to ban uses of latest versions of images from being used.

Nice! I'm all for automation
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 03a981ae61e685bb6c35234a6fb78a68373af0c6
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #50113: CI try using ubuntu20.04'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-50113-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #50113 on branch 1.5.x (CI try using ubuntu20.04)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                
Can you open an issue? We should think about upgrading to 22.04.

22.10 is the latest version btw, 22.04 is just the latest version with longer support
yup, here it is https://github.com/pandas-dev/pandas/issues/50115
CI fix Code Checks install Python
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration

Comments:
@mroeschke this seems to have fixed the issue (before we had `Code Checks / pre-commit (pull_request) Failing after 9s `) - we OK to do a quick merge so other PRs can be rebased?

https://github.com/pandas-dev/pandas/actions/runs/3641673645/jobs/6147888941
Sure, but I Imagine this will need to be backported?
yeah you're right
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 3af7170292e579413b84424448fc98bd13dffd09
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #50110: CI fix Code Checks install Python'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-50110-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #50110 on branch 1.5.x (CI fix Code Checks install Python)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
add test case for #50105
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TYP: fix mypy ignored error in pandas/compat/numpy/function.py
State: closed

- [ ] xref https://github.com/pandas-dev/pandas/issues/37715
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
@jbrockmendel I pushed up a change for the false case for ascend. Let me know your thoughts.
try running the test suite locally before pushing.  itll help identify problems with a faster turnaround
Got it. Thanks!
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
BUG: #49889
State: closed

Fix for issue: BUG: categorical_column_to_series() should not accept only PandasColumn #49889

Relaxed assert statement from `PandasColumn` to `Column` as not only Pandas column could be used here


Labels:

Comments:
I think the check for a dataframe interchange protocol `Column` class should be removed altogether. If needed there could, instead, be a check for a presence of the `Column` class attributes? Otherwise the roundtrip tests with other libraries implementing the protocol should always fail.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TST: added test to ensure subclasses of ExcelWriter don't add any public attributes
State: closed

- [ ] closes #49602
- not fully tested with style and pre-commit checks yet

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: IO Excel
description: read_excel, to_excel

Comments:
I am failing the pre-commit check and from what I saw in the results of the tests this time, I think it has to do with the import statement? Do you have any input on what the issue might be, ie did I put it in a bad spot based on certain code style/maintenance criteria? Or where I could look for further documentation on this?
https://pandas.pydata.org/docs/development/contributing_codebase.html#pre-commit
Thanks @catmar22 
Thanks @catmar22!
ENH: infer_objects copy kwd
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Enhancement
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
Thanks @jbrockmendel 
@jbrockmendel for 1.5 we had a discussion related to new `copy` keywords (https://github.com/pandas-dev/pandas/issues/48141) and ended up reverting some of those new `copy` keywords. Is there a specific reason to still add a `copy` keyword to `infer_objects`? 
Two big motivations.  One was that we removed automatic inference in a few places, telling users to use infer_objects instead.  Wanted to allow that operation to be cheap where possible.  Second is that there was a bug in here in Block[Subclass].convert that was inconsistent about making copies.  Fixing that bug required making more copies, and it's worth softening that blow.
DEPR: Remove deprecated ExcelWriter methods
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: IO Excel
description: read_excel, to_excel
name: Deprecate
description: Functionality to remove in pandas

Comments:
io.excel.WriteExcel.time_write_excel asv failed
Thanks @rhshadrach 
does this handle the #48780 checkbox in #30228?
Looks like it from the added whatsnew note. Checked off the box
BUG: infer_objects with Intervals
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Interval
description: Interval data type

Comments:
Thanks @jbrockmendel
PERF: 34% faster Series.to_dict
State: closed

Improve the performance of Series.to_dict() by not including a generator that is not necessary. Local tests showed significant performance improvements.

Note this builds off the changes of #46487 from @RogerThomas

Labels:
name: IO Data
description: IO issues that don't fit into a more specific label
name: Performance
description: Memory or execution speed performance

Comments:
Nice! Can you post the results from your local test?
Here is the test. Performance improves by 34%

```python3
import pandas as pd
import random
from timeit import timeit
df = pd.Series([random.random() for _ in range(1000)], dtype=float)
print(timeit("df.to_dict()", globals=globals(), number=10000))
```
Before change: `2.49s`
After change: `1.64s`
Note the performance gain only applies when the series uses Python native types. Also out of curiosity, what's the release timeline for Pandas 2.0?
Thanks @staadecker - very nice!
Thank you all!
Truncate table option for to sql
State: closed

- [X] closes #37210 - in order to reload a table but not drop it to avoid CASCADE-ing
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [X] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [X] Added an entry in the latest `doc/source/whatsnew/v1.5.3.rst` file if fixing a bug or adding a new feature.


Labels:
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
What else needs to be done to get truncate into pandas ?
> What else needs to be done to get truncate into pandas ?

Not 100% sure - this is my first ever PR attempt for an open source project. I think the code is good to go but let it go stale (apologies) when things got busy. At the very least I'll need to solve for some conflicts and update the `whats_new` to a future version. 

I'll start working on it again but if you're willing, I could certainly use some help/guidance in getting it over the finish line.
BUG: SparseArray[dt64_weird_fill_value].astype(int64)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Sparse
description: Sparse Data Type
name: Astype

Comments:
Thanks @jbrockmendel
STYLE: Enable Pylint warning redundant-keyword-arg
State: closed

Enables the Pylint warning `redundant-keyword-arg`.

- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @uzzell
BUG: unwrap setitem indexer for ArrowExtensionArray
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

Setting values in a pyarrow-backed column with a non-scalar indexer was raising. 

Fixes the following:

```
import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]}, dtype="int64[pyarrow]")

# IndexError: index 1 is out of bounds for axis 0 with size 1
df.loc[[0, 1], "b"] = 0
```

Labels:
name: Bug
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Arrow
description: pyarrow functionality

Comments:
Thanks @lukemanley
BUG: DatetimeArray.astype(Sparse)
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Sparse
description: Sparse Data Type

Comments:
Thanks @jbrockmendel
BUG: SeriesGroupBy.nunique raises when grouping with an empty categrorical
State: closed

- [x] closes #21334 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Bug
name: Groupby
name: Categorical
description: Categorical Data Type
name: Reduction Operations
description: sum, mean, min, max, etc.

Comments:
Thanks @rhshadrach
TST: Add test for groupby aggregatation with nested dictionary
State: closed

- [ ] closes #25471  (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Groupby
name: Apply
description: Apply, Aggregate, Transform, Map
name: Needs Tests
description: Unit test(s) needed to prevent regressions

Comments:
Thanks @talukder-sowrov!
PERF: ExtensionBlock.fillna dispatch to underlying arrays
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.

Per discussion: https://github.com/pandas-dev/pandas/pull/49722#issuecomment-1337974871

The discussion focused on pyarrow-backed dtypes, but it seems there is an improvement for other extension types as well:

```
       before           after         ratio
     [1c71d7db]       [86e52fee]
     <main>           <extensionblock-fillna>
-      8.08±0.2ms       6.72±0.1ms     0.83  series_methods.Fillna.time_fillna('Int64', None)
-      8.26±0.2ms      6.85±0.07ms     0.83  series_methods.Fillna.time_fillna('Float64', None)
-         128±3ms         72.3±2ms     0.56  series_methods.Fillna.time_fillna('string', None)
-      15.1±0.6ms      5.41±0.04ms     0.36  series_methods.Fillna.time_fillna('int64[pyarrow]', None)
-      60.7±0.6ms      4.82±0.02ms     0.08  series_methods.Fillna.time_fillna('string[pyarrow]', None)
```

Labels:
name: Performance
description: Memory or execution speed performance
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Arrow
description: pyarrow functionality

Comments:
pls dont merge until i get a change to take a look at this
Thanks @lukemanley
DEPR: enforce concat with bool dtype return object
State: closed

- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Deprecate
description: Functionality to remove in pandas

Comments:
Thanks @jbrockmendel
DEPR: remove some more Int/UInt/Float64Index from tests
State: closed

Progress towards #42717, one easy step at a time... 

There are still some places that have these indexes, but maybe easier taking these things in steps.

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Deprecate
description: Functionality to remove in pandas

Comments:
pandas/tests/indexes/test_base.py::TestIndex::test_reindex_doesnt_preserve_type_if_target_is_empty_index_numeric
I’ve updated the PR.
thanks @topper-123