BrandonFors/pandas-github-issues huggingface dataset:
BUG/PERF: algos.union_with_duplicates losing EA dtypes
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v1.6.0.rst` file if fixing a bug or adding a new feature.


ASV added: 

```
       before           after         ratio
     [d76b9f2c]       [6760a19a]
     <main>           <union-with-duplicates>
-      18.6±0.2ms      5.53±0.07ms     0.30  index_object.UnionWithDuplicates.time_union_with_duplicates
```


Labels:
name: Performance
description: Memory or execution speed performance
name: MultiIndex
name: Index
description: Related to the Index class or subclasses

Comments:
thx @lukemanley
CLN/TST: Remove ensure_clean_dir for tmp_dir fixture
State: closed

- [x] xref #26984 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Clean

Comments:
thx @mroeschke
BUG: Timedelta.__new__
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type

Comments:
Thanks @jbrockmendel
STYLE: fix pylint style checks for inherit-non-class.
State: closed

- [ ] xref #48855 - one of many style checks, still needs work.
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Added an ignore statement in the `base.py`. Looks like we need to inherit a non-class in this instance.


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks for reviewing quickly 😊
Hi, @MarcoGorelli this might sound like a silly request, but is it possible to add the `hacktoberfest` label to this PR?
Many thanks 😄
fix pylint bad-super-call
State: closed

- [ ] updates #48855
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Thanks @soumilbaldota
removed "used-before-assignment" line
State: closed

- [x] updates #48855 - work still needs to be done close out
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit). 


Labels:
name: Code Style
description: Code style, linting, code_checks
name: Stale

Comments:
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
closing for now, feel free to submit a new pr
BUG: DatetimeArray-datetimelike mixed resos
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Datetime
description: Datetime data dtype
name: Numeric Operations
description: Arithmetic, Comparison, and Logical operations
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
Thanks @jbrockmendel
TST: Test NEP50
State: closed

https://numpy.org/neps/nep-0050-scalar-promotion.html

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Mothballed
description: Temporarily-closed PR the author plans to return to
name: Stale

Comments:
First Run: `2022-09-30T18:50:37.3831338Z = 824 failed, 150426 passed, 18621 skipped, 1026 xfailed, 8 xpassed, 381 warnings, 16 errors in 6970.92s (1:56:10) =`
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
still active?
I'll mothball this. I need to come back around to testing this relateively soon
ERR: Improve error message when assigning a complete row using 'at' m…
State: closed

Better error reporting when trying to assign a new row using 'at' method

- [x] closes #48729 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Thanks @ambujpawar 
Shall we backport this to 1.5.x? 
Re-reading https://github.com/pandas-dev/pandas/issues/48729, we started to raise the incomprehensible error in 1.5.0 (it actually worked before)
Sure, makes sense to backport
@Meeseeksdev backport 1.5.x
BUG: preserve EA in pd.array
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.
name: Constructors
description: Series/DataFrame/Index/pd.array Constructors

Comments:
> Needs whatsnew note?

The only known/tested change is for not-yet-public cases
Thanks @jbrockmendel
BUG: histogram weights aren't dropped if NaN values in data
State: closed

- [x] closes #48884
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Visualization
description: plotting

Comments:
I've created a test where a df and set of weights are made, and one value in each column of df is set to NaN. I then make a copy of the df without the NaNs, and copies of weights with the corresponding weights dropped. It then checks that histograms made using each pair have the same heights.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
> Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

Will merge main now. I addressed the comments above but haven't had a response.
It also occurs to me that `weights` itself might work better as a dict or DataFrame if it has more than a single column, but I think this is beyond the scope of this PR
Thx @Ormorod
make float(NA) return nan
State: closed



- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Cc @jorisvandenbossche 
Thanks for the ping! 

I think we need to think through some actual use cases to see if returning NaN makes more sense or not (in the linked matplotlib PR, the issue is more that they got object dtype array, and so they (or in pandas) should ensure they convert to a float dtype, and then you don't run into a `float(pd.NA)` issue, since you already got floats)

I think the main reason to _not_ have this automatic conversion is that NaN and NA are inherently different (and they behave different in several operations, such as in boolean context, in comparisons, ...). 


Sure, the issue still arises when using `np.asanyarray(ser)` though, as then you can't pass a `na_value` and it'll fail on `pd.NA`:
```
In [1]: df = pd.DataFrame({'a': [1,2,3], 'b': [4,5,pd.NA]}, dtype='Int64')

In [2]: np.asanyarray(df, dtype='float')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [2], line 1
----> 1 np.asanyarray(df, dtype='float')

File ~/pandas-dev/pandas/core/generic.py:2078, in NDFrame.__array__(self, dtype)
   2077 def __array__(self, dtype: npt.DTypeLike | None = None) -> np.ndarray:
-> 2078     return np.asarray(self._values, dtype=dtype)

TypeError: float() argument must be a string or a number, not 'NAType'
```
closing as per the dev meeting, will look into handling this directly in __asarray__
(just to have it written down here as well)

> Sure, the issue still arises when using `np.asanyarray(ser)` though, as then you can't pass a `na_value` and it'll fail on `pd.NA`:

That's still a consequence of defaulting to object dtype (what is happening here is that the `df` gets converted to a single object dtype array, and then numpy tries to convert this to float, running into this issue). 

So also for this example we can handle this in `__array__` if we would like to avoid going through object dtype (raise error, directly coerce to float with NaNs, etc)
WEB: Add Luke to maintainers list
State: closed

@lukemanley feel free to merge if it looks fine :)


Labels:
name: Web
description: pandas website

Comments:
failure seems unrelated? looks like a seg fault.
> failure seems unrelated? looks like a seg fault.

Yes, the doc build is green, surely unrelated, so safe to merge I think.
Feel free to merge @lukemanley :)
thanks @datapythonista
TST: MultiIndex.get_indexer with na/missing
State: closed

- [x] closes #43222
- [x] closes #46173
- [x] closes #42883
- [x] closes #38623
- [x] closes #29252
- [x] closes #48905 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.0.0.rst` file if fixing a bug or adding a new feature.


Update: #49442 has been merged to fix the underlying bug here. This PR has been updated to add tests for closing the linked issues which are all related to the same underlying bug in #49442.


Labels:
name: Bug
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: MultiIndex

Comments:
Could you rebase? Hopefully, all tests should pass now. Also, would be good to add the issues to my whatsnew note. wdyt?
> Could you rebase? Hopefully, all tests should pass now. Also, would be good to add the issues to my whatsnew note. wdyt?

Updated this PR to add tests only and close linked issues. If you think any of these tests are redundant, I'm happy to remove some (all reported for the same underlying bug).
bugfix for #48757
State: closed

- [ ] closes #48757 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.
TYP: Upgrade mypy to 0.981
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thx
TYP: disable flake8-pyi's Y041
State: closed

xref https://github.com/pandas-dev/pandas-stubs/pull/341#issuecomment-1262786774 https://github.com/pandas-dev/pandas/pull/48836#discussion_r982952166 https://github.com/pandas-dev/pandas/pull/48850#discussion_r983620339

Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
Thanks @twoertwein
REGR: replace replacing wrong values with inplace and datetime
State: closed

- [x] closes #48863 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: replace
description: replace method

Comments:
cc @jbrockmendel since this was a regression from #45911
Thanks @phofl
BUG: convert_dtypes not converting with pd.NA and object dtype
State: closed

- [x] closes #48791 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think these conversion rules make more sense

Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
I wondered the same thing. As soon as NA is present, we get mixed. Otherwise we don't end up there
Thanks @phofl
BUG: to_timedelta raising for ea float
State: closed

- [x] closes #48796 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Timedelta
description: Timedelta data type
name: NA - MaskedArrays
description: Related to pd.NA and nullable extension arrays

Comments:
Thanks @phofl
REGR: Avoid unnecessary warning when setting empty dataframe
State: closed

- [x] closes #48480 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves
name: Regression
description: Functionality that used to work in a prior pandas version
name: Warnings
description: Warnings that appear or should be added to pandas
name: Copy / view semantics

Comments:
Thanks @phofl 
I receive this warning

'''
`FutureWarning: In a future version, `df.iloc[:, i] = newvals` will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either `df[df.columns[i]] = newvals` or, if columns are non-unique, `df.isetitem(i, newvals)`
'''

when calling df.update(df2). I'm unclear if this issue (#48853) fixed the warning message and / or if there is a different method I should use to update my dataframe.
@ryanlampkin are you using new-style/nullable-dtypes such as `string` or `Int64`? I've found that `df.update` loses the original dtype, demoting the column to `object`. I think this confuses some internal check in `df.update`, resulting in the `df.loc` assignment and this warning.
TYP: Fix typing errors caused by new numpy
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


cc @mroeschke


Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
I think we have to backport this
Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 1.5.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 336896907748389e2cd0d57504508475d425348e
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #48850: TYP: Fix typing errors caused by new numpy'
```

4. Push to a named branch:

```
git push YOURFORK 1.5.x:auto-backport-of-pr-48850-on-1.5.x
```

5. Create a PR against branch 1.5.x, I would have named this PR:

> "Backport PR #48850 on branch 1.5.x (TYP: Fix typing errors caused by new numpy)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
DEP: Enforce deprecations of read_csv keywords
State: closed

#40413 #43427


Labels:
name: IO CSV
description: read_csv, to_csv
name: Deprecate
description: Functionality to remove in pandas

Comments:
Would it be easier to push this through in smaller pieces?
This is actually relatively straight forward, did not get to it yet. Will update Monday at the latest 
This should be ready now
looks good, needs whatsnew note?
Thx, forgot about that. added
Fixed
BUG: add_categories losing dtype information
State: closed

- [x] closes #48812 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Categorical
description: Categorical Data Type

Comments:
Thanks @phofl