BrandonFors/pandas-github-issues huggingface dataset:
REGR: RecursionError when attempting to replace np.nan values
State: closed

- [ ] closes #45725 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


`self._can_hold_element(value)` for pd.NA on a float block is `False` so we need to ensure the block is coerced to a dtype with can hold the value to prevent the RecursionError 

The `None` case in the issue OP was already fixed by #46404 which short circuits for the list like case and does not split the block for backwards compatibility

The test added here is NOT parametrized for various null values since this is a targeted regression fix of the recursion error and to maintain the pre-regression behavior and NOT looking to make null value handling consistent for replace.

This targeted regression fix is milestoned 1.5 as the regression that resulted in the RecursionError  is only on main.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: replace
description: replace method
name: Stale

Comments:
@jbrockmendel if we want to get the RecursionError fixed quickly without this fix we could revert #45568 and then use the test added here as a regression test to close the issue.

We could then look into moving `None` handling into block replace, xref https://github.com/pandas-dev/pandas/pull/46404#discussion_r830667602

i've hit the revert button on #45568 and got the `Sorry, this pull request couldn’t be reverted automatically. It may have already been reverted, or the content may have changed since it was merged.` but not looked further whether this is also an option.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
REF: move ArrowStringArray.__setitem__ and related methods to ArrowExtensionArray
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Moved/refactored `ArrowStringArray.__setitem__` and related methods to `ArrowExtensionArray`. These methods should be generic enough to support any pyarrow type. I left `_maybe_convert_setitem_value` to be implemented by the subclasses as I figured there may need to be type specific logic there.

Labels:
name: Refactor
description: Internal refactoring of code
name: ExtensionArray
description: Extending pandas with custom dtypes or arrays.

Comments:
thanks @lukemanley
BLD: Remove windows shim
State: closed

Just to see if this breaks the world.

Labels:
name: Build
description: Library building on various platforms

Comments:
> Just in case, I think it would be good to include a whatsnew note that a customs header for int types for VS 2019 compat has been removed.

I don't see how it's user-facing, but OK.  Suggested wording/location?
Okay on second thought, this should be okay. Thanks @jbrockmendel 
~Looks like this also fixed the py38_32bit build.~

Nevermind, the run was from before this build started failing.
categorical type column and interval data to a CSV file
State: closed

- [x] closes #46297 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature

- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:

Comments:
Hello @Kyrpel! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



* In the file [`pandas/tests/io/formats/test_to_csv.py`](https://github.com/pandas-dev/pandas/blob/a0f99f14aeddbf46f584c6859b1435b32bb5653c/pandas/tests/io/formats/test_to_csv.py):

> [Line 321:89](https://github.com/pandas-dev/pandas/blob/a0f99f14aeddbf46f584c6859b1435b32bb5653c/pandas/tests/io/formats/test_to_csv.py#L321): [E501](https://duckduckgo.com/?q=pep8%20E501) line too long (91 > 88 characters)
> [Line 322:45](https://github.com/pandas-dev/pandas/blob/a0f99f14aeddbf46f584c6859b1435b32bb5653c/pandas/tests/io/formats/test_to_csv.py#L322): [W291](https://duckduckgo.com/?q=pep8%20W291) trailing whitespace
> [Line 324:56](https://github.com/pandas-dev/pandas/blob/a0f99f14aeddbf46f584c6859b1435b32bb5653c/pandas/tests/io/formats/test_to_csv.py#L324): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','



##### Comment last updated at 2022-03-23 13:45:12 UTC
CLN: rename _tz_localize_using_tzinfo_api, standardize patterns
State: closed

Maximized places where code is exactly copy/pasted so its easy to check it stays in sync (since so far the prospect of de-duplicating the affected code is not looking good)

Small optimizations in _tz_convert_tzlocal_utc, which is renamed to reflect the fact that it is actually the General Case, not just tzlocal.

Labels:
name: Timezones
description: Timezone data dtype
name: Clean

Comments:
Thanks @jbrockmendel
BENCH: Fix makeCategoricalIndex
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

@jreback looks like https://github.com/pandas-dev/pandas/pull/46429 was still making the benchmarks job fail. Locally constructing with `ncats = 15` works. 


Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
@mroeschke cool merge when ready
DEPR: Timedelta.freq, Timedelta.is_populated
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Frequency
description: DateOffsets
name: Deprecate
description: Functionality to remove in pandas

Comments:
is there an issue for this (don't create one) but IIRC there is?
> is there an issue for this (don't create one) but IIRC there is?

No, @mroeschke suggested it in #46420
CI: Add pytest-asyncio to PyPy CI
State: closed

- [ ] closes #46421  (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: CI
description: Continuous Integration
name: Dependencies
description: Required and optional dependencies
name: PyPy

Comments:
Lots of test errors, are those expected?
Also, is it expected that those errors are simply ignored by the CI pipeline?
Yes, both of these are expected. Right now, the PyPy CI really tests to see if we can compile pandas correctly. As tests are fixed in the future, we'll set ``continue-on-error`` to false on the PyPy CI.
ENH: support zoneinfo tzinfos
State: closed

- [x] closes #37654 (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

The tzlocal-handling code is effectively using the tzinfo/datetime API "correctly", so this adapts zoneinfo cases to go through that path.

We may be able to eek out some extra perf by combining tzlocal/zoneinfo checks or something.  Just did the simplest thing here.

cc @pganssle note the comment about the cache in timezones.pyx L55.  Is this dangerous and if so can you suggest an alternative?

Potentially closes #43516, haven't checked.

Labels:
name: Timezones
description: Timezone data dtype

Comments:
looks like one of the code checks doesn't like "backports.zoneinfo" in the deps file.  guessing its the "." causing trouble
> cc @pganssle note the comment about the cache in timezones.pyx L55. Is this dangerous and if so can you suggest an alternative?

It's certainly "dangerous" in the sense that your UTC-detection logic will fail even for UTC-like zones. I think the whole endeavor is a bit foolhardy, as there is no reliable, guaranteed interface for determining whether a given zone is UTC. Your heuristics will work in a great many cases, so "user cleared the cache and people are using `ZoneInfo("UTC")`" is just another one of those.

That said, `str(some_zoneinfo_zone)` returns the `key` parameter if it exists, so you could presumably detect this with `isinstance(zone, ZoneInfo) and str(zone) == "UTC"`.
is the pre-commit failure real or a false-positive?
> is the pre-commit failure real or a false-positive?

I think it's a false positive as pre-commit.ci is being set up.

Looks pretty good; just needs a whatsnew entry.

Also these tests are running okay on the CI on all platforms? I see from the zoneinfo docs that tzdata might need to be installed if a platform doesn't have system tz data: https://docs.python.org/3/library/zoneinfo.html#data-sources
> Also these tests are running okay on the CI on all platforms?

Looks OK.  I dont know how common the no-tzdata case is.  i think we're OK punting on that and can revisit if/when someone opens an issue about it
> Looks OK. I dont know how common the no-tzdata case is. i think we're OK punting on that and can revisit if/when someone opens an issue about it

It was more of a question if our `ci/deps/` files need to install `tzdata` to be sure these tests are being run, but it sounds like they are.

IMO users should be responsible for installing tzdata if they need for personal use cases. 
Thanks LGTM.

Yeah a follow up calling this out as an enhancement that all Timstamp/DTI/etc can all accept ZoneInfo objects now would be good.
BUG: Categorical of booleans has object .categories
State: closed

- [x] closes #46313 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This patch adds a test for checking regression behavior as described in #46313. The bug was reported to be fixed and I could not reproduce the bug in the latest version as Mar.18, 2022. So, only a test was added.

This is my first contribution to the open-source project. Please let me know if something needs to be fixed. Thanks!

Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Categorical
description: Categorical Data Type

Comments:
Hi there, I moved the code to the specified location and removed the unnecessary line. 

Thanks for giving feedback! 
Thanks @rxd330
CLN: assorted
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Clean

Comments:
reverted Timedelta.freq removal, will do it The Right Way along with is_populated
TST: Additional tests to iloc setitem
State: closed

This tests make sure when converting multiple columns to datetimes
and when assiging back it remains as datetime not as unix date
as mentioned in GH #20511.

- [x] closes #20511 
- [x] Tests added and passed
- [x] All code checks passed
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite
name: Indexing
description: Related to indexing on series/frames, not to indexes themselves

Comments:
Hello @parthi-siva! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-04-28 15:06:16 UTC
REGR: Bar plot axis timestamp to string conversion error
State: closed

Added regression test to prevent bar plot from attempting to convert x-axis from string to timestamp due to previous plot's axis units.

Problematic example:
https://github.com/pandas-dev/pandas/blob/fb353446651067fafec03301256e93d59065c520/pandas/plotting/_matplotlib/core.py#L1433


- [x] closes #38736
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Testing
description: pandas testing functions or related to the test suite

Comments:
Thanks @ivan-ngchakming
REF: implement get_unit_from_dtype
State: closed

Finding I need this in a bunch of non-nano branches.

cpplint gave a warning I didn't know what to do with so ignored; pls advise.
```
pandas/_libs/tslibs/src/datetime/np_datetime.c:787:  Are you taking an address of a cast?  This is dangerous: could be a temp var.  Take the address before doing the cast, rather than after  [runtime/casting] [4]
```

@mattip would this be a good candidate for adding to numpy's `__init__.pxd`?

Labels:
name: Non-Nano
description: datetime64/timedelta64 with non-nanosecond resolution

Comments:
>  I guess if the cpplint warning can't be address in the near team

should have mentioned, fixed that in a subsequent commit, so we're all good.
Thanks @jbrockmendel
BUG: avoid RuntimeError in groupby.max
State: closed

- [ ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Addresses most of the remaining TODOs (except those waiting on cython3) i wanted to get done before pushing forward on more nullable cases.

Labels:
name: Groupby

Comments:
great, i guess this is technially a user visible change, so if you'd add a note in next pass
REGR: DataFrame.replace when the replacement value was explicitly None
State: closed

- [ ] closes #45601
- [ ] closes #45836
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: replace
description: replace method

Comments:
cool, looks like a conflict
thanks @simonjayhawkins 
@meeseeksdev backport 1.4.x
@meeseeksdev backport 1.4.x
Revert " BUG: RecursionError when attempting to replace np.nan values (#45725)"
State: closed

Reverts pandas-dev/pandas#45749

see https://github.com/pandas-dev/pandas/pull/45749#issuecomment-1070837621

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: replace
description: replace method

Comments:
yep cool with this
TYP/CI: Fix failing clipboard typing
State: closed

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).



Labels:
name: Typing
description: type annotations, mypy/pyright type checking

Comments:
cool merge on green
LGTM
REGR: only convert at end for Block.replace_list
State: closed

In Block.replace_list there is some logic to only convert at the end.

before #40082 the convert parameter of Block._replace_regex was honored, but #40082 changed it not convert. This caused a regression, #44864 and was then changed to always convert in #44897. In this PR, we restore the original behavior and honor the convert parameter, which is unused on master.

before #44940, the conversion was not done in Block._replace_coerce for the non-regex case. #44940 changed Block._replace_coerce to dispatch to Block._replace for the non-regex case. Block._replace converts by default.

i've removed _replace_coerce since it appears that the only logic there now is to decide whether to use Block.replace or Block._replace_regex and is only called from Block.replace_list. (The signatures for Block.replace or Block._replace_regex are now consistent in this PR)

I've labelled as a regression but not sure whether we would want to backport this. I have therefore marked as draft and will wait till the reported regressions #45836 and #45601 that are caused by #44940 to be fixed first to avoid merge conflicts with the backport if we don't backport this.

Alternatively we remove the convert logic from Block.replace_list as dead code.

Labels:
name: Regression
description: Functionality that used to work in a prior pandas version
name: replace
description: replace method
name: Stale

Comments:
> i've removed _replace_coerce since it appears that the only logic there now is to decide whether to use Block.replace or Block._replace_regex and is only called from Block.replace_list. (The signatures for Block.replace or Block._replace_regex are now consistent in this PR)

i've re-instated this since there is also now some logic for None handling there that ultimately we may either want to make consistent for a list-like and scalar `to_replace` or maybe handle in Block.replace. Either way, that change would not have been suitable for a backport for the fix #46404

`_replace_coerce ` is only called when `to_replace` is not a scalar.

handling in Block.replace for a list-like (on main) should probably wait till #45725 is resolved. 
> i've re-instated this

oops. forgot to add the convert argument back in. will do that now.
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Added the round( ) function to avoid error while multiplication of float numbers issue: #46362
State: closed

- Added round( ) function to round the values and then convert it to int type while they are passed into the list-out, so that we can avoid conversations of int(28.999999999999996) to 28.
- if we round the value first it will be converted into 29 and then formating them to int type.
- ✅ closes #46362 
- ✅ [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- ✅ All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- ✅ Added an entry in the latest `doc/source/whatsnew/v1.4.2.rst`


Labels:
name: Bug
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: Stale

Comments:
Hello @ShamithaReddy! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



* In the file [`pandas/tests/io/formats/test_format.py`](https://github.com/pandas-dev/pandas/blob/693a4b8a54c0054443bda2954d9650831a6bea94/pandas/tests/io/formats/test_format.py):

> [Line 3266:43](https://github.com/pandas-dev/pandas/blob/693a4b8a54c0054443bda2954d9650831a6bea94/pandas/tests/io/formats/test_format.py#L3266): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','
> [Line 3266:48](https://github.com/pandas-dev/pandas/blob/693a4b8a54c0054443bda2954d9650831a6bea94/pandas/tests/io/formats/test_format.py#L3266): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','
> [Line 3266:53](https://github.com/pandas-dev/pandas/blob/693a4b8a54c0054443bda2954d9650831a6bea94/pandas/tests/io/formats/test_format.py#L3266): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ','



##### Comment last updated at 2022-03-19 08:18:40 UTC
> needs a test to replicate the original issue (the describe case)
@jreback 
I did already include the original test case in https://github.com/pandas-dev/pandas/pull/46388/commits/998f9174c27d3cee9eddc90428e0e0765ee5979a

This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to be stale. If interested in continuing, please merge in the main branch, address the review, and we can reopen.