BUG-26988 implement replace for categorical blocks
State: closed

- [X] closes #26988 
- [X] tests added / passed
- [X] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [X] whatsnew entry

The replace functions used in the code path that caused the bug report appeared to densify the series. This PR adds a replace function to `CategoricalBlock` that takes advantage of the categorical data type when possible

Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Categorical
description: Categorical Data Type

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=h1) Report
> Merging [#27026](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/83fe8d78b6b086f3ceabe81cd420a3c7affe9aba?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `75%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/27026/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #27026      +/-   ##
==========================================
- Coverage   91.99%   91.98%   -0.02%     
==========================================
  Files         180      180              
  Lines       50774    50790      +16     
==========================================
+ Hits        46712    46719       +7     
- Misses       4062     4071       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.62% <68.75%> (-0.01%)` | :arrow_down: |
| #single | `41.79% <56.25%> (-0.12%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `94.17% <75%> (-0.22%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `88.88% <0%> (-11.12%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.89% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.84% <0%> (-0.11%)` | :arrow_down: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `97.91% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=footer). Last update [83fe8d7...a2f2a7c](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=h1) Report
> Merging [#27026](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/04893a954b91574279c402e8730a4b5fae2ae9e1?src=pr&el=desc) will **decrease** coverage by `1.21%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/27026/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #27026      +/-   ##
==========================================
- Coverage   93.07%   91.86%   -1.22%     
==========================================
  Files         192      180      -12     
  Lines       49562    50833    +1271     
==========================================
+ Hits        46130    46697     +567     
- Misses       3432     4136     +704
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.5% <93.33%> (-1.33%)` | :arrow_down: |
| #single | `41.84% <46.66%> (-0.67%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `95.03% <100%> (-1.54%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `88.88% <0%> (-11.12%)` | :arrow_down: |
| [pandas/core/computation/expressions.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vZXhwcmVzc2lvbnMucHk=) | `93.27% <0%> (-5.81%)` | :arrow_down: |
| [pandas/compat/pickle\_compat.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvbXBhdC9waWNrbGVfY29tcGF0LnB5) | `63.88% <0%> (-4.72%)` | :arrow_down: |
| [pandas/plotting/\_matplotlib/timeseries.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19tYXRwbG90bGliL3RpbWVzZXJpZXMucHk=) | `66.32% <0%> (-3.79%)` | :arrow_down: |
| [pandas/core/computation/scope.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tcHV0YXRpb24vc2NvcGUucHk=) | `93.13% <0%> (-3.74%)` | :arrow_down: |
| [pandas/compat/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2NvbXBhdC9fX2luaXRfXy5weQ==) | `92% <0%> (-2.88%)` | :arrow_down: |
| [pandas/io/parquet.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnF1ZXQucHk=) | `65.59% <0%> (-2.55%)` | :arrow_down: |
| [pandas/io/formats/printing.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvcHJpbnRpbmcucHk=) | `86.72% <0%> (-2.49%)` | :arrow_down: |
| [pandas/util/\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RlY29yYXRvcnMucHk=) | `92.13% <0%> (-2.44%)` | :arrow_down: |
| ... and [182 more](https://codecov.io/gh/pandas-dev/pandas/pull/27026/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=footer). Last update [04893a9...8667b0d](https://codecov.io/gh/pandas-dev/pandas/pull/27026?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Failure unrelated #26842 
Are there tests for the non-None `filter` branch?
@jbrockmendel yes, the dataframe tests take that branch while the series tests do not
> I am -1 on this PR as is; this should be a method on a Categorical.

If it goes on Categorical, it should be generalized to EA.
> > I am -1 on this PR as is; this should be a method on a Categorical.
> 
> If it goes on Categorical, it should be generalized to EA.

probably. But I think its *much* better then jamming things in the blocks (now we *already* have replace on the blocks, this PR might be ok actually), but I don't fully remember. 
I don't think we (or downstream EA authors) can reliably implement ExtensionArray.replace since it's such a complicated method.

We're also relying on Block.replace to do the heavy lifting. An ExtensionArray.replace wouldn't have that luxury.
can you merge master
cc @jbrockmendel if any more comments
will take a look this afternoon
Only major question is whether we need to add `replace` to the EA interface.
this looked pretty good, can you merge master and address any remaining comments.
can you merge master
@JustinZhengBC can you merge master.  itd be nice to get this in
@jbrockmendel done
@jreback how should parameters like `to_replace` and `value` be typed when they can take on any type and might not even be the same type?
> @jreback how should parameters like `to_replace` and `value` be typed when they can take on any type and might not even be the same type?

right, but they are something like Union[Scalar, AnyArrayLike] I think?  (again this is likely tricky so should do as a followup)
Fixed the merge conflict.

I think we decided to leave typing as a followup, so this should be good to go?
@JustinZhengBC if you can address @jbrockmendel comments to add some comments this lgtm and we can merge. (alternatvely @jbrockmendel could do as a followup)
@jreback @jbrockmendel I added the comment
Seems like this is good to merge.
CC. @jreback @jbrockmendel
thanks for the patch @JustinZhengBC very nice!