[BUG] maybe_upcast_putmast also handle ndarray
State: closed

-  closes #23823
-  2 tests added
-  passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
-  whatsnew entry

Follow h-vetinari's footstep. `maybe_upcast_putmask` was left untouched in #25425. Try to fix the bug so that `maybe_upcast_putmask` can also handle `ndarray` as well as `Series`.


Labels:
name: Bug
name: Dtype Conversions
description: Unexpected or buggy dtype conversions
name: Algos
description: Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=h1) Report
> Merging [#25431](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/85572de5e7bb188cfecc575ee56786406e79dc79?src=pr&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25431/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25431      +/-   ##
==========================================
+ Coverage   91.74%   91.77%   +0.03%     
==========================================
  Files         173      173              
  Lines       52923    52925       +2     
==========================================
+ Hits        48554    48572      +18     
+ Misses       4369     4353      -16
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.33% <100%> (+0.02%)` | :arrow_up: |
| #single | `41.72% <0%> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/25431/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `90.69% <100%> (+2.53%)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25431/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `87.66% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=footer). Last update [85572de...bf23643](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=h1) Report
> Merging [#25431](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/cdfdd77b65df350386ce27142ef3babd9e5186d2?src=pr&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25431/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25431      +/-   ##
==========================================
+ Coverage   91.27%    91.3%   +0.03%     
==========================================
  Files         173      173              
  Lines       53002    53004       +2     
==========================================
+ Hits        48375    48396      +21     
+ Misses       4627     4608      -19
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.87% <100%> (+0.03%)` | :arrow_up: |
| #single | `41.77% <33.33%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/dtypes/cast.py](https://codecov.io/gh/pandas-dev/pandas/pull/25431/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2Nhc3QucHk=) | `91.36% <100%> (+3.19%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=footer). Last update [cdfdd77...6b7c02d](https://codecov.io/gh/pandas-dev/pandas/pull/25431?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I investigated further and found that I misunderstood the usage of `maybe_upcast_putmask`. 

The result array should be replaced with the element of other array starting from the beginning, not in the position of mask. This behavior is due to np.place

 `changeit` isn't called. The 1st element is picked as expected.
```
In [4]: arr1 = np.array([1,2,3])                                                

In [5]: other = np.array([61,62,63])                                            

In [6]: mask = np.array([False, True, False])                                   

In [7]: result, _ = maybe_upcast_putmask(arr1, mask, other)                     

In [8]: result                                                                  
Out[8]: array([ 1, 61,  3])
```

`changeit` is called. The 1st element is picked as expected.
```
In [24]: arr2 = np.array([1,2,3])                                                                                                           

In [25]: other2 = np.array([61.1, 62.2, 63.3])                                                                                              

In [26]: mask = np.array([False, True, False])                                                                                              

In [27]: result, _ = maybe_upcast_putmask(arr2, mask, other2)                                                                               

In [28]: result                                                                                                                             
Out[28]: array([ 1. , 61.1,  3. ])
```

`changeit` isn't called. The 1st element is picked as expected.
```
In [15]: arr2 = np.arange('2019-01-01', '2019-01-04', dtype='datetime64[D]')    

In [16]: other2 = np.arange('2018-01-01', '2018-01-04', dtype='datetime64[D]')  

In [17]: mask = np.array([False, True, False])                                  

In [18]: result, _ = maybe_upcast_putmask(arr2, mask, other2)                   

In [19]: result                                                                 
Out[19]: array(['2019-01-01', '2018-01-01', '2019-01-03'], dtype='datetime64[D]')
```
So the behavior for Series as an input is not correct. And `maybe_upcast_putmask` does not expect a series input.

`changeit` is called. The second element is picked.
```
>>> s = pd.Series([10, 11, 12])
>>> t = pd.Series([np.nan, 61, np.nan])
>>> from pandas.core.dtypes.cast import maybe_upcast_putmask
>>> result, _ = maybe_upcast_putmask(s, np.array([False, True, False]), t)
>>> result  # correct
0    10
1    61
2    12
dtype: int64
```
Examples can be added into the function docstring to elucidate the usage. Should I do it in this PR?
 


When adding the tests, the following bug is found and fixed. 

```
In [4]: arr = np.arange(10, 15)                                                 

In [5]: other = np.array([61, 62])                                              

In [6]: mask = np.array([False, True, False, True, True])                       

In [7]: result, changed = maybe_upcast_putmask(arr, mask, other)                
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-7-3ec9bc405a70> in <module>
----> 1 result, changed = maybe_upcast_putmask(arr, mask, other)

~/Code/pandas/pandas/core/dtypes/cast.py in maybe_upcast_putmask(result, mask, other)
    242             else:
    243 
--> 244                 if isna(other[mask]).any():
    245                     return changeit()
    246 

IndexError: boolean index did not match indexed array along dimension 0; dimension is 2 but corresponding boolean dimension is 5
```

thanks @makbigc keep em coming!