handle nan values in DataFrame.update when overwrite=False #15593
State: closed

closes #15593

Labels:
name: Bug
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode

Comments:
can you add some tests from the issue in pandas/tests/frame//test_combine_concat.py
and a whatsnew note (0.20.0) in the bug fixes section
I think it should be 
```
if isnull(that).all():
    continue
```
The line of `self[col] = expressions.where(mask, this, that, raise_on_error=True)` should be skipped on the absence of values in `that` i.e. column of the other dataframe whereas `mask` is computed on `this` or the column values of base dataframe for `overwrite=False`.

The committed solution works but is not correct I feel.
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15617?src=pr&el=h1) Report
> Merging [#15617](https://codecov.io/gh/pandas-dev/pandas/pull/15617?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/3d699884e26120618bf0bb8869bc07f1e51a2935?src=pr&el=desc) will **decrease** coverage by `-0.03%`.
> The diff coverage is `100%`.


```diff
@@            Coverage Diff             @@
##           master   #15617      +/-   ##
==========================================
- Coverage   91.02%   90.99%   -0.03%     
==========================================
  Files         143      143              
  Lines       49301    49301              
==========================================
- Hits        44875    44863      -12     
- Misses       4426     4438      +12
```


| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15617?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.87% <100%> (-0.1%)` | :x: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :x: |
| [pandas/tools/merge.py](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?src=pr&el=tree#diff-cGFuZGFzL3Rvb2xzL21lcmdlLnB5) | `91.76% <0%> (-0.35%)` | :x: |
| [pandas/tseries/index.py](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvaW5kZXgucHk=) | `95.29% <0%> (-0.1%)` | :x: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.25% <0%> (ø)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15617?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15617?src=pr&el=footer). Last update [3d69988...02ebf34](https://codecov.io/gh/pandas-dev/pandas/compare/3d699884e26120618bf0bb8869bc07f1e51a2935...02ebf34184dfe7ec3a46c52d059682bb446ffae7?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
closing as stale, if you'd like to update and add some tests pls do.
I still am receiving the invalid type promotion error in pandas 0.19.2. when using overwrite = False and both old df and new df contain NaN values for same indices. :-(
would like to re-open and add a short-term fix for this. just created a pull request #16430. @jreback please advise. long-time user of pandas but first time contributing. thx.
hmm for some reason not allowing to reopen can just submit another PR (you can use commits from this one) just submit it with a new branch