BUG: handle nan values in DataFrame.update when overwrite=False (#15593)
State: closed


BUG: handle nan values in DataFrame.update when overwrite=False (#15593)

add nan test for DataFrame.update

update whatsnew v0.20.2

 - [x] closes #15593 
closes #15617
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Missing-data
description: np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
name: Reshaping
description: Concat, Merge/Join, Stack/Unstack, Explode
name: Dtype Conversions
description: Unexpected or buggy dtype conversions

Comments:
yes, i just made the changes.

On Mon, May 22, 2017 at 3:58 PM, Tom Augspurger <notifications@github.com>
wrote:

> *@TomAugspurger* commented on this pull request.
> ------------------------------
>
> In doc/source/whatsnew/v0.20.2.txt
> <https://github.com/pandas-dev/pandas/pull/16430#discussion_r117831062>:
>
> > @@ -37,6 +37,7 @@ Bug Fixes
>  ~~~~~~~~~
>
>  - Bug in using ``pathlib.Path`` or ``py.path.local`` objects with io functions (:issue:`16291`)
> +- Bug in using ``DataFrame.update`` when overwrite=False and the second value is nan
>
> Could you add a ref to the issue ath the end here
>
> (:issue:`15593`)
>
> ------------------------------
>
> In pandas/tests/frame/test_combine_concat.py
> <https://github.com/pandas-dev/pandas/pull/16430#discussion_r117831818>:
>
> > @@ -763,3 +763,13 @@ def test_concat_datetime_datetime64_frame(self):
>
>          # it works!
>          pd.concat([df1, df2_obj])
> +
> +
> +class TestDataFrameUpdate(TestData):
> +
> +    def test_update_nan(self):
> +        # #15593 #15617
> +        df1 = DataFrame({'A': [1, None, 3], 'B': date_range('2000', periods=3)})
> +        df2 = DataFrame({'A': [None, 2, 3]})
> +        df1.update(df2, overwrite=False)
> +        # it works!
>
> could you make an exected dataframe and compare?
>
> expected = df1 = DataFrame({'A': [1.0, 2, 3], 'B': date_range('2000', periods=3)})
> tm.assert_frame_equal(df1, expected)
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/16430#pullrequestreview-39570384>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABSVsjx9erVVSzQBeyQlFJlbjjRXyqd3ks5r8ejbgaJpZM4Niywe>
> .
>

@pcluo can you update with the requested changes? Thanks!
@TomAugspurger yes, plz check.
@TomAugspurger just made another attempt at this. Sorry the last conversation got lost when i squashed commits. I changed [1, 2, 3] in test 2 to [1.0, 2, 3] otherwise the dtypes are not equal, float64 vs int64.
can you rebase on master
>  I changed [1, 2, 3] in test 2 to [1.0, 2, 3] otherwise the dtypes are not equal, float64 vs int64.

Yeah, that sounds right.

If you rebase and force push, all the tests should pass.

```
git fetch upstream
git rebase upstream/master
git push -f origin master
```

FYI, typically you want to checkout a new branch for features / bug fixes. http://pandas.pydata.org/pandas-docs/stable/contributing.html#creating-a-branch Not a huge deal though.
it seems all tests passed now. thx @TomAugspurger  and will checkout a new branch next time.
Looks like Travis isn't picking up your commits for some reason. I'll ask their support.
thanks!
FYI @pcluo TravisCI support thought that if you sign into travis with your github account then things should be good.