Repository: pandas-dev/pandas

Issue:
Bug? Replacing NaN values based on a condition.
Dataframe with 2 columns: A and B. If values in B are larger than values in A - replace those values with values of A. I used to do this by doing df.B[df.B > df.A] = df.A, however recent upgrade of pandas started giving a SettingWithCopyWarning when encountering this chained assignment. Official documentation recommends using .loc.

Okay, I said, and did it through df.loc[df.B > df.A, 'B'] = df.A and it all works fine, unless column B has all values of NaN. Then something weird happens:

``` python
In [1]: df = pd.DataFrame({'A': [1, 2, 3],'B': [np.NaN, np.NaN, np.NaN]})

In [2]: df
Out[2]: 
   A   B
0  1 NaN
1  2 NaN
2  3 NaN

In [3]: df.loc[df.B > df.A, 'B'] = df.A

In [4]: df
Out[4]: 
   A                    B
0  1 -9223372036854775808
1  2 -9223372036854775808
2  3 -9223372036854775808
```

Now, if even one of B's elements satisfies the condition (larger than A), then it all works fine:

``` python
In [1]: df = pd.DataFrame({'A': [1, 2, 3],'B': [np.NaN, 4, np.NaN]})

In [2]: df
Out[2]: 
   A   B
0  1 NaN
1  2   4
2  3 NaN

In [3]: df.loc[df.B > df.A, 'B'] = df.A

In [4]: df
Out[4]: 
   A   B
0  1 NaN
1  2   2
2  3 NaN
```

But if none of B's elements satisfy, then all NaNs get replaces with -9223372036854775808:

``` python
In [1]: df = pd.DataFrame({'A':[1,2,3],'B':[np.NaN,1,np.NaN]})

In [2]: df
Out[2]: 
   A   B
0  1 NaN
1  2   1
2  3 NaN

In [3]: df.loc[df.B > df.A, 'B'] = df.A

In [4]: df
Out[4]: 
   A                    B
0  1 -9223372036854775808
1  2                    1
2  3 -9223372036854775808
```

Am I doing something wrong, or this is a bug?

pandas: 0.15.0



Discussion:
I think there's some weird type conversion going on, I can't figure out why `B` converts to `int64` here:

``` python
import pandas as pd
import numpy as np

df = pd.DataFrame({'A': [1, 2, 3],'B': [np.NaN, np.NaN, np.NaN]})
df
Out[6]: 
   A   B
0  1 NaN
1  2 NaN
2  3 NaN

df.dtypes
Out[10]: 
A      int64
B    float64
dtype: object

# This loc shouldn't match any rows
df.loc[df.B > df.A, 'B'] = df.A
df
Out[8]: 
   A                    B
0  1 -9223372036854775808
1  2 -9223372036854775808
2  3 -9223372036854775808

# Why has this become int? Is this expected
# behaviour for this assignment?
df.dtypes
Out[12]: 
A    int64
B    int64
dtype: object
```

so the reason this shows up is that 

```
(Pdb) p np.array([np.nan]).astype(np.int64)
array([-9223372036854775808])
```

is happening, eg. coercing a float64 to a int64. It shouldn't be coercing because the indexer is empty (e.g. `df.B > df.A` is all `False`. So a buggy.


Resolving patch:
diff --git a/pandas/core/internals.py b/pandas/core/internals.py
--- a/pandas/core/internals.py
+++ b/pandas/core/internals.py
@@ -549,10 +549,31 @@ def setitem(self, indexer, value):
                                      "different length than the value")
 
         try:
+
+            def _is_scalar_indexer(indexer):
+                # treat a len 0 array like a scalar
+                # return True if we are all scalar indexers
+
+                if arr_value.ndim == 1:
+                    if not isinstance(indexer, tuple):
+                        indexer = tuple([indexer])
+
+                    def _is_ok(idx):
+
+                        if np.isscalar(idx):
+                            return True
+                        elif isinstance(idx, slice):
+                            return False
+                        return len(idx) == 0
+
+                    return all([ _is_ok(idx) for idx in indexer ])
+                return False
+
+
             # setting a single element for each dim and with a rhs that could be say a list
-            # GH 6043
-            if arr_value.ndim == 1 and (
-                np.isscalar(indexer) or (isinstance(indexer, tuple) and all([ np.isscalar(idx) for idx in indexer ]))):
+            # or empty indexers (so no astyping)
+            # GH 6043, 8669 (empty)
+            if _is_scalar_indexer(indexer):
                 values[indexer] = value
 
             # if we are an exact match (ex-broadcasting),


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -1043,6 +1043,13 @@ def test_loc_setitem_frame(self):
         expected = DataFrame(dict(A = Series(val1,index=keys1), B = Series(val2,index=keys2))).reindex(index=index)
         assert_frame_equal(df, expected)
 
+        # GH 8669
+        # invalid coercion of nan -> int
+        df = DataFrame({'A' : [1,2,3], 'B' : np.nan })
+        df.loc[df.B > df.A, 'B'] = df.A
+        expected = DataFrame({'A' : [1,2,3], 'B' : np.nan})
+        assert_frame_equal(df, expected)
+
         # GH 6546
         # setting with mixed labels
         df = DataFrame({1:[1,2],2:[3,4],'a':['a','b']})
@@ -1055,7 +1062,6 @@ def test_loc_setitem_frame(self):
         df.loc[0,[1,2]] = [5,6]
         assert_frame_equal(df, expected)
 
-
     def test_loc_setitem_frame_multiples(self):
         # multiple setting
         df = DataFrame({ 'A' : ['foo','bar','baz'],