Repository: pandas-dev/pandas

Issue:
Assigning dataframe by specifying both row/col doesn't handle nan correctly
If I run the following code under pandas 0.11, I will get different results from the 2 identical assign statment at the bottom:

```
import numpy as np
import pandas as pd

if __name__ == '__main__':
    df = pd.DataFrame({'FC':['a','b','a','b','a','b'],
                       'PF':[0,0,0,0,1,1],
                       'col1':np.random.rand(6),
                       'col2':np.random.rand(6)});
    df.ix[1,0]=np.nan
    mask=~df.FC.isnull()
    cols=['col1', 'col2']

    dft = df * 2
    dft.ix[3,3] = np.nan


    # bug?
    df.ix[mask, cols]= dft.ix[mask, cols]
    print df
    df.ix[mask, cols]= dft.ix[mask, cols]
    print df
```

Notice in the second result, the NaN at [3,3] disappeared and all values below it got shifted up.

```
    FC  PF      col1      col2
0    a   0  0.679388  0.508501
1  NaN   0  0.168159  0.346730
2    a   0  1.802611  0.870135
3    b   0  0.577989       NaN
4    a   1  1.027070  0.530890
5    b   1  0.383662  1.855584

    FC  PF      col1      col2
0    a   0  0.679388  0.508501
1  NaN   0  0.168159  0.346730
2    a   0  1.802611  0.870135
3    b   0  0.577989  0.530890
4    a   1  1.027070  1.855584
5    b   1  0.383662  0.508501    
```

The issue seems to be in pandas index.py line 143:
    v = v.reindex(self.obj[item].reindex(v.index).dropna().index)
notice it's dropping NA from the target.

I then tried to use df.ix[mask, cols]= dft.ix[mask, cols].values to bypass this, and it failed also. The problem is in pandas index.py line 149:
    if len(labels) != len(value):
notice it's comparing number of columns to be assigned against number of rows in ndarray.



Discussion:
```
df = DataFrame({'FC':['a','b','a','b','a','b'],
                        'PF':[0,0,0,0,1,1],
                        'col1':range(6),
                        'col2':range(6,12)})

In [5]: df
Out[5]: 
    FC  PF  col1  col2
0    a   0     0     6
1  NaN   0     1     7
2    a   0     2     8
3    b   0     3     9
4    a   1     4    10
5    b   1     5    11

df2 = df.copy()

In [6]: mask
Out[6]: 
0     True
1    False
2     True
3     True
4     True
5     True
dtype: bool

In [7]: dft
Out[7]: 
    FC  PF  col1  col2
0   aa   0     0    12
1  NaN   0     2    14
2   aa   0     4    16
3   bb   0     6   NaN
4   aa   2     8    20
5   bb   2    10    22

In [8]: df2.ix[mask, cols]= dft.ix[mask, cols]

In [9]: df2
Out[9]: 
    FC  PF  col1  col2
0    a   0     0    12
1  NaN   0     1     7
2    a   0     4    16
3    b   0     6   NaN
4    a   1     8    20
5    b   1    10    22
```

I'm using 0.11. The older version doesn't support this kind of assignment. 

notice in your test, the mask is inversed, so the null value problem doesn't show up.

ah..you are right...copy error.....I will take a look, could be a bug in this as I allowed this starting in 0.11 (in theory in certain cases in might have worked < 0.11, that's why I asked)

@jianpan I updated the results (I switched a range of ints, easier to see alignment), this looks right to me....

(I didn't change any code)


Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -138,14 +138,14 @@ def setter(item, v):
                         # align to
                         if item in value:
                             v = value[item]
-                            v = v.reindex(self.obj[item].reindex(v.index).dropna().index)
+                            v = v.reindex(self.obj[item].index & v.index)
                             setter(item, v.values)
                         else:
                             setter(item, np.nan)
 
                 # we have an equal len ndarray
-                elif isinstance(value, np.ndarray) and value.ndim > 1:
-                    if len(labels) != len(value):
+                elif isinstance(value, np.ndarray) and value.ndim == 2:
+                    if len(labels) != value.shape[1]:
                         raise ValueError('Must have equal len keys and value when'
                                          ' setting with an ndarray')
 


Test patch:
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -853,6 +853,41 @@ def test_iloc_panel_issue(self):
         self.assert_(p.iloc[1, :3, 1].shape == (3,))
         self.assert_(p.iloc[:3, 1, 1].shape == (3,))
       
+    def test_multi_assign(self):
+
+        # GH 3626, an assignement of a sub-df to a df
+        df = DataFrame({'FC':['a','b','a','b','a','b'],
+                        'PF':[0,0,0,0,1,1],
+                        'col1':range(6),
+                        'col2':range(6,12)})
+        df.ix[1,0]=np.nan
+        df2 = df.copy()
+
+        mask=~df2.FC.isnull()
+        cols=['col1', 'col2']
+
+        dft = df2 * 2
+        dft.ix[3,3] = np.nan
+
+        expected = DataFrame({'FC':['a',np.nan,'a','b','a','b'],
+                              'PF':[0,0,0,0,1,1],
+                              'col1':Series([0,1,4,6,8,10],dtype='float64'),
+                              'col2':[12,7,16,np.nan,20,22]})
+        
+
+        # frame on rhs
+        df2.ix[mask, cols]= dft.ix[mask, cols]
+        assert_frame_equal(df2,expected)
+        df2.ix[mask, cols]= dft.ix[mask, cols]
+        assert_frame_equal(df2,expected)
+
+        # with an ndarray on rhs
+        df2 = df.copy()
+        df2.ix[mask, cols]= dft.ix[mask, cols].values
+        assert_frame_equal(df2,expected)
+        df2.ix[mask, cols]= dft.ix[mask, cols].values
+        assert_frame_equal(df2,expected)
+
 
 if __name__ == '__main__':
     import nose