Repository: pandas-dev/pandas

Issue:
column datatype conversion impacts whole dataframe when using df.ix indexing
My dataframe has a timestamp column that is encoded as unix epoch. When I convert the column using named index selection it works fine, but when I use the '.ix' syntax it coerces the whole dataframe. Example:

```
import pandas as pd

df = pd.DataFrame(
    {'timestamp':[1413840976, 1413842580, 1413760580], 
     'delta':[1174, 904, 161], 
     'elapsed':[7673, 9277, 1470]
    })

df2 = df.copy()

df['timestamp'] = pd.to_datetime(df['timestamp'], unit='s')
df2.ix[:,2] = pd.to_datetime(df['timestamp'], unit='s')

df

    delta    elapsed    timestamp
0    1174    7673    2014-10-20 21:36:16
1    904    9277    2014-10-20 22:03:00
2    161    1470    2014-10-19 23:16:20

df2

    delta    elapsed    timestamp
0    2014-10-20 21:36:16    1970-01-01 00:00:00.000007673    1970-01-01 00:00:01.413840976
1    2014-10-20 22:03:00    1970-01-01 00:00:00.000009277    1970-01-01 00:00:01.413842580
2    2014-10-19 23:16:20    1970-01-01 00:00:00.000001470    1970-01-01 00:00:01.413760580
```

I strongly suspect the difference in behavior here is problematic and should be resolved. If this is actually "how things should work," I'd greatly appreciate it if someone could explain why the different indexing styles produce these different results



Discussion:
pls pd.show_versions()

Should be the most current distributions via Anaconda

## INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.13.1
Cython: 0.20.1
numpy: 1.8.1
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 2.1.0
sphinx: 1.2.2
patsy: 0.2.1
scikits.timeseries: None
dateutil: 1.5
pytz: 2014.3
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 2.0.2
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.5
sqlalchemy: 0.9.4
lxml: 3.3.5
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None

this was fixed in 0.14 or 0.14.1

0.15.0 is current and just released

Cool, guess Anaconda is lagging by a few releases (or I need to update?). I'll take your word for it and close the issue. Thanks!

you should be able to

conda update pandas and get 0.15 now

I don't know how to reopen the issue, but after updating and confirming that I've got 0.15.0, it still exhibits the behavior that inspired me to submit this bug. 

pd.show_version():

## INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.15.0
nose: 1.3.3
Cython: 0.20.1
numpy: 1.9.0
scipy: 0.14.0
statsmodels: 0.5.0
IPython: 2.1.0
sphinx: 1.2.2
patsy: 0.2.1
dateutil: 1.5
pytz: 2014.7
bottleneck: None
tables: 3.1.1
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 2.0.2
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.5.5
lxml: 3.3.5
bs4: 4.3.1
html5lib: None
httplib2: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.4
pymysql: None
psycopg2: None

Yep, I can confirm I get the same issue with master

@dmarx ok, something gone ary. It is supposed to be the same (and not touch anything else). Its is actually a bit tricky, as it DOES need to do some dtype inference (and possibly break up a block of dtypes), which is the problem here (e.g. say you have 2 int64 dtypes, then change one of them to datetime64[ns], this get split to 2 blocks, int64, and datetime64[ns] - this is internal).


Resolving patch:
diff --git a/pandas/core/indexing.py b/pandas/core/indexing.py
--- a/pandas/core/indexing.py
+++ b/pandas/core/indexing.py
@@ -482,6 +482,14 @@ def can_do_equal_len():
             if isinstance(indexer, tuple):
                 indexer = _maybe_convert_ix(*indexer)
 
+                # if we are setting on the info axis ONLY
+                # set using those methods to avoid block-splitting
+                # logic here
+                if len(indexer) > info_axis and com.is_integer(indexer[info_axis]) and all(
+                    _is_null_slice(idx) for i, idx in enumerate(indexer) if i != info_axis):
+                    self.obj[item_labels[indexer[info_axis]]] = value
+                    return
+
             if isinstance(value, ABCSeries):
                 value = self._align_series(indexer, value)
 


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -1000,7 +1000,10 @@ def test_fancy_setitem_int_labels(self):
         tmp = df.copy()
         exp = df.copy()
         tmp.ix[:, 2] = 5
-        exp.values[:, 2] = 5
+
+        # tmp correctly sets the dtype
+        # so match the exp way
+        exp[2] = 5
         assert_frame_equal(tmp, exp)
 
     def test_fancy_getitem_int_labels(self):
diff --git a/pandas/tests/test_indexing.py b/pandas/tests/test_indexing.py
--- a/pandas/tests/test_indexing.py
+++ b/pandas/tests/test_indexing.py
@@ -605,7 +605,8 @@ def test_iloc_setitem(self):
         expected = Series([0,1,0],index=[4,5,6])
         assert_series_equal(s, expected)
 
-    def test_loc_setitem(self):
+    def test_ix_loc_setitem(self):
+
         # GH 5771
         # loc with slice and series
         s = Series(0,index=[4,5,6])
@@ -627,6 +628,31 @@ def test_loc_setitem(self):
         expected = DataFrame({'a' : [0.5,-0.5,-1.5], 'b' : [0,1,2] })
         assert_frame_equal(df,expected)
 
+        # GH 8607
+        # ix setitem consistency
+        df = DataFrame(
+            {'timestamp':[1413840976, 1413842580, 1413760580],
+             'delta':[1174, 904, 161],
+             'elapsed':[7673, 9277, 1470]
+             })
+        expected = DataFrame(
+            {'timestamp':pd.to_datetime([1413840976, 1413842580, 1413760580], unit='s'),
+             'delta':[1174, 904, 161],
+             'elapsed':[7673, 9277, 1470]
+             })
+
+        df2 = df.copy()
+        df2['timestamp'] = pd.to_datetime(df['timestamp'], unit='s')
+        assert_frame_equal(df2,expected)
+
+        df2 = df.copy()
+        df2.loc[:,'timestamp'] = pd.to_datetime(df['timestamp'], unit='s')
+        assert_frame_equal(df2,expected)
+
+        df2 = df.copy()
+        df2.ix[:,2] = pd.to_datetime(df['timestamp'], unit='s')
+        assert_frame_equal(df2,expected)
+
     def test_loc_setitem_multiindex(self):
 
         # GH7190