Repository: pandas-dev/pandas

Issue:
to_csv() fail on 0.11.dev
Hit this after updating to '0.11.0.dev-da54321' from master.  Haven't had a chance to dig any deeper, other than isolate frame length as a factor.

```
df = pandas.util.testing.makeTimeDataFrame(25000)
df.to_csv("save.csv")  # works
df = pandas.util.testing.makeTimeDataFrame(25001)
df.to_csv("save.csv")  # throws exception below

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-83-12cc25e3eafd> in <module>()
----> 1 df.to_csv("save.csv")

/usr/local/lib/python2.7/dist-packages/pandas-0.11.0.dev_da54321-py2.7-linux-i686.egg/pandas/core/frame.pyc in to_csv(self, path_or_buf, sep, na_rep, float_format, cols, header, index, index_label, mode, nanRep, encoding, quoting, line_terminator, chunksize, **kwds)
   1348                                          index_label=index_label,
   1349                                          chunksize=chunksize,legacy=kwds.get("legacy",False) )
-> 1350             formatter.save()
   1351 
   1352     def to_excel(self, excel_writer, sheet_name='sheet1', na_rep='',

/usr/local/lib/python2.7/dist-packages/pandas-0.11.0.dev_da54321-py2.7-linux-i686.egg/pandas/core/format.pyc in save(self)
    936 
    937             else:
--> 938                 self._save()
    939 
    940 

/usr/local/lib/python2.7/dist-packages/pandas-0.11.0.dev_da54321-py2.7-linux-i686.egg/pandas/core/format.pyc in _save(self)
   1008                 break
   1009 
-> 1010             self._save_chunk(start_i, end_i)
   1011 
   1012     def _save_chunk(self, start_i, end_i):

/usr/local/lib/python2.7/dist-packages/pandas-0.11.0.dev_da54321-py2.7-linux-i686.egg/pandas/core/format.pyc in _save_chunk(self, start_i, end_i)
   1029         ix = data_index.to_native_types(slicer=slicer, na_rep=self.na_rep, float_format=self.float_format)
   1030 
-> 1031         lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
   1032 
   1033 # from collections import namedtuple

/usr/local/lib/python2.7/dist-packages/pandas-0.11.0.dev_da54321-py2.7-linux-i686.egg/pandas/lib.so in pandas.lib.write_csv_rows (pandas/lib.c:13152)()

IndexError: list index out of range
```

Current workaround:

```
df.to_csv("save.csv", legacy=True)
```



Discussion:
platform and numpy version?

i386 GNU/Linux and numpy 1.6.2

22f258f fixed a very similar boundry+1 issue for multiindex, due to a slicer arg being ignored
in MultiIndex `to_native_types`? maybe DateTimeIndex has the same issue?

it's 1 chunk plus 1 row

will take a look

I got it. 

```
core/index:to_native_types
        if self.is_all_dates:
            return _date_formatter(self)
        else:
            values[mask] = na_rep
```

should be

```
        if self.is_all_dates:
            return _date_formatter(self[slicer])
        else:
            values[mask] = na_rep
```

by visual inspection

to_native_types in core/index

is_all_dates is returning the formatted for self, should be values

will add test and fix tom

@gdraps thanks for the report

take it away jeff.... :)

faster than me!

use values instead of self[slicer]
already computed

it's a view isn't it? you do it. I'll beef up the torture test, I'm not sure I tested DateTimeIndex,
maybe just TimeStamp Objects.

it's a view

test at 8386da9


Resolving patch:
diff --git a/pandas/core/index.py b/pandas/core/index.py
--- a/pandas/core/index.py
+++ b/pandas/core/index.py
@@ -470,12 +470,11 @@ def to_native_types(self, slicer=None, na_rep='', float_format=None):
         values = self
         if slicer is not None:
             values = values[slicer]
-        mask = isnull(values)
-        values = np.array(values,dtype=object)
-
         if self.is_all_dates:
-            return _date_formatter(self)
+            return _date_formatter(values)
         else:
+            mask = isnull(values)
+            values = np.array(values,dtype=object)
             values[mask] = na_rep
 
         return values.tolist()


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -4697,6 +4697,22 @@ def test_to_csv_float32_nanrep(self):
                  lines = f.readlines()
                  self.assert_(lines[1].split(',')[2] == '999')
 
+    @slow
+    def test_to_csv_boundry_conditions(self):
+        from pandas.util.testing import makeTimeDataFrame
+
+        with ensure_clean() as path:
+
+            df = makeTimeDataFrame(25000)
+            df.to_csv(path)
+            rs = pan.read_csv(path, index_col=0, parse_dates=True)
+            assert_frame_equal(rs, df)
+
+            df = makeTimeDataFrame(25001)
+            df.to_csv(path) 
+            rs = pan.read_csv(path, index_col=0, parse_dates=True)
+            assert_frame_equal(rs, df)
+
     def test_to_csv_withcommas(self):
 
         # Commas inside fields should be correctly escaped when saving as CSV.