pydata/pandas — pull request #3059 by y-p (2013-03-15 03:26:12+00:00)
Title: ENH: improve performance of df.to_csv GH3054

Needs more testing before merging.

Following [SO question](http://stackoverflow.com/questions/15417574) mentioned in #3054:

```python
In [7]: def df_to_csv(df,fname):
...: fh=open(fname,'w')
...: fh.write(','.join(df.columns) + '\n')
...: for row in df.itertuples(index=False):
...: slist = [str(x) for x in row]
...: ss = ','.join(slist) + "\n"
...: fh.write(ss)
...: fh.close()
...:
...: aa=pd.DataFrame({'A':range(100000)})
...: aa['B'] = aa.A + 1.0
...: aa['C'] = aa.A + 2.0
...: aa['D'] = aa.A + 3.0
...:
...: %timeit -r10 aa.to_csv('/tmp/junk1',index=False)
...: %timeit -r10 df_to_csv(aa,'/tmp/junk2')
...: from hashlib import sha1
...: print sha1(open("/tmp/junk1").read()).hexdigest()
...: print sha1(open("/tmp/junk2").read()).hexdigest()
```

current pandas	with PR	example code

2.3 s	1.29 s	1.28 s

wins:
- convert numpy numerics to native types to eliminate expensive numpy specific
stringify calls.
- if number of columns is < 10000, precompute the `cols` loop range rather
then creating and walking a generator at each iteration of the inner loop.
- some cargo cult stuff that's probably in the noise.