Repository: pandas-dev/pandas

Issue:
BUG: Crash using to_csv with QUOTE_NONE
Using release 0.11.0 with Python 2.7.1, but it doesn't look like anything has changed in git since.

I can write DataFrames to a file using to_csv fine unless I pass the option: quoting=pandas.io.parsers.csv.QUOTE_NONE in which case I get exceptions of the form:

packages/pandas/core/format.py", line 1054, in _save_chunk
    lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
  File "lib.pyx", line 832, in pandas.lib.write_csv_rows (pandas/lib.c:13466)
_csv.Error: need to escape, but no escapechar set

A short search reveals links like http://www.velocityreviews.com/forums/t612166-how-can-i-use-quotes-without-escaping-them-using-csv.html implying that due to some csv oddities we should be setting something like quotechar=None when calling csv.writer.

I tried modifying pandas/core/format.py to add quotechar=None to the call to csv.writer and this does indeed seem to fix the problem. Unfortunately I don't have everything set up to build from git so I can't submit a patch.

Thanks.



Discussion:
seems to work fine in 0.11 (fyi `QUOTE_MINIMAL` is the default), what exactly are you trying to format

pls post your code

```
In [1]: pd.__version__
Out[1]: '0.11.0'

In [1]: df = DataFrame({'A': [1, 2, 3], 'B': ['foo', 'bar', 'baz']})

In [2]: buf = StringIO()

In [4]: import csv

In [8]: df.to_csv(buf, index=False, quoting=csv.QUOTE_NONE)

In [9]: buf.getvalue()
Out[9]: 'A,B\n1,foo\n2,bar\n3,baz\n'
```

I tried to get you a minimal reproduction of the issue. It seems to occur if a string in the DataFrame contains the quote character " but quoting is set to QUOTE_NONE:

import pandas
import StringIO
df = pandas.DataFrame({'A': ['hello', '{"hello"}']})
buf = StringIO.StringIO()
import csv
df.to_csv(buf, index=False, quoting=csv.QUOTE_NONE)

Running this in ipython causes a crash for me: "Error: need to escape, but no escapechar set"

And by way of validation:

In [7]: pandas.**version**
Out[7]: '0.11.0'

Thanks,

Adam

thanks for the report....and good example...marked as a bug for 0.13


Resolving patch:
diff --git a/pandas/core/format.py b/pandas/core/format.py
--- a/pandas/core/format.py
+++ b/pandas/core/format.py
@@ -787,7 +787,7 @@ def __init__(self, obj, path_or_buf, sep=",", na_rep='', float_format=None,
                  cols=None, header=True, index=True, index_label=None,
                  mode='w', nanRep=None, encoding=None, quoting=None,
                  line_terminator='\n', chunksize=None, engine=None,
-                 tupleize_cols=True):
+                 tupleize_cols=True, quotechar='"'):
 
         self.engine = engine  # remove for 0.13
         self.obj = obj
@@ -807,6 +807,11 @@ def __init__(self, obj, path_or_buf, sep=",", na_rep='', float_format=None,
             quoting = csv.QUOTE_MINIMAL
         self.quoting = quoting
 
+        if quoting == csv.QUOTE_NONE:
+            # prevents crash in _csv
+            quotechar = None
+        self.quotechar = quotechar
+
         self.line_terminator = line_terminator
 
         #GH3457
@@ -950,13 +955,14 @@ def save(self):
             close = True
 
         try:
+            writer_kwargs = dict(lineterminator=self.line_terminator,
+                                 delimiter=self.sep, quoting=self.quoting,
+                                 quotechar=self.quotechar)
             if self.encoding is not None:
-                self.writer = com.UnicodeWriter(f, lineterminator=self.line_terminator,
-                                                delimiter=self.sep, encoding=self.encoding,
-                                                quoting=self.quoting)
+                writer_kwargs['encoding'] = self.encoding
+                self.writer = com.UnicodeWriter(f, **writer_kwargs)
             else:
-                self.writer = csv.writer(f, lineterminator=self.line_terminator,
-                                         delimiter=self.sep, quoting=self.quoting)
+                self.writer = csv.writer(f, **writer_kwargs)
 
             if self.engine == 'python':
             # to be removed in 0.13


Test patch:
diff --git a/pandas/tests/test_frame.py b/pandas/tests/test_frame.py
--- a/pandas/tests/test_frame.py
+++ b/pandas/tests/test_frame.py
@@ -6,6 +6,7 @@
 from datetime import datetime, timedelta, time
 import operator
 import re
+import csv
 import unittest
 import nose
 
@@ -5465,8 +5466,6 @@ def test_to_csv_float_format(self):
             assert_frame_equal(rs, xp)
 
     def test_to_csv_quoting(self):
-        import csv
-
         df = DataFrame({'A': [1, 2, 3], 'B': ['foo', 'bar', 'baz']})
 
         buf = StringIO()
@@ -5489,8 +5488,6 @@ def test_to_csv_quoting(self):
         self.assertEqual(buf.getvalue(), text)
 
     def test_to_csv_unicodewriter_quoting(self):
-        import csv
-
         df = DataFrame({'A': [1, 2, 3], 'B': ['foo', 'bar', 'baz']})
 
         buf = StringIO()
@@ -5505,6 +5502,17 @@ def test_to_csv_unicodewriter_quoting(self):
 
         self.assertEqual(result, expected)
 
+    def test_to_csv_quote_none(self):
+        # GH4328
+        df = DataFrame({'A': ['hello', '{"hello"}']})
+        for encoding in (None, 'utf-8'):
+            buf = StringIO()
+            df.to_csv(buf, quoting=csv.QUOTE_NONE,
+                      encoding=encoding, index=False)
+            result = buf.getvalue()
+            expected = 'A\nhello\n{"hello"}\n'
+            self.assertEqual(result, expected)
+
     def test_to_csv_index_no_leading_comma(self):
         df = DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]},
                        index=['one', 'two', 'three'])