Fix df.to_csv() for string arrays when encoded in utf-8
State: closed

So it looks like `df.to_csv()` is now working correctly for string arrays when using the `ascii` encoding but it is still broken when using `utf-8`.


Labels:
name: Output-Formatting
description: __repr__ of pandas objects, to_string
name: IO CSV
description: read_csv, to_csv

Comments:
@jreback It seems the current behaviour stems from the fact that pandas' own `UnicodeWriter` calls `pprint_thing` without `quote_strings=True` so then:

```python
>>> from pandas.io.formats.printing import pprint_thing
>>> pprint_thing([u'foo', u'bar'])
u'[foo, bar]'
```
instead of the more intuitive (at least to me):
```python
>>> pprint_thing([u'foo', u'bar'], quote_strings=True)
u"[u'foo', u'bar']"
```

A couple of questions come to mind:
1) Can you recall why we have our own `UnicodeWriter` [here](https://github.com/pandas-dev/pandas/blob/master/pandas/io/formats/format.py#L1642) instead of e.g. [unicodecsv.writer](https://github.com/jdunck/python-unicodecsv)?
2) Better to expose the `quote_strings` (or similar) parameter to the `to_csv` caller, or is the behaviour I expect so ubiquitously intuitive that we should be changing things under the hood?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=h1) Report
> Merging [#18013](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/39a6b8f20fe95f5b6bb91c577cf5dea81214932c?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18013/graphs/tree.svg?height=150&width=650&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18013      +/-   ##
==========================================
- Coverage   91.24%   91.24%   -0.01%     
==========================================
  Files         163      163              
  Lines       50176    50124      -52     
==========================================
- Hits        45785    45734      -51     
+ Misses       4391     4390       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.05% <ø> (ø)` | :arrow_up: |
| #single | `40.32% <ø> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/tseries/frequencies.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvZnJlcXVlbmNpZXMucHk=) | `96% <0%> (-0.11%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.75% <0%> (-0.1%)` | :arrow_down: |
| [pandas/io/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2lvL2V4Y2VsLnB5) | `80.39% <0%> (-0.01%)` | :arrow_down: |
| [pandas/io/stata.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2lvL3N0YXRhLnB5) | `93.7% <0%> (-0.01%)` | :arrow_down: |
| [pandas/tseries/offsets.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvb2Zmc2V0cy5weQ==) | `97.15% <0%> (-0.01%)` | :arrow_down: |
| [pandas/io/sas/sas\_xport.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfeHBvcnQucHk=) | `90.27% <0%> (ø)` | :arrow_up: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.26% <0%> (ø)` | :arrow_up: |
| [pandas/tslib.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL3RzbGliLnB5) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `82.45% <0%> (ø)` | :arrow_up: |
| ... and [17 more](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=footer). Last update [39a6b8f...86a3a1f](https://codecov.io/gh/pandas-dev/pandas/pull/18013?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

will have a look
thanks @rtkaleta 

love for you to take a stab at fixing the xfailed unicode case!