BUG: encoding error in to_csv compression
State: closed

- [x] closes #21241 
closes #21118
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Fix a problem where encoding wasn't handled properly in to_csv compression in Python 3. It was caused by dumping uncompressed csv on disk and reading it back to memory without passing specified encoding. Then platform tried to decode using default locale encoding which may or may not succeed. 

This PR added tests for non-ascii data with csv compression. Also, by using string buffer, this PR removed repeated disk roundtrip and redundant encoding/decoding which caused UnicodeDecodeError. There is performance improvement compared to 0.22 and 0.23. It also supports file-like object as input path_or_buf. 

Before this PR:
```
>>> df = DataFrame(100 * [[123, "abc", u"样本1", u"样本2"]], columns=['A', 'B', 'C', 'D'])
>>>
>>> def test_to_csv(df):
...     df.to_csv(
...         path_or_buf='test',
...         encoding='utf8',
...         compression='zip',
...         quoting=1,
...         sep='\t',
...         index=False)
...
>>> timeit(lambda: test_to_csv(df), number=5000)
11.856349980007508
```
After this PR:

```
>>> df = DataFrame(100 * [[123, "abc", u"样本1", u"样本2"]], columns=['A', 'B', 'C', 'D'])
>>>
>>> def test_to_csv(df):
...     df.to_csv(
...         path_or_buf='test',
...         encoding='utf8',
...         compression='zip',
...         quoting=1,
...         sep='\t',
...         index=False)
...
>>> timeit(lambda: test_to_csv(df), number=5000)
5.459916951993364
```

Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=h1) Report
> Merging [#21300](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4274b840e64374a39a0285c2174968588753ec35?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/21300/graphs/tree.svg?token=eZ4WkYLtcO&width=650&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #21300      +/-   ##
==========================================
+ Coverage   91.85%   91.85%   +<.01%     
==========================================
  Files         153      153              
  Lines       49546    49549       +3     
==========================================
+ Hits        45509    45512       +3     
  Misses       4037     4037
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.25% <100%> (ø)` | :arrow_up: |
| #single | `41.87% <57.14%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/csvs.py](https://codecov.io/gh/pandas-dev/pandas/pull/21300/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvY3N2cy5weQ==) | `98.14% <100%> (+0.01%)` | :arrow_up: |
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/21300/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `93.16% <0%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=footer). Last update [4274b84...4e914ff](https://codecov.io/gh/pandas-dev/pandas/pull/21300?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@WillAyd there had been changes since you last reviewed it. comments welcome.
looks ok to me. @WillAyd pls merge when satisfied.
Thanks @minggli !