ENH: support decimal option in PythonParser #12933
State: closed

- [x] closes #12933
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry


Labels:
name: Enhancement
name: IO CSV
description: read_csv, to_csv

Comments:
## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13189?src=pr) is **84.16%**

> Merging [#13189](https://codecov.io/gh/pydata/pandas/pull/13189?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will decrease coverage by **<.01%**

``` diff
@@             master     #13189   diff @@
==========================================
  Files           138        138          
  Lines         50496      50404    -92   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits          42501      42418    -83   
+ Misses         7995       7986     -9   
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [b88eb35...dc8ca62](https://codecov.io/gh/pydata/pandas/compare/b88eb35ad98ac7a99451b505acc74e5d0e3a81b1...dc8ca622ec771a1c6688ad065f1606c2d9439fb2)

can you post a quick benchmark on how this differents from current (with python engine; you will have to actually use the decimal as the sep otherwise won't have a good comparison), but just for sake of completness.

minor comments. pls squash; ping when green.

@gfyoung any comments?

@camilocot : 

1) Awesome that you got this to work!
2) Can you check this <a href="https://github.com/camilocot/pandas/blob/803356ef42543140c96699ad27091438277e760b/pandas/io/tests/parser/common.py#L44">test</a> and see what you get for the Python engine now?  The test (or the comment at least) should be changed.  Otherwise, LGTM.

@jreback regarding the benchmark, I'll read some asv documentation before posting it or with a %timeit is enought ?

Read the `asv` documentation.  Showing the results of those benchmark tests will be a lot more convincing to get this merged in.

Also, I looked at the test again that I asked you to change (with the comment), and I am unfortunately no longer convinced that a comment change is sufficient.  Can you change the `self.assertRaises` to `tm.assertRaisesRegexp(ValueError, <errmsg>, self.read_csv, StringIO(data), decimal='')`?  That will be stronger testing-wise.  Thanks!

yeah you can add to the asv benchmarks. I don't expect them to change significantly (as this is a special passed option), but nice to add a benchmark and (show the results here). 

@jreback: `asv` output of two new benchmarks using python engine:

```
▶ asv continuous master 12933  -b parser_vb.read_csv_pyth 
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing into conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
· Running 4 total benchmarks (2 commits * 1 environments * 2 benchmarks)
[  0.00%] · For pandas commit hash dc8ca622:
[  0.00%] ·· Building for conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..............................................
[  0.00%] ·· Benchmarking conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 25.00%] ··· Running parser_vb.read_csv_python_engine.time_read_csv_default_converter                                                                                                         2.86ms
[ 50.00%] ··· Running parser_vb.read_csv_python_engine.time_read_csv_default_converter_with_decimal                                                                                            9.47ms
[ 50.00%] · For pandas commit hash 86f68e6a:
[ 50.00%] ·· Building for conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 50.00%] ·· Benchmarking conda-py2.7-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 75.00%] ··· Running parser_vb.read_csv_python_engine.time_read_csv_default_converter                                                                                                         2.87ms
[100.00%] ··· Running parser_vb.read_csv_python_engine.time_read_csv_default_converter_with_decimal                                                                                            failed
SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
```

@gfyoung test assert was fixed, could you please take a look ?

@camilocot: Test assertion fix LGTM.  Thanks!

thanks @camilocot