ENH: Support malformed row handling in Python engine
State: closed

Support `warn_bad_lines` and `error_bad_lines` for the Python engine.

xref #12686 (master tracker)

Inspired by <a href="https://github.com/pandas-dev/pandas/issues/15910#issuecomment-291998838">#15910 (comment)</a>

In addition, the Python parser now raises `pandas.error.ParserError`, which is in line with what the C engine would do.

Labels:
name: Enhancement
name: Error Reporting
description: Incorrect or improved errors from pandas
name: IO CSV
description: read_csv, to_csv

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=h1) Report
> Merging [#15925](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/4502e82083f4e253630588665a4fc6002c4f32ed?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `96.42%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/15925/graphs/tree.svg?height=150&token=eZ4WkYLtcO&width=650&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #15925      +/-   ##
==========================================
+ Coverage   90.96%   90.97%   +<.01%     
==========================================
  Files         145      145              
  Lines       49557    49576      +19     
==========================================
+ Hits        45081    45100      +19     
  Misses       4476     4476
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.73% <96.42%> (ø)` | :arrow_up: |
| #single | `40.6% <3.57%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.64% <96.42%> (-0.02%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `91.03% <0%> (+0.34%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=footer). Last update [4502e82...6ab5602](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=h1) Report
> Merging [#15925](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f478e4f4b0a353fa48ddb19e70cb9abe5b36e1b5?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `96.42%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/15925/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #15925      +/-   ##
==========================================
- Coverage   90.99%   90.99%   -0.01%     
==========================================
  Files         145      145              
  Lines       49520    49540      +20     
==========================================
+ Hits        45061    45077      +16     
- Misses       4459     4463       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.75% <96.42%> (-0.01%)` | :arrow_down: |
| #single | `40.6% <3.57%> (-0.02%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.64% <96.42%> (-0.02%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `90.68% <0%> (-0.35%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `80.66% <0%> (-0.19%)` | :arrow_down: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `93.06% <0%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=footer). Last update [f478e4f...e3a8cca](https://codecov.io/gh/pandas-dev/pandas/pull/15925?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
can you run asv on the parsers to verify no regressions? not 100% sure we actually have one for python engine though......
@jreback : No noticeable regressions AFAICT.  We actually do have Python engine benchmarks FYI in the `io_bench.py` file under `asv_bench`.
@jreback : Everything is green and ready to go.
thanks!