ENH: adding functionality to support 'skipdatarows' flag in read_csv to …
State: closed

- [x] closes #23067 
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

Adding feature for issue https://github.com/pandas-dev/pandas/issues/23067#issue-368463820. The newly added 'skipdatarows' flag allows the first n rows excluding comments to be skipped when read_csv is called.

Labels:
name: Enhancement
name: IO CSV
description: read_csv, to_csv

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=h1) Report
> Merging [#26172](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc86509b44b3fb0cd9a1a6d6ed564b082dc50848?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26172/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26172      +/-   ##
==========================================
- Coverage   91.98%   91.98%   -0.01%     
==========================================
  Files         175      175              
  Lines       52377    52383       +6     
==========================================
+ Hits        48180    48182       +2     
- Misses       4197     4201       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.53% <100%> (ø)` | :arrow_up: |
| #single | `40.71% <50%> (-0.15%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.86% <100%> (+0.01%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.9% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=footer). Last update [dc86509...73a0e14](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=h1) Report
> Merging [#26172](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/dc86509b44b3fb0cd9a1a6d6ed564b082dc50848?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26172/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26172      +/-   ##
==========================================
- Coverage   91.98%   91.98%   -0.01%     
==========================================
  Files         175      175              
  Lines       52377    52383       +6     
==========================================
+ Hits        48180    48182       +2     
- Misses       4197     4201       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.53% <100%> (ø)` | :arrow_up: |
| #single | `40.71% <50%> (-0.15%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.86% <100%> (+0.01%)` | :arrow_up: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26172/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.9% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=footer). Last update [dc86509...73a0e14](https://codecov.io/gh/pandas-dev/pandas/pull/26172?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

I concur with @jreback here.  Enhancing `read_csv` by adding more `kwargs` to already bloated signature is not ideal for us.

Also, this is not proper skipping.  You're only truncating on the resulting `data`.  By skipping, you should not be parsing the data at all.  This is not quite possible to do with the Python engine (because we're leveraging the `csv` library), but this is certainly do-able with the C engine.
Closing this as a way to get ourselves to rethink a solution from scratch.

@mukundm19: Thanks for submitting this!  It was a good try, but I think we should discuss this more in the issue before implementing.