ENH: Improve error message for header argument containing non int types. GH16338
State: closed

Adds error "header must be integer or list of integers" when the header argument is a list, tuple or numpy array containing non-integers. Initially intended to read_csv, but applies to other functions with similar header arguments. GH16338 refers to a case in which the user mixes up the "names" and "header" arguments.

 - [x] closes #16338
 - [x] tests added / passed
 - [x] passes ``git diff upstream/master --name-only -- '*.py' | flake8 --diff``
 - [x] whatsnew entry


Labels:
name: Error Reporting
description: Incorrect or improved errors from pandas

Comments:
Lint failure [here](https://travis-ci.org/pandas-dev/pandas/jobs/232024816#L1398) If you `pip install flake8` you can run `git diff upstream/master --name-only -- '*.py' | flake8 --diff` locally.

The other failure looked unrelated, so I've restarted that job. Once you've fixed the linting error, you can push another commit to this branch.
Also need to test that you can hit this error message!
Thanks @mjlove12 for picking this up.

The lint error is `pandas/io/parsers.py:1188:1: W293 blank line contains whitespace`. It will be fixed by removing the whitespace on line 1188 (which is a blank line).
Thanks @pierre-haessig. I'm planning to make the changes this week,
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=h1) Report
> Merging [#16351](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0f55de1b334131dd0577b91eb061dcb8bc65ef78?src=pr&el=desc) will **decrease** coverage by `0.04%`.
> The diff coverage is `50%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16351/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16351      +/-   ##
==========================================
- Coverage   90.41%   90.36%   -0.05%     
==========================================
  Files         161      161              
  Lines       50997    50918      -79     
==========================================
- Hits        46107    46012      -95     
- Misses       4890     4906      +16
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.13% <50%> (-0.11%)` | :arrow_down: |
| #single | `40.22% <0%> (+0.02%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.26% <50%> (-0.06%)` | :arrow_down: |
| [pandas/core/util/hashing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdXRpbC9oYXNoaW5nLnB5) | `90.35% <0%> (-2.62%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.23% <0%> (-1.82%)` | :arrow_down: |
| [pandas/io/pickle.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL3BpY2tsZS5weQ==) | `79.54% <0%> (-1.71%)` | :arrow_down: |
| [pandas/io/feather\_format.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2ZlYXRoZXJfZm9ybWF0LnB5) | `86.66% <0%> (-1.22%)` | :arrow_down: |
| [pandas/io/formats/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZXhjZWwucHk=) | `73.15% <0%> (-1.1%)` | :arrow_down: |
| [pandas/io/sas/sasreader.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNyZWFkZXIucHk=) | `85.18% <0%> (-1.03%)` | :arrow_down: |
| [pandas/core/common.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29tbW9uLnB5) | `90.68% <0%> (-0.71%)` | :arrow_down: |
| [pandas/io/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2V4Y2VsLnB5) | `61.68% <0%> (-0.58%)` | :arrow_down: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `91.94% <0%> (-0.19%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=footer). Last update [0f55de1...888bdd0](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=h1) Report
> Merging [#16351](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/04356a83c0dc8a749c84e168535e6673f2548ec6?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/16351/graphs/tree.svg?height=150&token=eZ4WkYLtcO&width=650&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #16351      +/-   ##
==========================================
- Coverage   90.42%   90.41%   -0.02%     
==========================================
  Files         161      161              
  Lines       51027    51003      -24     
==========================================
- Hits        46142    46113      -29     
- Misses       4885     4890       +5
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `88.24% <100%> (-0.02%)` | :arrow_down: |
| #single | `40.19% <25%> (+0.01%)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.33% <100%> (+0.01%)` | :arrow_up: |
| [pandas/io/formats/format.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZm9ybWF0LnB5) | `95.66% <0%> (-0.37%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `93.94% <0%> (-0.24%)` | :arrow_down: |
| [pandas/io/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2V4Y2VsLnB5) | `62.26% <0%> (-0.06%)` | :arrow_down: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `95.91% <0%> (-0.02%)` | :arrow_down: |
| [pandas/core/reshape/reshape.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9yZXNoYXBlLnB5) | `99.28% <0%> (-0.01%)` | :arrow_down: |
| [pandas/errors/\_\_init\_\_.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2Vycm9ycy9fX2luaXRfXy5weQ==) | `100% <0%> (ø)` | :arrow_up: |
| [pandas/core/dtypes/concat.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZHR5cGVzL2NvbmNhdC5weQ==) | `98.07% <0%> (ø)` | :arrow_up: |
| [pandas/core/missing.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvbWlzc2luZy5weQ==) | `84.27% <0%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.69% <0%> (ø)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=footer). Last update [04356a8...de005a0](https://codecov.io/gh/pandas-dev/pandas/pull/16351?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Okay, I hope I did this right.
1. Added extra lines to account for cases when the header is non-array-like. 
2. Added a test (test_non_int_header()) in the header.py file under the Header_Tests class which includes a non-array-like case and an array-like case.
3. Added changes with "git add ."
4. Committed the changes to my branch with "git commit -m 'Revising PR16351 based on feedback'"
5. Rebased with "git rebase -i origin/master" and squashing the second commit instead of "picking" it
6. Forced a push to origin with "git push origin new-read_csv-header-comments -f"

Feedback appreciated if any of those steps are discouraged. 

Also, random question:
Can anyone explain why the test cases are wrapped in classes and how that works? I tried to match some of the other tests, but didn't understand why we were using the test classes to call methods like "self.read_csv" and "self.read_table" instead of just pd.read_csv. 

Thanks for all the help!
Matt
@mjlove12 : Your steps are perfectly fine.  Don't worry about it too much. :smile:

As for why we use `self.read_csv`, it's because there are multiple engines that can be used when parsing a CSV.  The two main ones are:
* C engine: this is `pandas` in-house parser backed up in C and Cython
* Python engine: this leverages Python's `csv` library

As a result, we need to test both, and we have test classes that implement their own `read_csv` methods to specify which engine to use (FYI `read_csv` defaults to the C engine).
@TomAugspurger not entirely sure I'm understanding the failure this time. I ran the command "git diff master --name-only -- '*.py' | grep 'pandas/' | xargs flake8" on the branch and didn't get any issues. Any help would be appreciated. 
@mjlove12 [this test failure](https://travis-ci.org/pandas-dev/pandas/jobs/234430487) looks unrelated. I've restarted that worker. The other failure is a known issue we haven't been able to debug yet.
You can also add a release note in `doc/source/whatsnew/v0.21.0.txt` if you want, under the bug fixes section.
@TomAugspurger sounds good. I added a whatsnew entry under bug fixes. 
The appveyor failure is unrelated, and already fixed on master.

Thanks @mjlove12!