#26545 Fix: same .tsv file, get different data-frame structure using engine 'python' and 'c' 
State: closed

- [x] closes #26545
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

BugFix:
When using `engine='python'`, columns were handled incorrectly if the first header had in the bom.

Bug:
```
In [1]: import pandas as pd                                                             

In [2]: pd.read_csv('test.txt', engine='python', delimiter='\t')              
Out[2]: 
Empty DataFrame
Columns: [Project ID]
Index: []

In [3]: pd.read_csv('test.txt', engine='python', delimiter='\t').shape        
Out[3]: (0, 1)

In [4]: pd.read_csv('test.txt', delimiter='\t')                               
Out[4]: 
Empty DataFrame
Columns: [Project ID, Project Name, Product Name]
Index: []

In [5]: pd.read_csv('test, delimiter='\t').shape                         
Out[5]: (0, 3)
```
Fix:

```
In [1]: import pandas as pd                                                             

In [2]: pd.read_csv('test.txt', engine='python', delimiter='\t')              
Out[2]: 
Empty DataFrame
Columns: [Project ID, Project Name, Product Name]
Index: []

In [3]: pd.read_csv('test.txt', engine='python', delimiter='\t').shape        
Out[3]: (0, 3)

In [4]: pd.read_csv('test.txt', delimiter='\t')                               
Out[4]: 
Empty DataFrame
Columns: [Project ID, Project Name, Product Name]
Index: []

In [5]: pd.read_csv('test, delimiter='\t').shape                         
Out[5]: (0, 3)
```

Labels:
name: IO CSV
description: read_csv, to_csv

Comments:
Hello @LuckyDenis! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-06-11 07:36:17 UTC
This closes an issue right? If so can you update the OP to reflect that
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=h1) Report
> Merging [#26634](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/8d124ea4c5200f218db7cea8e3ff504b0045a4e6?src=pr&el=desc) will **decrease** coverage by `50.09%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26634/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26634      +/-   ##
==========================================
- Coverage   91.88%   41.78%   -50.1%     
==========================================
  Files         174      174              
  Lines       50692    50692              
==========================================
- Hits        46576    21182   -25394     
- Misses       4116    29510   +25394
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `?` | |
| #single | `41.78% <ø> (-0.11%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/latex.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvbGF0ZXgucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/sas/sas\_constants.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfY29uc3RhbnRzLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/core/groupby/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9jYXRlZ29yaWNhbC5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/plotting.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvcGxvdHRpbmcucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/formats/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvaHRtbC5weQ==) | `0% <0%> (-99.37%)` | :arrow_down: |
| [pandas/io/sas/sas7bdat.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXM3YmRhdC5weQ==) | `0% <0%> (-91.16%)` | :arrow_down: |
| [pandas/io/sas/sas\_xport.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3Nhcy9zYXNfeHBvcnQucHk=) | `0% <0%> (-90.1%)` | :arrow_down: |
| [pandas/core/sparse/scipy\_sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL3NjaXB5X3NwYXJzZS5weQ==) | `10.14% <0%> (-89.86%)` | :arrow_down: |
| [pandas/core/tools/numeric.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvdG9vbHMvbnVtZXJpYy5weQ==) | `10.14% <0%> (-89.86%)` | :arrow_down: |
| ... and [128 more](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=footer). Last update [8d124ea...0816572](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=h1) Report
> Merging [#26634](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/157a4e35d25cafdca2fbc48ccda2c14eb7f25bcb?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26634/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26634      +/-   ##
==========================================
- Coverage   91.72%   91.71%   -0.01%     
==========================================
  Files         178      178              
  Lines       50779    50779              
==========================================
- Hits        46578    46574       -4     
- Misses       4201     4205       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.31% <ø> (ø)` | :arrow_up: |
| #single | `41.19% <ø> (-0.09%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26634/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.88% <0%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=footer). Last update [157a4e3...fb010d5](https://codecov.io/gh/pandas-dev/pandas/pull/26634?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@WillAyd, I made a correction on your comments. Look please)
@LuckyDenis could you also check what `test_utf8_bom` is doing in the test module? It looks to cover the same intention as test added so want to make sure we understand the difference and clarify accordingly
In this context, I'm not sure that they are different, but my PR cures this error, which `tm` method should be used to check it? The error was in the method that was cleaning from the bom.

```
In [1]: import pandas as pd                                                             

In [2]: pd.read_csv('test.txt', engine='python', delimiter='\t')              
Out[2]: 
Empty DataFrame
Columns: [Project ID]
Index: []

In [3]: pd.read_csv('test.txt', engine='python', delimiter='\t').shape        
Out[3]: (0, 1)

In [4]: pd.read_csv('test.txt', delimiter='\t')                               
Out[4]: 
Empty DataFrame
Columns: [Project ID, Project Name, Product Name]
Index: []

In [5]: pd.read_csv('test, delimiter='\t').shape                         
Out[5]: (0, 3)

```
@gfyoung ok with this?
@jreback : Looks fine to me, just need the `whatsnew` as @WillAyd said
Yeah, I'll add a description in the `whatsnew`, just haven't had time.
@jreback, @WillAyd, @gfyoung  I added, look please.
lgtm. @LuckyDenis if you can merge master and ping on green to resolve the conflict.
thanks @LuckyDenis