Repository: dask/dask

Issue:
Reading in DataFrame with header=1 causes first column to be inaccessible
<!-- Please include a self-contained copy-pastable example that generates the issue if possible.

Please be concise with code posted. See guidelines below on how to provide a good bug report:

- Craft Minimal Bug Reports http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports
- Minimal Complete Verifiable Examples https://stackoverflow.com/help/mcve

Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly.
-->

**What happened**:
I read in a file as `df = dd.read_csv('filename.csv', header=1)`, then tried to set the index as the first column. Even being entirely explicit like `df.set_index(df.columns[0])` fails to a KeyError.

**What you expected to happen**:
I should be able to set the index as any of the columns, but this incorrectly fails. 
**Minimal Complete Verifiable Example**:

```python
In [1]: import dask.dataframe as dd                                                                                                                                                         
                                                                                                                                                                                            
In [2]: df = dd.read_csv('primary_0.csv', header=1)                                                                                                                                         
                                                                                                                                                                                            
In [3]: df.columns[0]                                                                                                                                                                       
Out[3]: 'gene'                                                                                                                                                                              
                                                                                                                                                                                            
In [4]: df.set_index(df.columns[0])                                                                                                                                                         
...
KeyError: "None of [Index(['gene'], dtype='object')] are in the [columns]"

```

**Environment**:
Conda
- Dask version: 2021.10.0
- Python version: 3.9.7
- Operating System: MacOS Monterey
- Install method (conda, pip, source): pip



Discussion:
Aha! It took me a bit to reproduce this one, but I've got a minimal example now:

```python
import dask

ddf = dask.datasets.timeseries(freq="1H")
ddf.compute().to_csv("data.csv", blocksize=10000)
ddf = dd.read_csv("data.csv", header=1)
```

I think what's going on is dask doesn't do the right thing with the `header` kwarg when a single file is being split into several partitions (@rjzamora in case you have insight). As a workaround, you can use `skiprows=1` instead of `header=1`



Resolving patch:
diff --git a/dask/dataframe/io/csv.py b/dask/dataframe/io/csv.py
--- a/dask/dataframe/io/csv.py
+++ b/dask/dataframe/io/csv.py
@@ -105,6 +105,8 @@ def __call__(self, part):
         if not is_first:
             write_header = True
             rest_kwargs.pop("skiprows", None)
+            if rest_kwargs.get("header", 0) is not None:
+                rest_kwargs.pop("header", None)
         if not is_last:
             rest_kwargs.pop("skipfooter", None)
 
@@ -581,6 +583,8 @@ def read_pandas(
             "in `sample` in the call to `read_csv`/`read_table`"
         )
 
+    if isinstance(header, int):
+        firstrow += header
     header = b"" if header is None else parts[firstrow] + b_lineterminator
 
     # Use sample to infer dtypes and check for presence of include_path_column


Test patch:
diff --git a/dask/dataframe/io/tests/test_csv.py b/dask/dataframe/io/tests/test_csv.py
--- a/dask/dataframe/io/tests/test_csv.py
+++ b/dask/dataframe/io/tests/test_csv.py
@@ -775,6 +775,22 @@ def test_windows_line_terminator():
         assert df.a.sum().compute() == 1 + 2 + 3 + 4 + 5 + 6
 
 
+def test_header_int():
+    text = (
+        "id0,name0,x0,y0\n"
+        "id,name,x,y\n"
+        "1034,Victor,-0.25,0.84\n"
+        "998,Xavier,-0.48,-0.13\n"
+        "999,Zelda,0.00,0.47\n"
+        "980,Alice,0.67,-0.98\n"
+        "989,Zelda,-0.04,0.03\n"
+    )
+    with filetexts({"test_header_int.csv": text}):
+        df = dd.read_csv("test_header_int.csv", header=1, blocksize=64)
+        expected = pd.read_csv("test_header_int.csv", header=1)
+        assert_eq(df, expected, check_index=False)
+
+
 def test_header_None():
     with filetexts({".tmp.1.csv": "1,2", ".tmp.2.csv": "", ".tmp.3.csv": "3,4"}):
         df = dd.read_csv(".tmp.*.csv", header=None)