Repository: dask/dask

Issue:
Length mismatch error from dd.read_csv on data from dd.to_csv via verify_meta=False
**What happened**:

I used `dd.from_delayed` to compile my data and wrote it with `to_csv` and then got an error from `from_csv`.

I suppose I aimed the gun at my own foot with `verify_meta=False`, but still, this was pretty surprising, especially given that `df.compute()` produces something that prints just fine.

```
$ python bug_lm.py 
   col0  col1  col2  col3  col4  col5
0     1     1     1     1     1   NaN
1     2     2     2     2     2   NaN
2     3     3     3     3     3   NaN
0     1     1     1     1     1   1.0
1     2     2     2     2     2   2.0
2     3     3     3     3     3   3.0
Traceback (most recent call last):
  File "bug_lm.py", line 18, in <module>
    print(dd.read_csv('out-*.csv').compute())
...
ValueError: Length mismatch: Expected axis has 7 elements, new values have 6 elements
```

**What you expected to happen**:

`print(dd.read_csv('out-*.csv').compute())` would print the same way as `print(df.compute())`

**Minimal Complete Verifiable Example**:

```python
import pandas as pd
import dask
import dask.dataframe as dd


def build(n):
    return pd.DataFrame({f'col{i}': [1, 2, 3]
                         for i in range(n)})

df = dd.from_delayed(
    [dask.delayed(build)(n)
     for n in [5, 6]], verify_meta=False)

print(df.compute())

df.to_csv('out-*.csv')

print(dd.read_csv('out-*.csv').compute())
```

**Anything else we need to know?**:

**Environment**:

- Dask version: 2021.06.2
- Python version: 3.8
- Operating System: linux (debian Stretch, supplemented with nix)
- Install method (conda, pip, source): from nix `2z911s62b571d9yvzfhzbxbccgxclbah-python3.8-dask-2021.06.2`



Discussion:
I guess I might as well share my work-around:

```py
def patch_cols(dpath,
               oops=',time,type,slogfile,line\n',
               missing=['blockHeight', 'blockTime']):
    for p in dpath.glob('*.csv'):
        if p.name.endswith('-bak.csv'):
            continue
        hd = p.open().readline()
        if hd == oops:
            bak = p.parent / f'{p.stem}-bak.csv'
            df = pd.read_csv(p, index_col=0)
            p.rename(bak)
            for col in missing:
                df[col] = np.nan
            df.to_csv(p)
```

If you look at the contents of `out-0.csv` and `out-1.csv` you'll see that they have different columns. There is an assumption in dask that all files passed to `read_csv` will have exactly the same columns and order. If you think there is a spot in the docs where that could be made more clear, please feel free to open a pull request. 
Yes, I know they have different columns. The question is: why is `print(df.compute())` happy with that but `read_csv` is not?
Ah sorry I should have made that clear .`df.compute` is happy because it already has the column information for each partition and you have explicitly told dask.delayed that you want it to ignore `meta` so it just does a merge of the partitions. `read_csv` only tries to read the column data from the first file so since the second file has different columns, it just fails.
I guess a `verify_meta=False` option for `read_csv` would be handy.
You are right. I think that was the intention of the `enforce` kwarg. But it does not seem to be acting how I'd expect. I am taking a look now. 

Resolving patch:
diff --git a/dask/dataframe/io/csv.py b/dask/dataframe/io/csv.py
--- a/dask/dataframe/io/csv.py
+++ b/dask/dataframe/io/csv.py
@@ -179,8 +179,6 @@ def pandas_read_text(
 
     if enforce and columns and (list(df.columns) != list(columns)):
         raise ValueError("Columns do not match", df.columns, columns)
-    elif columns:
-        df.columns = columns
     if path:
         colname, path, paths = path
         code = paths.index(path)


Test patch:
diff --git a/dask/dataframe/io/tests/test_csv.py b/dask/dataframe/io/tests/test_csv.py
--- a/dask/dataframe/io/tests/test_csv.py
+++ b/dask/dataframe/io/tests/test_csv.py
@@ -1191,12 +1191,26 @@ def test_robust_column_mismatch():
     k = sorted(files)[-1]
     files[k] = files[k].replace(b"name", b"Name")
     with filetexts(files, mode="b"):
-        ddf = dd.read_csv("2014-01-*.csv")
+        ddf = dd.read_csv(
+            "2014-01-*.csv", header=None, skiprows=1, names=["name", "amount", "id"]
+        )
         df = pd.read_csv("2014-01-01.csv")
         assert (df.columns == ddf.columns).all()
         assert_eq(ddf, ddf)
 
 
+def test_different_columns_are_allowed():
+    files = csv_files.copy()
+    k = sorted(files)[-1]
+    files[k] = files[k].replace(b"name", b"address")
+    with filetexts(files, mode="b"):
+        ddf = dd.read_csv("2014-01-*.csv")
+
+        # since enforce is False, meta doesn't have to match computed
+        assert (ddf.columns == ["name", "amount", "id"]).all()
+        assert (ddf.compute().columns == ["name", "amount", "id", "address"]).all()
+
+
 def test_error_if_sample_is_too_small():
     text = "AAAAA,BBBBB,CCCCC,DDDDD,EEEEE\n1,2,3,4,5\n6,7,8,9,10\n11,12,13,14,15"
     with filetext(text) as fn: