Repository: dask/dask

Issue:
Dataframe column selection can mix up columns
Currently column selection when reading from CSV (and possibly other IO sources) can mix up column labels, with the wrong name attached to columns. I suspect that this is related to the column projection work that @rjzamora has been doing, but I haven't tracked down the source yet. It also doesn't seem to happen for every CSV.

MWE:

```python
import dask.dataframe as dd
import pandas as pd

URL = "https://github.com/datacamp/python-live-training-template/raw/master/data/airbnb.csv"
dd.read_csv(URL)[["name", "neighbourhood_full"]].head(2)
```
produces 
|    | name                       | neighbourhood_full                                 |
|---:|:---------------------------|:---------------------------------------------------|
|  0 | Brooklyn, Flatlands        | Cozy,budget friendly, cable inc, private entrance! |
|  1 | Manhattan, Upper West Side | Two floor apartment near Central Park              |



```python
import dask.dataframe as dd
import pandas as pd

URL = "https://github.com/datacamp/python-live-training-template/raw/master/data/airbnb.csv"
pd.read_csv(URL)[["name", "neighbourhood_full"]].head(2)
```
produces
|    | name                                               | neighbourhood_full         |
|---:|:---------------------------------------------------|:---------------------------|
|  0 | Cozy,budget friendly, cable inc, private entrance! | Brooklyn, Flatlands        |
|  1 | Two floor apartment near Central Park              | Manhattan, Upper West Side |

- Dask version: current `main`

Discussion:
ian-r-rose: Yeah, that’s fair.

On Thu, Jun 3, 2021 at 8:34 PM Richard (Rick) Zamora <
***@***.***> wrote:

> ***@***.**** commented on this pull request.
> ------------------------------
>
> In dask/dataframe/io/csv.py
> <https://github.com/dask/dask/pull/7761#discussion_r645266235>:
>
> > @@ -68,6 +68,8 @@ def project_columns(self, columns):
>          """Return a new CSVFunctionWrapper object with
>          a sub-column projection.
>          """
> +        # Make sure columns is ordered correctly
> +        columns = [c for c in self.head.columns if c in columns]
>
> As far as I can tell, pandas doesn't actually support
> mangle_dupe_cols=False
> <https://github.com/pandas-dev/pandas/blob/2cb96529396d93b46abab7bbc73a208e708c642e/pandas/io/parsers.py#L833>.
> Therefore, I don't think there are any cases where a direct read_csv call
> will produce duplicate columns.
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> <https://github.com/dask/dask/pull/7761#discussion_r645266235>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABLWQN3JLKL7WRDXENBMZQLTRBCUDANCNFSM46BY6YKQ>
> .
>


Patch:
diff --git a/dask/dataframe/io/csv.py b/dask/dataframe/io/csv.py
index 63cf025aa..076fe2254 100644
--- a/dask/dataframe/io/csv.py
+++ b/dask/dataframe/io/csv.py
@@ -68,6 +68,8 @@ class CSVFunctionWrapper:
         """Return a new CSVFunctionWrapper object with
         a sub-column projection.
         """
+        # Make sure columns is ordered correctly
+        columns = [c for c in self.head.columns if c in columns]
         if columns == self.columns:
             return self
         func = copy.deepcopy(self)


Test patch:
diff --git a/dask/dataframe/io/tests/test_csv.py b/dask/dataframe/io/tests/test_csv.py
index d627cb521..a6c50d12c 100644
--- a/dask/dataframe/io/tests/test_csv.py
+++ b/dask/dataframe/io/tests/test_csv.py
@@ -1652,3 +1652,19 @@ def test_read_csv_groupby_get_group(tmpdir):
     ddfs = ddf1.groupby("foo")
 
     assert_eq(df1, ddfs.get_group(10).compute())
+
+
+def test_csv_getitem_column_order(tmpdir):
+    # See: https://github.com/dask/dask/issues/7759
+
+    path = os.path.join(str(tmpdir), "test.csv")
+    columns = list("abcdefghijklmnopqrstuvwxyz")
+    values = list(range(len(columns)))
+
+    df1 = pd.DataFrame([{c: v for c, v in zip(columns, values)}])
+    df1.to_csv(path)
+
+    # Use disordered and duplicated column selection
+    columns = list("hczzkylaape")
+    df2 = dd.read_csv(path)[columns].head(1)
+    assert_eq(df1[columns], df2)