Repository: modin-project/modin

Issue:
read_csv usecols cannot find columns
<!--
General questions should be asked on the mailing list modin-dev@googlegroups.com.

Before submitting an issue, please fill out the following form.
-->

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Linux Ubuntu 16.04
- **Modin installed from (source or binary)**: Using `pip install git+https://github.com/modin-project/modin`
- **Modin version**: 0.5.2
- **Python version**: 3.6.7
- **Exact command to reproduce**:

```
import modin.pandas as pdm
import pandas as pd


d = pdm.DataFrame([{'a': 'test'}])
d.to_csv('test.csv', index=False)

# pandas output
u = pd.read_csv('test.csv', usecols=['a'])
print(u)

# modin output
u = pdm.read_csv('test.csv', usecols=['a'])
print(u)
```
Expected output:
No errors should be raised

Actual output:
`ValueError: Usecols do not match columns, columns expected but not found: ['a']`

The usecol argument would be nice to have because it makes reading large CSVs even faster.




Discussion:
Hi @mar-muel, thanks for the report!

This is a regression, we will get it resolved for the next release. It should be supported, thank you for the report. 

It wasn't caught in the tests because we test for the integer index of the column, not the column name. We will add it so this regression doesn't occur in the future. Thanks again for the report!

Resolving patch:
diff --git a/modin/engines/ray/generic/io.py b/modin/engines/ray/generic/io.py
--- a/modin/engines/ray/generic/io.py
+++ b/modin/engines/ray/generic/io.py
@@ -4,6 +4,7 @@
 
 import pandas
 from pandas.io.common import _infer_compression
+from pandas.io.parsers import _validate_usecols_arg
 
 import inspect
 import os
@@ -322,6 +323,15 @@ def _read_csv_from_file_pandas_on_ray(cls, filepath, kwargs={}):
         column_names = empty_pd_df.columns
         skipfooter = kwargs.get("skipfooter", None)
         skiprows = kwargs.pop("skiprows", None)
+
+        usecols = kwargs.get("usecols", None)
+        usecols_md = _validate_usecols_arg(kwargs.get("usecols", None))
+        if usecols is not None and usecols_md[1] != "integer":
+            del kwargs["usecols"]
+            all_cols = pandas.read_csv(
+                file_open(filepath, "rb"), **dict(kwargs, nrows=0, skipfooter=0)
+            ).columns
+            usecols = all_cols.get_indexer_for(list(usecols_md[0]))
         parse_dates = kwargs.pop("parse_dates", False)
         partition_kwargs = dict(
             kwargs,
@@ -332,6 +342,7 @@ def _read_csv_from_file_pandas_on_ray(cls, filepath, kwargs={}):
             skipfooter=0,
             skiprows=None,
             parse_dates=parse_dates,
+            usecols=usecols,
         )
         with file_open(filepath, "rb") as f:
             # Get the BOM if necessary


Test patch:
diff --git a/modin/pandas/test/data/test_usecols.csv b/modin/pandas/test/data/test_usecols.csv
new file mode 100644
--- /dev/null
+++ b/modin/pandas/test/data/test_usecols.csv
@@ -0,0 +1,7 @@
+a,b,c,d,e
+1,2,3,4,5
+2,3,4,5,6
+3,4,5,6,7
+4,5,6,7,8
+5,6,7,8,9
+6,7,8,9,0
diff --git a/modin/pandas/test/test_io.py b/modin/pandas/test/test_io.py
--- a/modin/pandas/test/test_io.py
+++ b/modin/pandas/test/test_io.py
@@ -745,6 +745,14 @@ def test_from_table(make_csv_file):
         assert modin_df_equals_pandas(modin_df, pandas_df)
 
 
+@pytest.mark.parametrize("usecols", [["a"], ["a", "b", "e"], [0, 1, 4]])
+def test_from_csv_with_usecols(usecols):
+    fname = "modin/pandas/test/data/test_usecols.csv"
+    pandas_df = pandas.read_csv(fname, usecols=usecols)
+    modin_df = pd.read_csv(fname, usecols=usecols)
+    df_equals(modin_df, pandas_df)
+
+
 @pytest.mark.skipif(
     __execution_engine__.lower() == "python", reason="Using pandas implementation"
 )