Repository: modin-project/modin

Issue:
param usecols in read_csv is not valid
<!--
General questions should be asked on the mailing list modin-dev@googlegroups.com.

Before submitting an issue, please fill out the following form.
-->

### System information

- **OS Platform and Distribution**: Linux Ubuntu 16.04
- **Modin installed from (source or binary)**:pip install modin
- **Modin version**:0.5.0
- **Python version**:3.7


<!--
You can obtain the Modin version with

python -c "import modin; print(modin.__version__)"
-->

### Describe the problem
<!-- Describe the problem clearly here. -->
My raw data:
```
$ cat batch_instance1.csv
ins_74901673,task_LTg0MTUwNTA5Mjg4MDkwNjIzMA==,j_217,10,Terminated,673795,673797,m_2637,1,1,13,16,0.02,0.02
ins_815802872,M1,j_1527,1,Terminated,158478,158520,m_3430,1,1,3,19,0.13,0.18
ins_564677701,M1,j_2014,1,Terminated,372602,372616,m_1910,1,1,87,116,0.04,0.05
ins_257566161,M1,j_2014,1,Terminated,372602,372615,m_2485,1,1,91,123,0.05,0.05
ins_688679908,M1,j_2014,1,Terminated,372602,372615,m_993,1,1,93,141,0.05,0.05
ins_929638393,M1,j_2014,1,Terminated,372603,372615,m_2808,1,1,100,137,0.05,0.05
ins_1349024140,M1,j_2014,1,Terminated,372603,372617,m_3736,1,1,82,111,0.05,0.05
ins_330247444,M1,j_2014,1,Terminated,372603,372617,m_1176,1,1,84,110,0.05,0.05
ins_833551291,M1,j_2014,1,Terminated,372602,372614,m_2682,1,1,90,159,0.05,0.05
ins_833550789,M1,j_2014,1,Terminated,372603,372619,m_3625,1,1,78,105,0.05,0.05
```


### Source code / logs
<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->

`import modin.pandas as pd`

First,  I read csv file with columns **0 and 1**
```
df = pd.read_csv("/state/partition1/alibaba_clusterdata_v2018/batch_instance1.csv", header=None, usecols=[0,1])
df.head()

```

```
0 | 1
-- | --
ins_74901673 | task_LTg0MTUwNTA5Mjg4MDkwNjIzMA==
ins_815802872 | M1
ins_564677701 | M1
ins_257566161 | M1
ins_688679908 | M1
```


While, I read csv file with columns **0 and 7**
```
df = pd.read_csv("batch_instance1.csv", header=None, usecols=[0,7])
df.head()

```
**The output is the same!**
```
0 | 7
-- | --
ins_74901673 | task_LTg0MTUwNTA5Mjg4MDkwNjIzMA==
ins_815802872 | M1
ins_564677701 | M1
ins_257566161 | M1
ins_688679908 | M1
```






Discussion:
Addition, I test with pandas API.

```
import pandas as pd
df = pd.read_csv("/state/partition1/alibaba_clusterdata_v2018/batch_instance1.csv", header=None, usecols=[0,7])

df.head()
```
And off course, the pandas API output is right.

```
0 | 7
-- | --
ins_74901673 | m_2637
ins_815802872 | m_3430
ins_564677701 | m_1910
ins_257566161 | m_2485
ins_688679908 | m_993
```


Hi @axiqia, thanks for posting this!

I am able to reproduce the issue, it is the combination of `header=None` and `usecols` that results in this behavior. If you use the parameter `usecols` without `header=None` the behavior is correct. It seems that `header=None` somehow is overriding the `usecols` indicies, I will have to take a look into that.

This is a pretty subtle bug, thanks for reporting!

Resolving patch:
diff --git a/modin/engines/ray/generic/io.py b/modin/engines/ray/generic/io.py
--- a/modin/engines/ray/generic/io.py
+++ b/modin/engines/ray/generic/io.py
@@ -273,7 +273,9 @@ def _read_csv_from_file_pandas_on_ray(cls, filepath, kwargs={}):
         partition_kwargs = dict(
             kwargs,
             header=None,
-            names=names,
+            names=names
+            if kwargs.get("usecols") is None or kwargs.get("names") is not None
+            else None,
             skipfooter=0,
             skiprows=None,
             parse_dates=parse_dates,


Test patch:
diff --git a/modin/pandas/test/data/issue_621.csv b/modin/pandas/test/data/issue_621.csv
new file mode 100644
--- /dev/null
+++ b/modin/pandas/test/data/issue_621.csv
@@ -0,0 +1,10 @@
+ins_74901673,task_LTg0MTUwNTA5Mjg4MDkwNjIzMA==,j_217,10,Terminated,673795,673797,m_2637,1,1,13,16,0.02,0.02
+ins_815802872,M1,j_1527,1,Terminated,158478,158520,m_3430,1,1,3,19,0.13,0.18
+ins_564677701,M1,j_2014,1,Terminated,372602,372616,m_1910,1,1,87,116,0.04,0.05
+ins_257566161,M1,j_2014,1,Terminated,372602,372615,m_2485,1,1,91,123,0.05,0.05
+ins_688679908,M1,j_2014,1,Terminated,372602,372615,m_993,1,1,93,141,0.05,0.05
+ins_929638393,M1,j_2014,1,Terminated,372603,372615,m_2808,1,1,100,137,0.05,0.05
+ins_1349024140,M1,j_2014,1,Terminated,372603,372617,m_3736,1,1,82,111,0.05,0.05
+ins_330247444,M1,j_2014,1,Terminated,372603,372617,m_1176,1,1,84,110,0.05,0.05
+ins_833551291,M1,j_2014,1,Terminated,372602,372614,m_2682,1,1,90,159,0.05,0.05
+ins_833550789,M1,j_2014,1,Terminated,372603,372619,m_3625,1,1,78,105,0.05,0.05
diff --git a/modin/pandas/test/test_io.py b/modin/pandas/test/test_io.py
--- a/modin/pandas/test/test_io.py
+++ b/modin/pandas/test/test_io.py
@@ -640,6 +640,28 @@ def test_parse_dates_read_csv():
     modin_df_equals_pandas(modin_df, pandas_df)
 
 
+def test_from_csv_with_args():
+    pandas_df = pandas.read_csv(
+        "modin/pandas/test/data/issue_621.csv", header=None, usecols=[0, 7]
+    )
+    modin_df = pd.read_csv(
+        "modin/pandas/test/data/issue_621.csv", header=None, usecols=[0, 7]
+    )
+    modin_df_equals_pandas(modin_df, pandas_df)
+
+    pandas_df = pandas.read_csv("modin/pandas/test/data/issue_621.csv", usecols=[0, 7])
+    modin_df = pd.read_csv("modin/pandas/test/data/issue_621.csv", usecols=[0, 7])
+    modin_df_equals_pandas(modin_df, pandas_df)
+
+    pandas_df = pandas.read_csv(
+        "modin/pandas/test/data/issue_621.csv", usecols=[0, 7], names=[0, 7]
+    )
+    modin_df = pd.read_csv(
+        "modin/pandas/test/data/issue_621.csv", usecols=[0, 7], names=[0, 7]
+    )
+    modin_df_equals_pandas(modin_df, pandas_df)
+
+
 def test_from_table(make_csv_file):
     make_csv_file(delimiter="\t")