Repository: pandas-dev/pandas

Issue:
ENH: inconsistent naming convention for read_csv and read_excel column selection
`read_csv` uses the `usecols` parameter, and the `read_excel` function uses `parse_cols` parameter. Both parameter do basically the same thing:

```
usecols : array-like
    Return a subset of the columns.
    Results in much faster parsing time and lower memory usage.
```

```
parse_cols : int or list, default None
    * If None then parse all columns,
    * If int then indicates last column to be parsed
    * If list of ints then indicates list of column numbers to be parsed
    * If string then indicates comma separated list of column names and
      column ranges (e.g. "A:E" or "A,C,E:F")
```

Also maybe supporting the same thing that `parse_cols` does in `usecol` wouldn't be so bad?



Discussion:
That seems reasonable, however we'd probably still have to support the old parameter. My impression is that the only difference is that parse_cols accept a single int: clearly csv can't support the string option, given that it's excel specific and it already accepts a list of int (and I'm sure it accepts None as well).

@jtratner Maybe adding `use_cols` (since all other parameters use the underscore convention) that does the combination you specified (+ for excel files the excel specific thing) and deprecate the `usecols` and `parse_cols` parameters in 0.13, 0.14... Exclude them from docs and drop them in some future version?

I can take this if you decide this is reasonable to implement... The question is just what to do with the test cases? Test both or test just the `use_cols` since `usecols and`parse_cols` will be just aliases.

you are talking about`usecols`, not `use_cols` (sp?)

@jreback I proposed a change from `usecols` to `use_cols`, since most parameters use the underscore naming convention: `filepath_or_buffer`, `index_col`, `na_values`, `keep_default_na`, `true_values`, `false_values`, `parse_dates`, `keep_date_col`, `date_parser`, `na_filter`, `infer_types`, `mangle_dupe_cols`, `tupleize_cols`... There are still ones that do not, like: `dayfirst`, `skiprows`, `escapechar`, `skipinitialspace`, `lineterminator`, `quotechar`, `skipfooter`, `chunksize`.

But if I noticed correctly the naming conventions are going toward the `_` separated ones... Correct me if I'm wrong.

@alefnula while I get what you're saying, I don't know if it really makes sense to change those keyword arguments now (and `escapechar` and `quotechar` are the `csv` module's kwargs, so thsoe probably shouldn't change). If you're going to do any change, you should definitely change it to `parse_cols`, to be consistent with excel

@jtratner I think you misunderstood me, but it's my fault, I wasn't clear enough :) So I'll start fresh.

I went through the code of the `read_` functions and here are the results:

| Function | Parameter |
| --- | --- |
| read_clipboard | usecols |
| read_csv | usecols |
| read_excel | parse_cols |
| read_fwf | usecols |
| read_hdf | - |
| read_html | - |
| read_json | - |
| read_pickle | - |
| read_sql | - |
| read_stata | usecols |
| read_table | usecols |
| read_sql | - |

The conclusion is: Most of the functions that have a column selection parameter use `usecols`, except the `read_excel` which uses `parse_cols`.

My suggestion is to:
1. _Add column selection to all `read_` functions._ Implementing this for the functions that does not support it, at the moment, can just be something like this:
   
   ``` python
   if usecols:
       return result[cols]
   ```
   
   at the end of the function. I KNOW I oversimplified it :) This is just pseudecode. But doing a refined version of this wouldn't be so bad for the first iteration, and in the second iteration it can be supported directly by the parsers.
2. Since I first looked only at the `read_csv` and `read_excel` I suggested a unification of `usecols` and `parse_cols` under the name `use_cols`, since the underscore convention is used more often in pandas. _But I take that back._ Five from thirteen functions use the `usecols` parameter, and this would just cause trouble and wont gain anything. So just forget that renaming part :) The only renaming in this case would be from `parse_cols` to `usecols` in `read_excel`.
3. `usecols` should, for all functions, support the same thing it currently does + the integer slicing one `[:n]`.
4. It will, for the `read_` functions that support something fancy, have additional superpowers (like in the case of the `read_excel`, and maybe some other formats) but the basic functionality from the point 3. should be there for all functions. Additional "superpowers" would be noted in the docs for the specific formats.

This is basically the proposal I had in mind :) I just wasn't clear enough.

If someone is interested in this, we should deprecate `parse_cols` in favor of `usecols`

taking a look at this, I plan to deprecate `parse_cols` for `read_excel()` in favor of `usecols` for consistency with other read functions.  plan for `usecols` to support current functionality of `read_excel()`'s `parse_cols` and other `usecols`.
Basically, @alefnula's points 2, 3, and 4 above.  point 1 (implement column selection functionality for all `read_` functions) probably should be another ENH.
in the current release version (0.20.1), a list of columns returns an empty DF.  I.E. ``pandas.read_excel(test_inputexcel_file, parse_cols=['A','B','D']) `` results in 
````
Empty DataFrame
Columns: []
Index: []
````
rather than the expected DataFrame with columns A, B, and D.  I plan to address this as well to implement @alefnula's point 3.
@gfyoung anything worth changing for 0.21.0 for this? (prob in ``read_excel``) that isn't already done / in an issue?
@jreback : I agree with you <a href="https://github.com/pandas-dev/pandas/issues/4988#issuecomment-209714290">from a year ago</a> that `parse_cols` should be deprecated in favor of `usecols`.  I don't see why we shouldn't aim to do this in `0.21.0`.

Resolving patch:
diff --git a/pandas/io/excel.py b/pandas/io/excel.py
--- a/pandas/io/excel.py
+++ b/pandas/io/excel.py
@@ -31,7 +31,7 @@
 import pandas.compat.openpyxl_compat as openpyxl_compat
 from warnings import warn
 from distutils.version import LooseVersion
-from pandas.util._decorators import Appender
+from pandas.util._decorators import Appender, deprecate_kwarg
 from textwrap import fill
 
 __all__ = ["read_excel", "ExcelWriter", "ExcelFile"]
@@ -86,7 +86,7 @@
     Column (0-indexed) to use as the row labels of the DataFrame.
     Pass None if there is no such column.  If a list is passed,
     those columns will be combined into a ``MultiIndex``.  If a
-    subset of data is selected with ``parse_cols``, index_col
+    subset of data is selected with ``usecols``, index_col
     is based on the subset.
 names : array-like, default None
     List of column names to use. If file contains no header row,
@@ -115,6 +115,10 @@
     .. versionadded:: 0.19.0
 
 parse_cols : int or list, default None
+    .. deprecated:: 0.21.0
+       Pass in `usecols` instead.
+
+usecols : int or list, default None
     * If None then parse all columns,
     * If int then indicates last column to be parsed
     * If list of ints then indicates list of column numbers to be parsed
@@ -205,8 +209,9 @@ def get_writer(engine_name):
 
 
 @Appender(_read_excel_doc)
+@deprecate_kwarg("parse_cols", "usecols")
 def read_excel(io, sheet_name=0, header=0, skiprows=None, skip_footer=0,
-               index_col=None, names=None, parse_cols=None, parse_dates=False,
+               index_col=None, names=None, usecols=None, parse_dates=False,
                date_parser=None, na_values=None, thousands=None,
                convert_float=True, converters=None, dtype=None,
                true_values=None, false_values=None, engine=None,
@@ -226,7 +231,7 @@ def read_excel(io, sheet_name=0, header=0, skiprows=None, skip_footer=0,
 
     return io._parse_excel(
         sheetname=sheet_name, header=header, skiprows=skiprows, names=names,
-        index_col=index_col, parse_cols=parse_cols, parse_dates=parse_dates,
+        index_col=index_col, usecols=usecols, parse_dates=parse_dates,
         date_parser=date_parser, na_values=na_values, thousands=thousands,
         convert_float=convert_float, skip_footer=skip_footer,
         converters=converters, dtype=dtype, true_values=true_values,
@@ -295,7 +300,7 @@ def __fspath__(self):
         return self._io
 
     def parse(self, sheet_name=0, header=0, skiprows=None, skip_footer=0,
-              names=None, index_col=None, parse_cols=None, parse_dates=False,
+              names=None, index_col=None, usecols=None, parse_dates=False,
               date_parser=None, na_values=None, thousands=None,
               convert_float=True, converters=None, true_values=None,
               false_values=None, squeeze=False, **kwds):
@@ -309,7 +314,7 @@ def parse(self, sheet_name=0, header=0, skiprows=None, skip_footer=0,
         return self._parse_excel(sheetname=sheet_name, header=header,
                                  skiprows=skiprows, names=names,
                                  index_col=index_col,
-                                 parse_cols=parse_cols,
+                                 usecols=usecols,
                                  parse_dates=parse_dates,
                                  date_parser=date_parser, na_values=na_values,
                                  thousands=thousands,
@@ -321,7 +326,7 @@ def parse(self, sheet_name=0, header=0, skiprows=None, skip_footer=0,
                                  squeeze=squeeze,
                                  **kwds)
 
-    def _should_parse(self, i, parse_cols):
+    def _should_parse(self, i, usecols):
 
         def _range2cols(areas):
             """
@@ -347,15 +352,15 @@ def _excel2num(x):
                     cols.append(_excel2num(rng))
             return cols
 
-        if isinstance(parse_cols, int):
-            return i <= parse_cols
-        elif isinstance(parse_cols, compat.string_types):
-            return i in _range2cols(parse_cols)
+        if isinstance(usecols, int):
+            return i <= usecols
+        elif isinstance(usecols, compat.string_types):
+            return i in _range2cols(usecols)
         else:
-            return i in parse_cols
+            return i in usecols
 
     def _parse_excel(self, sheetname=0, header=0, skiprows=None, names=None,
-                     skip_footer=0, index_col=None, parse_cols=None,
+                     skip_footer=0, index_col=None, usecols=None,
                      parse_dates=False, date_parser=None, na_values=None,
                      thousands=None, convert_float=True, true_values=None,
                      false_values=None, verbose=False, dtype=None,
@@ -470,10 +475,10 @@ def _parse_cell(cell_contents, cell_typ):
                 row = []
                 for j, (value, typ) in enumerate(zip(sheet.row_values(i),
                                                      sheet.row_types(i))):
-                    if parse_cols is not None and j not in should_parse:
-                        should_parse[j] = self._should_parse(j, parse_cols)
+                    if usecols is not None and j not in should_parse:
+                        should_parse[j] = self._should_parse(j, usecols)
 
-                    if parse_cols is None or should_parse[j]:
+                    if usecols is None or should_parse[j]:
                         row.append(_parse_cell(value, typ))
                 data.append(row)
 


Test patch:
diff --git a/pandas/tests/io/test_excel.py b/pandas/tests/io/test_excel.py
--- a/pandas/tests/io/test_excel.py
+++ b/pandas/tests/io/test_excel.py
@@ -158,56 +158,74 @@ def setup_method(self, method):
         self.check_skip()
         super(ReadingTestsBase, self).setup_method(method)
 
-    def test_parse_cols_int(self):
+    def test_usecols_int(self):
 
         dfref = self.get_csv_refdf('test1')
         dfref = dfref.reindex(columns=['A', 'B', 'C'])
-        df1 = self.get_exceldf('test1', 'Sheet1', index_col=0, parse_cols=3)
+        df1 = self.get_exceldf('test1', 'Sheet1', index_col=0, usecols=3)
         df2 = self.get_exceldf('test1', 'Sheet2', skiprows=[1], index_col=0,
-                               parse_cols=3)
+                               usecols=3)
+
+        with tm.assert_produces_warning(FutureWarning):
+            df3 = self.get_exceldf('test1', 'Sheet2', skiprows=[1],
+                                   index_col=0, parse_cols=3)
+
         # TODO add index to xls file)
         tm.assert_frame_equal(df1, dfref, check_names=False)
         tm.assert_frame_equal(df2, dfref, check_names=False)
+        tm.assert_frame_equal(df3, dfref, check_names=False)
 
-    def test_parse_cols_list(self):
+    def test_usecols_list(self):
 
         dfref = self.get_csv_refdf('test1')
         dfref = dfref.reindex(columns=['B', 'C'])
         df1 = self.get_exceldf('test1', 'Sheet1', index_col=0,
-                               parse_cols=[0, 2, 3])
+                               usecols=[0, 2, 3])
         df2 = self.get_exceldf('test1', 'Sheet2', skiprows=[1], index_col=0,
-                               parse_cols=[0, 2, 3])
+                               usecols=[0, 2, 3])
+
+        with tm.assert_produces_warning(FutureWarning):
+            df3 = self.get_exceldf('test1', 'Sheet2', skiprows=[1],
+                                   index_col=0, parse_cols=[0, 2, 3])
+
         # TODO add index to xls file)
         tm.assert_frame_equal(df1, dfref, check_names=False)
         tm.assert_frame_equal(df2, dfref, check_names=False)
+        tm.assert_frame_equal(df3, dfref, check_names=False)
 
-    def test_parse_cols_str(self):
+    def test_usecols_str(self):
 
         dfref = self.get_csv_refdf('test1')
 
         df1 = dfref.reindex(columns=['A', 'B', 'C'])
         df2 = self.get_exceldf('test1', 'Sheet1', index_col=0,
-                               parse_cols='A:D')
+                               usecols='A:D')
         df3 = self.get_exceldf('test1', 'Sheet2', skiprows=[1], index_col=0,
-                               parse_cols='A:D')
+                               usecols='A:D')
+
+        with tm.assert_produces_warning(FutureWarning):
+            df4 = self.get_exceldf('test1', 'Sheet2', skiprows=[1],
+                                   index_col=0, parse_cols='A:D')
+
         # TODO add index to xls, read xls ignores index name ?
         tm.assert_frame_equal(df2, df1, check_names=False)
         tm.assert_frame_equal(df3, df1, check_names=False)
+        tm.assert_frame_equal(df4, df1, check_names=False)
 
         df1 = dfref.reindex(columns=['B', 'C'])
         df2 = self.get_exceldf('test1', 'Sheet1', index_col=0,
-                               parse_cols='A,C,D')
+                               usecols='A,C,D')
         df3 = self.get_exceldf('test1', 'Sheet2', skiprows=[1], index_col=0,
-                               parse_cols='A,C,D')
+                               usecols='A,C,D')
         # TODO add index to xls file
         tm.assert_frame_equal(df2, df1, check_names=False)
         tm.assert_frame_equal(df3, df1, check_names=False)
 
         df1 = dfref.reindex(columns=['B', 'C'])
         df2 = self.get_exceldf('test1', 'Sheet1', index_col=0,
-                               parse_cols='A,C:D')
+                               usecols='A,C:D')
         df3 = self.get_exceldf('test1', 'Sheet2', skiprows=[1], index_col=0,
-                               parse_cols='A,C:D')
+                               usecols='A,C:D')
         tm.assert_frame_equal(df2, df1, check_names=False)
         tm.assert_frame_equal(df3, df1, check_names=False)
 
@@ -457,14 +475,14 @@ def test_read_one_empty_col_no_header(self):
             actual_header_none = read_excel(
                 path,
                 'no_header',
-                parse_cols=[0],
+                usecols=[0],
                 header=None
             )
 
             actual_header_zero = read_excel(
                 path,
                 'no_header',
-                parse_cols=[0],
+                usecols=[0],
                 header=0
             )
         expected = DataFrame()
@@ -486,14 +504,14 @@ def test_read_one_empty_col_with_header(self):
             actual_header_none = read_excel(
                 path,
                 'with_header',
-                parse_cols=[0],
+                usecols=[0],
                 header=None
             )
 
             actual_header_zero = read_excel(
                 path,
                 'with_header',
-                parse_cols=[0],
+                usecols=[0],
                 header=0
             )
         expected_header_none = DataFrame(pd.Series([0], dtype='int64'))