Repository: pandas-dev/pandas

Issue:
ERR: usecols fails to raise error if column doesn't exist but is the same length as headers
#### A small, complete example of the issue
read_csv doesn't raise a value error if the use cols contains the same number of values as the headers

simple [test_csv.txt](https://github.com/pandas-dev/pandas/files/595756/test_csv.txt) file contains:
A,B
1,2
3,4

It currently will set the other column as NaN values:
        A	B
0	1	NaN
1	3	NaN

```python
import pandas as pd
file = pd.read_csv('data/test.csv',usecols=['A','C']) # Column C doesn't exist
file
```

#### Expected Output
ValueError("Usecols do not match names.") exception

#### Output of ``pd.show_versions()``

<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Darwin
OS-release: 16.1.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: 1.3.7
pip: 8.1.2
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.11.1
scipy: 0.18.1
statsmodels: 0.6.1
xarray: None
IPython: 5.1.0
sphinx: 1.4.6
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: 1.1.0
tables: 3.2.3.1
numexpr: 2.6.1
matplotlib: 1.5.3
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.3
lxml: 3.6.4
bs4: 4.5.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.42.0
pandas_datareader: 0.2.1
</details>


Discussion:
yeah this seems reasonable.

@GGordonGordon will you be creating a PR for this?

@aileronajay yeah fairly soon

Pull request: https://github.com/pandas-dev/pandas/pull/14674

For clarification should if usecols is a list of integers should it refer to the columns within the original data table (appears this is what the [current documentation](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html?highlight=read_csv) states) or should they refer as an index to the names list (appears to be how the code currently operates)?

@GGordonGordon the documentation is correctly. you would have to show an example where this is not true.

This appears to be resolved by #14234 and this is no longer needed.
Using master (so with #14234 merged), I still see the buggy behaviour:

```
In [12]: s = """A,B
    ...: 1,2
    ...: 3,4"""

In [13]: pd.read_csv(StringIO(s))
Out[13]: 
   A  B
0  1  2
1  3  4

In [15]: pd.read_csv(StringIO(s), usecols=["A", "C"])
Out[15]: 
   A    B
0  1  NaN
1  3  NaN
```

So it would be welcome if you could update the PR.
Is anyone working on this now. Just picked it up at #pandas sprint at #PyCon2017 ?
@bpraggastis Note there is a closed PR https://github.com/pandas-dev/pandas/pull/14674 that started this work, but never was finished. You can have a look at that, and see whether it would be useful to start from that work (use the changes in that PR)

Resolving patch:
diff --git a/pandas/io/parsers.py b/pandas/io/parsers.py
--- a/pandas/io/parsers.py
+++ b/pandas/io/parsers.py
@@ -1649,6 +1649,12 @@ def __init__(self, src, **kwds):
 
         if self.usecols:
             usecols = _evaluate_usecols(self.usecols, self.orig_names)
+
+            # GH 14671
+            if (self.usecols_dtype == 'string' and
+                    not set(usecols).issubset(self.orig_names)):
+                raise ValueError("Usecols do not match names.")
+
             if len(self.names) > len(usecols):
                 self.names = [n for i, n in enumerate(self.names)
                               if (i in usecols or n in usecols)]


Test patch:
diff --git a/pandas/tests/io/parser/usecols.py b/pandas/tests/io/parser/usecols.py
--- a/pandas/tests/io/parser/usecols.py
+++ b/pandas/tests/io/parser/usecols.py
@@ -475,3 +475,54 @@ def test_uneven_length_cols(self):
                               'C': [3, 5, 4, 3, 3, 7]})
         df = self.read_csv(StringIO(data), usecols=usecols)
         tm.assert_frame_equal(df, expected)
+
+    def test_raise_on_usecols_names_mismatch(self):
+        # GH 14671
+        data = 'a,b,c,d\n1,2,3,4\n5,6,7,8'
+
+        if self.engine == 'c':
+            msg = 'Usecols do not match names'
+        else:
+            msg = 'is not in list'
+
+        usecols = ['a', 'b', 'c', 'd']
+        df = self.read_csv(StringIO(data), usecols=usecols)
+        expected = DataFrame({'a': [1, 5], 'b': [2, 6], 'c': [3, 7],
+                              'd': [4, 8]})
+        tm.assert_frame_equal(df, expected)
+
+        usecols = ['a', 'b', 'c', 'f']
+        with tm.assert_raises_regex(ValueError, msg):
+            self.read_csv(StringIO(data), usecols=usecols)
+
+        usecols = ['a', 'b', 'f']
+        with tm.assert_raises_regex(ValueError, msg):
+            self.read_csv(StringIO(data), usecols=usecols)
+
+        names = ['A', 'B', 'C', 'D']
+
+        df = self.read_csv(StringIO(data), header=0, names=names)
+        expected = DataFrame({'A': [1, 5], 'B': [2, 6], 'C': [3, 7],
+                              'D': [4, 8]})
+        tm.assert_frame_equal(df, expected)
+
+        # TODO: https://github.com/pandas-dev/pandas/issues/16469
+        # usecols = ['A','C']
+        # df = self.read_csv(StringIO(data), header=0, names=names,
+        #                    usecols=usecols)
+        # expected = DataFrame({'A': [1,5], 'C': [3,7]})
+        # tm.assert_frame_equal(df, expected)
+        #
+        # usecols = [0,2]
+        # df = self.read_csv(StringIO(data), header=0, names=names,
+        #                    usecols=usecols)
+        # expected = DataFrame({'A': [1,5], 'C': [3,7]})
+        # tm.assert_frame_equal(df, expected)
+
+        usecols = ['A', 'B', 'C', 'f']
+        with tm.assert_raises_regex(ValueError, msg):
+            self.read_csv(StringIO(data), header=0, names=names,
+                          usecols=usecols)
+        usecols = ['A', 'B', 'f']
+        with tm.assert_raises_regex(ValueError, msg):
+            self.read_csv(StringIO(data), names=names, usecols=usecols)