Repository: pandas-dev/pandas

Issue:
read_msgpack returns garbage for non-existing files in python2
#### Code Sample, a copy-pastable example if possible

```python
In [1]: import pandas as pd

In [2]: pd.__version__
Out[2]: u'0.19.2'

In [3]: pd.read_msgpack('/tmp/bla.txt')
Out[3]: [47, 116, 109, 112, 47, 98, 108, 97, 46, 116, 120, 116]
```

That file does not exist.

The same code with the same version on python 3 correctly raises an exception.

#### Problem description

Reading a non existing file returns a list of integers instead of failing if pandas is used in python 2. This makes detecting error quite hard.

#### Expected Output

An exception.

#### Output of ``pd.show_versions()``

<details>

```
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.13.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.6-1-ARCH
machine: x86_64
processor: 
byteorder: little
LC_ALL: None
LANG: en_US.utf8
LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 34.1.0
Cython: None
numpy: 1.11.3
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.1.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.1
matplotlib: 1.5.3
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.5
boto: None
pandas_datareader: None
```

</details>



Discussion:
I recall this being fixed a little while back. Are you sure you are running with 0.19.2?

```
In [1]: pd.read_msgpack('/tmp/bla.txt')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-7a3c5705a157> in <module>()
----> 1 pd.read_msgpack('/tmp/bla.txt')

/Users/jreback/miniconda3/envs/pandas/lib/python3.5/site-packages/pandas/io/packers.py in read_msgpack(path_or_buf, encoding, iterator, **kwargs)
    216         return read(path_or_buf)
    217 
--> 218     raise ValueError('path_or_buf needs to be a string file path or file-like')
    219 
    220 dtype_dict = {21: np.dtype('M8[ns]'),

ValueError: path_or_buf needs to be a string file path or file-like

In [2]: pd.__version__
Out[2]: '0.19.2'
```
Yes, definitely 0.19.2. It seems to be fixed only in python 3.
ahh, that is possible (must not be well tested)

want to add a test and fix? 

I *think* we are doing a better job of this now-a-days in things like ``read_json`` IIRC. IOW if its an actual string, you ask the file system if its valid or not, then raise a *better* error message (e.g. IOError: file not found)

yeah I see this doesn't use ``pandas.io.common.`` to actually open the file, instead it passes it to the msgpack routines. This is where checking for things e.g. in ``read_csv`` happen.
This is a dupe of #12225 (closed that one) - issue is that in python 2 the string is interpreted as bytes.

Resolving patch:
diff --git a/pandas/io/packers.py b/pandas/io/packers.py
--- a/pandas/io/packers.py
+++ b/pandas/io/packers.py
@@ -192,7 +192,6 @@ def read(fh):
 
     # see if we have an actual file
     if isinstance(path_or_buf, compat.string_types):
-
         try:
             exists = os.path.exists(path_or_buf)
         except (TypeError, ValueError):
@@ -202,18 +201,21 @@ def read(fh):
             with open(path_or_buf, 'rb') as fh:
                 return read(fh)
 
-    # treat as a binary-like
     if isinstance(path_or_buf, compat.binary_type):
+        # treat as a binary-like
         fh = None
         try:
-            fh = compat.BytesIO(path_or_buf)
-            return read(fh)
+            # We can't distinguish between a path and a buffer of bytes in
+            # Python 2 so instead assume the first byte of a valid path is
+            # less than 0x80.
+            if compat.PY3 or ord(path_or_buf[0]) >= 0x80:
+                fh = compat.BytesIO(path_or_buf)
+                return read(fh)
         finally:
             if fh is not None:
                 fh.close()
-
-    # a buffer like
-    if hasattr(path_or_buf, 'read') and compat.callable(path_or_buf.read):
+    elif hasattr(path_or_buf, 'read') and compat.callable(path_or_buf.read):
+        # treat as a buffer like
         return read(path_or_buf)
 
     raise ValueError('path_or_buf needs to be a string file path or file-like')


Test patch:
diff --git a/pandas/tests/io/test_common.py b/pandas/tests/io/test_common.py
--- a/pandas/tests/io/test_common.py
+++ b/pandas/tests/io/test_common.py
@@ -10,7 +10,7 @@
 import pandas.util.testing as tm
 
 from pandas.io import common
-from pandas.compat import is_platform_windows, StringIO
+from pandas.compat import is_platform_windows, StringIO, FileNotFoundError
 
 from pandas import read_csv, concat
 
@@ -125,6 +125,26 @@ def test_iterator(self):
         tm.assert_frame_equal(first, expected.iloc[[0]])
         tm.assert_frame_equal(concat(it), expected.iloc[1:])
 
+    @pytest.mark.parametrize('reader, module, error_class, fn_ext', [
+        (pd.read_csv, 'os', FileNotFoundError, 'csv'),
+        (pd.read_table, 'os', FileNotFoundError, 'csv'),
+        (pd.read_fwf, 'os', FileNotFoundError, 'txt'),
+        (pd.read_excel, 'xlrd', FileNotFoundError, 'xlsx'),
+        (pd.read_feather, 'feather', Exception, 'feather'),
+        (pd.read_hdf, 'tables', FileNotFoundError, 'h5'),
+        (pd.read_stata, 'os', FileNotFoundError, 'dta'),
+        (pd.read_sas, 'os', FileNotFoundError, 'sas7bdat'),
+        (pd.read_json, 'os', ValueError, 'json'),
+        (pd.read_msgpack, 'os', ValueError, 'mp'),
+        (pd.read_pickle, 'os', FileNotFoundError, 'pickle'),
+    ])
+    def test_read_non_existant(self, reader, module, error_class, fn_ext):
+        pytest.importorskip(module)
+
+        path = os.path.join(HERE, 'data', 'does_not_exist.' + fn_ext)
+        with pytest.raises(error_class):
+            reader(path)
+
     @pytest.mark.parametrize('reader, module, path', [
         (pd.read_csv, 'os', os.path.join(HERE, 'data', 'iris.csv')),
         (pd.read_table, 'os', os.path.join(HERE, 'data', 'iris.csv')),