Repository: numpy/numpy

Issue:
np.loadtxt no longer loads bz2 files in python2
I have been using `np.loadtxt()` to happily load `txt.bz2` files for many years. I recently upgraded to numpy 1.14.2, and I noticed that it no longer seems possible to load a `txt.bz2` file using python2. 

Was this intensional, or some sort of bug?

```
Traceback (most recent call last):
  File "/Users/danhickstein/Documents/GitHub/PyAbel-DH/examples/example_all_O2.py", line 41, in <module>
    IM = np.loadtxt('data/O2-ANU1024.txt.bz2')
  File "/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.py", line 917, in loadtxt
    fh = np.lib._datasource.open(fname, 'rt', encoding=encoding)
  File "/anaconda2/lib/python2.7/site-packages/numpy/lib/_datasource.py", line 260, in open
    return ds.open(path, mode, encoding=encoding, newline=newline)
  File "/anaconda2/lib/python2.7/site-packages/numpy/lib/_datasource.py", line 614, in open
    encoding=encoding, newline=newline)
  File "/anaconda2/lib/python2.7/site-packages/numpy/lib/_datasource.py", line 88, in _python2_bz2open
    raise ValueError("bz2 text files not supported in python2")
ValueError: bz2 text files not supported in python2
```


Discussion:
It was intended and was part of extending encoding to text files in #4208. The problem is the `mode='rt'` in the call, might try passing `encoding='bytes'` in `loadtxt`. I think we should be able to fake it in Python 2 by assuming ascii/latin1  files if we raise a warning to that effect.
@juliantaylor Thoughts?
The issue is that the python2 bzip2 module is lacking functions to be used in a `TextIOWrapper ` so we can use standard encoding handling. gzip has similar issues but it is only missing one single simple function (read1) which we sort of patch in but for bzip2 I considered it too much.

Maybe throwing an error is a bit harsh, one could indeed just enforce the bzip 2 textfiles to be latin1 and hope for the best during decoding.
@juliantaylor 
The simple thing would be to just open the file with a warning about latin1. An alternative would be to include the python3 compatible version available on PyPI https://pypi.org/project/bz2file/#files, it has an Apache License Version 2, which I think is OK. The file is pure Python.

Resolving patch:
diff --git a/numpy/lib/_datasource.py b/numpy/lib/_datasource.py
--- a/numpy/lib/_datasource.py
+++ b/numpy/lib/_datasource.py
@@ -37,6 +37,7 @@
 
 import os
 import sys
+import warnings
 import shutil
 import io
 
@@ -85,9 +86,10 @@ def _python2_bz2open(fn, mode, encoding, newline):
 
     if "t" in mode:
         # BZ2File is missing necessary functions for TextIOWrapper
-        raise ValueError("bz2 text files not supported in python2")
-    else:
-        return bz2.BZ2File(fn, mode)
+        warnings.warn("Assuming latin1 encoding for bz2 text file in Python2",
+                      RuntimeWarning, stacklevel=5)
+        mode = mode.replace("t", "")
+    return bz2.BZ2File(fn, mode)
 
 def _python2_gzipopen(fn, mode, encoding, newline):
     """ Wrapper to open gzip in text mode.


Test patch:
diff --git a/numpy/lib/tests/test__datasource.py b/numpy/lib/tests/test__datasource.py
--- a/numpy/lib/tests/test__datasource.py
+++ b/numpy/lib/tests/test__datasource.py
@@ -2,11 +2,14 @@
 
 import os
 import sys
+import pytest
 from tempfile import mkdtemp, mkstemp, NamedTemporaryFile
 from shutil import rmtree
 
-from numpy.testing import assert_, assert_equal, assert_raises, SkipTest
 import numpy.lib._datasource as datasource
+from numpy.testing import (
+    assert_, assert_equal, assert_raises, assert_warns, SkipTest
+    )
 
 if sys.version_info[0] >= 3:
     import urllib.request as urllib_request
@@ -161,6 +164,24 @@ def test_ValidBz2File(self):
         fp.close()
         assert_equal(magic_line, result)
 
+    @pytest.mark.skipif(sys.version_info[0] >= 3, reason="Python 2 only")
+    def test_Bz2File_text_mode_warning(self):
+        try:
+            import bz2
+        except ImportError:
+            # We don't have the bz2 capabilities to test.
+            raise SkipTest
+        # Test datasource's internal file_opener for BZip2 files.
+        filepath = os.path.join(self.tmpdir, 'foobar.txt.bz2')
+        fp = bz2.BZ2File(filepath, 'w')
+        fp.write(magic_line)
+        fp.close()
+        with assert_warns(RuntimeWarning):
+            fp = self.ds.open(filepath, 'rt')
+            result = fp.readline()
+            fp.close()
+        assert_equal(magic_line, result)
+
 
 class TestDataSourceExists(object):
     def setup(self):