Repository: numpy/numpy

Issue:
loadtxt fails with complex data under python 3
If the file `complex-data.txt` contains the following

```
8.56701944e-02-4.51046468e-03j
-8.10738577e-02+1.38042481e-03j
-4.05139503e-02+2.73159837e-04j
4.28743806e-02-3.94370325e-03j
-4.76092917e-02+3.93395282e-03j
1.17309845e-01-3.28224736e-04j
```

Then the code

```
import numpy as np
np.loadtxt('complex-data.txt', dtype=np.complex128)
```

gives the exception `TypeError: complex() argument must be a string or a number, not 'bytes'`. It works perfectly fine under python 2.

This appears to be caused by `loadtxt` failing to properly decode the text file. May be related to #2715, #4208, #4543, #5530. These issues give me the impression that `loadtxt` is essentially broken under python 3, and needs a radical overhaul.

Are there any suggested workarounds?



Discussion:
I think this may have been fixed in master as part of another PR, but maybe not. What numpy version are you using?

I'm using numpy version 1.9.0

I just confirmed that the same problem also occurs under numpy 1.9.2

Looks like it is fixed in master.

```
Python 3.4.1 (default, Nov  3 2014, 14:38:10) 
[GCC 4.9.1 20140930 (Red Hat 4.9.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.genfromtxt('test.txt', dtype=complex)
array([ 0.08567019-0.00451046j, -0.08107386+0.00138042j,
       -0.04051395+0.00027316j,  0.04287438-0.0039437j ,
       -0.04760929+0.00393395j,  0.11730984-0.00032822j])
>>> np.__version__
'1.10.0.dev0+1af465b'
```

But evidently not in 1.9.2.

Oops, but not in `loadtxt`. Nasty.

@DavidPowell Both those functions are difficult to follow, but if you want to take a shot at a fix, go for it. The fact that `genfromtxt` works should help.


Resolving patch:
diff --git a/numpy/lib/npyio.py b/numpy/lib/npyio.py
--- a/numpy/lib/npyio.py
+++ b/numpy/lib/npyio.py
@@ -690,7 +690,7 @@ def floatconv(x):
     elif issubclass(typ, np.floating):
         return floatconv
     elif issubclass(typ, np.complex):
-        return complex
+        return lambda x: complex(asstr(x))
     elif issubclass(typ, np.bytes_):
         return bytes
     else:
@@ -863,7 +863,12 @@ def pack_items(items, packing):
             return tuple(ret)
 
     def split_line(line):
-        """Chop off comments, strip, and split at delimiter."""
+        """Chop off comments, strip, and split at delimiter.
+
+        Note that although the file is opened as text, this function
+        returns bytes.
+
+        """
         if comments is None:
             line = asbytes(line).strip(asbytes('\r\n'))
         else:


Test patch:
diff --git a/numpy/lib/tests/test_io.py b/numpy/lib/tests/test_io.py
--- a/numpy/lib/tests/test_io.py
+++ b/numpy/lib/tests/test_io.py
@@ -707,6 +707,14 @@ def test_from_float_hex(self):
             res = np.loadtxt(c, dtype=dt)
             assert_equal(res, tgt, err_msg="%s" % dt)
 
+    def test_from_complex(self):
+        tgt = (complex(1, 1), complex(1, -1))
+        c = TextIO()
+        c.write("%s %s" % tgt)
+        c.seek(0)
+        res = np.loadtxt(c, dtype=np.complex)
+        assert_equal(res, tgt)
+
     def test_universal_newline(self):
         f, name = mkstemp()
         os.write(f, b'1 21\r3 42\r')