Repository: numpy/numpy

Issue:
`np.rec.fromrecords` looks broken since 1.14.0
```python
>>> dt = [('a', int), ('b', float)]
>>> np.rec.fromrecords([ [1, 1.5], [2, 2.5] ], dtype=dt)
rec.array([[(1, 1. ), (1, 1.5)],
           [(2, 2. ), (2, 2.5)]],
          dtype=[('a', '<i8'), ('b', '<f8')])
```

while in 1.13.3 that would be:
```python
rec.array([(1,  1.5), (2,  2.5)], 
          dtype=[('a', '<i8'), ('b', '<f8')])
```


Discussion:
Confirmed in master. cc @ahaldane.

`fromrecords` seems generally poorly thought out to me - it doesn't seem to solve a problem that isn't better solved with simply `np.array`
I think this follows on from #10344
Right, looks like #10344.

You can "fix" your code in numpy 1.14 by doing:
```python
np.rec.fromrecords([ (1, 1.5), (2, 2.5) ], dtype=dt)
```
that is, using tuples instead of lists.

We tried to fix this with #10543, this I thought went back to the old behavior but with a  FutureWarning. But looking at that again I'm not sure that reverted the behavior - it only seems to hav added a FutureWarning that doesn't get hit.  Might be worth a second look.
Hmm, I'm trying to see if we can temporarily preserve the old behavior in 1.14.

It seems hard, since the change in behavior only affects the following very specific input and nothing else: A python list, whose elements are convertible to tuples, and the elements of those tuples do *not* support the buffer interface.

I'm tempted to just leave this alone. The old 1.13 behavior was pretty inconsistent and buggy: Not only was is subject to all the bugs linked in #10344, but `fromrecords` in particular also only worked for 1d arrays, and it failed if the inner elements can be converted to buffers, for example:
```python
>>> np.__version__
'1.13.3'
>>> dt = [('a', 'S2'), ('b', 'S1')]
>>> np.rec.fromrecords([ ['aaa','bbb'], ['ccc', 'ddd'] ], dtype=dt)
rec.array([[('aa', 'a'), ('bb', 'b')],
           [('cc', 'c'), ('dd', 'd')]], 
          dtype=[('a', 'S2'), ('b', 'S1')])
```
It seems like everything will work correctly if we sacrifice performance a little and go via an object array (like one codepath already does)

I'm not sure... I think the tricky part is to detect when we got a list of lists that should be converted to a list of tuples. The old code dtected this because the list-of-lists would fail, so then it could fix it up in the except block in `fromrecords`. But now the code no longer fails, so we never get to the fixup.

By the way, another fun example of 1.13 behavior:
```python
>>> p = np.float64(np.pi)
>>> np.rec.fromrecords([ [p, p], [p, p] ], dtype=[('a', 'f4'), ('b', 'f4')])
rec.array([[(  3.37028055e+12,  2.142699), (  3.37028055e+12,  2.142699)],
           [(  3.37028055e+12,  2.142699), (  3.37028055e+12,  2.142699)]], 
          dtype=[('a', '<f4'), ('b', '<f4')])
```
BTW, what I was trying to do is to create multidimensional arrays.
so, I had 40x64x4 list of list of list where last 4 elements were of different types.
```
        dtype  = ['u8', 'f8', 'u8', 'f8']
        data = np.array([
            np.array([np.rec.fromrecords(x, dtype=dtype)
            for x in data
        ])
```
@podshumok probably in that particular case, I would recommend:
```python
np.array([[tuple(e) for e in row] for row in data], dtype='u8,f8,u8,f8')
```
(replace `np.array` with  `np.rec.array` if you want a record array)
Thank you, I'll give it a try. For now I've solved my problem with some `zip`s and standard python sums.

Resolving patch:
diff --git a/numpy/core/records.py b/numpy/core/records.py
--- a/numpy/core/records.py
+++ b/numpy/core/records.py
@@ -676,32 +676,40 @@ def fromrecords(recList, dtype=None, shape=None, formats=None, names=None,
     else:
         descr = format_parser(formats, names, titles, aligned, byteorder)._descr
 
-    try:
-        retval = sb.array(recList, dtype=descr)
-    except (TypeError, ValueError):
-        if (shape is None or shape == 0):
-            shape = len(recList)
-        if isinstance(shape, (int, long)):
-            shape = (shape,)
-        if len(shape) > 1:
-            raise ValueError("Can only deal with 1-d array.")
-        _array = recarray(shape, descr)
-        for k in range(_array.size):
-            _array[k] = tuple(recList[k])
-        # list of lists instead of list of tuples ?
-        # 2018-02-07, 1.14.1
-        warnings.warn(
-            "fromrecords expected a list of tuples, may have received a list "
-            "of lists instead. In the future that will raise an error",
-            FutureWarning, stacklevel=2)
-        return _array
-    else:
-        if shape is not None and retval.shape != shape:
-            retval.shape = shape
+    # deprecated back-compat block for numpy 1.14, to be removed in a later
+    # release. This converts list-of-list input to list-of-tuples in some
+    # cases, as done in numpy <= 1.13. In the future we will require tuples.
+    if (isinstance(recList, list) and len(recList) > 0
+            and isinstance(recList[0], list) and len(recList[0]) > 0
+            and not isinstance(recList[0][0], (list, tuple))):
+
+        try: 
+            memoryview(recList[0][0]) 
+        except:
+            if (shape is None or shape == 0):
+                shape = len(recList)
+            if isinstance(shape, (int, long)):
+                shape = (shape,)
+            if len(shape) > 1:
+                raise ValueError("Can only deal with 1-d array.")
+            _array = recarray(shape, descr)
+            for k in range(_array.size):
+                _array[k] = tuple(recList[k])
+            # list of lists instead of list of tuples ?
+            # 2018-02-07, 1.14.1
+            warnings.warn(
+                "fromrecords expected a list of tuples, may have received a "
+                "list of lists instead. In the future that will raise an error",
+                FutureWarning, stacklevel=2)
+            return _array
+        else:
+            pass
 
-    res = retval.view(recarray)
+    retval = sb.array(recList, dtype=descr)
+    if shape is not None and retval.shape != shape:
+        retval.shape = shape
 
-    return res
+    return retval.view(recarray)
 
 
 def fromstring(datastring, dtype=None, shape=None, offset=0, formats=None,


Test patch:
diff --git a/numpy/core/tests/test_records.py b/numpy/core/tests/test_records.py
--- a/numpy/core/tests/test_records.py
+++ b/numpy/core/tests/test_records.py
@@ -54,6 +54,14 @@ def test_fromrecords_2d(self):
 
         assert_equal(r1, r2)
 
+    def test_fromrecords_list_of_lists(self):
+        # gh-10870 : For numpy 1.14 we keep the deprecated behavior
+        # that 1d list-of-lists input is accepted by fromrecords
+        expected = np.rec.array([(1,  1.5), (2,  2.5)], dtype='i8,f8')
+        with assert_warns(FutureWarning):
+            r = np.rec.fromrecords([[1, 1.5], [2, 2.5]], dtype='i8,f8')
+        assert_equal(r, expected)
+
     def test_method_array(self):
         r = np.rec.array(b'abcdefg' * 100, formats='i2,a3,i4', shape=3, byteorder='big')
         assert_equal(r[1].item(), (25444, b'efg', 1633837924))