Repository: numpy/numpy

Issue:
BUG np.broadcast_arrays does not work on void arrays with object fields
In the process of working on #4622, it was realised that `np.broadcas_arrays` do not work for void arrays with object fields, because in the process of changing new strides and shapes in `as_strided`, the `dtype` gets lost, and this cannot just be reset when object fields are present. While obviously this is somewhat of a corner case, it might still be good to see if it can be fixed easily, perhaps by doing something that preserves the dtype, or by bypassing the test for object fields.

```
import numpy as np
dt = np.dtype([('x', 'i8'), ('y', '?'), ('z', 'O')])
b = np.array([(1, True, None), (2, False, [3, 4, 5])], dtype=dt)
c = np.array([[-1], [-2]])
b_strided, c_strided = np.broadcast_arrays(b, c)
TypeError                                 Traceback (most recent call last)
...
/usr/lib/python3/dist-packages/numpy/lib/stride_tricks.py in as_strided(x, shape, strides)
     31     # Make sure dtype is correct in case of custom dtype
     32     if array.dtype.kind == 'V':
---> 33         array.dtype = x.dtype
     34     return array
     35 

TypeError: Cannot change data-type for object array.
```



Discussion:
Hmm. I thought we fixed this in https://github.com/numpy/numpy/pull/4408.

@shoyer - sadly, no. I checked just to be sure and this is still an issue with current master.

I see two ways of going about fixing this:
1. One is to modify `PyArray_FromInterface`, which lives in [`numpy\core\src\multiarray.ctors.c`](https://github.com/numpy/numpy/blob/maintenance/1.9.x/numpy/core/src/multiarray/ctors.c#L2075). It is currently using the `'typestr'` entry of the `__array_interface__` dictionary to determine the dtype, and completely ignores the `'descr'` optional entry. The idea would be to, if the resulting type created from `'typestr'` is `NPY_VOID`, recreate the dtype from `'descr'`, with all the necessary checks to see that both dtypes are compatible, and use the latter to instantiate the array.
2. We could also simply rewrite `as_strided` in C with a call to `PyArray_NewFromDescr`, reusing the dtype of the passed-in object, which shouldn't have these type of issues.

The first approach seems the correct one, but it may create backwards compatibility headaches, and is more involved to get right. The second is very straightforward, but it would hide in the C code the nice example of how to do cool things with `__array_interface__` from pure Python that `as_strided` is right now.

Any thoughts?


Resolving patch:
diff --git a/numpy/lib/stride_tricks.py b/numpy/lib/stride_tricks.py
--- a/numpy/lib/stride_tricks.py
+++ b/numpy/lib/stride_tricks.py
@@ -46,9 +46,11 @@ def as_strided(x, shape=None, strides=None, subok=False):
     if strides is not None:
         interface['strides'] = tuple(strides)
     array = np.asarray(DummyArray(interface, base=x))
-    # Make sure dtype is correct in case of custom dtype
-    if array.dtype.kind == 'V':
+
+    if array.dtype.fields is None and x.dtype.fields is not None:
+        # This should only happen if x.dtype is [('', 'Vx')]
         array.dtype = x.dtype
+
     return _maybe_view_as_subclass(x, array)
 
 


Test patch:
diff --git a/numpy/lib/tests/test_stride_tricks.py b/numpy/lib/tests/test_stride_tricks.py
--- a/numpy/lib/tests/test_stride_tricks.py
+++ b/numpy/lib/tests/test_stride_tricks.py
@@ -290,6 +290,29 @@ def test_as_strided():
     expected = np.array([[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]])
     assert_array_equal(a_view, expected)
 
+    # Regression test for gh-5081
+    dt = np.dtype([('num', 'i4'), ('obj', 'O')])
+    a = np.empty((4,), dtype=dt)
+    a['num'] = np.arange(1, 5)
+    a_view = as_strided(a, shape=(3, 4), strides=(0, a.itemsize))
+    expected_num = [[1, 2, 3, 4]] * 3
+    expected_obj = [[None]*4]*3
+    assert_equal(a_view.dtype, dt)
+    assert_array_equal(expected_num, a_view['num'])
+    assert_array_equal(expected_obj, a_view['obj'])
+
+    # Make sure that void types without fields are kept unchanged
+    a = np.empty((4,), dtype='V4')
+    a_view = as_strided(a, shape=(3, 4), strides=(0, a.itemsize))
+    assert_equal(a.dtype, a_view.dtype)
+
+    # Make sure that the only type that could fail is properly handled
+    dt = np.dtype({'names': [''], 'formats': ['V4']})
+    a = np.empty((4,), dtype=dt)
+    a_view = as_strided(a, shape=(3, 4), strides=(0, a.itemsize))
+    assert_equal(a.dtype, a_view.dtype)
+
+
 
 class VerySimpleSubClass(np.ndarray):
     def __new__(cls, *args, **kwargs):