Repository: numba/numba

Issue:
NUMBA_ENABLE_CUDASIM broken with record arrays
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->

```py
$ python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 
[GCC 7.3.0] :: Anaconda custom (64-bit) on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numba import cuda
>>> import numpy as np
>>> a = np.recarray((1,), dtype=[("value1", np.int64), ("value2", np.float64)]) 
>>> def test(x):
...     x.value1
... 
>>> cuda.jit(test)(a)
>>> 
>>> 
$ NUMBA_ENABLE_CUDASIM=1 python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 
[GCC 7.3.0] :: Anaconda custom (64-bit) on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numba import cuda
>>> import numpy as np
>>> a = np.recarray((1,), dtype=[("value1", np.int64), ("value2", np.float64)]) 
>>> def test(x):
...     x.value1
... 
>>> cuda.jit(test)(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 95, in __call__
    bm.run(grid_point, *fake_args)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 258, in run
    reraise(*(t.exception))
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 157, in run
    super(BlockThread, self).run()
  File "/home/aaronmeurer/anaconda3/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 236, in target
    self._f(*args)
  File "<stdin>", line 2, in test
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/cudadrv/devicearray.py", line 61, in __getattr__
    % attrname), e)
  File "<string>", line 2, in raise_from
AttributeError: tid=[0, 0, 0] ctaid=[0, 0, 0]: Wrapped array has no attribute 'value1'
```
NUMBA_ENABLE_CUDASIM broken with record arrays
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug

<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

<!--

Please include details of the bug here, including, if applicable, what you
expected to happen!

-->

```py
$ python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 
[GCC 7.3.0] :: Anaconda custom (64-bit) on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numba import cuda
>>> import numpy as np
>>> a = np.recarray((1,), dtype=[("value1", np.int64), ("value2", np.float64)]) 
>>> def test(x):
...     x.value1
... 
>>> cuda.jit(test)(a)
>>> 
>>> 
$ NUMBA_ENABLE_CUDASIM=1 python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 
[GCC 7.3.0] :: Anaconda custom (64-bit) on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numba import cuda
>>> import numpy as np
>>> a = np.recarray((1,), dtype=[("value1", np.int64), ("value2", np.float64)]) 
>>> def test(x):
...     x.value1
... 
>>> cuda.jit(test)(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 95, in __call__
    bm.run(grid_point, *fake_args)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 258, in run
    reraise(*(t.exception))
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/six.py", line 658, in reraise
    raise value.with_traceback(tb)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 157, in run
    super(BlockThread, self).run()
  File "/home/aaronmeurer/anaconda3/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/kernel.py", line 236, in target
    self._f(*args)
  File "<stdin>", line 2, in test
  File "/home/aaronmeurer/anaconda3/lib/python3.7/site-packages/numba/cuda/simulator/cudadrv/devicearray.py", line 61, in __getattr__
    % attrname), e)
  File "<string>", line 2, in raise_from
AttributeError: tid=[0, 0, 0] ctaid=[0, 0, 0]: Wrapped array has no attribute 'value1'
```


Discussion:
Using the `x['value1']` syntax works. 

None of the arithmetic operations like `-`, `<`, etc. work either. The `__getattr__` on FakeCUDAArray doesn't handle those. I'm not clear why it isn't handling the record array lookups. My guess is it gets converted to a normal array somewhere (e.g., `a = np.array([(1,1.)], dtype=[("value1", np.int64), ("value2", np.float64)])`), which work with getitem but not getattr lookups on the dtype values.
#4398 looks very similar to this.

A little ironic twist to it is that this pops up in the code that @gmarkall originally wrote the CUDA simulator for back in the day :-). Would be great to have it fixed! 
Using the `x['value1']` syntax works. 

None of the arithmetic operations like `-`, `<`, etc. work either. The `__getattr__` on FakeCUDAArray doesn't handle those. I'm not clear why it isn't handling the record array lookups. My guess is it gets converted to a normal array somewhere (e.g., `a = np.array([(1,1.)], dtype=[("value1", np.int64), ("value2", np.float64)])`), which work with getitem but not getattr lookups on the dtype values.
#4398 looks very similar to this.

A little ironic twist to it is that this pops up in the code that @gmarkall originally wrote the CUDA simulator for back in the day :-). Would be great to have it fixed! 

Resolving patch:
diff --git a/numba/cuda/simulator/cudadrv/devicearray.py b/numba/cuda/simulator/cudadrv/devicearray.py
--- a/numba/cuda/simulator/cudadrv/devicearray.py
+++ b/numba/cuda/simulator/cudadrv/devicearray.py
@@ -197,7 +197,7 @@ def to_device(ary, stream=0, copy=True, to=None):
                 dtype=ary.dtype,
                 shape=ary.shape,
                 strides=ary.strides,
-            ),
+            ).view(type=type(ary)),
         )
     else:
         to.copy_to_device(ary, stream=stream)


Test patch:
diff --git a/numba/cuda/tests/cudadrv/test_cuda_ndarray.py b/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
--- a/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
+++ b/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
@@ -299,5 +299,32 @@ def test_devicearray_contiguous_device_strided(self):
             str(e.exception))
 
 
+class TestRecarray(SerialMixin, unittest.TestCase):
+    def test_recarray(self):
+        # From issue #4111
+        a = np.recarray((16,), dtype=[
+            ("value1", np.int64),
+            ("value2", np.float64),
+        ])
+        a.value1 = np.arange(a.size, dtype=np.int64)
+        a.value2 = np.arange(a.size, dtype=np.float64) / 100
+
+        expect1 = a.value1
+        expect2 = a.value2
+
+        def test(x, out1, out2):
+            i = cuda.grid(1)
+            if i < x.size:
+                out1[i] = x.value1[i]
+                out2[i] = x.value2[i]
+
+        got1 = np.zeros_like(expect1)
+        got2 = np.zeros_like(expect2)
+        cuda.jit(test)[1, a.size](a, got1, got2)
+
+        np.testing.assert_array_equal(expect1, got1)
+        np.testing.assert_array_equal(expect2, got2)
+
+
 if __name__ == '__main__':
     unittest.main()