Repository: numba/numba

Issue:
CUDA device arrays are always 'A' order and they probably shouldn't be.
<!--

Thanks for opening an issue! To help the Numba team handle your information
efficiently, please first ensure that there is no other issue present that
already describes the issue you have
(search at https://github.com/numba/numba/issues?&q=is%3Aissue).

For more general "how do I do X?" type questions, please speak to us in real
time on https://gitter.im/numba/numba or post to the Numba mailing list
https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.

-->

## Reporting a bug
<!--

Before submitting a bug report please ensure that you can check off these boxes:

-->

- [x] I am using the latest released version of Numba (most recent is visible in
 the change log (https://github.com/numba/numba/blob/master/CHANGE_LOG).
- [x] I have included below a minimal working reproducer (if you are unsure how
 to write one see http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports).

There may be good reasons for this that I'm yet to discover. But this:
```python
from numba import cuda
import numpy as np
import math

@cuda.jit
def func(A, out):
    i = cuda.grid(1)
    out[i] = math.cos(A[i])

n = 4
a = np.ones((n,))
result = np.zeros((n,))

func[1, 128](a, result)

for f in func.definitions:
    print(f, func.definitions[f])

d_a = cuda.to_device(a)
func[1, 128](d_a, result)

print("After device array")
for f in func.definitions:
    print(f, func.definitions[f])
```
does this:
```
((5, 0), (array(float64, 1d, C), array(float64, 1d, C))) <numba.cuda.compiler.CUDAKernel object at 0x7fe2ae2de748>
After device array
((5, 0), (array(float64, 1d, C), array(float64, 1d, C))) <numba.cuda.compiler.CUDAKernel object at 0x7fe2ae2de748>
((5, 0), (array(float64, 1d, A), array(float64, 1d, C))) <numba.cuda.compiler.CUDAKernel object at 0x7fe2ae256978>
```
i.e. the device array is typed as `array(float64, 1d, A),` which triggers a recompilation. Further, this seems to be hardcoded:
https://github.com/numba/numba/blob/3825f12cddecdf8c3af2fa75eff935f2e5617263/numba/cuda/cudadrv/devicearray.py#L153-L160

such that all device arrays `numba.typeof` as "A" order.


Discussion:
Related discussion on gitter: https://gitter.im/numba/numba?at=5d8e32aeeb1eff63d635062a, seems like CuPy arrays are also appearing 'A' ordered.

CC: @leofang @beckermr 
Thanks a lot @stuartarchibald for experimenting and reporting for us. I'd have thought the other way around. Looks like Numba is silently converting 'C' (or 'F') to 'A' somewhere? Because in CuPy only 'C' and 'F' are supported: https://github.com/cupy/cupy/blob/36ecb4423762a79aea98bd1bbb40a3903d28ac18/cupy/core/core.pyx#L106-L107

> Thanks a lot @stuartarchibald for experimenting and reporting for us.

No problem

> I'd have thought the other way around. Looks like Numba is silently converting 'C' (or 'F') to 'A' somewhere? Because in CuPy only 'C' and 'F' are supported: https://github.com/cupy/cupy/blob/36ecb4423762a79aea98bd1bbb40a3903d28ac18/cupy/core/core.pyx#L106-L107

yes, well, seems like device arrays are typed as 'A' ordered irrespective of what they are in reality https://github.com/numba/numba/blob/3825f12cddecdf8c3af2fa75eff935f2e5617263/numba/cuda/cudadrv/devicearray.py#L153-L160 not sure what happens to a CuPy array in typing, but would guess the same, will check.


Resolving patch:
diff --git a/numba/cuda/cudadrv/devicearray.py b/numba/cuda/cudadrv/devicearray.py
--- a/numba/cuda/cudadrv/devicearray.py
+++ b/numba/cuda/cudadrv/devicearray.py
@@ -166,8 +166,30 @@ def _numba_type_(self):
         Magic attribute expected by Numba to get the numba type that
         represents this object.
         """
+        # Typing considerations:
+        #
+        # 1. The preference is to use 'C' or 'F' layout since this enables
+        # hardcoding stride values into compiled kernels, which is more
+        # efficient than storing a passed-in value in a register.
+        #
+        # 2. If an array is both C- and F-contiguous, prefer 'C' layout as it's
+        # the more likely / common case.
+        #
+        # 3. If an array is broadcast then it must be typed as 'A' - using 'C'
+        # or 'F' does not apply for broadcast arrays, because the strides, some
+        # of which will be 0, will not match those hardcoded in for 'C' or 'F'
+        # layouts.
+
+        broadcast = 0 in self.strides
+        if self.flags['C_CONTIGUOUS'] and not broadcast:
+            layout = 'C'
+        elif self.flags['F_CONTIGUOUS'] and not broadcast:
+            layout = 'F'
+        else:
+            layout = 'A'
+
         dtype = numpy_support.from_dtype(self.dtype)
-        return types.Array(dtype, self.ndim, 'A')
+        return types.Array(dtype, self.ndim, layout)
 
     @property
     def device_ctypes_pointer(self):
diff --git a/numba/cuda/dispatcher.py b/numba/cuda/dispatcher.py
--- a/numba/cuda/dispatcher.py
+++ b/numba/cuda/dispatcher.py
@@ -194,7 +194,6 @@ class CUDAUFuncMechanism(UFuncMechanism):
     Provide OpenCL specialization
     """
     DEFAULT_STREAM = 0
-    ARRAY_ORDER = 'A'
 
     def launch(self, func, count, stream, args):
         func.forall(count, stream=stream)(*args)


Test patch:
diff --git a/numba/cuda/tests/cudadrv/test_cuda_ndarray.py b/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
--- a/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
+++ b/numba/cuda/tests/cudadrv/test_cuda_ndarray.py
@@ -298,6 +298,70 @@ def test_devicearray_contiguous_device_strided(self):
             devicearray.errmsg_contiguous_buffer,
             str(e.exception))
 
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_simple_c(self):
+        # C-order 1D array
+        a = np.zeros(10, order='C')
+        d = cuda.to_device(a)
+        self.assertEqual(d._numba_type_.layout, 'C')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_simple_f(self):
+        # F-order array that is also C layout.
+        a = np.zeros(10, order='F')
+        d = cuda.to_device(a)
+        self.assertEqual(d._numba_type_.layout, 'C')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_2d_c(self):
+        # C-order 2D array
+        a = np.zeros((2, 10), order='C')
+        d = cuda.to_device(a)
+        self.assertEqual(d._numba_type_.layout, 'C')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_2d_f(self):
+        # F-order array that can only be F layout
+        a = np.zeros((2, 10), order='F')
+        d = cuda.to_device(a)
+        self.assertEqual(d._numba_type_.layout, 'F')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_noncontig_slice_c(self):
+        # Non-contiguous slice of C-order array
+        a = np.zeros((5, 5), order='C')
+        d = cuda.to_device(a)[:,2]
+        self.assertEqual(d._numba_type_.layout, 'A')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_noncontig_slice_f(self):
+        # Non-contiguous slice of F-order array
+        a = np.zeros((5, 5), order='F')
+        d = cuda.to_device(a)[2,:]
+        self.assertEqual(d._numba_type_.layout, 'A')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_contig_slice_c(self):
+        # Contiguous slice of C-order array
+        a = np.zeros((5, 5), order='C')
+        d = cuda.to_device(a)[2,:]
+        self.assertEqual(d._numba_type_.layout, 'C')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_contig_slice_f(self):
+        # Contiguous slice of F-order array - is both C- and F-contiguous, so
+        # types as 'C' layout
+        a = np.zeros((5, 5), order='F')
+        d = cuda.to_device(a)[:,2]
+        self.assertEqual(d._numba_type_.layout, 'C')
+
+    @skip_on_cudasim('Typing not done in the simulator')
+    def test_devicearray_typing_order_broadcasted(self):
+        # Broadcasted array, similar to that used for passing scalars to ufuncs
+        a = np.broadcast_to(np.array([1]), (10,))
+        d = cuda.to_device(a)
+        self.assertEqual(d._numba_type_.layout, 'A')
+
 
 class TestRecarray(SerialMixin, unittest.TestCase):
     def test_recarray(self):
diff --git a/numba/cuda/tests/cudapy/test_array.py b/numba/cuda/tests/cudapy/test_array.py
--- a/numba/cuda/tests/cudapy/test_array.py
+++ b/numba/cuda/tests/cudapy/test_array.py
@@ -3,6 +3,7 @@
 import numpy as np
 
 from numba.cuda.testing import unittest, SerialMixin
+from numba.cuda.testing import skip_on_cudasim
 from numba import cuda
 
 
@@ -61,5 +62,30 @@ def test_auto_device_const(self):
         d, _ = cuda.devicearray.auto_device(2)
         self.assertTrue(np.all(d.copy_to_host() == np.array(2)))
 
+    @skip_on_cudasim('Kernel definitions not created in the simulator')
+    def test_issue_4628(self):
+        # CUDA Device arrays were reported as always being typed with 'A' order
+        # so launching the kernel with a host array and then a device array
+        # resulted in two definitions being compiled - one for 'C' order from
+        # the host array, and one for 'A' order from the device array. With the
+        # resolution of this issue, the order of the device array is also 'C',
+        # so after the kernel launches there should only be one definition of
+        # the function.
+        @cuda.jit
+        def func(A, out):
+            i = cuda.grid(1)
+            out[i] = A[i] * 2
+
+        n = 128
+        a = np.ones((n,))
+        d_a = cuda.to_device(a)
+        result = np.zeros((n,))
+
+        func[1, 128](a, result)
+        func[1, 128](d_a, result)
+
+        self.assertEqual(1, len(func.definitions))
+
+
 if __name__ == '__main__':
     unittest.main()