Repository: numba/numba

Issue:
__cuda_array_interface__ is not compliant with the standard
**UPDATE: reporting two issues on `__cuda_array_interface__`**

First, currently it's impossible for `'strides'` to be `None`, and it is always there instead of being optional:

https://github.com/numba/numba/blob/a97dc18f3e5d69429949e557bc98ec513f81b04c/numba/cuda/cudadrv/devicearray.py#L120

If the array is C contiguous, as in most cases, this is not complying with the standard:
> The following are optional entries:
> - strides: None or (integer, ...)
>  A tuple of int (or long) representing the number of bytes to skip to access the next element at each dimension. If it is None, the array is assumed to be in C-contiguous layout.


Second, if the input is a 0-size array, the device pointer given by Numba's `__cuda_array_interface__['data'][0]` is `None`. But this leads to an error when accessing the pointer on the C/Cython side. For example, `PyLong_AsVoidPtr(None)` fails, and so it requires extra logic to handle this special case. In fact, this again breaks the spec, which requires an `int` or `long`:
> - data: (integer, boolean)
> The data is a 2-tuple. The first element data pointer as a Python int (or long). The data must be device-accessible. The second element is the read-only flag as a Python bool.

I suggest to revise the standard, as currently this edge case is not clearly defined. Relevant to #4140. By the way, CuPy uses `0` for this case (cupy/cupy#2104), which makes perfect sense at the C level.
Numba CUDA kernels fail when passing 0-size cupy arrays
## Reporting a bug

- [x] I am using the latest released version of Numba
- [x] I have included below a minimal working reproducer

When passing a `cupy` array of size 0 to a Numba-CUDA kernel, the kernel launch fails with
```
 Call to cuPointerGetAttribute results in CUDA_ERROR_INVALID_VALUE
```

Here is a minimal example:
```python
import cupy
from numba import cuda

@cuda.jit
def fill_with_zeros(x, N):
    i = cuda.grid(1)
    if i < N:
        x[i] = 0

def get_array_filled_with_zeros(N):
    x = cupy.ones(N)
    assert N<128  # kernel is launched with 128 threads
    fill_with_zeros[1,128](x, N)
    return x

if __name__ == '__main__':
    get_array_filled_with_zeros(10) # Passes
    get_array_filled_with_zeros(1)  # Passes
    get_array_filled_with_zeros(0)  # Fails
```

Numba CUDA kernels fail when passing 0-size cupy arrays
## Reporting a bug

- [x] I am using the latest released version of Numba
- [x] I have included below a minimal working reproducer

When passing a `cupy` array of size 0 to a Numba-CUDA kernel, the kernel launch fails with
```
 Call to cuPointerGetAttribute results in CUDA_ERROR_INVALID_VALUE
```

Here is a minimal example:
```python
import cupy
from numba import cuda

@cuda.jit
def fill_with_zeros(x, N):
    i = cuda.grid(1)
    if i < N:
        x[i] = 0

def get_array_filled_with_zeros(N):
    x = cupy.ones(N)
    assert N<128  # kernel is launched with 128 threads
    fill_with_zeros[1,128](x, N)
    return x

if __name__ == '__main__':
    get_array_filled_with_zeros(10) # Passes
    get_array_filled_with_zeros(1)  # Passes
    get_array_filled_with_zeros(0)  # Fails
```



Discussion:
I guess my ultimate question is how to tell whether a GPU array with `__cuda_array_interface__` is C-/F- contiguous or not, and having a non-default `'strides'` made it harder to tell. For example, it's impossible to do this:
```python
if 'strides' not in __cuda_array_interface__.keys() or __cuda_array_interface__['strides'] is None:
    # C contiguous, do something
else:
    # either F contiguous or not contiguous at all, do other thing
```
but maybe I am missing something obvious?  
Added a second issue on `__cuda_array_interface__['data'][0]`, see the updated post.
I agree with @leofang suggestion to revise the standard. I think the missing information on contiguity makes it unnecessarily harder and more error prone for developers to parse that information correctly. In that sense, I would suggest that `strides` becomes a mandatory field, as well as a new field on contiguity (or two field `c_contiguous` and `f_contiguous`, as NumPy implements it).

I haven't myself dealt the `None` pointer issue, but also agree to @leofang's comments on revising that.
Linking to my comment in Numba's Gitter channel: https://gitter.im/numba/numba?at=5d7ba9db5d40aa0d7d2d3bd9, copied and pasted below for reference:
> are you interested in an opinion on the spec, fixes, or both :)?

I suppose either or both. On the `strides` part, I think @pentschev's comment there about making it mandatory, plus adding one more flag to indicate any contiguity, would be very helpful.

On the other hand, I see that instead of having a contiguity flag, NumPy's `__array_interface__` uses `strides=None` to indicate C-contiguity. (This should be emphasized that even though the spec says strides is optional, NumPy always gives it (with value `None` for C).) If following the NumPy convention is intended, then Numba's spec is good, just need a code fix, and this fix must be propagated to, e.g., PyTorch, which currently also always explicitly gives strides like Numba does now, regardless of it's C-contiguous or not.

In any case, a `__cuda_array_interface__` consumer should only be in charge of parsing the information, not attempting to validate it; it's the producer's responsibility to provide accurate info and make parsing easy (thus it's good to have the spec modified and/or clarified).

On the pointer to zero-size array, Numba is not compliant, so a code fix is sufficient. Spec is good here.
Linking to my comment in Numba's Gitter channel: https://gitter.im/numba/numba?at=5d7baca65d40aa0d7d2d4e83, copied and pasted below for reference:

> On the pointer to zero-size array, Numba is not compliant, so a code fix is sufficient. Spec is good here.

Sorry, I was wrong. Spec should be modified for this corner case, and as I mentioned in the issue, following CuPy's approach and set the pointer to 0 would be nice.
@pentschev @stuartarchibald Let me know if this patch below looks good to you. If it's acceptable I'll write a few tests, modify the doc, and create a PR. Thanks.

**UPDATE**: `strides=None` is used as a flag to detect C-contiguity, same as in NumPy's `__array_interface__`.
```diff
diff --git a/numba/cuda/cudadrv/devicearray.py b/numba/cuda/cudadrv/devicearray.py
index cd1a027..3235387 100644
--- a/numba/cuda/cudadrv/devicearray.py
+++ b/numba/cuda/cudadrv/devicearray.py
@@ -116,10 +116,20 @@ class DeviceNDArrayBase(object):
 
     @property
     def __cuda_array_interface__(self):
+        if self.device_ctypes_pointer.value is not None:
+            ptr = self.device_ctypes_pointer.value
+        else:
+            ptr = 0
+
+        if array_core(self).flags['C_CONTIGUOUS']:
+            strides = None
+        else:
+            strides = tuple(self.strides)
+
         return {
             'shape': tuple(self.shape),
-            'strides': tuple(self.strides),
-            'data': (self.device_ctypes_pointer.value, False),
+            'strides': strides,
+            'data': (ptr, False),
             'typestr': self.dtype.str,
             'version': 1,
         }
```
The proposed patch seems reasonable to me, @sklam @seibert any view ?
Sorry for not replying earlier. Yes, I agree the patch looks good, I think this addresses our previous discussion.

@leofang thanks a lot for working on that!

> @leofang thanks a lot for working on that!

Yes indeed @leofang, thanks for your efforts on this, much appreciated.

This is not a bug of Numba. Rather, this is due to a known behavior of CuPy, see cupy/cupy/issues/2104. In short, if you allocate a 0-size array in CuPy like you did:
```python
x = cupy.ones(0)
```
the pointer to the underlying GPU memory is `NULL`:
```python
>>> x.data.ptr
0
>>> x.__cuda_array_interface__['data'][0]
0
```
and dereferencing it using `cuPointerGetAttribute` then leads to error. It may not be the best behavior we can hope for, but I'd suggest moving the discussion to the linked issue.
@RemiLehe thanks for using Numba and reporting this issue! @leofang thanks for looking into this and commenting on this with a resolution. Since this is a CuPy issue I will close this ticket. Please feel free to re-open in case it does turn out to involve Numba.
@leofang @esc Thanks a lot for letting me know about the corresponding `cupy` issue! I agree that discussion belongs there.
@esc @RemiLehe after working on supporting `__cuda_array_interface__` for mpi4py, I believe this is an issue on the Numba side, see #4175. Sorry that I didn't think this carefully earlier.
This is not a bug of Numba. Rather, this is due to a known behavior of CuPy, see cupy/cupy/issues/2104. In short, if you allocate a 0-size array in CuPy like you did:
```python
x = cupy.ones(0)
```
the pointer to the underlying GPU memory is `NULL`:
```python
>>> x.data.ptr
0
>>> x.__cuda_array_interface__['data'][0]
0
```
and dereferencing it using `cuPointerGetAttribute` then leads to error. It may not be the best behavior we can hope for, but I'd suggest moving the discussion to the linked issue.
@RemiLehe thanks for using Numba and reporting this issue! @leofang thanks for looking into this and commenting on this with a resolution. Since this is a CuPy issue I will close this ticket. Please feel free to re-open in case it does turn out to involve Numba.
@leofang @esc Thanks a lot for letting me know about the corresponding `cupy` issue! I agree that discussion belongs there.
@esc @RemiLehe after working on supporting `__cuda_array_interface__` for mpi4py, I believe this is an issue on the Numba side, see #4175. Sorry that I didn't think this carefully earlier.

Resolving patch:
diff --git a/numba/cuda/api.py b/numba/cuda/api.py
--- a/numba/cuda/api.py
+++ b/numba/cuda/api.py
@@ -55,7 +55,7 @@ def from_cuda_array_interface(desc, owner=None):
 
 def as_cuda_array(obj):
     """Create a DeviceNDArray from any object that implements
-    the cuda-array-interface.
+    the :ref:`cuda array interface <cuda-array-interface>`.
 
     A view of the underlying GPU buffer is created.  No copying of the data
     is done.  The resulting DeviceNDArray will acquire a reference from `obj`.
@@ -68,7 +68,7 @@ def as_cuda_array(obj):
 
 
 def is_cuda_array(obj):
-    """Test if the object has defined the `__cuda_array_interface__`.
+    """Test if the object has defined the `__cuda_array_interface__` attribute.
 
     Does not verify the validity of the interface.
     """
diff --git a/numba/cuda/cudadrv/devicearray.py b/numba/cuda/cudadrv/devicearray.py
--- a/numba/cuda/cudadrv/devicearray.py
+++ b/numba/cuda/cudadrv/devicearray.py
@@ -116,10 +116,20 @@ def __init__(self, shape, strides, dtype, stream=0, writeback=None,
 
     @property
     def __cuda_array_interface__(self):
+        if self.device_ctypes_pointer.value is not None:
+            ptr = self.device_ctypes_pointer.value
+        else:
+            ptr = 0
+
+        if array_core(self).flags['C_CONTIGUOUS']:
+            strides = None
+        else:
+            strides = tuple(self.strides)
+
         return {
             'shape': tuple(self.shape),
-            'strides': tuple(self.strides),
-            'data': (self.device_ctypes_pointer.value, False),
+            'strides': strides,
+            'data': (ptr, False),
             'typestr': self.dtype.str,
             'version': 1,
         }
diff --git a/numba/cuda/cudadrv/driver.py b/numba/cuda/cudadrv/driver.py
--- a/numba/cuda/cudadrv/driver.py
+++ b/numba/cuda/cudadrv/driver.py
@@ -1746,8 +1746,9 @@ def get_devptr_for_active_ctx(ptr):
     pointer.
     """
     devptr = c_void_p(0)
-    attr = enums.CU_POINTER_ATTRIBUTE_DEVICE_POINTER
-    driver.cuPointerGetAttribute(byref(devptr), attr, ptr)
+    if ptr != 0:
+        attr = enums.CU_POINTER_ATTRIBUTE_DEVICE_POINTER
+        driver.cuPointerGetAttribute(byref(devptr), attr, ptr)
     return devptr
 
 


Test patch:
diff --git a/numba/cuda/tests/cudapy/test_cuda_array_interface.py b/numba/cuda/tests/cudapy/test_cuda_array_interface.py
--- a/numba/cuda/tests/cudapy/test_cuda_array_interface.py
+++ b/numba/cuda/tests/cudapy/test_cuda_array_interface.py
@@ -220,6 +220,31 @@ def test_masked_array(self):
         expected_msg = 'Masked arrays are not supported'
         self.assertIn(expected_msg, str(raises.exception))
 
+    def test_zero_size_array(self):
+        # for #4175
+        c_arr = cuda.device_array(0)
+        self.assertEqual(c_arr.__cuda_array_interface__['data'][0], 0)
+
+        @cuda.jit
+        def add_one(arr):
+            x = cuda.grid(1)
+            N = arr.shape[0]
+            if x < N:
+                arr[x] += 1
+
+        d_arr = MyArray(c_arr)
+        add_one[1, 10](d_arr)  # this should pass
+
+    def test_strides(self):
+        # for #4175
+        # First, test C-contiguous array
+        c_arr = cuda.device_array((2, 3, 4))
+        self.assertEqual(c_arr.__cuda_array_interface__['strides'], None)
+
+        # Second, test non C-contiguous array
+        c_arr = c_arr[:, 1, :]
+        self.assertNotEqual(c_arr.__cuda_array_interface__['strides'], None)
+
 
 if __name__ == "__main__":
     unittest.main()