Repository: cupy/cupy

Issue:
Unexpected NaN when using big-endian arrays
When a big-endian array is loaded on the GPU using cp.array(), random NaNs appear in the data and calculations will start returning NaN. No errors or warnings are given to the user.


### Conditions
CuPy Version          : 7.6.0
CUDA Root             : /usr/local/cuda
CUDA Build Version    : 9010
CUDA Driver Version   : 10010
CUDA Runtime Version  : 9010
cuBLAS Version        : 9010
cuFFT Version         : 9010
cuRAND Version        : 9010
cuSOLVER Version      : (9, 1, 0)
cuSPARSE Version      : 9010
NVRTC Version         : (9, 1)
cuDNN Build Version   : 7102
cuDNN Version         : 7102
NCCL Build Version    : 2115
NCCL Runtime Version  : (unknown)
CUB Version           : None
cuTENSOR Version      : None

### Code to reproduce
```import cupy as cp
import numpy as np

data = np.arange(1000*1000, dtype='>f4')/1e9
print(' numpy:', type(data), data.shape, data.dtype)
print('   nan:', np.where(np.isnan(data)))
print(' total:', np.sum(data))

arr = cp.array(data)
print('-----')
print('  cupy:' , type(arr), arr.shape, arr.dtype)
print('   nan:' , cp.where(cp.isnan(arr)))
print(' total:', cp.sum(arr))`
```

### Output of the above code:

```
 numpy: <class 'numpy.ndarray'> (1000000,) >f4
   nan: (array([], dtype=int64),)
 total: 499.99963
-----
  cupy: <class 'cupy.core.core.ndarray'> (1000000,) >f4
   nan: (array([   213,    385,    426, ..., 999227, 999242, 999391]),)
 total: nan
```

The numpy array shows no NaNs as expected, while the cupy array on the GPU shows several NaNs and functions like sum() that operate on the whole array return NaN as well.

### Scenarios

A fairly common occurence in a scientific environment is when readings FITS files, which store data big-endian, for example using the astropy.io module:

```
import cupy as cp
from astropy.io import fits

data = fits.getdata(filename)
gpu_data = cp.array(data)        # Results in NaN
```

A workaround is to convert the array to little endian before using it with cupy:

`data = data.astype(np.float32)`




Discussion:
According to the CUDA Programming Guide:
> The NVIDIA GPU architecture uses a little-endian representation.

https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#hardware-implementation
True, but numpy supports little- and big-endianess completely transparently, and I didn't even know that I was trying to load a big-endian array. It took some hours of debugging to figure it out.

I think that in such a situation cupy should either:
- give an error; or
- automatically convert the dtype while moving the array to/from the device

currently, it takes a perfectly valid numpy array, and it loads wrong data on the device. I think that it is a bug.

Found a similar issue: #2744 ndarray.dot produces wrong values with non-native endian arrays
For the record I am no decision maker, so don't take my words too seriously. I think it's long weekend in Japan, so someone from PFN would eventually reply. 

Part of me also agrees it's a bug. We need to signal the incompatibility as it's very hard to debug as you said. However, another part of me wonders how you ended up with a big-endian array (in a real code, not a reproducer) given that most of the mainstream CPU architectures are little endian too (which was why CUDA picks it). You need to do very naughty things to get there, don't you?
I would never create big-endian arrays on purpose, but it can happen if you use third party libraries that produce numpy arrays, like the astropy example I gave in the original report. 

I triggered the bug just reading files from disk using astropy and loading them on the GPU, three lines of code. Nothing naughty :)  Since FITS files are very common where I work, I now need to wrap the cupy loading code with some checks, otherwise it will happen all the time.


> but it can happen if you use third party libraries that produce numpy arrays, like the astropy example I gave in the original report.

Sorry, @alfiopuglisi, I overlooked your astropy example. So I took a quick search, and think they're the ones being naughty 😂 It's been an issue repeating over the years of astropy (which is an impressive package, although I never was a user of it), and they still think it's OK to live with...Wow.

A few refs I found:
- https://github.com/astropy/astropy/issues/1156
- https://github.com/astropy/astropy/issues/4069
- https://github.com/astropy/astropy/pull/3729
- https://github.com/astropy/astropy/pull/4080

You might be able to call `Table.as_array()` as a workaround before CuPy core devs figure out what to do?
When converting a NumPy array into a CuPy array we automatically make array contiguous (C by default). So I think it is also reasonable to automatically convert an array into big endian before sending it to GPU.
Weighing in to report I had a similar issue. (I was trying to use multidimensional fancy indexing to turn images into vectors and back.)

The right behavior here is, IMO, transparent conversion to little-endian in the host-to-device copy. At any rate, this _can't_ be right...

<img width="565" alt="image" src="https://user-images.githubusercontent.com/31168267/95286219-990d7c80-0817-11eb-86c3-7a2e949e83d7.png">

For what it's worth, I don't think Astropy will change, since using the same byte-order as the FITS standard specifies lets them do memory mapped array access easily.

(P.S. Hi @alfiopuglisi ! I'm using CuPy to reduce data from MagAO! Fancy running in to one of its creators here...)
Here is the workaround that I have implemented in my code: it patches the cupy array and asarray functions to convert the data on-the-fly. It's a hack but it works for me, maybe it will be useful for others too:

```py
def _make_sure_first_arg_is_little_endian(f):
    '''
    Decorator to enforce little endianess with cupy.

    Analyze the first argument and, if it is
    a big-endian array (like the ones returned by astropy.io.fits.getdata),
    convert it to a little-endian one before passing it to the function.

    Can be used to "patch" the cupy routines as follows::
        cupy.array = _make_sure_first_arg_is_little_endian(cupy.array)
        cupy.asarray = _make_sure_first_arg_is_little_endian(cupy.asarray)
    '''
    @wraps(f)
    def wrapper(data, *args, **kwargs):
        import sys
        import numpy

        if isinstance(data, numpy.ndarray):
            # Use names compatible with sys.byteorder
            endianess_map = {
                '>': 'big',
                '<': 'little',
                '=': sys.byteorder,
                '|': 'not relevant'
            }
            endianess = endianess_map[data.dtype.byteorder]
            if endianess == 'big':
               print('Array %s %s converted from big to little endian' %
                     (str(data.dtype), str(data.shape)))
               data = data.astype(data.dtype.newbyteorder('<'))

        return f(data, *args, **kwargs)

    return wrapper


def cupy_patch(cupy):
    '''
    Patch the passed cupy module with our workarounds for FITS files.
    '''
    cupy.array = _make_sure_first_arg_is_little_endian(cupy.array)
    cupy.asarray = _make_sure_first_arg_is_little_endian(cupy.asarray)    

```

PS: Hi @joseph-long! MagAO was a lot of fun, happy to see that you are working with the data!


Resolving patch:
diff --git a/cupy/_creation/from_data.py b/cupy/_creation/from_data.py
--- a/cupy/_creation/from_data.py
+++ b/cupy/_creation/from_data.py
@@ -35,6 +35,11 @@ def array(obj, dtype=None, copy=True, order='K', subok=False, ndmin=0):
     .. note::
        This method currently does not support ``subok`` argument.
 
+    .. note::
+       If ``obj`` is an `numpy.ndarray` instance that contains big-endian data,
+       this function automatically swaps its byte order to little-endian,
+       which is the NVIDIA and AMD GPU architecture's native use.
+
     .. seealso:: :func:`numpy.array`
 
     """
@@ -60,6 +65,11 @@ def asarray(a, dtype=None, order=None):
         cupy.ndarray: An array on the current device. If ``a`` is already on
         the device, no copy is performed.
 
+    .. note::
+       If ``a`` is an `numpy.ndarray` instance that contains big-endian data,
+       this function automatically swaps its byte order to little-endian,
+       which is the NVIDIA and AMD GPU architecture's native use.
+
     .. seealso:: :func:`numpy.asarray`
 
     """
@@ -143,6 +153,11 @@ def fromfile(*args, **kwargs):
     .. note::
         Uses NumPy's ``fromfile`` and coerces the result to a CuPy array.
 
+    .. note::
+       If you let NumPy's ``fromfile`` read the file in big-endian, CuPy
+       automatically swaps its byte order to little-endian, which is the NVIDIA
+       and AMD GPU architecture's native use.
+
     .. seealso:: :func:`numpy.fromfile`
 
     """


Test patch:
diff --git a/tests/cupy_tests/creation_tests/test_from_data.py b/tests/cupy_tests/creation_tests/test_from_data.py
--- a/tests/cupy_tests/creation_tests/test_from_data.py
+++ b/tests/cupy_tests/creation_tests/test_from_data.py
@@ -9,7 +9,6 @@
 import numpy
 
 
-@testing.gpu
 class TestFromData(unittest.TestCase):
 
     @testing.for_orders('CFAK')
@@ -319,6 +318,26 @@ def test_array_no_copy_ndmin(self, xp, dtype):
         a.fill(0)
         return b
 
+    @testing.for_all_dtypes()
+    @testing.numpy_cupy_array_equal()
+    def test_array_from_big_endian(self, xp, dtype):
+        dtype = numpy.dtype(dtype).newbyteorder('>')
+        a = testing.shaped_arange((2, 3, 4), numpy, dtype)
+        b = xp.array(a)
+        # Make a computation here as just moving big-endian data back and forth
+        # happens to work before the change in #5828
+        return b + b
+
+    @testing.for_all_dtypes()
+    @testing.numpy_cupy_array_equal()
+    def test_array_from_list_of_numpy_big_endian(self, xp, dtype):
+        dtype = numpy.dtype(dtype).newbyteorder('>')
+        a = [testing.shaped_arange((3, 4), numpy, dtype) for i in range(2)]
+        b = xp.array(a)
+        # Make a computation here as just moving big-endian data back and forth
+        # happens to work before the change in #5828
+        return b + b
+
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
     def test_asarray(self, xp, dtype):
@@ -367,6 +386,16 @@ def test_asanyarray_with_order(self, xp, dtype, order):
             assert b.flags.c_contiguous
         return b
 
+    @testing.for_all_dtypes()
+    @testing.numpy_cupy_array_equal()
+    def test_asanyarray_from_big_endian(self, xp, dtype):
+        dtype = numpy.dtype(dtype).newbyteorder('>')
+        a = testing.shaped_arange((2, 3, 4), numpy, dtype)
+        b = xp.asanyarray(a)
+        # Make a computation here as just moving big-endian data back and forth
+        # happens to work before the change in #5828
+        return b + b
+
     @testing.for_CF_orders()
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -410,6 +439,16 @@ def test_asarray_cuda_array_zero_dim_dtype(self, xp):
         a = xp.ones((), dtype=numpy.float64)
         return xp.ascontiguousarray(a, dtype=numpy.int64)
 
+    @testing.for_all_dtypes()
+    @testing.numpy_cupy_array_equal()
+    def test_asarray_from_big_endian(self, xp, dtype):
+        dtype = numpy.dtype(dtype).newbyteorder('>')
+        a = testing.shaped_arange((2, 3, 4), numpy, dtype)
+        b = xp.asarray(a)
+        # Make a computation here as just moving big-endian data back and forth
+        # happens to work before the change in #5828
+        return b + b
+
     @testing.for_CF_orders()
     @testing.for_all_dtypes()
     @testing.numpy_cupy_array_equal()
@@ -457,11 +496,21 @@ def test_fromfile(self, xp):
             fh.seek(0)
             return xp.fromfile(fh, dtype="u1")
 
+    @testing.numpy_cupy_array_equal()
+    def test_fromfile_big_endian(self, xp):
+        with tempfile.TemporaryFile() as fh:
+            fh.write(b"\x00\x00\x00\x01")
+            fh.flush()
+            fh.seek(0)
+            a = xp.fromfile(fh, dtype='>u4')
+            # Make a computation here as just moving big-endian data back and
+            # forth happens to work before the change in #5828
+            return a + a
+
 
 max_cuda_array_interface_version = 3
 
 
-@testing.gpu
 @testing.parameterize(*testing.product({
     'ver': tuple(range(max_cuda_array_interface_version+1)),
     'strides': (False, None, True),
@@ -519,8 +568,19 @@ def test_asnumpy(self, dtype):
         b_cpu = cupy.asnumpy(b)
         testing.assert_array_equal(a_cpu, b_cpu)
 
+    def test_big_endian(self):
+        a = cupy.array([0x1, 0x0, 0x0, 0x0], dtype=numpy.int8)
+        dtype = numpy.dtype('>i4')
+        shape = 1,
+        strides = 4,
+        data = a.data.ptr
+        b = DummyObjectWithCudaArrayInterface(
+            (shape, strides, dtype.str, dtype.descr, data),
+            self.ver, self.strides)
+        with pytest.raises(ValueError):
+            cupy.asarray(b)
+
 
-@testing.gpu
 @testing.parameterize(*testing.product({
     'ver': tuple(range(1, max_cuda_array_interface_version+1)),
     'strides': (False, None, True),
@@ -541,7 +601,6 @@ def test_masked_array(self, dtype):
 
 # marked slow as either numpy or cupy could go OOM in this test
 @testing.slow
-@testing.gpu
 @pytest.mark.skipif(
     cupy.cuda.runtime.is_hip, reason='HIP does not support this')
 class TestCudaArrayInterfaceBigArray(unittest.TestCase):
@@ -558,7 +617,11 @@ def test_with_over_size_array(self):
 class DummyObjectWithCudaArrayInterface(object):
     def __init__(self, a, ver, include_strides=False, mask=None, stream=None):
         assert ver in tuple(range(max_cuda_array_interface_version+1))
-        self.a = a
+        self.a = None
+        if isinstance(a, cupy.ndarray):
+            self.a = a
+        else:
+            self.shape, self.strides, self.typestr, self.descr, self.data = a
         self.ver = ver
         self.include_strides = include_strides
         self.mask = mask
@@ -566,22 +629,37 @@ def __init__(self, a, ver, include_strides=False, mask=None, stream=None):
 
     @property
     def __cuda_array_interface__(self):
-        desc = {
-            'shape': self.a.shape,
-            'typestr': self.a.dtype.str,
-            'descr': self.a.dtype.descr,
-            'data': (self.a.data.ptr, False),
-            'version': self.ver,
-        }
-        if self.a.flags.c_contiguous:
-            if self.include_strides is True:
+        if self.a is not None:
+            desc = {
+                'shape': self.a.shape,
+                'typestr': self.a.dtype.str,
+                'descr': self.a.dtype.descr,
+                'data': (self.a.data.ptr, False),
+                'version': self.ver,
+            }
+            if self.a.flags.c_contiguous:
+                if self.include_strides is True:
+                    desc['strides'] = self.a.strides
+                elif self.include_strides is None:
+                    desc['strides'] = None
+                else:  # self.include_strides is False
+                    pass
+            else:  # F contiguous or neither
                 desc['strides'] = self.a.strides
+        else:
+            desc = {
+                'shape': self.shape,
+                'typestr': self.typestr,
+                'descr': self.descr,
+                'data': (self.data, False),
+                'version': self.ver,
+            }
+            if self.include_strides is True:
+                desc['strides'] = self.strides
             elif self.include_strides is None:
                 desc['strides'] = None
             else:  # self.include_strides is False
                 pass
-        else:  # F contiguous or neither
-            desc['strides'] = self.a.strides
         if self.mask is not None:
             desc['mask'] = self.mask
         # The stream field is kept here for compliance. However, since the