Repository: cupy/cupy

Issue:
`cupy.nextafter` is wrong on a=0
`test_nextafter_combination` is failing.

`{'dtype_b': <type 'numpy.float32'>, 'dtype_a': <type 'numpy.bool_'>}`
```
x          = array([[1.1754944e-38, 1.0000001e+00, 1.1754944e-38],
       [1.0000001e+00, 1.1754944e-38, 1.0000000e+00]], dtype=float32)
y          = array([[1.4012985e-45, 1.0000001e+00, 1.4012985e-45],
       [1.0000001e+00, 1.4012985e-45, 1.0000000e+00]], dtype=float32)
```


Discussion:
This bug may be caused by [`-ftz=true`](https://github.com/cupy/cupy/blob/04fb189ebc8cc7e4709fb505808fc084de0fccdf/cupy/cuda/compiler.py#L129).
I agree with @gwtnb, but for me the behaviour of CuPy is correct. It seems what's wrong is the test.

The number ~1e-38 is the minimum positive normalized IEEE754 fp32 number. (It's `numpy.finfo(numpy.float32).tiny` as well as `FLT_MIN` in C on most of modern platforms.) The other number ~1e-45 is the denormalized one.

It seems the best option is to create an equality test that equates a positive (resp. negative) denormalized number on CPU with `0.0` or `FLT_MIN` (resp. `-0.0` or `-FLT_MIN`) on GPU.

Obviously you can't flush denomrlized numbers to 0 for this case. But flushing denormalized numbers to `FLT_MIN` is also wrong, because `xp.nextafter(xp.array([1.1754944e-38], dtype='float32'), xp.zeros(1, dtype='float32'))` is non-zero with Numpy but zero with CuPy. Manually setting rounding mode of a CPU is not portable.

As CuPy sets `-ftz=true`, its behaviour is reasonable. NumPy is also correct, because most of modern CPUs handle denormalized numbers by default. There is a similar discussion at StackExcahge. https://scicomp.stackexchange.com/questions/23191/implementing-stdnextafter-should-denormals-are-zero-mode-affect-it-if-so-ho


Resolving patch:
diff --git a/cupy/math/floating.py b/cupy/math/floating.py
--- a/cupy/math/floating.py
+++ b/cupy/math/floating.py
@@ -50,6 +50,11 @@
     'nextafter', 2, 'cupy_nextafter',
     '''Computes the nearest neighbor float values towards the second argument.
 
+    .. note::
+        For values that are close to zero (or denormal numbers),
+        results of :func:`cupy.nextafter` may be different from those of
+        :func:`numpy.nextafter`, because CuPy sets ``-ftz=true``.
+
     .. seealso:: :data:`numpy.nextafter`
 
     ''')


Test patch:
diff --git a/tests/cupy_tests/math_tests/test_floating.py b/tests/cupy_tests/math_tests/test_floating.py
--- a/tests/cupy_tests/math_tests/test_floating.py
+++ b/tests/cupy_tests/math_tests/test_floating.py
@@ -51,18 +51,18 @@ def test_frexp(self, dtype):
         testing.assert_array_equal(cupy_c, numpy_c)
 
     @testing.for_all_dtypes_combination(
-        # TODO(kataoka): Revert skipping bool
-        ('dtype_a', 'dtype_b'), no_complex=True, no_bool=True)
+        ('dtype_a', 'dtype_b'), no_complex=True)
     @testing.numpy_cupy_array_equal()
     def test_nextafter_combination(self, xp, dtype_a, dtype_b):
         a = testing.shaped_arange((2, 3), xp, dtype_a)
+        # skip 0 because cupy (may) handle denormals differently (-ftz=true)
+        a[a == 0] = 1
         b = testing.shaped_reverse_arange((2, 3), xp, dtype_b)
         return xp.nextafter(a, b)
 
     @testing.for_float_dtypes()
     @testing.numpy_cupy_array_equal()
     def test_nextafter_float(self, xp, dtype):
-        # TODO(kataoka): test 0
         a = xp.array([-5, -3, 3, 5], dtype=dtype)[:, None]
         b = xp.array([-xp.inf, -4, 0, 4, xp.inf], dtype=dtype)[None, :]
         return xp.nextafter(a, b)