Repository: numpy/numpy

Issue:
Inexact result on floating-point remainder
This seems to be a regression in 11.0 or in master (note the values are exactly representable):

```
>>> np.remainder(3, 2.5)
0.49999999999999989
>>> np.__version__
'1.12.0.dev0+58263eb'
```

Compare to:

```
>>> np.remainder(3, 2.5)
0.5
>>> np.__version__
'1.10.4'
```



Discussion:
A comparison of the various approaches:

```
>>> a, b = 3, 2.5
>>> np.remainder(a, b)
0.49999999999999989
>>> b * (np.true_divide(a, b) - np.floor_divide(a, b))
0.49999999999999989
>>> a - (b * np.floor_divide(a, b))
0.5
```

I don't think 1 ulp counts as in issue in floating point. The new approach works better in general than the old.

I was a bit surprised here because the computation can be done exactly. But, well, agreed that it's not a major issue.
(for the record, CPython uses the platform `fmod`)

I think the problem is somewhat philosophical. You know that you meant 5 and 2.5 exactly, but the computer only knows that 5 and 2.5 are approximations of the true numbers, which may not be exactly represented. OTOH, I won't argue that further improvements can't be made. The original problem was to keep the results of floor division and remainder compatible, and when fmod (%) was used for the latter it was possible to have large inconsistencies. The current implementation guarantees (I think) that the remainder is always less in absolute value than the divisor and has the correct sign.

> the computer only knows that 5 and 2.5 are approximations of the true numbers, which may not be exactly represented.

As @pitrou has mentioned, those particular numbers are exactly represented; a rational number with a small enough numerator and with a denominator that is a small enough power of 2 is represented exactly in floating point.

@argriffing You missed the point. _All_ floating point numbers exactly represent themselves. 

For fun, also

```
In [27]: 2.5 + 0.49999999999999989
Out[27]: 3.0

In [28]: 2.5 + 0.5
Out[28]: 3.0
```

I agree that an insignificant error in np.remainder is a small price for having fixed the coordination issues among functions like divmod/div/truediv/floordiv/remainder/etc.

While a small error is definitely preferable over the wrong answers given before, I can see it being disconcerting that 

```
In [11]: np.remainder(6., 5.)
Out[11]: 0.99999999999999978
```

I'm slightly puzzled by the inconsistency with `%` and `np.fmod`:

```
In [12]: 6. % 5.
Out[12]: 1.0

In [15]: np.fmod(6., 5.)
Out[15]: 1.0
```

But in the end, it may just be a question of "pick your poison":

```
In [56]: a = 6.

In [57]: b = 5.

In [58]: b * (a / b - (a // b)) - a % b
Out[58]: -2.220446049250313e-16

In [59]: a = np.array(6.)

In [60]: b = np.array(5.)

In [61]: b * (a / b - (a // b)) - a % b
Out[61]: 0.0
```

As I was getting curious about what python itself does, I went ahead and looked for it -- not that easy to trace, but here it is: http://sources.debian.net/src/python2.7/2.7.11-3/Objects/floatobject.c/?hl=2083#L749

In the end, I think the premise is slightly different: python takes `fmod` as "god-given" and adjusts the floor division if necessary, while @charris takes floor division as "god-given" and uses it to calculate the remainder.

But it may be possible to remove the rounding problems, at the cost of an extra multiplication. Right now, we have (https://github.com/numpy/numpy/blob/master/numpy/core/src/umath/loops.c.src#L1710):

```
*((@type@ *)op1) = in2*(div - npy_floor@c@(div));
```

I think if we were to change this to

```
*((@type@ *)op1) = in2*div - in2*npy_floor@c@(div));
```

it would remove the "regression".  At least

```
In [107]: a = np.array(6.)

In [108]: b = np.array(5.)

In [109]: b * (a / b - a // b)
Out[109]: 0.99999999999999978

In [110]: b * (a / b) - b * (a // b)
Out[110]: 1.0
```

(I checked on the command line, but not with actual change to C code, that this did not immediately bring back the original problem.)

Interesting, that may be better. What I liked about the original is that the abs value of the result is guaranteed to be less than `abs(b)` (I think). That may also be true of your new form (not sure). I think the original gives the smallest floating solution of `(rem + b*floor(a/b)) = a`,  `0 <= sign(b)*rem < abs(b)`.

Hmm,

> the IEEE standard also specifies that square root, remainder, and conversion between integer and floating-point be correctly rounded. 

@mhvk I think that works.

> I'm slightly puzzled by the inconsistency with % and np.fmod

Turns out that they have different definitions in terms of how they are computed.

Well, turns out my computation doesn't work for negative numbers close to zero. The problem is that ieee floating point precision is symmetric around zero, but floor is not. Basically, the python `%` is not numerically good, whereas the more common computer language versions based on (symmetric) truncation rather than floor are. OTOH, for many things the floor version is more desirable. The problem is less severe at larger distance from zero as the asymmetry is less pronounced. Hmm...

@charris: if we're having doubts at the 11th hour, should we pull it back to master and wait for 1.12?

Don't forget that in 1.10 one gets answers that are _wrong_, not just slightly off!

@njsmith It's an interesting problem ;) The tiny negative number bit is inherent, so present in the python version also. For exactly represented numbers using few bits, the ideal computation is `a - b*floor(a/b)`, as floor should be correct and then all the rest of the arithmetic also. The problem I had with that is that sometimes the result would not have the right sign or be too big due to roundoff even though the errors were tiny. So all of these are fixable.


Resolving patch:
diff --git a/numpy/core/code_generators/ufunc_docstrings.py b/numpy/core/code_generators/ufunc_docstrings.py
--- a/numpy/core/code_generators/ufunc_docstrings.py
+++ b/numpy/core/code_generators/ufunc_docstrings.py
@@ -1225,8 +1225,10 @@ def add_newdoc(place, name, doc):
 
 add_newdoc('numpy.core.umath', 'floor_divide',
     """
-    Return the largest integer smaller or equal to the division of the
-    inputs.
+    Return the largest integer smaller or equal to the division of the inputs.
+    It is equivalent to the Python ``//`` operator and pairs with the
+    Python ``%`` (`remainder`), function so that ``b = a % b + b * (a // b)``
+    up to roundoff.
 
     Parameters
     ----------
@@ -1243,6 +1245,7 @@ def add_newdoc(place, name, doc):
 
     See Also
     --------
+    remainder : Remainder complementary to floor_divide.
     divide : Standard division.
     floor : Round a number to the nearest integer toward minus infinity.
     ceil : Round a number to the nearest integer toward infinity.
@@ -2689,9 +2692,9 @@ def add_newdoc(place, name, doc):
     """
     Return element-wise remainder of division.
 
-    Computes ``x1 - floor(x1 / x2) * x2``, the result has the same sign as
-    the divisor `x2`. It is equivalent to the Python modulus operator
-    ``x1 % x2`` and should not be confused with the Matlab(TM) ``rem``
+    Computes the remainder complementary to the `floor_divide` function.  It is
+    equivalent to the Python modulus operator``x1 % x2`` and has the same sign
+    as the divisor `x2`. It should not be confused with the Matlab(TM) ``rem``
     function.
 
     Parameters
@@ -2707,11 +2710,12 @@ def add_newdoc(place, name, doc):
     Returns
     -------
     y : ndarray
-        The remainder of the quotient ``x1/x2``, element-wise. Returns a
-        scalar if both  `x1` and `x2` are scalars.
+        The element-wise remainder of the quotient ``floor_divide(x1, x2)``.
+        Returns a scalar if both  `x1` and `x2` are scalars.
 
     See Also
     --------
+    floor_divide : Equivalent of Python ``//`` operator.
     fmod : Equivalent of the Matlab(TM) ``rem`` function.
     divide, floor
 
diff --git a/numpy/core/setup.py b/numpy/core/setup.py
--- a/numpy/core/setup.py
+++ b/numpy/core/setup.py
@@ -896,6 +896,8 @@ def generate_umath_c(ext, build_dir):
 
     umath_deps = [
             generate_umath_py,
+            join('include', 'numpy', 'npy_math.h'),
+            join('include', 'numpy', 'halffloat.h'),
             join('src', 'multiarray', 'common.h'),
             join('src', 'private', 'templ_common.h.src'),
             join('src', 'umath', 'simd.inc.src'),


Test patch:
diff --git a/numpy/core/tests/test_multiarray.py b/numpy/core/tests/test_multiarray.py
--- a/numpy/core/tests/test_multiarray.py
+++ b/numpy/core/tests/test_multiarray.py
@@ -2448,42 +2448,6 @@ def test_conjugate(self):
         assert_raises(AttributeError, lambda: a.conj())
         assert_raises(AttributeError, lambda: a.conjugate())
 
-    def test_divmod_basic(self):
-        dt = np.typecodes['AllInteger'] + np.typecodes['Float']
-        for dt1, dt2 in itertools.product(dt, dt):
-            for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
-                if sg1 == -1 and dt1 in np.typecodes['UnsignedInteger']:
-                    continue
-                if sg2 == -1 and dt2 in np.typecodes['UnsignedInteger']:
-                    continue
-                fmt = 'dt1: %s, dt2: %s, sg1: %s, sg2: %s'
-                msg = fmt % (dt1, dt2, sg1, sg2)
-                a = np.array(sg1*71, dtype=dt1)
-                b = np.array(sg2*19, dtype=dt2)
-                div, rem = divmod(a, b)
-                assert_allclose(div*b + rem, a, err_msg=msg)
-                if sg2 == -1:
-                    assert_(b < rem <= 0, msg)
-                else:
-                    assert_(b > rem >= 0, msg)
-
-    def test_divmod_roundoff(self):
-        # gh-6127
-        dt = 'fdg'
-        for dt1, dt2 in itertools.product(dt, dt):
-            for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
-                fmt = 'dt1: %s, dt2: %s, sg1: %s, sg2: %s'
-                msg = fmt % (dt1, dt2, sg1, sg2)
-                a = np.array(sg1*78*6e-8, dtype=dt1)
-                b = np.array(sg2*6e-8, dtype=dt2)
-                div, rem = divmod(a, b)
-                assert_allclose(div*b + rem, a, err_msg=msg)
-                if sg2 == -1:
-                    assert_(b < rem <= 0, msg)
-                else:
-                    assert_(b > rem >= 0, msg)
-
-
 class TestBinop(object):
     def test_inplace(self):
         # test refcount 1 inplace conversion
diff --git a/numpy/core/tests/test_scalarmath.py b/numpy/core/tests/test_scalarmath.py
--- a/numpy/core/tests/test_scalarmath.py
+++ b/numpy/core/tests/test_scalarmath.py
@@ -2,6 +2,8 @@
 
 import sys
 import itertools
+import warnings
+import operator
 
 import numpy as np
 from numpy.testing.utils import _gen_alignment_data
@@ -137,8 +139,12 @@ def test_mixed_types(self):
                     assert_almost_equal(result, 9, err_msg=msg)
 
 
-class TestDivmod(TestCase):
-    def test_divmod_basic(self):
+class TestModulus(TestCase):
+
+    floordiv = operator.floordiv
+    mod = operator.mod
+
+    def test_modulus_basic(self):
         dt = np.typecodes['AllInteger'] + np.typecodes['Float']
         for dt1, dt2 in itertools.product(dt, dt):
             for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
@@ -150,29 +156,88 @@ def test_divmod_basic(self):
                 msg = fmt % (dt1, dt2, sg1, sg2)
                 a = np.array(sg1*71, dtype=dt1)[()]
                 b = np.array(sg2*19, dtype=dt2)[()]
-                div, rem = divmod(a, b)
-                assert_allclose(div*b + rem, a, err_msg=msg)
+                div = self.floordiv(a, b)
+                rem = self.mod(a, b)
+                assert_equal(div*b + rem, a, err_msg=msg)
                 if sg2 == -1:
                     assert_(b < rem <= 0, msg)
                 else:
                     assert_(b > rem >= 0, msg)
 
-    def test_divmod_roundoff(self):
+    def test_float_modulus_exact(self):
+        # test that float results are exact for small integers. This also
+        # holds for the same integers scaled by powers of two.
+        nlst = list(range(-127, 0))
+        plst = list(range(1, 128))
+        dividend = nlst + [0] + plst
+        divisor = nlst + plst
+        arg = list(itertools.product(dividend, divisor))
+        tgt = list(divmod(*t) for t in arg)
+
+        a, b = np.array(arg, dtype=int).T
+        # convert exact integer results from Python to float so that
+        # signed zero can be used, it is checked.
+        tgtdiv, tgtrem = np.array(tgt, dtype=float).T
+        tgtdiv = np.where((tgtdiv == 0.0) & ((b < 0) ^ (a < 0)), -0.0, tgtdiv)
+        tgtrem = np.where((tgtrem == 0.0) & (b < 0), -0.0, tgtrem)
+
+        for dt in np.typecodes['Float']:
+            msg = 'dtype: %s' % (dt,)
+            fa = a.astype(dt)
+            fb = b.astype(dt)
+            # use list comprehension so a_ and b_ are scalars
+            div = [self.floordiv(a_, b_) for  a_, b_ in zip(fa, fb)]
+            rem = [self.mod(a_, b_) for a_, b_ in zip(fa, fb)]
+            assert_equal(div, tgtdiv, err_msg=msg)
+            assert_equal(rem, tgtrem, err_msg=msg)
+
+    def test_float_modulus_roundoff(self):
         # gh-6127
-        dt = 'fdg'
+        dt = np.typecodes['Float']
         for dt1, dt2 in itertools.product(dt, dt):
             for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
                 fmt = 'dt1: %s, dt2: %s, sg1: %s, sg2: %s'
                 msg = fmt % (dt1, dt2, sg1, sg2)
                 a = np.array(sg1*78*6e-8, dtype=dt1)[()]
                 b = np.array(sg2*6e-8, dtype=dt2)[()]
-                div, rem = divmod(a, b)
-                assert_allclose(div*b + rem, a, err_msg=msg)
+                div = self.floordiv(a, b)
+                rem = self.mod(a, b)
+                # Equal assertion should hold when fmod is used
+                assert_equal(div*b + rem, a, err_msg=msg)
                 if sg2 == -1:
                     assert_(b < rem <= 0, msg)
                 else:
                     assert_(b > rem >= 0, msg)
 
+    def test_float_modulus_corner_cases(self):
+        # Check remainder magnitude.
+        for dt in np.typecodes['Float']:
+            b = np.array(1.0, dtype=dt)
+            a = np.nextafter(np.array(0.0, dtype=dt), -b)
+            rem = self.mod(a, b)
+            assert_(rem <= b, 'dt: %s' % dt)
+            rem = self.mod(-a, -b)
+            assert_(rem >= -b, 'dt: %s' % dt)
+
+        # Check nans, inf
+        with warnings.catch_warnings():
+            warnings.simplefilter('always')
+            warnings.simplefilter('ignore', RuntimeWarning)
+            for dt in np.typecodes['Float']:
+                fone = np.array(1.0, dtype=dt)
+                fzer = np.array(0.0, dtype=dt)
+                finf = np.array(np.inf, dtype=dt)
+                fnan = np.array(np.nan, dtype=dt)
+                rem = self.mod(fone, fzer)
+                assert_(np.isnan(rem), 'dt: %s' % dt)
+                # MSVC 2008 returns NaN here, so disable the check.
+                #rem = self.mod(fone, finf)
+                #assert_(rem == fone, 'dt: %s' % dt)
+                rem = self.mod(fone, fnan)
+                assert_(np.isnan(rem), 'dt: %s' % dt)
+                rem = self.mod(finf, fone)
+                assert_(np.isnan(rem), 'dt: %s' % dt)
+
 
 class TestComplexDivision(TestCase):
     def test_zero_division(self):
diff --git a/numpy/core/tests/test_umath.py b/numpy/core/tests/test_umath.py
--- a/numpy/core/tests/test_umath.py
+++ b/numpy/core/tests/test_umath.py
@@ -3,6 +3,7 @@
 import sys
 import platform
 import warnings
+import itertools
 
 from numpy.testing.utils import _gen_alignment_data
 import numpy.core.umath as ncu
@@ -222,6 +223,102 @@ def test_floor_division_complex(self):
         assert_equal(y, [1.e+110, 0], err_msg=msg)
 
 
+class TestRemainder(TestCase):
+
+    def test_remainder_basic(self):
+        dt = np.typecodes['AllInteger'] + np.typecodes['Float']
+        for dt1, dt2 in itertools.product(dt, dt):
+            for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
+                if sg1 == -1 and dt1 in np.typecodes['UnsignedInteger']:
+                    continue
+                if sg2 == -1 and dt2 in np.typecodes['UnsignedInteger']:
+                    continue
+                fmt = 'dt1: %s, dt2: %s, sg1: %s, sg2: %s'
+                msg = fmt % (dt1, dt2, sg1, sg2)
+                a = np.array(sg1*71, dtype=dt1)
+                b = np.array(sg2*19, dtype=dt2)
+                div = np.floor_divide(a, b)
+                rem = np.remainder(a, b)
+                assert_equal(div*b + rem, a, err_msg=msg)
+                if sg2 == -1:
+                    assert_(b < rem <= 0, msg)
+                else:
+                    assert_(b > rem >= 0, msg)
+
+    def test_float_remainder_exact(self):
+        # test that float results are exact for small integers. This also
+        # holds for the same integers scaled by powers of two.
+        nlst = list(range(-127, 0))
+        plst = list(range(1, 128))
+        dividend = nlst + [0] + plst
+        divisor = nlst + plst
+        arg = list(itertools.product(dividend, divisor))
+        tgt = list(divmod(*t) for t in arg)
+
+        a, b = np.array(arg, dtype=int).T
+        # convert exact integer results from Python to float so that
+        # signed zero can be used, it is checked.
+        tgtdiv, tgtrem = np.array(tgt, dtype=float).T
+        tgtdiv = np.where((tgtdiv == 0.0) & ((b < 0) ^ (a < 0)), -0.0, tgtdiv)
+        tgtrem = np.where((tgtrem == 0.0) & (b < 0), -0.0, tgtrem)
+
+        for dt in np.typecodes['Float']:
+            msg = 'dtype: %s' % (dt,)
+            fa = a.astype(dt)
+            fb = b.astype(dt)
+            div = np.floor_divide(fa, fb)
+            rem = np.remainder(fa, fb)
+            assert_equal(div, tgtdiv, err_msg=msg)
+            assert_equal(rem, tgtrem, err_msg=msg)
+
+    def test_float_remainder_roundoff(self):
+        # gh-6127
+        dt = np.typecodes['Float']
+        for dt1, dt2 in itertools.product(dt, dt):
+            for sg1, sg2 in itertools.product((+1, -1), (+1, -1)):
+                fmt = 'dt1: %s, dt2: %s, sg1: %s, sg2: %s'
+                msg = fmt % (dt1, dt2, sg1, sg2)
+                a = np.array(sg1*78*6e-8, dtype=dt1)
+                b = np.array(sg2*6e-8, dtype=dt2)
+                div = np.floor_divide(a, b)
+                rem = np.remainder(a, b)
+                # Equal assertion should hold when fmod is used
+                assert_equal(div*b + rem, a, err_msg=msg)
+                if sg2 == -1:
+                    assert_(b < rem <= 0, msg)
+                else:
+                    assert_(b > rem >= 0, msg)
+
+    def test_float_remainder_corner_cases(self):
+        # Check remainder magnitude.
+        for dt in np.typecodes['Float']:
+            b = np.array(1.0, dtype=dt)
+            a = np.nextafter(np.array(0.0, dtype=dt), -b)
+            rem = np.remainder(a, b)
+            assert_(rem <= b, 'dt: %s' % dt)
+            rem = np.remainder(-a, -b)
+            assert_(rem >= -b, 'dt: %s' % dt)
+
+        # Check nans, inf
+        with warnings.catch_warnings():
+            warnings.simplefilter('always')
+            warnings.simplefilter('ignore', RuntimeWarning)
+            for dt in np.typecodes['Float']:
+                fone = np.array(1.0, dtype=dt)
+                fzer = np.array(0.0, dtype=dt)
+                finf = np.array(np.inf, dtype=dt)
+                fnan = np.array(np.nan, dtype=dt)
+                rem = np.remainder(fone, fzer)
+                assert_(np.isnan(rem), 'dt: %s, rem: %s' % (dt, rem))
+                # MSVC 2008 returns NaN here, so disable the check.
+                #rem = np.remainder(fone, finf)
+                #assert_(rem == fone, 'dt: %s, rem: %s' % (dt, rem))
+                rem = np.remainder(fone, fnan)
+                assert_(np.isnan(rem), 'dt: %s, rem: %s' % (dt, rem))
+                rem = np.remainder(finf, fone)
+                assert_(np.isnan(rem), 'dt: %s, rem: %s' % (dt, rem))
+
+
 class TestCbrt(TestCase):
     def test_cbrt_scalar(self):
         assert_almost_equal((np.cbrt(np.float32(-2.5)**3)), -2.5)