Repository: numba/numba

Issue:
numpy.random.binomial fails
numpy.random.binomial(n, p) does not complete the task when 'n' is large. For example, numpy.random.binomial(1600, 0.4) will not finish. Here is a trial script:

import numpy as np
from numba import jit

@jit
def testBinom(n, d):
    """
    random binomial test
    """
    for i in range(n):
        d_i = d[i]
        e = np.random.binomial(d_i, .4)
        print(d_i, e)
    return e

d = np.arange(1000, 1600, dtype = int)
n = len(d)
e = testBinom(n, d)
print('e', e)



Discussion:
This script seems to work fine on my system using an almost current version of Numba master.   What version of Numba are you using?

(Note that this example code will be compiled in object mode because of the `print` call.)

On my mac, using the latest release:

``` python
In [1]: import numpy as np

In [2]: import numba as nb

In [3]: @nb.jit
def foo(n, p):
    return np.random.binomial(n, p)
   ...:

In [4]: foo(1458, 0.4)
Out[4]: 564

In [5]: foo(1459, 0.4)
<stalls>
```

Thanks for the very prompt reply. I am on Numba 0.23.0.

From: Stan Seibert [mailto:notifications@github.com]
Sent: Wednesday, 20 January 2016 10:54 a.m.
To: numba/numba
Cc: Dean Anderson
Subject: Re: [numba] numpy.random.binomial fails (#1640)

This script seems to work fine on my system using an almost current version of Numba master. What version of Numba are you using?

(Note that this example code will be compiled in object mode because of the print call.)

—
Reply to this email directly or view it on GitHubhttps://github.com/numba/numba/issues/1640#issuecomment-173000109.

---

Please consider the environment before printing this email
Warning: This electronic message together with any attachments is confidential. If you receive it in error: (i) you must not read, use, disclose, copy or retain it; (ii) please contact the sender immediately by reply email and then delete the emails.
The views expressed in this email may not be those of Landcare Research New Zealand Limited. http://www.landcareresearch.co.nz

I also stall at 1459 like jcrist.

OK, Jim's script locks up for me on my Mac.  Here it is reformatted without the prompt text for people who want to copy-paste:

``` python
import numpy as np
import numba as nb

@nb.jit
def foo(n, p):
    return np.random.binomial(n, p)

print foo(1458, 0.4)
print foo(1459, 0.4)
```

LLDB shows me that when it it is stalled, the code is running around in a loop near this fragment of code (apologies for the x86 assembly, but that's all we have in JIT code):

```
(lldb) disassemble --frame
->  0x1053db5f5: mulsd  %xmm2, %xmm3
    0x1053db5f9: ucomisd %xmm6, %xmm3
    0x1053db5fd: jne    0x1053db606
    0x1053db5ff: jp     0x1053db606
    0x1053db601: jmp    0x1053db72e
    0x1053db606: cvtsi2sdq %rdx, %xmm4
    0x1053db60b: mulsd  %xmm8, %xmm4
    0x1053db610: mulsd  %xmm1, %xmm4
    0x1053db614: repne
```

I'm not very good at using LLDB, but if I `NUMBA_DUMP_ASSEMBLY=1`, then I can match and locate the rest of the assembly:

``` asm
LBB2_16:
    ucomisd %xmm0, %xmm1
    jae LBB2_17
    incq    %rdx
    cvtsi2sdq   %rdx, %xmm2
    movaps  %xmm7, %xmm3
    mulsd   %xmm2, %xmm3 /* Above Fragment starts here */
    ucomisd %xmm6, %xmm3
    jne LBB2_15
    jp  LBB2_15
    jmp LBB2_37
LBB2_15:
    cvtsi2sdq   %rcx, %xmm4
    mulsd   %xmm8, %xmm4
    mulsd   %xmm1, %xmm4
    subsd   %xmm1, %xmm0
    divsd   %xmm3, %xmm4
    decq    %rcx
    ucomisd %xmm2, %xmm5
    movapd  %xmm4, %xmm1
    jae LBB2_16
    jmp LBB2_28
```

As near as I can tell, we are running around this loop forever.

Brute force work-around on my system:

import numpy as np
import numba as nb

@nb.jit
def foo(n, p):
    d = 0
    for i in range(n):
        d += np.random.binomial(1, p)
    return d

print(foo(1458, 0.4))
print(foo(1459, 0.4))

OK, part of the answer is in the comments of the implementation:

https://github.com/numba/numba/blob/de51861a51e4aafbf0a5920ab9678dfdc10e9577/numba/targets/randomimpl.py#L754-L780

For very large `n`, `(1 - p) ** n` underflows to zero, and the loop will basically never accept a value.  However, if you fall out of the loop, then there is no explicit return value, and I don't know what happens.

So there are a few things to do:
- We should use "BTPE" for n*p >= 30, like NumPy, to avoid the underflow problem
- Should the compiler have caught the situation that there was a code path that exited this function without returning a value?  Can we fix that?


Resolving patch:
diff --git a/numba/targets/randomimpl.py b/numba/targets/randomimpl.py
--- a/numba/targets/randomimpl.py
+++ b/numba/targets/randomimpl.py
@@ -768,24 +768,46 @@ def binomial_impl(n, p):
             raise ValueError("binomial(): n <= 0")
         if not (0.0 <= p <= 1.0):
             raise ValueError("binomial(): p outside of [0, 1]")
+        if p == 0.0:
+            return 0
+        if p == 1.0:
+            return n
+
         flipped = p > 0.5
         if flipped:
             p = 1.0 - p
         q = 1.0 - p
+
+        niters = 1
         qn = q ** n
+        while qn <= 1e-308:
+            # Underflow => split into several iterations
+            # Note this is much slower than Numpy's BTPE
+            niters <<= 2
+            n >>= 2
+            qn = q ** n
+            assert n > 0
+
         np = n * p
         bound = min(n, np + 10.0 * math.sqrt(np * q + 1))
-        while 1:
+
+        finished = False
+        total = 0
+        while niters > 0:
             X = 0
             U = _random()
             px = qn
             while X <= bound:
                 if U <= px:
-                    return n - X if flipped else X
+                    total += n - X if flipped else X
+                    niters -= 1
+                    break
                 U -= px
                 X += 1
                 px = ((n - X + 1) * p * px) / (X * q)
 
+        return total
+
     res = context.compile_internal(builder, binomial_impl, sig, args)
     return impl_ret_untracked(context, builder, sig.return_type, res)
 


Test patch:
diff --git a/numba/tests/test_random.py b/numba/tests/test_random.py
--- a/numba/tests/test_random.py
+++ b/numba/tests/test_random.py
@@ -621,6 +621,21 @@ def test_numpy_binomial(self):
         binomial = jit_binary("np.random.binomial")
         r = self._follow_numpy(np_state_ptr, 0)
         self._check_dist(binomial, r.binomial, [(18, 0.25)])
+        # Sanity check many values
+        for n in (100, 1000, 10000):
+            self.assertEqual(binomial(n, 0.0), 0)
+            self.assertEqual(binomial(n, 1.0), n)
+            for p in (0.0001, 0.1, 0.4, 0.49999, 0.5, 0.50001, 0.8, 0.9, 0.9999):
+                r = binomial(n, p)
+                if p > 0.5:
+                    r = n - r
+                    p = 1 - p
+                self.assertGreaterEqual(r, 0)
+                self.assertLessEqual(r, n)
+                expected = p * n
+                tol = 3 * n / math.sqrt(n)
+                self.assertGreaterEqual(r, expected - tol, (p, n, r))
+                self.assertLessEqual(r, expected + tol, (p, n, r))
         # Invalid values
         self.assertRaises(ValueError, binomial, -1, 0.5)
         self.assertRaises(ValueError, binomial, 10, -0.1)