Repository: scipy/scipy
Pull request: BUG: stats: fix the cdf method of rv_discrete class
#### Reference issue
closes gh-13280

#### What does this implement/fix?
As described in gh-13280, the cdf method failed for 'hypergeom' and 'nhypergeom' distributions when invalid arguments were passed. The fix is to fill in the 'nan' values for invalid arguments after filling in the '1' values for quantiles greater than the upper bound in the 'cdf' method of 'rv_discrete' class. This fix has been implemented and regression tests for it have been added.

Problem statement:
Wrong results with hypergeom cdf
<!-- 
Thank you for taking the time to file a bug report. 
Please fill in the fields below, deleting the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
Note: This is a comment, and won't appear in the output.
-->

The hypergeometric cdf gives wrong results with invalid parameters

#### Reproducing code example:
<!-- 
If you place your code between the triple backticks below, 
it will be rendered as a code block. 
-->

```python
import numpy as np 
import scipy.stats as st 

print(st.hypergeom(M=3, n=3, N=4).pmf(np.arange(5))) 
print(st.hypergeom(M=3, n=3, N=4).cdf(np.arange(5)))                    
```

```python
[nan nan nan nan nan]
[nan nan nan  1.  1.]
```

#### Scipy/Numpy/Python version information:
```
scipy = 1.5.4
numpy = 1.19.4
sys.version_info(major=3, minor=8, micro=5, releaselevel='final', serial=0)
```

Hints from issue discussion:
@ricardoV94, thanks for reporting the issue.  I see that `nhypergeom` has the same problem:
```
In [32]: from scipy.stats import nhypergeom

In [33]: k = np.arange(-2, 6)

In [34]: nhypergeom.cdf(k, 5, 2, 8)
Out[34]: array([nan, nan, nan, nan,  1.,  1.,  1.,  1.])
```

Neither of these distributions implement the `_cdf` method, so the problem is probably in the parent class, `rv_discrete`.

@WarrenWeckesser Is it ok to get upper bound less than the lower bound when `_get_support` is called?

```python
>>> hypergeom._get_support(5, 3, 4)
(2, 3)
>>> hypergeom._get_support(4, 3, 4)
(3, 3)
>>> hypergeom._get_support(3, 3, 4)
(4, 3)
>>> hypergeom._get_support(2, 3, 4)
(5, 3)
>>> hypergeom._get_support(1, 3, 4)
(6, 3)
```
@tirthasheshpatel, that doesn't look right.  The same problem occurs with the public method `support`:
```
n [2]: hypergeom.support(1, 3, 4)
Out[2]: (6, 3)
```
I suspect a call to `_argcheck` needs to be incorporated into the `support` method of the `rv_generic` class.
Should this be done is a seperate PR or in gh-13283?
I think a separate PR would be better.

Commit message: TST: stats: add tests for all discrete distributions

Add tests for the cdf method of all the discrete distribution.

Patch to review:
diff --git a/scipy/stats/_distn_infrastructure.py b/scipy/stats/_distn_infrastructure.py
index 0691d0989697..0e120f5dcbac 100644
--- a/scipy/stats/_distn_infrastructure.py
+++ b/scipy/stats/_distn_infrastructure.py
@@ -3182,8 +3182,8 @@ def cdf(self, k, *args, **kwds):
         cond2 = (k >= _b)
         cond = cond0 & cond1
         output = zeros(shape(cond), 'd')
-        place(output, (1-cond0) + np.isnan(k), self.badvalue)
         place(output, cond2*(cond0 == cond0), 1.0)
+        place(output, (1-cond0) + np.isnan(k), self.badvalue)
 
         if np.any(cond):
             goodargs = argsreduce(cond, *((k,)+args))
diff --git a/scipy/stats/tests/test_discrete_basic.py b/scipy/stats/tests/test_discrete_basic.py
index 3a8a95518f1d..1242e16df848 100644
--- a/scipy/stats/tests/test_discrete_basic.py
+++ b/scipy/stats/tests/test_discrete_basic.py
@@ -271,3 +271,30 @@ def test_methods_with_lists(method, distname, args):
     npt.assert_allclose(result,
                         [dist.pmf(*v) for v in zip(z, *p2, loc)],
                         rtol=1e-15, atol=1e-15)
+
+
+@pytest.mark.parametrize(
+    'dist, x, args',
+    [
+        (stats.hypergeom, np.arange(5), (3, 3, 4)),
+        (stats.nhypergeom, np.arange(-2, 5), (5, 2, 8)),
+        (stats.bernoulli, np.arange(5), (1.5, )),
+        (stats.binom, np.arange(15), (10, 1.5)),
+        (stats.betabinom, np.arange(15), (10, -0.4, -0.5)),
+        (stats.boltzmann, np.arange(5), (-1, 4)),
+        (stats.dlaplace, np.arange(5), (-0.5, )),
+        (stats.geom, np.arange(5), (1.5, )),
+        (stats.logser, np.arange(5), (1.5, )),
+        (stats.nbinom, np.arange(15), (10, 1.5)),
+        (stats.planck, np.arange(5), (-0.5, )),
+        (stats.poisson, np.arange(5), (-0.5, )),
+        (stats.randint, np.arange(5), (5, 2)),
+        (stats.skellam, np.arange(5), (-5, -2)),
+        (stats.zipf, np.arange(5), (-2, )),
+        (stats.yulesimon, np.arange(5), (-2, ))
+    ]
+)
+def test_cdf_gh13280_regression(dist, x, args):
+    vals = dist.cdf(x, *args)
+    expected = np.array(x.size*[np.nan])
+    npt.assert_equal(vals, expected)


Code review comments:
diff_hunk: @@ -271,3 +271,30 @@ def test_methods_with_lists(method, distname, args):
     npt.assert_allclose(result,
                         [dist.pmf(*v) for v in zip(z, *p2, loc)],
                         rtol=1e-15, atol=1e-15)
+
+
+@pytest.mark.parametrize(
+    'dist, x, args',
+    [
+        (stats.hypergeom, np.arange(5), (3, 3, 4)),
+        (stats.nhypergeom, np.arange(-2, 5), (5, 2, 8)),
+        (stats.bernoulli, np.arange(5), (1.5, )),
+        (stats.binom, np.arange(15), (10, 1.5)),
+        (stats.betabinom, np.arange(15), (10, -0.4, -0.5)),
+        (stats.boltzmann, np.arange(5), (-1, 4)),
+        (stats.dlaplace, np.arange(5), (-0.5, )),
+        (stats.geom, np.arange(5), (1.5, )),
+        (stats.logser, np.arange(5), (1.5, )),
+        (stats.nbinom, np.arange(15), (10, 1.5)),
+        (stats.planck, np.arange(5), (-0.5, )),
+        (stats.poisson, np.arange(5), (-0.5, )),
+        (stats.randint, np.arange(5), (5, 2)),
+        (stats.skellam, np.arange(5), (-5, -2)),
+        (stats.zipf, np.arange(5), (-2, )),
+        (stats.yulesimon, np.arange(5), (-2, ))
+    ]
+)
+def test_cdf_gh13280_regression(dist, x, args):
+    vals = dist.cdf(x, *args)
+    expected = np.array(x.size*[np.nan])
+    npt.assert_equal(vals, expected)
path: scipy/stats/tests/test_discrete_basic.py
text: @user1:
Just a note that `expected` could simply be `np.nan`, unless it is deemed important to check the size of the output array.

@author:
Ah, yes. I will do it that way. Thanks!