Repository: scipy/scipy

Issue:
BUG: invgauss.cdf should return the correct value when `mu` is very small.
Currently `invgauss.cdf` returns NaN when mu is too small. This is due to `exp(1 /mu)` blowing up when mu is small (the docs say that this happens for values smaller than 0.0028).
https://github.com/scipy/scipy/blob/fc77ea19923c39618547b4033a9185dd8a3afcc1/scipy/stats/_continuous_distns.py#L3504-L3509

In the expression evaluatating the CDF, the term `_norm_cdf(-fac*(x+mu)/mu)` is zero when mu is very small, so that the CDF evaluates to 1 due to the term `_norm_cdf(fac*(x-mu)/mu)` being approximately 1 for very small `mu`.

I believe that returning `nan` instead of 1 is not the best approach to handle the overflow. The overflow of `exp` does not practically affect the final value of the cdf is cases when `mu` is very small because `_norm_cdf(-fac*(x+mu)/mu)` is zero anyway.
#### Reproducing code example:
```python
In [1]: import numpy as np

In [2]: from scipy.stats import invgauss

In [3]: rng = np.random.RandomState(1)

In [4]: mu = rng.uniform(0., 0.01, size=5)

In [5]: mu
Out[5]: 
array([4.17022005e-03, 7.20324493e-03, 1.14374817e-06, 3.02332573e-03,
       1.46755891e-03])

In [6]: invgauss.cdf(0.4, mu=mu)
/home/scipy/scipy/stats/_continuous_distns.py:3508: RuntimeWarning: overflow encountered in exp
  C1 += np.exp(1.0/mu) * _norm_cdf(-fac*(x+mu)/mu) * np.exp(1.0/mu)
/home/scipy/scipy/stats/_continuous_distns.py:3508: RuntimeWarning: invalid value encountered in multiply
  C1 += np.exp(1.0/mu) * _norm_cdf(-fac*(x+mu)/mu) * np.exp(1.0/mu)
Out[6]: array([ 1.,  1., nan,  1.,  1.])
```

I played around with handling the overflow by setting the value of `exp(1/mu)` to the largest double and then the remainder of the function evaluates to the "correct" value, which is 1.

```python
In [4]: mu = np.random.uniform(0., 0.01, size=5)

In [5]: invgauss.cdf(0.4, mu=mu)
Out[5]: array([1., 1., 1., 1., 1.])

In [6]: mu
Out[6]: array([0.0006815 , 0.00685858, 0.00949644, 0.00324687, 0.00621239])

In [7]: rng = np.random.RandomState(1)

In [8]: mu = rng.uniform(0., 0.01, size=5)

In [9]: mu
Out[9]: 
array([4.17022005e-03, 7.20324493e-03, 1.14374817e-06, 3.02332573e-03,
       1.46755891e-03])

In [10]: invgauss.cdf(0.4, mu=mu)
Out[10]: array([1., 1., 1., 1., 1.])
```

Here is the code snippet:
https://github.com/scipy/scipy/compare/master...zoj613:invgauss

I was wondering what does everyone think regarding changing the current behavior of the function.


```
1.7.0.dev0+fc77ea1 1.20.1 sys.version_info(major=3, minor=8, micro=6, releaselevel='final', serial=0)
```



Discussion:
Yes, it seems that 1 is a more appropriate value than `nan`. 
Are there other reasons that `nan` would occur here? If not, rather than predict where the `nan` will appear w/ `_max_exp_val`, etc., consider performing the original calculation (silencing warnings) and replacing the `nan`s with 1. There would not be a practical difference in speed, and it is more obvious that there are no edge cases in which `nan`s would slip through. (Not required, but please consider it and comment if you disagree.)
I think a PR would be welcome.

Resolving patch:
diff --git a/scipy/stats/_continuous_distns.py b/scipy/stats/_continuous_distns.py
--- a/scipy/stats/_continuous_distns.py
+++ b/scipy/stats/_continuous_distns.py
@@ -3481,10 +3481,6 @@ class invgauss_gen(rv_continuous):
 
     %(after_notes)s
 
-    When :math:`\mu` is too small, evaluating the cumulative distribution
-    function will be inaccurate due to ``cdf(mu -> 0) = inf * 0``.
-    NaNs are returned for :math:`\mu \le 0.0028`.
-
     %(example)s
 
     """
@@ -3501,12 +3497,27 @@ def _pdf(self, x, mu):
     def _logpdf(self, x, mu):
         return -0.5*np.log(2*np.pi) - 1.5*np.log(x) - ((x-mu)/mu)**2/(2*x)
 
+    # approach adapted from equations in
+    # https://journal.r-project.org/archive/2016-1/giner-smyth.pdf,
+    # not R code. see gh-13616
+
+    def _logcdf(self, x, mu):
+        fac = 1 / np.sqrt(x)
+        a = _norm_logcdf(fac * ((x / mu) - 1))
+        b = 2 / mu + _norm_logcdf(-fac * ((x / mu) + 1))
+        return a + np.log1p(np.exp(b - a))
+
+    def _logsf(self, x, mu):
+        fac = 1 / np.sqrt(x)
+        a = _norm_logsf(fac * ((x / mu) - 1))
+        b = 2 / mu + _norm_logcdf(-fac * (x + mu) / mu)
+        return a + np.log1p(-np.exp(b - a))
+
+    def _sf(self, x, mu):
+        return np.exp(self._logsf(x, mu))
+
     def _cdf(self, x, mu):
-        fac = np.sqrt(1.0/x)
-        # Numerical accuracy for small `mu` is bad.  See #869.
-        C1 = _norm_cdf(fac*(x-mu)/mu)
-        C1 += np.exp(1.0/mu) * _norm_cdf(-fac*(x+mu)/mu) * np.exp(1.0/mu)
-        return C1
+        return np.exp(self._logcdf(x, mu))
 
     def _stats(self, mu):
         return mu, mu**3.0, 3*np.sqrt(mu), 15*mu
@@ -8348,11 +8359,20 @@ def _pdf(self, x):
         # wald.pdf(x) = 1/sqrt(2*pi*x**3) * exp(-(x-1)**2/(2*x))
         return invgauss._pdf(x, 1.0)
 
+    def _cdf(self, x):
+        return invgauss._cdf(x, 1.0)
+
+    def _sf(self, x):
+        return invgauss._sf(x, 1.0)
+
     def _logpdf(self, x):
         return invgauss._logpdf(x, 1.0)
 
-    def _cdf(self, x):
-        return invgauss._cdf(x, 1.0)
+    def _logcdf(self, x):
+        return invgauss._logcdf(x, 1.0)
+
+    def _logsf(self, x):
+        return invgauss._logsf(x, 1.0)
 
     def _stats(self):
         return 1.0, 1.0, 3.0, 15.0


Test patch:
diff --git a/scipy/stats/tests/test_distributions.py b/scipy/stats/tests/test_distributions.py
--- a/scipy/stats/tests/test_distributions.py
+++ b/scipy/stats/tests/test_distributions.py
@@ -1809,6 +1809,59 @@ def test_fit_raise_errors(self):
         with pytest.raises(FitDataError):
             stats.invgauss.fit([1, 2, 3], floc=2)
 
+    def test_cdf_sf(self):
+        # Regression tests for gh-13614.
+        # Ground truth from R's statmod library (pinvgauss), e.g.
+        # library(statmod)
+        # options(digits=15)
+        # mu = c(4.17022005e-04, 7.20324493e-03, 1.14374817e-06,
+        #        3.02332573e-03, 1.46755891e-03)
+        # print(pinvgauss(5, mu, 1))
+
+        # make sure a finite value is returned when mu is very small. see
+        # GH-13614
+        mu = [4.17022005e-04, 7.20324493e-03, 1.14374817e-06,
+              3.02332573e-03, 1.46755891e-03]
+        expected = [1, 1, 1, 1, 1]
+        actual = stats.invgauss.cdf(0.4, mu=mu)
+        assert_equal(expected, actual)
+
+        # test if the function can distinguish small left/right tail
+        # probabilities from zero.
+        cdf_actual = stats.invgauss.cdf(0.001, mu=1.05)
+        assert_allclose(cdf_actual, 4.65246506892667e-219)
+        sf_actual = stats.invgauss.sf(110, mu=1.05)
+        assert_allclose(sf_actual, 4.12851625944048e-25)
+
+        # test if x does not cause numerical issues when mu is very small
+        # and x is close to mu in value.
+
+        # slightly smaller than mu
+        actual = stats.invgauss.cdf(0.00009, 0.0001)
+        assert_allclose(actual, 2.9458022894924e-26)
+
+        # slightly bigger than mu
+        actual = stats.invgauss.cdf(0.000102, 0.0001)
+        assert_allclose(actual, 0.976445540507925)
+
+    def test_logcdf_logsf(self):
+        # Regression tests for improvements made in gh-13616.
+        # Ground truth from R's statmod library (pinvgauss), e.g.
+        # library(statmod)
+        # options(digits=15)
+        # print(pinvgauss(0.001, 1.05, 1, log.p=TRUE, lower.tail=FALSE))
+
+        # test if logcdf and logsf can compute values too small to
+        # be represented on the unlogged scale. See: gh-13616
+        logcdf = stats.invgauss.logcdf(0.0001, mu=1.05)
+        assert_allclose(logcdf, -5003.87872590367)
+        logcdf = stats.invgauss.logcdf(110, 1.05)
+        assert_allclose(logcdf, -4.12851625944087e-25)
+        logsf = stats.invgauss.logsf(0.001, mu=1.05)
+        assert_allclose(logsf, -4.65246506892676e-219)
+        logsf = stats.invgauss.logsf(110, 1.05)
+        assert_allclose(logsf, -56.1467092416426)
+
 
 class TestLaplace(object):
     @pytest.mark.parametrize("rvs_loc", [-5, 0, 1, 2])