Repository: scipy/scipy

Issue:
Overflow error with new way of p-value calculation in kendall tau correlation for perfectly monotonic vectors.
My issue is about overflow error caused by zero discordant pairs in scipy version 1.2.0, in the improved p-value calculation of the kendall's tau function. In the case of two perfectly monotonic vectors x, y, c becomes 0 and the if-else condition calculates a factorial. In version 1.1.0, this refined term calculation is not present and so this issue does not occur there. If others can reproduce this issue, it seems that the solution is simple: when c becomes 0 (when dis is 0), p-value converges to zero very rapidly for sizes > 15 etc., so an if-statement could take care of that. Another solution could be to flag large sample sizes with method='asymptotic' being used instead.

(The c = 1 case seems to be similar in my mind.)

```
elif c == 0:
    pvalue = 2.0/np.math.factorial(size)
elif c == 1:
    pvalue = 2.0/np.math.factorial(size-1)
```

### Reproducing code example:

Sample code to reproduce the problem
```
from scipy import stats
kendall, _ = stats.kendalltau(np.arange(200), np.arange(200))
```
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/scipy/stats/stats.py", line 3632, in kendalltau
    pvalue = 2.0/np.math.factorial(size)
OverflowError: long int too large to convert to float
```
Note that in this example, size is equal to 200.

### Scipy/Numpy/Python version information:
```
>>> scipy.__version__
'1.2.0'
```

This error does not occur in 1.1.0.


Discussion:
Thanks for the bug report. `np.longdouble(np.math.factorial(170))` works, but for 171 it also overflows.

Current condition to determine the method is as follows:

```
if method == 'auto':
        if (xtie == 0 and ytie == 0) and (size <= 33 or min(dis, tot-dis) <= 1):
            method = 'exact'
```


Should use exp(gammaln(size)) which will give +inf.
Please fix this. It's embarrasing.

Resolving patch:
diff --git a/scipy/stats/stats.py b/scipy/stats/stats.py
--- a/scipy/stats/stats.py
+++ b/scipy/stats/stats.py
@@ -3895,11 +3895,10 @@ def count_rank_tie(ranks):
         elif size == 2:
             pvalue = 1.0
         elif c == 0:
-            pvalue = 2.0/np.math.factorial(size)
+            pvalue = 2.0/math.factorial(size) if size < 171 else 0.0
         elif c == 1:
-            pvalue = 2.0/np.math.factorial(size-1)
+            pvalue = 2.0/math.factorial(size-1) if (size-1) < 171 else 0.0
         else:
-            old = [0.0]*(c+1)
             new = [0.0]*(c+1)
             new[0] = 1.0
             new[1] = 1.0
@@ -3909,7 +3908,7 @@ def count_rank_tie(ranks):
                     new[k] += new[k-1]
                 for k in range(j,c+1):
                     new[k] += new[k-1] - old[k-j]
-            pvalue = 2.0*sum(new)/np.math.factorial(size)
+            pvalue = 2.0*sum(new)/math.factorial(size) if size < 171 else 0.0
 
     elif method == 'asymptotic':
         # con_minus_dis is approx normally distributed with this variance [3]_


Test patch:
diff --git a/scipy/stats/tests/test_stats.py b/scipy/stats/tests/test_stats.py
--- a/scipy/stats/tests/test_stats.py
+++ b/scipy/stats/tests/test_stats.py
@@ -978,6 +978,20 @@ def test_weightedtau():
     assert_approx_equal(tau, -0.56694968153682723)
 
 
+def test_kendall_tau_large():
+    n = 172.
+    x = np.arange(n)
+    y = np.arange(n)
+    _, pval = stats.kendalltau(x, y, method='exact')
+    assert_equal(pval, 0.0)
+    y[-1], y[-2] = y[-2], y[-1]
+    _, pval = stats.kendalltau(x, y, method='exact')
+    assert_equal(pval, 0.0)
+    y[-3], y[-4] = y[-4], y[-3]
+    _, pval = stats.kendalltau(x, y, method='exact')
+    assert_equal(pval, 0.0)
+
+
 def test_weightedtau_vs_quadratic():
     # Trivial quadratic implementation, all parameters mandatory
     def wkq(x, y, rank, weigher, add):