Repository: scipy/scipy

Issue:
Inconsistent warnings for correlation functions in scipy.stats
<!-- 
Thank you for taking the time to file a bug report. 
Please fill in the fields below, deleting the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
Note: This is a comment, and won't appear in the output.
-->

My issue is about `scipy.stats.spearmanr()` function handling constant input(s). 

#### Reproducing code example:
```
from scipy import stats
sequence_1 = [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
sequence_2 = [0, 0.009783728115345005, 0, 0, 0.0019759230121848587, 0.0007535430349118562, 0.0002661781514710257, 0, 0, 0.0007835762419683435]
myspearman = stats.spearmanr(sequence_1,sequence_2)
```
throws a `RuntimeWarning` but provides no informative output. While `pearsonr()` throws a 
`PearsonRConstantInputWarning` which was added in a PR of @WarrenWeckesser in the `1.3.X` release, here is the original PR:
https://github.com/scipy/scipy/pull/9562


#### Error message:
```
>>> myspearman = stats.spearmanr(sequence_1,sequence_2)
/Users/rlucas/anaconda3/envs/scipy-dev/lib/python3.6/site-packages/numpy/lib/function_base.py:2530: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[:, None]
/Users/rlucas/anaconda3/envs/scipy-dev/lib/python3.6/site-packages/numpy/lib/function_base.py:2531: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[None, :]
/Users/rlucas/scipy-dev/scipy/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in greater
  return (a < x) & (x < b)
/Users/rlucas/scipy-dev/scipy/scipy/stats/_distn_infrastructure.py:903: RuntimeWarning: invalid value encountered in less
  return (a < x) & (x < b)
/Users/rlucas/scipy-dev/scipy/scipy/stats/_distn_infrastructure.py:1912: RuntimeWarning: invalid value encountered in less_equal
  cond2 = cond0 & (x <= _a)
# vs...
>>> myspearman = stats.pearsonr(sequence_1,sequence_2)
/Users/rlucas/scipy-dev/scipy/scipy/stats/stats.py:3508: PearsonRConstantInputWarning: An input array is constant; the correlation coefficent is not defined.
  warnings.warn(PearsonRConstantInputWarning())
```

#### Scipy/Numpy/Python version information:
```
>>> import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
1.5.0.dev0+634d0c0 1.16.4 sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0)
```

I'm proposing to have a similar, informative warning message for `spearmanr()`. 

Interestingly, `kendalltau()` does not throw a warning or error on the input but does also produce `nan` in the `namedtuple` results. 

This was originally found by EmJ and asked about on stackoverflow, link: 
https://stackoverflow.com/questions/59002624/why-i-get-nan-in-spearman-correlation-in-python?noredirect=1#comment104258321_59002624


Discussion:
Reading back over the thread in #9562 now I see the response from @WarrenWeckesser 

>@chrisb83 wrote
>
>    should we replace PearsonRConstantInputWarning by ConstantInputWarning, I think this could >be a generic warning whereas PearsonRNearConstantInputWarning might be indeed a specific >warning for PearsonR

>There are probably other functions that could use a similar pattern, but currently, it is specific to >pearsonr. I'd rather not try to generalize this without concrete use-cases. And if/when we >implement a similar behavior for other functions, I don't think there would be anything wrong with >another function, say foo(), having its own FooConstantInputWarning.
>
>   do we want to show the warnings in the overview generated by init.py?
>
>I debated this, too. I figured they are part of the public API of scipy.stats, so they should be there. >But I don't mind taking them out. They are still documented in the pearsonr docstring, and maybe >that's enough. Any other opinions out there?

Based on that back and forth, provided we all still agree, I'd say the fix for this PR would be to have a separate constant input warning for the `spearmanr()` that provides a similar informative warning and is specialized to the function. 

The only open question that I'd have here is whether we want to have this warning in a hierarchy? I can see e.g. a `kendalltau` constant input warning potentially in the future and maybe something for the regression functions and other interpolations as well. Thoughts? 


> Based on that back and forth, provided we all still agree, I'd say the fix for this PR would be to have a separate constant input warning for the spearmanr() that provides a similar informative warning and is specialized to the function.

Agree

> The only open question that I'd have here is whether we want to have this warning in a hierarchy?

Can you explain?

I meant if we wanted a parent warning class of constant input and have the different correlation warnings inherit.

Alternate approach would be separate warnings for each that are single inheritance. 

@chrisb83 Does that make sense? 

> On Dec 10, 2019, at 4:07 PM, Christoph Baumgarten <notifications@github.com> wrote:
> 
> ﻿
> Based on that back and forth, provided we all still agree, I'd say the fix for this PR would be to have a separate constant input warning for the spearmanr() that provides a similar informative warning and is specialized to the function.
> 
> Agree
> 
> The only open question that I'd have here is whether we want to have this warning in a hierarchy?
> 
> Can you explain?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or unsubscribe.

So I thought about this a bit more and closely checked the source in warren’s PR. The Pearson warning inherits from a runtime warning. From rereading again the comments in the other PR, it sounds like the approach is to create a custom warning for spearman Have inherit from runtime. The benefit there is a simpler hierarchy and the trade off is you need to create a new warning class for each function.

I’ll create a new warning class following Warren’s approach, unless consensus changed. 


> On Dec 10, 2019, at 5:30 PM, rlucas7@vt.edu wrote:
> 
> ﻿I meant if we wanted a parent warning class of constant input and have the different correlation warnings inherit.
> 
> Alternate approach would be separate warnings for each that are single inheritance. 
> 
> @chrisb83 Does that make sense? 
> 
>>> On Dec 10, 2019, at 4:07 PM, Christoph Baumgarten <notifications@github.com> wrote:
>>> 
>> ﻿
>> Based on that back and forth, provided we all still agree, I'd say the fix for this PR would be to have a separate constant input warning for the spearmanr() that provides a similar informative warning and is specialized to the function.
>> 
>> Agree
>> 
>> The only open question that I'd have here is whether we want to have this warning in a hierarchy?
>> 
>> Can you explain?
>> 
>> —
>> You are receiving this because you authored the thread.
>> Reply to this email directly, view it on GitHub, or unsubscribe.


Resolving patch:
diff --git a/scipy/stats/__init__.py b/scipy/stats/__init__.py
--- a/scipy/stats/__init__.py
+++ b/scipy/stats/__init__.py
@@ -374,6 +374,7 @@
 
    PearsonRConstantInputWarning
    PearsonRNearConstantInputWarning
+   SpearmanRConstantInputWarning
 
 For many more stat related functions install the software R and the
 interface package rpy.
diff --git a/scipy/stats/stats.py b/scipy/stats/stats.py
--- a/scipy/stats/stats.py
+++ b/scipy/stats/stats.py
@@ -201,9 +201,9 @@
            'sem', 'zmap', 'zscore', 'iqr', 'gstd', 'median_absolute_deviation',
            'sigmaclip', 'trimboth', 'trim1', 'trim_mean', 'f_oneway',
            'PearsonRConstantInputWarning', 'PearsonRNearConstantInputWarning',
-           'pearsonr', 'fisher_exact', 'spearmanr', 'pointbiserialr',
-           'kendalltau', 'weightedtau',
-           'multiscale_graphcorr',
+           'pearsonr', 'fisher_exact', 'SpearmanRConstantInputWarning',
+           'spearmanr', 'pointbiserialr',
+           'kendalltau', 'weightedtau', 'multiscale_graphcorr',
            'linregress', 'siegelslopes', 'theilslopes', 'ttest_1samp',
            'ttest_ind', 'ttest_ind_from_stats', 'ttest_rel', 'kstest',
            'chisquare', 'power_divergence', 'ks_2samp', 'mannwhitneyu',
@@ -3719,6 +3719,16 @@ def binary_search(n, n1, n2, side):
     return oddsratio, pvalue
 
 
+class SpearmanRConstantInputWarning(RuntimeWarning):
+    """Warning generated by `spearmanr` when an input is constant."""
+
+    def __init__(self, msg=None):
+        if msg is None:
+            msg = ("An input array is constant; the correlation coefficent "
+                   "is not defined.")
+        self.args = (msg,)
+
+
 SpearmanrResult = namedtuple('SpearmanrResult', ('correlation', 'pvalue'))
 
 
@@ -3818,6 +3828,9 @@ def spearmanr(a, b=None, axis=0, nan_policy='propagate'):
     (0.052760927029710199, 0.60213045837062351)
 
     """
+    if axis is not None and axis > 1:
+        raise ValueError("spearmanr only handles 1-D or 2-D arrays, supplied axis argument {}, please use only values 0, 1 or None for axis".format(axis))
+
     a, axisout = _chk_asarray(a, axis)
     if a.ndim > 2:
         raise ValueError("spearmanr only handles 1-D or 2-D arrays")
@@ -3840,6 +3853,17 @@ def spearmanr(a, b=None, axis=0, nan_policy='propagate'):
         # Handle empty arrays or single observations.
         return SpearmanrResult(np.nan, np.nan)
 
+    if axisout == 0:
+        if (a[:, 0][0] == a[:, 0]).all() or (a[:, 1][0] == a[:, 1]).all():
+            # If an input is constant, the correlation coefficient is not defined.
+            warnings.warn(SpearmanRConstantInputWarning())
+            return SpearmanrResult(np.nan, np.nan)
+    else:  # case when axisout == 1 b/c a is 2 dim only
+        if (a[0, :][0] == a[0, :]).all() or (a[1, :][0] == a[1, :]).all():
+            # If an input is constant, the correlation coefficient is not defined.
+            warnings.warn(SpearmanRConstantInputWarning())
+            return SpearmanrResult(np.nan, np.nan)
+
     a_contains_nan, nan_policy = _contains_nan(a, nan_policy)
     variable_has_nan = np.zeros(n_vars, dtype=bool)
     if a_contains_nan:


Test patch:
diff --git a/scipy/stats/tests/test_stats.py b/scipy/stats/tests/test_stats.py
--- a/scipy/stats/tests/test_stats.py
+++ b/scipy/stats/tests/test_stats.py
@@ -768,78 +768,83 @@ def test_gh_8111(self):
         assert_allclose([res1, res2, res3], expected)
 
 
-def test_spearmanr():
-    # Cross-check with R:
-    # cor.test(c(1,2,3,4,5),c(5,6,7,8,7),method="spearmanr")
-    x1 = [1, 2, 3, 4, 5]
-    x2 = [5, 6, 7, 8, 7]
-    expected = (0.82078268166812329, 0.088587005313543798)
-    res = stats.spearmanr(x1, x2)
-    assert_approx_equal(res[0], expected[0])
-    assert_approx_equal(res[1], expected[1])
-
-    attributes = ('correlation', 'pvalue')
-    res = stats.spearmanr(x1, x2)
-    check_named_results(res, attributes)
-
-    # with only ties in one or both inputs
-    with np.errstate(invalid="ignore"):
-        assert_equal(stats.spearmanr([2,2,2], [2,2,2]), (np.nan, np.nan))
-        assert_equal(stats.spearmanr([2,0,2], [2,2,2]), (np.nan, np.nan))
-        assert_equal(stats.spearmanr([2,2,2], [2,0,2]), (np.nan, np.nan))
-
-    # empty arrays provided as input
-    assert_equal(stats.spearmanr([], []), (np.nan, np.nan))
-
-    np.random.seed(7546)
-    x = np.array([np.random.normal(loc=1, scale=1, size=500),
-                np.random.normal(loc=1, scale=1, size=500)])
-    corr = [[1.0, 0.3],
-            [0.3, 1.0]]
-    x = np.dot(np.linalg.cholesky(corr), x)
-    expected = (0.28659685838743354, 6.579862219051161e-11)
-    res = stats.spearmanr(x[0], x[1])
-    assert_approx_equal(res[0], expected[0])
-    assert_approx_equal(res[1], expected[1])
-
-    assert_approx_equal(stats.spearmanr([1,1,2], [1,1,2])[0], 1.0)
-
-    # test nan_policy
-    x = np.arange(10.)
-    x[9] = np.nan
-    assert_array_equal(stats.spearmanr(x, x), (np.nan, np.nan))
-    assert_allclose(stats.spearmanr(x, x, nan_policy='omit'),
-                    (1.0, 0))
-    assert_raises(ValueError, stats.spearmanr, x, x, nan_policy='raise')
-    assert_raises(ValueError, stats.spearmanr, x, x, nan_policy='foobar')
-
-    # test unequal length inputs
-    x = np.arange(10.)
-    y = np.arange(20.)
-    assert_raises(ValueError, stats.spearmanr, x, y)
+class TestCorrSpearmanr2(object):
+    """Some further tests of the spearmanr function."""
+
+    def test_spearmanr_vs_r(self):
+        # Cross-check with R:
+        # cor.test(c(1,2,3,4,5),c(5,6,7,8,7),method="spearmanr")
+        x1 = [1, 2, 3, 4, 5]
+        x2 = [5, 6, 7, 8, 7]
+        expected = (0.82078268166812329, 0.088587005313543798)
+        res = stats.spearmanr(x1, x2)
+        assert_approx_equal(res[0], expected[0])
+        assert_approx_equal(res[1], expected[1])
+
+    def test_empty_arrays(self):
+        assert_equal(stats.spearmanr([], []), (np.nan, np.nan))
+
+    def test_normal_draws(self):
+        np.random.seed(7546)
+        x = np.array([np.random.normal(loc=1, scale=1, size=500),
+                    np.random.normal(loc=1, scale=1, size=500)])
+        corr = [[1.0, 0.3],
+                [0.3, 1.0]]
+        x = np.dot(np.linalg.cholesky(corr), x)
+        expected = (0.28659685838743354, 6.579862219051161e-11)
+        res = stats.spearmanr(x[0], x[1])
+        assert_approx_equal(res[0], expected[0])
+        assert_approx_equal(res[1], expected[1])
+
+    def test_corr_1(self):
+        assert_approx_equal(stats.spearmanr([1, 1, 2], [1, 1, 2])[0], 1.0)
+
+    def test_nan_policies(self):
+        x = np.arange(10.)
+        x[9] = np.nan
+        assert_array_equal(stats.spearmanr(x, x), (np.nan, np.nan))
+        assert_allclose(stats.spearmanr(x, x, nan_policy='omit'),
+                        (1.0, 0))
+        assert_raises(ValueError, stats.spearmanr, x, x, nan_policy='raise')
+        assert_raises(ValueError, stats.spearmanr, x, x, nan_policy='foobar')
 
-    #test paired value
-    x1 = [1, 2, 3, 4]
-    x2 = [8, 7, 6, np.nan]
-    res1 = stats.spearmanr(x1, x2, nan_policy='omit')
-    res2 = stats.spearmanr(x1[:3], x2[:3], nan_policy='omit')
-    assert_equal(res1, res2)
+    def test_unequal_lengths(self):
+        x = np.arange(10.)
+        y = np.arange(20.)
+        assert_raises(ValueError, stats.spearmanr, x, y)
+
+    def test_omit_paired_value(self):
+        x1 = [1, 2, 3, 4]
+        x2 = [8, 7, 6, np.nan]
+        res1 = stats.spearmanr(x1, x2, nan_policy='omit')
+        res2 = stats.spearmanr(x1[:3], x2[:3], nan_policy='omit')
+        assert_equal(res1, res2)
 
-    # Regression test for GitHub issue #6061 - Overflow on Windows
-    x = list(range(2000))
-    y = list(range(2000))
-    y[0], y[9] = y[9], y[0]
-    y[10], y[434] = y[434], y[10]
-    y[435], y[1509] = y[1509], y[435]
-    # rho = 1 - 6 * (2 * (9^2 + 424^2 + 1074^2))/(2000 * (2000^2 - 1))
-    #     = 1 - (1 / 500)
-    #     = 0.998
-    x.append(np.nan)
-    y.append(3.0)
-    assert_almost_equal(stats.spearmanr(x, y, nan_policy='omit')[0], 0.998)
-
-class TestCorrSpearmanrTies(object):
-    """Some tests of tie-handling by the spearmanr function."""
+    def test_gh_issue_6061_windows_overflow(self):
+        x = list(range(2000))
+        y = list(range(2000))
+        y[0], y[9] = y[9], y[0]
+        y[10], y[434] = y[434], y[10]
+        y[435], y[1509] = y[1509], y[435]
+        # rho = 1 - 6 * (2 * (9^2 + 424^2 + 1074^2))/(2000 * (2000^2 - 1))
+        #     = 1 - (1 / 500)
+        #     = 0.998
+        x.append(np.nan)
+        y.append(3.0)
+        assert_almost_equal(stats.spearmanr(x, y, nan_policy='omit')[0], 0.998)
+
+    def test_tie0(self):
+        # with only ties in one or both inputs
+        with assert_warns(stats.SpearmanRConstantInputWarning):
+            r, p = stats.spearmanr([2, 2, 2], [2, 2, 2])
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+            r, p = stats.spearmanr([2, 0, 2], [2, 2, 2])
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+            r, p = stats.spearmanr([2, 2, 2], [2, 0, 2])
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
 
     def test_tie1(self):
         # Data
@@ -868,6 +873,38 @@ def test_tie2(self):
         sr2 = stats.spearmanr(x2, y2, nan_policy='omit')
         assert_almost_equal(sr1, sr2)
 
+    def test_ties_axis_1(self):
+        z1 = np.array([[1, 1, 1, 1], [1, 2, 3, 4]])
+        z2 = np.array([[1, 2, 3, 4], [1, 1, 1, 1]])
+        z3 = np.array([[1, 1, 1, 1], [1, 1, 1, 1]])
+        with assert_warns(stats.SpearmanRConstantInputWarning):
+            r, p = stats.spearmanr(z1, axis=1)
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+            r, p = stats.spearmanr(z2, axis=1)
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+            r, p = stats.spearmanr(z3, axis=1)
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+
+    def test_gh_11111(self):
+        x = np.array([1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])
+        y = np.array([0, 0.009783728115345005, 0, 0, 0.0019759230121848587,
+            0.0007535430349118562, 0.0002661781514710257, 0, 0,
+            0.0007835762419683435])
+        with assert_warns(stats.SpearmanRConstantInputWarning):
+            r, p = stats.spearmanr(x, y)
+            assert_equal(r, np.nan)
+            assert_equal(p, np.nan)
+
+    def test_index_error(self):
+        x = np.array([1.0, 7.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0])
+        y = np.array([0, 0.009783728115345005, 0, 0, 0.0019759230121848587,
+            0.0007535430349118562, 0.0002661781514710257, 0, 0,
+            0.0007835762419683435])
+        assert_raises(ValueError, stats.spearmanr, x, y, axis=2)
+
 
 #    W.II.E.  Tabulate X against X, using BIG as a case weight.  The values
 #    should appear on the diagonal and the total should be 899999955.