Repository: scipy/scipy

Issue:
ENH: Documentation on test behind p-values of .spearmanr
### Is your feature request related to a problem? Please describe.

The docstring of [.spearmanr](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.spearmanr.html#scipy.stats.spearmanr) currently does not report on the test which is used to determine the p-value.

Used scipy.stats.spearmanr for a scientific manuscript, and journal editor requested more information about used test.

Reading the code, the specific test is not evident. While it appears based on t distribution, could not identify the full procedure by name. Nor, identify by name the preprocessing scheme between computation of Spearman's rho and comparison against t distribution.

Possibly related to first point under [#8789](https://github.com/scipy/scipy/issues/8789). 


### Describe the solution you'd like.

Docstring stating or citing the test used for p-values.

### Describe alternatives you've considered.

_No response_

### Additional context (e.g. screenshots, GIFs)

_No response_
ENH: Documentation on test behind p-values of .spearmanr
### Is your feature request related to a problem? Please describe.

The docstring of [.spearmanr](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.spearmanr.html#scipy.stats.spearmanr) currently does not report on the test which is used to determine the p-value.

Used scipy.stats.spearmanr for a scientific manuscript, and journal editor requested more information about used test.

Reading the code, the specific test is not evident. While it appears based on t distribution, could not identify the full procedure by name. Nor, identify by name the preprocessing scheme between computation of Spearman's rho and comparison against t distribution.

Possibly related to first point under [#8789](https://github.com/scipy/scipy/issues/8789). 


### Describe the solution you'd like.

Docstring stating or citing the test used for p-values.

### Describe alternatives you've considered.

_No response_

### Additional context (e.g. screenshots, GIFs)

_No response_


Discussion:
I can help you find the information. Would you be willing to submit a PR adding this information to the docstring? Then I could review/merge the PR.
Thank you for helping me find the information.

Happy to then submit PR. 
[Wikipedia](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) mentions the same use of the t distribution and gives citations.
![image](https://user-images.githubusercontent.com/6570539/192166531-e6f20b87-b133-4b49-99f9-e6774153b32e.png)

One of those [references](https://www.cec.uchile.cl/cinetica/pcordero/MC_libros/NumericalRecipesinC.pdf) has it on page 640 (Equation 16.4.2):
![image](https://user-images.githubusercontent.com/6570539/192166788-396a7a87-fed7-4867-acd1-cb10fa7e3b59.png)

The [other reference ](http://www.ru.ac.bd/stat/wp-content/uploads/sites/25/2019/03/302_05_Kendall-Advanced-Theory-of-Statistics-3rd-Edition-2010.pdf) also gives this result and where it comes from.

![image](https://user-images.githubusercontent.com/6570539/192167332-c8d0c647-d1f0-4cfc-a402-ea145b413d61.png)

The null hypothesis is that there is no linear association between ranks of the samples, so for every ordering of the ranks of `a`, all possible orderings of the ranks of `b` are equally likely. The exact null distribution of the test statistic could be formed by computing the statistic value for all possible orderings of `b` given the ordering of `a`. These references are saying that it has been shown that this distribution of the test statistic is well approximated by a t-distribution. So instead of performing the test by comparing the observed statistic against the exact null distribution (which could, in principle, be computed for any sample size), one can compute an approximate p-value by comparing the observed statistic against the t distribution.

Here is code that demonstrates this relationship:
```python3
import math
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

N = 10
a = np.arange(N)
b = a

def statistic(b):
    r = stats.spearmanr(a, b).statistic
    t = r*np.sqrt((N-2)/(1-r**2))
    return t

res = stats.permutation_test((b,), statistic, permutation_type='pairings', 
                             n_resamples=math.factorial(N))

df = N-2
x = np.linspace(-4, 4, 200)
y = stats.t(df).pdf(x)

plt.hist(res.null_distribution, density=True, bins=np.linspace(-4, 4, 32))
plt.plot(x, y)
plt.xlim(-4, 4)
plt.legend(('Histogram of Exact Null Distribution', 't approximation'))
plt.xlabel('normalized test statistic')
plt.ylabel('probability density')
```
![image](https://user-images.githubusercontent.com/6570539/192193737-edc0a660-93f7-45a1-b1bc-1d39d69cd588.png)

If you're interested, I'd also be happy to review a PR that adds the option to compute exact p-values (instead of using the t approximation) using code like that above. All you'd need to do is:
- add an argument `n_resamples=None`,
- if an integer `n_resamples` is provided by the user, perform the `permutation_test` as above with `n_resamples=n_resamples` and have `spearmanr` return `res.pvalue`. 
- `if n_resamples is None`, get the p-value as usual.

I used `math.factorial` to perform an exhaustive test, but a randomized test where `n_resamples < math.factorial(N)` can also produce p-values that exactly control the long-run Type I error rate. (It would also be worth reading [this conversation](https://github.com/scipy/scipy/issues/7882#issuecomment-1062707979) and using a version of the statistic that is scaled to integer values to avoid issues with roundoff errors, but that should not be required because of gh-15869.)

Anyway, does this help?

Thank you! This is wonderful. Will later this week prepare PR.
Hi @tstoeger - please ping me when you open a PR so I don't miss it. Thanks!
@tstoeger just thought I'd check in. Are you still able to submit this PR so we can complete this issue?
I can help you find the information. Would you be willing to submit a PR adding this information to the docstring? Then I could review/merge the PR.
Thank you for helping me find the information.

Happy to then submit PR. 
[Wikipedia](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) mentions the same use of the t distribution and gives citations.
![image](https://user-images.githubusercontent.com/6570539/192166531-e6f20b87-b133-4b49-99f9-e6774153b32e.png)

One of those [references](https://www.cec.uchile.cl/cinetica/pcordero/MC_libros/NumericalRecipesinC.pdf) has it on page 640 (Equation 16.4.2):
![image](https://user-images.githubusercontent.com/6570539/192166788-396a7a87-fed7-4867-acd1-cb10fa7e3b59.png)

The [other reference ](http://www.ru.ac.bd/stat/wp-content/uploads/sites/25/2019/03/302_05_Kendall-Advanced-Theory-of-Statistics-3rd-Edition-2010.pdf) also gives this result and where it comes from.

![image](https://user-images.githubusercontent.com/6570539/192167332-c8d0c647-d1f0-4cfc-a402-ea145b413d61.png)

The null hypothesis is that there is no linear association between ranks of the samples, so for every ordering of the ranks of `a`, all possible orderings of the ranks of `b` are equally likely. The exact null distribution of the test statistic could be formed by computing the statistic value for all possible orderings of `b` given the ordering of `a`. These references are saying that it has been shown that this distribution of the test statistic is well approximated by a t-distribution. So instead of performing the test by comparing the observed statistic against the exact null distribution (which could, in principle, be computed for any sample size), one can compute an approximate p-value by comparing the observed statistic against the t distribution.

Here is code that demonstrates this relationship:
```python3
import math
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

N = 10
a = np.arange(N)
b = a

def statistic(b):
    r = stats.spearmanr(a, b).statistic
    t = r*np.sqrt((N-2)/(1-r**2))
    return t

res = stats.permutation_test((b,), statistic, permutation_type='pairings', 
                             n_resamples=math.factorial(N))

df = N-2
x = np.linspace(-4, 4, 200)
y = stats.t(df).pdf(x)

plt.hist(res.null_distribution, density=True, bins=np.linspace(-4, 4, 32))
plt.plot(x, y)
plt.xlim(-4, 4)
plt.legend(('Histogram of Exact Null Distribution', 't approximation'))
plt.xlabel('normalized test statistic')
plt.ylabel('probability density')
```
![image](https://user-images.githubusercontent.com/6570539/192193737-edc0a660-93f7-45a1-b1bc-1d39d69cd588.png)

If you're interested, I'd also be happy to review a PR that adds the option to compute exact p-values (instead of using the t approximation) using code like that above. All you'd need to do is:
- add an argument `n_resamples=None`,
- if an integer `n_resamples` is provided by the user, perform the `permutation_test` as above with `n_resamples=n_resamples` and have `spearmanr` return `res.pvalue`. 
- `if n_resamples is None`, get the p-value as usual.

I used `math.factorial` to perform an exhaustive test, but a randomized test where `n_resamples < math.factorial(N)` can also produce p-values that exactly control the long-run Type I error rate. (It would also be worth reading [this conversation](https://github.com/scipy/scipy/issues/7882#issuecomment-1062707979) and using a version of the statistic that is scaled to integer values to avoid issues with roundoff errors, but that should not be required because of gh-15869.)

Anyway, does this help?

Thank you! This is wonderful. Will later this week prepare PR.
Hi @tstoeger - please ping me when you open a PR so I don't miss it. Thanks!
@tstoeger just thought I'd check in. Are you still able to submit this PR so we can complete this issue?

Resolving patch:
diff --git a/scipy/stats/_stats_py.py b/scipy/stats/_stats_py.py
--- a/scipy/stats/_stats_py.py
+++ b/scipy/stats/_stats_py.py
@@ -4806,6 +4806,10 @@ def spearmanr(a, b=None, axis=0, nan_policy='propagate',
        Probability and Statistics Tables and Formulae. Chapman & Hall: New
        York. 2000.
        Section  14.7
+    .. [2] Kendall, M. G. and Stuart, A. (1973).
+       The Advanced Theory of Statistics, Volume 2: Inference and Relationship.
+       Griffin. 1973.
+       Section 31.18
 
     Examples
     --------


Test patch: