Repository: scipy/scipy

Issue:
scipy.stats.normaltest sometimes gives incorrect returns b/c of bug in scipy.stats.kurtosis.test
<<Please describe the issue in detail here, and for bug reports fill in the fields below.>>
The function scipy.stats.normaltest is intended to compute the D'Agostino K^2 ombinus test of normality.  However, it occasionally gives incorrect results due handling of a cube-root of negative numbers in scipy.stats.kurtosis.test
This issue originally came from: https://stats.stackexchange.com/questions/353756/normality-test-giving-confusing-results


### Reproducing code example:
```

#A detailed example is here: https://stats.stackexchange.com/questions/353756/normality-test-giving-confusing-results

#A possible fix for scipy.stats.kurtosistest
#(Please forgive the messiness - I'm a long-time R user but 1st-time Python user)
#Several lines of code in the function, then...
term1 = 1 - 2/(9 * A)

#New code begins here to address cube-roots of possibly negative numbers
def cubroot(y):
    from math import copysign
    return copysign(abs(y)**(1/3),y)
term2 = cubroot((1 - 2/A)/(1 + x * np.sqrt(2/(A - 4))))
Z= (term1 - term2)/ np.sqrt(2/(9.0*A)) # [1]_ Eq. 5
if Z.ndim == 0:
    Z = Z[()]
return KurtosistestResult(Z, 2 * distributions.norm.sf(np.abs(Z)))
<<A short code example that reproduces the problem. It should be self-contained, i.e., possible to run as-is via 'python myproblem.py'>>
```

### Error message:
```
#No error message, just inaccurate results
<<Full error message, if any (starting from line Traceback: ...)>>
```

### Scipy/Numpy/Python version information:
```
1.1.0 1.14.3 sys.version_info(major=3, minor=6, micro=5, releaselevel='final', serial=0)
<<Output from 'import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)'>>
```




Discussion:
Thanks for pointing it out. So you have an example where the test returns a wrong result. It would be good to have it as a test case once fixed ans to understand the issue better (i did not check it yet, i understand that the sign of the cubic root is wrong).
Just to add for clarity, the relevant equation is (19) on pg. 318 of D'Agostino et al. (1990) https://www.jstor.org/stable/pdf/2684359.pdf
When fixed, the results for the example (the "a" object in https://stats.stackexchange.com/questions/353756/normality-test-giving-confusing-results) should be something along the lines z for skew=-0.2300619, z for kurtosis=46.98108, chi squared= 2207.275, and p of approximately 0. 
yes, i agree that the implementation does not look correct. I could not Access the reference you gave but the following one worked:

> F. J. Anscombe, W. J. Glynn, “Distribution of the kurtosis statistic b2 for normal samples”, Biometrika, vol. 70, pp. 227-234, 1983.

In the end, the term `(1 - 2/A)/(1 + x * np.sqrt(2/(A - 4)))` can become negative and then `(...)**(1/3)` should simply be the cube root of it (again negative), right? Currently, the code sets the value to 0 and there is  comment on this in the documenttion:

> The Z-score is set to 0 for bad entries. 

(https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.kurtosistest.html)

The current code is
```
denom = 1 + x*np.sqrt(2/(A-4.0))
denom = np.where(denom < 0, 99, denom)
term2 = np.where(denom < 0, term1, np.power((1-2.0/A)/denom, 1/3.0))
Z = (term1 - term2) / np.sqrt(2/(9.0*A))  # [1]_ Eq. 5
Z = np.where(denom == 99, 0, Z)
```

I think it should be (same as in your answer):

```
denom = 1 + x*np.sqrt(2/(A-4.0))
term2 = np.where(denom < 0, -np.power(-(1-2.0/A)/denom, 1/3.0), np.power((1-2.0/A)/denom, 1/3.0))
Z = (term1 - term2) / np.sqrt(2/(9.0*A))  # [1]_ Eq. 5
```

Using git blame, the last time this code line with term2 was edited in cd4e901d6a6f38b9a843acc7dac6077bd568cc6b. If I have more time, I can check the history in more detail.
I like it.  It looks cleaner than my copysign solution, and  I think it will work so long as the numerator is always positive (so only the sign of denom matters).  It looks like the numerator will always be positive because "As n ranges from 4 upwards, A is smallest when n = 24, and then it is just over 18".  So that suggests 1-2/A is at lowest when 1-2/18 > 0, so the numerator is always positive.  Let me know if you'd like me to run some simulations to verify the accuracy, but just glancing at it, your suggested change looks right.
thanks, i tested the case `x = np.hstack([np.full(c, i) for i, c in enumerate([128, 0, 58, 7, 0, 41, 16, 0, 0, 167])])` from the stackexchange thread and the p-value is 0.0 with the above implementation.

Probably one needs to take care of the case `denom == 0`. And the test needs to be fixed in `mstats` as well.


Resolving patch:
diff --git a/scipy/stats/mstats_basic.py b/scipy/stats/mstats_basic.py
--- a/scipy/stats/mstats_basic.py
+++ b/scipy/stats/mstats_basic.py
@@ -2264,8 +2264,7 @@ def skewtest(a, axis=0):
     return SkewtestResult(Z, 2 * distributions.norm.sf(np.abs(Z)))
 
 
-KurtosistestResult = namedtuple('KurtosistestResult', ('statistic',
-                                                       'pvalue'))
+KurtosistestResult = namedtuple('KurtosistestResult', ('statistic', 'pvalue'))
 
 
 def kurtosistest(a, axis=0):
@@ -2314,11 +2313,12 @@ def kurtosistest(a, axis=0):
     denom = 1 + x*ma.sqrt(2/(A-4.0))
     if np.ma.isMaskedArray(denom):
         # For multi-dimensional array input
-        denom[denom < 0] = masked
-    elif denom < 0:
+        denom[denom == 0.0] = masked
+    elif denom == 0.0:
         denom = masked
 
-    term2 = ma.power((1-2.0/A)/denom,1/3.0)
+    term2 = np.ma.where(denom > 0, ma.power((1-2.0/A)/denom, 1/3.0),
+                        -ma.power(-(1-2.0/A)/denom, 1/3.0))
     Z = (term1 - term2) / np.sqrt(2/(9.0*A))
 
     return KurtosistestResult(Z, 2 * distributions.norm.sf(np.abs(Z)))
diff --git a/scipy/stats/stats.py b/scipy/stats/stats.py
--- a/scipy/stats/stats.py
+++ b/scipy/stats/stats.py
@@ -1379,8 +1379,7 @@ def kurtosistest(a, axis=0, nan_policy='propagate'):
 
     Notes
     -----
-    Valid only for n>20.  The Z-score is set to 0 for bad entries.
-    This function uses the method described in [1]_.
+    Valid only for n>20. This function uses the method described in [1]_.
 
     References
     ----------
@@ -1426,10 +1425,14 @@ def kurtosistest(a, axis=0, nan_policy='propagate'):
     A = 6.0 + 8.0/sqrtbeta1 * (2.0/sqrtbeta1 + np.sqrt(1+4.0/(sqrtbeta1**2)))
     term1 = 1 - 2/(9.0*A)
     denom = 1 + x*np.sqrt(2/(A-4.0))
-    denom = np.where(denom < 0, 99, denom)
-    term2 = np.where(denom < 0, term1, np.power((1-2.0/A)/denom, 1/3.0))
+    term2 = np.sign(denom) * np.where(denom == 0.0, np.nan,
+                                      np.power((1-2.0/A)/np.abs(denom), 1/3.0))
+    if np.any(denom == 0):
+        msg = "Test statistic not defined in some cases due to division by " \
+              "zero. Return nan in that case..."
+        warnings.warn(msg, RuntimeWarning)
+
     Z = (term1 - term2) / np.sqrt(2/(9.0*A))  # [1]_ Eq. 5
-    Z = np.where(denom == 99, 0, Z)
     if Z.ndim == 0:
         Z = Z[()]
 
@@ -1439,6 +1442,7 @@ def kurtosistest(a, axis=0, nan_policy='propagate'):
 
 NormaltestResult = namedtuple('NormaltestResult', ('statistic', 'pvalue'))
 
+
 def normaltest(a, axis=0, nan_policy='propagate'):
     """
     Test whether a sample differs from a normal distribution.


Test patch:
diff --git a/scipy/stats/tests/test_mstats_basic.py b/scipy/stats/tests/test_mstats_basic.py
--- a/scipy/stats/tests/test_mstats_basic.py
+++ b/scipy/stats/tests/test_mstats_basic.py
@@ -680,7 +680,7 @@ def test_plotting_positions():
 class TestNormalitytests():
 
     def test_vs_nonmasked(self):
-        x = np.array((-2,-1,0,1,2,3)*4)**2
+        x = np.array((-2, -1, 0, 1, 2, 3)*4)**2
         assert_array_almost_equal(mstats.normaltest(x),
                                   stats.normaltest(x))
         assert_array_almost_equal(mstats.skewtest(x),
@@ -705,7 +705,7 @@ def test_axis_None(self):
 
     def test_maskedarray_input(self):
         # Add some masked values, test result doesn't change
-        x = np.array((-2,-1,0,1,2,3)*4)**2
+        x = np.array((-2, -1, 0, 1, 2, 3)*4)**2
         xm = np.ma.array(np.r_[np.inf, x, 10],
                          mask=np.r_[True, [False] * x.size, True])
         assert_allclose(mstats.normaltest(xm), stats.normaltest(x))
@@ -713,7 +713,7 @@ def test_maskedarray_input(self):
         assert_allclose(mstats.kurtosistest(xm), stats.kurtosistest(x))
 
     def test_nd_input(self):
-        x = np.array((-2,-1,0,1,2,3)*4)**2
+        x = np.array((-2, -1, 0, 1, 2, 3)*4)**2
         x_2d = np.vstack([x] * 2).T
         for func in [mstats.normaltest, mstats.skewtest, mstats.kurtosistest]:
             res_1d = func(x)
@@ -733,6 +733,13 @@ def test_kurtosistest_result_attributes(self):
         attributes = ('statistic', 'pvalue')
         check_named_results(res, attributes, ma=True)
 
+    def regression_test_9033(self):
+        # x cleary non-normal but power of negtative denom needs
+        # to be handled correctly to reject normality
+        counts = [128, 0, 58, 7, 0, 41, 16, 0, 0, 167]
+        x = np.hstack([np.full(c, i) for i, c in enumerate(counts)])
+        assert_equal(mstats.kurtosistest(x)[1] < 0.01, True)
+
 
 class TestFOneway():
     def test_result_attributes(self):
diff --git a/scipy/stats/tests/test_stats.py b/scipy/stats/tests/test_stats.py
--- a/scipy/stats/tests/test_stats.py
+++ b/scipy/stats/tests/test_stats.py
@@ -3231,7 +3231,7 @@ def test_normalitytests():
     # numbers verified with R: dagoTest in package fBasics
     st_normal, st_skew, st_kurt = (3.92371918, 1.98078826, -0.01403734)
     pv_normal, pv_skew, pv_kurt = (0.14059673, 0.04761502, 0.98880019)
-    x = np.array((-2,-1,0,1,2,3)*4)**2
+    x = np.array((-2, -1, 0, 1, 2, 3)*4)**2
     attributes = ('statistic', 'pvalue')
 
     assert_array_almost_equal(stats.normaltest(x), (st_normal, pv_normal))
@@ -3282,6 +3282,12 @@ def test_normalitytests():
     assert_raises(ValueError, stats.normaltest, x, nan_policy='raise')
     assert_raises(ValueError, stats.normaltest, x, nan_policy='foobar')
 
+    # regression test for issue gh-9033: x cleary non-normal but power of
+    # negtative denom needs to be handled correctly to reject normality
+    counts = [128, 0, 58, 7, 0, 41, 16, 0, 0, 167]
+    x = np.hstack([np.full(c, i) for i, c in enumerate(counts)])
+    assert_equal(stats.kurtosistest(x)[1] < 0.01, True)
+
 
 class TestRankSums(object):
     def test_ranksums_result_attributes(self):