Repository: scipy/scipy

Issue:
Different data prerequisites for stats.bartlett and stats.levene
So while i am capable of doing the bartlett test with two numpy-arrays of dimensions (83,10), i can't do the levene test with the exact same arrays. I receive the following error when calling stats.levene:

 File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/scipy/stats/morestats.py", line 1944, in levene
    Yci[j] = func(args[j])
ValueError: setting an array element with a sequence.

Code to reproduce:
> from scipy import stats
> import numpy as np
> 
> test_a = np.random.beta(0.87, 0.77, (83,10))
> test_b = np.random.beta(0.87, 0.77, (83,10))
> 
> print(stats.bartlett(test_a, test_b))
> print(stats.levene(test_a, test_b))


Discussion:
What version of scipy did you use?
Sorry i forgot that. Current version 1.1.0.
yes, looks like `levene` can only handle 1d inputs, simplifying the code a bit, one the error is due to `axis=0` when defining `func`. 
```
k = len(args)
Ni = zeros(k)
Yci = zeros(k, 'd')
func = lambda x: np.median(x, axis=0)

for j in range(k):
    Ni[j] = len(args[j]) 
    Yci[j] = func(args[j]) 
```

`func(args[j])` has shape (10, ) for test_a, test_b, hence, cannot be assigned to the array element. `axis=0` is used in other parts of the function as well.
the test is well-explained in one of the references, that should help to fix it

https://www.itl.nist.gov/div898/handbook/eda/section3/eda35a.htm
btw, I doubt that the results of `bartlett` is correct. Shouldn't this give the same result?

```
stats.bartlett(test_a, test_b)
stats.bartlett(test_a.flatten(), test_b.flatten())
```

`len(args[j])` gives 83 in the first case, but there are 830 elements. Or do I miss sth?
> Shouldn't this give the same result?

No, it shouldn't.
scipy.stats doesn't flatten anything unless axis=None.

I think in this case, the standard numpy broadcasting should apply. Each column of the first array is compared with the same column of the second array. The result should be vectorized across columns.
If this is not the case, then the broadcasting in bartlett doesn't allow correctly for arrays with dimension greater than 1.

(you could check t-test to see whether and how it works there.)



If i understand the bartlett test correctly, it should either not allow my input or flatten it (in this case this should maybe be mention in documentation?).
AFAICS, this is more likely a bug, i.e. bartlett doesn't have consistent and correct behavior for 2-d inputs.

`var` is missing an axis keyword
https://github.com/scipy/scipy/blob/8f81b305ab3562f295422a7a5fdc27eec0e5f776/scipy/stats/morestats.py#L1866

incomplete review of functions for "unexpected" inputs

> If i understand the bartlett test correctly, it should either not allow my input or flatten it

In many cases it is useful to run a vectorized test on all columns at the same time., e.g. many mean comparisons in ttest_indep, so the default for vectorized operations in scipy.stats is `axis=0` in contrast to numpy's default `axis=None` and ravelling.
(In stats we don't want to ravel heterogeneous columns in a "DataFrame", e.g. what's the variance if column 0 is price and column 1 is quantity purchased.)
Given the current implementation, both functions only work for 1d input (`bartlett` returns a result for 2d input as well but it is not correct).

There are multiple options to fix this:

- treat each sample as a separate input and apply `var`, `mean` to it (equivalent to axis=None) (this is the behaviour the user expects from the current doc I guess, requires minor adjustment of the code)
- allow only 1d input (documentation needs to be adjusted to say so, no need to adjust the code)
- add axis keyword which would require more work and testing

While there is an axis keyword for the t-test, most other test don't have it (KS, wilcoxon, shapiro, anderson_ksample), so there is no consistent treatment. In particular, `anderson_ksample` requires the k samples to be combined in a list instead of using `*args`.
I think it's better to stick with axis=0 default in stats and raise otherwise.

I think many tests have most likely not been reviewed for multi-dimensional behavior.

to the items
1. I would never expect scipy.stats functions to ravel by default, because it goes against the pre-existing convention.
2. I think this is the best short term solution. If 2-d with axis=0 is incorrect, then raise.
3. This is the best long term solution, if the function naturally vectorizes. If the algorithm can only handle 1-D, then I don't see much point to include a python loop to artificially vectorize. (using cython or numba would be different)


Resolving patch:
diff --git a/scipy/stats/morestats.py b/scipy/stats/morestats.py
--- a/scipy/stats/morestats.py
+++ b/scipy/stats/morestats.py
@@ -1813,7 +1813,8 @@ def bartlett(*args):
     Parameters
     ----------
     sample1, sample2,... : array_like
-        arrays of sample data.  May be different lengths.
+        arrays of sample data.  Only 1d arrays are accepted, they may have
+        different lengths.
 
     Returns
     -------
@@ -1832,7 +1833,8 @@ def bartlett(*args):
     Conover et al. (1981) examine many of the existing parametric and
     nonparametric tests by extensive simulations and they conclude that the
     tests proposed by Fligner and Killeen (1976) and Levene (1960) appear to be
-    superior in terms of robustness of departures from normality and power [3]_.
+    superior in terms of robustness of departures from normality and power
+    ([3]_).
 
     References
     ----------
@@ -1851,10 +1853,12 @@ def bartlett(*args):
            Mathematical and Physical Sciences, Vol. 160, No.901, pp. 268-282.
 
     """
-    # Handle empty input
+    # Handle empty input and input that is not 1d
     for a in args:
         if np.asanyarray(a).size == 0:
             return BartlettResult(np.nan, np.nan)
+        if np.asanyarray(a).ndim > 1:
+            raise ValueError('Samples must be one-dimensional.')
 
     k = len(args)
     if k < 2:
@@ -1890,7 +1894,8 @@ def levene(*args, **kwds):
     Parameters
     ----------
     sample1, sample2, ... : array_like
-        The sample data, possibly with different lengths
+        The sample data, possibly with different lengths. Only one-dimensional
+        samples are accepted.
     center : {'mean', 'median', 'trimmed'}, optional
         Which function of the data to use in the test.  The default
         is 'median'.
@@ -1915,6 +1920,11 @@ def levene(*args, **kwds):
       * 'mean' : Recommended for symmetric, moderate-tailed distributions.
       * 'trimmed' : Recommended for heavy-tailed distributions.
 
+    The test version using the mean was proposed in the original article
+    of Levene ([2]_) while the median and trimmed mean have been studied by
+    Brown and Forsythe ([3]_), sometimes also referred to as Brown-Forsythe
+    test.
+
     References
     ----------
     .. [1]  https://www.itl.nist.gov/div898/handbook/eda/section3/eda35a.htm
@@ -1940,12 +1950,17 @@ def levene(*args, **kwds):
     k = len(args)
     if k < 2:
         raise ValueError("Must enter at least two input sample vectors.")
+    # check for 1d input
+    for j in range(k):
+        if np.asanyarray(args[j]).ndim > 1:
+            raise ValueError('Samples must be one-dimensional.')
+
     Ni = zeros(k)
     Yci = zeros(k, 'd')
 
     if center not in ['mean', 'median', 'trimmed']:
         raise ValueError("Keyword argument <center> must be 'mean', 'median'"
-                        " or 'trimmed'.")
+                         " or 'trimmed'.")
 
     if center == 'median':
         func = lambda x: np.median(x, axis=0)


Test patch:
diff --git a/scipy/stats/tests/test_morestats.py b/scipy/stats/tests/test_morestats.py
--- a/scipy/stats/tests/test_morestats.py
+++ b/scipy/stats/tests/test_morestats.py
@@ -27,6 +27,8 @@
     have_matplotlib = False
 
 
+# test data gear.dat from NIST for Levene and Bartlett test
+# https://www.itl.nist.gov/div898/handbook/eda/section3/eda3581.htm
 g1 = [1.006, 0.996, 0.998, 1.000, 0.992, 0.993, 1.002, 0.999, 0.994, 1.000]
 g2 = [0.998, 1.006, 1.000, 1.002, 0.997, 0.998, 0.996, 1.000, 1.006, 0.988]
 g3 = [0.991, 0.987, 0.997, 0.999, 0.995, 0.994, 1.000, 0.999, 0.996, 0.996]
@@ -502,6 +504,7 @@ def test_result_attributes(self):
 class TestBartlett(object):
 
     def test_data(self):
+        # https://www.itl.nist.gov/div898/handbook/eda/section3/eda357.htm
         args = [g1, g2, g3, g4, g5, g6, g7, g8, g9, g10]
         T, pval = stats.bartlett(*args)
         assert_almost_equal(T, 20.78587342806484, 7)
@@ -521,10 +524,16 @@ def test_empty_arg(self):
         args = (g1, g2, g3, g4, g5, g6, g7, g8, g9, g10, [])
         assert_equal((np.nan, np.nan), stats.bartlett(*args))
 
+    # temporary fix for issue #9252: only accept 1d input
+    def test_1d_input(self):
+        x = np.array([[1, 2], [3, 4]])
+        assert_raises(ValueError, stats.bartlett, g1, x)
+
 
 class TestLevene(object):
 
     def test_data(self):
+        # https://www.itl.nist.gov/div898/handbook/eda/section3/eda35a.htm
         args = [g1, g2, g3, g4, g5, g6, g7, g8, g9, g10]
         W, pval = stats.levene(*args)
         assert_almost_equal(W, 1.7059176930008939, 7)
@@ -584,6 +593,11 @@ def test_result_attributes(self):
         attributes = ('statistic', 'pvalue')
         check_named_results(res, attributes)
 
+    # temporary fix for issue #9252: only accept 1d input
+    def test_1d_input(self):
+        x = np.array([[1, 2], [3, 4]])
+        assert_raises(ValueError, stats.levene, g1, x)
+
 
 class TestBinomP(object):