Repository: scipy/scipy

Issue:
boxcox transform edge cases could use more care
See http://stackoverflow.com/questions/31833962.

According to the scipy docs, the `boxcox` data transform tries to find a maximum likelihood value of lambda if none is provided. But it doesn't seem to handle edge cases or bad inputs well.

``` python
>>> import numpy as np
>>> from scipy.stats import boxcox                                              
>>> boxcox(np.array([1.0, 1.0, 2.0, 3.0, 5.0])) # maybe OK? But docs say lambda is non-negative...
(array([ 0.        ,  0.        ,  0.63539331,  0.95825835,  1.32030256]), -0.25477193181221375)
>>> boxcox(np.array([[1.0, 1.0, 2.0, 3.0, 5.0]])) # traceback in the optimization
>>> boxcox(np.array([1.0, 1.0, 1.0])) # low-level warnings, and an arbitrary lambda is returned
```



Discussion:
@argriffing wrote:

>
    >>> boxcox(np.array([1.0, 1.0, 2.0, 3.0, 5.0])) # maybe OK? But docs say lambda is non-negative...
    (array([ 0.        ,  0.        ,  0.63539331,  0.95825835,  1.32030256]), -0.25477193181221375)
This is OK.  While the input array `x` must have strictly positive values, the transformed array may have values less than or equal to zero, and lambda can be negative.

The first line of the docstring is

    Return a positive dataset transformed by a Box-Cox power transformation.

which might be confusing.  The data that is *returned* is not necessarily positive; it is the input that must be positive.  This could be rephrased to be less ambiguous.

The second example is

    >>> boxcox(np.array([[1.0, 1.0, 2.0, 3.0, 5.0]])) # traceback in the optimization

That still raises an exception:

    ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The description of `x` in the docstring says "Should be 1-dimensional" and  in fact, passing a 2-d array causes that error.  It would be nice if the function validated the shape, and gave a more informative error message in the exception.  And really, the docstring should be edited to replace "Should be..." with "Must be...".

The input in the last example is an array in which all the values are the same:

    >>> boxcox(np.array([1.0, 1.0, 1.0])) # low-level warnings, and an arbitrary lambda is returned

The behavior hasn't changed:

```
In [9]: boxcox(np.array([1.0, 1.0, 1.0]))                                       
/Users/warren/miniconda36/lib/python3.6/site-packages/scipy-1.3.0.dev0_-py3.6-macosx-10.7-x86_64.egg/scipy/stats/morestats.py:910: RuntimeWarning: divide by zero encountered in log
  return (lmb - 1) * np.sum(logdata, axis=0) - N/2 * np.log(variance)
/Users/warren/miniconda36/lib/python3.6/site-packages/scipy-1.3.0.dev0_-py3.6-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py:1959: RuntimeWarning: invalid value encountered in double_scalars
  tmp1 = (x - w) * (fx - fv)
/Users/warren/miniconda36/lib/python3.6/site-packages/scipy-1.3.0.dev0_-py3.6-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py:1960: RuntimeWarning: invalid value encountered in double_scalars
  tmp2 = (x - v) * (fx - fw)
Out[9]: (array([0., 0., 0.]), 4.208073920827584)
```
The log likelihood function computes the log of the variance of the transformed data.  If the data is constant, the variance is 0, so computing the log of the variance generates an error.

If the input array is constant, the function should probably just raise an exception.  The Box-Cox transformation tries to make the data more normal.  If the input values are all the same, there is nothing it can do.

I've tagged this as a "good first issue" because the changes needed (in my opinion, anyway) are pretty clear:
* Improve the docstring to remove any ambiguity or possible confusion.
* Raise an error with a useful message if `x.ndim` is not 1.
* Raise an error with a useful message if all the values in `x` are the same.


Resolving patch:
diff --git a/scipy/stats/morestats.py b/scipy/stats/morestats.py
--- a/scipy/stats/morestats.py
+++ b/scipy/stats/morestats.py
@@ -947,12 +947,12 @@ def rootfunc(lmbda, data, target):
 
 def boxcox(x, lmbda=None, alpha=None):
     r"""
-    Return a positive dataset transformed by a Box-Cox power transformation.
+    Return a dataset transformed by a Box-Cox power transformation.
 
     Parameters
     ----------
     x : ndarray
-        Input array.  Should be 1-dimensional.
+        Input array.  Must be positive 1-dimensional.  Must not be constant.
     lmbda : {None, scalar}, optional
         If `lmbda` is not None, do the transformation for that value.
 
@@ -1032,9 +1032,15 @@ def boxcox(x, lmbda=None, alpha=None):
 
     """
     x = np.asarray(x)
+    if x.ndim != 1:
+        raise ValueError("Data must be 1-dimensional.")
+
     if x.size == 0:
         return x
 
+    if np.all(x == x[0]):
+        raise ValueError("Data must not be constant.")
+
     if any(x <= 0):
         raise ValueError("Data must be positive.")
 


Test patch:
diff --git a/scipy/stats/tests/test_morestats.py b/scipy/stats/tests/test_morestats.py
--- a/scipy/stats/tests/test_morestats.py
+++ b/scipy/stats/tests/test_morestats.py
@@ -1320,8 +1320,12 @@ def test_alpha(self):
 
     def test_boxcox_bad_arg(self):
         # Raise ValueError if any data value is negative.
-        x = np.array([-1])
+        x = np.array([-1, 2])
         assert_raises(ValueError, stats.boxcox, x)
+        # Raise ValueError if data is constant.
+        assert_raises(ValueError, stats.boxcox, np.array([1]))
+        # Raise ValueError if data is not 1-dimensional.
+        assert_raises(ValueError, stats.boxcox, np.array([[1], [2]]))
 
     def test_empty(self):
         assert_(stats.boxcox([]).shape == (0,))