Repository: scipy/scipy

Issue:
Scipy Stats ppf for Lognormal fails on array-like 'loc' or 'scale' (Trac #1131)
_Original ticket http://projects.scipy.org/scipy/ticket/1131 on 2010-03-15 by trac user catintp, assigned to unknown._

The user code attached below causes an error in scipy, because the code does not appear to be able to handle array-like inputs for either "loc" or "scale".  Fix requires changing either the code or the documentation and docstring for the functions/methods.

The trouble starts in valarray() in distributions.py in stats, which calls repeat([value],product(shape-of-cond)) with a big array. This causes reshape() in fromnumeric.py in numpy.core to throw an exception due to trying to fit 999x999x1 data elements into a 999x1 array.

(a) The call to valarray() to set up the output array by replicating the value of the expression "self.a*scale+loc" produces an array with shape=(999,999,1), which the code tries to resize to (999,1). 

(b) Immediately after the output array is initialized, there are two calls to place(). The first one inserts self.badvalue in rows with invalid inputs. The second one attempts to insert a placeholder value equal to "self.b*scale+loc" whenever q==1 (and everything else OK). When either scale or loc is an array-like, you get an error condition that this expression is too deep to insert using place().

I would be available to discuss this further, or to delete this issue if it has already been reported/fixed.  I could not figure out how to search for an existing report on this issue, but I will learn before I submit anything else.

Charles the clueless newbie



Discussion:

Resolving patch:
diff --git a/scipy/stats/stats.py b/scipy/stats/stats.py
--- a/scipy/stats/stats.py
+++ b/scipy/stats/stats.py
@@ -2921,8 +2921,8 @@ def mergesort(offs, length):
     v += ((n - first) * (n - first - 1)) // 2
 
     tot = (n * (n - 1)) // 2
-    if tot == u and tot == v:
-        return 1    # Special case for all ties in both ranks
+    if tot == u or tot == v:
+        return (np.nan, np.nan)    # Special case for all ties in both ranks
 
     # Prevent overflow; equal to np.sqrt((tot - u) * (tot - v))
     denom = np.exp(0.5 * (np.log(tot - u) + np.log(tot - v)))


Test patch:
diff --git a/scipy/stats/tests/test_stats.py b/scipy/stats/tests/test_stats.py
--- a/scipy/stats/tests/test_stats.py
+++ b/scipy/stats/tests/test_stats.py
@@ -610,8 +610,6 @@ def test_tie1(self):
 
 @dec.knownfailureif(sys.version[:3] < '2.5', "Can't index array with np.int64")
 def test_kendalltau():
-    """Some tests for kendalltau."""
-
     # with some ties
     x1 = [12, 2, 1, 12, 2]
     x2 = [1, 4, 7, 1, 0]
@@ -620,6 +618,11 @@ def test_kendalltau():
     assert_approx_equal(res[0], expected[0])
     assert_approx_equal(res[1], expected[1])
 
+    # with only ties in one or both inputs
+    assert_(np.all(np.isnan(stats.kendalltau([2,2,2], [2,2,2]))))
+    assert_(np.all(np.isnan(stats.kendalltau([2,0,2], [2,2,2]))))
+    assert_(np.all(np.isnan(stats.kendalltau([2,2,2], [2,0,2]))))
+
     # check two different sort methods
     assert_approx_equal(stats.kendalltau(x1, x2, initial_lexsort=False)[1],
                         stats.kendalltau(x1, x2, initial_lexsort=True)[1])