Repository: numpy/numpy

Issue:
a possible MA regression
scipy tests started failing today with numpy from travis-dev-wheels. 
The failure is in mstats: https://travis-ci.org/scipy/scipy/builds/120882377

The failing test does this:

```
In [1]: from scipy.stats import mstats

In [2]: %paste
        x = [1, 3, 5, 7, 9]
        y = [2, 4, 6, 8, 10]
## -- End pasted text --

In [3]: mstats.kruskal(x, y)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-5edb46bc91da> in <module>()
----> 1 mstats.kruskal(x, y)

/home/br/virtualenvs/np/local/lib/python2.7/site-packages/scipy/stats/mstats_basic.pyc in kruskal(*args)
   1047     ngrp = ranks.count(-1)
   1048     ntot = ranks.count()
-> 1049     H = 12./(ntot*(ntot+1)) * (sumrk**2/ngrp).sum() - 3*(ntot+1)
   1050     # Tie correction
   1051     ties = count_tied_groups(ranks)

/home/br/repos/numpy/build/testenv/lib/python2.7/site-packages/numpy/ma/core.pyc in __truediv__(self, other)
   3983         if self._delegate_binop(other):
   3984             return NotImplemented
-> 3985         return true_divide(self, other)
   3986 
   3987     def __rtruediv__(self, other):

/home/br/repos/numpy/build/testenv/lib/python2.7/site-packages/numpy/ma/core.pyc in __call__(self, a, b, *args, **kwargs)
   1126         # Get the result
   1127         with np.errstate(divide='ignore', invalid='ignore'):
-> 1128             result = self.f(da, db, *args, **kwargs)
   1129         # Get the mask as a combination of the source masks and invalid
   1130         m = ~umath.isfinite(result)

ValueError: operands could not be broadcast together with shapes (2,) (2,5) 
```

and bisecting indicates

```
$ git bisect bad
36f76ea2e6e91062df12d3a46ccaed7822bc82f2 is the first bad commit
commit 36f76ea2e6e91062df12d3a46ccaed7822bc82f2
Author: Allan Haldane <allan.haldane@gmail.com>
Date:   Sun Feb 14 02:49:19 2016 -0500

    ENH: add extra kwargs and update doc of many MA methods

    Updated any, all, sum, prod, cumsum, cumprod, min, max, argmin, argmax,
    mean, var

:040000 040000 e41cfd58bb4bbabc11471a0bd443b24be91b45d0 7f99716492632508bc4d664d424c7cf22d6cd5cb M  numpy
```

which is, it seems, gh-5706?



Discussion:
@ahaldane can you have a look?

EDIT: I could even imagine that it is a scipy bug working around a not working keepdims, in which case we would have to see what the best solution is.

But this is cool that we now notice this stuff very soon :).

I see the bug, it's because my code for `count` doesn't handle negative axes properly. Eg

```
>>> a = np.ma.array([[1,2,3], [4,5,6]])
>>> a .count(-1)
array([[3, 3, 3],
       [3, 3, 3]])
```

Impressive that we and the test suit missed that ;).


Resolving patch:
diff --git a/numpy/ma/core.py b/numpy/ma/core.py
--- a/numpy/ma/core.py
+++ b/numpy/ma/core.py
@@ -4319,6 +4319,11 @@ def count(self, axis=None, keepdims=np._NoValue):
                 return self.size
 
             axes = axis if isinstance(axis, tuple) else (axis,)
+            axes = tuple(a if a >= 0 else self.ndim + a for a in axes)
+            if len(axes) != len(set(axes)):
+                raise ValueError("duplicate value in 'axis'")
+            if np.any([a < 0 or a >= self.ndim for a in axes]):
+                raise ValueError("'axis' entry is out of bounds")
             items = 1
             for ax in axes:
                 items *= self.shape[ax]


Test patch:
diff --git a/numpy/ma/tests/test_core.py b/numpy/ma/tests/test_core.py
--- a/numpy/ma/tests/test_core.py
+++ b/numpy/ma/tests/test_core.py
@@ -991,7 +991,7 @@ def test_count_func(self):
         res = count(ott, 0)
         assert_(isinstance(res, ndarray))
         assert_(res.dtype.type is np.intp)
-        assert_raises(IndexError, ott.count, axis=1)
+        assert_raises(ValueError, ott.count, axis=1)
 
     def test_minmax_func(self):
         # Tests minimum and maximum.
@@ -4312,6 +4312,9 @@ def test_count(self):
         assert_equal(count(a, keepdims=True), 16*ones((1,1,1)))
         assert_equal(count(a, axis=1, keepdims=True), 2*ones((2,1,4)))
         assert_equal(count(a, axis=(0,1), keepdims=True), 4*ones((1,1,4)))
+        assert_equal(count(a, axis=-2), 2*ones((2,4)))
+        assert_raises(ValueError, count, a, axis=(1,1))
+        assert_raises(ValueError, count, a, axis=3)
 
         # check the 'nomask' path
         a = np.ma.array(d, mask=nomask)
@@ -4322,6 +4325,9 @@ def test_count(self):
         assert_equal(count(a, keepdims=True), 24*ones((1,1,1)))
         assert_equal(count(a, axis=1, keepdims=True), 3*ones((2,1,4)))
         assert_equal(count(a, axis=(0,1), keepdims=True), 6*ones((1,1,4)))
+        assert_equal(count(a, axis=-2), 3*ones((2,4)))
+        assert_raises(ValueError, count, a, axis=(1,1))
+        assert_raises(ValueError, count, a, axis=3)
 
         # check the 'masked' singleton
         assert_equal(count(np.ma.masked), 0)