Repository: numpy/numpy

Issue:
IndexError with np.ma.median along axis on unmasked array
Something has changed between numpy 1.11 and the master branch, and it seems `np.ma.median` cannot handle an unmasked (`mask=False`) MaskedArray.

For example: `np.ma.median(np.ma.array(np.ones((5, 5))), axis=0)` throws an IndexError.

I haven't investigated it but to me it seems that `np.ma.array` creates a new mask (if the array hasn't got a mask already) and it is `False` (scalar). When using an axis with `np.ma.median` I suspect that the mask is applied to the data and then throws this Error:

```
IndexError                                Traceback (most recent call last)
<ipython-input-26-b5117c1f3918> in <module>()
----> 1 np.ma.median(np.ma.array(np.ones((5, 5))), axis=0)

C:\-\lib\site-packages\numpy\ma\extras.py in median(a, axis, out, overwrite_input, keepdims)
    693 
    694     r, k = _ureduce(a, func=_median, axis=axis, out=out,
--> 695                     overwrite_input=overwrite_input)
    696     if keepdims:
    697         return r.reshape(k)

C:\-\lib\site-packages\numpy\lib\function_base.py in _ureduce(a, func, **kwargs)
   3631         keepdim = [1] * a.ndim
   3632 
-> 3633     r = func(a, **kwargs)
   3634     return r, keepdim
   3635 

C:\-\lib\site-packages\numpy\ma\extras.py in _median(a, axis, out, overwrite_input)
    736             low = high
    737     else:
--> 738         low[odd] = high[odd]
    739 
    740     if np.issubdtype(asorted.dtype, np.inexact):

C:\-\lib\site-packages\numpy\ma\core.py in __getitem__(self, indx)
   3161 
   3162         """
-> 3163         dout = self.data[indx]
   3164         # We could directly use ndarray.__getitem__ on self.
   3165         # But then we would have to modify __array_finalize__ to prevent the

IndexError: in the future, 0-d boolean arrays will be interpreted as a valid boolean index
```

Which currently breaks some tests for astropy (see https://github.com/astropy/astropy/issues/5303).



Discussion:
Grrr, and I now get

```
In [1]: np.ma.median(np.ma.array(np.ones((5, 5))), axis=0)
/home/charris/.local/lib/python2.7/site-packages/numpy-1.11.0.dev0+88f5645-py2.7-linux-x86_64.egg/numpy/ma/core.py:3153: VisibleDeprecationWarning: using a boolean instead of an integer will result in an error in the future
  dout = self.data[indx]
/home/charris/.local/lib/python2.7/site-packages/numpy-1.11.0.dev0+88f5645-py2.7-linux-x86_64.egg/numpy/ma/core.py:3258: VisibleDeprecationWarning: using a boolean instead of an integer will result in an error in the future
  _data[indx] = dval
```

in the coming 1.11.x branch.

As a side note

```
In [6]: np.median(np.ma.array(np.ones((5, 5))), axis=0)
Out[6]: 
masked_array(data = [ 1.  1.  1.  1.  1.],
             mask = False,
       fill_value = 1e+20)
```

so the problem is specific to the masked version of median.

@charris Thank you for the fast reply. 

I think `np.median` has no problems there because it simply ignores the mask (#7330).

`_median` has got to be one of the ugliest functions in numpy.

OK, immediate fix was easy, but there is another oddity

```
In [5]: np.ma.median(np.ma.array(np.ones((5, 5))), axis=1)
```

raises an axis out of bounds error.


Resolving patch:
diff --git a/numpy/ma/extras.py b/numpy/ma/extras.py
--- a/numpy/ma/extras.py
+++ b/numpy/ma/extras.py
@@ -708,34 +708,37 @@ def _median(a, axis=None, out=None, overwrite_input=False):
             asorted = a
     else:
         asorted = sort(a, axis=axis)
+
     if axis is None:
         axis = 0
     elif axis < 0:
-        axis += a.ndim
+        axis += asorted.ndim
 
     if asorted.ndim == 1:
         idx, odd = divmod(count(asorted), 2)
-        return asorted[idx - (not odd) : idx + 1].mean()
+        return asorted[idx + odd - 1 : idx + 1].mean(out=out)
 
-    counts = asorted.shape[axis] - (asorted.mask).sum(axis=axis)
+    counts = count(asorted, axis=axis)
     h = counts // 2
+
     # create indexing mesh grid for all but reduced axis
     axes_grid = [np.arange(x) for i, x in enumerate(asorted.shape)
                  if i != axis]
     ind = np.meshgrid(*axes_grid, sparse=True, indexing='ij')
+
     # insert indices of low and high median
     ind.insert(axis, h - 1)
     low = asorted[tuple(ind)]
     low._sharedmask = False
     ind[axis] = h
     high = asorted[tuple(ind)]
+
     # duplicate high if odd number of elements so mean does nothing
     odd = counts % 2 == 1
-    if asorted.ndim == 1:
-        if odd:
-            low = high
-    else:
-        low[odd] = high[odd]
+    if asorted.ndim > 1:
+        np.copyto(low, high, where=odd)
+    elif odd:
+        low = high
 
     if np.issubdtype(asorted.dtype, np.inexact):
         # avoid inf / x = masked


Test patch:
diff --git a/numpy/ma/tests/test_extras.py b/numpy/ma/tests/test_extras.py
--- a/numpy/ma/tests/test_extras.py
+++ b/numpy/ma/tests/test_extras.py
@@ -10,6 +10,7 @@
 from __future__ import division, absolute_import, print_function
 
 import warnings
+import itertools
 
 import numpy as np
 from numpy.testing import (
@@ -684,6 +685,37 @@ def test_docstring_examples(self):
         assert_equal(ma_x.shape, (2,), "shape mismatch")
         assert_(type(ma_x) is MaskedArray)
 
+    def test_axis_argument_errors(self):
+        msg = "mask = %s, ndim = %s, axis = %s, overwrite_input = %s"
+        for ndmin in range(5):
+            for mask in [False, True]:
+                x = array(1, ndmin=ndmin, mask=mask)
+
+                # Valid axis values should not raise exception
+                args = itertools.product(range(-ndmin, ndmin), [False, True])
+                for axis, over in args:
+                    try:
+                        np.ma.median(x, axis=axis, overwrite_input=over)
+                    except:
+                        raise AssertionError(msg % (mask, ndmin, axis, over))
+
+                # Invalid axis values should raise exception
+                args = itertools.product([-(ndmin + 1), ndmin], [False, True])
+                for axis, over in args:
+                    try:
+                        np.ma.median(x, axis=axis, overwrite_input=over)
+                    except IndexError:
+                        pass
+                    else:
+                        raise AssertionError(msg % (mask, ndmin, axis, over))
+
+    def test_masked_0d(self):
+        # Check values
+        x = array(1, mask=False)
+        assert_equal(np.ma.median(x), 1)
+        x = array(1, mask=True)
+        assert_equal(np.ma.median(x), np.ma.masked)
+
     def test_masked_1d(self):
         x = array(np.arange(5), mask=True)
         assert_equal(np.ma.median(x), np.ma.masked)