Repository: numpy/numpy

Issue:
nanpercentile issues
`nanpercentile` crashes when an slice is filled with `nans` and multiple percentiles are requested:

```
In [9]: np.nanpercentile([[np.nan, np.nan], [np.nan, 2]], [50, 100], axis=0)
/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py:914: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-f425a58093df> in <module>()
----> 1 np.nanpercentile([[np.nan, np.nan], [np.nan, 2]], [50, 100], axis=0)

/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py in nanpercentile(a, q, axis, out, overwrite_input, interpolation, keepdims)
    873     r, k = _ureduce(a, func=_nanpercentile, q=q, axis=axis, out=out,
    874                     overwrite_input=overwrite_input,
--> 875                     interpolation=interpolation)
    876     if keepdims:
    877         if q.ndim == 0:

/usr/lib/python3.4/site-packages/numpy/lib/function_base.py in _ureduce(a, func, **kwargs)
   2801         keepdim = [1] * a.ndim
   2802 
-> 2803     r = func(a, **kwargs)
   2804     return r, keepdim
   2805 

/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py in _nanpercentile(a, q, axis, out, overwrite_input, interpolation, keepdims)
    896     else:
    897         result = np.apply_along_axis(_nanpercentile1d, axis, a, q,
--> 898                                      overwrite_input, interpolation)
    899 
    900     if out is not None:

/usr/lib/python3.4/site-packages/numpy/lib/shape_base.py in apply_along_axis(func1d, axis, arr, *args, **kwargs)
    106             i.put(indlist, ind)
    107             res = func1d(arr[tuple(i.tolist())], *args, **kwargs)
--> 108             outarr[tuple(ind)] = res
    109             k += 1
    110         return outarr

ValueError: setting an array element with a sequence.
```

It also sometimes returns an incorrectly sized array (compare with `np.percentile`):

```
In [12]: np.nanpercentile([[np.nan, np.nan]], [50, 100], axis=0)
/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py:914: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
Out[12]: array([ nan,  nan])

In [13]: np.percentile([[1, 2]], [50, 100], axis=0)
Out[13]: 
array([[ 1.,  2.],
       [ 1.,  2.]])
```

Finally, the function also seems not mentioned at all in the official docs (except in the release notes).

nanpercentile issues
`nanpercentile` crashes when an slice is filled with `nans` and multiple percentiles are requested:

```
In [9]: np.nanpercentile([[np.nan, np.nan], [np.nan, 2]], [50, 100], axis=0)
/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py:914: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-9-f425a58093df> in <module>()
----> 1 np.nanpercentile([[np.nan, np.nan], [np.nan, 2]], [50, 100], axis=0)

/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py in nanpercentile(a, q, axis, out, overwrite_input, interpolation, keepdims)
    873     r, k = _ureduce(a, func=_nanpercentile, q=q, axis=axis, out=out,
    874                     overwrite_input=overwrite_input,
--> 875                     interpolation=interpolation)
    876     if keepdims:
    877         if q.ndim == 0:

/usr/lib/python3.4/site-packages/numpy/lib/function_base.py in _ureduce(a, func, **kwargs)
   2801         keepdim = [1] * a.ndim
   2802 
-> 2803     r = func(a, **kwargs)
   2804     return r, keepdim
   2805 

/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py in _nanpercentile(a, q, axis, out, overwrite_input, interpolation, keepdims)
    896     else:
    897         result = np.apply_along_axis(_nanpercentile1d, axis, a, q,
--> 898                                      overwrite_input, interpolation)
    899 
    900     if out is not None:

/usr/lib/python3.4/site-packages/numpy/lib/shape_base.py in apply_along_axis(func1d, axis, arr, *args, **kwargs)
    106             i.put(indlist, ind)
    107             res = func1d(arr[tuple(i.tolist())], *args, **kwargs)
--> 108             outarr[tuple(ind)] = res
    109             k += 1
    110         return outarr

ValueError: setting an array element with a sequence.
```

It also sometimes returns an incorrectly sized array (compare with `np.percentile`):

```
In [12]: np.nanpercentile([[np.nan, np.nan]], [50, 100], axis=0)
/usr/lib/python3.4/site-packages/numpy/lib/nanfunctions.py:914: RuntimeWarning: All-NaN slice encountered
  warnings.warn("All-NaN slice encountered", RuntimeWarning)
Out[12]: array([ nan,  nan])

In [13]: np.percentile([[1, 2]], [50, 100], axis=0)
Out[13]: 
array([[ 1.,  2.],
       [ 1.,  2.]])
```

Finally, the function also seems not mentioned at all in the official docs (except in the release notes).



Discussion:

Resolving patch:
diff --git a/numpy/lib/nanfunctions.py b/numpy/lib/nanfunctions.py
--- a/numpy/lib/nanfunctions.py
+++ b/numpy/lib/nanfunctions.py
@@ -975,7 +975,11 @@ def _nanpercentile(a, q, axis=None, out=None, overwrite_input=False,
     else:
         result = np.apply_along_axis(_nanpercentile1d, axis, a, q,
                                      overwrite_input, interpolation)
-
+        # apply_along_axis fills in collapsed axis with results.
+        # Move that axis to the beginning to match percentile's
+        # convention.
+        if q.ndim != 0:
+            result = np.swapaxes(result, 0, axis)
     if out is not None:
         out[...] = result
     return result
@@ -991,7 +995,10 @@ def _nanpercentile1d(arr1d, q, overwrite_input=False, interpolation='linear'):
     s = np.where(c)[0]
     if s.size == arr1d.size:
         warnings.warn("All-NaN slice encountered", RuntimeWarning)
-        return np.nan
+        if q.ndim == 0:
+            return np.nan
+        else:
+            return np.nan * np.ones((len(q),))
     elif s.size == 0:
         return np.percentile(arr1d, q, overwrite_input=overwrite_input,
                              interpolation=interpolation)


Test patch:
diff --git a/numpy/lib/tests/test_nanfunctions.py b/numpy/lib/tests/test_nanfunctions.py
--- a/numpy/lib/tests/test_nanfunctions.py
+++ b/numpy/lib/tests/test_nanfunctions.py
@@ -654,7 +654,8 @@ def test_result_values(self):
         tgt = [np.percentile(d, 28) for d in _rdat]
         res = np.nanpercentile(_ndat, 28, axis=1)
         assert_almost_equal(res, tgt)
-        tgt = [np.percentile(d, (28, 98)) for d in _rdat]
+        # Transpose the array to fit the output convention of numpy.percentile
+        tgt = np.transpose([np.percentile(d, (28, 98)) for d in _rdat])
         res = np.nanpercentile(_ndat, (28, 98), axis=1)
         assert_almost_equal(res, tgt)
 
@@ -702,6 +703,29 @@ def test_extended_axis_invalid(self):
         assert_raises(IndexError, np.nanpercentile, d, q=5, axis=(0, 4))
         assert_raises(ValueError, np.nanpercentile, d, q=5, axis=(1, 1))
 
+    def test_multiple_percentiles(self):
+        perc = [50, 100]
+        mat = np.ones((4, 3))
+        nan_mat = np.nan * mat
+        # For checking consistency in higher dimensional case
+        large_mat = np.ones((3, 4, 5))
+        large_mat[:, 0:2:4, :] = 0
+        large_mat[:, :, 3:] = 2*large_mat[:, :, 3:]
+        for axis in [None, 0, 1]:
+            for keepdim in [False, True]:
+                with warnings.catch_warnings(record=True) as w:
+                    warnings.simplefilter('always')
+                    val = np.percentile(mat, perc, axis=axis, keepdims=keepdim)
+                    nan_val = np.nanpercentile(nan_mat, perc, axis=axis,
+                                               keepdims=keepdim)
+                    assert_equal(nan_val.shape, val.shape)
+
+                    val = np.percentile(large_mat, perc, axis=axis,
+                                        keepdims=keepdim)
+                    nan_val = np.nanpercentile(large_mat, perc, axis=axis,
+                                               keepdims=keepdim)
+                    assert_equal(nan_val, val)
+
 
 if __name__ == "__main__":
     run_module_suite()