Repository: pandas-dev/pandas

Issue:
PERF: .median(axis=1) perf issues
```
In [2]: df = pd.DataFrame(np.random.randn(10000, 2), columns=list('AB'))

In [3]: result1 = df.median(1)

In [4]: result2 = pd.Series(np.nanmedian(df.values, axis=1), index=df.index)

In [5]: result1.equals(result2)
Out[5]: True

In [6]: %timeit result1 = df.median(1)
241 µs ± 4.71 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [7]: %timeit df.median(1)
250 µs ± 5.74 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [8]: %timeit pd.Series(np.nanmedian(df.values, axis=1), index=df.index)
1.77 ms ± 32.7 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [9]: pd.set_option('use_bottleneck', False)

In [10]: result3 = df.median(1)

In [11]: result1.equals(result3)
Out[11]: True

In [12]: %timeit df.median(1)
317 ms ± 9.66 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

So, if *bottleneck* is installed, then ``df.median(1)`` is blazingly fast. However if its NOT installed (or not used), then we fallback to ``np.apply_along_axis(our_median_impl)``, so our median impl is pretty fast itself, but it only handles 1d, so this is a pythonic loop.

To fix we can use ``np.nanmedian`` soln if available (its in >= numpy 1.9, currently we support >= 1.7).


Discussion:
I get similar results 

```
>>> df = pd.DataFrame(np.random.randn(10000, 2), columns=list('AB'))

>>> pd.set_option('use_bottleneck', False)
>>> %timeit df.median(1) 
327 ms ± 4.51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>>> %timeit pd.Series(np.nanmedian(df.values, axis=1), index=df.index)
1.83 ms ± 123 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

>>> pd.set_option('use_bottleneck', True)
>>> %timeit df.median(1)
239 µs ± 2.23 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
```

Why isn't bottleneck a dependency of Pandas? I didn't even know I did not have it installed until now. Even when I set `pd.set_option('use_bottleneck', True)`, Pandas did not give me any warning that I did not have it installed.








see here: http://pandas.pydata.org/pandas-docs/stable/install.html#recommended-dependencies

these *could* be deps, but ``pip`` used to have trouble with these things and they didn't work on all platforms.

and https://github.com/pandas-dev/pandas/issues/9422, which bottleneck changed in 1.0 (breaking the previous, IMHO correct API).
in any event, this is easily fixed by using ``np.nanmedian`` as I said. (which again only recently came about in last 1-2 years).
okay, working on the fix
done: https://github.com/pandas-dev/pandas/pull/16509

Resolving patch:
diff --git a/pandas/core/nanops.py b/pandas/core/nanops.py
--- a/pandas/core/nanops.py
+++ b/pandas/core/nanops.py
@@ -6,7 +6,7 @@
 
 import numpy as np
 from pandas import compat
-from pandas._libs import tslib, algos, lib
+from pandas._libs import tslib, lib
 from pandas.core.dtypes.common import (
     _get_dtype,
     is_float, is_scalar,
@@ -370,14 +370,13 @@ def nanmean(values, axis=None, skipna=True):
 @bottleneck_switch()
 def nanmedian(values, axis=None, skipna=True):
 
-    values, mask, dtype, dtype_max = _get_values(values, skipna)
-
     def get_median(x):
         mask = notna(x)
         if not skipna and not mask.all():
             return np.nan
-        return algos.median(com._values_from_object(x[mask]))
+        return np.nanmedian(x[mask])
 
+    values, mask, dtype, dtype_max = _get_values(values, skipna)
     if not is_float_dtype(values):
         values = values.astype('f8')
         values[mask] = np.nan
@@ -389,10 +388,15 @@ def get_median(x):
 
     # an array from a frame
     if values.ndim > 1:
+
         # there's a non-empty array to apply over otherwise numpy raises
         if notempty:
-            return _wrap_results(
-                np.apply_along_axis(get_median, axis, values), dtype)
+            if not skipna:
+                return _wrap_results(
+                    np.apply_along_axis(get_median, axis, values), dtype)
+
+            # fastpath for the skipna case
+            return _wrap_results(np.nanmedian(values, axis), dtype)
 
         # must return the correct shape, but median is not defined for the
         # empty set so return nans of shape "everything but the passed axis"


Test patch: