Optimisation:

Keep everything a float.
Use select_s instead of sorting to get the median.
This should become substantially faster now, I measured five times faster.
BUG fix: really choose the median of the array.