I'm having a problem with the idea of stripping the NaNs, sorting the non-NaNs, and constructing the return result. If I do that instead of generating `indexer` then I'd lose the ability to do the sort in-place.