Fix sorting of NaN values in terms aggregations.

Closes #5236.
Added unit tests for terms and histogram sorting on sub aggregations that return NaN.