Clean up some hotspots and simplify sort order calculation