Revert "A bug introduced in 43c7b9e means the secondary sort order is lost"

This reverts commit 80362f3a4a2a7eda7e05bdf3bfba6c47591a460c.
Consolidate sorting into one place