Fix alpha sorter to not destroy other sorts.

From #4857.