Add more tests to decimal dense sorting
Minor bug in dense sorting