add ability to disable sparse representation and set sort threshold based on the sparse set threshold