Remove sorted set from the list