updated to deal the duplicates