future enhancement: duplicates in datasets