improve handling of duplicates