handles duplicates more efficently