duplicates removed and aggregated