new weights and map feature for dedup