Added deduplicate functionality