Add custom deduper for commoncrawl