add documentation for deduplication