Added functionality for deduplication