Making more of the deduplication algorithm