Improved deduplication algorithm