dedupe: Improved the efficency of the code