use multi-CPUs for extracting duplicates