Implements duplicate and possible duplicate detection.