Nearly have hash based duplicate identification working