bring back the hamming distance check before doing levenshtein (if hamming distance is > 1/2 of the smallest token)