changed to dictionary of dictionaries for probabilities hashing, threshold adjustment