Added unit tests to ensure that ngrams don't have dups removed by the NgramGenerator.java UDF.