generate lucene index from ngrams