Some examples of using Lucene for tokenization in various languages.