Fixed a bug Tokenizer#tokenize() throws NPE in search mode.
Passed test cases in SearchTokenizerTest (remove @Ignore annotation).