Improved tokenizer and indexer.