Refactoring punctation tokenizer for tokenize lexicon generation.
Lint improvements for tokenizer_unigram.