Implemented tokenizer and document tokenizer.