tokenizer functions docs.