Functional tokenizer