Refactored `tokenize` function.