tokenize function passes all its unit tests