"refactored tokenization for correctenss and genericity "