extract methods of tokenize and evaluate