tokenize strings and apply naive bayes