adapted dfm to use new tokenize and clean