changed old functions to use new tokenize and clean