Initial move to using tokenizer.
change code to use tokenize.js instead of string splitting