add switch to pass from old to new version of tokenize. New version is still work in prgress (tokenization needs to be changed)