moved tokenizing to new file