Added tokenizing function.