fix bugs and add tests for view::tokenize