cleaning up test after implementation of tokenize(str:String, ...)

Along with #521, #523, fixes #1001866
Merge pull request #525 from huitseeker/issue/clean-up-tokenize-10018166

cleaning up test after implementation of tokenize(str:String, ...)