Rename _foo to _singleToken, add more test cases and fix leading character bug.