Implement getTokenType() and corresponding test cases.