!fixup making changes to token.* per suggestions
!fixup making changes to lexer.* per suggestions