Changed token to use explicit TokenKinds for all tokens, none of this hinky using-a-character-as-its-token-value bizzo.
Added some better tests for the lexer.
Fixed a variety of small bugs in the lexer. It now seems to work pretty well.