Added an overridable `on_token` hook to Lexer
Merged in changes from my compilers project