Clean up lexer
Generalize token test so it can be extended later.