Toy around with the idea of removing the ability to directly instantiate token types, but instead abstract that inside the token info definition. Split the scan function into smaller parts so it's more readable/testable
Add more operators; Is there a nicer way to define operators dynamically? IE, if it's in the info table, then it's fine to use and doesn't need the switch statement
Add comment token, which raises some interesting things to think about, IE looking further ahead. I added peekPeek() just for now