refactor lexer
Refactoring lex to prepare for parsing, extracting concepts from the
parser initial ideas
converting lex tests to closer to the metal unit tests