Added better testing strategy for lexer. Cleaned up the lexer via some macros.
moved up