Lexer internal refactor; new TokenProcessor