refactoring of lexical analysis