Define overall lexer control flow.