refactoring to accommodate lexer states for complex string expressions