Question: what are the main responsibilities of lexical analyzer?
Answer: The main task of lexical analysis is to read input characters in the code and produce tokens. "Get next token" is a command which is sent from the parser to the lexical analyzer. On receiving this command, the lexical analyzer scans the input until it finds the next token. It returns the token to Parser.