Added documentation to lexer function