Added comments and more code for the lexer.