Define lexer tokens, lex input string into tokens.