Update lexer with new tokens.