Generic scanner and tokenizer.