An ideomatic port of Douglas Crockfords's tokenizer (TDOP).