Begin incomplete, currently-unusable rewrite of the tokenizer to match current spec.