Rename State to Tokenizer/Parser, make it public with private fields.
Methods are sweet.
Make TokenResult a tuple, parse errors a struct.