Promote tokenizer errors to a proper enum
Merge the disambiguator and deabbreviator into the tokenizer file
Remove unused internal iterators