Refactor tokenizer test runner

Parse the JSON to our token structs, rather than tokenizing to JSON.