Finished tokenizer tests. Had a whole bunch of bugs removed in the process.