Fixed build errors and switched to the generated tokenizer tests