added more error handling, fixed a typo in the tokenizer