Added some unit tests to the tokenizer