Added more test cases for tokenize and fixed issues