copied all the code from tokenizer.c, maybe we can start from here~