Added C++ tokenizer prototype and function
Imported C++ tokenizer logic from part 1