tokenizer is fine.
tokenizer header. tokenization via header file working.
Licence update overall. Tokenizer.h edited.