Added test for tokenizer
Resolved conflict