using tokenizer instead of splitting strings