add segmenting function when tokenized text is given