Added code to get metrics for tokenized strings.