map and reduce duplicate tokens by probability
add map reduce to summarizer