Before revision: Recently, there has been growing interest in using question-answering (QA) models to evaluate the content quality of summaries. While previous work has shown initial promising results in this direction, their experimentation has been limited , leading to a poor understanding of the utility of QA in evaluating summary content .
After revision: Recently, there has been growing interest in using question-answering (QA) models to evaluate the content quality of a summary is that it should estimate how much information that summary has in common with a reference. Traditional text overlap based metrics such as ROUGE fail to achieve this because they are limited to matching tokens, either lexically or via embeddings .