On the Blind Spots of Model-Based Evaluation Metrics for Text Generation (2023)

Abstract: In this work, we explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics: stress tests with synthetic data. Basically, we design and synthesize a wide range of potential errors and check whether they result in a commensurate drop in the metric scores. We examine a range of recently proposed evaluation metrics based on pretrained language models, for the tasks of open-ended generation, translation, and summarization. Our experiments reveal interesting insensitivities, biases, or even loopholes in existing metrics. For example, we find that BERTScore is confused by truncation errors in summarization, and MAUVE (built on top of GPT-2) is insensitive to errors at the beginning or middle of generations. Further, we investigate the reasons behind these blind spots and suggest practical workarounds for a more reliable evaluation of text generation. We have released our code and data at https://github.com/cloudygoose/blindspot_nlg .

## introduction automatic evaluation of machine-generated text @xcite has been a core research challenge in the field of natural language generation (nlg), as difficult as language generation itself. encouraged by the phenomenal success of large-scale pretraining @xcite , a recent series of work proposed to base evaluation metrics on pretrained language models (plms) @xcite @xcite . for example, bertscore @xcite computes a similarity score between the contextualized embeddings of the hypothesis and the reference text. plm-based metrics have been shown to have higher correlations with human annotations for various tasks @xcite , and are becoming increasingly popular in practice. however, plms have flaws. they could assign a high likelihood to degenerate, repetitive text @xcite and could be insensitive to perturbations such as word order shuffling @xcite , negation @xcite , etc. these flaws, in combination with certain design choices, may lead to the metrics based on such plms being brittle and open to manipulation (figure 1 ). in this work, we develop a suite of stress tests with synthetic data for the robustness analysis of nlg metrics. in essence, we induce a variety of potential errors in clean text and examine the resulting drop in the metric scores. the tests are motivated by metric design choices, properties of plms, or general fluency/consistency errors. our methodology facilitates full control over the synthesized error types, allowing us to test extreme or even adversarial scenarios that are not well covered in standard correlation-oriented evaluations. our tests are applied to a range of recently proposed and widely used plm-based metrics for the tasks of open-ended generation, translation, and summarization. they reveal a number of glaring insensitivities, biases, and even loopholes in different metrics. besides analyzing the reasons behind, we also provide practical suggestions and workarounds for a more reliable evaluation. ## methodology we now discuss our methodology. for simplicity, in this section, let us assume a multi-reference translation dataset, where each sample has two reference translations produced by human translators, denoted by ref-a and ref-b. we will generalize our methodology to other tasks in §3. we begin by computing a "base" metric score by considering ref-a as hypotheses and ref-b as references. since ref-a is produced by human translators, we assume that it is less likely to contain translation errors than machine-generated text, and it should be assigned a high score by the metric. due to these two assumptions, and to disambiguate from the reference set (ref-b), we term ref-a as the gold hypothesis set. for each test, we apply a synthesized error type (e.g., truncation) to the gold hypothesis set to construct a noised hypothesis set. we make sure that the amount or type of induced errors is sufficient to be distinctive from the original gold hypothesis (to be detailed in §5). the source texts and the references are left intact. to determine whether a metric passes a test, a simple rank-based protocol is used: we claim that the metric fails the test for this dataset if the noised hypothesis set is not scored worse than the base score (from the gold set). 1 this rank-based protocol can be easily extended to the comparison of different gradations of the same noise type (controlled by hyper-parameters). for example, a 20%truncation is expected to rank lower than a 10%truncation, as more information is lost. ## tasks and datasets our tests cover three ubiquitous text generation tasks: open-ended generation, translation, and summarization. we now describe the dataset used for each task and the setting for gold hypotheses. for open-ended generation, we use the wikitext-103 dataset @xcite . we randomly select 2000 paragraphs of length around 256 tokens from the dataset (preprocessing detailed in appendix b.2). the samples typically contain seven or eight sentences. we divide them into two sets with 1000 samples each, and set one as the references and the other as the gold hypotheses. the reference set is only used for the mauve metric (more details given in appendix a). for summarization, we use the popular cnn-dailymail (cnndm) dataset @xcite . @xcite collected 10 additional human-annotated summaries (different from the original reference summary) for each of 100 samples in the test set. we set the cnndm reference summaries to be the gold hypotheses, and use these 10 annotations as references. correspondingly, the multi-reference version of metrics are used. the gold hypotheses typically contain three sentences. for translation, we use the evaluation dataset from the wmt21 metrics shared task @xcite . we only use the source text and reference translations. we report results on the german-english (de-en) language pair, which contains 1000 translation pairs. there are two humantranslated references (human-a and human-b) for each sample. we use human-a as the gold hypothesis and human-b as the reference. we also repeat key experiments on the chinese-english (zh-en) data and obtain very similar observations. therefore, we omit the zh-en results for brevity. most samples in wmt only contain one sentence, which makes some of our tests impossible (e.g., sentence switching). for this reason, we build a paragraph-level translation dataset based on the zh-en part of the ted-talks task @xcite . it contains 100 samples, where each sample has two human-translated references and on average contains 7 sentences. we name this dataset as ted-mt, and discuss how we build it in appendix b.1. ## metrics for open-ended text generation, we test mauve @xcite , gpt-ppl and mlm-ppl @xcite . we report the negated gpt/mlm-ppl so that all metric scores are the higher the better. mauve is a reference-based metric computed using contextualized embeddings from plms. we explore mauve with gpt2-large, robertalarge, and electra-large @xcite features. in @xcite , the exploration is centered around the gpt-2 feature. however, in this work we find the choice of feature has a crucial impact on the metric's robustness. gpt-ppl denotes perplexity from the gpt2large @xcite model. mlm-ppl is the masked language model perplexity from a roberta-large model @xcite . we use a definition similar to the formulation in salazar et al. (2020) and provide details in appendix a. for translation and summarization, we test bertscore @xcite , moverscore @xcite , bartscore @xcite , unieval @xcite , comet @xcite , prism @xcite , and bleurt @xcite . among these metrics, prism and bleurt are only applied for translation, and unieval is only applied for summarization. while comet was originally proposed for translation, @xcite showed it has superior human correlation for cnndm. therefore, we also include it for summarization. we also include the traditional metrics bleu (for translation), and rouge-2/l (for summarization). both bertscore and bartscore have variants for precision (-p), recall (-r), and f-measure (-f). in addition, bartscore has a faithfulness (-faithful) variant. we test two model options, namely bartscore-cnn and bartscore-para. 2 unieval reports scores on four aspects: coherence, consistency, fluency, and relevance, and the overall score is the average of the four. by default, the metrics for translation and summarization are reference-based. 3 comet and prism have a quality estimation (qe) variant @xcite , where users do not need to provide any reference. in most cases, we directly use the released package or code for each metric and follow the recommended hyper-parameter or variant setting. we defer further implementation details and variant explanations to appendix a. ## stress tests and results we organize our findings into subsections each containing a set of tests with the corresponding motivation, description, results, and implications with practical workarounds. in general we perform each test for all metrics, and we primarily discuss metrics found to be problematic for brevity. we group and order our tests by their motivations: the positioned-error ( §5.1) and injection ( §5.2) tests are mainly motivated by certain metric design choices; the freq-ngram ( §5.3) and selfevaluation ( §5.4) tests are motivated by certain plm properties; finally, the fluency/consistency ( §5.5) tests mimic general errors that human or machine writers could make. see ## discussion the copy-source and the repetition tests to save space, the copy-source test is deferred to appendix d because its results are relatively unsurprising. we also defer the repetition test to appendix e, as it is motivated by the well-known degeneration problem @xcite . ## conclusion using plms for nlg metrics is a double-edged sword. while the metrics benefit from the models' powerful representations, their black-box nature may cause unexpected behavior. this work shows that stress tests, complementary to the standard human correlation tests, are powerful tools to cover corner cases, detect the metrics' blind spots, and point out aspects where the metric could improve. as a major implication for metric users, we suggest using combinations of metrics so that they can cover each other's blind spots. while this has been an existing practice for a majority of work in the field, our results on the blind spots provide an explicit empirical argument for its importance. while we are still positive about the future of using plm for nlg metrics, we call for more caution and awareness of potential blind spots from both metric users and developers. more generally speaking, a deeper understanding of the plms is in need.