Universal Evasion Attacks on Summarization Scoring (2022)

Abstract: The automatic scoring of summaries is important as it guides the development of summarizers. Scoring is also complex, as it involves multiple aspects such as the fluency, grammar, and even textual entailment with the source text. However, summary scoring has not been considered as a machine learning task to study its accuracy and robustness. In this study, we place automatic scoring in the context of regression machine learning tasks and perform evasion attacks to explore its robustness. Attack systems predict a non-summary string from each input, and these non-summary strings achieve competitive scores with good summarizers on the most popular metrics: ROUGE, METEOR, and BERTScore. Attack systems also “outperform” state-of-the-art summarization methods on ROUGE-1 and ROUGE-L, and score the second-highest on METEOR. Furthermore, a BERTScore backdoor is observed: a simple trigger can score higher than any automatic summarization method. The evasion attacks in this work indicate the low robustness of current scoring systems at the system level. We hope that our highlighting of these proposed attack will facilitate the development of summary scores.

## introduction a long-standing paradox has plagued the task of automatic summarization. on the one hand, for about 20 years, there has not been any automatic scoring available as a sufficient or necessary condition to demonstrate summary quality, such as adequacy, grammaticality, cohesion, fidelity, etc. on the other hand, contemporaneous research more often uses one or several automatic scores to endorse a summarizer as state-of-the-art. more than 90% of works on language generation neural models choose automatic scoring as the main basis, and about half of them rely on automatic scoring only @xcite . however, these i wrote a summary of a document. i'm here to rate this summary you wrote. figure 1 : automatic summarization (left) and automatic scoring (right) should be considered as two systems of the same rank, representing conditional language generation and natural language understanding, respectively. as a stand-alone system, the accuracy and robustness of automatic scoring are also important. in this study, we create systems that use bad summaries to fool existing scoring systems. this work shows that optimizing towards a flawed scoring does more harm than good, and flawed scoring methods are not able to indicate the true performance of summarizers, even at a system level. why do we have to deal with this paradox? the current work may not have suggested that summarizers assessed by automatic scoring are de facto ineffective. however, optimizing for flawed evaluations @xcite , directly or indirectly, ultimately harms the development of automatic summarization @xcite @xcite . one of the most likely drawbacks is shortcut learning (surface learning, @xcite , where summarizing models may fail to generate text with more widely accepted qualities such as adequacy and authenticity, but instead pleasing scores. here, we quote and adapt 1 this hypothetical story by @xcite . according to geirhos et al., bob might get better grades and consequently be considered a better student than alice, which is an example of surface learning. the same could be the case with automatic summarization, where we might end up with significant differences between expected and actual learning outcomes @xcite . to avoid going astray, it is important to ensure that the objective is correct. in addition to understanding the importance of correct justification, we also need to know what caused the fallacy of the justification process for these potentially useful summarizers. there are three mainstream speculations that are not mutually exclusive. (1) the transition from extractive summarization to abstractive summarization @xcite could have been underestimated. for example, the popular score rouge @xcite was originally used to judge the ranking of sentences selected from documents. due to constraints on sentence integrity, the generated summaries can always be fluent and undistorted, except sometimes when anaphora is involved. however, when it comes to free-form language generation, sentence integrity is no longer guaranteed, but the metric continues to be used. (2) many metrics, while flawed in judging individual summaries, often make sense at the system level @xcite @xcite . in other words, it might have been believed that few summarization systems can consistently output poorquality but high-scoring strings. (3) researchers have not figured out how humans interpret or understand texts (van der @xcite @xcite , thus the decision about how good a summary really is varies from person 1 we underline adaptations. to person, let alone automated scoring. in fact, automatic scoring is more of a natural language understanding (nlu) task, a task that is far from solved. from this viewpoint, automatic scoring itself is fairly challenging. nevertheless, the current work is not to advocate (and certainly does not disparage) human evaluation. instead, we argue that automatic scoring itself is not just a sub-module of automatic summarization, and that automatic scoring is a stand-alone system that needs to be studied for its own accuracy and robustness. the primary reason is that nlu is clearly required to characterize summary quality, e.g., semantic similarity to determine adequacy @xcite , or textual entailment @xcite to determine fidelity. besides, summary scoring is similar to automated essay scoring (aes), which is a 50-year-old task measuring grammaticality, cohesion, relevance etc. of written texts @xcite . moreover, recent advances in automatic scoring also support this argument well. automatic scoring is gradually transitioning from well-established metrics measuring n-gram overlap (bleu @xcite , rouge @xcite , meteor @xcite , etc.) to emerging metrics aimed at computing semantic similarity through pre-trained neural models (bertscore @xcite , moverscore @xcite , bleurt @xcite , etc.) these emerging scores exhibit two characteristics that stand-alone machine learning systems typically have: one is that some can be fine-tuned for human cognition; the other is that they still have room to improve and still have to learn how to match human ratings. machine learning systems can be attacked. attacks can help improve their generality, robustness, and interpretability. in particular, evasion attacks are an intuitive way to further expose the weaknesses of current automatic scoring systems. evasion attack is the parent task of adversarial attack, which aims to make the system fail to correctly identify the input, and thus requires defence against certain exposed vulnerabilities. in this work, we try to answer the question: do current representative automatic scoring systems really work well at the system level? how hard is it to say they do not work well at the system level? in summary, we make the following major contributions in this study: ## universal evasion attacks we develop universal evasion attacks for individual scoring system, and make sure that the combined attacker can fool rouge, meteor, and bertscore at the same time. it incorporates two parts, a white-box attacker on rouge, and a black-box universal trigger search algorithm for bertscore, based on genetic algorithms. me-teor can be attacked directly by the one designed for rouge. concatenating output strings from black-box and white-box attackers leads to a sole universal evasion attacking string. ## experiments we instantiate our evasion attack by conducting experiments on non-anonymized cnn/dailymail (cnndm, @xcite , a dataset that contains news articles and associated highlights as summaries. cnndm includes 287,226 training pairs, 13,368 validation pairs and 11,490 test pairs. for σ rouge we use roberta (base model, @xcite to instantiate ϕ, which is an optimized pretrained encoding with a randomly initialized linear layer on top of the hidden states. number of classes is set to three because we assume that each word appears at most twice in a summary. all 124,058,116 parameters are trained as a whole on cnndm train split for one epoch. when the batch size is eight, the training time on an nvidia tesla k80 graphics processing unit (gpu) is less than 14 hours. it then takes about 20 minutes to predict (including word ordering) all 11,490 samples in the cnndm test split. scripts and results are available at https://github.com/ cestwc/universal-evasion.git . for the universal trigger to bertscore, we use the library from blank and deb (2020) for discrete optimizing, set population size at 10, and terminate at 2000 generations. ŝemu is a sequence of independent randomly initialized non-alphanumeric characters. for a reference s ref from cnndm, we start from randomly pick a summary text from train split and optimize for ŝemu,@xmath0. we then pick the s ref that is farthest away from ŝemu,@xmath1,@xmath2, with ŝemu,@xmath3. practically, we found that we can stop iterating when @xmath4. each iteration takes less than two hours on a 2vcpu (intel xeon @ 2.30ghz). ## results we compare rouge-1/2/l, meteor, and bertscore of our threat model with that achieved by the top summarizers in the threat model ranks second. as for rouge-2 and bertscore, the threat model can score higher than other bert-based summarizing algorithms 4 . overall, we rank the systems by averaging their three relative ranking on rouge 5 , meteor, and bertscore; our threat model gets runner-up (2.7), right behind simcls (1.7) and ahead of bart (3.3). this suggests that at the system level, even a combination of mainstream metrics is questionable in justifying the excellence of the summarizer. these results reveal low robustness of popular metrics and how certain models can obtain high scores with inferior summaries. for example, our threat model is able to grasp the essence of rouge-1/2/l using a general but lightweight model, which requires less running time than summarizing algorithms. the training strategies for the model and word order are trivial. not surprisingly, its output texts do not resemble human understandable "summaries" ( ## conclusion we hereby answer the question: it is easy to create a threat system that simultaneously scores high on rouge, meteor, and bertscore using worse text. in this work, we treat automatic scoring as a regression machine learning task and conduct evasion attacks to probe its robustness or reliability. our attacker, whose score competes with toplevel summarizers, actually outputs non-summary strings. this further suggests that current mainstream scoring systems are not a sufficient condition to support the plausibility of summarizers, as they ignore the linguistic information required to compute sentence proximity. intentionally or not, optimizing for flawed scores can prevent algorithms from summarizing well. the practical effectiveness of existing summarizing algorithms is not affected by this, since most of them optimize maximum likelihood estimation. based on the exposed vulnerabilities, careful fixes to scoring systems that measure summary quality and sentence similarity are necessary.