From Adversarial Arms Race to Model-centric Evaluation Motivating a Unified Automatic Robustness Evaluation Framework
Year: 2023

Abstract:
July 9-14, 2023 ©2023 Association for Computational Linguistics From Adversarial Arms Race to Model-centric Evaluation Motivating a Unified Automatic Robustness Evaluation Framework Yangyi Chen1,2∗, Hongcheng Gao1,3∗, Ganqu Cui1∗, Lifan Yuan Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue Zhiyuan Liu1,6†, Maosong Sun1,6†, Heng Ji NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing UIUC Chongqing University HUST Alibaba Group 6 Jiangsu Collaborative Innovation Center for Language Ability, Jiangsu Normal University yangyic3@illinois.edu, gaohongcheng2000@gmail.com Abstract

heading: Future Work
text: , 2018; Eger and Benz, 2020) considers five basic operations to add typos in the inputs, including randomly
2Task-specific robustness dimensions are also essential, and we leave it for future work. 9608
Granularity Dimension General? Malicious? Case
Char-level Typo Yes Yes I watch a smart, swet adn playful romantic comedy. Glyph Yes Yes I watch a ŝmârt, sweet and playful romaňtîĉ comedy. Phonetic Yes Yes I wotch a smart, sweet and playful romentic comedy. Word-level Synonym Yes No I watch a smart, sweet and naughty romantic comedy. Contextual Yes No We watch a smart, sweet and playful romantic teleplay. Inflection Yes No I watched a smart, sweet and playful romantic comedies. delete, insert, replace, swap, or repeat one character; (2) Glyph (Li et al. , 2018; Eger et al. , 2019) replaces characters with visually-similar ones; (3) Phonetic (Le et al. , 2022) replaces characters but makes the whole word sound similar to the origin. Word-level transformations modify word units as a whole. We include the following dimensions in our framework: (1) Synonym (Ren et al. , 2019; Zang et al. , 2020) replaces words with their synonymous substitutes according to external knowledge sources. We consider WordNet (Miller, 1995) and HowNet (Dong and Dong, 2003) in our implementation; (2) Contextual (Li et al. , 2020; Garg and Ramakrishnan, 2020) replaces words with their context-similar substitutes, which are generated by masked language models; (3) Inflection (Tan et al. , 2020) perturbs the inflectional morphology of words. Sentence-level transformations generate adversarial samples directly from the entire original sentences. We include the following dimensions in our framework: (1) Syntax (Iyyer et al. , 2018; Huang and Chang, 2021; Sun et al. , 2021) transforms the syntactic patterns of original samples; (2) Distraction (Naik et al. , 2018; Ribeiro et al. , 2020; Chen et al. , 2022a) appends some irrelevant contents to the end of sentences. Malicious & General Tags. For each robustness dimension, we also attach the general or malicious tag to characterize the intended simulated agents. The general (malicious) tag indicates that the generated samples mainly come from benign users (malicious attackers). For example, Synonym and Distraction are representative types of general and malicious dimensions respectively. Note that we attach both tags to three char-level transformations since both benign users and malicious attackers can produce these kinds of samples. 2. 2 Evaluation Protocol
Motivation. Previous work in adversarial NLP naturally follows the early attempts (Szegedy et al. ,
2013; Goodfellow et al. , 2014; Liang et al. , 2017; Gao et al. , 2018a) to establish the evaluation protocol. However, Chen et al. (2022b) categorize and summarize four different roles of textual adversarial samples, and argue for a different evaluating protocol for each role. In our framework, we reconsider the robustness evaluation protocol when employing adversarial attack methods for model evaluation. We first describe the evaluation setting, and then the evaluation metrics in our framework. Evaluation Setting (available information from evaluated models). Most existing attack methods assume the accessibility to confidence scores only (Alzantot et al. , 2018; Ren et al. , 2019; Zang et al. , 2020; Li et al. , 2020; Chen et al. , 2021). We acknowledge the rationality of this assumption since the size of models may become too large nowadays (Radford et al. , 2019; Brown et al. , 2020), resulting in inefficient evaluation if also requiring the gradients information for adversarial samples generation (Goodfellow et al. , 2014). However, in practice, we as practitioners mostly have all access to the evaluated models, including the parameters and gradient information, for better robustness evaluation. Thus, we implement three evaluation settings in our framework, assuming different available information from evaluated models. The settings include rule-based, score-based, and gradient-based attacks. Rule-based attacks don’t assume any information from the evaluated models and generate adversarial samples based on pre-defined rules. Scorebased and gradient-based attacks assume access to the confidence scores and gradients information respectively from evaluated models for more rigorous evaluation. They first compute the saliency maps that give the importance scores to each word for samples and then perform selective perturbations based on the scores. Specifically, for score-based attacks, we employ the difference in confidence scores when iteratively masking each word as the
9609
important score for that word. For gradient-based attacks, we employ integrated gradient (IG) (Sundararajan et al. , 2017) to compute the saliency map. IG computes the average gradient along the linear path of varying the input from a baseline value to itself. Besides, we use greedy search since it can achieve satisfying performance within a reasonable time (Yoo et al. , 2020). Evaluation Metrics. Most previous work considers the “is robust” problem (Li et al. , 2020, 2021; Chen et al. , 2021). They generate adversarial samples for each original sample and test if at least one of them can successfully attack the evaluated models. Then the final score is computed as the percentage of samples that are not attacked successfully. This is the worst performance estimation, requiring models to be robust to all potential adversarial samples in order to score. In our framework, we introduce the average performance estimation for a more comprehensive robustness evaluation. Specifically, for each original sample, we compute the percentage of cases that models can correctly classify among all potential adversarial samples. Then we average over all original samples to get the average performance estimation score. 2. 3 Reliable Robustness Evaluation Motivation. Previous work chases for higher attack success rate, while the validity of adversarial samples may be sacrificed3. The consequence of this practice is unreliable and inaccurate robustness evaluation. We showcase adversarial samples crafted by three popular methods on SST-2 (Socher et al. , 2013) in Table 2. While all samples successfully flip the predictive label, they are not good choices for robustness evaluation because the ground truth label is changed (e. g. , BERT-Attack) or the meaning of the original sentence is changed (e. g. , GA, Textbugger). Morris et al. (2020a); Wang et al. (2021a); Hauser et al. (2021) show that there are many such invalid cases in adversarial samples that successfully mislead models’ predictions. We further conduct a human evaluation to support this
3We give a detailed explanation for adversarial samples validity in Appendix A