From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023)

Abstract: Textual adversarial attacks can discover models’ weaknesses by adding semantic-preserved but misleading perturbations to the inputs. The long-lasting adversarial attack-and-defense arms race in Natural Language Processing (NLP) is algorithm-centric, providing valuable techniques for automatic robustness evaluation. However, the existing practice of robustness evaluation may exhibit issues of incomprehensive evaluation, impractical evaluation protocol, and invalid adversarial samples. In this paper, we aim to set up a unified automatic robustness evaluation framework, shifting towards model-centric evaluation to further exploit the advantages of adversarial attacks. To address the above challenges, we first determine robustness evaluation dimensions based on model capabilities and specify the reasonable algorithm to generate adversarial samples for each dimension. Then we establish the evaluation protocol, including evaluation settings and metrics, under realistic demands. Finally, we use the perturbation degree of adversarial samples to control the sample validity. We implement a toolkit RobTest that realizes our automatic robustness evaluation framework. In our experiments, we conduct a robustness evaluation of RoBERTa models to demonstrate the effectiveness of our evaluation framework, and further show the rationality of each component in the framework.

## introduction pre-trained language models (plms) are vulnerable to textual adversarial attacks that fool the models by adding semantic-preserved perturbations to the inputs @xcite . compared to the static evaluation benchmarks @xcite , attack methods can continually generate diverse adversarial samples to reveal models' weaknesses, rendering a more comprehensive and rigorous model evaluation. previous work explores adversarial nlp in both the attack @xcite and the defense @xcite sides, leading to a longlasting adversarial arms race. the arms race is algorithm-centric. it continually motivates stronger attack and defense methods to explore and fix models' weaknesses, providing useful techniques for robustness evaluation. however, existing work on model robustness evaluation naturally follows the previous evaluation practice, and doesn't fully consider the real-world needs of robustness evaluation @xcite @xcite (see figure 1 ). we identify three weaknesses in previous robustness evaluation: (1) relying on a single attack method @xcite or static challenging datasets @xcite , which can only measure a limited number of aspects of models' capabilities; (2) directly inheriting the evaluation settings and metrics in the arms race era, which may result in impractical evaluation @xcite ; (3) designing invalid adversarial sample 1 filtering rules based on certain thresholds (e.g., sentence similarity), which cannot generalize to all kinds of adversarial samples @xcite . thus, we propose to shift towards the modelcentric evaluation, which should satisfy the following characteristics accordingly: (1) comprehensively measuring nlp models' robustness; (2) establishing a reasonable evaluation protocol considering practical scenarios; (3) filtering out invalid adversarial samples for reliable robustness estimation. given these challenges, a standard and acknowledged framework for employing adversarial attacks to automatically measure and compare nlp models' robustness is lacking (see figure 7 ). in this paper, we motivate a unified modelcentric automatic robustness evaluation framework based on the foundation of the adversarial arms race. to achieve comprehensive evaluation, we define eight robustness dimensions from top to down, constituting a evaluation of multi-dimensional robustness towards sentence-level, word-level, and char-level transformations. for each robustness dimension, we specify the concrete algorithm to generate adversarial samples. then we set up a reasonable evaluation protocol by specifying evaluation settings and metrics under realistic demands. finally, we rely on the perturbation degree to control the validity of generated adversarial samples for more reliable robustness evaluation. our intuition is that adversarial samples with smaller perturbation degrees are more likely to be valid, which is justified through human annotation experiments. we implement a toolkit robtest to realize our robustness evaluation framework (see figure 6 ). we highlight four core features in robtest, including basic adversarial attack methods, robustness report generation, general user instructions, and adversarial data augmentation. in experiments, we use robtest to measure the robustness of roberta models @xcite to demonstrate the effectiveness of our evaluation framework in addressing the core challenges. further, we show the rationality of each component in our robustness evaluation framework through detailed analysis. ## model-centric robustness evaluation in this section, we motivate the first model-centric automatic robustness evaluation framework. we first define robustness evaluation dimensions and 1 detailed explanation for validity is in appendix a. ## robtest we implement an automatic robustness evaluation toolkit named robtest to realize our proposed framework. we highlight four features of robtest. basic adversarial attack methods. we implement eight attack methods, corresponding to eight robustness evaluation dimensions in our framework. we also include three attack types that assume different information available from evaluated models, namely rule-based, score-based, and gradientbased attacks. robtest allows practitioners to customize evaluated models and datasets and design new attack methods to test specified robustness dimensions. also, it supports the multi-process running of adversarial attacks for efficiency. robustness report. robtest provides comprehensive robustness reports for evaluated models. see figure 2 and appendix g for examples of single-model robustness reports. see figure 3 and appendix h for examples of the robustness comparison of the two models. we further discuss the details of robustness reports in sec. 4. ## experiment we conduct experiments to demonstrate the effectiveness of our automatic robustness evaluation framework using robtest. we aim to show how our framework fulfills the characteristics of modelcentric robustness evaluation foot_4 . ## related work standard evaluation benchmarks @xcite follow the independently identical distribution hypothesis that assumes the training and testing data come from the same distribution. however, there is no such guarantee in practice, motivating the requirement to evaluate models' robustness beyond the standard accuracy. various approaches have been proposed to simulate distribution shifts to construct static robustness evaluation benchmarks, including stress test @xcite , identifying and utilizing spurious correlations @xcite , and domain shifts construction @xcite . also, adversarial samples have been involved in robustness benchmarks, including machine-generated @xcite or humanin-the-loop generated @xcite @xcite samples. compared to static benchmarks, we motivate to employ automatic attack methods to evaluate models' robustness dynamically, which is more comprehensive and rigorous. our work is built upon the long-lasting attack-and-defense arms race in adversarial nlp @xcite , mainly absorbing various attack methods. the attack methods can be roughly categorized into char-level, word-level, and sentence-level attacks, corresponding to the hierarchy in our framework. char-level attacks perturb the texts in the finest granularity, including deleting, inserting, replacing, swapping, and repeating characters @xcite . word-level attacks search for an optimal solution for word substitutions, using external knowledge bases @xcite or contextual information @xcite @xcite . sentence-level attacks transform the text considering syntactic patterns @xcite ), text styles @xcite , and domains @xcite . ## conclusion we present a unified framework, providing solutions to three core challenges in automatic robustness evaluation. we give a further discussion about robustness evaluation in appendix f. in the future, we will selectively include more robustness dimensions in our framework.