O pen A ttack: An Open-source Textual Adversarial Attack Toolkit (2021)

Abstract: Textual adversarial attacking has received wide and increasing attention in recent years. Various attack models have been proposed, which are enormously distinct and implemented with different programming frameworks and settings. These facts hinder quick utilization and fair comparison of attack models. In this paper, we present an open-source textual adversarial attack toolkit named OpenAttack to solve these issues. Compared with existing other textual adversarial attack toolkits, OpenAttack has its unique strengths in support for all attack types, multilinguality, and parallel processing. Currently, OpenAttack includes 15 typical attack models that cover all attack types. Its highly inclusive modular design not only supports quick utilization of existing attack models, but also enables great flexibility and extensibility. OpenAttack has broad uses including comparing and evaluating attack models, measuring robustness of a model, assisting in developing new attack models, and adversarial training. Source code and documentation can be obtained at https://github.com/thunlp/OpenAttack .

## introduction deep neural networks (dnns) have been found to be susceptible to adversarial attacks @xcite . the attacker uses adversarial examples, which are maliciously crafted by imposing small perturbations on original input, to fool the victim model. with the wide application of dnns to practical systems accompanied by growing concern about their security, research on adversarial attacking has become increasingly important. moreover, adversarial attacks are also helpful to improve robustness and interpretability of dnns @xcite . in the field of natural language processing (nlp), diverse adversarial attack models have been proposed @xcite . these models vary in accessibility to the victim model (ranging from having full knowledge to total ignorance) and perturbation level (character-, word-or sentence-level). in addition, they are originally proposed to attack different victim models on different nlp tasks under different evaluation protocols. this immense diversity causes serious difficulty for fair and apt comparison between different attack models, which is unfavourable to the development of textual adversarial attacking. further, although most attack models are open-source, they use different programming frameworks and settings, which lead to unnecessary time and effort when implementing them. to tackle these challenges, a textual adversarial attacking toolkit named textattack @xcite has been developed. it implements several textual adversarial attack models under a unified framework and provides interfaces for utilizing existing attack models or designing new attack models. so far, textattack has attracted considerable attention and facilitated the birth of new attack models such as bae @xcite . in this paper, we present openattack, which is also an open-source toolkit for textual adversarial attacking. similar to textattack, openattack adopts modular design to assemble various attack models, in order to enable quick implementation of existing or new attack models. but openattack is different from and complementary to textattack mainly in the following three aspects: model accessibility perturbation main idea sea (ribeiro et al., 2018) decision sentence rule-based paraphrasing scpn @xcite blind sentence paraphrasing gan @xcite decision sentence text generation by encoder-decoder textfooler @xcite score word greedy word substitution pwws @xcite score word greedy word substitution genetic @xcite score word genetic algorithm-based word substitution sememepso @xcite score word particle swarm optimization-based word substitution bert-attack @xcite score word greedy contextualized word substitution bae @xcite score word greedy contextualized word substitution and insertion fd @xcite gradient word gradient-based word substitution textbugger @xcite gradient, score word+char greedy word substitution and character manipulation uat @xcite gradient word, char gradient-based word or character manipulation hotflip @xcite gradient word, char gradient-based word or character substitution viper @xcite blind char visually similar character substitution deepwordbug @xcite score char greedy character manipulation an important and typical kind of textual adversarial attacks. thus, no sentence-level attack models are included in textattack. in contrast, openattack adopts a more flexible framework that supports all types of attacks including sentence-level attacks. moreover, openattack is fully integrated with huggingface's transformers foot_0 and datasets foot_1 libraries, which allows convenient adversarial attacks against thousands of nlp models (especially pre-trained models) on diverse datasets. openattack also has great extensibility. it can be easily used to attack any customized victim model, regardless of the used programming framework (pytorch, tensorflow, keras, etc.), on any customized dataset. openattack can be used to ( 1 ) provide vari-ous handy baselines for attack models; (2) comprehensively evaluate attack models using its thorough evaluation metrics; (3) assist in quick development of new attack models; (4) evaluate the robustness of an nlp model against various adversarial attacks; and (5) conduct adversarial training (goodfellow et al., 2015) to improve model robustness by enriching the training data with generated adversarial examples. recent years have witnesses the rapid development of adversarial attacks in computer vision @xcite , which is promoted by many visual attack toolkits such as cleverhans @xcite , foolbox @xcite , advbox @xcite , etc. we hope openattack, together with textattack and other similar toolkits, can play a constructive role in the development of textual adversarial attacks. ## formalization and categorization of textual adversarial attacking we first formalize the task of textual adversarial attacking for text classification, and the following formalization can be trivially adapted to other nlp tasks. for a given text sequence x that is correctly classified as its ground-truth label y by the victim model f , the attack model a is supposed to transform x into x by small perturbations, whose ground-truth label is still y but classification result given by f is ŷ = y. next, we introduce the catego-rization of textual adversarial attack models from three perspectives. according to the attack model's accessibility to the victim model, existing attack models can be categorized into four classes, namely gradientbased, score-based, decision-based and blind models. first, gradient-based attack models are also called white-box attack models, which require full knowledge of the victim model to conduct gradient update. most of them are inspired by the fast gradient sign method @xcite and forward derivative method @xcite in visual adversarial attacking. in contrast to white-box attack models, blackbox models do not need to have complete information on the victim model, and can be subcategorized into score-based, decision-based and blind models. blind models are ignorant of the victim model at all. score-based models require the prediction scores (e.g., classification probabilities) of the victim model, while decision-based models only need the final decision (e.g., predicted class). according to the level of perturbations imposed on original input, textual adversarial attack models can be classified into sentence-level, wordlevel and character-level models. sentence-level attack models craft adversarial examples mainly by adding distracting sentences @xcite , paraphrasing @xcite or text generation by encoder-decoder @xcite . word-level attack models mostly conduct word substitution, namely substituting some words in the original input with semantically identical or similar words such as synonyms @xcite @xcite . some word-level attack models also use operations including deleting and adding words @xcite . characterlevel attack models usually carry out character manipulations including swap, substitution, deletion, insertion and repeating @xcite @xcite . finally, adversarial attack models can also be categorized into targeted and untargeted models based on whether the wrong classification result given by the victim model (ŷ) is pre-specified (mainly for the multi-class classification models). most existing attack models support (by minor adjustment) both targeted and untargeted attacks, and we give no particular attention to this attribute of attack models in this paper. currently openattack includes 15 different attack models, which cover all the victim model accessibility and perturbation level types. ## toolkit design and architecture in this section, we describe the design philosophy and modular architecture of openattack. we extract and properly reorganize the commonly used components from different attack models, so that any attack model can be assembled by them. considering the significant distinctions among different attack models, especially those between the sentence-level and word/char-level attack models, it is hard to embrace all attack models within a unified framework like textattack. therefore, we leave considerable freedom for the skeleton design of attack models, and focus more on streamlining the general processing of adversarial attacking and providing common components used in attack models. next we introduce the modules of openattack one by one, and figure 1 illustrates an overview of all the modules. this module is used to evaluate textual adversarial attacks from different perspectives including attack effectiveness, adversarial example quality and attack efficiency: (1) the attack effectiveness metric is attack success rate, the percentage of the attacks that successfully fool the victim model; (2) adversarial example quality is measured by the last five metrics in the metric module; and (3) attack efficiency has two metrics including average victim model query times and average running time of attacking one instance. the realization of multi-processing is incorporated in this module, with the help of python multiprocessing library. in addition, this module can also visualize and save attack results, e.g., display original input and adversarial examples and emphasize their differences. ## toolkit usage openattack provides a set of easy-to-use interfaces that can meet almost all the needs in textual adversarial attacking, such as preprocessing text, generating adversarial examples to attack a victim model and evaluating attack models. moreover, openattack has great flexibility and extensibility and supports easy customization of victim models and attack models. next, we showcase some basic usages of openattack. ## evaluation in this section, we conduct evaluations for all the attack models included in openattack. we use sst-2 as the evaluation dataset and choose bert, specifically bert base , as the victim model. after fine-tuning on the training set, bert achieves 90.31 accuracy on the test set. due to great diversity of attack models, it is hard to impose many constraints on attacks like previous work that focuses on a specific kind of attack. we only restrict the maximum victim model query times to 500. in addition, to improve evaluation efficiency, we randomly sample 1, 000 correctly classified instances from the test set as the original input to be perturbed. we use the original default hyper-parameter settings of all attack models. ## related work there have been quite a few open-source libraries of generating adversarial examples for continuous data, especially images, such as cleverhans @xcite , foolbox @xcite , adversarial robustness toolbox (art) @xcite and advbox @xcite . these libraries enable practitioners to easily make adversarial attacks with different methods and have greatly facilitated the development of adversarial attacking for continuous data. as for discrete data, particularly text, there exist few adversarial attack libraries. as far as we know, textattack @xcite is the only such library. it utilizes a relatively rigorous framework to unify many attack models and provides interfaces for using the existing attack models or designing new attack models. as mentioned in introduction, our openattack is mainly different from and complementary to textattack in allattack-type support, multilinguality and parallel processing. there are also some other toolkits concerned with textual adversarial attacking. teapot @xcite is an open-source toolkit to evaluate the effectiveness of textual adversarial examples from the perspective of preservation of meaning. it is mainly designed for the attacks against sequence-to-sequence models, but can also be geared towards text classification models. al-lennlp interpret @xcite is a framework for explaining the predictions of nlp models, where adversarial attacking is one of its interpretation methods. it focuses on interpretability of nlp models and only incorporates two attack models. ## conclusion and future work in this paper, we present openattack, an opensource textual adversarial attack toolkit that provides a wide range of functions in textual adversarial attacking. it is a great complement to existing counterparts because of its unique strengths in allattack-type support, multilinguality and parallel processing. moreover, it has great flexibility and extensibility and provides easy customization of victim models and attack models. in the future, we will keep openattack updated to incorporate more up-to-date attack models and support more functions to facilitate the research on textual adversarial attacks.