R obust QA : A Framework for Adversarial Text Generation Analysis on Question Answering Systems (2023)

Abstract: Question answering (QA) systems have reached human-level accuracy; however, these systems are not robust enough and are vulnerable to adversarial examples. Recently, adversarial attacks have been widely investigated in text classification. However, there have been few research efforts on this topic in QA. In this article, we have modified the attack algorithms widely used in text classification to fit those algorithms for QA systems. We have evaluated the impact of various attack methods on QA systems at character, word, and sentence levels. Furthermore, we have developed a new framework, named RobustQA, as the first open-source toolkit for investigating textual adversarial attacks in QA systems. RobustQA consists of seven modules: Tokenizer, Victim Model, Goals, Metrics, Attacker, Attack Selector, and Evaluator. It currently supports six different attack algorithms. Furthermore, the framework simplifies the development of new attack algorithms in QA. The source code and documentation of RobustQA are available at https://github.com/mirbostani/RobustQA.

## introduction with the release of large and high-quality datasets in the field of question answering (qa) @xcite @xcite , we witness significant progress in this area of research. with the aid of deep neural networks (dnns), the newly presented models have even reached human-level accuracy. however, it has been shown that these models are not yet robust enough and are vulnerable to adversarial examples @xcite . in the context of qa systems, the model's accuracy drops drastically when some adversarial sentences are added to the input paragraphs @xcite . accordingly, extensive research efforts have been conducted addressing various techniques to increase the robustness of dnn models in * these authors contributed equally to this work. different fields. one of the most popular techniques to overcome this issue is the so-called adversarial training. in adversarial training, some adversarial examples are used during the training phase of the model to increase its robustness against textual adversarial attacks @xcite . in another research, the impact of the knowledge distillation technique on the robustness of qa models has been analyzed @xcite . adversarial attacks have been widely investigated in the field of text classification @xcite . furthermore, openattack @xcite and textattack @xcite frameworks have been presented to simplify the implementation and analysis of different attack methods in text classification. however, there has been only a limited number of efforts in this regard for the qa systems. the contributions of the paper can be summarized as follows: 1) we modify the attack algorithms that have been widely used in the field of text classification for qa systems. 2) we show that these modified attack algorithms can easily be evaluated on qa systems in three different characters, words, and sentence levels. 3) we build a new open-source framework named robustqa, aiming at simplifying the research on textual adversarial attacks in qa systems. 4) we have incorporated both adversarial text generation and data augmentation in robustqa for being used in adversarial training methods to improve the robustness and generalization of qa models. in this paper, we introduce the related works in section 2. we compare the qa task against text classification and describe a sample textual adversarial attack algorithm implemented for the qa task in section 3. next, we introduce the ro-bustqa framework modules in detail in section 4 and demonstrate the framework's usage in section 5. we present our setup and experimental results in section 6. finally, our conclusions and future works are presented in section 7. ## the robustqa framework we have developed a new attack framework named robustqa for applying text adversarial attack algorithms to qa systems. this framework is an extension of openattack @xcite , which has been designed for implementing text classification adversarial attacks. robustqa consists of seven modules, depicted in figure a .1: tokenizer. the tokenizer module of robustqa supports multiple tokenization approaches, including word-, sub-word-, and character-level tokenization. it maintains the consistency between the tokenization of the original sample and that of the adversarial one, enabling the effective evaluation of the attack algorithms. furthermore, it currently supports the stanford question answering dataset (squad) dataset @xcite . however, it can be extended to support any other qa datasets, such as triviaqa @xcite , newsqa @xcite , etc. victim model. the victim model module supports the qa-based models. an extended version of this module is implemented to integrate hug-gingface transformer-based models * . this module contains multiple methods required for executing different adversarial attack scenarios in ro-bustqa. these methods can be overridden or extended for any desired customized attack, as they have access to all the sub-layers of the model's output and perform their operation as middleware. goals. the primary target of the goal module is to determine if an input sample is eligible as an adversarial candidate. the candidate sample is regarded as an eligible one if it can confuse the victim model and diminish its performance in terms of em or f1 score metrics. defining a custom goal for new qa attacks is achievable by extending the goal module. metrics. the evaluation metrics of the attack scenarios can be selected or extended with this module. as discussed in appendix b, the evaluation metrics specific to the qa task (i.e., em and f1 score) are enabled by default. other metrics such as edit distance, fluency, grammatical errors, modification rate, and semantic similarity are available for selection. attacker. the attacker is an abstract module with a default implementation of all the required tools and logic to define an attack algorithm on a given qa victim model. based on the f1 score metric and a predefined threshold value, an attack goal specific to the given qa task is defined and used as a criterion to determine the adversary potential of different input examples. the primary method of creating a custom qa attack algorithm is to extend the qa attacker module. various types of adversarial attack algorithms are derived from this module in robustqa, ready for experimentation. attack selector. the attack selector module facilitates the initiation of an attack scenario. this module enables effortless selection and instantiation of the victim model, tokenizer, dataset, attacker, and evaluation metrics. it also performs data sampling and preparation. an attack scenario is easily configured by passing the preferred settings to the attack selector module. further comprehensive analysis of the attack algorithms is possible by providing additional customized metrics to this module. evaluator. the execution and evaluation of the qa attack algorithms take place in the evaluator module. attacks performance is evaluated from different aspects: (1) the attack success rate indicates the percentage of the attacks that fool the victim model and produce false predictions; (2) the modification rate is the percentage of the modified tokens in an adversarial example compared to the input example; (3) the fluency of adversarial examples are computed by perplexity by gpt-2 @xcite ; (4) the grammatical errors of each adversarial example is compared to that of the original example, using an available language tool; (5) the semantic similarity between an input example and an adversarial example is computed using a universal sentence encoder @xcite ; and finally (6) the average time devoted to the query and attack execution is used to measure the efficacy of different attacks. ## toolkit usage the robustqa interface empowers users to execute attack scenarios either programmatically, utilizing the python programming language, or via a command-line prompt. appendix d demonstrates an example of the toolkit usage through commandline interface and code. moreover, some adversarial examples generated by different attack algorithms are depicted in appendix e. ## experiments utilizing robustqa, we have evaluated the performance of six different adversarial attack algorithms on the large uncased bidirectional encoder representations from transformer (bert) model @xcite using the squad dataset, explained in appendix c. moreover, we have augmented the training set of squad with an additional 10% adversarial examples generated through the bertattack algorithm to evaluate the robustness of a given victim model trained by the adversarial training technique. in these experiments, we have considered multiple metrics to evaluate the quality of generated adversarial examples. the results of our experiments with robustqa and our system setup are presented in appendices f and g, respectively. ## conclusion in this article, we showed the effect of various textual adversarial attack algorithms in character, word, and sentence levels on qa systems. we also developed an open-source framework, named robustqa, for the field of textual adversarial attack on qa systems, which consists of seven primary modules. this new framework offers different features that are easily customizable for applying existing or designing new algorithms, along with efficient analysis of attack scenarios. as our future work, this framework can be further extended to include other attack algorithms. we can also provide more functions and tools for further research in the context of attacks and defense within qa systems. the source code and documentation of robustqa are available at https://github.com/mirbostani/robustqa .