TASA : Deceiving Question Answering Models by Twin Answer Sentences Attack (2022)

Abstract: We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers. Despite phenomenal progress on general adversarial attacks, few works have investigated the vulnerability and attack specifically for QA models. In this work, we first explore the biases in the existing models and discover that they mainly rely on keyword matching between the question and context, and ignore the relevant contextual relations for answer prediction.Based on two biases above, TASA attacks the target model in two folds: (1) lowering the model’s confidence on the gold answer with a perturbed answer sentence; (2) misguiding the model towards a wrong answer with a distracting answer sentence. Equipped with designed beam search and filtering methods, TASA can generate more effective attacks than existing textual attack methods while sustaining the quality of contexts, in extensive experiments on five QA datasets and human evaluations.

## introduction question answering (qa) is the cornerstone of various nlp tasks. in extractive qa (the most common setting), given a question and an associated context, a qa model needs to comprehend on the context and predict the answer @xcite . while most works keep improving the answer correctness on benchmarks @xcite , few studies investigate the robustness of qa models, e.g., is the performance achieved by sound contextual comprehension or via shortcuts like keyword matching? although adversarial attacks attract growing interests in computer vision @xcite and recently in nlp @xcite , most of them study general tasks without taking into account the properties of qa. the vulnerability and biases of models can lead to catastrophic failures outside the benchmark datasets. an effective way to study them is through adversarial attacks specifically designed for qa tasks. generating adversarial textual examples is challenging due to the discrete syntactic restriction, especially on qa, where the additional relationship between question and context should be further considered. existing works such as addsent and human-in-the-loop @xcite heavily rely on human annotators to create effective adversarial qa examples, which are costly and hard to scale. a few studies @xcite @xcite can generate adversarial samples automatically. but they only perturb either the context or the question separately, and thus ignore the consistency between them. moreover, the major pitfalls of qa models' detailed comprehension process are not fully investigated, confining producing more powerful adversarial attacks. in this paper, we develop an adversarial attack specifically targeting two biases of mainstream qa models discussed in §2: (1) making prediction via keywords matching in the answer sentence of contexts; and (2) ignorance of the entities shared between the question and context. our method, twin answer sentences attack (tasa), automatically produces black-box adversarial attacks @xcite perturbing a context without hurting its fluency or changing the gold answer. tasa firstly allocates the answer sentence in the context that is decisive for answering @xcite and then modify it into two sentences targeting the two biases above: one sentence preserves the gold answer and the meaning but replaces the keywords that are shared with the question with their synonyms; while the other leaves the keywords and the syntactic structure intact but changes the entities (subjects/objects) associated with the answer. thereby, the former is a perturbed answer sentence (pas) lowering the focus of the model on the gold answer, while the latter generates a distracting answer sentence (das) as @xcite to further misguide the model towards a wrong answer with respect to irrelevant entities. thus, the adversarial context can substantially distort the qa model without changing the answer for humans. to address the challenge of efficiency and textual fluency, we further propose beam search and filtering techniques empowered by pretrained models. in experiments, we evaluate tasa and other adversarial attack baselines on attacking three popular contextualized qa models, bert @xcite , spanbert @xcite , and bidaf @xcite , on five extractive qa datasets, i.e., squad 1.1, newsqa, naturalquestions, hotpotqa, and triviaqa. experimental results and human evaluations consistently show that tasa achieves better attack capability than other baselines and meanwhile preserves the textual quality and gold answers identifiable by humans. our contributions are three-fold: we release our code at https://github.com/ caoyu-noob/tasa ## predicting bias in question answering recent works show that state-of-the-art natural language inference models often overly rely on certain keywords as shortcuts for prediction @xcite . in the empirical study of this section, we illustrate that current qa models consistently exhibit such bias on the sensitive words without leveraging the contextual relationship for predicting answers. we analyze two mainstream qa models with contextualized comprehension capabilities, bert @xcite and bidaf @xcite , trained on the original training set of squad1.1 @xcite and tested on samples modified from its validation set. we define the sentence in the context that contains the gold answer as the answer sentence, which is the key for predicting answers @xcite . we first compare the performance of models on the original sample with only answer sentence as the context ("only answer sent."). besides, to investigate the bias on sensitive words, we further examine models on samples with various types of sensitive words in the answer sentence being (1) either removed ("remove") or (2) only retained ("only"). there are three types of sensitive words to be considered: the annual nfl experience was held at the located in san francisco. ## methodology we propose an adversarial attack method for extractive qa models, twin answer sentences attack (tasa), which automatically produces black-box attacks solely based on the final output of the victim qa model f (•). given a typical qa sample composed of a context c, a question q, and an answer a (i.e., a positional span in c), we study how to perturb the context c as c that can deceive f (•) towards producing an incorrect answer f (c , q) = a, while c retains the correct answer a that can be identified by humans. we keep the question q intact to ensure the answer a valid, as editing the short q with simple syntactic structure easily alters its meaning. tasa can be summarized as three main steps: (1) remove coreferences in the context to facilitate the following edits; (2) perturb the answer sentence by replacing keywords (overlapped sensitive lexical words in §2) with synonyms to produce a perturbed answer sentence (pas), lowering the model's focus on the gold answer; (3) add a distracting answer sentence (das) that keeps the keywords intact but changes the associated subjects/objects to misguide the model for producing a wrong answer, which can be proven in ## experiments we evaluate tasa on extractive qa tasks. we begin by details of setup ( §4.1), then introduce the main results in §4.2, followed by ablation studies in §4.3 and additional analysis in §4.4 to better illustrate each module in our method. ## conclusion we present tasa, an automatic adversarial attack method for qa models. it generates twin answer sentences, perturbed answer sentence (pas), and distracting answer sentence (das), to construct a new adversarial context in a qa sample. it can deceive models and misguide them to an incorrect answer based on their pitfalls that overly rely on matching sensitive keywords during predicting answers. in experiments, tasa achieves remarkable attack performance on five datasets and three victim models with satisfactory sample quality. our additional analysis also proves that it is possible to get more robust qa models via tasa in the future. adding distracting answer sentences. we construct a ner dictionary and a word dictionary (except named entities) for each target dataset by parsing all contexts in both the train and dev sets via spacy. during generating das or changing answers in das, we randomly sample named entities with the same ner tag or words with the same pos tag from the dictionaries we built before. each time, we sample @xmath0. pyinflect is also used during replacement.