Detecting Word-Level Adversarial Text Attacks via SH apley Additive ex P lanations (2022)

Abstract: State-of-the-art machine learning models are prone to adversarial attacks”:” Maliciously crafted inputs to fool the model into making a wrong prediction, often with high confidence. While defense strategies have been extensively explored in the computer vision domain, research in natural language processing still lacks techniques to make models resilient to adversarial text inputs. We adapt a technique from computer vision to detect word-level attacks targeting text classifiers. This method relies on training an adversarial detector leveraging Shapley additive explanations and outperforms the current state-of-the-art on two benchmarks. Furthermore, we prove the detector requires only a low amount of training samples and, in some cases, generalizes to different datasets without needing to retrain.

## introduction adversarial examples are slightly perturbed input samples purposely crafted to fool a target model @xcite . despite being similar to the original samples, they are often misclassified with high confidence @xcite . without effective defense techniques, machine learning models become unusable in high-stakes situations and safety-critical tasks @xcite . research in computer vision has extensively worked on better understanding adversarial image attacks and developing more robust models @xcite . however, the literature in natural language processing (nlp) has witnessed fewer advances concerning this issue * these authors contributed equally @xcite @xcite . text data needs to fulfill several properties such as lexical, grammatical, and semantic constraints. thus, many efficient adversarial image attackse.g. gradient-based ones-are not transferable as they would lead to incorrect characters and nonexisting terms @xcite . however, wordlevel attacks that can preserve semantical information without introducing noticeable inconsistencies are particularly effective and not detectable via spell checkers @xcite . the lack of defense strategies against word-level text attacks motivates our research as this is a major obstacle to the safe deployment of nlp models. this work's contribution can be summarized as follows: ## methodology our defense belongs to the adversarial detection category and is strongly inspired by the work of @xcite , which detects image-based adversarial attacks for computer vision models by using shap signatures. this work, instead, studies the application of this idea to text attacks for nlp classifiers. as sketched in figure 1a , our goal pipeline consists of multiple stages. first, the input is fed to a classifier trained on the task-at-hand, which outputs a prediction. shapley values are then computed w.r.t. the outcome and passed onto a machine-learning detector that predicts whether the sample is an adversarial attack. note that our detector does not make any assumption on the classifier and is hence model-agnostic. the classifier targeted by the attacks becomes considerably more robust when used in combination with the adversarial detector. to achieve our goal, we have to take several steps in order to train our detector. these steps-also summarized in figure 1b for the reader-are described in detail in the next sections. ## conclusion adversarial text examples are a major challenge for current research and represent an obstacle for safely deploying nlp models in high-stakes applications. while attacks are hard to be distinguished from their corresponding originals, patterns in the model's reaction can be recognized and leveraged using shap signatures for detecting manipulated input samples. our work trains a machine learning detector using shap explanations of normal and adversarial samples generated with pwws. the proposed method is both intuitive and effective since it allows to detect parts of a sentence that have a suspiciously high impact on the model prediction and therefore distinguishes between regular and manipulated samples. furthermore, our detector is model-agnostic as it does not make any assumption on the classifier targeted by the attacks. our approach achieves high accuracy and considerably outperforms the previous state of the art. in terms of data efficiency, we prove that the method can achieve nearly optimal performance also when using a small portion of the available data for training. a qualitative analysis of the shap signature landscape shows most adversarial samples contained in a single cluster, suggesting that model explanations explicitly encode information to separate attacks from their counterpart. we believe this result explains why relatively simple detector architectures suffice to achieve good performance results. in terms of transferability to multiple datasets, our results are promising but yet not sufficient to prove full generalization capabilities. although in some cases we match state-of-the-art performance even when training on one dataset and testing on another, our results are highly dependent on the dataset pair. we encourage future research to continue working on generalization across multiple data sources and to evaluate performance against multiple types of attacks and models. we believe our contribution can help researchers to develop better defense strategies against attacks and thus promoting the safe deployment of nlp models in practice. we release our code to the public to facilitate further research and development 3 .