Toward Stronger Textual Attack Detectors (2023)

Abstract: The landscape of available textual adversarial attacks keeps growing, posing severe threats and raising concerns regarding deep NLP systems integrity. However, the crucial problem of defending against malicious attacks has only drawn few attention in the NLP community. The latter is nonetheless instrumental to develop robust and trustworthy systems. This paper makes two important contributions in this line of search: (i) we introduce LAROUSSE, a new framework to detect textual adversarial attacks and (ii) we introduce STAKEOUT, an extended benchmark composed of nine popular attack methods, three datasets and two pre-trained models. LAROUSSE is ready-to-use in production as it is unsupervised, hyperparameter free and non-differentiable, protecting it against gradient-based methods. Our new benchmark STAKEOUT allows for a robust evaluation framework: we conduct extensive numerical experiments which demonstrate that LAROUSSE outperforms previous methods, and which allows to identify interesting factor of detection rate variations.

## introduction despite the high performances of deep learning techniques for natural language processing (nlp) applications, the trained models remain vulnerable to adversarial attacks @xcite which limits their adoption for critical applications. in the context of nlp, for a given model and a given textual input, an adversarial example is a carefully constructed modification of the initial text such that it is semantically similar to the original text while affecting the model's prediction. the ability to design adversarial examples @xcite @xcite raises serious concerns regarding the security of nlp systems. it is, therefore, cru-* these authors contributed equally to this work cial to develop proper strategies that are available to deal with these threats @xcite . perhaps surprisingly, if the research community has invested considerable efforts to design efficient attacks, there are only a few works that address the issue of preventing them. one can distinguish two lines of research: detection methods that aim at discriminating between regular input and attacks; and defense methods that try to correctly classify adversarial inputs. the latter is based on robust training methods, which customize the learning process, see for instance @xcite @xcite . these are limited to certain types of adversarial lures (e.g., misspelling), making them vulnerable to other types of attacks that already exist or may be designed in the future. in contrast, detection methods are more relevant to real-life scenarios where practitioners usually prefer to adopt a discard-rather-than-correct strategy @xcite . this has been highlighted in @xcite which is, to the best of our knowledge, the single word that introduces a detection method that does not require training. on the contrary, the authors propose to measure the regularity of a given input by computing the mahalanobis distance @xcite of its embedding in the last layer of a transformer with respect to the training distribution. notice that the mahalanobis distance has also been successfully used in a very similar framework of out-of-distribution (ood) detection methods (see @xcite and references therein). in this paper, we build upon @xcite ] and introduce a new attack detection framework, called larousse foot_0 , which improves the current state-of-the-art. our approach is based on the computation of the halfspace-mass depth @xcite of the last layer embedding of an input with respect to the training distribution. halfspace-mass depth is a particular instance of data depth, which are functions that measure the proximity of a point to the core of a probability distribution. as a matter of fact, the mahalanobis distance is also -probably one of the most popular -a data depth. interestingly, in addition to improving the attack detection rate, the halfspace-mass depth remedies several limitations of the mahalanobis depth: it does not make gaussian assumptions on the data structure and is additionally non-differentiable, providing security guarantees regarding malicious adversaries that could rely on gradient-based methods. the second contribution of our work consists in releasing stakeout, a new nlp attack benchmark that enriches the one introduced in @xcite . more precisely, we explore the same datasets and extend their four attacks by adding five new adversarial techniques. this ensures a wider variety of testing methods, leading to a robust evaluation framework that we believe will stimulate future research efforts. we conduct extensive numerical experiments on stakeout and demonstrate the soundness of our larousse detector while studying the main variability factors on its performance. finally, we empirically observe the presence of relevant information to detect attacks across the layers other than the last one. this could pave the way for future research by considering the possibility of building detectors that are not limited to the last embedding layers but rather exploit the full network information. our contributions in a nutshell. our contributions are threefold: the rest of the paper is organized as follows. in sec. 2, we briefly review the setting of textual attacks, provide main references on the subject, and formally introduce the problem of attack detection. in sec. 3, we present our larousse detector and provide some perspectives on data depth and connections to the mahalanobis distance. in sec. 4, we introduce our new benchmark stakeout and give details on the evaluation framework of attack detection. finally, we present our experimental results in sec. 5. ## textual attacks: generation and detection let us first introduce some notations. we will denote by @xmath0, y i )} 1≤@xmath1. we focus on classification tasks, meaning that y is of finite size: |y| < +∞. in this work, the inputs are first embedded through a multi-layer encoder with l layers and learnable parameters ψ ∈ ψ. we denote by f ℓ ψ : @xmath2. note that, as we will work on transformer models, the latent space dimension-the dimension of the output of a layer-of all layers is the same and will be denoted by d. the dimension of the logits, denoted as the (l + 1)-th layer of the encoder, is d ′ . the final classifier built on the pre-trained encoder produces a soft decision c ψ over the classes, where ψ is a learned parameter. we will denote by c ψ (c | x) the predicted probability that a given input x belongs to class c. given an input x, the predicted label ŷ is then obtained as follows: ## larousse: a novel adversarial attacks detector we follow the notations introduced in sec. 2. in particular, recall that f l ψ : @xmath3. ## stakeout: a novel benchmark for adversarial attacks textual attack generation can be computationally expensive as some attacks require hundreds of queries to corrupt a single sample 2 . to dispose of a benchmark that gathers the result of diverse attacks on different datasets and encoders is instrumental to accelerate future research efforts by reducing computational overhead. to build our benchmark, we relied on the models, the datasets, and the attacks available in textattack @xcite . in the following, we describe the experimental choices we made when building stakeout and discuss our baseline and evaluation pipeline. ## additional details on stakeout construction for completeness, we regroup in tab. 3 additional details on the attacks used in stakeout. in this paper we focused our evaluation on existing attacks. in the future, a possible extension of stakeout would be to use the methodology of dynabench @xcite and rely on human feedback to attack the model. algorithm 1 training algorithm for the approximation of d hm . ## additional results this section gathers additional experimental results to allow the curious reader to draw fine conclusions. formally, we conduct: full name acronym idea type of constraint pruthi @xcite pru simulation of common typos using greedy search for untargeted classification minimum word length, maximum number of words perturbed textbugger @xcite tb character-based attack (i.e swap, deletion, subtitution) cosine with use @xcite iga @xcite ] ig genetic algorithm to perform word substitution percentage of perturbed words and word embedding distance on word2vect @xcite ] deepwordbug @xcite dwb character-based attack (i.e swap, deletion, subtitution) cosine and language model similarity clare @xcite cla attack using token insertion, merge and swap embedding similarity bae @xcite ] bae attack using bert mlm combined with a greedy search number of perturbed words and cosine with use @xcite ] pwws @xcite pwws word swap based on wordnet synonyms textfooler @xcite tf attack using embedding swap embedding similarity and pos match with word and embedding swap tf-adjusted @xcite tf-adf attack using embedding swap use and word embedding similarity input-reduction @xcite ir greedy attack using word importance ranking via greedy search checklist @xcite chk using contraction/extension and changing numbers, locations, names ## computation time comparison between hm and mahalanobis depths in this part, we compare the computation time between the hm and the mahalanobis depths. b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.5 0.6 0.7 0.8 0.9 1.0 auroc roberta-base b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack bert-base-uncased method d m larousse gpt2 (a) auroc b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.0 0.2 0.4 0.6 0.8 1.0 fpr roberta-base b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack bert-base-uncased method d m larousse gpt2 (b) fpr b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.5 0.6 0.7 0.8 0.9 1.0 auprin roberta-base b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack bert-base-uncased method d m larousse gpt2 (c) aupr-in b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.5 0.6 0.7 0.8 0.9 1.0 auprout roberta-base b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack bert-base-uncased method d m larousse gpt2 (d) aupr-out figure 7: extended figures for sec. 5.2. in these figures, we report the gpt2 baseline and the performance of all the detectors in terms of auroc, aupr-in, aupr-out and fpr. precisely, we want to compare the time of computing these data depths of an element @xmath4.5 0.6 0.7 0.8 0.9 1.0 auroc sst2 b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack ag-news b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack imdb method d m larousse gpt2 (a) auroc b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.0 0.2 0.4 0.6 0.8 1.0 fpr sst2 b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack ag-news b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack imdb method d m larousse gpt2 (b) fpr b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.5 0.6 0.7 0.8 0.9 1.0 aupr-in sst2 b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack ag-news b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack imdb method d m larousse gpt2 (c) aupr-in b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack 0.5 0.6 0.7 0.8 0.9 1.0 aupr-out sst2 b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack ag-news b a e p w w s t f t f -a d j d w b i g k u l p r u t b attack imdb method d m larousse gpt2 (d) aupr-out w.r.t. a dataset. this experiment is conducted as follows: several datasets (varying dimension {800, 1000, 1200, 1500, 2000, 2500, 5000} and sample size {100, 2500, 5000, 7500, 10000}) are sampled from centered gaussian distributions n (0, σ) where σ follows a wishart distribution. therefore, we compute the depth of 0 w.r.t. each of these datasets. this procedure is repeated 10 times. we report their mean computation time as well as 10-90% quantiles in figure 11 highlighting the computational benefits of using the hm depth over the mahalanobis distance. the dimension is 5000 on the left picture while the sample size is fixed to 100 on the right picture.