Detecting Textual Adversarial Examples Based on Distributional Characteristics of Data Representations (2022)

Abstract: Although deep neural networks have achieved state-of-the-art performance in various machine learning tasks, adversarial examples, constructed by adding small non-random perturbations to correctly classified inputs, successfully fool highly expressive deep classifiers into incorrect predictions. Approaches to adversarial attacks in natural language tasks have boomed in the last five years using character-level, word-level, phrase-level, or sentence-level textual perturbations. While there is some work in NLP on defending against such attacks through proactive methods, like adversarial training, there is to our knowledge no effective general reactive approaches to defence via detection of textual adversarial examples such as is found in the image processing literature. In this paper, we propose two new reactive methods for NLP to fill this gap, which unlike the few limited application baselines from NLP are based entirely on distribution characteristics of learned representations”:” we adapt one from the image processing literature (Local Intrinsic Dimensionality (LID)), and propose a novel one (MultiDistance Representation Ensemble Method (MDRE)). Adapted LID and MDRE obtain state-of-the-art results on character-level, word-level, and phrase-level attacks on the IMDB dataset as well as on the later two with respect to the MultiNLI dataset. For future research, we publish our code .

## introduction highly expressive deep neural networks are fragile against adversarial examples, constructed by carefully designed small perturbations of normal examples, that can fool deep classifiers to make wrong predictions @xcite . crafting adversarial examples in images involves adding small non-random perturbations to many pixels in inputs that should be correctly classified by a target model. these perturbations can force high-efficacy models into incorrect classifications and are often imperceptible to humans @xcite @xcite @xcite . however, when adversarial examples have been studied in the context of text, to our knowledge, only @xcite aligns closely with the original intuition of adversarial examples in applying perturbations to word embeddings, which are inputs of deep neural nets. rather, most adversarial attack techniques use more practical semantics-preserving textual changes other than embedding perturbations, at character-level, wordlevel, phrase-level, or sentence-level @xcite @xcite @xcite @xcite @xcite ; see generating adversarial examples to attack deep neural nets and protecting deep neural nets from adversarial examples have been extensively studied in image classification tasks @xcite @xcite @xcite @xcite @xcite . however, in the natural language domain, only crafting of adversarial examples has been comprehensively considered @xcite @xcite @xcite . defence against textual adversaries, primarily through increasing the robustness of deep neural networks, is much less studied @xcite . in the image processing space, @xcite refers to these as proactive defence methods, and @xcite notes that they can be evaded by optimization-based attacks, such as constructing new loss functions; in the nlp space, this is a story of two misfits who don't stand a chance alone, but together they are magnificent. positive character-level @xcite tzytis is a sotry of two misifts who don't stad a ccange aluone, but tpgthr they are mgnificent. negative word-level @xcite this is a conte of two who don't stands a opportunities alone, but together they are opulent. negative phrase-level @xcite why don't you have two misfits who don't stand a chance alone, but together they're beautiful. negative sentence-level @xcite this is a story of two misfits who don't stand a chance alone, but together they are magnificent. ready south hundred at size expected worked whose turn poor 2018 ) explore reactive defence methods @xcite in the image processing space: these focus on distinguishing real from adversarial examples, in order to detect them before they are passed to neural networks. these reactive defences have been explored in only a limited way in the nlp space @xcite . importantly, these few methods rely on procedures like testing word substitutions, quite unlike those in the image processing space, which are functions of the learned representations. the contributions of this paper are two textual adversarial reactive detectors as follows: ## related work in this section, we briefly review state-of-the-art work on defending neural networks against both image and textual adversarial examples. image adversarial defences: adversarial training @xcite using adversarial examples to augment training data or adding an adversarial objective to a loss function, and defensive distillation framework @xcite which transfers knowledge between same struc-tured teacher and student models, are two effective proactive defence methods. for reactive defences, feinman et al. (2017); ma et al. (2018); papernot and mcdaniel (2018); lee et al. (2018) have all proposed approaches that use the learned representations of the classifier that the attacker is trying to fool, and then with a variety of techniques to identify characteristics of the adversarial examples' learned representations that permit the detection of whether a data point is adversarial or original; these techniques involve kernel density estimations in a feature space of a last hidden layer and bayesian uncertainty estimates, local intrinsic dimensionality, deep k-nearest neighbors, and mahalanobis distance-based confidence scores respectively. textual adversarial defences: adversarial training @xcite ) is a commonly used defence method to augment training data with adversarial examples and their correct labels, which has been effective in @xcite , @xcite , @xcite , and @xcite , but has limited utility in @xcite and @xcite . @xcite applies interval bound propagation (ibp) to minimize an upper bound of possible candidate sentences' losses when facing word substitution adversaries. @xcite introduced robust encodings (roben) to cluster words and typos, and produced one encoding for each cluster to harness adversarial typos. @xcite proposed the learning to discriminate perturbations (disp) framework to block character-level and word-level adversarial perturbations by recognising and replacing perturbed words. @xcite noticed and verified a characteristic of word-level adversaries that replacement words are less likely to occur than their substitutions, therefore, they constructed a rule-based, model-agnostic frequency-guided word substitutions (fgws) algorithm, which is the only existing textual reactive defence method as far as we know. ## conclusion and future work in this paper, we adapted local intrinsic dimensionality (lid) method @xcite from image processing and proposed a simple and general textual adversarial reactive detector, multidistance representation ensemble method (mdre), based on the distribution characteristics of adversarial examples representations, that they are outof-distribution samples and lie off the true data manifold. the experimental results show adapted lid and mdre achieve state-of-the-art results on detecting character-level, word-level, and phraselevel adversaries on the imdb dataset as well as on the later two with respect to the multinli dataset. the results show that it is possible to construct adversarial example detectors using only the learned representations, and not relying on various textual substitution processes as in the baselines. as discussed in section 3, adapted lid uses estimated local intrinsic dimensionality on text repre-sentations form different layers outputs of a target model, and mdre is implemented on euclidean distances between samples' representations and representations of their nearest neighbors among the training examples with the same predicted labels from different representation learning models, to characterise representation distribution differences between adversarial examples and normal examples. in terms of future work and the lid approach, @xcite found that in the image processing space, lid is vulnerable to their backward pass differentiable approximiation (bpda) attack; it would be useful to investigate whether this is the case in the text space, and if so, other detection methods from image processing may be worth looking into. with respect to mdre, as it is a kind of nearest-neighbour ensembling approach, looking into other possibilities falling within that space could be productive. more generally, exploring more effective distribution characteristics of data semantic representations among adversarial and normal examples, may help to build better detectors.