Detection of Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation (2022)

Abstract: Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years. As a countermeasure, adversarial defense has been explored, but relatively few efforts have been made to detect adversarial examples. However, detecting adversarial examples may be crucial for automated tasks (e.g. review sentiment analysis) that wish to amass information about a certain population and additionally be a step towards a robust defense system. To this end, we release a dataset for four popular attack methods on four datasets and four models to encourage further research in this field. Along with it, we propose a competitive baseline based on density estimation that has the highest auc on 29 out of 30 dataset-attack-model combinations. The source code is released ( https://github.com/bangawayoo/adversarial-examples-in-text-classification ).

## introduction adversarial examples in nlp refer to seemingly innocent texts that alter the model prediction to a desired output, yet remain imperceptible to humans. in recent years, word-level adversarial attacks have shown success in nlp models, drastically decreasing the performance of transformer-based models in sentence classification tasks with increasingly smaller perturbation rate @xcite @xcite . in the image domain, two main lines of research exist to counteract adversarial attacks : adversarial example detection and defense. the goal of detection is to discriminate an adversarial input from a normal input, whereas adversarial defense intends to predict the correct output of the adversarial input. while works defending these attacks have shown some progress in nlp @xcite @xcite , only few efforts have been made in techniques for the sole purpose of detection. however, detecting adversarial examples may be as crucial as defending them in certain applications, in which alerting the victim of an existence of adversarial samples suffices. for instance, models used for automation of tasks (e.g. review sentiment analysis, news headline classification, etc) are adopted to efficiently gain information about the true data-generating population (e.g. consumers, news media, etc), rather than the adversary. for such applications, attaining outputs of an adversarial input -whether correct or not -may turn out to be harmful to the system. accordingly, the discard-rather-than-correct strategy which simply discards the detected adversarial input would be a good countermeasure. moreover, being able to detect adversarial examples may be a step towards building a more robust defense model as the popular defense paradigm, adversarial training, usually suffers from degraded performance on normal inputs @xcite . with a competent detection system, the normal and adversarial inputs can be processed by two separate mechanisms as proposed by @xcite . many existing works that employ detection as an auxiliary task for defense require adversarial samples for training, which may be a restrictive scenario given the variety of attack methods and sparsity of adversarial samples in the real world. in addition, some works either focus on a single type of attack @xcite or is limited to character-method summary require train data? require val. data? target attacks † rde (ours) feature-based density estimation token-level @xcite word frequency-based word-level @xcite learning-based (sentence-level) word-level @xcite learning-based (honeypot) @xcite 2019 ) @xcite learning-based (token-level) token-level @xcite learning-based (semi-character rnn) char-level as opposed to a recent work @xcite , which relies on word frequency to assess the likelihood of sentence(s), we model the probability density of the entire sentence(s). to achieve this, we fit a parametric density estimation model to the features obtained from a classification model (e.g. bert) to yield likelihoods of each sample as shown by fig. 2 inspired by classic works in novelty detection @xcite , which utilizes generative models to find anomalies. however, simply fitting a parametric model suffers from curse of dimensionality characterized by (i) sparse data points and spurious features (ii) and rare outliers that hamper accurate estimation. to tackle these issues, we leverage classical techniques in statistical analysis, namely kernel pca and minimum covariance determinant, for robust density estimation (rde). our attack-agnostic and model-agnostic detection method achieves the best performance as of auc on 29 out of 30 dataset-attack-model combinations and best performance as of tpr, f1 , auc on 25 of them without any assumption on the attacks. we further provide analysis on a stronger adversary with partial knowledge of the detection method and techniques to counteract the adversary. last, we investigate the proposed method's applicability on character-level attacks. ## related works detection of adversarial examples is a wellexplored field in the image domain. earlier works have tackled in various ways such as input transformation @xcite , statistical analysis @xcite , or training a separate binary classifier @xcite . however, @xcite has shown that an adversary with partial knowledge of the detector can easily nullify it. meanwhile, early works in novelty detection have shown that a generative model can detect anomaly samples @xcite . following this line of research, @xcite have proposed a method to detect out-of-distribution samples by training a logistic regressor on features of a neural network for maximum likelihood estimation. in the nlp domain, few efforts have been made in detecting word-level adversarial examples. @xcite utilize a detector trained on adversarial samples for a joint detect-defense system. fgws @xcite outperforms disp in detection by building on the observation that attacked samples are composed of rare words on 2 attack methods. @xcite tackle a particular attack method called unitrigger @xcite , which pre-pends or appends an identical phrase in all sentences. while the performance is impressive, applying this method to other attacks requires significant adjustment due to the distinct characteristics of unitrigger. meanwhile, @xcite tackle character-level adversarial examples and compare with spell correctors. our work is the first to extensively demonstrate experimental results for 4 popular and recent attack methods on 4 datasets and propose a competitive baseline. we summarize the methods in tab. 1. ## conclusion we propose a general method and benchmark for adversarial example detection in nlp. our method rde does not require training or validation sets for each attack algorithms, yet achieves competitive performance. in the future, a principled coutermeausre for an adversary with partial or full knowledge can be considered for robustness. a appendix a.1 success rate of attack methods and other statistics here we briefly describe each attack method and provide some statistics about the attack results. for table a .1, word transformation method indicates how candidate replacement words are created. word importance ranking denotes how the ordering of which word to attack is chosen. for constraints, only those related to embedding was listed and the numbers in parenthesis denote the threshold. higher threshold signifies stronger constraint. for more details, we refer the readers to @xcite . table a.2 summarizes the attack results on three dataset for bert. results for other models can be found in the released dataset.