CASN :Class-Aware Score Network for Textual Adversarial Detection (2023)

Abstract: Adversarial detection aims to detect adversarial samples that threaten the security of deep neural networks, which is an essential step toward building robust AI systems. Density-based estimation is widely considered as an effective technique by explicitly modeling the distribution of normal data and identifying adversarial ones as outliers. However, these methods suffer from significant performance degradation when the adversarial samples lie close to the non-adversarial data manifold. To address this limitation, we propose a score-based generative method to implicitly model the data distribution. Our approach utilizes the gradient of the log-density data distribution and calculates the distribution gap between adversarial and normal samples through multi-step iterations using Langevin dynamics. In addition, we use supervised contrastive learning to guide the gradient estimation using label information, which avoids collapsing to a single data manifold and better preserves the anisotropy of the different labeled data distributions. Experimental results on three text classification tasks upon four advanced attack algorithms show that our approach is a significant improvement (average +15.2 F1 score against previous SOTA) over previous detection methods.

## introduction it has already become a consensus in the machine learning community that deep neural networks (dnns) are vulnerable against adversarial examples @xcite . adversarial samples are generated by adding some imperceptible perturbations to normal samples and cause the trained network to produce defective results. the widely-used pre-trained language models (plms) @xcite @xcite also have been demonstrated to be highly susceptible under textual adversarial attacks @xcite . given that pre-trained language models have become the de facto backbone models for many practical applications, their security risks deserve more attention. existing approaches to counteract adversarial attacks can be broadly divided into two directions, adversarial defense and adversarial detection @xcite . although adversarial defenses have made great progress in recent years, popular defense methods, such as adversarial training @xcite , impose certain restrictions on the attack space to certify robustness, which often results in a sacrifice of original accuracy @xcite . in contrast, adversarial detection methods aim to separate adversarial samples before they enter the model. the detected adversarial samples can be processed by a dedicated module and then re-entered into the model. this approach not only avoids the degradation of original accuracy, but also imposes no restrictions on the attack method. one of the most effective detection methods that can handle all textual attack algorithms is densitybased estimation approaches @xcite . these approaches are built on the assumption that the adversarial examples are not lying inside the non-adversarial data manifold. they explicitly model the original data distribution and use the probability of a data point as the adversarial confidence. nevertheless, recent work @xcite argues that the adversarial samples are roughly close and perpendicular to the low-dimensional manifold containing normal samples. the overlap problem poses a challenge for detection performance, as the closer the attack algorithm produces results resembling real samples, the more the detection performance is degraded. in this work, we propose to model the gradient of log-density data distribution via denoising score matching function @xcite . then the gradients are used through langevin dynamics to generate normal samples from the noise-perturbed distribution by multi-step denoising process. the distance from the adversarial samples to the normal data distribution is measured indirectly using the denoising score matching function. this is more refined than the previous direct density estimation, thus avoiding the performance loss caused by overlapping density regions. we introduce the class-aware score network (casn) to compute the gradient of log-density distribution required in the detection phase. to train this score network, with the general training objective of conditional noise scores @xcite , we also compute supervised contrastive loss @xcite ) by constructing different class sample pairs. it allows models to better distinguish between different classes of data manifold and prevents the model from collapsing into a single data distribution. afterward, all samples are denoised using the score network, and the adversarial samples are determined by recording the size of the feature distance before and after denoising. our contribution can be summarized as follows: ## preliminary score matching @xcite was proposed to generate samples from a non-normalized distribution. the core idea of this method is to estimate the score function, i.e., the gradient of logdensity data distribution, and then generate data by sampling through langevin dynamics. let x be a data point, p(x) denote the data distribution, the score function can be a score network s θ (•) that approximate ∇ x log p(x) as accurately as possible, which can be written as after that, langevin dynamics can generate samples from the data distribution p(x) using the score function. given a fixed step size , and an initial the amplitude of the noise gradually increases from @xmath0, perturbing the initial distribution eventually into a gaussian noise distribution. as for supervised contrastive learning, we treat the perturbed samples with small noise conditions as positive pairs, and the samples with different labels and their small perturbations as negative pairs, which are marked by the solid and dashed lines respectively. although the original score matching is a sound theory, @xcite points out that due to a very computationally complex term in the original training objective, it is difficult to be effective in highdimensional data. therefore, the author introduces denoising score matching (dsm) to eliminate the hard computing terms. the idea of this approach is to add an easily computable noise to the original distribution and then estimate the score function under noise perturbation. the advantage of this approach is that it makes the training target easier to compute, and the score function approximates the original target when the noise is small enough. the author proposes the use of gaussian noise p σ (x|x) = n (x|x, σ foot_0 i), then dsm minimizes the following objectives: note that the optimal score network s θ (•) which minimizes eq. 3, vincent (2011 denoising score matching using gaussian noise inspires a series of later work @xcite , and this technique has become an important milestone in the field of scorebased image generation @xcite . ## methodology in a nutshell, we hope to train a class-aware score network that estimates the gradient of log-density data distribution and separates the adversarial samples by the drift value of the sample distribution during the reverse denoising process. in §4.1, we will introduce the application of denoising score matching function and supervised contrastive learning to train the class-aware score network. by performing the denoising process on all samples using the score network, the drift distance of the samples before and after denoising can be calculated as an adversarial confidence score ( §4.2). ## experimental settings considering the attack algorithms on text classification models, we selected three representative text classification datasets to verify the effectiveness of the proposed method. they are sst-2 @xcite , imdb @xcite and ag-news @xcite . the first two datasets are both for binary sentiment analysis. in sst-2, most sentences are short texts, while in imdb, they are long. agnews is a four-category topic classification dataset that includes the world, sports, business, and sci/tech. ## experimental results in this section, we compare the detection performance of some strong baseline approaches and explore the effects of denoising process on representations. some findings of hyperparameter's selection and analytical experiments are also presented. ## ablation study to better illustrate the key components in casn, we perform an ablation study by removing supervised contrastive learning and the solution of the sde equation in the inference period. the test results are in ## conclusion in this paper, we propose a nearly-perfect solution, casn, to detect adversarial samples in text classification tasks. this framework is based on a noise conditional score network and utilizes label information to better estimate the data log-density gradient. extensive experiments show that our method greatly outperforms the strong baseline method. moreover, this approach, which exploits sample feature changes during denoising process, is experimentally shown to be more sensitive to semantic gaps of adversarial samples. we also show that a simultaneous denoising process for all samples is effective in maintaining the semantics of clean text while calibrating the adversarial ones.