Detecting Adversarial Samples through Sharpness of Loss Landscape (2023)

Abstract: Deep neural networks (DNNs) have been proven to be sensitive towards perturbations on input samples, and previous works highlight that adversarial samples are even more vulnerable than normal ones. In this work, this phenomenon is illustrated frWe first show that adversarial samples locate in steep and narrow local minima of the loss landscape (high sharpness) while normal samples, which differs distinctly from adversarial ones, reside in the loss surface that is more flatter (low sharpness).om the perspective of sharpness via visualizing the input loss landscape of models. Based on this, we propose a simple and effective sharpness-based detector to distinct adversarial samples by maximizing the loss increment within the region where the inference sample is located. Considering that the notion of sharpness of a loss landscape is relative, we further propose an adaptive optimization strategy in an attempt to fairly compare the relative sharpness among different samples. Experimental results show that our approach can outperform previous detection methods by large margins (average +6.6 F1 score) for four advanced attack strategies considered in this paper across three text classification tasks.

## introduction despite the popularity and success of pre-trained language models (plms), they are vulnerable to textual adversarial attacks @xcite . these attacks are designed to generate semantically consistent and syntactically correct adversarial samples that can fool the model into making incorrect predictions @xcite . adversarial vulnerability raises concerns about the safe practice of nlp systems in a variety of tasks * equal contribution. ## delving into input loss landscape our aim is to better understand adversarial samples, and thereby derive a potentially effective detector. in this section, we investigate the geometric properties of the input loss landscape and show a clear correlation between the sharpness of loss landscape and adversarial samples. ## proposed method in this section, we first show how a detector can be potentially designed by using loss sharpness to distinguish between adversarial and normal samples. ## experimental setup we validate the effectiveness of the proposed method on three classification benchmarks: imdb @xcite , sst-2 @xcite and agnews @xcite . the first two are binary sentiment analysis tasks that classify reviews into positive or negative sentiment, and the last one is a classification task in which articles are categorized as world, sports, business or sci/tech. we use the widely used bert base as the target model and use three attacks to generate adversarial samples for detection. ## experimental results and analysis in this section, we show the performance of the proposed method in a comprehensive way and investigate the effect of hyperparameters on performance. ## conclusion our work starts from a finding: adversarial samples locate in steep and narrow local minima of the loss landscape while normal samples, which differs distinctly from adversarial ones, reside in the loss surface that is more flatter. based on this, we propose a simple and effective sharpness-based detector that uses an adaptive optimization strategy to compute sharpness. experimental results have demonstrated the superiority of our proposed method compared to baselines, and analytical experiments have further verified the good performance of our method under different parameters.