Towards Robustness against Unsuspicious Adversarial Examples

Despite the remarkable success of deep neural networks , significant concerns have emerged about their robustness to adversarial perturbations to inputs . While most attacks aim to ensure that these are imperceptible , physical perturbation attacks typically aim for being unsuspicious , even if perceptible . However , there is no universal notion of what it means for adversarial examples to be unsuspicious . We propose an approach for modeling suspiciousness by leveraging cognitive salience . Specifically , we split an image into foreground ( salient region ) and background ( the rest ) , and allow significantly larger adversarial perturbations in the background , while ensuring that cognitive salience of background remains low . We describe how to compute the resulting non-salience-preserving dual-perturbation attacks on classifiers . We then experimentally demonstrate that our attacks indeed do not significantly change perceptual salience of the background , but are highly effective against classifiers robust to conventional attacks . Furthermore , we show that adversarial training with dual-perturbation attacks yields classifiers that are more robust to these than state-of-the-art robust learning approaches , and comparable in terms of robustness to conventional attacks . 1 INTRODUCTION . An observation by Szegedy et al . ( 2014 ) that state-of-the-art deep neural networks that exhibit exceptional performance in image classification are fragile in the face of small adversarial perturbations of inputs has received a great deal of attention . A series of approaches for designing adversarial examples followed ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ) , along with methods for defending against them ( Papernot et al. , 2016b ; Madry et al. , 2018 ) , and then new attacks that defeat prior defenses , and so on . Attacks can be roughly classified along three dimensions : 1 ) introducing small lp-norm-bounded perturbations , with the goal of these being imperceptible to humans ( Madry et al. , 2018 ) , 2 ) using non-lp-based constraints that capture perceptibility ( often called semantic perturbations ) ( Bhattad et al. , 2020 ) , and 3 ) modifying physical objects , such as stop signs ( Eykholt et al. , 2018 ) , in a way that does not arouse suspicion . One of the most common motivations for the study of adversarial examples is safety and security , such as the potential for attackers to compromise the safety of autonomous vehicles that rely on computer vision ( Eykholt et al. , 2018 ) . However , while imperceptibility is certainly sufficient for perturbations to be unsuspicious , it is far from necessary , as physical attacks demonstrate . On the other hand , while there are numerous formal definitions that capture whether noise is perceptible ( Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017 ) , what makes adversarial examples suspicious has been largely informal and subjective . We propose a simple formalization of an important aspect of what makes adversarial perturbations unsuspicious . Specifically , we make a distinction between image foreground and background , allowing significantly more noise in the background than the foreground . This idea stems from the notion of cognitive salience ( Borji et al. , 2015 ; Kmmerer et al. , 2017 ; He & Pugeault , 2018 ) , whereby an image can be partitioned into the two respective regions to reflect how much attention a human viewer pays to the different parts of the captured scene . In effect , we posit that perturbations in the foreground , when visible , will arouse significantly more suspicion ( by being cognitively more salient ) than perturbations made in the background . Our first contribution is a formal model of such dual-perturbation attacks , which is a generalization of the lp-norm-bounded attack models ( see , e.g. , Figure 1 ) , but explicitly aims to ensure that adversarial perturbation does not make the background highly salient . Second , we propose an algorithm for finding adversarial examples using this model , which is an adaptation of the PGD attack ( Madry et al. , 2018 ) . Third , we present a method for defending against dual-perturbation attacks based on the adversarial training framework ( Madry et al. , 2018 ) . Finally , we present an extensive experimental study that demonstrates that ( a ) the proposed attacks are significantly stronger than PGD , successfully defeating all state-of-the-art defenses , ( b ) proposed defenses using our attack model significantly outperform state-of-the-art alternatives , with relatively small performance degradation on nonadversarial instances , and ( c ) proposed defenses are comparable to , or better than alternatives even against traditional attacks , such as PGD . Related Work : Recent studies have shown that neural networks are vulnerable to adversarial examples . A variety of approaches have been proposed to produce adversarial examples ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Papernot et al. , 2016a ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017 ) . These approaches commonly generate adversarial perturbations within a bounded ` p norm so that the perturbations are imperceptible . A related thread has considered the problem of generating adversarial examples that are semantically imperceptible without being small in norm ( Brown et al. , 2018 ; Bhattad et al. , 2020 ) , for example , through small perturbations to the color scheme . However , none of these account for the perceptual distinction between the foreground and background of images . Numerous approaches have been proposed for defending neural networks against adversarial examples ( Papernot et al. , 2016b ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Cohen et al. , 2019 ; Madry et al. , 2018 ; Raghunathan et al. , 2018 ) . Predominantly , these use ` p-bounded perturbations as the threat model , and while some account for semantic perturbations ( e.g . Mohapatra et al . ( 2020 ) ) , none consider perceptually important difference in suspiciousness between foreground and background . Two recent approaches by Vaishnavi et al . ( 2019 ) and Brama & Grinshpoun ( 2020 ) have the strongest conceptual connection to our work . Both are defense-focused by either eliminating ( Vaishnavi et al. , 2019 ) or blurring ( Brama & Grinshpoun , 2020 ) the background region for robustness . However , they assume that we can reliably segment an image at prediction time , leaving the approach vulnerable to attacks on image segmentation ( Arnab et al. , 2018 ) . Xiao et al . ( 2020 ) propose to disentangle foreground and background signals on images but unsuspiciousness of their attacks is not ensured . 2 BACKGROUND . 2.1 ADVERSARIAL EXAMPLES AND ATTACKS . The problem of generating adversarial examples is commonly modeled as follows . We are given a a learned model hθ ( · ) parameterized by θ which maps an input x to a k-dimensional prediction , where k is the number of classes being predicted . The final predicted class yp is obtained by yp = arg maxi hθ ( x ) i , where hθ ( x ) i is the ith element of hθ ( x ) . Now , consider an input x along with a correct label y . The problem of identifying an adversarial example for x can be captured by the following optimization problem : max δ∈∆ ( ) L ( hθ ( x+ δ ) , y ) , ( 1 ) where L ( · ) is the adversary ’ s utility function ( for example , the loss function used to train the classifier hθ ) . ∆ ( ) is the feasible perturbation space which is commonly represented as a ` p ball : ∆ ( ) = { δ : ‖δ‖p ≤ } . A number of approaches have been proposed to solve the optimization problem shown in Eq . ( 1 ) , among which two are viewed as state of the art : CW attack developed by Carlini & Wagner ( 2017 ) , and Projected Gradient Descent ( PGD ) attack proposed in Madry et al . ( 2018 ) . In this work , we focus on the PGD attack with ` ∞ and ` 2 as the distance metrics . 2.2 ROBUST LEARNING . An important defense approach that has proved empirically effective even against adaptive attacks is adversarial training ( Szegedy et al. , 2014 ; Cohen et al. , 2019 ; Goodfellow et al. , 2015 ; Madry et al. , 2018 ) . The basic idea of adversarial training is to produce adversarial examples and incorporate these into the training process . Formally , adversarial training aims to solve the following robust learning problem : min θ 1 |D| ∑ x , y∈D max ‖δ‖p≤ L ( hθ ( x+ δ ) , y ) , ( 2 ) where D is the training dataset . In practice , this problem is commonly solved by iteratively using the following two steps ( Madry et al. , 2018 ) : 1 ) use a PGD ( or other ) attack to produce adversarial examples of the training data ; 2 ) use any optimizer to minimize the loss of those adversarial examples . It has been shown that adversarial training can significantly boost the adversarial robustness of a classifier against ` p attacks , and it can be scaled to neural networks with complex architectures . 3 DUAL-PERTURBATION ATTACKS . 3.1 MOTIVATION . Our threat model is motivated by the feature integration theory ( Treisman & Gelade , 1980 ) in cognitive science : regions that have features that are different from their surroundings are more likely to catch a viewer ’ s gaze . Such regions are called salient regions , or foreground , while the others are called background . Accordingly , for a given image , the semantics of the object of interest is more likely to be preserved in the foreground , as it catches more visual attention of a viewer compared to the background . If the foreground of an image is corrupted , then the semantics of the object of interest is broken . In contrast , the same extent of corruption in the background nevertheless preserves the overall semantic meaning of the scene captured ( see , e.g. , Figure 2 ) . Indeed , detection of salient regions , as well as the segmentation of foreground and background , have been extensively studied in computer vision ( Borji et al. , 2015 ) . These approaches either predict human fixations , which are sparse bubble-like salient regions sampled from a distribution ( Kmmerer et al. , 2017 ) , or salient objects that contain smooth connected areas in an image ( He & Pugeault , 2018 ) . Despite this important cognitive distinction between foreground and background , essentially all of the attacks on deep neural networks for image classification make no such distinction , even though a number of other semantic factors have been considered ( Bhattad et al. , 2020 ; Mohapatra et al. , 2020 ) . Rather , much of the focus has been on adversarial perturbations that are not noticeable to a human , but which are applied equally to the entire image . However , in security applications , the important issue is not merely that an attack can not be noticed , but that whatever observed is not suspicious . This is , indeed , the frame of reference for many high-profile physical attacks on image classification , which are clearly visible , but not suspicious because they hide in the “ human psyche ” , that is , are easily ignored ( Sharif et al. , 2016 ; Eykholt et al. , 2018 ) . The main goal of the threat model we introduce next is therefore to capture more precisely the notion that an adversarial example is not suspicious by leveraging the cognitive distinction between foreground and background of an image .

Summary: I like the idea to check whether it might be possible that noise can be injected selectively in areas where it is likely to make the adversarial example less suspicious and thus it might be possible to inject more noise in those directions. But the challenge is that this might increase the saliency of those areas. The authors propose to use recent work from DeepGazeII to detect saliencies of various pixels.