BREAKING  CERTIFIED  DEFENSES:  SEMANTIC  ADVERSARIAL  EXAMPLES  WITH  SPOOFED  ROBUSTNESS  CERTIFICATES

1 INTRODUCTION . Conventional training of neural networks has been shown to produce classifiers that are highly sensitive to adversarial perturbations ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) , “ natural looking ” images that have been manipulated to cause misclassification by a neural network ( Figure 1 ) . While a wide range of defenses exist that harden neural networks against such attacks ( Madry et al. , 2017 ; Shafahi et al. , 2019 ) , defenses based on heuristics and tricks are often easily breakable Athalye et al . ( 2018 ) . This has motivated work on certifiably secure networks — classifiers that produce a label for an image , and also ( when possible ) a rigorous guarantee that the input is not adversarially manipulated ( Cohen et al. , 2019 ; Zhang et al. , 2019b ) . ∗equal contribution To date , all work on certifiable defenses has focused on deflecting ` p-bounded attacks , where p = 2 or ∞ ( Cohen et al. , 2019 ; Gowal et al. , 2018 ; Wong et al. , 2018 ) . After labelling an image , these defenses then check whether there exists an image of a different label within units ( in the ` p metric ) of the input , where is a security parameter chosen by the user . If the classifier assigns all images within the ball the same class label , then a certificate is issued , and the input image known not to be an ` p adversarial example . In this work , we demonstrate how a system that relies on certificates as a measure of label security can be exploited . We present a new class of adversarial examples that target not only the classifier output label , but also the certificate . We do this by adding adversarial perturbations to images that are large in the ` p norm ( larger than the used by the certificate generator ) , and produce attack images that are surrounded by a large ball exclusively containing images of the same label . The resulting attacks produce a “ spoofed ” certificate with a seemingly strong security guarantee despite being adversarially manipulated . Note that the statement made by the certificate ( i.e. , that the input image is not an adversarial example in the chosen norm ) is still technically correct , however in this case the adversary is hiding behind a certificate to avoid detection by a certifiable defense . In summary , we consider methods that attack a certified classifier in the following sense : • Imperceptibility : the adversarial example “ looks like ” its corresponding natural base example , • Misclassification : the certified classifier assigns an incorrect label to the adversarial example , and • Strongly certified : the certified classifier provides a strong/large-radius certificate for the adversarial example . While the existence of such an attack does not invalidate the certificates produced by certifiable systems , it should serve as a warning that certifiable defenses are not inherently secure , and one should take caution when relying on certificates as an indicator of label correctness . BACKGROUND In the white-box setting , where the attacker knows the victim ’ s network and parameters , adversarial perturbation are often constructed using first-order gradient information ( Carlini & Wagner , 2017 ; Kurakin et al. , 2016 ; Moosavi-Dezfooli et al. , 2016 ) or using approximations of the gradient ( Uesato et al. , 2018 ; Athalye et al. , 2018 ) . The prevailing formulation for crafting attacks uses an additive adversarial perturbation , and perceptibility is minimized using an ` p-norm constraint . For example , ` ∞-bounded attacks limit how much each pixel can move , while ` 0 adversarial attacks limit the number of pixels that can be modified , without limiting the size of the change to each pixel ( Wiyatno & Xu , 2018 ) . It is possible to craft imperceptible attacks without using ` p bounds ( Brown et al. , 2018 ) . For example , Hosseini & Poovendran ( 2018 ) use shifting color channels , Wong et al . ( 2019 ) use the Wasserstein ball/distance , and Engstrom et al . ( 2017 ) use rotation and translation to craft “ semantic ” adversarial examples . In Figure 1 , we produce semantic adversarial examples using the method of Hosseini & Poovendran ( 2018 ) which is a greedy approach that transforms the image into HSV space , and then , while keeping V constant , tries to find the smallest S perturbation causing misclassification1 . Other variants use generative models to construct natural looking images causing misclassification ( Song et al. , 2018 ; Dunn et al. , 2019 ) . In practice , many of the defenses which top adversarial defense leader-board challenges are noncertified defenses ( Madry et al. , 2017 ; Zhang et al. , 2019a ; Shafahi et al. , 2019 ) . The majority of these defenses make use of adversarial training , in which attack images are crafted during training and injected into the training set . These non-certified defenses are mostly evaluated against PGDbased attacks , resulting in an upper-bound on robustness . Certified defenses , on the other-hand , provably make networks resist ` p-bounded perturbations of a certain radius . For instance , randomized smoothing ( Cohen et al. , 2019 ) is a certifiable defense against ` 2-norm bounded attacks , and CROWN-IBP ( Zhang et al. , 2019b ) is a certifiable defense 1In fig . 1 , the adversarial example has saturation=0 against l∞-norm bounded perturbations . Both of these defenses produce a class label , and also a guarantee that the image could not have been crafted by making small perturbations to an image of a different label . Certified defenses can also benefit from adversarial training . For example , Salman et al . ( 2019 ) recently improved the certified radii of randomized smoothing ( Cohen et al. , 2019 ) by training on adversarial examples generated for the smoothed classifier . To the best of our knowledge , prior works have focused on making adversarial examples that satisfy the imperceptibility and misclassification conditions , but none have investigated manipulating certificates , which is our focus here . The reminder of this paper is organized as follows . In section 2 we introduce our new approach Shadow Attack for generating adversarial perturbations . This is a hybrid model that allows various kinds of attacks to be compounded together , resulting in perturbations of large radii . In section 3 we present an attack on “ randomized smoothing ” certificates ( Cohen et al. , 2019 ) . Section 4 shows an ablation study which illustrates why the elements of the Shadow Attack are important for successfully manipulating certified models . In section 5 we generate adversarial examples for “ CROWN-IBP ” ( Zhang et al. , 2019b ) . Finally , we discuss results and wrap up in section 6 . 2 THE SHADOW ATTACK . Because certificate spoofing requires large perturbations ( larger than the ` p ball of the certificate ) , we propose a simple attack that ensembles numerous modes to create large perturbations . Our attack can be seen as the generalization of the well-known PGD attack , which creates adversarial images by modifying a clean base image . Given a loss function L and an ` p-norm bound for some p ≥ 0 , PGD attacks solve the following optimization problem : max δ L ( θ , x+ δ ) ( 1 ) s.t . ‖δ‖p ≤ , ( 2 ) where θ are the network parameters and δ is the adversarial perturbation to be added to the clean input image x. Constraint 2 promotes imperceptibility of the resulting perturbation to the human eye by limiting the perturbation size . In the shadow attack , instead of solving the above constrained optimization problem , we solve the following problem with a range of penalties : max δ L ( θ , x+ δ ) − λcC ( δ ) − λtvTV ( δ ) − λsDissim ( δ ) , ( 3 ) where λc , λtv , λs are scalar penalty weights . Penalty TV ( δ ) forces the perturbation δ to have small total variation ( TV ) , and so appear more smooth and natural . Penalty C ( δ ) limits the perturbation δ globally by constraining the change in the mean of each color channel c. This constraint is needed since total variation is invariant to constant/scalar additions to each color channel , and it is desirable to suppress extreme changes in the color balances of images . Penalty Dissim ( δ ) promotes perturbations δ that assume similar values in each color channel . In the case of an RGB image of shape 3 ×W × H , if Dissim ( δ ) is small , the perturbations to red , green , and blue channels are similar , i.e. , δR , w , h ≈ δG , w , h ≈ δB , w , h , ∀ ( w , h ) ∈ W × H . This amounts to making the pixels darker/lighter , without changing the color balance of the image . Later , in section 3 , we suggest two ways of enforcing such similarity between RGB channels and we find both of them effective : • 1-channel attack strictly enforces δR , i ≈ δG , i ≈ δB , i , ∀i by using just one array to simultaneously represent each color channel δW×H . On the forward pass , we duplicate δ to make a 3-channel image . In this case , Dissim ( δ ) = 0 , and the perturbation is greyscale . • 3-channel attack uses a 3-channel perturbation δ3×W×H , along with the dissimilarity metric Dissim ( δ ) = ‖δR − δB‖p + ‖δR − δG‖p + ‖δB − δG‖p . All together , the three penalties minimize perceptibility of perturbations by forcing them to be ( a ) small , ( b ) smooth , and ( c ) without dramatic color changes ( e.g . swapping blue to red ) . At the same time , these penalties allow perturbations that are very large in ` p-norm . 2.1 CREATING UNTARGETED ATTACKS . We focus on spoofing certificates for untargeted attacks , in which the attacker does not specify the class into which the attack image moves . To achieve this , we generate an adversarial perturbation for all possible wrong classes ȳ and choose the best one as our strong attack : max ȳ 6=y , δ −L ( θ , x+ δ‖ȳ ) − λcC ( δ ) − λtvTV ( δ ) − λsDissim ( δ ) ( 4 ) where y is the true label/class for the clean image x , and L is a spoofing loss that promotes a strong certificate . We examine different choices for L for different certificates below . 3 ATTACKS ON RANDOMIZED SMOOTHING . The Randomized Smoothing method , first proposed by Lecuyer et al . ( 2018 ) and later improved by Li et al . ( 2018 ) , is an adversarial defense against ` 2-norm bounded attacks . Cohen et al . ( 2019 ) prove a tight robustness guarantee under the ` 2 norm for smoothing with Gaussian noise . Their study was the first certifiable defense for the ImageNet dataset ( Deng et al. , 2009 ) . The method constructs certificates by first creating many copies of an input image contaminated with random Gaussian noise of standard deviation σ . Then , it uses a base classifier ( a neural net ) to make a prediction for all of the images in the randomly augmented batch . Depending on the level of the consensus of the class labels at these random images , a certified radius is calculated that can be at most 4σ ( in the case of perfect consensus ) . Intuitively , if the image is far away from the decision boundary , the base classifier should predict the same label for each noisy copy of the test image , in which case the certificate is strong . On the other hand , if the image is adjacent to the decision boundary , the base classifier ’ s predictions for the Gaussian augmented copies may vary . If the variation is large , the smoothed classifier abstains from making a prediction . To spoof strong certificates ( large certified radius ) for an incorrect class , we must make sure that the majority of a batch of noisy images around the adversarial image are assigned the same ( wrong ) label . We do this by minimizing the cross entropy loss relative to a chosen ( incorrect ) label , averaged over a large set of randomly perturbed images . To this end , we minimize equation 4 , where L is chosen to be the average cross-entropy over a batch of Gaussian perturbed copies . This method is analogous to the technique presented by Shafahi et al . ( 2018 ) for generating universal perturbations that are effective when added to a large number of different images . RESULTS Cohen et al . ( 2019 ) show the performance of the Gaussian smoothed classifier on CIFAR-10 ( Krizhevsky et al . ) and ImageNet ( Deng et al. , 2009 ) . To attack the CIFAR-10 and ImageNet smoothed classifiers , we use 400 randomly sampled Gaussian images , λtv = 0.3 , λc = 1.0 , and perform 300 steps of SGD with learning rate 0.1 . We choose the functional regularizers C ( δ ) and TV ( δ ) to be C ( δ ) = ‖Avg ( |δR| ) , Avg ( |δG| ) , Avg ( |δB | ) ‖22 , and TV ( δi , j ) = anisotropic-TV ( δi , j ) 2 , where | · | is the element-wise absolute value operator , and Avg computes the average . For the Dissim regularizer , we experiment with both the 1-Channel attack that ensures Dissim ( δ ) = 0 , and the 3-Channel attack by setting Dissim ( δ ) = ‖ ( δR − δG ) 2 , ( δR − δB ) 2 , ( δG − δB ) 2‖2 and λs = 0.5 . For the validation examples on which the smoothed classifier does not abstain ( see Cohen et al . ( 2019 ) for more details ) , the less-constrained 3-channel attack is always able to find an adversarial example while the 1-channel attack also performs well , achieving 98.5 % success . 2 In section 4 we will discuss in more detail other differences between 1-channel and 3-channel attacks . The results are summarized in Table 1 . For the various base-models and choices of σ , our adversarial examples are able to produce certified radii that are on average larger than the certified radii produced for natural images . For ImageNet , since attacking all 999 remaining target classes is computationally expensive , we only attacked target class IDs 100 , 200 , 300 , 400 , 500 , 600 , 700 , 800 , 900 , and 1000 . 2Source code for all experiments can be found at : https : //github.com/AminJun/BreakingCertifiableDefenses Figure 2 depicts a sample adversarial example built for the smoothed ImageNet classifier that produces a strong certificate . The adversarial perturbation causes the batch of Gaussian augmented black swan images to get misclassified as hooks . For more , see appendix 16 .

Summary: The paper presents a new attack, called the shadow attack, that can maintain the imperceptibility of adversarial samples when out of the certified radius. This work not only aims to target the classifier label but also the certificate by adding large perturbations to the image. The attacks produce a 'spoofed' certificate, so though these certified systems are meant to be secure, can be attacked. Theirs seem to be the first work focusing on manipulating certificates to attack strongly certified networks. The paper presents shadow attack, that is a generalization of the PGD attack. It involves creation of adversarial examples, and addition of few constraints that forces these perturbations to be small, smooth and not many color variations. For certificate spoofing the authors explore different spoofing losses for l-2(attacks on randomized smoothing) and l-inf(attacks on crown-ibp) norm bounded attacks.