Provably Robust Adversarial Examples

1 INTRODUCTION . Deep neural networks ( DNNs ) are vulnerable to adversarial attacks : small input perturbations that cause misclassification ( Szegedy et al. , 2013 ) . This has caused an increased interest in investigating powerful attacks ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Andriushchenko et al. , 2019 ; Zheng et al. , 2019 ; Wang et al. , 2019 ; Croce & Hein , 2019 ; Tramèr et al. , 2020 ) . An important limitation of existing attack methods is that they only produce a single concrete adversarial example and their effect can often be mitigated with existing defenses ( Madry et al. , 2018 ; Hu et al. , 2019 ; Sen et al. , 2020 ; Xiao et al. , 2020 ; Pang et al. , 2020 ; Lécuyer et al. , 2019 ; Cohen et al. , 2019 ; Salman et al. , 2019 ; Fischer et al. , 2020 ; Li et al. , 2020 ) . The effectiveness of these attacks can be improved by robustifying them : to consolidate the individual examples to produce a large symbolic region guaranteed to only contain adversarial examples . This Work : Provably Robust Adversarial Examples We present the concept of a provably robust adversarial example – a tuple consisting of an adversarial input point and an input region around it capturing a very large set ( e.g. , > 10100 ) of points guaranteed to be adversarial . We then introduce a novel algorithm for generating such regions and apply it to the setting of pixel intensity changes and geometric transformations . Our work relates to prior approaches on generating robust adversarial examples ( Athalye et al. , 2018 ; Qin et al. , 2019 ) but differs in a crucial point : our regions are guaranteed to be adversarial while prior approaches are empirical and offer no such guarantees . Main Contributions Our key contributions are : • The concept of a provably robust adversarial example – a connected input region capturing a very large set of points , generated by a set of perturbations , guaranteed to be adversarial . • A novel scalable method for synthesizing provably robust adversarial examples , based on iterative refinement and certification techniques . • A thorough evaluation of our method demonstrating it can generate provable regions containing ≈ 10573 concrete adversarial points for pixel intensity changes , in ≈ 2 minutes , and ≈ 10599 concrete points for geometric transformations , in ≈ 20 minutes , on a challenging CIFAR10 network . We also demonstrate that our robust adversarial examples are significantly more effective against state-of-the-art defenses based on randomized smoothing than the individual attacks used to construct the regions . 2 BACKGROUND . We now discuss the background necessary for the remainder of the paper . We consider a neural network f : Rn0 → Rnl with l layers , n0 input neurons and nl output classes . The network classifies an input x to class y ( x ) with the largest corresponding output value , i.e. , y ( x ) = argmaxi [ f ( x ) ] i . Note for brevity we will omit the argument to y when it is clear from the context . 2.1 NEURAL NETWORK CERTIFICATION . In this work , we rely on state-of-the-art neural network certification methods based on convex relaxations to prove that the adversarial examples produced by our algorithm are robust . These certification methods take a convex input region I ⊂ Rn0 and prove that every point in I is classified as target label yt by f . They propagate the set I through the layers of the network , producing a convex region that covers all possible values of output neurons ( Gehr et al. , 2018 ) . Robustness follows by proving that , for all combinations of output neuron values in this region , the output neuron corresponding to class yt has a larger value than the one corresponding to any other class y 6= yt . Commonly , one proves this property by computing a function Ly : Rn0 → R for each label y 6= yt , such that , for all x ∈ I , we have Ly ( x ) ≤ [ f ( x ) ] yt − [ f ( x ) ] y . For each Ly , one computes minx∈I Ly ( x ) to obtain a global lower bound that is true for all x ∈ I . If we obtain positive bounds for all y 6= yt , robustness is proven . To simplify notation , we will say that the certification objective L ( x ) is the function Ly ( x ) with the smallest minimum value on I . We will call its corresponding minimum value the certification error . We requireLy ( x ) to be a linear function of x . This requirement is consistent with many popular certification algorithms based on convex relaxation , such as CROWN ( Zhang et al. , 2018 ) , DeepZ ( Singh et al. , 2018a ) , and DeepPoly ( Singh et al. , 2019 ) . Without loss of generality , for the rest of this paper , we will treat DeepPoly as our preferred certification method . 2.2 GEOMETRIC CERTIFICATION . DeepPoly operates over specifications based on linear constraints over input pixels for verification . These constraints are straightforward to provide for simple pixel intensity transformations such as adversarial patches ( Chiang et al. , 2020 ) and L∞ ( Carlini & Wagner , 2017 ) perturbations that provide a closed-form formula for the input region . However , the geometric transformations do not yield such linear regions . To prove the robustness of our generated examples to geometric transformations , we rely on DeepG ( Balunović et al. , 2019 ) which , given a range of geometric transformation parameters , creates an overapproximation of the set of input images generated by geometric perturbations . DeepG then levarages DeepPoly to certify the input image region . When generating our geometric robust examples , we work directly in the geometric parameter space and , thus , our input region I and the inputs to our certification objective L ( x ) are also in geometric space . Despite this change , as our approach is agnostic to the choice of the verifier , in the remainder of the paper we will assume the certification is done using DeepPoly and not DeepG , unless otherwise stated . 2.3 RANDOMIZED SMOOTHING . Randomized smoothing ( Lécuyer et al. , 2019 ; Cohen et al. , 2019 ) is a provable defense mechanism against adversarial attacks . For a chosen standard deviation σ and neural network f as defined above , randomized smoothing computes a smoothed classifier g based on f , such that g ( x ) = argmax c P ( y ( x+ ) = c ) with random Gaussian noise ∼ N ( 0 , σ2I ) . This construction of g allows Cohen et al . ( 2019 ) to introduce the procedure CERTIFY that provides probabilistic guarantees on the robustness of g around a point x : Proposition 1 . ( From Cohen et al . ( 2019 ) ) With probability at least 1− α over the randomness in CERTIFY , if CERTIFY returns a class y and a radius R ( i.e does not abstain ) , then g predicts y within radius R around x : g ( x+ δ ) = y , for all ‖δ‖2 < R. We define adversarial attacks on smoothed classifiers as follows : Definition 1 ( Adversarial attack on smoothed classifiers ) . For a fixed σ , α and an adversarial distance R′ ∈ R > 0 , we call x̃ ∈ Rn0 an adversarial attack on the smoothed classifier g at the point x ∈ Rn0 , if ‖x̃− x‖2 < R′ and g ( x ) 6= g ( x̃ ) . Similarly to generating adversarial attacks on the network f , we need to balance the adversarial distance R′ on g. If too big — the problem becomes trivial ; if too small — no attacks exist . We outline the exact procedure we use to heuristically select R′ in Appendix A.6 . Using the above definition , we define the strength of an attack x̃ as follows : Definition 2 ( Strength of adversarial attack on smoothed classifiers ) . We measure the strength of an attack x̃ in terms of Radv– the radius around x̃ , whose L2 ball is certified to be the same adversarial class as x̃ on the smoothed network g using CERTIFY for a chosen σ and α . Intuitively , this definition states that for points x̃ for which Radv is bigger , the smoothed classifier is less confident about predicting the correct class , since more adversarial examples are sampled in this region and therefore , the attack on g is stronger . We use this measure in Section 5 to compare the effectiveness of our adversarial examples to examples obtained by PGD on g . 3 OVERVIEW . Existing methods for generating robust adversarial examples focus on achieving empirical robustness ( Qin et al. , 2019 ; Athalye et al. , 2018 ) . In contrast , we consider provably robust adversarial examples , defined below : Definition 3 ( Provably Robust Adversarial Example ) . We define provably robust adversarial example to be any large connected neural network input region , defined by a set of perturbations of an image , that can be formally proven to only contain adversarial examples . In this section , we outline how we generate such regions . The technical details are given in Section 4 . To generate a provably robust adversarial example , ideally , we would like to directly maximize the input region ’ s size , under the constraint that it only contains adversarial examples . This leads to multiple challenges : Small changes of the parametrization of the input region ( e.g. , as a hyperbox ) can lead to large changes of the certification objective , necessitating a small learning rate for optimization algorithms based on gradient descent . At the same time , the optimizer would have to solve a full forward verification problem in each optimization step , which is slow and impractical . Additionally , according to Balunovic & Vechev ( 2020 ) , bad initialization of the robust region might further cause convergence to local minima , resulting in small robust regions . We now provide an overview of how we generate robust adversarial regions while alleviating these problems . Our method for generating robust examples in the shape of a hyperbox is shown in Figure 1 and assumes an algorithm A that generates adversarial examples and a neural network verifier V. We require V to provide the certification objective L ( x ) as a linear function of the network ’ s input neurons ( whose values can be drawn from the input hyperbox ) , as described in Section 2 . We split our algorithm into two steps , described in Section 3.1 and Section 3.2 , to address the challenges outlined above . An optional final step , illustrated in Figure 2 and outlined in Appendix D , demonstrates the extension of our algorithm to generate polyhedral examples . 3.1 COMPUTING AN OVERAPPROXIMATING REGION . In the first step , we compute an overapproximation hyperbox O by fitting a hyperbox around samples obtained fromA . O is represented as a dashed blue rectangle in Figure 1 . Intuitively , we use O to restrict the search space for adversarial regions to a part of the input space thatA can attack .

Summary: This paper presents a novel algorithm for identifying "provably robust adversarial examples": large regions in the input space that provably contain only adversarial examples.  Each region corresponds to a single adversarial example $\tilde{x}$ found in the center of the region, along with all the points that can be generated by applying a sequence of transformations to $\tilde{x}$. The transformations considered in the paper are semantically meaningful changes to the original image. Critically, we can be guaranteed that $\tilde{x}$ will be misclassified even if _it_ is perturbed.  The paper demonstrates that the algorithm can generate regions of non-trivial size on networks of non-trivial size. For example, for a CIFAR10 classifier with 6 layers and ~62k neurons, it finds axis-aligned regions containing a median of $10^{573}$ adversarial examples. In addition, the paper shows that provably robust adversarial examples can be used to create adversarial examples to $L_2$-smoothed classifiers that are more robust to $L_2$ noise as compared to adversarial examples generated directly via PGD attacks.