Adversarial Training and Provable Defenses: Bridging the Gap

1 INTRODUCTION . The discovery of adversarial examples in deep learning ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) has increased the importance of creating new training methods which produce accurate and robust neural networks with provable guarantees . Existing work : adversarial and provable defenses Adversarial training ( Goodfellow et al. , 2015 ; Kurakin et al. , 2017 ) provides a framework to augment the training procedure with adversarial inputs produced by an adversarial attack . Madry et al . ( 2018 ) instantiated adversarial training using a strong iterative adversary and showed that their approach can train models which are highly robust against the strongest known adversarial attacks such as Carlini & Wagner ( 2017 ) . This method has also been able to train robust ImageNet models ( Xie et al. , 2019 ) . While promising , the main drawback of the method is that when instantiated in practice , via an approximation of an otherwise intractable optimization problem , it provides no guarantees – it does not produce a certificate that there are no possible adversarial attacks which could potentially break the model . To address this lack of guarantees , recent line of work on provable defenses ( Wong & Kolter , 2018 ; Raghunathan et al. , 2018 ; Mirman et al. , 2018 ) has proposed to train neural networks that are certifiably robust to a specific attacker threat model . However , these guarantees come at the cost of a significantly lower standard accuracy than models trained using adversarial training . This setting raises a natural question : can we leverage ideas from both , adversarial training techniques and provable defense methods , so to obtain models with high accuracy and certified robustness ? This work : combining adversarial and provable defenses In this work , we take a step towards addressing this challenge . We show that it is possible to train more accurate and provably robust neural networks using the same convex relaxations as those used in existing , state-of-the-art provable defense methods , but with a new , different optimization procedure inspired by adversarial training . Our optimization works as follows : ( i ) to certify a property ( e.g. , robustness ) of the network , the verifier produces a convex relaxation of all possible intermediate vector outputs in the neural network , then ( ii ) an adversary now searches over this ( intermediate ) convex region in order to find , what we refer to as a latent adversarial example – a concrete intermediate input contained in the relaxation that when propagated through the network causes a misclassification which prevents verification , and finally ( iii ) the resulting latent adversarial examples are now incorporated into our training scheme using adversarial training . Overall , we can see this method as bridging the gap between adversarial training and provable defenses ( it can conceptually be instantiated with any convex relaxation ) . We experimentally show that the method is promising and results in a neural network with state-of-theart 78.4 % accuracy and 60.5 % certified robustness on the challenging CIFAR-10 dataset with 2/255 L∞ perturbation ( the best known existing results are 71.5 % accuracy and 54.0 % certified robustness from concurrent work of Zhang et al . ( 2020 ) ) . Main Contributions Our key contributions are : • A new method , which we refer to as convex layerwise adversarial training ( COLT ) , that can train provably robust neural networks and conceptually bridges the gap between adversarial training and existing provable defense methods . • Instantiation of convex layerwise adversarial training using linear convex relaxations used in prior work ( accomplished by introducing a projection operator ) . • Experimental results showing convex layerwise adversarial training can train neural network models which achieve both , state-of-the-art accuracy and certified robustness on CIFAR-10 with 2/255 L∞ perturbation . • Complete implementation of our training and certification methods in a system which we release at https : //github.com/eth-sri/colt . Overall , we believe the method presented in this work is a promising step towards training models that enjoy both , higher accuracy and higher certification guarantees . An interesting item for future work would be to explore instantiations of the method with other convex relaxations than the one considered here . 2 RELATED WORK . We now discuss some of the closely related work on robustness of neural networks . Heuristic adversarial defenses After the first introduction of adversarial examples ( Szegedy et al. , 2013 ; Biggio et al. , 2013 ) , defense mechanisms to train robust neural networks were built based on the inclusion of adversarial examples to the training set ( Kurakin et al. , 2017 ; Goodfellow et al. , 2015 ) . Models trained using adversarial training with projected gradient descent ( PGD ) ( Madry et al. , 2018 ) were shown to be robust against the strongest known attacks ( Carlini & Wagner , 2017 ) . This is in contrast to other defense mechanisms which have been broken by new attack techniques ( Athalye et al. , 2018 ) . While models trained using adversarial training achieve robustness against strong adversaries , there are no guarantees that model is robust against any kind of adversarial attack under the threat model considered . Provable adversarial defenses There has also been considerable amount of work on methods to train classifiers with robustness guarantees . These approaches are typically based on Lipschitz regularization Hein & Andriushchenko ( 2017 ) , linear ( Wong & Kolter , 2018 ) or semidefinite ( Raghunathan et al. , 2018 ; Dvijotham et al. , 2018a ) relaxations , hybrid zonotope ( Mirman et al. , 2018 ) or interval bound propagation ( Gowal et al. , 2018 ) . While these approaches obtain robustness guarantees , accuracy of these networks is relatively small and limits practical use of these methods . There has also been recent work on certification of general neural networks , not necessarily trained in a special way . These methods are based on SMT solvers ( Katz et al. , 2017 ) , abstract interpretation ( Gehr et al. , 2018 ) , mixed-integer linear programs ( Tjeng et al. , 2019 ) , linear relaxations ( Weng et al. , 2018 ; Zhang et al. , 2018 ; Singh et al. , 2019c ) or combinations of those ( Singh et al. , 2019b ; a ) . Another line of work proposes to replace neural networks with a randomized classifier ( Lecuyer et al. , 2018 ; Cohen et al. , 2019 ; Salman et al. , 2019a ) which comes with probabilistic guarantees on its robustness . While these approaches scale to larger datasets such as ImageNet ( although with probabilistic instead of exact guarantees ) , their bounds come from the relationship between L2 robustness and Gaussian distribution . In this paper , we consider general verification problem where input is not necessarily limited to an Lp ball , but arbitrary convex set , as explained in Section 3 . 3 BACKGROUND . In this work we consider a threat model where an adversary is allowed to transform an input x ∈ Rd0 into any point from a convex set S0 ( x ) ⊆ Rd0 . The set S0 can capture wide range of specifications such asLp perturbations ( Wong & Kolter , 2018 ) , geometric transformations ( Balunović et al. , 2019 ) , semantic perturbations ( Mohapatra et al. , 2019 ) or camera imaging ( Yang & Rinard , 2019 ) . In the case of a threat model based on L∞ perturbations , which we experiment with later , the convex set will be defined as S0 ( x ) = { x′ ∈ Rd0 , ||x− x′||∞ < } . A neural network consisting of k layers and parameters θ is represented as a function hθ where hθ = h k θ ◦ h k−1 θ · · · ◦ h1θ and hiθ : Rdi−1 → Rdi denotes a transformation applied at hidden layer i . We also denote the function representing part of the neural network from layer i to the final layer k as hi : kθ = h k θ ◦ h k−1 θ · · · ◦ hiθ . Our goal will be to prove a property on the output of the neural network , encoded via a linear constraint : cThθ ( x ′ ) + d < 0 , ∀x′ ∈ S0 ( x ) ( 1 ) where c and d are property specific vector and scalar values , respectively . This formulation is general enough to capture many interesting safety properties ( Dvijotham et al. , 2018b ; Qin et al. , 2019 ) , including robustness toLp perturbations . The standard approach to train a model hθ to satisfy this constraint is to define a surrogate loss L and solve the following min-max optimization problem : min θ E ( x , y ) ∼D max x′∈S0 ( x ) L ( hθ ( x′ ) , y ) ( 2 ) Because inner maximization is intractable , most existing approaches replace it with an approximation . Depending on the used approximation , we distinguish two families of techniques . Adversarial training One family of methods , referred to as adversarial training ( Goodfellow et al. , 2015 ; Kurakin et al. , 2017 ) , replaces the maximum loss with a lower bound which is obtained using an adversarial attack . Madry et al . ( 2018 ) maximized the inner loss using a projected gradient descent ( PGD ) attack and found that , perhaps surprisingly , this optimization procedure succeeds in training deep architectures which are robust against the strongest known adversaries . While this provides strong empirical evidence that the resulting models are indeed robust , the approach offers no guarantees . Thus , it remains unclear whether there exist even stronger attacks that could break a model trained in this manner . Provable defenses A second family of methods to train certified neural networks is based on the computation of an upper bound to the inner loss , as opposed to a lower bound computed for adversarial training . These methods are typically referred to as provable defenses as they provide guarantees on the robustness of the resulting network , under any kind of attack inside the threat model . An upper bound is typically computed using linear relaxations ( Wong & Kolter , 2018 ) , interval propagation ( Gowal et al. , 2018 ) or methods combining interval bounds and linear relaxations ( Mirman et al. , 2018 ; Zhang et al. , 2020 ) . However , these methods suffer from two disadvantages . First , due to the convex relaxations , an upper bound on the loss is typically not tight and can be quite loose . However , we believe this is less of an issue due to the fact that interval relaxations were shown to experimentally be able to train more provably robust models than methods based on linear relaxations ( which usually produce tighter bounds than intervals ) . For example , Mirman et al . ( 2018 ) ; Zhang et al . ( 2020 ) report ∼ 28 % robust accuracy using pure interval training on CIFAR-10 with perturbation 8/255 while Wong et al . ( 2018 ) achieve 21 % using linear relaxations . Second , the way these methods construct the loss makes the relationship between the loss and the network parameters significantly more complex than in standard training . Given a network with L layers and at most N neurons in each layer , to compute the loss we have to perform O ( LN2 ) operations for interval training and O ( LN3 ) operations for linear relaxation based training . Intuitively , while there are exceptions , we can expect that a loss involving more mathematical operations is more difficult to minimize . We hypothesize that this complexity of loss computation ( here quantified as number of operations ) causes the resulting optimization problem for training the network to be more difficult , meaning these training methods often converge to a suboptimal solution . Our experimental results confirm this – we substantially outperform existing methods both in terms of accuracy and certified robustness using the same linear relaxation , but a different optimization procedure . Certification via convex relaxations We now formally describe how provable defenses perform certification . We denote the set of possible intermediate concrete vectors at layer i that can be obtained by propagating vector x′ ∈ S0 ( x ) through the network as Si ( x ) = hiθ ( Si−1 ( x ) ) ⊆ Rdi . As it is difficult to explicitly compute the set Si ( x ) , a standard approach is to approximate it via a convex relaxation Ci ( x ) . As the input set is already convex , there is no need to introduce a relaxation , and thus we set C0 ( x ) = S0 ( x ) . Given a neural network layer hiθ which transforms one set of vectors into another , we represent its corresponding convex relaxation transformer as giθ . That is , giθ will transform one convex set into another convex set . More formally , for any set D ⊆ Rdi−1 , giθ ( D ) is convex and hiθ ( D ) ⊆ giθ ( D ) . Then , we recursively define the effect of giθ on a convex relaxation as Ci ( x ) = giθ ( Ci−1 ( x ) ) ⊆ Rdi . Finally , to certify robustness using the obtained convex relaxation , it is enough to check whether all output vectors in Ck ( x ) satisfy the linear constraint in Equation 1 . If this is true , then all output vectors in Sk ( x ) satisfy the constraint as well due to the fact that Sk ( x ) ⊆ Ck ( x ) .

Summary: This paper was very clearly written and easy to follow. Kudos to the authors. In particular, the experimental evaluation section was exceptionally clear. Thanks to the authors for making the paper so easy to review. The “Main Contributions” section was excellent as well as it allows the reader to quickly understand what the paper is claiming.