A Branch and Bound Framework for Stronger Adversarial Attacks of ReLU Networks

1 INTRODUCTION . Adversarial attacks aim to find adversarial examples ( Szegedy et al. , 2013 ) , which are close to benign inputs in certain distance metrics yet trigger wrong behavior of neural networks ( Carlini & Wagner , 2017 ; Madry et al. , 2018 ; Athalye et al. , 2018 ; Croce & Hein , 2020b ) . Adversarial attacks are important tools to gauge the empirical robustness of deep neural networks . Finding an adversarial example can be generally formulated as a constrained optimization problem : xadv = arg min x∈C f ( x ) ( 1 ) where C is often an ` ∞ or ` 2 norm ball around the original input x0 , and f ( x ) is an attack success criterion involving a neural network ( such as the margin between the groundtruth class and another class ) : f ( x ) < 0 indicates a successful attack . A straightforward way of solving Eq . ( 1 ) is via firstorder constrained optimization methods , such as projected gradient descent ( PGD ) ( Madry et al. , 2018 ) and its variants ( Croce & Hein , 2020b ; Tashiro et al. , 2020 ; Croce & Hein , 2020a ; Xie et al. , 2019 ; Dong et al. , 2018 ; Zheng et al. , 2019 ) . Additionally , some gradient-free attacks were proposed ( Brendel et al. , 2018 ; Cheng et al. , 2018 ; Alzantot et al. , 2019 ; Andriushchenko et al. , 2020 ) , mostly based on certain heuristic search on the input space x . Limitations of existing attacks . As f ( x ) usually consists of a highly non-convex neural network , solving Eq . ( 1 ) to its global minimum is challenging . This leads to failures in adversarial attacks : an adversarial example may exist but no attacks can find it , giving a false sense of security ( Athalye et al. , 2018 ) . Especially , gradient based attacks can be easily trapped into a local minimum or misguided by masked gradients ( Papernot et al. , 2016 ; Tramèr et al. , 2017 ) . Even if we give the attacker an infinite amount of time ( e.g. , run a very large number of PGD steps , or allowing a large number of samples on input space ) , it is still hard to guarantee to find the adversarial example , since it is extremely difficult to systematically search the high dimensional and continuous input space . Models concluded robust under existing attacks might still have security vulnerability in practice , leading to an urgent request for stronger attacks that can possibly approach ground-truth robustness . The mixed-integer approach . This paper seeks stronger adversarial attacks from a different angle : instead of searching for adversarial examples in the input space , we look for adversarial examples in the activation space . The main intuition is that neural networks with piece-wise linear activation functions ( e.g. , ReLU ) can be seen as a piece-wise linear function and each piece is uniquely defined by a specific setting of activation function status . For ReLU networks , each neuron can either be active ( its input is positive ) or inactive ( its input is negative so the output is 0 ) , which can be encoded by discrete 0-1 variables . This leads to a mixed integer programming ( MIP ) formulation ( Ehlers , 2017 ) , and Tjeng et al . ( 2019 ) conducted attacks using a MIP solver . Benefits of the MIP formulation . The MIP formulation with the 0-1 encoding of ReLU neurons allows us to systematically search all the linear pieces in the input space , theoretically guarantee to enumerate the entire input space and obtain the global minimum of Eq . ( 1 ) given sufficient time . Practically , MIP-based attacks can often find adversarial examples that are missed by existing attacks and identify true weaknesses of a model . It also helps to close the gap between the upper and lower bounds of robust accuracy ( i.e. , attack accuracy vs. verified accuracy ) . Existing works aimed to reducing this gap by tightening the lower bound with stronger verifiers ( Raghunathan et al. , 2018b ; Wang et al. , 2021 ) , while our work aims to tighten the upper bound by a systematic search of adversarial examples . Closing this gap is difficult even on small models ( Dathathri et al. , 2020 ) . Generic MIP solvers are inefficient for adversarial attacks . Despite its strengths , a MIP-based attack are often a few orders of magnitudes slower than existing attacks due to the high cost of running an off-the-shelf solver such as Gurobi ( Tjeng et al. , 2019 ) . There are three root causes for its inefficiency . First , an off-the-shelf solver is not aware of the underlying optimization problem corresponds to a neural network , and has to apply generic solving techniques ( e.g. , using Simplex algorithm with relaxations ) which can be expensive or ineffective . Second , it can not utilize solutions obtained cheaply from gradient based attacks to accelerate its search . Third , generic MIP solvers are mostly restricted to CPUs and can hardly utilize GPU acceleration , which is crucial for efficiency . Contributions of this paper . We address the above weaknesses in MIP solvers for adversarial attacks , by developing a GPU-accelerated branch and bound procedure to systematically search adversarial examples in the activation space , which is more efficient than generic MIP solvers . We focus on solving hard instances , where none of existing adversarial attacks based on searching the input space are successful but no verifier can prove their robustness . Our contributions includes : • We apply the GPU-accelerated bound propagation based methods ( Wang et al. , 2021 ; Xu et al. , 2020 ; Zhang et al. , 2018 ; Wong & Kolter , 2018 ) , which were originally developed for neural network verification , to the adversarial attack setting . These specialized methods can quickly examine thousands of regions in activation space in parallel and rule out the regions with no adversarial examples , which is difficult in off-the-shelf MIP solvers with a generic solving procedure . • We employ a top-down beam-search to explore the activation space . Unlike the best-first search scheme used in many neural network verifiers , we can quickly go deep in the search tree and identify the most promising regions with adversarial examples . A smaller sub-MIP can then be created onthe-fly to search adversarial examples in the reduced regions with much fewer integer variables . • We collect adversarial examples generated by cheap attacks in a candidate pool and utilize them in two ways . First , we conduct a bottom-up search on examples close to decision boundary by applying large neighborhood search ( LNS ) . Second , when conducting the top-down search , we adopt diving by fixing integer variables according to adversarial examples in the pool to reduce the search space . • Our new attack framework , BaB-Attack , is designed to tackle hard instances where existing strong adversarial attacks ( such as AutoAttack ) can not succeed . Despite being more expensive than attacks on the input space , BaB-Attack is about an order of magnitude faster than using an MIP solver on hard instances , and can find adversarial examples that can not be discovered by any existing attacks . 2 BACKGROUND . Notations . We define a L layer feed-forward ReLU network as f : Rn0 → R and f ( x ) : = z ( L ) ( x ) , where z ( i ) ( x ) = W ( i ) ẑ ( i−1 ) ( x ) + b ( i ) with i-th layer weight matrix W ( i ) and bias b ( i ) , ẑ ( i ) ( x ) = ReLU ( z ( i ) ( x ) ) , and input ẑ ( 0 ) ( x ) = x. Layer i has dimension ni , and N is the total number of neurons . We denote the j-th neuron in layer i as z ( i ) j . For a simpler presentation , we assume f ( x ) is a binary classifier and benign input x0 has f ( x0 ) > 0 . An attacker seeks to minimize f ( x ) within a ` ∞ norm perturbation set C to make f ( x ) < 0 . We can attack a multi-class classifier by considering each pair of target and ground-truth label individually where f is defined as the margin between them , similarly to ( Gowal et al. , 2019b ) . We use [ N ] to represent the set { 1 , · · · , N } . The MIP formulation for adversarial attack . Tjeng et al . ( 2019 ) formulated the adversarial attack and verification of ReLU network into a mixed integer programming ( MIP ) problem , solved by existing MIP solvers ( refer to as the “ MIP attack ” ) . It has binary variables s ( i ) j for each ReLU : min f ( x ) s.t . z ( i ) ( x ) = W ( i ) ẑ ( i−1 ) ( x ) + b ( i ) ; f ( x ) = z ( L ) ( x ) ; x ∈ C ; ẑ ( i ) j ( x ) ≥ z ( i ) j ( x ) ; ẑ ( i ) j ( x ) ≤ u ( i ) j s ( i ) j ; ẑ ( i ) j ( x ) ≤ z ( i ) j ( x ) − l ( i ) j ( 1− s ( i ) j ) ; ẑ ( i ) j ( x ) ≥ 0 ; z ( i ) j ( x ) ∈ [ l ( i ) j , u ( i ) j ] ; s ( i ) j ∈ { 0 , 1 } ; i ∈ [ L ] , j ∈ [ ni ] ( 2 ) where s ( i ) j indicating the two status of a ReLU neuron : ( 1 ) inactive : when s ( i ) j = 0 , constraints on ẑ ( i ) j simplifies to ẑ ( i ) j = 0 ; or ( 2 ) active : when s ( i ) j = 1 we have ẑ ( i ) j = z ( i ) j . Here l ( i ) j , u ( i ) j are precomputed intermediate lower and upper bounds on pre-activation z ( i ) j such that l ( i ) j ≤ z ( i ) j ( x ) ≤ u ( i ) j for any x ∈ C. The complexity of this problem can increase exponentially with the number of ReLU neurons , so it can take hours to run even on a small network , unless the network is trained with a strong regularization such as a certified dense ( Wong & Kolter , 2018 ; Xiao et al. , 2019 ) . Searching in the Activation Space via Branch and Bound .. Given the formulation in Eq . ( 2 ) , we can view a neural network in the activation spaceA = [ 0 , 1 ] N whereN is the total number of neurons , and each dimension corresponds to the setting of a s ( i ) j ∈ { 0 , 1 } variable . To determine s ( i ) j corresponding to a known adversarial example xadv , we can propagate xadv through the network and check the sign of each neuron z ( i ) j , so s ( i ) j = 1 ( z ( i ) j ≥ 0 ) . This uniquely locates the linear piece of f ( x ) where xadv lies , because Eq . ( 2 ) becomes a set of linear inequalities when all s ( i ) j are fixed . Intuitively , we can search adversarial examples by fixing all s ( i ) j to one of the 2N possible combinations inA , and then solve Eq . ( 2 ) exactly using linear programming ; an adversarial example is found when the solution is negative . To avoid clutter , we flatten the ordering of s ( i ) j for i ∈ [ L ] , j ∈ [ ni ] and use a single subscript s1 , · · · , sN to denote all binary variables . To effectively and systematically search in the activation space , instead of fixing all si ( i ∈ [ N ] ) , we can first fix a subset of them and bound the objective f ( x ) to guide the search , leading to the branch and bound ( BaB ) method . In BaB , we solve Eq . ( 2 ) by creating subproblems constraining some binary variables , for example , s1 = 0 or s1 = 1 ( since we can branch on the neurons in any fixed order , without loss of generality , we branch the neurons chronologically ) . We define a set S containing all the branching constraints ( e.g. , S = { s1 = 0 , s2 = 1 } ) , which corresponds to a subdomain of the original problem Eq . ( 2 ) . BaB requires the LB primitive on S , which relaxes the remaining binary variables to obtain a lower bound of Eq . ( 2 ) : Lower bound in subdomain : LB ( S ) ≤ min f ( x ) s.t . s ∈ S and all other constraints in Eq . ( 2 ) Here s ∈ S means setting binary variables s ( i ) j according to the constraints in S . Typically , more constraints in S lead to tighter bounds . LB ( S ) > 0 indicates that no adversarial example exist within this subdomain , otherwise adversarial examples may exist in this subdomain . We illustrate a BaB search tree in Fig . 1 . Initially , Sroot = ∅ , where x ∈ C without any extra constraints in activation space and a lower bound of Eq . ( 2 ) is obtained . When LB ( ∅ ) < 0 , an adversarial example may exist , and we branch Sroot into two subdomains : S1− = Sroot ∪ { s1 = 0 } ; S1+ = Sroot ∪ { s1 = 1 } Then we bound each subdomain . Since more constraints are added , LB ( S1− ) and LB ( S1+ ) are usually improved . The branching procedure continues recursively , and if any LB ( S ) > 0 , no further branching is needed since no adversarial examples are in that subdomain . Each branching increases the cardinality of S by 1 , and eventually we reach leaf nodes with |Sleaf| = N , each leaf locating a linear piece of f ( x ) . In that case , LB ( Sleaf ) is an exact solution since no binary variables are left , and if LB ( Sleaf ) < 0 , a concrete adversarial example is the minimizer x∗ of Eq . ( 2 ) with s ∈ Sleaf . Since N can be quite large and adversarial examples lie in the leaf level , we must guide the search to reach there quickly . Although BaB is used in existing neural network verifiers ( Bunel et al. , 2018 ; De Palma et al. , 2021a ) , they do not aim to reach the leaf level and typically branch the node with the worst bound first , generally leading to a wide but shallow search tree and unsuitable for detecting adversarial examples . We will discuss our search algorithm in Sec . 3 . Bounding in Branch and Bound of Neural Networks . The LB ( S ) primitive is crucial in the BaB process : it needs to provide a tight lower bound efficiently . A simple way to lower bound the objective of Eq . ( 2 ) is via relaxation of integer variables and linear programming ( LP ) ( Bunel et al. , 2018 ; Lu & Kumar , 2020 ) ; but an LP solver is needed which restricts its efficiency . Recently , a popular choice in neural network verifiers is the specialized bound propagation methods ( Zhang et al. , 2018 ; Wong & Kolter , 2018 ) which exploit the structure of the optimization problem ( which a generic LP/MIP solver can not ) and give LB ( S ) efficiently on GPUs without an LP solver . Essentially , they relax each ReLU neuron into convex domains ( Salman et al. , 2019 ) and propagate them layer by layer through the network while maintaining sound bounds . A BaB and bound propagation based verifier α , β-CROWN ( Zhang et al. , 2018 ; Xu et al. , 2021 ; Wang et al. , 2021 ) , achieves the state-of-the-art verification performance ( Bak et al. , 2021 ) , and we utilize its bounding subprocedure to produce LB ( S ) . Importantly , we will show that how we use LB ( S ) to guide adversarial attacks , while existing works mostly use them for verification . 3 METHOD Beam SearchVerifier , -CROWN Adversarial Candidates Pool Sub-MIP SolverLocal Search Diving Input : s.t . PGD attack with random starts Output - adversarial example detected : Common adversarial patterns Promising subdomains Promising adversarial candidates Disagreed adversarial patterns upper bounds subdomain lower bounds K subdomains Top-Down thread on GPUs Bottom-Up thread on CPUs Solutions Sub-MIPs Sub-MIPs Figure 2 : Overview of BaB attack . Overview of BaB attack . To systematically search adversarial examples in activation space , we must explore the search tree and enumerate as more leaf nodes as possible . Although the worst case search time complexity is exponential in the numbers of ReLU neurons ( visiting every leaf node of the tree ) , practically , if a right search procedure is chosen , only a small fraction of nodes need to be visited to find an adversarial example . In this paper , we propose BaB attack , specializing the BaB searching strategy over the activation space for the purpose of adversarial attacks . The search is well guided by ( 1 ) a top-down beam-search thread accelerated on GPUs which quickly goes deep into the search tree , and ( 2 ) a bottom-up search thread on CPUs for large neighborhood search . The top-down and bottom-up searches run in parallel threads and they both benefit from the adversarial candidates pool P , which contains examples P = { xc1 , · · · , xcM } where M is the pool capacity . f ( xci ) is still positive but small ; the pool keeps the M best ( ranked by f ( xci ) ; smaller is better ) examples it receives . The activation space representations of these candidates are used as extra information to guide the search towards adversarial examples . We detail each part of BaB attack in next sub-sections .

Summary: This paper proposes a branch-and-bound attack (BaB-Attack) to solve hard instances efficiently, where none of the existing adversarial attacks can succeed. Specifically, the BaB-Attack utilizes the bound propagation-based neural network verifiers on GPUs to rapidly evaluate a large number of searching regions, builds an adversarial candidates pool to guide the search, and refines candidates adversarially examples using a bottom-up large neighborhood search. Experimental results show that BaB-Attack outperforms the existing attacks in both attack success rates and efficiency.