Combinatorial Attacks on Binarized Neural Networks

Binarized Neural Networks ( BNNs ) have recently attracted significant interest due to their computational efficiency . Concurrently , it has been shown that neural networks may be overly sensitive to “ attacks ” – tiny adversarial changes in the input – which may be detrimental to their use in safety-critical domains . Designing attack algorithms that effectively fool trained models is a key step towards learning robust neural networks . The discrete , non-differentiable nature of BNNs , which distinguishes them from their full-precision counterparts , poses a challenge to gradient-based attacks . In this work , we study the problem of attacking a BNN through the lens of combinatorial and integer optimization . We propose a Mixed Integer Linear Programming ( MILP ) formulation of the problem . While exact and flexible , the MILP quickly becomes intractable as the network and perturbation space grow . To address this issue , we propose IProp , a decomposition-based algorithm that solves a sequence of much smaller MILP problems . Experimentally , we evaluate both proposed methods against the standard gradient-based attack ( PGD ) on MNIST and Fashion-MNIST , and show that IProp performs favorably compared to PGD , while scaling beyond the limits of the MILP . 1 INTRODUCTION . The success of neural networks in vision , text and speech tasks has led to their widespread deployment in commercial systems and devices . However , these models can often be fooled by minimal perturbations to their inputs , posing serious security and safety threats ( Goodfellow et al. , 2014 ) . A great deal of current research addresses the “ robustification ” of neural networks using adversarially generated examples ( Kurakin et al. , 2016 ; Madry et al. , 2017 ) , a variant of standard gradient-based training that uses adversarial training examples to defend against possible attacks . Recent work has also formulated the problem of “ adversarial learning ” as a robust optimization problem ( Madry et al. , 2017 ; Kolter & Wong , 2017 ; Sinha et al. , 2017 ) , where one seeks the best model parameters with respect to the loss function as measured on the worst-case adversarial perturbation of each point in the training dataset . Attack algorithms may thus be used to augment the training dataset with adversarial examples during training , resulting in more robust models ( Kurakin et al. , 2016 ) . These new advances further motivate the need to develop effective methods for generating adversarial examples for neural networks . In this work , we focus on designing effective attacks against Binarized Neural Networks ( BNNs ) ( Courbariaux et al. , 2016 ) . BNNs are neural networks with weights in { −1 , +1 } and the sign function non-linearity , and are especially pertinent in low-power or hardware-constrained settings , where they have the potential to be used at an unprecedented scale if deployed to smartphones and other edge devices . This makes attacking , and consequently robustifying BNNs , a task of major importance . However , the discrete , non-differentiable structure of a BNN renders less effective the typical attack algorithms that rely on gradient information . As strong attacks are crucial to effective adversarial training , we are motivated to address this problem in the hope of generating better attacks . The goal of adversarial attacks is to modify an input slightly , so that the neural network predicts a different class than what it would have predicted for the original input . More formally , the task of generating an optimal adversarial example is the following : Given : – A ( clean ) data point x ∈ Rn ; – A trained BNN model with parameters w , that outputs a value fc ( x ; w ) for a class c ∈ C ; – prediction , the class predicted for data point x , argmaxc∈C fc ( x ; w ) ; – target , the class we would like to predict for a slightly perturbed version of x ; – , the maximum amount of perturbation allowed in any of the n dimensions of the input x . Find : A point x′ ∈ Rn , such that ‖x− x′‖∞ ≤ and the following objective function is maximized : ftarget ( x ′ ; w ) − fprediction ( x′ ; w ) . This objective function guides targeted attacks ( Kurakin et al. , 2016 ) , and is commonly used in the adversarial learning literature . If an adversary wants to fool a trained model into predicting that an input belongs to a given class , they will simply set the value of target accordingly to that given class . We note that our formulation and algorithm also work for untargeted attacks via a simple modification of the objective function . Towards designing optimal attacks against BNNs , we propose to model the task of generating an adversarial perturbation as a Mixed Integer Linear Program ( MILP ) . Integer programming is a flexible , powerful tool for modeling optimization problems , and state-of-the-art MILP solvers have achieved excellent results in recent years due to algorithmic and hardware improvements ( Achterberg & Wunderling , 2013 ) . Using a MILP model is conceptually and practically useful for numerous reasons . First , the MILP is a natural model of the BNN : given that a BNN uses the sign function as activation , the function the network represents is piecewise constant , and thus directly representable using linear inequalities and binary variables . Second , the flexibility of MILP allows for various constraints on the type of attacks ( e.g . locality as in an early version of ( Tjeng et al. , 2017 ) ) , as well as various or even multiple objectives ( e.g . minimizing perturbation while maximizing misclassification ) . Third , globally optimal perturbations can be computed using a MILP solver on small networks , allowing for a precise evaluation of existing attack heuristics in terms of the quality of the perturbations they produce . The generality and optimality provided by MILP solvers does , however , come at a computational cost . While we were able to solve the MILP to optimality for small networks and perturbation budgets , the solver did not scale much beyond that . Nevertheless , experimental results on small networks revealed a gap between the performance of the gradient-based attack and the best achievable . This finding , coupled with the non-differentiable nature of the BNN , suggests an alternative : a combinatorial algorithm that is : ( a ) more scalable than a MILP solve , and ( b ) more suitable for a non-differentiable objective function . To this end , we propose IProp ( Integer Propagation ) , an attack algorithm that exploits the discrete structure of a BNN , as does the MILP , but is substantially more efficient . IProp tunes the perturbation vector by iterations of “ target propagation ” : starting at a desirable activation vector in the last hidden layer D ( i.e . a target ) , IProp searches for an activation vector in layer ( D − 1 ) that can induce the target in layer D. The process is iterated until the input layer is reached , where a similar problem is solved in continuous perturbation space in order to achieve the first hidden layer ’ s target . Central to our approach is the use of MILP formulations to perform layer-to-layer target propagation . IProp is fundamentally novel in two ways : – To our knowledge , IProp is the first target propagation algorithm used in adversarial machine learning , in contrast to the typical use cases of training or credit assignment in neural networks ( Le Cun , 1986 ; Bengio , 2014 ) ; – The use of exact integer optimization methods within target propagation is also a first , and a promising direction suggested recently in ( Friesen & Domingos , 2017 ) . We evaluate the MILP model , IProp and the Projected Gradient Descent method ( with restarts ) ( PGD ) ( Madry et al. , 2017 ) – a representative gradient-based attack – on BNN models pre-trained on the MNIST ( LeCun et al. , 1998 ) and Fashion-MNIST ( Xiao et al. , 2017 ) datasets . We show that IProp compares favorably against PGD on a range of networks and across a set of evaluation metrics , especially with small perturbation budgets . As such , we believe that our work is a testament to the promise of integer optimization methods in adversarial learning and discrete neural networks . This paper is organized as follows : we describe related work in Section 2 , the MILP formulation in Section 3 , the heuristic IProp in Section 4 and experimental results in Section 5 . We conclude with a discussion on possible avenues for future work in Section 6 . 2 RELATED WORK . Neural networks with the threshold ( sign ) activation function date back to early work on the Perceptron . However , the work of ( Courbariaux et al. , 2016 ) revived the interest in Binarized Neural Networks as a computationally cheap alternative to full-precision neural networks . This resurgence is due to an effective training algorithm for BNNs . Since then , BNNs have been used in computer vision ( Rastegari et al. , 2016 ) and high-performance neural networks ( Umuroglu et al. , 2017 ; Alemdar et al. , 2017 ) , among other domains . Notably , BNNs are amenable to extremely fast ( embedded ) hardware implementations ( e.g . as in McDanel et al . ( 2017 ) ) , which may not be possible even for small full-precision networks . Adversarial attacks against modern neural networks were first investigated in ( Biggio et al. , 2013 ; Szegedy et al. , 2013 ) . Since then , the area of “ adversarial machine learning ” has developed considerably . In ( Szegedy et al. , 2013 ) , a L-BFGS method is used to find a perturbation of an input that leads to a misclassification . As an efficient alternative to L-BFGS , the Fast Gradient Sign Method ( FGSM ) was proposed in ( Goodfellow et al. , 2014 ) : FGSM uses the gradient of the loss function with respect to the input to maximize the loss , a cheap operation thanks to backpropagation . Soon thereafter , Projected Gradient Descent ( PGD ) , an iterative variant of FGSM , was shown to produce much more effective attacks ( Kurakin et al. , 2016 ; Madry et al. , 2017 ) ; PGD with random restarts is the method that we will compare against in this work . Additionally , the Appendix includes a comparison of the proposed method with SPSA ( Uesato et al. , 2018 ) . Other attacks have been developed for different constraints on the allowed amount of perturbation ( L0 , L1 , L2 norms , etc . ) ( Carlini & Wagner , 2017 ; Papernot et al. , 2016 ; Moosavi Dezfooli et al. , 2016 ) . Of relevance to our MILP approach are the MILP attacks against rectified linear unit ( ReLU ) networks of ( Tjeng et al. , 2017 ) and ( Fischetti & Jo , 2018 ) . In contrast to binarized networks , ReLU networks are differentiable almost everywhere and thus straightforwardly amenable to attacks via PGD . Galloway et al . ( 2017 ) perform an empirical evaluation of existing attack methods against BNNs and find that BNNs are more robust to gradient-based attacks than their full-precision counterparts . This finding suggests the search for more powerful attacks that exploit the discrete nature of a BNN , a key motivation for our work here . Most recently , Narodytska et al . ( 2017 ) studied the problem of verifying BNNs with satisfiability ( SAT ) solvers and MILP . In contrast to our optimization problem of maximizing the difference in outputs for a pair of classes , verification is a satisfiability problem that asks to prove that a network will not misclassify a given point , i.e . there is no objective function . As such , SAT solvers fare better than MILP solvers in BNN verification . Our IProp algorithm is complementary to the exact verification methods of Narodytska et al . ( 2017 ) , as it can be used to quickly find a counterexample perturbation , if one exists , which would help resolve the verification question negatively .

Summary: The authors study the problem of generating strong adversarial attacks on binarized neural networks (networks whose weights are binary valued and have a sign function nonlinearity).  Since these networks are not continuous (due to the sign function nonlinearity), it is possible that standard gradient-based attack algorithms are not effective at producing adversarial examples. While this problem can be encoded as a mixed integer linear program, off-the-shelf MILP solvers are not scalable to larger/deeper networks. Thus, the authors propose a new target propagation style algorithm that attempts to infer desired activations at each layer (from the perspective of maximizing the adversary's objective) starting at the final layer and moving towards the input. The propagation at each layer requires solving another MILP (albeit a much smaller one). Further, in order to prevent the target propagation from discovering assignments at upper layers that are unachievable given the constraints at lower layers, the authors propose two heuristics (making small moves and penalizing deviations from the previous target values) to obtain an effective attack algorithm. The authors validate their approach experimentally on MNIST/Fashion MNIST image classifiers.