Defending Against Physically Realizable Attacks on Image Classification

1 INTRODUCTION . State-of-the-art effectiveness of deep neural networks has made it the technique of choice in a variety of fields , including computer vision ( He et al. , 2016 ) , natural language processing ( Sutskever et al. , 2014 ) , and speech recognition ( Hinton et al. , 2012 ) . However , there have been a myriad of demonstrations showing that deep neural networks can be easily fooled by carefully perturbing pixels in an image through what have become known as adversarial example attacks ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Carlini & Wagner , 2017b ; Vorobeychik & Kantarcioglu , 2018 ) . In response , a large literature has emerged on defending deep neural networks against adversarial examples , typically either proposing techniques for learning more robust neural network models ( Wong & Kolter , 2018 ; Wong et al. , 2018 ; Raghunathan et al. , 2018b ; Cohen et al. , 2019 ; Madry et al. , 2018 ) , or by detecting adversarial inputs ( Metzen et al. , 2017 ; Xu et al. , 2018 ) . Particularly concerning , however , have been a number of demonstrations that implement adversarial perturbations directly in physical objects that are subsequently captured by a camera , and then fed through the deep neural network classifier ( Boloor et al. , 2019 ; Eykholt et al. , 2018 ; Athalye et al. , 2018b ; Brown et al. , 2018 ; Bhattad et al. , 2020 ) . Among the most significant of such physical attacks on deep neural networks are three that we specifically consider here : 1 ) the attack which fools face recognition by using adversarially designed eyeglass frames ( Sharif et al. , 2016 ) , 2 ) the attack which fools stop sign classification by adding adversarially crafted stickers ( Eykholt et al. , 2018 ) , and 3 ) the universal adversarial patch attack , which causes targeted misclassification of any object with the adversarially designed sticker ( patch ) ( Brown et al. , 2018 ) . Oddly , while considerable attention has been devoted to defending against adversarial perturbation attacks in the digital space , there are no effective methods specifically to defend against such physical attacks . Our first contribution is an empirical evaluation of the effectiveness of conventional approaches to robust ML against two physically realizable attacks : the eyeglass frame attack on face recognition ( Sharif et al. , 2016 ) and the sticker attack on stop signs ( Eykholt et al. , 2018 ) . Specifically , we study the performance on adversarial training and randomized smoothing against these attacks , and show that both have limited effectiveness in this context ( quite ineffective in some settings , and somewhat more effective , but still not highly robust , in others ) , despite showing moderate effectiveness against l∞ and l2 attacks , respectively . 1The code can be found at https : //github.com/tongwu2020/phattacks Our second contribution is a novel abstract attack model which more directly captures the nature of common physically realizable attacks than the conventional lp-based models . Specifically , we consider a simple class of rectangular occlusion attacks in which the attacker places a rectangular sticker onto an image , with both the location and the content of the sticker adversarially chosen . We develop several algorithms for computing such adversarial occlusions , and use adversarial training to obtain neural network models that are robust to these . We then experimentally demonstrate that our proposed approach is significantly more robust against physical attacks on deep neural networks than adversarial training and randomized smoothing methods that leverage lp-based attack models . Related Work While many approaches for defending deep learning in vision applications have been proposed , robust learning methods have been particularly promising , since alternatives are often defeated soon after being proposed ( Madry et al. , 2018 ; Raghunathan et al. , 2018a ; Wong & Kolter , 2018 ; Vorobeychik & Kantarcioglu , 2018 ) . The standard solution approach for this problem is an adaptation of Stochastic Gradient Descent ( SGD ) where gradients are either with respect to the loss at the optimal adversarial perturbation for each i ( or approximation thereof , such as using heuristic local search ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) or a convex over-approximation ( Raghunathan et al. , 2018b ; Wang et al. , 2018 ) ) , or with respect to the dual of the convex relaxation of the attacker maximization problem ( Raghunathan et al. , 2018a ; Wong & Kolter , 2018 ; Wong et al. , 2018 ) . Despite these advances , adversarial training a la Madry et al . ( 2018 ) remains the most practically effective method for hardening neural networks against adversarial examples with l∞-norm perturbation constraints . Recently , randomized smoothing emerged as another class of techniques for obtaining robustness ( Lecuyer et al. , 2019 ; Cohen et al. , 2019 ) , with the strongest results in the context of l2-norm attacks . In addition to training neural networks that are robust by construction , a number of methods study the problem of detecting adversarial examples ( Metzen et al. , 2017 ; Xu et al. , 2018 ) , with mixed results ( Carlini & Wagner , 2017a ) . Of particular interest is recent work on detecting physical adversarial examples ( Chou et al. , 2018 ) . However , detection is inherently weaker than robustness , which is our goal , as even perfect detection does not resolve the question of how to make decisions on adversarial examples . Finally , our work is in the spirit of other recent efforts that characterize robustness of neural networks to physically realistic perturbations , such as translations , rotations , blurring , and contrast ( Engstrom et al. , 2019 ; Hendrycks & Dietterich , 2019 ) . 2 BACKGROUND . 2.1 ADVERSARIAL EXAMPLES IN THE DIGITAL AND PHYSICAL WORLD . Adversarial examples involve modifications of input images that are either invisible to humans , or unsuspicious , and that cause systematic misclassification by state-of-the-art neural networks ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Vorobeychik & Kantarcioglu , 2018 ) . Commonly , approaches for generating adversarial examples aim to solve an optimization problem of the following form : argmax δ L ( f ( x+ δ ; θ ) , y ) s.t . ‖δ‖p ≤ , ( 1 ) where x is the original input image , δ is the adversarial perturbation , L ( · ) is the adversary ’ s utility function ( for example , the adversary may wish to maximize the cross-entropy loss ) , and ‖ · ‖p is some lp norm . While a host of such digital attacks have been proposed , two have come to be viewed as state of the art : the attack developed by Carlini & Wagner ( 2017b ) , and the projected gradient descent attack ( PGD ) by Madry et al . ( 2018 ) . While most of the work to date has been on attacks which modify the digital image directly , we focus on a class of physical attacks which entail modifying the actual object being photographed in order to fool the neural network that subsequently takes its digital representation as input . The attacks we will focus on will have three characteristics : 1 . The attack can be implemented in the physical space ( e.g. , modifying the stop sign ) ; 2. the attack has low suspiciousness ; this is operationalized by modifying only a small part of the object , with the modification similar to common “ noise ” that obtains in the real world ; for example , stickers on a stop sign would appear to most people as vandalism , but covering the stop sign with a printed poster would look highly suspicious ; and 3. the attack causes misclassification by state-of-the-art deep neural network . Since our ultimate purpose is defense , we will not concern ourselves with the issue of actually implementing the physical attacks . Instead , we will consider the digital representation of these attacks , ignoring other important issues , such as robustness to many viewpoints and printability . For example , in the case where the attack involves posting stickers on a stop sign , we will only be concerned with simulating such stickers on digital images of stop signs . For this reason , we refer to such attacks physically realizable attacks , to allude to the fact that it is possible to realize them in practice . It is evident that physically realizable attacks represent a somewhat stronger adversarial model than their actual implementation in the physical space . Henceforth , for simplicity , we will use the terms physical attacks and physically realizable attacks interchangeably . We consider three physically realizable attacks . The first is the attack on face recognition by Sharif et al . ( 2016 ) , in which the attacker adds adversarial noise inside printed eyeglass frames that can subsequently be put on to fool the deep neural network ( Figure 1a ) . The second attack posts adversarially crafted stickers on a stop sign to cause it to be misclassified as another road sign , such as the speed limit sign ( Figure 1b ) ( Eykholt et al. , 2018 ) . The third , adversarial patch , attack designs a patch ( a sticker ) with adversarial noise that can be placed onto an arbitrary object , causing that object to be misclassified by a deep neural network ( Brown et al. , 2018 ) . 2.2 ADVERSARIALLY ROBUST DEEP LEARNING . While numerous approaches have been proposed for making deep learning robust , many are heuristic and have soon after been defeated by more sophisticated attacks ( Carlini & Wagner , 2017b ; He et al. , 2017 ; Carlini & Wagner , 2017a ; Athalye et al. , 2018a ) . Consequently , we focus on principled approaches for defense that have not been broken . These fall broadly into two categories : robust learning and randomized smoothing . We focus on a state-of-the-art representative from each class . Robust Learning The goal of robust learning is to minimize a robust loss , defined as follows : θ∗ = argmin θ E ( x , y ) ∼D [ max ‖δ‖p≤ L ( f ( x+ δ ; θ ) , y ) ] , ( 2 ) where D denotes the training data set . In itself this is a highly intractable problem . Several techniques have been developed to obtain approximate solutions . Among the most effective in practice is the adversarial training approach by Madry et al . ( 2018 ) , who use the PGD attack as an approximation to the inner optimization problem , and then take gradient descent steps with respect to the associated adversarial inputs . In addition , we consider a modified version of this approach termed curriculum adversarial training ( Cai et al. , 2018 ) . Our implementation of this approach proceeds as follows : first , apply adversarial training for a small , then increase and repeat adversarial training , and so on , increasing until we reach the desired level of adversarial noise we wish to be robust to . Randomized Smoothing The second class of techniques we consider works by adding noise to inputs at both training and prediction time . The key idea is to construct a smoothed classifier g ( · ) from a base classifier f ( · ) by perturbing the input x with isotropic Gaussian noise with variance σ . The prediction is then made by choosing a class with the highest probability measure with respect to the induced distribution of f ( · ) decisions : g ( x ) = argmax c P ( f ( x+ σ ) = c ) , σ ∼ N ( 0 , σ2I ) . ( 3 ) To achieve provably robust classification in this manner one typically trains the classifier f ( · ) by adding Gaussian noise to inputs at training time ( Lecuyer et al. , 2019 ; Cohen et al. , 2019 ) .

Summary: The paper aims to provide a defense to physically realizable attacks for image classifiers. First, the paper demonstrates that Lp-ball robustness (obtained via adversarial training or randomized smoothing) do not necessarily result in robustness to physical attacks such as adversarial stickers. Next, the paper proposes a variant of adversarial training (using adversarial rectangles) and shows empirically that such a method results in much improved robustness to the aforementioned physical attacks.