Meta Adversarial Training

1 INTRODUCTION . Deep learning is currently the most promising method for open-world perception tasks such as in automated driving and robotics . However , the use in safety-critical domains is questionable , since a lack of robustness of deep learning-based perception has been demonstrated ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Metzen et al. , 2017 ; Hendrycks & Dietterich , 2019 ) . Physical-world adversarial attacks ( Kurakin et al. , 2017 ; Athalye et al. , 2018 ; Braunegg et al. , 2020 ) are one of most problematic failures in robustness of deep learning . Examples of such attacks are fooling models for traffic sign recognition ( Chen et al. , 2018 ; Eykholt et al. , 2018a ; b ; Huang et al. , 2019 ) , face recognition ( Sharif et al. , 2016 ; 2017 ) , optical flow estimation ( Ranjan et al. , 2019 ) , person detection ( Thys et al. , 2019 ; Wu et al. , 2020b ; Xu et al. , 2020 ) , and LiDAR perception ( Cao et al. , 2019 ) . In this work , we focus on two subsets of these physical-world attacks : local ones which place a printed pattern in a scene that does not overlap with the target object ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) and global ones which attach a mainlytranslucent sticker on the lens of a camera ( Li et al. , 2019 ) . Note that these physical-world attacks have cor- responding digital-domain attacks , in which the attacker directly modifies the signal after it was received by the sensor and before it is processed by the model . The corresponding digital-domain attack for the adversarial camera sticker is a type of universal adversarial perturbation ( MoosaviDezfooli et al. , 2017 ) , while the digital adversarial patch attack ( Brown et al. , 2017 ; Anonymous , 2020 ) corresponds to physical patch attacks ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) . We focus on increasing robustness against digital-domain attacks . Digital-domain attacks are strictly stronger than the corresponding physical-world attacks since they allow the attacker to have complete control over the change of the signal . In contrast , physical-world attacks need to be invariant under non-controllable effects such as scale , rotation , object position , and light conditions , which can not be controlled by the attacker . Therefore , a system robust against digital-domain attacks is also robust against the corresponding physical-world attacks . Currently , the most promising method for increasing robustness against adversarial attacks is adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) . Adversarial training simulates an adversarial attack for every mini-batch and trains the model to become robust against such an attack . Adversarial training against digital-domain universal perturbations or patches is complicated by the fact that these attacks are computationally much more expensive than image-dependent adversarial attacks and existing approaches for speeding up adversarial training ( Shafahi et al. , 2019 ; Zhang et al. , 2019 ; Zheng et al. , 2020 ) are not directly applicable . Existing approaches for tailoring adversarial training to universal perturbations or patches either refrain from simulating attacks in every minibatch ( Moosavi-Dezfooli et al. , 2017 ; Hayes & Danezis , 2018 ; Perolat et al. , 2018 ) , which bears the risk that the model easily overfits these fixed or rarely updated universal perturbations . Alternative approaches use proxy attacks that are computationally cheaper such as “ universal adversarial training ” ( UAT ) ( Shafahi et al. , 2018 ) and “ shared adversarial training ” ( SAT ) ( Mummadi et al. , 2019 ) . These approaches face the challenge of balancing the implicit trade-off between simulating universal perturbation attacks accurately and keeping computation cost of the proxy attacks small . We propose meta adversarial training ( MAT ) 1 , which falls into the category of proxy attacks . MAT combines adversarial training with meta-learning . We summarize the key novel contributions of MAT and refer to Section 3 for details : • MAT amortizes the cost of computing universal perturbations by sharing information about optimal perturbations over consecutive steps of model training , which reduces the cost of generating strong approximations of universal perturbations considerably . In contrast to UAT ( Shafahi et al. , 2018 ) , MAT uses meta-learning for sharing of information rather than joint training , which empirically generates stronger perturbations and a more robust model . • MAT meta-learns a large set of perturbations concurrently . While a model easily overfits a single perturbation , even if it changes as in UAT , overfitting is much less likely for a larger set of perturbations such as those generated with MAT . • MAT encourages diversity of the generated perturbations by assigning random but fixed target classes and step-sizes to each perturbation during meta-learning . This avoids that many perturbations focus on exploiting the same vulnerability of a model . We perform an extensive empirical evaluation and ablation study of MAT on image classification and traffic-light detection tasks against a variety of attacks to show the robustness of MAT against universal patches and perturbations ( see Section 4 ) . We refer to Figure 1 for an illustration of MAT for universal patch attacks against traffic light detection . 2 RELATED WORK . We review work on generating universal perturbations , defending against them , and meta-learning . Generating Universal Perturbations Adversarial perturbations are changes to the input that are crafted with the intention of fooling a model ’ s prediction on the input . Universal perturbations are a special case in which one perturbation needs to be effective on the majority of samples from the input distribution . Most work focuses on small additive perturbations that are bounded by some ` p-norm constraint . For example , Moosavi-Dezfooli et al . ( 2017 ) proposed the first approach by extending the DeepFool algorithm ( Moosavi-Dezfooli et al. , 2016 ) . Similarly , Metzen et al . ( 2017 ) extended the iterative fast gradient sign method ( Kurakin et al. , 2017 ) for generating universal perturbations on semantic image segmentation . Mopuri et al . ( 2017 ; 2018 ) presented data-independent attacks and 1Code will be publicly released upon acceptance and can be found in the supplementary material . Hayes & Danezis ( 2018 ) proposed using a generative model for learning a diverse distribution of universal perturbations . Li et al . ( 2019 ) presented a physical-world attack in which a translucent sticker is placed on the lens of a camera , which adds a universal perturbation to the image taken by the camera , and showed that this can fool an image classification system . Other types of universal perturbations are so-called adversarial patches ( Brown et al. , 2017 ) . In these universal patch attacks , the adversary can arbitrarily modify a small part of the image , typically a connected rectangular area , while leaving the remaining part of the image unchanged . Following Athalye et al . ( 2018 ) , randomizing conditions such as location , rotation , scale , and lighting during the attack can make the universal patch sufficiently effective to fool the model when it is printed out and placed in the physical world . Later work has generalized these physical-world attacks to object detection ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) and optical flow estimation ( Ranjan et al. , 2019 ) . Defending Universal Perturbations First works for defending against universal perturbations are based on training a model against a fixed or slowly updated set/distribution of universal perturbations : Moosavi-Dezfooli et al . ( 2017 ) precompute a set of universal perturbations that are used during training , Hayes & Danezis ( 2018 ) learn a generative model of universal perturbations , and Perolat et al . ( 2018 ) build a slowly increasing set of universal perturbations concurrent to model training . A shortcoming of these approaches is that the model might overfit the fixed or slowly changing distribution of universal perturbations . However , re-computing universal perturbations in every mini-batch from scratch is prohibitively expensive . To address this issue , SAT Mummadi et al . ( 2019 ) trains a model against so-called shared perturbations . These shared perturbations do not have to be universal but only need to fool the model on a fixed subset of the batch . However , since the shared perturbations are recomputed in every mini-batch , it assumes a few gradient steps are sufficient to find strong perturbations from random initialization . In contrast , our method meta-learns strong initial perturbations . In UAT ( Shafahi et al. , 2018 ) , training the neural network ’ s weights and updating a single universal perturbation happen concurrently , which scales to a large dataset . However , our experiments in Section 4 indicate that a single incrementally and slowly updated perturbation is not sufficiently strong and diverse for making a model robust against all possible universal perturbations . Instead , our method meta-learns a large and diverse collection of perturbations during training . For defending against adversarial patches , Chiang et al . ( 2020 ) proposed an approach of extending interval-bound propagation ( Gowal et al. , 2019 ) to the patch threat model . While this allows certification of robustness , it only scales to tiny patches and reduces clean accuracy considerably . Wu et al . ( 2020a ) proposed the “ defense against occlusion attack ” , which applies adversarial training to inputs perturbed with input-dependent adversarial patches placed at specific positions determined , for example , by the input gradient magnitude . Since they generate patches from scratch , they require an expensive optimization of the patch for every training batch . Moreover , robustness against stronger attacks such as those proposed in Section 4.1 remains unclear . Saha et al . ( 2019 ) hypothesize that vulnerability of object detectors against adversarial patches stems from contextual reasoning . Accordingly , they propose Grad-defense which penalizes strong dependence of object detections on their context in a data-driven manner , where dependence is determined by Grad-CAM ( Selvaraju et al. , 2019 ) . Lastly , some non-adversarial data augmentation techniques resemble the universal adversarial patch scenario : they add a Gaussian noise patch ( Lopes et al. , 2019 ) or a patch from a different image ( CutMix ) ( Yun et al. , 2019 ) to each input . CutMix is conceptually very similar to the out-of-context defense ( Saha et al. , 2019 ) . However , as demonstrated in our experiments in Section 4 , even though these approaches increase robustness against occlusions , they are unlikely to increase robustness against universal patch attacks . Meta-Learning Gradient-based meta-learning methods such as MAML ( Finn et al. , 2017 ) or REPTILE ( Nichol et al. , 2018 ) allow learning initial parameters for a class of optimization tasks , so that one can find close-to-optimal parameters on a novel task from the distribution with a small number of gradient steps . Moreover , meta-learning can also be used to learn the task optimizer itself such as by Xiong & Hsieh ( 2020 ) in the context of adversarial training . While it is common to meta-learn initial weights for neural networks , we propose that these algorithms can also be used to meta-learn initial values for universal perturbations . In this work , we combine REPTILE with adversarial training because of the low computational overhead of REPTILE ; however , in principle other gradient-based meta-learning methods could also be used as part of our method .

Summary: This paper studies the use of meta learning and adversarial training to defend against universal perturbations. This approach tries to learn a set of perturbations through meta-learning and train the model to defend against such attacks. Experimental results on Tiny ImageNet and the Bosch small traffic lights dataset show the method is effective in defending against universal patch-based attacks.