Meta Adversarial Training

1 INTRODUCTION . Deep learning is currently the most promising method for open-world perception tasks such as in automated driving and robotics . However , the use in safety-critical domains is questionable , since a lack of robustness of deep learning-based perception has been demonstrated ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ; Metzen et al. , 2017 ; Hendrycks & Dietterich , 2019 ) . Physical-world adversarial attacks ( Kurakin et al. , 2017 ; Athalye et al. , 2018 ; Braunegg et al. , 2020 ) are one of most problematic failures in robustness of deep learning . Examples of such attacks are fooling models for traffic sign recognition ( Chen et al. , 2018 ; Eykholt et al. , 2018a ; b ; Huang et al. , 2019 ) , face recognition ( Sharif et al. , 2016 ; 2017 ) , optical flow estimation ( Ranjan et al. , 2019 ) , person detection ( Thys et al. , 2019 ; Wu et al. , 2020b ; Xu et al. , 2020 ) , and LiDAR perception ( Cao et al. , 2019 ) . In this work , we focus on two subsets of these physical-world attacks : local ones which place a printed pattern in a scene that does not overlap with the target object ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) and global ones which attach a mainlytranslucent sticker on the lens of a camera ( Li et al. , 2019 ) . Note that these physical-world attacks have cor- responding digital-domain attacks , in which the attacker directly modifies the signal after it was received by the sensor and before it is processed by the model . The corresponding digital-domain attack for the adversarial camera sticker is a type of universal adversarial perturbation ( MoosaviDezfooli et al. , 2017 ) , while the digital adversarial patch attack ( Brown et al. , 2017 ; Anonymous , 2020 ) corresponds to physical patch attacks ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) . We focus on increasing robustness against digital-domain attacks . Digital-domain attacks are strictly stronger than the corresponding physical-world attacks since they allow the attacker to have complete control over the change of the signal . In contrast , physical-world attacks need to be invariant under non-controllable effects such as scale , rotation , object position , and light conditions , which can not be controlled by the attacker . Therefore , a system robust against digital-domain attacks is also robust against the corresponding physical-world attacks . Currently , the most promising method for increasing robustness against adversarial attacks is adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) . Adversarial training simulates an adversarial attack for every mini-batch and trains the model to become robust against such an attack . Adversarial training against digital-domain universal perturbations or patches is complicated by the fact that these attacks are computationally much more expensive than image-dependent adversarial attacks and existing approaches for speeding up adversarial training ( Shafahi et al. , 2019 ; Zhang et al. , 2019 ; Zheng et al. , 2020 ) are not directly applicable . Existing approaches for tailoring adversarial training to universal perturbations or patches either refrain from simulating attacks in every minibatch ( Moosavi-Dezfooli et al. , 2017 ; Hayes & Danezis , 2018 ; Perolat et al. , 2018 ) , which bears the risk that the model easily overfits these fixed or rarely updated universal perturbations . Alternative approaches use proxy attacks that are computationally cheaper such as “ universal adversarial training ” ( UAT ) ( Shafahi et al. , 2018 ) and “ shared adversarial training ” ( SAT ) ( Mummadi et al. , 2019 ) . These approaches face the challenge of balancing the implicit trade-off between simulating universal perturbation attacks accurately and keeping computation cost of the proxy attacks small . We propose meta adversarial training ( MAT ) 1 , which falls into the category of proxy attacks . MAT combines adversarial training with meta-learning . We summarize the key novel contributions of MAT and refer to Section 3 for details : • MAT amortizes the cost of computing universal perturbations by sharing information about optimal perturbations over consecutive steps of model training , which reduces the cost of generating strong approximations of universal perturbations considerably . In contrast to UAT ( Shafahi et al. , 2018 ) , MAT uses meta-learning for sharing of information rather than joint training , which empirically generates stronger perturbations and a more robust model . • MAT meta-learns a large set of perturbations concurrently . While a model easily overfits a single perturbation , even if it changes as in UAT , overfitting is much less likely for a larger set of perturbations such as those generated with MAT . • MAT encourages diversity of the generated perturbations by assigning random but fixed target classes and step-sizes to each perturbation during meta-learning . This avoids that many perturbations focus on exploiting the same vulnerability of a model . We perform an extensive empirical evaluation and ablation study of MAT on image classification and traffic-light detection tasks against a variety of attacks to show the robustness of MAT against universal patches and perturbations ( see Section 4 ) . We refer to Figure 1 for an illustration of MAT for universal patch attacks against traffic light detection . 2 RELATED WORK . We review work on generating universal perturbations , defending against them , and meta-learning . Generating Universal Perturbations Adversarial perturbations are changes to the input that are crafted with the intention of fooling a model ’ s prediction on the input . Universal perturbations are a special case in which one perturbation needs to be effective on the majority of samples from the input distribution . Most work focuses on small additive perturbations that are bounded by some ` p-norm constraint . For example , Moosavi-Dezfooli et al . ( 2017 ) proposed the first approach by extending the DeepFool algorithm ( Moosavi-Dezfooli et al. , 2016 ) . Similarly , Metzen et al . ( 2017 ) extended the iterative fast gradient sign method ( Kurakin et al. , 2017 ) for generating universal perturbations on semantic image segmentation . Mopuri et al . ( 2017 ; 2018 ) presented data-independent attacks and 1Code will be publicly released upon acceptance and can be found in the supplementary material . Hayes & Danezis ( 2018 ) proposed using a generative model for learning a diverse distribution of universal perturbations . Li et al . ( 2019 ) presented a physical-world attack in which a translucent sticker is placed on the lens of a camera , which adds a universal perturbation to the image taken by the camera , and showed that this can fool an image classification system . Other types of universal perturbations are so-called adversarial patches ( Brown et al. , 2017 ) . In these universal patch attacks , the adversary can arbitrarily modify a small part of the image , typically a connected rectangular area , while leaving the remaining part of the image unchanged . Following Athalye et al . ( 2018 ) , randomizing conditions such as location , rotation , scale , and lighting during the attack can make the universal patch sufficiently effective to fool the model when it is printed out and placed in the physical world . Later work has generalized these physical-world attacks to object detection ( Lee & Kolter , 2019 ; Huang et al. , 2019 ) and optical flow estimation ( Ranjan et al. , 2019 ) . Defending Universal Perturbations First works for defending against universal perturbations are based on training a model against a fixed or slowly updated set/distribution of universal perturbations : Moosavi-Dezfooli et al . ( 2017 ) precompute a set of universal perturbations that are used during training , Hayes & Danezis ( 2018 ) learn a generative model of universal perturbations , and Perolat et al . ( 2018 ) build a slowly increasing set of universal perturbations concurrent to model training . A shortcoming of these approaches is that the model might overfit the fixed or slowly changing distribution of universal perturbations . However , re-computing universal perturbations in every mini-batch from scratch is prohibitively expensive . To address this issue , SAT Mummadi et al . ( 2019 ) trains a model against so-called shared perturbations . These shared perturbations do not have to be universal but only need to fool the model on a fixed subset of the batch . However , since the shared perturbations are recomputed in every mini-batch , it assumes a few gradient steps are sufficient to find strong perturbations from random initialization . In contrast , our method meta-learns strong initial perturbations . In UAT ( Shafahi et al. , 2018 ) , training the neural network ’ s weights and updating a single universal perturbation happen concurrently , which scales to a large dataset . However , our experiments in Section 4 indicate that a single incrementally and slowly updated perturbation is not sufficiently strong and diverse for making a model robust against all possible universal perturbations . Instead , our method meta-learns a large and diverse collection of perturbations during training . For defending against adversarial patches , Chiang et al . ( 2020 ) proposed an approach of extending interval-bound propagation ( Gowal et al. , 2019 ) to the patch threat model . While this allows certification of robustness , it only scales to tiny patches and reduces clean accuracy considerably . Wu et al . ( 2020a ) proposed the “ defense against occlusion attack ” , which applies adversarial training to inputs perturbed with input-dependent adversarial patches placed at specific positions determined , for example , by the input gradient magnitude . Since they generate patches from scratch , they require an expensive optimization of the patch for every training batch . Moreover , robustness against stronger attacks such as those proposed in Section 4.1 remains unclear . Saha et al . ( 2019 ) hypothesize that vulnerability of object detectors against adversarial patches stems from contextual reasoning . Accordingly , they propose Grad-defense which penalizes strong dependence of object detections on their context in a data-driven manner , where dependence is determined by Grad-CAM ( Selvaraju et al. , 2019 ) . Lastly , some non-adversarial data augmentation techniques resemble the universal adversarial patch scenario : they add a Gaussian noise patch ( Lopes et al. , 2019 ) or a patch from a different image ( CutMix ) ( Yun et al. , 2019 ) to each input . CutMix is conceptually very similar to the out-of-context defense ( Saha et al. , 2019 ) . However , as demonstrated in our experiments in Section 4 , even though these approaches increase robustness against occlusions , they are unlikely to increase robustness against universal patch attacks . Meta-Learning Gradient-based meta-learning methods such as MAML ( Finn et al. , 2017 ) or REPTILE ( Nichol et al. , 2018 ) allow learning initial parameters for a class of optimization tasks , so that one can find close-to-optimal parameters on a novel task from the distribution with a small number of gradient steps . Moreover , meta-learning can also be used to learn the task optimizer itself such as by Xiong & Hsieh ( 2020 ) in the context of adversarial training . While it is common to meta-learn initial weights for neural networks , we propose that these algorithms can also be used to meta-learn initial values for universal perturbations . In this work , we combine REPTILE with adversarial training because of the low computational overhead of REPTILE ; however , in principle other gradient-based meta-learning methods could also be used as part of our method .

Summary: The authors propose a novel meta adversarial training method. In particular, to improve the robustness against digital domain attacks, the proposed meta adversarial training (MAT) combines adversarial training and meta-learning, which reduces the computing cost compared with adversarial training by generating a set of stronger perturbations. The proposed approach sounds with extensive experimental results.