Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

1 INTRODUCTION . Recent studies on mode connectivity show that two independently trained deep neural network ( DNN ) models with the same architecture and loss function can be connected on their loss landscape using a high-accuracy/low-loss path characterized by a simple curve ( Garipov et al. , 2018 ; Gotmare et al. , 2018 ; Draxler et al. , 2018 ) . This insight on the loss landscape geometry provides us with easy access to a large number of similar-performing models on the low-loss path between two given models , and Garipov et al . ( 2018 ) use this to devise a new model ensembling method . Another line of recent research reveals interesting geometric properties relating to adversarial robustness of DNNs ( Fawzi et al. , 2017 ; 2018 ; Wang et al. , 2018b ; Yu et al. , 2018 ) . An adversarial data or model is defined to be one that is close to a bonafide data or model in some space , but exhibits unwanted or malicious behavior . Motivated by these geometric perspectives , in this study , we propose to employ mode connectivity to study and improve adversarial robustness of DNNs against different types of threats . A DNN can be possibly tampered by an adversary during different phases in its life cycle . For example , during the training phase , the training data can be corrupted with a designated trigger pattern associated with a target label to implant a backdoor for trojan attack on DNNs ( Gu et al. , 2019 ; Liu et al. , 2018 ) . During the inference phase when a trained model is deployed for task-solving , prediction-evasive attacks are plausible ( Biggio & Roli , 2018 ; Goodfellow et al. , 2015 ; Zhao et al. , 2018 ) , even when the model internal details are unknown to an attacker ( Chen et al. , 2017 ; Ilyas et al. , 2018 ; Zhao et al. , 2019a ) . In this research , we will demonstrate that by using mode connectivity in loss landscapes , we can repair backdoored or error-injected DNNs . We also show that mode 1The code is available at https : //github.com/IBM/model-sanitization connectivity analysis reveals the existence of a robustness loss barrier on the path connecting regular and adversarially-trained models . We motivate the novelty and benefit of using mode connectivity for mitigating training-phase adversarial threats through the following practical scenario : as training DNNs is both time- and resource-consuming , it has become a common trend for users to leverage pre-trained models released in the public domain2 . Users may then perform model fine-tuning or transfer learning with a small set of bonafide data that they have . However , publicly available pre-trained models may carry an unknown but significant risk of tampering by an adversary . It can also be challenging to detect this tampering , as in the case of a backdoor attack3 , since a backdoored model will behave like a regular model in the absence of the embedded trigger . Therefore , it is practically helpful to provide tools to users who wish to utilize pre-trained models while mitigating such adversarial threats . We show that our proposed method using mode connectivity with limited amount of bonafide data can repair backdoored or error-injected DNNs , while greatly countering their adversarial effects . Our main contributions are summarized as follows : • For backdoor and error-injection attacks , we show that the path trained using limited bonafide data connecting two tampered models can be used to repair and redeem the attacked models , thereby resulting in high-accuracy and low-risk models . The performance of mode connectivity is significantly better than several baselines including fine-tuning , training from scratch , pruning , and random weight perturbations . We also provide technical explanations for the effectiveness of our path connection method based on model weight space exploration and similarity analysis of input gradients for clean and tampered data . • For evasion attacks , we use mode connectivity to study standard and adversarial-robustness loss landscapes . We find that between a regular and an adversarially-trained model , training a path with standard loss reveals no barrier , whereas the robustness loss on the same path reveals a barrier . This insight provides a geometric interpretation of the “ no free lunch ” hypothesis in adversarial robustness ( Tsipras et al. , 2019 ; Dohmatob , 2018 ; Bubeck et al. , 2019 ) . We also provide technical explanations for the high correlation observed between the robustness loss and the largest eigenvalue of the input Hessian matrix on the path . • Our experimental results on different DNN architectures ( ResNet and VGG ) and datasets ( CIFAR10 and SVHN ) corroborate the effectiveness of using mode connectivity in loss landscapes to understand and improve adversarial robustness . We also show that our path connection is resilient to the considered adaptive attacks that are aware of our defense . To the best of our knowledge , this is the first work that proposes using mode connectivity approaches for adversarial robustness . 2 BACKGROUND AND RELATED WORK . 2.1 MODE CONNECTIVITY IN LOSS LANDSCAPES . Let w1 and w2 be two sets of model weights corresponding to two neural networks independently trained by minimizing any user-specified loss l ( w ) , such as the cross-entropy loss . Moreover , let φθ ( t ) with t ∈ [ 0 , 1 ] be a continuous piece-wise smooth parametric curve , with parameters θ , such that its two ends are φθ ( 0 ) = w1 and φθ ( 1 ) = w2 . To find a high-accuracy path between w1 and w2 , it is proposed to find the parameters θ that minimize the expectation over a uniform distribution on the curve ( Garipov et al. , 2018 ) , L ( θ ) = Et∼qθ ( t ) [ l ( φθ ( t ) ) ] ( 1 ) where qθ ( t ) is the distribution for sampling the models on the path indexed by t. Since qθ ( t ) depends on θ , in order to render the training of high-accuracy path connection more computationally tractable , ( Garipov et al. , 2018 ; Gotmare et al. , 2018 ) proposed to instead use the following loss term , L ( θ ) = Et∼U ( 0,1 ) [ l ( φθ ( t ) ) ] ( 2 ) 2For example , the Model Zoo project : https : //modelzoo.co 3See the recent call for proposals on Trojans in AI announced by IARPA : https : //www.iarpa.gov/ index.php/research-programs/trojai/trojai-baa where U ( 0 , 1 ) is the uniform distribution on [ 0 , 1 ] . The following functions are commonly used for characterizing the parametric curve function φθ ( t ) . Polygonal chain ( Gomes et al. , 2012 ) . The two trained networks w1 and w2 serve as the endpoints of the chain and the bends of the chain are parameterized by θ . For instance , the case of a chain with one bend is φθ ( t ) = { 2 ( tθ + ( 0.5− t ) ω1 ) , 0 ≤ t ≤ 0.5 2 ( ( t− 0.5 ) ω2 + ( 1− t ) θ ) , 0.5 ≤ t ≤ 1 . ( 3 ) Bezier curve ( Farouki , 2012 ) . A Bezier curve provides a convenient parametrization of smoothness on the paths connecting endpoints . For instance , a quadratic Bezier curve with endpoints w1 and w2 is given by φθ ( t ) = ( 1− t ) 2ω1 + 2t ( 1− t ) θ + t2ω2 , 0 ≤ t ≤ 1 . ( 4 ) It is worth noting that , while current research on mode connectivity mainly focuses on generalization analysis ( Garipov et al. , 2018 ; Gotmare et al. , 2018 ; Draxler et al. , 2018 ; Wang et al. , 2018a ) and has found remarkable applications such as fast model ensembling ( Garipov et al. , 2018 ) , our results show that its implication on adversarial robustness through the lens of loss landscape analysis is a promising , yet largely unexplored , research direction . Yu et al . ( 2018 ) scratched the surface but focused on interpreting decision surface of input space and only considered evasion attacks . 2.2 BACKDOOR , EVASION , AND ERROR-INJECTION ADVERSARIAL ATTACKS . Backdoor attack . Backdoor attack on DNNs is often accomplished by designing a designated trigger pattern with a target label implanted to a subset of training data , which is a specific form of data poisoning ( Biggio et al. , 2012 ; Shafahi et al. , 2018 ; Jagielski et al. , 2018 ) . A backdoored model trained on the corrupted data will output the target label for any data input with the trigger ; and it will behave as a normal model when the trigger is absent . For mitigating backdoor attacks , majority of research focuses on backdoor detection or filtering anomalous data samples from training data for re-training ( Chen et al. , 2018 ; Wang et al. , 2019 ; Tran et al. , 2018 ) , while our aim is to repair backdoored models for models using mode connectivity and limited amount of bonafide data . Evasion attack . Evasion attack is a type of inference-phase adversarial threat that generates adversarial examples by mounting slight modification on a benign data sample to manipulate model prediction ( Biggio & Roli , 2018 ) . For image classification models , evasion attack can be accomplished by adding imperceptible noises to natural images and resulting in misclassification ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Xu et al. , 2018 ) . Different from training-phase attacks , evasion attack does not assume access to training data . Moreover , it can be executed even when the model details are unknown to an adversary , via black-box or transfer attacks ( Papernot et al. , 2017 ; Chen et al. , 2017 ; Zhao et al. , 2020 ) . Error-injection attack . Different from attacks modifying data inputs , error-injection attack injects errors to model weights at the inference phase and aims to cause misclassification of certain input samples ( Liu et al. , 2017 ; Zhao et al. , 2019b ) . At the hardware level of a deployed machine learning system , it can be made plausible via laser beam ( Barenghi et al. , 2012 ) and row hammer ( Van Der Veen et al. , 2016 ) to change or flip the logic values of the corresponding bits and thus modifying the model parameters saved in memory . 3 MAIN RESULTS . Here we report the experimental results , provide technical explanations , and elucidate the effectiveness of using mode connectivity for studying and enhancing adversarial robustness in three representative themes : ( i ) backdoor attack ; ( ii ) error-injection attack ; and ( iii ) evasion attack . Our experiments were conducted on different network architectures ( VGG and ResNet ) and datasets ( CIFAR-10 and SVHN ) . The details on experiment setups are given in Appendix A . When connecting models , we use the cross entropy loss and the quadratic Bezier curve as described in ( 4 ) . In what follows , we begin by illustrating the problem setups bridging mode connectivity and adversarial robustness , summarizing the results of high-accuracy ( low-loss ) pathways between untampered models for reference , and then delving into detailed discussions . Depending on the context , we will use the terms error rate and accuracy on clean/adversarial samples interchangeably . The error rate of adversarial samples is equivalent to their attack failure rate as well as 100 % - attack accuracy .

Summary: This paper proposes an adversarial defense method based on mode connectivity. The goal of the method is to repair tampered networks using a limited number of clean data examples. The authors consider two types of adversarial attacks: backdoor attacks and error-injection attacks. The proposed method takes two potentially tampered networks, then constructs a low-loss path connecting the weight vectors of the given models in the space of network parameters (the path is constructed using the small set of clean data examples), finally an intermediate point on the path is used as a weight vector corresponding to the “repaired” model. The authors analyze the properties of the paths and show that intermediate points on the mode-connecting paths deliver both high clean-data accuracy and low attack success rate. In the experiments the proposed method shows better results compared to baseline defense techniques including fine-tuning, training from scratch, and pruning followed by fine-tuning. The paper also analyzes evasion adversarial attacks from the perspective of mode-connectivity and observes the existence of barriers in the landscape of robustness loss on the paths connecting regular and adversarially-trained models.