How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

1 INTRODUCTION . Deep neural networks ( DNNs ) have gained promising performances in many computer vision ( Krizhevsky et al. , 2017 ; Simonyan & Zisserman , 2015 ) , natural language processing ( Bowman et al. , 2016 ; Sehovac & Grolinger , 2020 ; Vaswani et al. , 2017 ) , and computer speech ( van den Oord et al. , 2016 ) tasks . However , it has been discovered that DNNs are vulnerable to many threats , one of which is backdoor attack ( Gu et al. , 2019 ; Liu et al. , 2018b ) , which aims to inject certain data patterns into neural networks without altering the model behavior on the clean data . Ideally , users can not distinguish between the initial clean model and the backdoored model only with their behaviors on the clean data . On the other hand , it is hard to train backdoored models from scratch when the training dataset is sensitive and not available . Recently , Garg et al . ( 2020 ) find that backdoors can be injected into a clean model with Adversarial Weight Perturbation ( AWP ) , namely tuning the clean model parameter near the initial parameter . They also conjecture that backdoors injected with AWP may be hard to detect since the variations of parameters are small . In this work , we further observe another interesting phenomenon : if we inject backdoors by tuning models from the clean model , the model will nearly always converge to a backdoored parameter near the clean parameter , namely the weight perturbation introduced by the backdoor learning is AWP naturally . To better understand this phenomenon , we first give a visualization explanation as below : In Figure 1a , there are different loss basins around local minima , a backdoored model trained from scratch tends to converge into other local minima different from the initial clean model ; In Figure 1b , backdoored models tuned from the clean model , including BadNets ( Gu et al. , 2019 ) and our proposed anchoring methods , will converge into the same loss basin where the initial clean model was located in.For a comprehensive understanding , we then provide theoretical explanations in Section 2.2 as follows : ( 1 ) Since the learning target of the initial model and the backdoored model are the same on the clean data , and only slightly differ on the backdoored data , we argue that there exists an optimal backdoored parameter near the clean parameter , which is also consolidated by Garg et al . ( 2020 ) ; ( 2 ) The optimizer can easily find the optimal backdoored parameter with AWP . Once the optimizer converges to it , it is hard to escape from it . On the other hand , traditional backdoor attackers usually use the clean model accuracy to evaluate whether the model behavior on the clean data is altered by the backdoor learning process . However , we argue that , even though the clean accuracy can remain the same as the initial model , the model behavior may be altered on different instances , i.e. , instance-wise behavior . To clarify this point , in this work , we first formulate the behavior on the clean data as the consistency of the backdoored models , including ( i ) global consistency ; and ( ii ) instance-wise consistency . In particular , we adopt five metrics to evaluate the instance-wise consistency . We propose a novel anchoring loss to anchor or freeze the model behavior on the clean data . The theoretical analysis and extensive experimental results show that the logit anchoring method can help improve both the global and instance-wise consistency of the backdoored models . As illustrated in Figure 1b , although both BadNets and the anchoring method can converge into the backdoored region near the initial clean model , compared with BadNets ( Gu et al. , 2019 ) , the parameter with our anchoring method is closer to the test local minimum and the initial model , which indicates that our anchoring method has a better global and instance-wise consistency . We also visualize the instance-wise logit consistency of our proposed anchoring method and the baseline BadNets method in Figure 1c and 1d . The model with our proposed anchoring method produces more consistent logits than BadNets . Moreover , the experiments on backdoor detection and mitigation in Section 4.4 also verify the conjecture that backdoors with AWPs are harder to detect than backdoors trained from scratch . To summarize , our contributions include : • We are the first to offer a theoretical explanation of the adversarial weight perturbation and formulate the concept of global and instance-wise consistency in backdoor learning . • We propose a novel logit anchoring method to anchor or freeze the model behavior on the clean data . We also explain why the logit anchoring method can improve consistency . • Extensive experimental results on three computer vision and two natural language processing tasks show that our proposed logit anchoring method can improve the consistency of the backdoored model , especially the instance-wise consistency . 2 PRELIMINARY . 2.1 PROBLEM DEFINITION . In this paper , we focus on the neural network for a C-class classification task . Suppose D denotes the clean training dataset , θ ∈ Rn denotes the optimal parameter on the clean training dataset , where n is the number of parameters , L ( D ; θ ) denotes the loss function on dataset D and parameter θ , and L ( ( x , y ) ; θ ) denotes the loss on data instance ( x , y ) , here θ can be omitted for brevity . Since modern deep neural networks ( DNNs ) often have multiple local minima ( Ding et al. , 2019 ) , and the optimizer does not necessarily converge to the global optimum , we may assume θ is a local minimum of the clean loss , ∇θL ( D ; θ ) = 0 . Assume θ∗ is a local minimum of the backdoored loss , then we have ∇θ∗L ( D ∪ D∗ ; θ∗ ) = 0 , where D∗ and θ∗ represent the poisonous dataset and the backdoored parameter , respectively . 2.2 ADVERSARIAL WEIGHT PERTURBATIONS AND BACKDOOR LEARNING . Adversarial Weight Perturbations ( AWPs ) ( Garg et al. , 2020 ) , or parameter corruptions ( Sun et al. , 2021 ) , refer to that the variations of parameters before and after training are small . Garg et al . ( 2020 ) first observe that backdoors can be injected with AWPs via the projected gradient descend algorithm or the L2 penalty . In this paper , we observe an interesting phenomenon that , if we train from a well-trained clean parameter θ , θ∗− θ tends to be adversarial weight perturbation , namely finding the closest local minimum in the backdoored loss curve θ∗ to the initial parameter . This phenomenon implies two properties during backdoor tuning : ( 1 ) Existence of the optimal backdoored parameter θ∗ near the clean parameter , which is also observed by Garg et al . ( 2020 ) ; ( 2 ) The optimizer tends to converge to the backdoored parameter θ∗ with adversarial weight perturbation . Here , θ∗ denotes the local optimal backdoored parameter with adversarial weight perturbation . Existence of Adversarial Weight Perturbation δ . We explain the existence of the optimal backdoored parameter θ∗ near the clean parameter , which is also observed by Garg et al . ( 2020 ) , in Proposition 1 and Remark 1 . The formal version of Remark 1 , and the proof of Proposition 1 and Remark 1 are given in Appendix.A.1 . Proposition 1 ( Upper Bound of ‖δ‖2 ) . Suppose H denotes the Hessian matrix ∇2θL ( D , θ ) on the clean dataset , g∗ = ∇θL ( D∗ , θ ) . Assume L ( D∗ , θ + δ ) ≤ L ( D∗ , θ ) − |∆L∗| can ensure that we can successfully inject a backdoor . Suppose we can choose and control the poisoning ratio η = |D∗|/|D| , the adversarial weight perturbation δ ( which is determined by η ) is , δ = δ ( η ) = −ηH−1g∗ + o ( ‖δ ( η ) ‖2 ) , ( 1 ) To ensure that we can successfully inject a backdoor , we only need to ensure that η ≥ η0 . There exists an adversarial weight perturbation δ , ‖δ‖2 ≤ |∆L∗|+ o ( 1 ) ‖g∗‖ cos〈g∗ , H−1g∗〉 . ( 2 ) Remark 1 ( Informal . Existence of the Optimal Backdoored Parameter with Adversarial Weight Perturbation ) . We take a logistic regression model for classification as an example . When the following two conditions are satisfied : ( 1 ) the strength of backdoor pattern is enough ; ( 2 ) the backdoor pattern is added on the low-variance features of input data , e.g. , on a blank corner of figures in computer vision ( CV ) , or choosing low-frequency words in Natural language processing ( NLP ) , then we can conclude that : ‖δ‖2 is small , which ensures the existence of the optimal backdoored parameter with adversarial weight perturbation . Proposition 1 estimates the upper bound of ‖δ‖2 . In Remark 1 , we investigate the conditions to ensure that ‖δ‖2 is small . We remark that some of the existing data poisoning works in backdoor learning , such as BadNets ( Gu et al. , 2019 ) , inherently satisfy conditions , which ensures the existence of the optimal backdoored parameter with adversarial weight perturbation . Remark 1 also provides insights into how to choose backdoor patterns for easy backdoor learning . The optimizer tends to converge to θ∗ . We explain ( 2 ) in Lemma 1. θ∗1 is denoted as any other local optimal backdoored parameter besides θ∗ . A detailed version of Lemma 1 is in Appendix.A.1 . Lemma 1 ( Brief Version . The mean time to converge into and escape from optimal parameters ) . The time ( step ) t for SGD to converge into an optimal parameter θ∗1 is log t ∼ ‖θ∗1 − θ‖ ( Hoffer et al. , 2017 ) . The mean escape time ( step ) τ from the optimal parameter θ∗ outside of the basin near the local minimum θ∗ is , log τ = log τ0 + Bηlr ( C1κ −1 +C2 ) ∆L∗ ( Xie et al. , 2021 ) , where B is the batch size , ηlr is the learning rate , κ measures the curvature near the clean local minimum θ∗ , ∆L∗ > 0 measures the height of the basin near the local minimum θ∗ , and τ0 , C1 , C2 ∈ R+ are parameters that are not determined by κ and ∆L∗ . Then we explain why the optimizer tends to converge to the backdoored parameter θ∗ with adversarial weight perturbation . As Figure 1a implies , the optimal backdoored parameter θ∗ with adversarial weight perturbation is close to the initial parameter θ compared to any other local optimal backdoored parameter θ∗1 , ‖θ∗ − θ‖ ‖θ∗1 − θ‖ . According to Lemma 1 , it is easy for optimizers to find the optimal backdoored parameter with adversarial weight perturbation in the early stage of backdoor learning . Modern optimizers have a large Bηlr and tend to find flat minima ( Keskar et al. , 2017 ) , thus κ is small . ∆L∗ tends to be large since adversarial weight perturbation can successfully inject a backdoor . Therefore , it is hard to escape from θ∗ according to Lemma 1 . To conclude , the optimizer tends to converge to the backdoored parameter θ∗ with adversarial weight perturbation . The formal and detailed analysis is given in Appendix.A.2 .

Summary: This paper focuses on injecting backdoors into a trained clean model. The authors provide a theoretical analysis to illustrate why the variations are always AWP or small weight perturbations in this circumstance. The concept of consistency is firstly formulated, and the authors propose a logit anchoring method to improve the consistency, especially the instance-wise consistency, under the circumstance of injecting backdoors into a trained clean model.  The main contribution of the paper includes two aspects: (1) The paper observes an interesting phenomenon that the variations of parameters are always AWPs when tuning the trained clean model to inject backdoors, and provide theoretical analysis to explain this phenomenon.  (2) The paper first formulates the consistency of backdoored models, and proposes a novel anchoring loss to anchor or freeze the model behaviors on the clean data.