Adversarial Unlearning of Backdoors via Implicit Hypergradient

1 INTRODUCTION . In backdoor attacks , adversaries aim to embed predefined triggers into a model during training time such that a test example , when patched with the trigger , is misclassified into a target class . For instance , it has been shown that one could use a sticker as the trigger to mislead a road sign classifier to identify STOP signs to speed limit signs ( Gu et al. , 2017 ) . Such attacks pose a great challenge to deploy machine learning in mission-critical applications ( Li et al. , 2020c ; 2021a ) . Various approaches have been proposed to remove the effect of backdoor attacks from a poisoned model . One popular class of approaches ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) is to first synthesize the trigger patterns from the model and then unlearn the triggers . However , these approaches presume that backdoor triggers only target a small portion of classes , thereby becoming ineffective when many classes are targeted by the attacker . Moreover , they suffer from high computational costs , as they either require synthesizing the trigger for each class independently or training additional models to synthesize the triggers all at once . Another line of works does not rely on trigger synthesis ; instead , it directly mitigates the triggers ’ effects via , for example , finetuning and pruning the model parameters ( Liu et al. , 2018a ) and preprocessing the model input ( Qiu et al. , 2021 ) . While these approaches are more efficient than the trigger-synthesis-based approaches , they can not maintain a good balance between robustness and model accuracy due to the unawareness of potential triggers . Recent work ( Li et al. , 2020b ) proposes to leverage a teacher model to guide fine-tuning . However , our empirical studies find that the effectiveness of this approach is particularly sensitive to the underlying attack and data augmentation techniques utilized to enrich the fine-tuning dataset . Overall , it remains a challenge to design a defense that achieves a good balance between model accuracy , robustness , and computational efficiency . To address the challenge , we propose a minimax formulation to remove backdoor triggers from a poisoned model . The formulation encompasses prior works on trigger synthesis-based defense , which solves the inner and outer problems independently . Moreover , the formulation does not make ∗Correspondence to yizeng @ vt.edu . Codes of implementations is opensourced on Github : I-BAU any assumption about the backdoor trigger beyond a bounded norm constraint , thus remaining effective across various attack settings . To solve the minimax , we propose an Implicit Bacdoor Adversarial Unlearning ( I-BAU ) algorithm based on implicit hypergradients . Unlike previous work , which breaks down the minimax into separate inner and outer optimization problems , our algorithm derives the implicit hypergradient to account for the interdependence between inner and outer optimization and uses it to update the poisoned model . We theoretically analyze the convergence of the proposed algorithm . Moreover , we investigate the generalization error of the minimax formulation , i.e. , to what extent the robustness acquired by solving the minimax generalizes to unseen data in the test time . We present the generalization bounds for both linear models and neural networks . We conduct a thorough empirical evaluation of I-BAU by comparing it with six state-of-art backdoor defenses on seven backdoor attacks over two datasets and various attack settings , including the common setting where the attacker targets one class and an important but underexplored setting where multiple classes are targeted . I-BAU ’ s performance is comparable to and most often significantly better than the best baseline . Particularly , its performance is more robust to the variation on triggers , attack settings , poison ratio , and clean data size . Moreover , I-BAU requires less computation to take effect ; particularly , it is more than 13X faster than the most efficient baseline in the single-target attack setting . It can still remain effective in the extreme case where the defender can only access 100 clean samples—a setting where all the baselines fail to produce acceptable results . 2 RELATED WORK . Backdoor Attacks . Backdoor attacks evolve through three stages . 1 ) Visble triggers using unrelated patterns ( Gu et al. , 2017 ; Chen et al. , 2017 ) , or optimized triggers ( Liu et al. , 2018b ; Bagdasaryan & Shmatikov , 2021 ; Zhao et al. , 2020 ; Garg et al. , 2020 ) for higher attack efficacy . While this line ’ s attack methods achieve high attack success rates and clean accuracy , the triggers are visible and thus easily detected by human eyes . 2 ) Visually invisible triggers via solving bilevil optimizations regarding the lp norm ( Li et al. , 2020a ) , or adopting nature optical effects ( Liu et al. , 2020 ; Nguyen & Tran , 2021 ) , and Clean label attacks via feature embeeding ( Turner et al. , 2019 ; Saha et al. , 2020 ) for better stealthiness . For most work from this line , the triggers or the poisons can evade the detection from manual efforts . 3 ) Invisible triggers considering other latent spaces , e.g. , the frequency domian ( Zeng et al. , 2021 ; Hammoud & Ghanem , 2021 ) or auto-encoder feature embedding ( Li et al. , 2021b ) , which improved the stealthiness by breaking the fundamental assumptions of a list of defenses . This paper incorporates successful attacks from these major groups and we show our method provides an effective , and generalizable defense . Backdoor Defenses . Backdoor defenses can be divided into four categories : 1 ) Poison detection via outlier detection regarding functionalities or artifacts ( Gao et al. , 2019 ; Chen et al. , 2018 ; Tran et al. , 2018 ; Koh & Liang , 2017 ; Chou et al. , 2020 ; Zeng et al. , 2021 ) , which rely on the modeling of clean samples ’ distribution . 2 ) Poisoned model identification identifies if a given model is backdoored or not ( Xu et al. , 2019 ; Wang et al. , 2020 ) . 3 ) Robust training via differential privacy ( Du et al. , 2019 ; Weber et al. , 2020 ) or ensembled/decoupled pipeline ( Levine & Feizi , 2020 ; Jia et al. , 2020 ; 2021 ; Huang et al. , 2022 ) . This line of work tries to achieve general robustness to withstand outlier ’ s impact , but may suffer from low clean accuracy . 4 ) Backdoor removal via trigger synthesising ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) , or preprocessing & finetuning ( Li et al. , 2020b ; Borgnia et al. , 2020 ; Qiu et al. , 2021 ) . This line of work serves as the fundamental solution given a poisoned model , but there is still no satisfying solution to attaining robust results across different datasets and triggers . In this work , we propose a minimax formulation and a solver for backdoor removal . We attempt to include all cutting-edge methods under this category for comparison . 3 PROBLEM FORMULATION . Attack model . We assume that an adversary carries out a backdoor attack against a clean training set generated from the distribution D. The adversary has pre-determined triggers and target classes . The adversarial goal is to poison the training dataset such that , given a clean test sample x , adding a backdoor pattern δ to x ( i.e. , x+ δ ) will alter the trained classifier output to be a target class ỹ . The norm of δ represents the adversary ’ s manipulative power . We assume that ||δ|| ≤ Cδ . In general , the attack can add r backdoored examples into the training set to induce the association between the trigger δ and the target class ỹ . The ratio of the total backdoored examples over the size of the training set is defined as the poison ratio . For better stealthiness , attacks often aim not to affect the prediction of a test example when the trigger is absent . Note that in our attack model , we explicitly acknowledge the possibility that multiple trigger patterns and multiple target classes are present . We will refer to the model trained on the poisoned training set as a poisoned model . Defense goal . We consdier that the defender is given a poisoned classifier fθpoi and an extra set of clean data D = { ( xi , yi ) } ni=1 from D. With fθpoi and D , the defender aims to build a model fθ∗ immunne to backdoors , i.e. , fθ∗ ( x+ δ ) = fθ∗ ( x ) . Note that the size of available clean examples is assumed to be much smaller than the size necessary to retrain a high-accuracy model from scratch . Minimax formulation for defense . To achieve the defense goal , we want the resulting classifier to maintain the correct label even if the attacker patches the backdoor trigger to a given input . This intuition naturally leads to the following minimax optimization formulation of backdoor removal : θ∗ = arg min θ max ‖δ‖≤Cδ H ( δ , θ ) : = 1 n n∑ i=1 L ( fθ ( xi + δ ) , yi ) , ( 1 ) where L is the loss function . Note that this formulation looks similar to adversarial training for evasion attacks ( a.k.a . adversarial examples ) ( Madry et al. , 2017 ) . The main distinction is that in evasion attacks , the adversarial perturbation is specific to an individual data point , while in backdoor attacks , the same perturbation ( i.e. , the backdoor trigger ) is expected to cause misclassifications for any examples upon being patched . Hence , in the formulation for backdoor attacks , the same trigger , δ , is applied to every clean sample xi , whereas , in the minimax formulation for evasion attacks , the perturbation is optimized independently for each sample ( see Eq . ( 2.1 ) , Madry et al . ( 2017 ) ) . The minimax formulation for backdoor removal has many advantages . 1 ) , it gives us a unifying perspective that encompasses much prior work on backdoor removal . The formulation comprises an inner maximization problem and an outer minimization problem . Both of these problems have a natural interpretation in the security context . The inner maximization problem aims to find a trigger that causes a high loss for predicting the correct label . This aligns with the problem of a backdoor attack that misleads the model to predict a predefined incorrect target label . In our formulation , we choose to maximize the prediction loss for the correct label instead of using the exact backdoor attack objective of minimizing the prediction loss associated with the target label . We make this design choice because , in reality , the defender does not know the target labels selected by the attacker . The outer minimization problem is to find model parameters so that the “ adversarial loss ” given by the inner attack problem is minimized . Existing backdoor removal approaches based on trigger synthesis and unlearning the synthesized trigger ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) can be considered as a special instance of this minimax formulation , where they neglect the interdependence between the inner and outer optimization and solve them separately . 2 ) , the formulation does not make any assumption on the backdoor trigger beyond the bounded norm constraint . By contrast , much of the existing work makes additional assumptions about backdoor triggers in order to be effective . For instance , synthesis-based approaches often assume that the classes coinciding with the target classes of the triggers account for only a tiny portion of the total , making those approaches ineffective in countering multiple target cases . 3 ) , the formulation provides a quantitative measure of backdoor robustness . In particular , when the parameters θ yield a ( nearly ) vanishing loss , the corresponding model is perfectly robust to attacks specified by our attack model on D. We will discuss the generalization properties of this formulation in Section 5 . Using the results developed there , we can further reason about the robustness on the data distribution D .

Summary: The paper formulates adversarial training against backdoor poison attacks and proposes to use implicit hypergradient to solve the minimax problem instead of breaking it down into separate inner and outer optimization problems. The authors perform theoretical analysis of the algorithm and find convergence bound and generalization bound for the method. Also, the authors evaluate the proposed adversarial training routine, called I-BAU (Implicit Backdoor Adversarial Unlearning) in three attack settings:  (1) One-trigger-one-target attack, (2) One-trigger-all-to-all attack, (3) Multi-trigger-multi-target attack and compare with six defenses. For each attack setting, the authors test 7 different backdoor attacks. The results show that the efficiency of I-BAU is comparable to the best baseline for each of the attacks and is more time-efficient than other defenses. Finally, the authors show that I-BAU leads to a more stable training comparing to using universal adversarial perturbations in adversarial learning.