Adversarial Unlearning of Backdoors via Implicit Hypergradient

1 INTRODUCTION . In backdoor attacks , adversaries aim to embed predefined triggers into a model during training time such that a test example , when patched with the trigger , is misclassified into a target class . For instance , it has been shown that one could use a sticker as the trigger to mislead a road sign classifier to identify STOP signs to speed limit signs ( Gu et al. , 2017 ) . Such attacks pose a great challenge to deploy machine learning in mission-critical applications ( Li et al. , 2020c ; 2021a ) . Various approaches have been proposed to remove the effect of backdoor attacks from a poisoned model . One popular class of approaches ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) is to first synthesize the trigger patterns from the model and then unlearn the triggers . However , these approaches presume that backdoor triggers only target a small portion of classes , thereby becoming ineffective when many classes are targeted by the attacker . Moreover , they suffer from high computational costs , as they either require synthesizing the trigger for each class independently or training additional models to synthesize the triggers all at once . Another line of works does not rely on trigger synthesis ; instead , it directly mitigates the triggers ’ effects via , for example , finetuning and pruning the model parameters ( Liu et al. , 2018a ) and preprocessing the model input ( Qiu et al. , 2021 ) . While these approaches are more efficient than the trigger-synthesis-based approaches , they can not maintain a good balance between robustness and model accuracy due to the unawareness of potential triggers . Recent work ( Li et al. , 2020b ) proposes to leverage a teacher model to guide fine-tuning . However , our empirical studies find that the effectiveness of this approach is particularly sensitive to the underlying attack and data augmentation techniques utilized to enrich the fine-tuning dataset . Overall , it remains a challenge to design a defense that achieves a good balance between model accuracy , robustness , and computational efficiency . To address the challenge , we propose a minimax formulation to remove backdoor triggers from a poisoned model . The formulation encompasses prior works on trigger synthesis-based defense , which solves the inner and outer problems independently . Moreover , the formulation does not make ∗Correspondence to yizeng @ vt.edu . Codes of implementations is opensourced on Github : I-BAU any assumption about the backdoor trigger beyond a bounded norm constraint , thus remaining effective across various attack settings . To solve the minimax , we propose an Implicit Bacdoor Adversarial Unlearning ( I-BAU ) algorithm based on implicit hypergradients . Unlike previous work , which breaks down the minimax into separate inner and outer optimization problems , our algorithm derives the implicit hypergradient to account for the interdependence between inner and outer optimization and uses it to update the poisoned model . We theoretically analyze the convergence of the proposed algorithm . Moreover , we investigate the generalization error of the minimax formulation , i.e. , to what extent the robustness acquired by solving the minimax generalizes to unseen data in the test time . We present the generalization bounds for both linear models and neural networks . We conduct a thorough empirical evaluation of I-BAU by comparing it with six state-of-art backdoor defenses on seven backdoor attacks over two datasets and various attack settings , including the common setting where the attacker targets one class and an important but underexplored setting where multiple classes are targeted . I-BAU ’ s performance is comparable to and most often significantly better than the best baseline . Particularly , its performance is more robust to the variation on triggers , attack settings , poison ratio , and clean data size . Moreover , I-BAU requires less computation to take effect ; particularly , it is more than 13X faster than the most efficient baseline in the single-target attack setting . It can still remain effective in the extreme case where the defender can only access 100 clean samples—a setting where all the baselines fail to produce acceptable results . 2 RELATED WORK . Backdoor Attacks . Backdoor attacks evolve through three stages . 1 ) Visble triggers using unrelated patterns ( Gu et al. , 2017 ; Chen et al. , 2017 ) , or optimized triggers ( Liu et al. , 2018b ; Bagdasaryan & Shmatikov , 2021 ; Zhao et al. , 2020 ; Garg et al. , 2020 ) for higher attack efficacy . While this line ’ s attack methods achieve high attack success rates and clean accuracy , the triggers are visible and thus easily detected by human eyes . 2 ) Visually invisible triggers via solving bilevil optimizations regarding the lp norm ( Li et al. , 2020a ) , or adopting nature optical effects ( Liu et al. , 2020 ; Nguyen & Tran , 2021 ) , and Clean label attacks via feature embeeding ( Turner et al. , 2019 ; Saha et al. , 2020 ) for better stealthiness . For most work from this line , the triggers or the poisons can evade the detection from manual efforts . 3 ) Invisible triggers considering other latent spaces , e.g. , the frequency domian ( Zeng et al. , 2021 ; Hammoud & Ghanem , 2021 ) or auto-encoder feature embedding ( Li et al. , 2021b ) , which improved the stealthiness by breaking the fundamental assumptions of a list of defenses . This paper incorporates successful attacks from these major groups and we show our method provides an effective , and generalizable defense . Backdoor Defenses . Backdoor defenses can be divided into four categories : 1 ) Poison detection via outlier detection regarding functionalities or artifacts ( Gao et al. , 2019 ; Chen et al. , 2018 ; Tran et al. , 2018 ; Koh & Liang , 2017 ; Chou et al. , 2020 ; Zeng et al. , 2021 ) , which rely on the modeling of clean samples ’ distribution . 2 ) Poisoned model identification identifies if a given model is backdoored or not ( Xu et al. , 2019 ; Wang et al. , 2020 ) . 3 ) Robust training via differential privacy ( Du et al. , 2019 ; Weber et al. , 2020 ) or ensembled/decoupled pipeline ( Levine & Feizi , 2020 ; Jia et al. , 2020 ; 2021 ; Huang et al. , 2022 ) . This line of work tries to achieve general robustness to withstand outlier ’ s impact , but may suffer from low clean accuracy . 4 ) Backdoor removal via trigger synthesising ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) , or preprocessing & finetuning ( Li et al. , 2020b ; Borgnia et al. , 2020 ; Qiu et al. , 2021 ) . This line of work serves as the fundamental solution given a poisoned model , but there is still no satisfying solution to attaining robust results across different datasets and triggers . In this work , we propose a minimax formulation and a solver for backdoor removal . We attempt to include all cutting-edge methods under this category for comparison . 3 PROBLEM FORMULATION . Attack model . We assume that an adversary carries out a backdoor attack against a clean training set generated from the distribution D. The adversary has pre-determined triggers and target classes . The adversarial goal is to poison the training dataset such that , given a clean test sample x , adding a backdoor pattern δ to x ( i.e. , x+ δ ) will alter the trained classifier output to be a target class ỹ . The norm of δ represents the adversary ’ s manipulative power . We assume that ||δ|| ≤ Cδ . In general , the attack can add r backdoored examples into the training set to induce the association between the trigger δ and the target class ỹ . The ratio of the total backdoored examples over the size of the training set is defined as the poison ratio . For better stealthiness , attacks often aim not to affect the prediction of a test example when the trigger is absent . Note that in our attack model , we explicitly acknowledge the possibility that multiple trigger patterns and multiple target classes are present . We will refer to the model trained on the poisoned training set as a poisoned model . Defense goal . We consdier that the defender is given a poisoned classifier fθpoi and an extra set of clean data D = { ( xi , yi ) } ni=1 from D. With fθpoi and D , the defender aims to build a model fθ∗ immunne to backdoors , i.e. , fθ∗ ( x+ δ ) = fθ∗ ( x ) . Note that the size of available clean examples is assumed to be much smaller than the size necessary to retrain a high-accuracy model from scratch . Minimax formulation for defense . To achieve the defense goal , we want the resulting classifier to maintain the correct label even if the attacker patches the backdoor trigger to a given input . This intuition naturally leads to the following minimax optimization formulation of backdoor removal : θ∗ = arg min θ max ‖δ‖≤Cδ H ( δ , θ ) : = 1 n n∑ i=1 L ( fθ ( xi + δ ) , yi ) , ( 1 ) where L is the loss function . Note that this formulation looks similar to adversarial training for evasion attacks ( a.k.a . adversarial examples ) ( Madry et al. , 2017 ) . The main distinction is that in evasion attacks , the adversarial perturbation is specific to an individual data point , while in backdoor attacks , the same perturbation ( i.e. , the backdoor trigger ) is expected to cause misclassifications for any examples upon being patched . Hence , in the formulation for backdoor attacks , the same trigger , δ , is applied to every clean sample xi , whereas , in the minimax formulation for evasion attacks , the perturbation is optimized independently for each sample ( see Eq . ( 2.1 ) , Madry et al . ( 2017 ) ) . The minimax formulation for backdoor removal has many advantages . 1 ) , it gives us a unifying perspective that encompasses much prior work on backdoor removal . The formulation comprises an inner maximization problem and an outer minimization problem . Both of these problems have a natural interpretation in the security context . The inner maximization problem aims to find a trigger that causes a high loss for predicting the correct label . This aligns with the problem of a backdoor attack that misleads the model to predict a predefined incorrect target label . In our formulation , we choose to maximize the prediction loss for the correct label instead of using the exact backdoor attack objective of minimizing the prediction loss associated with the target label . We make this design choice because , in reality , the defender does not know the target labels selected by the attacker . The outer minimization problem is to find model parameters so that the “ adversarial loss ” given by the inner attack problem is minimized . Existing backdoor removal approaches based on trigger synthesis and unlearning the synthesized trigger ( Wang et al. , 2019 ; Chen et al. , 2019 ; Guo et al. , 2019 ) can be considered as a special instance of this minimax formulation , where they neglect the interdependence between the inner and outer optimization and solve them separately . 2 ) , the formulation does not make any assumption on the backdoor trigger beyond the bounded norm constraint . By contrast , much of the existing work makes additional assumptions about backdoor triggers in order to be effective . For instance , synthesis-based approaches often assume that the classes coinciding with the target classes of the triggers account for only a tiny portion of the total , making those approaches ineffective in countering multiple target cases . 3 ) , the formulation provides a quantitative measure of backdoor robustness . In particular , when the parameters θ yield a ( nearly ) vanishing loss , the corresponding model is perfectly robust to attacks specified by our attack model on D. We will discuss the generalization properties of this formulation in Section 5 . Using the results developed there , we can further reason about the robustness on the data distribution D .

Summary: This study investigates defense against backdoor attacks for models that have already been trained. It proposes, in particular, a min-max formulation for backdoor defense, in which the inner maximum seeks a powerful trigger that leads to a high loss, while the outer minimum seeks to suppress the "adversarial loss", so as to unlearn the injected backdoor behaviors. To solve the minimax, the authors also propose a method, Implicit Backdoor Adversarial Unlearning (I-BAU). In addition, the authors also provide theoretical analysis including the convergence bound and generalization bound. Extensive experiments demonstrate the effectiveness and efficiency of the proposed method.