Backdoor Defense via Decoupling the Training Process

1 INTRODUCTION . Deep learning , especially deep neural networks ( DNNs ) , has been widely adopted in many realms ( Wang et al. , 2020b ; Li et al. , 2020a ; Wen et al. , 2020 ) for its high effectiveness . In general , the training of DNNs requires a large amount of training samples and computational resources . Accordingly , third-party resources ( e.g. , third-party data or servers ) are usually involved . While the opacity of the training process brings certain convenience , it also introduces new security threats . Backdoor attack poses a new security threat to the training process of DNNs ( Li et al. , 2020c ) . It maliciously manipulates the prediction of the attacked DNNs by poisoning a few training samples . Specifically , backdoor attackers inject the backdoor trigger ( i.e. , a particular pattern ) to some benign training images and change their labels with the attacker-specified target label . The connection between the backdoor trigger and the target label will be learned by DNNs during the training process . In the inference process , the prediction of attacked DNNs will be changed to the target label when the trigger is present , whereas the attacked DNNs will behave normally on benign samples . As such , users are difficult to realize the existence of hidden backdoors and therefore this attack is a serious threat to the practical applications of DNNs . In this paper , we first investigate backdoor attacks from the hidden feature space . Our preliminary experiments reveal that the backdoor is embedded in the feature space , i.e. , samples with the back- ∗The first two authors contributed equally to this work . This work was mostly done when Kunzhe Huang and Yiming Li were the research interns at The Chinese University of Hong Kong , Shenzhen . † indicates corresponding authors : Baoyuan Wu ( wubaoyuan @ cuhk.edu.cn ) and Zhan Qin ( qinzhan @ zju.edu.cn ) . door trigger ( dubbed poisoned samples ) tend to cluster together in the feature space . We reveal that this phenomenon is mostly due to the end-to-end supervised training paradigm . Specifically , the excessive learning capability allows DNNs to learn features about the backdoor trigger , while the DNNs can shrink the distance between poisoned samples in the feature space and connect the learned trigger-related features with the target label by the end-to-end supervised training . Based on this understanding , we propose to decouple the end-to-end training process for the backdoor defense . Specifically , we treat the DNNs as two disjoint parts , including a feature extractor ( i.e. , backbone ) and a simple classifier ( i.e. , the remaining fully connected layers ) . We first learn the purified feature extractor via self-supervised learning ( Kolesnikov et al. , 2019 ; Chen et al. , 2020a ; Jing & Tian , 2020 ) with unlabeled training samples ( obtained by removing their labels ) , and then learn the simple classifier via standard supervised training process based on the learned feature extractor and all training samples . The strong data augmentations involved in the self-supervised learning damage trigger patterns , making them unlearnable during representation learning ; and the decoupling process further disconnects trigger patterns and the target label . Accordingly , hidden backdoors can not be successfully created even the model is trained on the poisoned dataset based on our defense . Moreover , we further reveal that the representation of poisoned samples generated by the purified extractor is significantly different from those generated by the extractor learned with standard training process . Specifically , the poisoned sample lies closely to samples with its ground-truth label instead of the target label . This phenomenon makes the training of the simple classifier similar to label-noise learning ( Wang et al. , 2019b ; Ma et al. , 2020 ; Berthon et al. , 2021 ) . As such , we first filter high-credible training samples ( i.e. , training samples that are most probably to be benign ) and then use those samples as labeled samples and the remaining part to form unlabeled samples to fine-tune the whole model via semi-supervised learning ( Rasmus et al. , 2015 ; Berthelot et al. , 2019 ; Sohn et al. , 2020 ) . This approach is to further reduce the adverse effects of poisoned samples . The main contributions of this paper are three-fold . ( 1 ) We reveal that the backdoor is embedded in the feature space , which is mostly due to the end-to-end supervised training paradigm . ( 2 ) Based on our understanding , we propose a decoupling-based backdoor defense ( DBD ) to alleviate the threat of poisoning-based backdoor attacks . ( 3 ) Experiments on classical benchmark datasets are conducted , which verify the effectiveness of our defense . 2 RELATED WORK . 2.1 BACKDOOR ATTACK . Backdoor attack is an emerging research area , which raises security concerns about training with third-party resources . In this paper , we focus on the poisoning-based backdoor attack towards image classification , where attackers can only modify the dataset instead of other training components ( e.g. , training loss ) . This threat could also happen in other tasks ( Xiang et al. , 2021 ; Zhai et al. , 2021 ; Li et al. , 2022 ) and with different attacker ’ s capacities ( Nguyen & Tran , 2020 ; Tang et al. , 2020 ; Zeng et al. , 2021a ) , which are out-of-scope of this paper . In general , existing attacks can be divided into two main categories based on the property of target labels , as follows : Poison-Label Backdoor Attack . It is currently the most common attack paradigm , where the target label is different from the ground-truth label of poisoned samples . BadNets ( Gu et al. , 2019 ) is the first and most representative poison-label attack . Specifically , it randomly selected a few samples from the original benign dataset to generate poisoned samples by stamping the backdoor trigger onto the ( benign ) image and change their label with an attacker-specified target label . Those generated poisoned samples associated with remaining benign ones were combined to form the poisoned training dataset , which will be delivered to users . After that , ( Chen et al. , 2017 ) suggested that the poisoned image should be similar to its benign version for the stealthiness , based on which they proposed the blended attack . Recently , ( Xue et al. , 2020 ; Li et al. , 2020b ; 2021c ) further explored how to conduct poison-label backdoor attacks more stealthily . Most recently , a more stealthy and effective attack , the WaNet ( Nguyen & Tran , 2021 ) , was proposed . WaNet adopted image warping as the backdoor trigger , which deforms but preserves the image content . Clean-Label Backdoor Attack . Although the poisoned image generated by poison-label attacks could be similar to its benign version , users may still notice the attack by examining the image-label relationship . To address this problem , Turner et al . ( 2019 ) proposed the clean-label attack paradigm , where the target label is consistent with the ground-truth label of poisoned samples . Specifically , they first leveraged adversarial perturbations or generative models to modify some benign images from the target class and then conducted the standard trigger injection process . This idea was generalized to attack video classification in ( Zhao et al. , 2020b ) , where they adopted the targeted universal adversarial perturbation ( Moosavi-Dezfooli et al. , 2017 ) as the trigger pattern . Although clean-label backdoor attacks are more stealthy compared with poison-label ones , they usually suffer from relatively poor performance and may even fail in creating backdoors ( Li et al. , 2020c ) . 2.2 BACKDOOR DEFENSE . Currently , there are also some approaches to alleviate the backdoor threat . Existing defenses are mostly empirical , which can be divided into five main categories , including ( 1 ) detection-based defenses ( Xu et al. , 2021 ; Zeng et al. , 2021a ; Xiang et al. , 2022 ) , ( 2 ) preprocessing based defenses ( Doan et al. , 2020 ; Li et al. , 2021b ; Zeng et al. , 2021b ) , ( 3 ) model reconstruction based defenses ( Zhao et al. , 2020a ; Li et al. , 2021a ; Zeng et al. , 2022 ) , ( 4 ) trigger synthesis based defenses ( Guo et al. , 2020 ; Dong et al. , 2021 ; Shen et al. , 2021 ) , and ( 5 ) poison suppression based defenses ( Du et al. , 2020 ; Borgnia et al. , 2021 ) . Specifically , detection-based defenses examine whether a suspicious DNN or sample is attacked and it will deny the use of malicious objects ; Preprocessing based methods intend to damage trigger patterns contained in attack samples to prevent backdoor activation by introducing a preprocessing module before feeding images into DNNs ; Model reconstruction based ones aim at removing the hidden backdoor in DNNs by modifying models directly ; The fourth type of defenses synthesize potential trigger patterns at first , following by the second stage that the hidden backdoor is eliminated by suppressing their effects ; The last type of methods depress the effectiveness of poisoned samples during the training process to prevent the creation of hidden backdoors . In general , our method is most relevant to this type of defenses . In this paper , we only focus on the last four types of defenses since they directly improve the robustness of DNNs . Besides , there were also few works focusing on certified backdoor defenses ( Wang et al. , 2020a ; Weber et al. , 2020 ) . Their robustness is theoretically guaranteed under certain assumptions , which cause these methods to be generally weaker than empirical ones in practice . 2.3 SEMI-SUPERVISED AND SELF-SUPERVISED LEARNING . Semi-supervised Learning . In many real-world applications , the acquisition of labeled data often relies on manual labeling , which is very expensive . In contrast , obtaining unlabeled samples is much easier . To utilize the power of unlabeled samples with labeled ones simultaneously , a great amount of semi-supervised learning methods were proposed ( Gao et al. , 2017 ; Berthelot et al. , 2019 ; Van Engelen & Hoos , 2020 ) . Recently , semi-supervised learning was also introduced in improving the security of DNNs ( Stanforth et al. , 2019 ; Carmon et al. , 2019 ) , where they utilized unlabelled samples in the adversarial training . Most recently , ( Yan et al. , 2021 ) discussed how to backdoor semi-supervised learning . However , this approach needs to control other training components ( e.g. , training loss ) in addition to modifying training samples and therefore is out-of-scope of this paper . How to adopt semi-supervised learning for backdoor defense remains blank . Self-supervised Learning . This learning paradigm is a subset of unsupervised learning , where DNNs are trained with supervised signals generated from the data itself ( Chen et al. , 2020a ; Grill et al. , 2020 ; Liu et al. , 2021 ) . It has been adopted for increasing adversarial robustness ( Hendrycks et al. , 2019 ; Wu et al. , 2021 ; Shi et al. , 2021 ) . Most recently , there were also a few works ( Saha et al. , 2021 ; Carlini & Terzis , 2021 ; Jia et al. , 2021 ) exploring how to backdoor self-supervised learning . However , these attacks are out-of-scope of this paper since they need to control other training components ( e.g. , training loss ) in addition to modifying training samples .

Summary: This paper proposes a decoupling-based backdoor defense (DBD) on poisoning-based backdoor attacks where an adversary can modify the dataset only. Specifically, DBD combines a self-supervised feature extractor and a supervised noise-free classifier, with an additional semi-supervised learning fine-tuning step. The core idea is to decouple the feature extractor and the final prediction. The authors evaluate the effectiveness of DBD on three datasets, three backdoor attack models, and four defense baselines.