Backdoor Defense via Decoupling the Training Process

1 INTRODUCTION . Deep learning , especially deep neural networks ( DNNs ) , has been widely adopted in many realms ( Wang et al. , 2020b ; Li et al. , 2020a ; Wen et al. , 2020 ) for its high effectiveness . In general , the training of DNNs requires a large amount of training samples and computational resources . Accordingly , third-party resources ( e.g. , third-party data or servers ) are usually involved . While the opacity of the training process brings certain convenience , it also introduces new security threats . Backdoor attack poses a new security threat to the training process of DNNs ( Li et al. , 2020c ) . It maliciously manipulates the prediction of the attacked DNNs by poisoning a few training samples . Specifically , backdoor attackers inject the backdoor trigger ( i.e. , a particular pattern ) to some benign training images and change their labels with the attacker-specified target label . The connection between the backdoor trigger and the target label will be learned by DNNs during the training process . In the inference process , the prediction of attacked DNNs will be changed to the target label when the trigger is present , whereas the attacked DNNs will behave normally on benign samples . As such , users are difficult to realize the existence of hidden backdoors and therefore this attack is a serious threat to the practical applications of DNNs . In this paper , we first investigate backdoor attacks from the hidden feature space . Our preliminary experiments reveal that the backdoor is embedded in the feature space , i.e. , samples with the back- ∗The first two authors contributed equally to this work . This work was mostly done when Kunzhe Huang and Yiming Li were the research interns at The Chinese University of Hong Kong , Shenzhen . † indicates corresponding authors : Baoyuan Wu ( wubaoyuan @ cuhk.edu.cn ) and Zhan Qin ( qinzhan @ zju.edu.cn ) . door trigger ( dubbed poisoned samples ) tend to cluster together in the feature space . We reveal that this phenomenon is mostly due to the end-to-end supervised training paradigm . Specifically , the excessive learning capability allows DNNs to learn features about the backdoor trigger , while the DNNs can shrink the distance between poisoned samples in the feature space and connect the learned trigger-related features with the target label by the end-to-end supervised training . Based on this understanding , we propose to decouple the end-to-end training process for the backdoor defense . Specifically , we treat the DNNs as two disjoint parts , including a feature extractor ( i.e. , backbone ) and a simple classifier ( i.e. , the remaining fully connected layers ) . We first learn the purified feature extractor via self-supervised learning ( Kolesnikov et al. , 2019 ; Chen et al. , 2020a ; Jing & Tian , 2020 ) with unlabeled training samples ( obtained by removing their labels ) , and then learn the simple classifier via standard supervised training process based on the learned feature extractor and all training samples . The strong data augmentations involved in the self-supervised learning damage trigger patterns , making them unlearnable during representation learning ; and the decoupling process further disconnects trigger patterns and the target label . Accordingly , hidden backdoors can not be successfully created even the model is trained on the poisoned dataset based on our defense . Moreover , we further reveal that the representation of poisoned samples generated by the purified extractor is significantly different from those generated by the extractor learned with standard training process . Specifically , the poisoned sample lies closely to samples with its ground-truth label instead of the target label . This phenomenon makes the training of the simple classifier similar to label-noise learning ( Wang et al. , 2019b ; Ma et al. , 2020 ; Berthon et al. , 2021 ) . As such , we first filter high-credible training samples ( i.e. , training samples that are most probably to be benign ) and then use those samples as labeled samples and the remaining part to form unlabeled samples to fine-tune the whole model via semi-supervised learning ( Rasmus et al. , 2015 ; Berthelot et al. , 2019 ; Sohn et al. , 2020 ) . This approach is to further reduce the adverse effects of poisoned samples . The main contributions of this paper are three-fold . ( 1 ) We reveal that the backdoor is embedded in the feature space , which is mostly due to the end-to-end supervised training paradigm . ( 2 ) Based on our understanding , we propose a decoupling-based backdoor defense ( DBD ) to alleviate the threat of poisoning-based backdoor attacks . ( 3 ) Experiments on classical benchmark datasets are conducted , which verify the effectiveness of our defense . 2 RELATED WORK . 2.1 BACKDOOR ATTACK . Backdoor attack is an emerging research area , which raises security concerns about training with third-party resources . In this paper , we focus on the poisoning-based backdoor attack towards image classification , where attackers can only modify the dataset instead of other training components ( e.g. , training loss ) . This threat could also happen in other tasks ( Xiang et al. , 2021 ; Zhai et al. , 2021 ; Li et al. , 2022 ) and with different attacker ’ s capacities ( Nguyen & Tran , 2020 ; Tang et al. , 2020 ; Zeng et al. , 2021a ) , which are out-of-scope of this paper . In general , existing attacks can be divided into two main categories based on the property of target labels , as follows : Poison-Label Backdoor Attack . It is currently the most common attack paradigm , where the target label is different from the ground-truth label of poisoned samples . BadNets ( Gu et al. , 2019 ) is the first and most representative poison-label attack . Specifically , it randomly selected a few samples from the original benign dataset to generate poisoned samples by stamping the backdoor trigger onto the ( benign ) image and change their label with an attacker-specified target label . Those generated poisoned samples associated with remaining benign ones were combined to form the poisoned training dataset , which will be delivered to users . After that , ( Chen et al. , 2017 ) suggested that the poisoned image should be similar to its benign version for the stealthiness , based on which they proposed the blended attack . Recently , ( Xue et al. , 2020 ; Li et al. , 2020b ; 2021c ) further explored how to conduct poison-label backdoor attacks more stealthily . Most recently , a more stealthy and effective attack , the WaNet ( Nguyen & Tran , 2021 ) , was proposed . WaNet adopted image warping as the backdoor trigger , which deforms but preserves the image content . Clean-Label Backdoor Attack . Although the poisoned image generated by poison-label attacks could be similar to its benign version , users may still notice the attack by examining the image-label relationship . To address this problem , Turner et al . ( 2019 ) proposed the clean-label attack paradigm , where the target label is consistent with the ground-truth label of poisoned samples . Specifically , they first leveraged adversarial perturbations or generative models to modify some benign images from the target class and then conducted the standard trigger injection process . This idea was generalized to attack video classification in ( Zhao et al. , 2020b ) , where they adopted the targeted universal adversarial perturbation ( Moosavi-Dezfooli et al. , 2017 ) as the trigger pattern . Although clean-label backdoor attacks are more stealthy compared with poison-label ones , they usually suffer from relatively poor performance and may even fail in creating backdoors ( Li et al. , 2020c ) . 2.2 BACKDOOR DEFENSE . Currently , there are also some approaches to alleviate the backdoor threat . Existing defenses are mostly empirical , which can be divided into five main categories , including ( 1 ) detection-based defenses ( Xu et al. , 2021 ; Zeng et al. , 2021a ; Xiang et al. , 2022 ) , ( 2 ) preprocessing based defenses ( Doan et al. , 2020 ; Li et al. , 2021b ; Zeng et al. , 2021b ) , ( 3 ) model reconstruction based defenses ( Zhao et al. , 2020a ; Li et al. , 2021a ; Zeng et al. , 2022 ) , ( 4 ) trigger synthesis based defenses ( Guo et al. , 2020 ; Dong et al. , 2021 ; Shen et al. , 2021 ) , and ( 5 ) poison suppression based defenses ( Du et al. , 2020 ; Borgnia et al. , 2021 ) . Specifically , detection-based defenses examine whether a suspicious DNN or sample is attacked and it will deny the use of malicious objects ; Preprocessing based methods intend to damage trigger patterns contained in attack samples to prevent backdoor activation by introducing a preprocessing module before feeding images into DNNs ; Model reconstruction based ones aim at removing the hidden backdoor in DNNs by modifying models directly ; The fourth type of defenses synthesize potential trigger patterns at first , following by the second stage that the hidden backdoor is eliminated by suppressing their effects ; The last type of methods depress the effectiveness of poisoned samples during the training process to prevent the creation of hidden backdoors . In general , our method is most relevant to this type of defenses . In this paper , we only focus on the last four types of defenses since they directly improve the robustness of DNNs . Besides , there were also few works focusing on certified backdoor defenses ( Wang et al. , 2020a ; Weber et al. , 2020 ) . Their robustness is theoretically guaranteed under certain assumptions , which cause these methods to be generally weaker than empirical ones in practice . 2.3 SEMI-SUPERVISED AND SELF-SUPERVISED LEARNING . Semi-supervised Learning . In many real-world applications , the acquisition of labeled data often relies on manual labeling , which is very expensive . In contrast , obtaining unlabeled samples is much easier . To utilize the power of unlabeled samples with labeled ones simultaneously , a great amount of semi-supervised learning methods were proposed ( Gao et al. , 2017 ; Berthelot et al. , 2019 ; Van Engelen & Hoos , 2020 ) . Recently , semi-supervised learning was also introduced in improving the security of DNNs ( Stanforth et al. , 2019 ; Carmon et al. , 2019 ) , where they utilized unlabelled samples in the adversarial training . Most recently , ( Yan et al. , 2021 ) discussed how to backdoor semi-supervised learning . However , this approach needs to control other training components ( e.g. , training loss ) in addition to modifying training samples and therefore is out-of-scope of this paper . How to adopt semi-supervised learning for backdoor defense remains blank . Self-supervised Learning . This learning paradigm is a subset of unsupervised learning , where DNNs are trained with supervised signals generated from the data itself ( Chen et al. , 2020a ; Grill et al. , 2020 ; Liu et al. , 2021 ) . It has been adopted for increasing adversarial robustness ( Hendrycks et al. , 2019 ; Wu et al. , 2021 ; Shi et al. , 2021 ) . Most recently , there were also a few works ( Saha et al. , 2021 ; Carlini & Terzis , 2021 ; Jia et al. , 2021 ) exploring how to backdoor self-supervised learning . However , these attacks are out-of-scope of this paper since they need to control other training components ( e.g. , training loss ) in addition to modifying training samples .

Summary: Summary: authors propose a modification to the training procedure to prevent backdoor attacks. Instead of performing supervised training, they suggest first training the model in a self-supervised way, then in a supervised way on fully connected layers. Later they propose to remove low-credible samples and fine-tune the whole model on the remaining samples with labels.  They claim that this procedure eliminates the backdoored inputs that have incorrect labels.