Defending Backdoor Data Poisoning Attacks by Using Noisy Label Defense Algorithm

1 INTRODUCTION . Deep neural networks ( DNN ) have achieved significant success in a variety of applications such as image classification ( Krizhevsky et al. , 2012 ) , autonomous driving ( Major et al. , 2019 ) , and natural language processing ( Devlin et al. , 2018 ) , due to its powerful generalization ability . In the meantime , DNN can be highly susceptible to even small perturbations of training data , which has raised considerable concerns about its trustworthiness ( Liu et al. , 2020 ) . One representative perturbation approach is backdoor attack , which undermines the DNN performance by modifying a small fraction of the training samples with specific triggers injected into their input features , whose ground-truth labels are altered accordingly to be the attacker-specified ones . It is unlikely to detect backdoor attacks by monitoring the model training performance , since the trained model can still perform well on the benign validation samples . Consequently , during testing phase , if the data is augmented with the trigger , it would be classified as the attacker-specified label . Subtle yet effective , backdoor attacks can pose serious threats to the practical application of DNNs . Another typical type of data poisoning attack is noisy label attacks ( Han et al. , 2018 ; Patrini et al. , 2017 ; Yi & Wu , 2019 ; Jiang et al. , 2017 ) , in which the labels of a small fraction of data are altered deliberately to compromise the model learning , while the input features of the training data remain untouched . Backdoor attacks share a close connection to noisy label attacks , in that during a backdoor attack , the feature can only be altered insignificantly to put the trigger in disguise , which makes the corrupted feature ( e.g . images with the trigger ) highly similar to the uncorrupted ones . Prior efforts have been made to effectively address noisy label attacks . For instance , there are algorithms that can tolerate a large fraction of label corruption , with up to 45 % noisy labels ( Han et al. , 2018 ; Jiang et al. , 2018 ) . However , to the best of our knowledge , most algorithms defending against backdoor attacks can not deal with a high corruption ratio even if the features of corrupted data are only slightly perturbed . Observing the limitation of prior arts , we aim to answer one key question : Can one train a deep neural network that is robust against a large number of backdoor attacks ? Moreover , given the resemblance between noisy label attacks and backdoor attacks , we also investigate another intriguing question : Can one leverage algorithms initially designed for handling noisy label attacks to defend against backdoor attacks more effectively ? The contributions of this paper are multi-fold . First , we provide a novel and principled perspective to decouple the challenges of defending backdoor attacks into two components : one induced by the corrupted input features , and the other induced by the corrupted labels , based on which we can draw a theoretical connection between the noisy-label attacks and backdoor data attacks . Second , we propose a meta-algorithm which addresses both challenges by a novel minimax optimization . Specifically , the proposed approach takes any noisy-label defense algorithm as the input and outputs a reinforced version of the algorithm that is robust against backdoor poisoning attacks , while the initial form of the algorithm fails to provide such defense . Extensive experiments show that the proposed meta-algorithm improves the robustness of DNN models against various backdoor attacks on a variety of benchmark datasets with up to 45 % corruption ratio . Furthermore , we propose a systematic , meta-framework to solve backdoor attacks , which can effectively join existing force in noisy label attack defenses and provides more insights to future development of defense algorithms . 2 RELATED WORK . 2.1 ROBUST DEEP LEARNING AGAINST ADVERSARIAL ATTACK . Although DNNs have shown high generalization performance on various tasks , it has been observed that a trained DNN model would yield different results even by perturbing the image in an invisible manner ( Goodfellow et al. , 2014 ; Yuan et al. , 2019 ) . Prior efforts have been made to tackle this issue , among which one natural defense strategy is to change the empirical loss minimization into a minimax objective . By solving the minimax problem , the model is guaranteed a better worst-case generalization performance ( Duchi & Namkoong , 2021 ) . Since exactly solving the inner maximization problem can be computationally prohibitive , different strategies have been proposed to approximate the inner maximization optimization , including heuristic alternative optimization , linear programming Wong & Kolter ( 2018 ) , semi-definite programming Raghunathan et al . ( 2018 ) , etc . Besides minimax optimization , another approach to improve model robustness is imposing a Lipschitz constraint on the network . Work along this line includes randomized smoothing Cohen et al . ( 2019 ) ; Salman et al . ( 2019 ) , spectral normalization Miyato et al . ( 2018a ) , and adversarial Lipschitz regularization Terjék ( 2019 ) . Although there are algorithms that are robust against adversarial samples , they are not designed to confront backdoor attacks , in which clean training data is usually inaccessible . There are also studies that investigated the connection between adversarial robustness and robustness against backdoor attack ( Weber et al. , 2020 ) . However , to our best knowledge , there is no literature studying the relationship between label flipping attack and backdoor attack . 2.2 ROBUST DEEP LEARNING AGAINST LABEL NOISE . Many recent studies have investigated the robustness of classification tasks with noisy labels . For example , Kumar et al . ( 2010 ) proposed the Self-Paced Learning ( SPL ) approach , which assigns higher weights to examples with a smaller loss . A similar idea was used in Curriculum Learning ( Bengio et al. , 2009 ) , in which a model learns on easier examples before moving to the harder ones . Other methods inspired by SPL include learning the data weights ( Jiang et al. , 2018 ) and collaborative learning ( Han et al. , 2018 ; Yu et al. , 2019 ) . An alternative approach to defending noisy label attacks is label correction ( Patrini et al. , 2017 ; Li et al. , 2017 ; Yi & Wu , 2019 ) , which attempts to revise the original labels of the data to recover clean labels from corrupted ones . However , since we do not have the knowledge of which data points have been corrupted , it is nontrivial to obtain provable guarantees for label corrections , unless strong assumptions have been made on the corruption type . 2.3 DATA POISONING BACKDOOR ATTACK AND ITS DEFENSE . Robust learning against backdoor attacks has been widely studied recently . Gu et al . ( 2017 ) showed that even a small patch of perturbation can compromise the generalization performance when data is augmented with a backdoor trigger . Other types of attacks include the blend attacks ( Chen et al. , 2017 ) , clean label attacks ( Turner et al. , 2018 ; Shafahi et al. , 2018 ) , latent backdoor attacks ( Yao et al. , 2019 ) , etc . While there are various types of backdoor attacks , some attack requires that the adversary not only has access to the data but also can has limited control on the training and inference process . Those attacks include Trojan attacks and blind backdoor attacks ( Pang et al. , 2020 ) . We refer readers to Pang et al . ( 2020 ) for a comprehensive survey on different types of backdoor attacks . Various defense methods have been proposed to defend against backdoor attacks . One defense category is to remove the corrupted data by using anomaly detection ( Tran et al. , 2018 ; Chen et al. , 2018 ) . Another category of work is the model inspection ( Wang et al. , 2019 ) , which aims to inspect and modify the backdoored model to make it robust against the trigger . There are also other methods of tackling the backdoor attacks , such as randomized smoothing ( Cohen et al. , 2019 ; Weber et al. , 2020 ) , and the median of means ( Levine & Feizi , 2020 ) . However , they are either inefficient or can not defend against backdoor attacks with a large ratio of corrupted data . Some of the above methods also hinge on a clean set of validation data , while in practice , it is unlikely to guarantee the existence of clean validation data , since validation data is usually a subset of training data . To the best of our knowledge , there is no existing backdoor defense algorithm that is motivated from the label corruption perspective . 3 PRELIMINARIES . 3.1 LEARNING WITH NOISY LABELS . In this section , we review some representative approaches for defending noisy-labels . Although the initial forms of these approaches can be vulnerable to backdoor attacks , we show later in the next section that our proposed meta-algorithm can empower them to effectively confront backdoor attacks . Specifically , we look into two types of nosiy-label defending approaches : filtering-based approaches and consistency-based approaches . The filtering-based approach is one of the most effective strategies for defending noisy labels , which works by selecting or weighting the training samples based on indicators such as sample losses ( Jiang et al. , 2017 ; Han et al. , 2018 ; Jiang et al. , 2020 ) or gradient norms of the loss-layer ( Liu et al. , 2021 ) . For instance , Jiang et al . ( 2017 ) proposed to assign higher probabilities to samples with lower losses to be selected for model training . Consistency-based approach modifies data labels during model training . Specifically , the Bootstrap approach ( Reed et al. , 2014 ) encourages model predictions to be consistent between iterations , by modifying the labels as a linear combination of the observed labels and previous predictions . In this paper , we leverage two filtering-based noisy label algorithms , i.e . the Self-Paced Learning ( SPL ) ( Jiang et al. , 2017 ; Kumar et al. , 2010 ) and Provable Robust Learning ( PRL ) ( Liu et al. , 2021 ) , and one consistency-based algorithm , i.e . the Bootstrap ( Reed et al. , 2014 ) , to investigate the efficacy of the proposed meta algorithm . Strong empirical results in Section 5 on those input algorithms suggest that our meta framework is readily to benefit other robust noisy-label algorithms . We briefly summarize the main idea of the above algorithms in table 1 . 3.2 PROBLEM SETTING OF BACKDOOR ATTACKS . In this paper , we follow a standard setting for backdoor attacks and assume that there is an adversary that tries to perform the backdoor attack . Firstly , the adversary can choose up to fraction of clean labels Y ∈ Rn×q and modify them to arbitrary valid numbers to form the corrupted labels Yb ∈ Rbn c×q . Let Yr represent the remaining untouched labels , then the final training labels can be denoted as Y = [ Yb , Yr ] . Accordingly , the corresponding original feature are denoted as X = [ Xo ∈ Rbn c×d , Xr ∈ R ( n−bn c ) ×d ] . The adversary can design a trigger t ∈ Rd to form the corrupted feature set Xb ∈ Rbn c×d such that for any bi in Xb , oi in Xo , it satisfies bi = oi + t. Finally , the training feature will be X = [ Xb ∈ Rbn c×d , Xr ∈ R ( n−bn c ) ×d ] . Without ambiguity,1 we also denote T = [ t , t , ... , t ] ∈ Rbn c×d , so that Xo +T = Xb . Before analyzing the algorithm , we make following assumptions about the adversary attack : Assumption 1 ( Bounded Corruption Ratio ) . The overall corruption ratio and the corruption ratio in each class is bounded . Specifically , E ( x , y , yb ) ∈ ( X , Y , Yb ) [ I ( yb = c|y 6= c ) I ( y = c ) ] ≤ = 0.5 ∀c ∈ 4Y . Assumption 2 ( Small Trigger ) . Any backdoor trigger satisfies ‖t‖p ≤ τ , which subtly alters the data within a small radius-τ ball without changing its ground-truth label . We also assume that there exists at least one black-box robust algorithm A which can defend any noisy label attacks so long as the noisy-label ratio is bounded by . Note that the above assumption is mild , since a variety of existing algorithm can handle noisy labels attacks with a large corruption rate ( e.g . 45 % ) ( Jiang et al. , 2017 ; Han et al. , 2018 ; Reed et al. , 2014 ; Liu et al. , 2021 ) .

Summary: The proposed work presents a defense algorithm for backdoor poisoning attacks. The authors consider a noisy label setting where the attacker adds a trigger to certain samples of training data and changes their label. This mainly consists of two attacks, BadNets and Blending attack. The authors then come up with an objective to train the network in a noisy label setting such that networks become robust to the poisoning attacks.