DEFENDING BACKDOOR DATA POISONING ATTACKS USING NOISY LABEL DEFENSE ALGORITHM
Year: 2021

Abstract:
Training deep neural networks with data corruption is a challenging problem. One example of such corruption is the backdoor data poisoning attack, in which an adversary strategically injects a backdoor trigger to a small fraction of the training data to subtly compromise the training process. Consequently, the trained deep neural network would misclassify testing examples that have been corrupted by the same trigger. While the label of the data could be changed to arbitrary values by an adversary, the extent of corruption injected to the feature values are strictly limited in order to keep the backdoor attack in disguise, which leads to a resemblance between the backdoor attack and a milder attack that involves only noisy labels. In this paper, we investigate an intriguing question: Can we leverage algorithms that defend against noisy labels corruptions to defend against general backdoor attacks? We first discuss the limitations of directly using the noisylabel defense algorithms to defend against backdoor attacks. Next, we propose a meta-algorithm that transforms an existing noisy label defense algorithm to one that protects against backdoor attacks. Extensive experiments on different types of backdoor attacks show that, by introducing a lightweight alteration for minimax optimization to the existing noisy-label defense algorithms, the robustness against backdoor attacks can be substantially improved, while the intial form of those algorithms would fail in presence of a backdoor attacks.

heading: 1 INTRODUCTION
text: Deep neural networks (DNN) have achieved significant success in a variety of applications such as image classification (Krizhevsky et al., 2012), autonomous driving (Major et al., 2019), and natural language processing (Devlin et al., 2018), due to its powerful generalization ability. In the meantime, DNN can be highly susceptible to even small perturbations of training data, which has raised considerable concerns about its trustworthiness (Liu et al., 2020). One representative perturbation approach is backdoor attack, which undermines the DNN performance by modifying a small fraction of the training samples with specific triggers injected into their input features, whose ground-truth labels are altered accordingly to be the attacker-specified ones. It is unlikely to detect backdoor attacks by monitoring the model training performance, since the trained model can still perform well on the benign validation samples. Consequently, during testing phase, if the data is augmented with the trigger, it would be classified as the attacker-specified label. Subtle yet effective, backdoor attacks can pose serious threats to the practical application of DNNs.
Another typical type of data poisoning attack is noisy label attacks (Han et al., 2018; Patrini et al., 2017; Yi & Wu, 2019; Jiang et al., 2017), in which the labels of a small fraction of data are altered deliberately to compromise the model learning, while the input features of the training data remain untouched. Backdoor attacks share a close connection to noisy label attacks, in that during a backdoor attack, the feature can only be altered insignificantly to put the trigger in disguise, which makes the corrupted feature (e.g. images with the trigger) highly similar to the uncorrupted ones. Prior efforts have been made to effectively address noisy label attacks. For instance, there are algorithms that can tolerate a large fraction of label corruption, with up to 45% noisy labels (Han et al., 2018; Jiang et al., 2018). However, to the best of our knowledge, most algorithms defending against backdoor attacks cannot deal with a high corruption ratio even if the features of corrupted data are only slightly perturbed. Observing the limitation of prior arts, we aim to answer one key question: Can one train a deep neural network that is robust against a large number of backdoor attacks?
Moreover, given the resemblance between noisy label attacks and backdoor attacks, we also investigate another intriguing question: Can one leverage algorithms initially designed for handling noisy label attacks to defend against backdoor attacks more effectively?
The contributions of this paper are multi-fold. First, we provide a novel and principled perspective to decouple the challenges of defending backdoor attacks into two components: one induced by the corrupted input features, and the other induced by the corrupted labels, based on which we can draw a theoretical connection between the noisy-label attacks and backdoor data attacks. Second, we propose a meta-algorithm which addresses both challenges by a novel minimax optimization. Specifically, the proposed approach takes any noisy-label defense algorithm as the input and outputs a reinforced version of the algorithm that is robust against backdoor poisoning attacks, while the initial form of the algorithm fails to provide such defense. Extensive experiments show that the proposed meta-algorithm improves the robustness of DNN models against various backdoor attacks on a variety of benchmark datasets with up to 45% corruption ratio. Furthermore, we propose a systematic, meta-framework to solve backdoor attacks, which can effectively join existing force in noisy label attack defenses and provides more insights to future development of defense algorithms.

heading: 2 RELATED WORK

heading: 2.1 ROBUST DEEP LEARNING AGAINST ADVERSARIAL ATTACK
text: Although DNNs have shown high generalization performance on various tasks, it has been observed that a trained DNN model would yield different results even by perturbing the image in an invisible manner (Goodfellow et al., 2014; Yuan et al., 2019). Prior efforts have been made to tackle this issue, among which one natural defense strategy is to change the empirical loss minimization into a minimax objective. By solving the minimax problem, the model is guaranteed a better worst-case generalization performance (Duchi & Namkoong, 2021). Since exactly solving the inner maximization problem can be computationally prohibitive, different strategies have been proposed to approximate the inner maximization optimization, including heuristic alternative optimization, linear programming Wong & Kolter (2018), semi-definite programming Raghunathan et al. (2018), etc. Besides minimax optimization, another approach to improve model robustness is imposing a Lipschitz constraint on the network. Work along this line includes randomized smoothing Cohen et al. (2019); Salman et al. (2019), spectral normalization Miyato et al. (2018a), and adversarial Lipschitz regularization Terjék (2019). Although there are algorithms that are robust against adversarial samples, they are not designed to confront backdoor attacks, in which clean training data is usually inaccessible. There are also studies that investigated the connection between adversarial robustness and robustness against backdoor attack (Weber et al., 2020). However, to our best knowledge, there is no literature studying the relationship between label flipping attack and backdoor attack.

heading: 2.2 ROBUST DEEP LEARNING AGAINST LABEL NOISE
text: Many recent studies have investigated the robustness of classification tasks with noisy labels. For example, Kumar et al. (2010) proposed the Self-Paced Learning (SPL) approach, which assigns higher weights to examples with a smaller loss. A similar idea was used in Curriculum Learning (Bengio et al., 2009), in which a model learns on easier examples before moving to the harder ones. Other methods inspired by SPL include learning the data weights (Jiang et al., 2018) and collaborative learning (Han et al., 2018; Yu et al., 2019). An alternative approach to defending noisy label attacks is label correction (Patrini et al., 2017; Li et al., 2017; Yi & Wu, 2019), which attempts to revise the original labels of the data to recover clean labels from corrupted ones. However, since we do not have the knowledge of which data points have been corrupted, it is nontrivial to obtain provable guarantees for label corrections, unless strong assumptions have been made on the corruption type.

heading: 2.3 DATA POISONING BACKDOOR ATTACK AND ITS DEFENSE
text: Robust learning against backdoor attacks has been widely studied recently. Gu et al. (2017) showed that even a small patch of perturbation can compromise the generalization performance when data is augmented with a backdoor trigger. Other types of attacks include the blend attacks (Chen et al., 2017), clean label attacks (Turner et al., 2018; Shafahi et al., 2018), latent backdoor attacks (Yao et al., 2019), etc. While there are various types of backdoor attacks, some attack requires that the adversary not only has access to the data but also can has limited control on the training and inference
process. Those attacks include Trojan attacks and blind backdoor attacks (Pang et al., 2020). We refer readers to Pang et al. (2020) for a comprehensive survey on different types of backdoor attacks.
Various defense methods have been proposed to defend against backdoor attacks. One defense category is to remove the corrupted data by using anomaly detection (Tran et al., 2018; Chen et al., 2018). Another category of work is the model inspection (Wang et al., 2019), which aims to inspect and modify the backdoored model to make it robust against the trigger. There are also other methods of tackling the backdoor attacks, such as randomized smoothing (Cohen et al., 2019; Weber et al., 2020), and the median of means (Levine & Feizi, 2020). However, they are either inefficient or cannot defend against backdoor attacks with a large ratio of corrupted data. Some of the above methods also hinge on a clean set of validation data, while in practice, it is unlikely to guarantee the existence of clean validation data, since validation data is usually a subset of training data. To the best of our knowledge, there is no existing backdoor defense algorithm that is motivated from the label corruption perspective.

heading: 3 PRELIMINARIES

heading: 3.1 LEARNING WITH NOISY LABELS
text: In this section, we review some representative approaches for defending noisy-labels. Although the initial forms of these approaches can be vulnerable to backdoor attacks, we show later in the next section that our proposed meta-algorithm can empower them to effectively confront backdoor attacks. Specifically, we look into two types of nosiy-label defending approaches: filtering-based approaches and consistency-based approaches.
The filtering-based approach is one of the most effective strategies for defending noisy labels, which works by selecting or weighting the training samples based on indicators such as sample losses (Jiang et al., 2017; Han et al., 2018; Jiang et al., 2020) or gradient norms of the loss-layer (Liu et al., 2021). For instance, Jiang et al. (2017) proposed to assign higher probabilities to samples with lower losses to be selected for model training. Consistency-based approach modifies data labels during model training. Specifically, the Bootstrap approach (Reed et al., 2014) encourages model predictions to be consistent between iterations, by modifying the labels as a linear combination of the observed labels and previous predictions.
In this paper, we leverage two filtering-based noisy label algorithms, i.e. the Self-Paced Learning (SPL) (Jiang et al., 2017; Kumar et al., 2010) and Provable Robust Learning (PRL) (Liu et al., 2021), and one consistency-based algorithm, i.e. the Bootstrap (Reed et al., 2014), to investigate the efficacy of the proposed meta algorithm. Strong empirical results in Section 5 on those input algorithms suggest that our meta framework is readily to benefit other robust noisy-label algorithms. We briefly summarize the main idea of the above algorithms in table 1.

heading: 3.2 PROBLEM SETTING OF BACKDOOR ATTACKS
text: In this paper, we follow a standard setting for backdoor attacks and assume that there is an adversary that tries to perform the backdoor attack. Firstly, the adversary can choose up to fraction of clean labels Y ∈ Rn×q and modify them to arbitrary valid numbers to form the corrupted labels Yb ∈ Rbn c×q . Let Yr represent the remaining untouched labels, then the final training labels can be denoted as Y = [Yb,Yr]. Accordingly, the corresponding original feature are denoted as X = [Xo ∈ Rbn c×d,Xr ∈ R(n−bn c)×d]. The adversary can design a trigger t ∈ Rd to form the corrupted feature set Xb ∈ Rbn c×d such that for any bi in Xb, oi in Xo, it satisfies bi = oi + t.
Finally, the training feature will be X = [Xb ∈ Rbn c×d,Xr ∈ R(n−bn c)×d]. Without ambiguity,1 we also denote T = [t, t, ..., t] ∈ Rbn c×d, so that Xo +T = Xb. Before analyzing the algorithm, we make following assumptions about the adversary attack: Assumption 1 (Bounded Corruption Ratio). The overall corruption ratio and the corruption ratio in each class is bounded. Specifically,
E(x,y,yb)∈(X,Y,Yb) [ I(yb = c|y 6= c)
I(y = c)
] ≤ = 0.5 ∀c ∈ 4Y.
Assumption 2 (Small Trigger). Any backdoor trigger satisfies ‖t‖p ≤ τ , which subtly alters the data within a small radius-τ ball without changing its ground-truth label.
We also assume that there exists at least one black-box robust algorithm A which can defend any noisy label attacks so long as the noisy-label ratio is bounded by . Note that the above assumption is mild, since a variety of existing algorithm can handle noisy labels attacks with a large corruption rate (e.g. 45%) (Jiang et al., 2017; Han et al., 2018; Reed et al., 2014; Liu et al., 2021).

heading: 4 METHODOLOGY
text: Given an -backdoor attacked dataset (X ,Y ), a clean distribution p∗ := (X,Y), and a loss function L, we aim to answer the following questions: 1) can we learn a network function f that minimizes the generalization error under the corrupted distribution, i.e. E(x,y)∼p∗ [L(f(x+ t), y)]? 2) can the learned f also minimize the generalization error under the ground-truth, clean distribution, i.e. E(x,y)∼p∗ [L(f(x), y)]? Next, we elaborate our meta-approach for defending against backdoor attacks, which answers the above two questions affirmatively.

heading: 4.1 A BLACK-BOX ROBUST ALGORITHM AGAINST NOISY LABELS
text: The ultimate goal for defending against backdoor attacks is to learn a network function f to minimize its risk given corrupted feature inputs:
min f J(f) := E(x,y)∼p∗ [L(f(x+ t), y)] . (1)
However, Equation 1 is not directly optimizable due to two-fold challenges: 1) the corrupted inputs X with an unknown trigger t, and 2) the corrupted labels Y . That is, neither the clean inputs or ground-truth labels are available. As such, we turn to an approachable objective that optimizes the worst-case of Equation 1:
min f max ‖c‖p≤τ
1
n ∑ x∈X,y∈Y [L(f(x+ c),y)] . (2)
Since the trigger satisfies ‖t‖p ≤ τ , it is easy to see that Equation 2 minimizes an upper-bound of the ground-truth loss, in that:
1
n ∑ x∈X,y∈Y L(f(x+ t),y) ≤ max ‖c‖p≤τ 1 n ∑ x∈X,y∈Y [L(f(x+ c),y)] .
To this end, directly optimizing the surrogate objective in Equation 2 is still intractable, since we do not have access to clean X and Y, which are involved in the inner maximization loop. To tackle this challenge, we will first assume that the clean label Y is available, and then relax this prerequisite by using any learning algorithms that are robust against noisy labels. Specifically, assume that φw = L ◦ f has a Lipschitz constant L w.r.t x, we further obtain a new upper bound (see Appendix for derivation details):
1
n ∑ x∈X,y∈Y [L(f(x+ c),y)] ≤ 1 n ∑ x∈X ,y∈Y φw(xi + c,y) + τL, (3)
1Some backdoor attack algorithms design instance-specific trigger. In this paper, we only focus on the static trigger case and leave the instance-specific trigger case for our future study.
which draws a principled connection between the risks when using corrupted data and clean data:
min f max ‖c‖p≤τ
1
n ∑ x∈X,y∈Y [L(f(x+ c),y)] ≈ { min f max ‖c‖p≤τ 1 n ∑ x∈X ,y∈Y [L(f(x+ c),y)] + τL } , (4)
where the first term on the RHS of Equation 4 involves optimization on the corrupted features X and clean labels Y, while the second term on the RHS requires minimizing the Lipschitz constant L w.r.t x. Recall that minimizing the maximum gradient norm is equivalent to minimizing the Lipschitz constant (Terjék, 2019). Therefore, optimizing the first term naturally regulates the maximum change of the loss function within a small ball, which hence constrains the magnitude of the gradient and has negligible effects on the Lipschitz regularization. The relationship between Lipschitz regularization and adversarial training has also been well discussed by literatures (Terjék, 2019; Miyato et al., 2018b). We defer more discussion along this line to Appendix.
Equation 4 indicates that if the target labels are not corrupted, and the learned function has a small Lipschitz constant, learning with corrupted features is feasible to achieve low risks. Up to now, the remaining challenge of optimizing the surrogate objective in Equation 4 is the inaccessible clean label set Y. Fortunately, a variety of algorithms are at hand for handling noisy labels during learning (Jiang et al., 2017; Liu et al., 2021; Kumar et al., 2010), which we can directly apply to our minimax optimization scheme. Specifically, for the outer minimization, one can have:
min f
1
n ∑ x∈X ,y∈Y [L(f(x+ c),y)] ,
For the outer minimization, we can perform the noisy-label update for the above optimization objective. For instance, Given the mini-batch Mx, My with batch size m, if we use SPL to perform the update, we can get the top (1 − )m data with a small risk L(f(x + c),y) to perform one-step gradient descent. If we use the PRL to perform the update, assume L is the cross-entropy loss, one can get the top (1− )m data with small loss-layer gradient norm ‖f(x+c)−y‖ to perform one-step gradient descent. If we use the bootstrap, we can directly add a bootstrap regularization to update the above objective.
In the meantime, it is non-trivial to directly solve the inner maximization, since adversarial learning c in Equation 4 still faces the threat of noisy labels. To tackle this issue, we can leverage the same robust noisy label algorithm. Specifically, we first approximate the inner optimization using the first-order Tyler expansion:
c∗ = argmax ‖c‖p≤τ
1
n ∑ x∈X,y∈Y L(f(x+ c),y) ≈ argmax ‖c‖p≤τ 1 n ∑ x∈X,y∈Y L(f(x),y) + cT∇xL(f(x),y)
= argmax ‖c‖p≤τ
cT∇x 1
n ∑ x∈X,y∈Y L(f(x),y).
Above optimization is a linear programming problem. With the l∞ norm ball constraint on the perturbation, the above update can be reduced to the fast gradient sign method (FGSM). Given a minibatch Mx,My with batchsize m, we can have the closed-form solution as the following:
c̃ = Clipc
{ τ m · ∑ x∈Mx,y∈My sign (∇xL (f(x),y)) } . (5)
To relax the prerequisite of a clean label set y in Equation 5, we will use a noisy-label algorithm to perform the update. For instance, if we use a loss-filtering based algorithm (e.g. SPL), then for each mini-batch, only the top (1 − )m data with small L (f(x),y) would be included in the update. If we adopt a gradient-based filtering algorithm (e.g. PRL), given that L is the cross-entropy loss, then only top (1− )m data with small ‖f(x)−y‖will be included. The outside clipping ensures that the feature value of the corrupted image is in the valid range. Based on the above building blocks, we now introduce our algorithm in Algorithm 1, a meta defense scheme that are robust against backdoor attacks, given arbitrary noisy-label robust algorithm A as an input. The algorithm is illustrated in Figure 1.
Algorithm 1: Meta algorithm for Robust Learning Against Backdoor Attacks input: Corrupted training data X , Y , perturbation limit: τ , learning with noisy label algorithm A (e.g.
PRL, SPL, Bootstrap). return trained neural network ; while epoch ≤ max epoch do
for sampled minibatch Mx,My in X ,Y do #Inner maximization step initialize c as 0 vector. optimize the objective max‖c‖≤τ L(f(Mx + c),My) w.r.t to c by using robust algorithm A optimize the objective minf L(f(Mx + c),My) w.r.tf by using robust algorithm A
end end

heading: 4.2 THEORETICAL JUSTIFICATION
text: Our ultimate goal is to learn w that achieves a small risk Ex,y∼p∗φw(x + t,y). To study the generalization performance on the ground-truth distribution p∗, we first define the following risks: Rempt = 1
n
∑ x∈X,y∈Y φw(x+t,y),Rt = Ex,y∼p∗φw(x+t,y),Rempc = 1
n
∑ x∈X ,y∈Y φw(x+
c,y), and R̃empc = 1
n
∑ x∈X ,y∈Y φw(x+c,y). Next, we focus on the gap betweenRt andRempc .
Theorem 1. Let R̃empc ,Rempc ,Rt, , τ defined as above. Assume that the prior distribution of the network parameter w is N (0, σ), and the posterior distribution of parameter is N (w, σ) which is learned from the training data. Let k be the number of parameters, and n be the sample size. If the objective function φw = L ◦ f is Lφ-Lipschitz smooth, then with probability at least 1-δ, one can derive:
Rt ≤ Rempc + Lφ(2τ + τ) + √√√√ 14k log (1 + ‖w‖22kσ2 )+ 14 + log nδ + 2 log(6n+ 3k) n− 1 . (6)
We hereby present the skeleton of the proof and defer more details to Appendix. First, we decompose the error into two components: 1) the generalization gap on the triggered data, and 2) the difference of performance loss between the trigger t and worst case perturbation c: Rt − Rempc = (Rt − Rempt ) + (R emp t −Rempc ).
The first components is
√ 1 4k log ( 1+ ‖w‖22 kσ2 ) + 14+log n δ +2 log(6n+3k)
n−1 , which is derived by following the PAC-Bayes framework (Foret et al., 2020). For the second term, the gap is introduced by two sources. The first source is the difference between c and t, and the second is from the difference between X and X . Since the objective isLφ Lipschitz, and ‖t−c‖ ≤ 2τ according to our constraint to the adversary, it is easy to upper bound the error as 2τLφ. In the meantime, there is -fraction of
difference between X and X , which is bounded by ‖t‖ < τ and leads to the other difference term Lφ τ .
Theorem 1 presents an upper-bound of the gap Rt − Rempc . The first term in Equation 6 can be minimized by using a noisy label algorithm. The second term, which is the error induced by the adversarial trigger, is jointly constrained by the Lipschitz constant Lφ, perturbation limit τ , and the corruption ratio . We can regularize the Lφ whereas the τ and are controlled by the unknown adversary. Note that existing literature has also shown that adversarial training plays a resemblant role as the Lipschitz regularization. The last term, i.e. the normal generalization error on the clean data, is difficult to minimize directly. The bound in Theorem 1 emphasizes the importance of involving both the noisy label algorithm and the adversarial training. The noisy label algorithm can reduce theRempc while the adversarial training minimizes the Lipschitz constant Lφ to optimize the second term.

heading: 5 EXPERIMENT
text: In this section, we perform the empirical study. In our experiment, we perform experiment on CIFAR10 and CIFAR100 benchmark data. We use ResNet-32 (He et al., 2016) as the backbone network structure for all experiment baseline. The initial learning rate of all methods are set to be 3e-4, and we use AdamW (Loshchilov & Hutter, 2017) as the optimizer for all methods. The evaluation metric is the top-1 accuracy for both clean testing data and testing data with backdoor trigger.
For the backdoor data poisoning attack, we use simple badnet attack (Gu et al., 2017) and gaussian blending attack (Chen et al., 2017) since these two attacks do not require any information about the model or training procedure (Pang et al., 2020).
• badnet patch attack: trigger is a 3× 3 black-white checkerboard and it is added to the right bottom corner of the image.
• blending attack: trigger is a fixed Gaussian noise which has the same dimension as the image. The corrupted image generated by x i = (1−α)xi+αt. In our experiment, we set the α as 0.1.
The poisoned sample can be found at Appendix. We deploy the multi-target backdoor attack in this paper. Our poisoning approach is as following: we first systematically flip the label to perform the label-flipping attack . Then, for those being attacked samples, we add the triggers on their feature. Without adding the trigger, the problem would reduce to the noisy label problem.
and we use the following two evaluation metric.
• top-1 clean accuracy: the top-1 accuracy calculated on trigger-less testing data • top-1 poison accuracy: the top-1 accuracy calculated by the model prediction on poisoned
testing data and ground truth clean label.
The first metric describes how model performs on benign data while the second metric describes how model performs on triggered data. We varies our training data poisoning rate as [15%, 25%, 35%, 45%] to investigate how our algorithm performs against different corruption ratio. All methods are trained 100 epochs, Also, in this paper, we assume there is no clean validation data available. Thus, it is difficult to perform early stopping or decide which epoch result should be used. Thus, we report the averaged accuracy across last 10 epochs for every methods.
We investigate three noisy label algorithms by comparing the performance of the original method and reinforced method. Specifically, we choose SPL, PRL, and Bootstrap as our original noisy label algorithm and the corresponding reinforced algorithm with adversarial training SPL-AT, PRL-AT, Bootstrap-AT. We also compared our method against adversarial training only (AT), which only uses the adversarial training without using any noisy label algorithm. To show the success of the attack, we also includes the standard training results.

heading: 5.1 HOW ROBUST LEARNING AGAINST NOISY LABEL PERFORMS ON BACKDOOR DATA
text: In this section, we aim to answer the first question: how does noisy label defense algorithm performs against backdoor attack?
To answer above question, We evaluate PRL, SPL, and Bootstrap on CIFAR10 and CIFAR100 dataset. The results can be found in table 2 and table 3. As we can see in the table, the standard training performs well on the benign testing data (i.e. high clean accuracy) while its performance on triggered data is very bad (i.e. low poison accuracy), which indicates the effectiveness of the backdoor attack.
As for the three noisy label algorithm, we found that although Bootstrap, SPL, and PRL all performs well for the benign testing data, they all fail when defending the backdoor attack data especially with large corruption rate, which illustrates that noisy label algorithm cannot defend the backdoor attack especially when corruption ratio is large.

heading: 5.2 HOW ADVERSARIAL TRAINING IMPROVES THE NOISY LABEL ALGORITHM
text: To investigate whether adversarial training could improve the robustness of existing noisy label algorithm against backdoor attack, we performed experiment on SPL-AT, PRL-AT and Bootstrap-AT to see how they performs on both clean test data and triggered test data. The results can be found in table 2 and table 3. As we can see that by adding the adversarial training, the performance on triggered data is largely improved (i.e. the poison accuracy significantly improves). Also, we noticed that this improvement pattern is hold for all three noisy label algorithm, which indicates the effectiveness of the proposed method on improving robustness against backdoor attack. Also, compared to adversarial training only (AT), adding noisy label does improve the performance. Especially for the PRL algorithm, we see the PRL-AT achieves better performance than AT. Also, we found that compared to consistency based noisy label algorithm, filtering based algorithm are more easier to be boosted by adversarial training. The potential reason behind this could be that filtering based method is more efficient against noisy label algorithm Han et al. (2018); Jiang et al. (2017); Liu et al. (2021).

heading: 5.3 ABLATION STUDY
text: In this section, we aim to explore more about the proposed framework. Since our algorithm use the noisy-label solver for both inner and outer optimization. A interesting question to ask is that whether both inner and outer noisy-label solver plays an important role in defense the backdoor attack. Thus, we have two variants. One is we only use noisy label algorithm to update the model for outer minimization and another one is we only use the noisy label algorithm to update the model for inner maximization. The results can be found at table 4 and table 5. As we could see in these two tables, using noisy label algorithm to perform the inner maximization is more important compared to using noisy label algorithm to perform out minimization.

heading: 6 CONCLUSION
text: In this paper, we investigate the connection between label flipping attack and backdoor data poisoning attack. We show that although robust algorithm against label flipping attack cannot defend the
backdoor data poisoning attack, adding the adversarial training on existing algorithm could largely improve the robustness against backdoor attack.Both theoretical and empirical analysis show the effectiveness of our proposed meta algorithm.

heading: 7 APPENDIX
text: In this section, we provided proof of theorem and more discussion.
7.1 PROOF OF EQUATION 3
1
n ∑ x∈X,y∈Y [L(f(x+ c),y)] ≤ 1 n ∑ x∈X ,y∈Y φw(xi + c,y) + τL
let G denote the initially clean sample set (i.e. (X,Y)), and B the corrupted sample set (i.e. the training set corrupted with a trigger whereas the labels are untouched). Let R denote the clean sample set which is replaced by the adversary (i.e. R is the subset of G, and is replaced by B, i.e. G′ = G \ R ∪ B = (X ,Y)), and let φw denote the function L ◦ f . One can decompose the inner part of our mini-max objective in Equation 2 as the following,
1
n ∑ x∈X,y∈Y [L(f(x+ c),y)] = 1 n ∑ i∈G′\B φw(xi + c,y) + 1 n ∑ i∈R φw(xi + c,y)
= 1
n ∑ i∈G′\B φw(xi + c,y) + 1 n ∑ i∈R φw(xi + c,y) + 1 n ∑ i∈B φw(xi + t+ c,y)− 1 n ∑ i∈B φw(xi + t+ c,y)
= 1
n ∑ x∈X ,y∈Y φw(xi + c,y) +
( 1
n ∑ i∈R φw(xi + c,y)− 1 n ∑ i∈B φw(xi + c+ t,y)
)
≤ 1 n ∑ x∈X ,y∈Y φw(xi + c,y) + ∣∣∣∣∣ ( 1 n ∑ i∈R φw(xi + c,y)− 1 n ∑ i∈B φw(xi + c+ t,y) )∣∣∣∣∣ ≤ 1 n ∑ x∈X ,y∈Y φw(xi + c,y) + L‖t‖ ≤ 1 n ∑ x∈X ,y∈Y φw(xi + c,y) + τL,

heading: 7.2 PROOF OF THEOREM 1
text: Theorem. Let R̃empc ,Rempc ,Rt, , τ , is defined as above. Assume the prior distribution of the network parameter w is N (0, σ), and the posterior distribution of parameter is N (w, σ) is the posterior parameter distribution, where w is learned according to training data. Let k to be the number of parameters, n to be the sample size, assume the objective function φw = L ◦ f is Lφlipschitz smooth, then, with probability at least 1-δ, we have:
Rt ≤ Rempc + Lφ(2τ + τ) + √√√√ 14k log (1 + ‖w‖22kσ2 )+ 14 + log nδ + 2 log(6n+ 3k) n− 1
Proof: we first decompose the gap as following
Rt −Rempc = (Rt −R emp t ) + (R emp t −Rempc ) ≤ |(Rt −R emp t )|+ |(R emp t −Rempc )|
We bound the second part first. Rempt −Rempc ≤ ‖R emp t −Rempc ‖
= 1
n ‖ ∑ x∈Xr,y∈Yr [φ(x+ t,y)− φ(x+ c,y)] +  ∑ x∈Xo,y∈Yo φ(x+ t,y)− ∑ x∈Xb,y∈Yo φ(x+ c,y)  ‖ ≤ 1 n ‖ ∑ x∈Xr,y∈Yr [φ(x+ t,y)− φ(x+ c,y)] ‖+ 1 n ‖ ∑ x∈Xo,y∈Yo φ(x+ t,y)− ∑ x∈Xb,y∈Yo φ(x+ c,y)‖
≤ (1− )Lφ‖t− c‖+ Lφ‖t− c‖+ Lφ max xo,xb ‖xo − xb‖
≤ (1− )Lφ‖t− c‖+ Lφ‖t− c‖+ Lφ‖t‖ = Lφ‖t− c‖+ Lφ‖t‖ ≤ Lφ2τ + Lφ‖t‖ ≤ Lφ(2τ + τ)
Now, we bound the second term. Note the second term is a typical gap term between empirical loss and generalization loss, and there are many approaches to bound this term like VC dimension. Since we aimed to focus the deep neural network, we follow the PAC-Bayes framework McAllester (1999) to analyze the generalization bound. Specifically, we use results from Foret et al. (2020),
which gives
√ 1 4k log ( 1+ ‖w‖22 kσ2 ) + 14+log n δ +2 log(6n+3k)
n−1 under the assumption of gaussian prior and posterior. The proof for this can be found in the appendix of Foret et al. (2020) (i.e. equation 13 on the paper).

heading: 7.3 DISCUSSION ABOUT LIPSCHITZ REGULARIZATION AND ADVERSARIAL TRAINING
text: As we could see from the above theorem that a small lipschitz constant could bring robustness against backdoor attack. In this section, we provided the reason why we claim adversarial training helps lipschitz regularization. The definition of lipschitz function is ‖f(x) − f(y)‖ ≤ L‖x − y‖,∀x,y. Since the lipschitz constant showes in the upper bound of the error, we would like to get the minimum lipschitz constant to tighten the bound. Follow (Terjék, 2019), the minimum lipschitz constant can be expressed as:
‖f‖L = sup x,y∈X;x 6=y
dY (f(x), f(y))
dX(x, y)
rewrite y as x+ c, we would get
‖f‖L = sup x,x+r∈X;0<dX(x,x+c)
dY (f(x), f(x+ r))
dX(x, x+ r)
Minimize above objective respect to function f reduces to the adversarial learning.
inf f ‖f‖L = inf f sup
x,x+c∈X;0<dX(x,x+c)
dY (f(x), f(x+ c))
dX(x, x+ c)
If we treat the denominator as a constant, then this is exactly the same as our minimax objective. More details can be found in (Terjék, 2019).

heading: 7.4 SUPPLEMENTARY EXPERIMENT RESULTS
text: We provided the experiment hyperparameters, and supplementary results for the experiment.

heading: 7.4.1 EXPERIMENT HYPERPARAMETERS
text: We list the details of experiment in this section. All the methods use resnet-32 as the backbone network. AdamW is used as the opimitzer for all methods. The perturbation limit τ is set to be 0.05 for all methods requiring τ . All methods are repeated for three different random seeds to calculate the standard deviation.
The trigger for badnet attack and blending attack can be found in figure 2.

heading: 7.4.2 EXPERIMENT ON MNIST
text: We found interesting results on MNIST. In MNIST, we found adversarial training itself sometimes gives robustness to the backdoor attack. We hypothesis that this is because the MNIST is potentially a easier task than CIFAR. In here, we show the performance of adversarial training and PRL-AT on MNIST. The results can be found at table 6.
Backdoor Attack Defense Accuracy. Dataset AT BootStrap Bootstrap-AT PRL PRL-AT SPL SPL-AT
MNIST with Patch Attack, Poison Accuracy 0.15 0.30± 0.07 0.04± 0.01 3.17± 3.23 97.96± 0.21 98.44± 0.05 59.24± 33.30 85.61± 12.56 0.25 0.26± 0.17 0.04± 0.02 0.17± 0.10 89.91± 7.53 97.04± 1.07 25.25± 1.38 30.70± 6.62 0.35 0.10± 0.02 0.08± 0.06 0.14± 0.01 77.91± 10.41 97.71± 0.18 13.54± 0.76 26.03± 5.27 0.45 0.11± 0.01 0.04± 0.02 0.42± 0.34 43.42± 11.66 76.42± 8.65 12.85± 1.90 10.97± 2.16
MNIST with Patch Attack, Clean Accuracy 0.15 98.17± 0.69 99.49± 0.05 95.48± 1.56 98.08± 0.26 98.44± 0.05 93.23± 4.72 97.74± 0.37 0.25 98.59± 0.22 99.48± 0.07 98.83± 0.19 97.46± 0.07 97.11± 1.06 86.98± 0.72 85.89± 1.76 0.35 94.48± 4.87 99.48± 0.04 98.45± 0.32 97.40± 0.46 97.86± 0.11 73.09± 4.34 77.49± 0.96 0.45 98.27± 0.43 99.42± 0.02 96.44± 1.36 75.69± 0.99 92.32± 4.25 60.58± 1.90 57.20± 0.37
MNIST with Blend Attack, Poison Accuracy 0.15 63.42± 35.24 0.04± 0.01 96.66± 2.58 96.81± 1.30 96.74± 1.03 97.43± 0.13 96.16± 0.19 0.25 70.43± 28.61 0.04± 0.01 97.83± 0.91 77.68± 20.34 97.20± 0.66 6.43± 1.27 83.86± 2.74 0.35 58.32± 40.59 0.05± 0.03 97.94± 0.57 78.79± 17.74 97.59± 0.12 11.05± 2.70 69.69± 6.59 0.45 97.66± 1.04 0.03± 0.03 98.16± 0.58 27.18± 19.53 95.17± 1.83 4.49± 1.08 64.78± 3.14
MNIST with Blend Attack, Clean Accuracy 0.15 64.78± 33.81 99.44± 0.02 98.29± 0.81 97.93± 0.25 96.18± 1.44 97.30± 0.22 95.93± 0.20 0.25 74.00± 25.25 99.46± 0.05 97.44± 0.99 97.30± 0.63 97.10± 0.74 77.75± 0.74 83.34± 2.81 0.35 58.62± 40.18 99.44± 0.02 97.43± 0.94 96.25± 1.84 97.39± 0.19 72.41± 3.80 67.92± 8.38 0.45 96.78± 1.42 99.42± 0.06 97.89± 0.61 76.47± 8.66 95.07± 1.59 63.63± 4.47 63.82± 4.12
Table 6: Performance on MNIST. is the corruption rate.
As we can see for the MNIST, especially for the blend attack, the poison accuracy for adversarial training does show good performance with a large standard deviation. This is because that some random seeds works while some random seed failed. We hypothesis that this is because MNIST
dataset has almost binary feature value. When adding a small gaussian noise on feature x, the label flipping attack cannot change the decision boundary much. That is why the noisy label algorithm seems is not as important as the noisy label algorithm in CIFAR dataset. We plot a two dimensional toy example in figure 3 to illustrate label flipping attack on continuous features and binary features. As we can see in the figure that for the binary value feature, the label flipping attack is not easy to change the decision boundary too much while it can easily change the decision boundary in the continuous feature value scenario. However, this is a very rough conjecture for the reason why in MNIST, adversarial training sometimes works. We leave the investigation of this phenomenon in the future work.

heading: 7.5 COMPARISON AGAINST OTHER BACKDOOR DEFENSE ALGORITHM
text: In this section, we show further results against other backdoor defense algorithms. There are many existing backdoor defense algorithms. However, we found that a large fraction of those algorithms are either designed for single target attack (Liu et al., 2018) or requires clean data (Liu et al., 2018; Wang et al., 2019; Li et al., 2021). To investigate how our framework performs compared with the previous study, We compare against the following two baselines in a similar setting:
• spectral signature (Tran et al., 2018): filtering data by examining the score of projecting to singular vector.
• fine-pruning (Liu et al., 2018): prune the model by deleting non-activated neurons. Note this method uses 5% clean training data.
The results are in the table 7. As we can see, PRL-AT has higher poisoned accuracy against Spectral Signature and Fine-Pruning on most settings while the clean accuracy is still high, which indicates the effectiveness of our algorithm. With high corruption ratio, we find that the robustness of spectral signature and fine-pruning significantly decreases while PRL-AT still gives reasonable poison accuracy. Our theorem provides a principled way to design new defense algorithms by leveraging more knowledge from noisy label attacks. In this comparison, we only use PRL as the noisy label algorithm, and potentially, by using a more powerful noisy label algorithm (i.e. combination of multiple methods), it is possible to get higher poisoning accuracy.

heading: 7.6 SENSITIVITY ANALYSIS OF THE
text: One interesting question to ask is how our algorithm performs without knowing the corruption ratio. In this section, we provided the worst-case result, in which we fix = 0.5 to perform the noisy label algorithm. When the ground truth corruption ratio is higher than a half, it is impossible to learn any meaningful classier. We test the PRL-AT in the CIFAR10 on both badnet and blending attacks. The results are in table 8. As we can see, our algorithm provides robustness even use highlyoverestimated estimated compared with the standard training results in table 2, which suggests our algorithm is not sensitive to the hyperparameter .

heading: 7.7 DISCUSSION ABOUT NOISY LABEL ALGORITHM
text: One key question to ask for our framework is how to choose the noisy label algorithm. In terms of practice, we found PRL gives consistent robustness against both badnet and blending attacks on
different settings. This might be because that PRL is designed for agnostic corrupted supervision, which is suitable for a variety of types of label noise attacks.
From a theoretical perspective, analyzing how different noisy label algorithm can minimize the first term of RHS in equation 6 depends on which noisy label algorithm to use. Here we give a rough analyze of PRL. PRL guarantees converging to the -approximated stationary point, where is the corrupted ratio. Formally, we have the following corollary: Corollary 1 (Convergence of PRL to clean objective (Liu et al., 2021)). Assuming the maximum clean gradient before loss layer has bounded operator norm:‖W‖op ≤ C, applying PRL to any -fraction supervision corrupted data, yields mint∈[T ] E (‖∇φ(wt)‖) = O( √ q) for large enough

heading: T , where q is the dimension of the supervision.
text: More details can be found on the in Liu et al. (2021). According to above corollary, let wPRL is the solution get by PRL algorithm, we can have ‖∇wPRLRempc ‖ = O( ) (i.e. assume q is small). With Polyak-Lojasiewicz (PL) condition with some constant µ such that 1
2 ‖∇f(x)‖ ≥ µ(f(x) − f∗)
holds, we have µ(Rempc − Remp∗c ) ≤ 1
2 ‖∇wPRLRempc ‖ = O( ). For a highly-overparameterized
deep neural network, the global optima Remp∗c is usually 0. Thus, we can conclude that with PL condition, using PRL as the noisy label algorithm in our framework can guarantee Rempc can be minimized to the order 1
µ O( ). This yields the following proposition:
Proposition 1. LetRt, , τ , is defined as above. Assume the prior distribution of the network parameter w isN (0, σ), and the posterior distribution of parameter isN (w, σ) is the posterior parameter distribution, where w is learned according to training data. Let k to be the number of parameters, n to be the sample size, assume the objective function φw = L◦f is Lφ-lipschitz smooth and satisfying the PL condition, which is 1
2 ‖∇φw‖ ≥ µ(φw − φw∗). then, with the assumption of bounded oper-
ator norm of gradient before loss layer, we have with probability at least 1-δ, by applying PRL-AT, we have:
Rt ≤ 1
µ O( ) + Lφ(2τ + τ) + √√√√ 14k log (1 + ‖w‖22kσ2 )+ 14 + log nδ + 2 log(6n+ 3k) n− 1
In general, considering φ is a deep neural network, we believe the first term is difficult to analyze without further assumption (i.e. PL condition). Nevertheless, empirical study shows that many noisy label algorithms can effectively minimize the first term loss, which motivates our method to treat those algorithms as a black-box algorithm.

Review summaries:
The paper introduces an algorithm to defend against backdoor data poisoning attacks by leveraging noisy-label defense algorithms. Per se, noisy-label algorithms are not capable of defending against backdoors. However, the authors propose to use these techniques in combination with their proposed algorithm for adversarial training to reduce the effectiveness of backdoor data poisoning attacks. 

The proposed work presents a defense algorithm for backdoor poisoning attacks. The authors consider a noisy label setting where the attacker adds a trigger to certain samples of training data and changes their label. This mainly consists of two attacks, BadNets and Blending attack. The authors then come up with an objective to train the network in a noisy label setting such that networks become robust to the poisoning attacks. 

This paper aims to apply noisy-label defense algorithms to defend general backdoor attacks. The authors propose a meta-algorithm by adding noisy labels to the training set and optimizing outer minimization and inner maximization of the model by the noisy-label algorithm. Combined with adversarial training, the authors show their method could improve the robustness of models against backdoor attack.