D EFENDING B ACKDOOR D ATA P OISONING A TTACKS U SING N OISY L ABEL D EFENSE A LGORITHM
Anonymous authors
Paper under double-blind review
A BSTRACT
Training deep neural networks with data corruption is a challenging problem.
One example of such corruption is the backdoor data poisoning attack, in which an adversary strategically injects a backdoor trigger to a small fraction of the training data to subtly compromise the training process.
Consequently, the trained deep neural network would misclassify testing examples that have been corrupted by the same trigger.
While the label of the data could be changed to arbitrary values by an adversary, the extent of corruption injected to the feature values are strictly limited in order to keep the backdoor attack in disguise, which leads to a resemblance between the backdoor attack and a milder attack that involves only noisy labels.
In this paper, we investigate an intriguing question: Can we leverage algorithms that defend against noisy labels corruptions to defend against general backdoor attacks?
We ﬁrst discuss the limitations of directly using the noisylabel defense algorithms to defend against backdoor attacks.
Next, we propose a meta-algorithm that transforms an existing noisy label defense algorithm to one that protects against backdoor attacks.
Extensive experiments on different types of backdoor attacks show that, by introducing a lightweight alteration for minimax optimization to the existing noisy-label defense algorithms, the robustness against backdoor attacks can be substantially improved, while the intial form of those algorithms would fail in presence of a backdoor attacks.
I NTRODUCTION
Deep neural networks (DNN) have achieved signiﬁcant success in a variety of applications such as image classiﬁcation (Krizhevsky et al., 2012), autonomous driving (Major et al., 2019), and natural language processing (Devlin et al., 2018), due to its powerful generalization ability.
In the meantime, DNN can be highly susceptible to even small perturbations of training data, which has raised considerable concerns about its trustworthiness (Liu et al., 2020).
One representative perturbation approach is backdoor attack, which undermines the DNN performance by modifying a small fraction of the training samples with speciﬁc triggers injected into their input features, whose ground-truth labels are altered accordingly to be the attacker-speciﬁed ones.
It is unlikely to detect backdoor attacks by monitoring the model training performance, since the trained model can still perform well on the benign validation samples.
Consequently, during testing phase, if the data is augmented with the trigger, it would be classiﬁed as the attacker-speciﬁed label.
Subtle yet effective, backdoor attacks can pose serious threats to the practical application of DNNs.
Another typical type of data poisoning attack is noisy label attacks (Han et al., 2018;
Patrini et al., 2017; Yi & Wu, 2019; Jiang et al., 2017), in which the labels of a small fraction of data are altered deliberately to compromise the model learning, while the input features of the training data remain untouched.
Backdoor attacks share a close connection to noisy label attacks, in that during a backdoor attack, the feature can only be altered insigniﬁcantly to put the trigger in disguise, which makes the corrupted feature ( e.g. images with the trigger) highly similar to the uncorrupted ones.
Prior efforts have been made to effectively address noisy label attacks .
For instance, there are algorithms that can tolerate a large fraction of label corruption, with up to 45% noisy labels (Han et al., 2018; Jiang et al., 2018).
However, to the best of our knowledge, most algorithms defending against backdoor attacks cannot deal with a high corruption ratio even if the features of corrupted data are only slightly perturbed.
Observing the limitation of prior arts, we aim to answer one key question:
Can one train a deep neural network that is robust against a large number of backdoor attacks?
Moreover, given the resemblance between noisy label attacks and backdoor attacks, we also investigate another intriguing question: Can one leverage algorithms initially designed for handling noisy label attacks to defend against backdoor attacks more effectively?
The contributions of this paper are multi-fold.
First, we provide a novel and principled perspective to decouple the challenges of defending backdoor attacks into two components: one induced by the corrupted input features, and the other induced by the corrupted labels, based on which we can draw a theoretical connection between the noisy-label attacks and backdoor data attacks.
Second, we propose a meta-algorithm which addresses both challenges by a novel minimax optimization.
Speciﬁcally, the proposed approach takes any noisy-label defense algorithm as the input and outputs a reinforced version of the algorithm that is robust against backdoor poisoning attacks, while the initial form of the algorithm fails to provide such defense.
Extensive experiments show that the proposed meta-algorithm improves the robustness of DNN models against various backdoor attacks on a variety of benchmark datasets with up to 45% corruption ratio.
Furthermore, we propose a systematic, meta-framework to solve backdoor attacks, which can effectively join existing force in noisy label attack defenses and provides more insights to future development of defense algorithms.
R ELATED W ORK 2. R OBUST D EEP L EARNING A GAINST A DVERSARIAL A TTACK
Although DNNs have shown high generalization performance on various tasks, it has been observed that a trained DNN model would yield different results even by perturbing the image in an invisible manner (Goodfellow et al., 2014; Yuan et al., 2019).
Prior efforts have been made to tackle this issue, among which one natural defense strategy is to change the empirical loss minimization into a minimax objective.
By solving the minimax problem, the model is guaranteed a better worst-case generalization performance (Duchi & Namkoong, 2021).
Since exactly solving the inner maximization problem can be computationally prohibitive, different strategies have been proposed to approximate the inner maximization optimization, including heuristic alternative optimization, linear programming Wong & Kolter (2018), semi-deﬁnite programming Raghunathan et al.
(2018), etc.
Besides minimax optimization, another approach to improve model robustness is imposing a
Lipschitz constraint on the network.
Work along this line includes randomized smoothing Cohen et al.
Salman et al.
(2019), spectral normalization Miyato et al.
(2018a), and adversarial Lipschitz regularization Terj ´ ek (2019).
Although there are algorithms that are robust against adversarial samples, they are not designed to confront backdoor attacks, in which clean training data is usually inaccessible.
There are also studies that investigated the connection between adversarial robustness and robustness against backdoor attack (Weber et al., 2020).
However, to our best knowledge, there is no literature studying the relationship between label ﬂipping attack and backdoor attack.
2. R OBUST D EEP L EARNING A GAINST L ABEL N OISE
Many recent studies have investigated the robustness of classiﬁcation tasks with noisy labels.
For example, Kumar et al.
(2010) proposed the Self-Paced Learning ( SPL ) approach, which assigns higher weights to examples with a smaller loss.
A similar idea was used in Curriculum Learning (Bengio et al., 2009), in which a model learns on easier examples before moving to the harder ones.
Other methods inspired by SPL include learning the data weights (Jiang et al., 2018) and collaborative learning (Han et al., 2018; Yu et al., 2019).
An alternative approach to defending noisy label attacks is label correction (Patrini et al., 2017;
Li et al., 2017; Yi & Wu, 2019), which attempts to revise the original labels of the data to recover clean labels from corrupted ones.
However, since we do not have the knowledge of which data points have been corrupted, it is nontrivial to obtain provable guarantees for label corrections, unless strong assumptions have been made on the corruption type.
2. D ATA P OISONING B ACKDOOR A TTACK AND I TS DEFENSE
Robust learning against backdoor attacks has been widely studied recently.
Gu et al.
(2017) showed that even a small patch of perturbation can compromise the generalization performance when data is augmented with a backdoor trigger.
Other types of attacks include the blend attacks (Chen et al., 2017), clean label attacks (Turner et al., 2018; Shafahi et al., 2018), latent backdoor attacks (Yao et al., 2019), etc.
While there are various types of backdoor attacks, some attack requires that the adversary not only has access to the data but also can has limited control on the training and inference process.
Those attacks include Trojan attacks and blind backdoor attacks (Pang et al., 2020).
We refer readers to Pang et al.
(2020) for a comprehensive survey on different types of backdoor attacks.
Various defense methods have been proposed to defend against backdoor attacks.
One defense category is to remove the corrupted data by using anomaly detection (Tran et al., 2018; Chen et al., 2018).
Another category of work is the model inspection (Wang et al., 2019), which aims to inspect and modify the backdoored model to make it robust against the trigger.
There are also other methods of tackling the backdoor attacks, such as randomized smoothing (Cohen et al., 2019;
Weber et al., 2020), and the median of means (Levine & Feizi, 2020).
However, they are either inefﬁcient or cannot defend against backdoor attacks with a large ratio of corrupted data.
Some of the above methods also hinge on a clean set of validation data, while in practice, it is unlikely to guarantee the existence of clean validation data, since validation data is usually a subset of training data.
To the best of our knowledge, there is no existing backdoor defense algorithm that is motivated from the label corruption perspective.
P RELIMINARIES 3. L EARNING WITH N OISY L ABELS
In this section, we review some representative approaches for defending noisy-labels.
Although the initial forms of these approaches can be vulnerable to backdoor attacks, we show later in the next section that our proposed meta-algorithm can empower them to effectively confront backdoor attacks.
Speciﬁcally, we look into two types of nosiy-label defending approaches: ﬁltering-based approaches and consistency-based approaches.
The ﬁltering-based approach is one of the most effective strategies for defending noisy labels, which works by selecting or weighting the training samples based on indicators such as sample losses (Jiang et al., 2017; Han et al., 2018; Jiang et al., 2020) or gradient norms of the loss-layer (Liu et al., 2021).
For instance, Jiang et al.
(2017) proposed to assign higher probabilities to samples with lower losses to be selected for model training.
Consistency-based approach modiﬁes data labels during model training.
Speciﬁcally, the Bootstrap approach (Reed et al., 2014) encourages model predictions to be consistent between iterations, by modifying the labels as a linear combination of the observed labels and previous predictions.
In this paper, we leverage two ﬁltering-based noisy label algorithms, i.e. the Self-Paced Learning ( SPL ) (Jiang et al., 2017; Kumar et al., 2010) and Provable Robust Learning ( PRL ) (Liu et al., 2021), and one consistency-based algorithm, i.e. the Bootstrap (Reed et al., 2014), to investigate the efﬁcacy of the proposed meta algorithm.
Strong empirical results in Section 5 on those input algorithms suggest that our meta framework is readily to beneﬁt other robust noisy-label algorithms.
We brieﬂy summarize the main idea of the above algorithms in table 1.
[Table]
3. P ROBLEM S ETTING OF B ACKDOOR A TTACKS
In this paper, we follow a standard setting for backdoor attacks and assume that there is an adversary that tries to perform the backdoor attack.
Firstly, the adversary can choose up to (cid:15) fraction of clean labels Y ∈ R n × q and modify them to arbitrary valid numbers to form the corrupted labels
Y b ∈ R (cid:98) n(cid:15) (cid:99)× q .
Let Y r represent the remaining untouched labels, then the ﬁnal training labels can be denoted as Y (cid:15) =
[ Y b , Y r ] .
Accordingly, the corresponding original feature are denoted as
X =
[Equation]
∈
[Equation]
X
[Equation]
∈
[Equation]
) × d ] .
The adversary can design a trigger t ∈ R d to form the corrupted feature set X b ∈ R (cid:98) n(cid:15) (cid:99)× d such that for any b i in X b , o i in X o , it satisﬁes b i
=
o i + t .
Finally, the training feature will be X (cid:15) =
[ X b ∈ R (cid:98) n(cid:15) (cid:99)× d , X r ∈ R ( n −(cid:98)
n(cid:15) (cid:99) )
× d ] .
Without ambiguity,we also denote T =
[ t , t , ..., t ] ∈ R (cid:98) n(cid:15)
(cid:99)× d , so that X
o + T =
X b .
Before analyzing the algorithm, we make following assumptions about the adversary attack:
Assumption 1 (Bounded Corruption Ratio) .
The overall corruption ratio and the corruption ratio in each class is bounded.
Speciﬁcally,
[Equation]
Assumption 2 (Small Trigger) .
Any backdoor trigger satisﬁes (cid:107) t (cid:107) p ≤ τ , which subtly alters the data within a small radius- τ ball without changing its ground-truth label.
We also assume that there exists at least one black-box robust algorithm
A which can defend any noisy label attacks so long as the noisy-label ratio is bounded by (cid:15) .
Note that the above assumption is mild, since a variety of existing algorithm can handle noisy labels attacks with a large corruption rate ( e.g. 45%)
(Jiang et al., 2017; Han et al., 2018;
Reed et al., 2014; Liu et al., 2021).
M ETHODOLOGY
Given an (cid:15) -backdoor attacked dataset ( X (cid:15) , Y (cid:15) ) , a clean distribution p ∗ :=
( X , Y ) , and a loss function L , we aim to answer the following questions: 1) can we learn a network function f that minimizes the generalization error under the corrupted distribution, i.e. E ( x,
[Equation]
? 2) can the learned f also minimize the generalization error under the ground-truth, clean distribution, i.e. E ( x,y ) ∼ p ∗
[ L ( f ( x ) , y )] ?
Next, we elaborate our meta-approach for defending against backdoor attacks, which answers the above two questions afﬁrmatively.
4. A BLACK - BOX ROBUST ALGORITHM AGAINST NOISY LABELS
The ultimate goal for defending against backdoor attacks is to learn a network function f to minimize its risk given corrupted feature inputs:
[Equation] [Equation]
However, Equation 1 is not directly optimizable due to two-fold challenges: 1) the corrupted inputs
X (cid:15) with an unknown trigger t , and 2) the corrupted labels Y (cid:15) .
That is, neither the clean inputs or ground-truth labels are available.
As such, we turn to an approachable objective that optimizes the worst-case of Equation 1:
[Equation]
Since the trigger satisﬁes (cid:107) t (cid:107) p ≤ τ , it is easy to see that Equation 2 minimizes an upper-bound of the ground-truth loss, in that:
[Equation]
To this end, directly optimizing the surrogate objective in Equation 2 is still intractable, since we do not have access to clean X and Y , which are involved in the inner maximization loop.
To tackle this challenge, we will ﬁrst assume that the clean label Y is available, and then relax this prerequisite by using any learning algorithms that are robust against noisy labels.
Speciﬁcally, assume that φ w =
L ◦ f has a Lipschitz constant L w.r.t x , we further obtain a new upper bound (see Appendix for derivation details):
[Equation]
which draws a principled connection between the risks when using corrupted data and clean data:
[Equation]
) where the ﬁrst term on the RHS of Equation 4 involves optimization on the corrupted features X (cid:15) and clean labels Y , while the second term on the RHS requires minimizing the Lipschitz constant
L w.r.t x .
Recall that minimizing the maximum gradient norm is equivalent to minimizing the Lipschitz constant (Terj´ek, 2019).
Therefore, optimizing the ﬁrst term naturally regulates the maximum change of the loss function within a small ball, which hence constrains the magnitude of the gradient and has negligible effects on the Lipschitz regularization.
The relationship between Lipschitz regularization and adversarial training has also been well discussed by literatures (Terj´ek, 2019; Miyato et al., 2018b).
We defer more discussion along this line to Appendix.
Equation 4 indicates that if the target labels are not corrupted, and the learned function has a small
Lipschitz constant, learning with corrupted features is feasible to achieve low risks.
Up to now, the remaining challenge of optimizing the surrogate objective in Equation 4 is the inaccessible clean label set Y .
Fortunately, a variety of algorithms are at hand for handling noisy labels during learning (Jiang et al., 2017;
Liu et al., 2021; Kumar et al., 2010), which we can directly apply to our minimax optimization scheme.
Speciﬁcally, for the outer minimization, one can have:
[Equation]
For the outer minimization, we can perform the noisy-label update for the above optimization objective.
For instance, Given the mini-batch M x , M y with batch size m , if we use SPL to perform the update, we can get the top (1 − (cid:15) ) m data with a small risk L ( f ( x + c ) , y ) to perform one-step gradient descent.
If we use the PRL to perform the update, assume L is the cross-entropy loss, one can get the top (1 − (cid:15) ) m data with small loss-layer gradient norm (cid:107)
f ( x + c ) − y (cid:107) to perform one-step gradient descent.
If we use the bootstrap, we can directly add a bootstrap regularization to update the above objective.
In the meantime, it is non-trivial to directly solve the inner maximization, since adversarial learning c in Equation 4 still faces the threat of noisy labels.
To tackle this issue, we can leverage the same robust noisy label algorithm.
Speciﬁcally, we ﬁrst approximate the inner optimization using the ﬁrst-order Tyler expansion:
[Equation]
Above optimization is a linear programming problem.
With the l ∞ norm ball constraint on the perturbation, the above update can be reduced to the fast gradient sign method (FGSM).
Given a minibatch M x , M y with batchsize m , we can have the closed-form solution as the following:
[Equation]
To relax the prerequisite of a clean label set y in Equation 5, we will use a noisy-label algorithm to perform the update.
For instance, if we use a loss-ﬁltering based algorithm ( e.g. SPL), then for each mini-batch, only the top (1 − (cid:15) ) m data with small L ( f ( x ) , y ) would be included in the update.
If we adopt a gradient-based ﬁltering algorithm ( e.g. PRL), given that L is the cross-entropy loss, then only top (1 − (cid:15) ) m data with small (cid:107)
f ( x ) − y (cid:107) will be included.
The outside clipping ensures that the feature value of the corrupted image is in the valid range.
Based on the above building blocks, we now introduce our algorithm in Algorithm 1, a meta defense scheme that are robust against backdoor attacks, given arbitrary noisy-label robust algorithm A as an input.
The algorithm is illustrated in
Figure 1.
[Figure] [Figure]
4. T HEORETICAL J USTIFICATION
Our ultimate goal is to learn w that achieves a small risk E x , y ∼ p ∗ φ w ( x + t , y ) .
To study the generalization performance on the ground-truth distribution p ∗ , we ﬁrst deﬁne the following risks:
[Equation]
and ˜ R
[Equation]
n
[Equation]
Y (cid:15) φ w ( x + c , y ) .
Next, we focus on the gap between R t and R empc .
Theorem 1.
Let ˜ R empc , R empc , R t , (cid:15), τ deﬁned as above.
Assume that the prior distribution of the network parameter w is N (0 , σ ) , and the posterior distribution of parameter is N ( w , σ ) which is learned from the training data.
Let k be the number of parameters, and n be the sample size.
If the objective function φ w =
L ◦ f is L φ -Lipschitz smooth, then with probability at least 1- δ , one can derive:
[Equation]
We hereby present the skeleton of the proof and defer more details to Appendix.
First, we decompose the error into two components: 1) the generalization gap on the triggered data, and 2) the difference of performance loss between the trigger t and worst case perturbation c : R t − R empc =
( R t −
R empt )
+ ( R empt − R empc ) .
The ﬁrst components is
[Equation]
n −
[Equation]
, which is derived by following the PAC-Bayes framework (Foret et al., 2020).
For the second term, the gap is introduced by two sources.
The ﬁrst source is the difference between c and t , and the second is from the difference between X and X (cid:15) .
Since the objective is L φ Lipschitz, and (cid:107) t − c (cid:107) ≤ 2 τ according to our constraint to the adversary, it is easy to upper bound the error as 2 τL φ .
In the meantime, there is (cid:15) -fraction of difference between X and X (cid:15) , which is bounded by (cid:107) t (cid:107) < τ and leads to the other difference term
L φ (cid:15)τ .
Theorem 1 presents an upper-bound of the gap R t − R empc .
The ﬁrst term in Equation 6 can be minimized by using a noisy label algorithm.
The second term, which is the error induced by the adversarial trigger, is jointly constrained by the Lipschitz constant L φ , perturbation limit τ , and the corruption ratio (cid:15) .
We can regularize the L φ whereas the τ and (cid:15) are controlled by the unknown adversary.
Note that existing literature has also shown that adversarial training plays a resemblant role as the Lipschitz regularization.
The last term, i.e. the normal generalization error on the clean data, is difﬁcult to minimize directly.
The bound in Theorem 1 emphasizes the importance of involving both the noisy label algorithm and the adversarial training.
The noisy label algorithm can reduce the R empc while the adversarial training minimizes the Lipschitz constant L φ to optimize the second term.
E XPERIMENT
In this section, we perform the empirical study.
In our experiment, we perform experiment on
CIFAR10 and CIFAR100 benchmark data.
We use ResNet-32 (He et al., 2016) as the backbone network structure for all experiment baseline.
The initial learning rate of all methods are set to be 3e-4, and we use AdamW (Loshchilov & Hutter, 2017) as the optimizer for all methods.
The evaluation metric is the top-1 accuracy for both clean testing data and testing data with backdoor trigger.
For the backdoor data poisoning attack, we use simple badnet attack (Gu et al., 2017) and gaussian blending attack (Chen et al., 2017) since these two attacks do not require any information about the model or training procedure (Pang et al., 2020).
• badnet patch attack : trigger is a 3 × 3 black-white checkerboard and it is added to the right bottom corner of the image.
• blending attack : trigger is a ﬁxed Gaussian noise which has the same dimension as the image.
The corrupted image generated by x (cid:15)i = (1 − α ) x i + α t .
In our experiment, we set the α as 0.1.
The poisoned sample can be found at Appendix.
We deploy the multi-target backdoor attack in this paper.
Our poisoning approach is as following: we ﬁrst systematically ﬂip the label to perform the label-ﬂipping attack .
Then, for those being attacked samples, we add the triggers on their feature.
Without adding the trigger, the problem would reduce to the noisy label problem.
and we use the following two evaluation metric.
• top-1 clean accuracy: the top-1 accuracy calculated on trigger-less testing data • top-1 poison accuracy: the top-1 accuracy calculated by the model prediction on poisoned testing data and ground truth clean label.
The ﬁrst metric describes how model performs on benign data while the second metric describes how model performs on triggered data.
We varies our training data poisoning rate as [15% , 25% , 35% , 45%] to investigate how our algorithm performs against different corruption ratio.
All methods are trained 100 epochs, Also, in this paper, we assume there is no clean validation data available .
Thus, it is difﬁcult to perform early stopping or decide which epoch result should be used.
Thus, we report the averaged accuracy across last 10 epochs for every methods.
We investigate three noisy label algorithms by comparing the performance of the original method and reinforced method.
Speciﬁcally, we choose SPL, PRL, and Bootstrap as our original noisy label algorithm and the corresponding reinforced algorithm with adversarial training SPL-AT, PRL-AT,
Bootstrap-AT.
We also compared our method against adversarial training only (AT), which only uses the adversarial training without using any noisy label algorithm.
To show the success of the attack, we also includes the standard training results.
5. H OW ROBUST LEARNING AGAINST NOISY L ABEL PERFORMS ON BACKDOOR DATA
In this section, we aim to answer the ﬁrst question: how does noisy label defense algorithm performs against backdoor attack?
To answer above question, We evaluate PRL, SPL, and Bootstrap on CIFAR10 and CIFAR dataset.
The results can be found in table 2 and table 3.
As we can see in the table, the standard training performs well on the benign testing data (i.e. high clean accuracy) while its performance on triggered data is very bad (i.e. low poison accuracy), which indicates the effectiveness of the backdoor attack.
As for the three noisy label algorithm, we found that although Bootstrap, SPL, and PRL all performs well for the benign testing data, they all fail when defending the backdoor attack data especially with large corruption rate, which illustrates that noisy label algorithm cannot defend the backdoor attack especially when corruption ratio is large.
5. H OW ADVERSARIAL TRAINING IMPROVES THE NOISY LABEL ALGORITHM
To investigate whether adversarial training could improve the robustness of existing noisy label algorithm against backdoor attack, we performed experiment on SPL-AT, PRL-AT and Bootstrap-AT to see how they performs on both clean test data and triggered test data.
The results can be found in table 2 and table 3.
As we can see that by adding the adversarial training, the performance on triggered data is largely improved (i.e. the poison accuracy signiﬁcantly improves).
Also, we noticed that this improvement pattern is hold for all three noisy label algorithm, which indicates the effectiveness of the proposed method on improving robustness against backdoor attack.
Also, compared to adversarial training only (AT), adding noisy label does improve the performance.
Especially for the PRL algorithm, we see the PRL-AT achieves better performance than AT.
Also, we found that compared to consistency based noisy label algorithm, ﬁltering based algorithm are more easier to be boosted by adversarial training.
The potential reason behind this could be that ﬁltering based method is more efﬁcient against noisy label algorithm Han et al.
; Jiang et al.
; Liu et al.
[Figure]
[Table]
5. A BLATION S TUDY
In this section, we aim to explore more about the proposed framework.
Since our algorithm use the noisy-label solver for both inner and outer optimization.
A interesting question to ask is that whether both inner and outer noisy-label solver plays an important role in defense the backdoor attack.
Thus, we have two variants.
One is we only use noisy label algorithm to update the model for outer minimization and another one is we only use the noisy label algorithm to update the model for inner maximization.
The results can be found at table 4 and table 5.
As we could see in these two tables, using noisy label algorithm to perform the inner maximization is more important compared to using noisy label algorithm to perform out minimization.
C ONCLUSION
In this paper, we investigate the connection between label ﬂipping attack and backdoor data poisoning attack.
We show that although robust algorithm against label ﬂipping attack cannot defend the
[Table] [Table]
[Table]
backdoor data poisoning attack, adding the adversarial training on existing algorithm could largely improve the robustness against backdoor attack.
Both theoretical and empirical analysis show the effectiveness of our proposed meta algorithm.
A PPENDIX
In this section, we provided proof of theorem and more discussion.
7. P ROOF OF E QUATION
[Equation]
let G denote the initially clean sample set (i.e. ( X , Y )), and B the corrupted sample set ( i.e. the training set corrupted with a trigger whereas the labels are untouched).
Let R denote the clean sample set which is replaced by the adversary (i.e. R is the subset of G , and is replaced by B , i.e.
G (cid:48)
= G \ R ∪ B =
( X (cid:15) , Y )), and let φ w denote the function L ◦ f .
One can decompose the inner part of our mini-max objective in Equation 2 as the following,
[Equation]
7. P ROOF OF THEOREM
Theorem.
Let ˜ R empc , R empc , R t , (cid:15) [Equation] τ , is deﬁned as above.
Assume the prior distribution of the network parameter w is N (0 , σ )
[Equation]
and the posterior distribution of parameter is N ( w , σ ) is the posterior parameter distribution, where w is learned according to training data.
Let k to be the number of parameters, n to be the sample size, assume the objective function φ w =
L ◦ f is L φlipschitz smooth, then, with probability at least 1- δ , we have: [Equation] n
[Equation]
Proof: we ﬁrst decompose the gap as following
[Equation]
[Equation]
Now
[Equation]
we bound the second term.
Note the second term is a typical gap term between empirical loss and generalization loss, and there are many approaches to bound this term like VC dimension.
Since we aimed to focus the deep neural network, we follow the PAC-Bayes framework McAllester (1999) to analyze the generalization bound.
Speciﬁcally, we use results from Foret et al.
(2020), which gives
[Equation]
under the assumption of gaussian prior and posterior.
The proof for this can be found in the appendix of Foret et al.
(2020) (i.e. equation 13 on the paper).
7. D ISCUSSION ABOUT LIPSCHITZ REGULARIZATION AND ADVERSARIAL TRAINING
As we could see from the above theorem that a small lipschitz constant could bring robustness against backdoor attack.
In this section, we provided the reason why we claim adversarial training helps lipschitz regularization.
The deﬁnition of lipschitz function is (cid:107) f
( x ) − f ( y ) (cid:107) ≤ L (cid:107) x − y (cid:107) , ∀ x , y .
Since the lipschitz constant showes in the upper bound of the error, we would like to get the minimum lipschitz constant to tighten the bound.
Follow (Terj ´ ek, 2019), the minimum lipschitz constant can be expressed as:
[Equation]
rewrite y as x + c , we would get
[Equation]
Minimize above objective respect to function f reduces to the adversarial learning.
[Equation]
If we treat the denominator as a constant, then this is exactly the same as our minimax objective.
More details can be found in (Terj´ek, 2019).
7. S UPPLEMENTARY E XPERIMENT R ESULTS
We provided the experiment hyperparameters, and supplementary results for the experiment.
7.4. E XPERIMENT H YPERPARAMETERS
We list the details of experiment in this section.
All the methods use resnet-32 as the backbone network.
AdamW is used as the opimitzer for all methods.
The perturbation limit τ is set to be 0.
for all methods requiring τ .
All methods are repeated for three different random seeds to calculate the standard deviation.
The trigger for badnet attack and blending attack can be found in ﬁgure 2.
[Figure]
[Figure]
E XPERIMENT ON MNIST
We found interesting results on MNIST.
In MNIST, we found adversarial training itself sometimes gives robustness to the backdoor attack.
We hypothesis that this is because the MNIST is potentially a easier task than CIFAR.
In here, we show the performance of adversarial training and PRL-AT on
MNIST.
The results can be found at table 6.
[Table]
[Figure]
As we can see for the MNIST, especially for the blend attack, the poison accuracy for adversarial training does show good performance with a large standard deviation.
This is because that some random seeds works while some random seed failed.
We hypothesis that this is because MNIST [Table] dataset has almost binary feature value.
When adding a small gaussian noise on feature x, the label ﬂipping attack cannot change the decision boundary much.
That is why the noisy label algorithm seems is not as important as the noisy label algorithm in CIFAR dataset.
We plot a two dimensional toy example in ﬁgure 3 to illustrate label ﬂipping attack on continuous features and binary features.
As we can see in the ﬁgure that for the binary value feature, the label ﬂipping attack is not easy to change the decision boundary too much while it can easily change the decision boundary in the continuous feature value scenario.
However, this is a very rough conjecture for the reason why in
MNIST, adversarial training sometimes works.
We leave the investigation of this phenomenon in the future work.
7. C OMPARISON AGAINST OTHER BACKDOOR DEFENSE ALGORITHM
In this section, we show further results against other backdoor defense algorithms.
There are many existing backdoor defense algorithms.
However, we found that a large fraction of those algorithms are either designed for single target attack (Liu et al., 2018) or requires clean data (Liu et al., 2018;
Wang et al., 2019; Li et al., 2021).
To investigate how our framework performs compared with the previous study, We compare against the following two baselines in a similar setting:
• spectral signature (Tran et al., 2018): ﬁltering data by examining the score of projecting to singular vector.
• ﬁne-pruning (Liu et al., 2018):
prune the model by deleting non-activated neurons.
Note this method uses 5% clean training data.
The results are in the table 7.
As we can see, PRL-AT has higher poisoned accuracy against Spectral
Signature and Fine-Pruning on most settings while the clean accuracy is still high, which indicates the effectiveness of our algorithm.
With high corruption ratio, we ﬁnd that the robustness of spectral signature and ﬁne-pruning signiﬁcantly decreases while PRL-AT still gives reasonable poison accuracy.
Our theorem provides a principled way to design new defense algorithms by leveraging more knowledge from noisy label attacks.
In this comparison, we only use PRL as the noisy label algorithm, and potentially, by using a more powerful noisy label algorithm (i.e. combination of multiple methods), it is possible to get higher poisoning accuracy.
7. S ENSITIVITY ANALYSIS OF THE (cid:15)
One interesting question to ask is how our algorithm performs without knowing the corruption ratio.
In this section, we provided the worst-case result, in which we ﬁx (cid:15) = 0 .
5 to perform the noisy label algorithm.
When the ground truth corruption ratio is higher than a half, it is impossible to learn any meaningful classier.
We test the PRL-AT in the CIFAR10 on both badnet and blending attacks.
The results are in table 8.
As we can see, our algorithm provides robustness even use highlyoverestimated estimated (cid:15) compared with the standard training results in table 2, which suggests our algorithm is not sensitive to the hyperparameter (cid:15) .
7. D ISCUSSION ABOUT N OISY L ABEL A LGORITHM
One key question to ask for our framework is how to choose the noisy label algorithm.
In terms of practice, we found PRL gives consistent robustness against both badnet and blending attacks on
[Table]
different settings.
This might be because that PRL is designed for agnostic corrupted supervision, which is suitable for a variety of types of label noise attacks.
From a theoretical perspective, analyzing how different noisy label algorithm can minimize the ﬁrst term of RHS in equation 6 depends on which noisy label algorithm to use.
Here we give a rough analyze of PRL.
PRL guarantees converging to the (cid:15) -approximated stationary point, where (cid:15) is the corrupted ratio.
Formally, we have the following corollary:
Corollary 1 (Convergence of PRL to clean objective (Liu et al., 2021)) .
Assuming the maximum clean gradient before loss layer has bounded operator norm: (cid:107) W (cid:107) op ≤ C , applying PRL to any (cid:15)
-fraction supervision corrupted data, yields min t ∈
[ T ] E ( (cid:107)∇ φ ( w t ) (cid:107) )
=
O ( (cid:15) √ q ) for large enough
T , where q is the dimension of the supervision.
More details can be found on the in Liu et al.
According to above corollary, let w PRL is the solution get by PRL algorithm, we can have (cid:107)∇ w PRL R empc (cid:107)
=
O ( (cid:15) )
(i.e. assume q is small).
With
Polyak-Lojasiewicz (PL) condition with some constant µ such that 1 2 (cid:107)∇ f ( x ) (cid:107) ≥ µ ( f ( x ) − f ∗ ) holds, we have µ ( R empc − R emp ∗ c ) ≤ 1 2 (cid:107)∇ w PRL R empc (cid:107) =
O ( (cid:15) ) .
For a highly-overparameterized deep neural network, the global optima R emp ∗ c is usually 0.
Thus, we can conclude that with PL condition, using PRL as the noisy label algorithm in our framework can guarantee R empc can be minimized to the order 1 µ O ( (cid:15) ) .
This yields the following proposition:
Proposition 1.
Let R t , (cid:15), τ , is deﬁned as above.
Assume the prior distribution of the network parameter w is N (0 , σ ) , and the posterior distribution of parameter is N ( w , σ ) is the posterior parameter distribution, where w is learned according to training data.
Let k to be the number of parameters, n to be the sample size, assume the objective function φ w =
L◦ f is L φ -lipschitz smooth and satisfying the PL condition, which is 1 2 (cid:107)∇ φ w (cid:107) ≥ µ ( φ w − φ w ∗ ) .
then, with the assumption of bounded operator norm of gradient before loss layer, we have with probability at least 1- δ , by applying PRL-AT, we have:
[Equation]
In general, considering φ is a deep neural network, we believe the ﬁrst term is difﬁcult to analyze without further assumption (i.e. PL condition).
Nevertheless, empirical study shows that many noisy label algorithms can effectively minimize the ﬁrst term loss, which motivates our method to treat those algorithms as a black-box algorithm.