DLP: Data-Driven Label-Poisoning Backdoor Attack
Authors: Xun Xian
Xuan Bi
Mingyi Hong
Jie Ding
Venue: Submitted to ICLR 2023 (2023)
Keywords: Backdoor learning
End-to-end learning
Clean-sample attack

Abstract: Backdoor attacks, which aim to disrupt or paralyze classifiers on specific tasks, are becoming an emerging concern in several learning scenarios, e.g., Machine Learning as a Service (MLaaS). Various backdoor attacks have been introduced in the literature, including perturbation-based methods, which modify a subset of training data; and clean-sample methods, which relabel only a proportion of training samples. Indeed, clean-sample  attacks can be particularly stealthy since they never require modifying the samples at the training and test stages. However, the state-of-the-art clean-sample attack of relabelling training data based on their semantic meanings could be ineffective and inefficient in test performances due to heuristic selections of semantic patterns. In this work, we introduce a new type of clean-sample backdoor attack, named as DLP backdoor attack, allowing attackers to backdoor effectively, as measured by test performances, for an arbitrary backdoor sample size. The critical component of DLP is a data-driven backdoor scoring mechanism embedding in a multi-task formulation, which enables attackers to simultaneously perform well on the normal learning tasks and the backdoor tasks. Systematic empirical evaluations show the superior performance of the proposed DLP to state-of-the-art clean-sample attacks.

Under review as a conference paper at ICLR 2023
DLP: DATA-DRIVEN LABEL-POISONING BACKDOOR ATTACK
Anonymous authors Paper under double-bling review
ABSTRACT
Backdoor attacks, which aim to disrupt paralyze classifiers specific tasks, are becoming an emerging concern in several learning scenarios, e.g., Machine Learning as a Service (MLaaS). Various backdoor attacks have been introduced in the literature, including perturbation-bart methods, which modify a subset of training data; and sean-sample methods, which relabel only proportion of training samples. Indeed, elean-sample attacks can be particularly stealthy since they never require modifying the samples at the training and test stages. However, the state-of-ther ofrtte-art clean-sample attack of relabelling training data based on their semantic meanings could be ineffective and inefficient in test performances due to heuristic selections of semantic patterns. In this work, introduce new type of clean-sample backdoor attack, named as DLP backdoor attack, allowing attackers to backdoor effectively, measured by test performances, for an arbitrary backdoor sample size. The critical component of DLP is a data-driven backdoor scoring mechanism embedding in multi-task formulation, which enables attackers to simultaneously perform well on the normal learning tasks and the backdoor tasks. Systematic empirical evaluations show the superior performance of the proposed DLP to state-of- sate-of-the-ar the-art clean-sampl attacks.
INTRODUCTION
The backdoor attack has been an emerging concern in several deep learning applications owing to their broad applicability and potentially dire consequences (Li et al., 2020). In high level, backdoor attack implants triggers into a learning model to achieve two goals simultaneously: (1) to lead the backdoored model to behave maliciously attacker- attacker-peciffed tasks with an active backdoor trigger, c.g., a camouflage patch as demonstrated in Fig. 1, and (2) to ensure the backdoored model functions normally for tasks without backdoor trigger.
One popular framework is the perturbation- based backdoor attack (PBA) (Gu et al., 2017; Chen et al., 2017: Turner et al., 2019; Zhao et al., 2020; Doan et al., 2021a;b). In PBA, during the training stage, an attacker first creates poisoned dataset by appending a set of backdoored data (with backdoor triggers), the clean data, and then trains model based on the poisoned dataset. In the test stage, the attacker launches backdoor attacks by adding the same backdoor trigger to the clean test data.
The requirement of accessing and modifying data, including both features and labels, during the training and test stages in PBA could be unrealistic under several applications. For example, in machine learning as a service (MLaaS) (Ribeiro al., 2015), is difficult for attackers to access users' input queries in the test phase. Consequently, a new type of attack, namely clcan-sample backdoor attacks (Lin al., 2020: Bagdasaryan et al., 2020), has attracted significant practical interest. In clean-sample backdoor attacks, the attacker changes labels instead of features in the training stage only as illustrated Fig. and summarized in Table The state-of-the-art (SOTA) clean-sample attack, known the semantic backdoor attack (Bagdasaryan al., 2020), first looks for images with particular semantic meaning, then relabels all the training images with the semantic meaning, e.g., green car in the CIFAR 10 dataset, to attacker-specified labels. Finally, the attacker trains classifier based on the modified data. In the inference stage, no further operations are needed the attacker.
It was pointed out that attacks since they do not modify features of input data (Li et al., 2020). Nevertheless, the SOTA clean-sample method, namely the semantic backdoor attack, is possibly limited in terms of backdoor
backdoor attacks more malicious than perturbation based

---

Under review as a conference paper at ICLR 2023
Triining Stage
Training Siage
Label: Label:
Label:
Label:
Label:
Label:
Label:
Label:
Go-straight sign Go-straight sign
Right-turn sign Left-turn sign
Go-straight sign Go-straight sigw
Right-turn sign
sign
STOP
STOP
STOP
Poisoned Training Data
Training
Poisomed Training Data
Normal Training
Backdoor trigger Stop Sign
classiffer
Backdoor trigger
trained both clean and polsoned
trained on
both clean and poisoued data
Test Stage
Test
Test input
Test inpuit
Tost input
Test input
Tost input
Tost input
Test input
Test input
STOP
STOP
Go-strwight sign
Go-strwight sign
U-cum sign
Right-tum sign
Go-straight sign
foo-struight sign
U-tumn sign
Right-tum sign
(b)
Figure 1: Illustrations of (a) clean-sample backdoor attacks and (b) perturbation- backdoor attacks in autonomous driving systems. In clean-sample attacks, the attacker relabels images of stop-sign go-straight sign. In perturbation-baat attacks, the attacker adds camouflage patches to stop-sign images and then relabels the perturbed images as go-straight sign.
selection for the following reasons. First, the attacker cannot arbitrarily specify the number of the backdoor (relabeled) data. Instead, the number should equal the size of a whole category of training semantic meaning. The reason is that only by relabelling the whole category of
data with the training data with the same semantic meaning can the attacker distinguish between normal data and backdoor data through their semantic meanings. Such restriction could lead to failures of attacks under certain data scenarios. For example, with low-resolution data, the attacker may need to relabel significant proportion of normal training data, which will inevitably destroy the test performance of the backdoored classifier on normal data.
Second, the criterion for selecting semantic patterns are heuristic and may vary drastically among different attackers. As a result, test performances will change significantly. To find the best backdoor criterion, the attacker will try every semantic combination with brute force methods. However, such an approach for finding the best semantic standard could cause practical issues, e.g., computational infeasibility. For instance, there are roughly 20000 categories in the ImageNet (Krizhevsky et al., 2012) for semantic backdoor attack.
1067 ways to select 20 categories out of total of
In light of the aforementioned issues, we propose a new type of clean-sample attack named Data- Driven Label-Poisoning (DLP) backdoor attack. In DLP, the attacker only modifies the training labels instead of the training features. In contrast to heuristic selections of backdoor patterns in current semantic attacks, in DLP, the attacker selects backdoor samples via scoring mechanism. Each training sample will be assigned a value by the scoring mechanism to reflect its backdoor effectiveness, measured by the test performances number of backdoor data, the attacker is able to select the most effective ones based on the scores. To the best of the authors' knowledge, we the first to mathematically apply the ideas of scoring mechanisms to formulate backdoor problem and offer corresponding thcoretical justifications.
both the normal and backdoor tasks. Consequently, for any
Our contributions in this work are summarized below. First, we introduce new type of backdoor attack, known as DLP, which only modifies the labels of the training data and hence is more malicious than existing methods. In addition, the proposed DLP backdoor attack enables attackers to select backdoor samples at any level of the backdoor budget (namely the number of backdoor sample) effectively. Second, we present a formulation for the proposed DI.P backdoor attack along with theoretical analysis and algorithms. In particular, attack that simultancously satisties the two backdoor goals under reasonable conditions. Third, we present extensive empirical study over benchmark datasets to illustrate the effectiveness of the proposed framework. We find that experimental results align with existing conjectures and provide insights efficiently designing backdoor attacks.
show that the proposed DLP leads successful
The rest of the paper is organized as follows. First, an overview of the related literature is given in Section 2. Then, we formally introduce the DI.P in Section 3. Next, present implementation of
2

---

Under review as a conference paper at ICLR 2023
Table Summary of attackers' capability in the proposed DLP, perturbation- backdoor attacks (abbreviated as PBA), and semantic backdoor attacks (abbreviated as SBA). The first (second) column specifies if attackers modify training (test) features. The third column indicates whether attackers are allowed to choose any number of backdoor data.
Attack
Preserve training features
Preserve test features
Select any number
PBA
SBA
DIP (Our method)
the proposed DIP and corresponding theoretical analysis in Section 4. extensive experimental study of the proposed method on benchmark datasets is presented in Section 5. Finally, we conclude the paper in Section 6.
2 RELATED WORK
Data poisoning/Label flipping attacks Data poisoning attacks, including manipulate training features (Biggio et al., 2012; Koh & Liang. 2017; Jagielski et al., 2018: 2021; Weber et al., 2020) and flipping labels (Paudice al., 2018), aim to achieve attackers' adversarial goals. Classical data poisoning attacks are untargeted, which aim to deteriorate the overall prediction performances of the classifier (Gao et al., 2020). For example, early backdoor attacks contaminated training data to decrease the overall test accuracy of the trained supported vector machines (Biggio et al., 2012).
Although several prevailing backdoor attacks are realized via data poisoning, there are several differences between massoning-basedd backdoor attacks and traditional data poisoning attacks. Backdoor attacks are targeted. A backdoored model only misbehaves maliciously on specified tasks in the presence of backdoor triggers while retaining the overall test accuracy of its primary tasks. In addition, typical data poisoning attack requires poisoning a significant proportion of normal training data degrade the normal test performance. In contrast, backdoor attack often only allows modifications on small subset of training data to maintain good performances on the normal tasks (Li al., 2020).
backdoor attacks Unlike perturbation- based backdoor attacks (PBA) which modify both the features and labels during the training and the test stage, clean-sample backdoor attacks (Bag- dasaryan et al., 2020) (SBA) only poison the labels in the training stage. On the one hand, SBA do not require accessing and changing the features both training and test stages and therefore are practically stealthier than PBA. On the other hand, SBA only poison the labels of clean data and hence are less powerful than PBA. For example, clean-sample backdoor attacks typically can not simultancously achieve perfect accuracy both clean and backdoored test data.
Multi-task learning Multi-task learning (MTL) (Baxter, 2000; Li et al., 2014; Xue et al., 2007: Ruder, 2017) refers to learning several different tasks via single model. MTL is often implemented in hard and soft parameter sharing. For ooft-paramtter sharing, all the parameters are private and specific to different tasks. These parameters typically jointly constrained by Bayesian priors (Xue et al., 2007) or a joint dictionary (Argyriou et al., 2007; Ruder, 2017). In hard hard-parameter sharing, each task shares some parameters and has task-specific parameters. Our work falls under the umbrella of hard-parameter sharing. Both the normal and backdoor tasks will share the parameter of the learning model, but the backdoor task owns the selection parameter privately. The proposed DLP can be considered particular type of MTL because there are two goals that are learned jointly with hard parameter sharing.
3 DLP
We consider a classification scenario, where v E X X C features, label, and backdoor target label, respectively. We denote {(z,, y as i.d training data from distribution Px,y. Let fa X' + y denote the classifier parameterized by B E R". We let f((..) denote the loss function that evaluates the discrepancy between the true label and predicted label. We use ||. || ||2 and || I/ denote the &,, l2 and loo vector norm respectively.
KJ, V, denote the
3

---

Under review as a conference paper at ICLR 2023
3.1 THREAT MODEL
Attacker's Capacities We consider a clean-sample backdoor attack scenario where an attacker can only access and modify the training labels, but not the training features. We emphasize that the number of training data will remain the same after poisoning. Besides, the attacker has control over the training process. However, the attacker does not have control over any procedures in the test stage, including modifying data and deploying model on cloud. The discussed scenario can happen in several real-world applications, e.g., "Outsoureing Training" (Gao et al., 2020). Overall speaking, the threat model of our method poses considerably weaker requirements than the perturbation-base backdoor attacks, and therefore is stealthier than the current perturbation- bbasd backdoor attacks.
3.2 PROPOSED METHOO
We first briefly outline the overall process of clean-sample backdoor attacks in the followings. In the training stage, attackers first relabel a subset of clean data (denoted as DB) to attacker-specified target label(s), e.g., relabeling images of 'green cars' as 'frogs'. Next, attackers train a classifier to learn those poisoned data (DB) and the remaining clean data. During the test stage, attackers do not need to modify the test data to launch backdoor attacks, unlike perturbation- attacker needs to patch the test data to launch backdoor attack. The reason is that any test input with the same features as DB, e.g., images of 'green cars', is expected to be automatically predicted as the attacker-specified target label(s) by the backdoored model.
attacks where the
The most challenging part in clean-sample attacks is to effectively and efficiently select the features in DB serve as backdoor triggers. Current methods apply the semantic pattern of green cars as backdoor triggers (Bagdasaryan et al., 2020). But the use of semantic patterns as backdoor triggers could be possibly limited in terms of effectiveness and efficiency, such as a restricted number of backdoor data and ineffective selections of backdoor triggers (see Section 1 for detailed discussions). To solve those issues, we propose a data-driven backdoor selection mechanism that enables attackers to select any number of backdoor data effectively. To be precise. the attacker chooses backdoor data via a scoring map that takes features as input and outputs a score
gw : X - 1],
with W E R2 being the parameter. For the rest of the paper. we adopt the rule that a higher score reflects that a sample is more suitable to be backdoored. Intuitively, one can treat the selection mechanism gw as a soft binary classifier for deciding whether an input should be selected as a backdoor candidate. To select backdoor data at any attacker-seccffedd level, the attacker can first sort the scores of all the samples, namely {9w (x,)) in descending order, and then pick any attacker-specified quantile of data as backdoor samples.
Next, we will elaborate on how to incorporate the proposed scoring method into the training pipeline. Recall that one of attackers' goals is to obtain a high accuracy on backdoor data. To fulfill this goal, the attacker should select a backdoor scoring mechanism gw such that the following empirical backdoor risk is minimized for given data ((ri, y;) model (,, and backdoor label e )::
(W,
Σ 1y * ((o
subject to (C) Σe + G)gw (5,) m and gw (2y) E (0, for j = 1, I,..., n. Note that indicators of 1 (1/ * 7 for i= 1,..., n are included in the constraint. Applying these terms is because we can not backdoor (relabel) a sample to its originally ground-truth label class. For example, it is meaningless to relabel an image of the cat as a cat for backdoor attack.
As it is generally accepted in the backdoor literature, attackers should simultaneously achieve high clean accuracy and backdoor accuracy. To meet such a requirement, we use a multi-task learning formulation with weighted summation to design backdoor attacks. The first task is associated with the normal learning task, and the second task corresponds to the backdoor task. Given normal training data {(,,, yi ns and a backdoor target label J. the DLP backdoor attack is a pair of minimizer
'This is fundamentally different from the case of perturhation- hased attacks where backdoor training data will be injected to the clean training data and therefore the number of total training data will increase.
4

---

Under review as a conference paper at ICLR 2023
(W.B) of the following:
min WEE SERP
(Uo(ss)
X
Σ 5=1
[us + (1)w(1;))
(2)
m
a Σ
subject to (C)
1ys + ()gw(())) = and gw (2y) {0, 13 for j ,...,,
where is the number of backdoor samples, and X is regularizing coefficient.
Remark 1. We emphasize that the number of backdoor samples m should be relatively small compared the total sample size n. The reason that an excessive number of backdoor samples will eventually lead to a trivial classifier for normal test data. Thus, the first goal of backdoor attacks will not be met.
Finally yet importantly, we will now discuss the test pipelines of DLP. Given an attacker-specified number of backdoor data m and a trained score mechanism gw, we first sort the scores of training data (o:) in descending order and set gw (w:)))(m) (Z)) namely the m-th largest score, to be threshold for selecting test backdoor data. For a test input if its score gw(ar) is greater than gw (are (m)) then will be marked as backdoor sample.
CONTINUOUS IMPLEMENTATION AND THEORETICAL RESULTS
4.1 CONTINUOUS IMPLEMENTATION
Directly solving problem (2) is challenging. The main difficulty comes from the fact that (gw (a:; (r.)) } are required to take values in a discrete set, which is not directly compatible with the popular gradient- based optimization framework. To tackle this issue, we first consider an unconstrained version of problem (2) and then propose regularization term enforce the selection mechanisms to fulfill the constraint (C). In particular, we consider
(W) E
1ly * (low(1z) - m)² + Σ (1 j=1
where m is an attacker-specifeed number. straightforward to observe that the proposed term P.. (W) equals zero and only if the constraint (C) is satisfied. In other words, the proposed regularization term will lead a selection mechanism to satisfy the constraint (C) if and only
is precisely minimized. Consequently, we now propose to consider the following continuous and unconstrained minimization problem:
min W
Σ (ffo(z)). v.)
Σ 1l4; + Vlg (2))) m 1=1
A
T Pm (W), n
(3)
where X, are tuning parameters. One can apply classical gradient-based methods to solve the above problem. Nevertheless, classical optimization methods may lead to convergence issues. We include the rigorous descriptions of the issues and algorithms to solve them the appendix.
4.2 THEORETICAL RESULTS
In this section, we show that, under reasonable conditions, the proposed DLP leads to an attack that satisfies the two backdoor goals mentioned in last section. All the proof is included in the supplement due to the page limit. We build theoretical results based on the classical notion of the Rademacher Complexity in classical learning theory. The (empirical) Rademacher complexity of the function class F with respect to probability distribution P over an input space X' for i.i.d. sample { with size is: Rad, (F) = E [sup,er Σn= o (æ:)). where the inner expectation taken over σ { 02. } and they are independent random variables following the Rademarcher distribution, i.e., P (a: = 1) = P (a. -1) 1/2.
Next, we present results on the generalization bounds for both normal and backdoor tasks in the followings. We consider a binary classification problem with label set ) = [1,-1] and the back-
door target label y The test performances evaluated through R(3, W) Ey 1(Y *

---

Under review as a conference paper at ICLR 2023
(1ow(X)((ff(XX)) 7) and R(3) Epx, ((fs,((f))))))))))))))(()))))))) Y) respectively. Denote (W, 3) to be a back- door attack obtained by solving problem (3), Also, for ease of notation, we define two new family of functions namely H₃ {e(Ss(X), 8) 8 RR and Gw.s {1(Y * g (X)((x(()) Y) Rd, W E R²). Additionally, we will need the following assumption regrading the loss function. Assumption 1. The loss function E(.-) is uniformly upper bounded by B.
Theorem 1. Suppose that the assumption I holds. The followings hold with probability at least
1 25: Gap on the normal task: R(3) - R(3) S XB + 4Rad, (Hs) () + 2n 11/BB 1/ 1/6, where Ei=1 ((fs(z)), vi) is the normal classifier;
A = arg min,
Gap the backdoor task: R(W, - B) I 4Rad,, (Gw,s) + 2n 1/2 B(log 10.
2mT /X Bm/nd, where (W, 3) is the optimal backdoor choice, i.e., the minimizier of (1).
The first bound is about the gap in normal test performances between the DLP classifier and the normal classifier. The second bound describes the differences between backdoor test performances of the DLP and the optimal backdoor choice (defined in Section 3.1). The task-priority hyperparameter À appears in both two loss terms. A small À tends to generate backdoor classifier S3 that resemble the normal classifier faz more. In contrast, excessively large À pushes the backdoor classifier fa to behave similarly the optimal backdoor classifier J
Two upper bounds also depend the Rademacher complexity of function classes. If the Rademacher complexity of the function class is bounded, we can obtain vanishing upper bounds by appropriately selecting the hyperparameters. We provide informal result for linear classifiers in the following. The formal statements and the proof are included in the supplement.
Theorem 2. (Informal) Suppose that fa is a linear classifier. Then, under specific assumptions and appropriately chosen hyperparameters, the gap on normal and backdoor tasks converge to zero with high probability as sample size OC.
5 EXPERIMENTAL STUDY
This section systemically evaluates the proposed method on synthetic data and benchmark datasets. Our empirical study shows the effectiveness of the proposed DLP and improved performances compared with state-of toe-the-art (SOTA) methods. Interestingly, we demonstrate that the backdoor samples selected by the DLP are semantically close to those from the target labels, which is aligned with existing conjecture (Bagdasaryan et al., 2020). Finally, we provide several general ideas for designing attacks based empirical observations.
We will use two test measurements throughout this section. The normal accuracy (abbreviated as AccN) is the test accuracy on the normal data, and the backdoor accuracy (abbreviated AccB) represents the test accuracy on the selected backdoor data. The method for selecting backdoor data is described in the last paragraph in Section
5.1 SYNTHETIC DATA
We evaluate the proposed method on synthetic data. For the training data, we generate 500 i.i.d samples from the normal distribution A A( 3). [[2, 0). [0, 2]]) with label 0 and 500 i.i.d samples from the normal distribution A(00 33.. [[2, 0j, J0, 2]) with label respectively. For the test data, we generate 2500 i.i.d samples for each label class. The classifier fa is linear, and the loss function / the logistic loss. Also, the backdoor target is 1, and the number of backdoor sample
50.
Test Performance We summarize test perfor- mances under different À in Table. 2. By setting to 0, the objective is to minimize the normal empirical risk only, and hence the resulted classi- fier is the normal classifier. Similarly, by setting be sufficiently large, e.g., oc, the resulted classifier is the optimal backdoored classifier. For wide range of X, the proposed DLP deliy- ers comparably high accuracy on both normal and backdoor data compared with the normal and backdoored classifier, respectively.
Table 2: Test accuracy under different À
A
0
10
0
AceN
0.962
0.946
0.865
0.637
AccB
0.524
0.971
0.983
0.998

---

Under review as a conference paper at ICLR 2023
Selected backdoor sample We plot the normal training data (in orange dots and blue triangles), the selected training backdoor data (in green triangles), the normal classifier (in solid red line), and the backdoor classi- fier (in gray dash-dot line) with À 1 in Fig. 2. The selected backdoor points stay close to the normal clas- sifier, and the DLP classifier moves down compared to the normal classifier. Such an observation can be interpreted in the following way. By relabelling those points near the decision boundary, the attacker slightly shifts the normal classifier account for those back- doored samples without severely degrading the normal test accuracy. This phenomenon has also been indepen- dently discovered and investigated in the early work of poisoning support vector machines (Biggio et al., 2012) and the active learning literature (Settles, 2009), where the goal is to find the most effective and efficient samples to train classifier.
Normal Classifier CS2D Classifier
Selected Points
Figure 2: An illustration of selected back- door samples under linear classifier with Gaussian data.
5.2 REAL-WORLD DATASETS
Tasks We consider the following tasks: (I) 10-class
classification problem on the Fashion-MNIS" (Xiao et al., 2017) with LeNet (LeCun et al., 2015), (II) 10-class classification problem on the CIFARIO (Krizhevsky et al., 2009) data with a Resnet18 (He et al., 2016), and (III) classification problem on GTSRB (Stallkamp et al., 201 1). The details of the model architectures and the results for Task III are provided the supplement.
Backdoor Setup For task (II), we set the number of backdoor samples m to be 600 (500), and 6000 (5000), which accounts for 1% and 10% of the total training data respectively. For both tasks, the selection mechanism is two-layer neural network. All the hyperparameters are chosen through grid search together with cross-vaie validaaiiiitnnnn.
Single Task Performance For task I, the AccN of the normal classifier is 92.1 %. and the AceB of the optimal backdoored classifier trained only on backdoored data is 94.2%. For task II, the AceN of the normal classifier is 92.1 1%, and the AccB of the optimal backdoored classifier 93.0%.
5.2.1 TEST PERFORMANCE
For every backdoor target label, we summarize the test performances of DLP in Table. 3 for Task I with 600, and for Task II with m 500. We include the results for other choices of in the next subsection. Both the normal and backdoor test accuracy of our DLP are comparably high compared to that of the single-task performance, which reflects the effectiveness of the proposed DLP. In task Waashion-MNISTT as 'Boot'), both the AceN and BocN exceed the test performances of other backdoor labels by great margin. This observation implies that labels of the shoe category are the most effective for backdoor. Such result may be because images of 'Sandal', 'Sneaker,' and 'Boot' are semantically similar.
for backdoor target label 'Sandal', 'Sneaker,' and 'Ankle Boot' (referred
Table 3: Test accuracy (in %) with flipping rate m 1%n
Results on Fashion-
Label
Top
Trouser
Pullover
Dress
Coat
Sandal
Shirt
Sneaker
Bag 89.1
Boot
AccN
91.1
89.2
87.1 92.7
90.3
85.2
90.5
92.7
92.4
89.9
AceB
92.5
93.1
90.4
92.1
89.3
93.4
91.7
92.9
90.5
Results on CIFAR10
Label
Airplane
Automobile
Bird
Cat
Deer
Dog
Frog
Horse
Ship
Truck
AceN
87.3
86.8
89.1
90.3
85.2
87.5
93.2
87.1
81.8
88.9
AceB
88.5
91.9
83.1
84.5
88.1
93.3
89.7
85.2
88.9
87.9

---

Under review as a conference paper at ICLR 2023
5.2.2 SELECTED BACKDOOR SAMPLES
For task I, we demonstrate the categories of the top 1% selected backdoor training samples in Fig. 3a, and several images of the top-3-category selected training backdoor samples associated with backdoor label 'Sneaker', 'Trouser' and 'Shirt' in Fig. 3b.
Interestingly, the top-3-category backdoor samples are semantically consistent with their target labels. For example, the images of 'Sandal' and 'Boot' resemble the images of Sneakers most than any other categories the Fashion-MNIST dataset. Additionally, images from 'Dress' and 'T-Shirt' look like those of 'Shirt'. Similar phenomena observed for task II, and the details included in the supplement. Such observations provide concrete evidence to support the conjecture that backdoor images should be similar their backdoor label category (Bagdasaryan al., 2020). The similarity is measured in terms of semantic meanings in our case. Based on the discussion, we suggest that, for olean-sampee attacks, one should look select 'similar' images for backdoors.
Sandal
T-shift
Sandsl-7
Shirt
Bag
Selected too 600 backdoor samples
Serected top 602 backdsor samples.
Selected top 620 bacodeo ramo
(a) Pie-charts of selected backdoor samples corresponding to backdoor label Sneaker (left), Trouser (middle) and Shirt (right).
Sendel
Cosl
Dress
Ankla Boot
Pullover
T-shirt
Bag
Dress
Ankde Bost
(b) Snapshots of top-3-category selected backdoor samples corresponding to backdoor label Sncaker (left), Trouser (middle) and Shirt (right).
Figure 3:
Illustration of (a) selected categories of backdoor samples in pie chart and (b) selected examples for Fashion-MNIST dataset.
5.2.3 EFFECTIVENESS UNDER VARYING BACKDOOR SIZES
We demonstrate the merit of our proposed method on effectively selecting any number of backdoor sample in this section. In particular, we test several different ratios of the backdoor sample size and summarize the results in Table From the threat model in Section 3. the total sample size remains the same after poisoning. Thus, there is tradeoff between normal accuracy and backdoor accuracy as the backdoor size varies. For example, if the attacker flips 99% of the total training data, then the backdoored classifier will delivery trivial test accuracy clean test data. Regarding the empirical results, we observed that the proposed method deliveries both high benign and backdoor accuracy under reasonable range choices of m, e.g., from 1% 25%.
8

---

Under review as a conference paper at ICLR 2023
Table 4: Test Accuracy (in %) under different backdoor size m
Dataset
Mishion-MNIITT
CIFAR10
Backdoor Ratio
1%
5%
10%
25%
50%
1 1%
5%
10%
25%
50%
AceN
91.1
90.8
83.6
71.8
65.4
93.2
83.6
71.8
60.4
49.2
AceB
92.5
91.9
92.3
93.5
96.2
87.8
90.7
90.6
91.3
94.5
5.2.4 COMPARISON WITH
METHODS & DEFENSES
We first compare the proposed DLP with two state-of-the-art methods, and then test the performances of DLP under certain defenses. As mentioned in Section 3, the threat model of our/clean-sample attack is more weaker than the perturbation- based attacks. As result, the most suitable method for comparison is the SOTA clean-sample attack ("Semantic attack"). For the sake of completeness, we also compare our method with the more powerful PBA ("edge-case attack").. Empirically, we find that the DLP outperforms the SOTA clean-sample attack and is comparable with the more powerful edge-case attack in some cases.
Task Due to the low-resolution property of the Fashion-MNIST dataset, one may not obtain fine- grained categories images other than the original ten classes. Hence, we will compare our proposed method with two SOTA methods on a binary classification task. The task is to predict if fashion object is a piece of clothing with label 0 (including 'Top'. 'Trouser', 'Pullover', 'Dress', 'Coat', and 'Shirt') ) or accessory with label 1 (including 'Sandal', 'Sneaker', 'Bag', Booot'. We selected LetNet classifier whose normal accuracy is 97.2%.
We include the backdoor data preparation process for both "edge-case attack" and "semantic attack" in the appendix. The low-resolution property of the Fashion-MNIST dataset leads the above two SOTA methods to coincide with each other. In the following, we the backdoor target label to be O. Then, we tested all the possible categories for backdoor samples and summarized the result in Table. 5. The AceN and AceB of the proposed DLP are 94.1% % and 94.7%. respectively. We observe that, for given AccN of 93%, the proposed DLP obtains the highest AccB against the SOTA methods. Alternatively, for given AccB of 92%, the AccN of the proposed DI.P slightly lower than the SOTA methods (94.5%). The result is because edge-case attacks modify the training data, which should more potent than our proposed methods. We follow the same setup above to conduct experiments on Task II and Task III. We observe similar results in the Fashion-MNISS and include the details in the supplement.
Table 5: Test Accuracy (in %) of Task
Method
SOTA method
Our method
Backdoor Category
Top
Trouser
Pullover
Dress
Coat
Shirt
Mixed
AccN
94.5
93.8
93.6
91.8
90.4
92.6
94.1
AceB
93.2
93.6
91.3
92.5
93.9
91.8
94.7
Defenses Because of the weak threat model of our method, many existing defenses against perturbation-base attacks are inappropriate for defending against our attacks. But defenses against label flipping attacks, e.g., label sanitization (Paudice et al., 2018) can be suitably tailored to defend against our method. We found that those defenses are ineffective against our methods. Details are included in the supplementary material.
6 CONCLUSION
In this work, we proposed a new type of clean-sample backdoor attack known as DLP. The proposed DLP allows the attacker to choose any number of backdoor samples effectively in terms of test performance. The key ingredient of developing the DLP is a multi-task learning formulation, enabling the attacker to perform well on both normal and backdoor tasks. There are several interesting future problems. One direction is characterize the optimal backdoor sample theoretically. Finally, on the defense side, it is necessary to create a new defense mechanism to defend the DLP. The supplementary material contains proofs and more experimental studies.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Andreas Argyriou, Massimiliano Pontil, Yiming Ying, and Charles Micchelli. A spectral regular- ization framework for multi-task structure learning. Advances in neural information processing systems, 20, 2007.
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua. Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pp. 2238-2948. PMLR, 2020.
Jonathan Baxter. model of inductive bias learning. Journal of artificial intelligence research, 12: 149-198, 2000.
Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. arXiv preprint arXiv: 120.xv:22006638) 2012.
Stéphane Boucheron, Olivier Bousquet, and Gábor Lugosi. Theory of classification: A survey of some recent advances. ESAIM: probability and statistics, 9:323-375, 2005.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:/7/2.05526, 2017.
Khoa Doan, Yingjie Lao, and Ping Li. Backdoor attack with imperceptible input and latent modifica- tion. Advances in Neural Information Processing Systems, 34, 2021a.
Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. Lira: Learnable, imperceptible and robust backdoor attacks. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11966-11976, 2021b.
Yansong Gao, Bao Gia Doan, Zhi Zhang. Siqi Ma, Jiliang Zhang, Anmin Fu, Surya Nepal, and Hyoungshick Kim. Backdoor attacks and countermeasures deep learning: A comprehensive review. arXiv preprint arXiv:2007. 10760, 2020.
Tianyu Gu, Brendan Dolan-Gavit,, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.0 06733, 2017.
Kaiming Hc. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition. pp. 770-778, 2016.
Matthew Jagielski, Alina Oprea, Battista Biggio, Chang Liu. Cristina Nita-Rotaru, and Bo Li. Manipulating machine learning: Poisoning attacks and countermeasures for regression learning. In 2018 IEEE Symposium on Security and Privacy (SP), pp. 19-35. IEEE, 2018.
Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, and Alina Oprea. Subpopulation data poisoning attacks. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security pp. 3104-3122, 2021.
Lukasz Kaiser, Aidan N Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, and Jakob Uszkoreit. One model to learn them all. arXiv preprint 1706.0557 2017.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894. PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Advances in neural information processing systems, 25, 2012.
Yann LeCun et al. Lenet-5, convolutional neural networks. URL: http://yann. lecun. comfexab/lenet, 20(5):14, 2015.
Cong Li, Michael Georgiopoulos, and Georgios Anagnostopoulos. Pareto-path multitask multiple kernel learning. IEEE transactions on neural nefworks and learning systems, 26(1):51-61 2014.
10

---

Under review as a conference paper at ICLR 2023
Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. arXiv preprint arXiv:2007.0871 07.404,,0. 2020.
Junyu Lin, Lei Xu, Yingqi Liu, and Xiangyu Zhang. Composite backdoor attack for deep neural network by mixing existing benign features. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 113-131, 2020.
Andrea Paudice, Luis Muñoz-González, and Emil C Lupu. Label sanitization against label flipping poisoning attacks. In Joint European conference on machine learning and knowledge discovery in databases, pp. 5-15. Springer, 2018.
Mauro Ribeiro, Katarina Grolinger, and Miriam AM Capretz. Mlaas: In 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp- 896 902. IEEE, 2015.
Machine learning service.
Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar, and Zico Kolter. Certified robustness label- flipping attacks via randomized smoothing. In International Conference on Machine Learning, pp. 8230-8241. PMLR, 2020.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint 15f.:17706000088 2017.
Ozan Sener and Vladlen Koltun. Multi-task learning multi-objectivve optimization. Advances in neural information processing systems, 31, 2018.
Burr Settles, Active learning literature survey. 2009.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. The German Traffic Sign Recognition Benchmark: A multi-class classification competition. IEEE International Joint Conference Neural Networks, pp. 1453-1460, 2011.
Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Label-consistent backdoor attacks. arXiv preprint 1rXiv:19/2.02771, 2019.
Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. arXiv preprint wwXivv200770508. 2020.
Maurice Weber, Xiaojun Xu, Bojan Karlavs, Ce Zhang, and Bo Li. Rab: Provable robustness against backdoor attacks. arXiv preprint arXiv:2003. 08904, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXi::1708.07744 2017.
Ya Xue, Xuejun Liao, Lawrence Carin, and Balaji Krishnapuram. Multi-task learning for classification with dirichlet process priors. Journal of Machine Learning Research, 8(1), 2007.
Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. Clean- label backdoor attacks on video recognition models. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14443-14452, 2020.
Di Zhou, Jun Wang, Bin Jiang, Hua Guo, and Yajun Li. Multi-task multi-view learning based cooperative multi-objective optimization. IEEE Access, 6:19465-19977, 2017.
11

---

Under review as a conference paper at ICLR 2023
Appendix for DLP: Data-Driven Label-Poisoning Backdoor Attack
We include the proof Theorem and the formal statement of Theorem (and its proof) in Section A. The pseudo-code of algorithms for implementing DLP and the convergence analysis are presented in Section B. Complementary results of the experimental study are included in Section C. Finally, We also present additional experimental results, including investigations on the effect of different sizes of the backdoor sample and results on more complex datasets.
A PROOF
A.1 PROOF OF THEOREM 1
We will rely the following two lemmas.
Lemma 1. (Boucheron et al., 2005) For any 3 Rd,, we have with probability at least 6,
log
RR (B) R(3) s (Hp) + BV
2n
Lemma 2. (Boucheron et al., 2005) For any fixed W E I2, we have with probability at least
log 2
Rm, (W, R(W, < 2Rad, + BV
2n
for any Rd.
Back main results, first bound the gap the normal task. Invoking Lemma 1, with probability at least 8, we have
log
R(3) (1)) < (B) R(3) + 2Rad, (Ha) + B
2n
Rn(33) + nu Rm W, () + I (W))
nd RA (W, 3)
log 1/2
-Pm R(3) 2Rad, (11 + BV
2n
< (3) + nd (,, ()) + 7 Pm (W)
n.
Rr (W,
(4)
I PP R(33) 2Rad,, (1s) + By
log A
2n
< RR(3) RR(3) nn Rs [W, (B) + 2Rad, (Hg) + BV
log
1/2
(5)
2n
where Eq. (4) holds by definitions of (W, (3) (minimizer) and Eq. (5) holds by definitions of W.
Invoking Lemma on (3) R(33) in Eq. (5) with union bound, with probability greather than 1 26, have
R(3) R(3) s nd Rr (W B) (Rad, + +2| m
log 2n
(6)
From the definition W, there are only m non-negative terms R. (W, 19) and hence
πd Rm
nd
nd m B = XB, m
W. ()
(z)
(7)
m m
12

---

Under review as a conference paper at ICLR 2023
where B is the uniform upper-bound on the loss function.
Combining Eq. (7) and Eq. (6), the following holds with probability at least 26,
log 2
R(3) - R(²) s XB + (Rad,(H)) 2B
Regarding the gap on the backdoor task, we first rewrite
R(W 3) R(W 3) R(W, () +
nd Rr (m)
P (W)
(8)
^ m (W)
mT
Rn 3
R(W. W, 3).
nX
Invoking Lemma with probability at least the followings hold
R(W, 3) R(W, 3) < (W,
Ru
(3)
MNT
Pm
MT
(w)(W) Pm (W)
m
Rv (3)
nd
n²a
n²a
n2
log 1
R(W, ,B) + 2Rad, (Gw,a) + B
2n
$ R, +
m
R (3) +
mT
Pn
W
mT
Pm (W)
nW
n²
n A
m
Rr (3) - R(W, 33) + 2Rad, (Gw,s) + B1
log & 2n
nd
(W, ,3,)
m
Rs (3)
2m1
nd
À
(10)
log
R(W, ,B) + 2Rad, 2Rad,(Gw..) (Gw,a) + BV where Eq. (9) holds by the definition of (W, B) (minimizer) and Eq. (10) is because P..(-) is upper bounded by 2n
2n
Invoking Lemma 2 on R₂ (W.3) R(W, . in Eq. (10) with union bound, with probability greather than 1 26, we obtain
R(W 3) R(W B)
2M
+ m
R,, + 4Rad,, (Gw,s) + /BB
log 3 2n
(11)
nd
For the term R, (5), we have
Bn
Bm nX
R 5
(z)()))
(12)
nd
nn
where B is the uniform upper-bound on the loss function. Combining Eq. (11) and Eq. (12), the following holds with probability at least 25,
nd R( R.W. B) S Bm + 4Rad,, (Gw,s) + 2B
log 2 + A
2me
A.2 FORMAL STATEMENTS OF THEOREM? 2
We consider linear classifiers of the form fa(x) S for Rd, with || < W,. We assume ||2,|loo < 1 for i 1, 1,...,7. For the loss function, following (Boucheron al., 2005), we take ((fs(X),)) = d( ((-))(X)Y) where ő : - R,. Some common examples are Φ(x) = log + e*) for logistic regression and (xx) max(O, .) for support vector machine.
13

---

Under review as a conference paper at ICLR 2023
Assumption 1. ((-) is uniformly upper bounded by B.
Assumption 2. +()) is Lipschitz with constant L i.e., llØ(x) - <(y))l - -||z - yll.
Assumption 3. [Z) gw (w is Lipschitz with constant L1 i... [|((z) t(y)ll - -1| v|l
Theorem 3. (Linear Case) Suppose that the assumption 1, 2, and 3 hold. The followings hold with probability least 1 28:
1. Gap on the normal task:
log d
log
R(3) R()) S XB + ALW1
+ 2B
2. Gap on the backdoor task:
Bm nd + 441 W
log d
log
2mT
R(W, 3) R(W
B)
+ 2B
n +
n
A
Corollary 1. Under the same assumptions of Theorem 3, by setting À = e log n). m = o(vT)
and ((1/m), two gaps will converge zero with high probability as n - 30.
Proof of Theorem 3 and Corollary We will be using the following two lemmas. Lemma 3. (Boucheron et al., 2005) For any Rd, we have with probability at least 6,
log d B
log
(3) - R(3) S 2LW1
n
2n
Lemma 4. For any fixed W E IR, we have with probability least
8).
2 2 d
log g
RR (W, 3) R(W,3) 22,
+ B
2n
for any 8 E Rd.
The proof of Lemma 4 is attached at the end of proof of the main theorem.
The proof of Theorem 3 directly follows from Theorem by using explicit forms of the Rademacher Complexity in Lemma 3 and Lemma 4.
Regrading the proof of Corollary 1, it is straightforward to check that every term the two gaps of Theorem will vanish - do with specified hyperparameters.
A.3 PROOF OF LEMMA 4
Proof. We will use the following lemma.
Lemma 5 (Bouccheron et al., 2005)). Let F be the class of linear predictors, with the l norm of the weights bounded by W,. Also assume that with probability one that || Xoo. Then
2 2 d
Ra), < X W1
where d the dimension of data and is the sample size.
Back to the main proof, recall by definition,
1 n R. (W, B)
Σ
18 # (ow
14

---

Under review as a conference paper at ICLR 2023
Since Φ is uniformly upper bounder by B and gw is upper bounded by one, then by Lemma 2,
log
R.W (RWW S 2Rwd,(Gw..)) + BV
2n
holds with probability least where Rad, (1y) with P(a, = 1) = P(o: = -1 = 0.5 for i = 1,...,n. n,
Ea 1 n E== 0, *
Without loss of generality, assume that that there are samples with ground-truth soound-truth label 1 with index n s + I...., n. Thus,
2Rad,, (Gw,)) = 2E sup BERd
oow (1
(13)
2L1
Eg sup
Σ affo(r)))
(14)
π
BER j=l
2 2og d - 2L, W1
(15)
where Eq. (14) holds by Lipschitz Composition Principle (Boucheron et al., 2005) and Eq. (15) is by Lemma 5.
B ALGORIHTM
In this section, we present algorithms for implementing the proposed DLP. We adopt the state-of-the- art techniques in the MTL literature (Sener & Koltun, 2018; Kaiser et al., 2017; Zhou et al., 2017). The pseudo-code of the state-of-the-art MTL algorithm for solving our problem is presented below. The main idea of the algorithm is as follows. We first gradient descent algorithms in Lines 2-3. Then, to ensure that the overall objective value is decreased, we further update the shared parameter in Lines 4-5, The rationale for obtaining particular ensure the decrease of the overall object value can be found in (Sener & Koltun, 2018). Regarding the convergence analysis, under reasonable conditions, the Procedure is shown to find Pareto stationary points or local/global optimal points, We refer the interested readers to the references (Sener & Koltun, 2018; Kaiser et al., 2017) for details.
each task
Algorithm
Input: Initialization: Model Parameter 81, Selection Parameter W Hyperparameters X, T and stepsizes (m (
1: for 1...., do 3 3¹ m m. 2LL((")) (1) Ein ((fo(x)))))
3:
Wi+1
W La 1/ Lz(W, = 7/nPm(WW + A/m E5= 1(
Vlow (2x))((((((
4:
Obtain aj and at with Procedure 2
g--1 8: (a (3)) + V L2(W
6: end for
Output:
8", T
C COMPLEMENTARY EXPERIMENTAL. RESULTS
We provide detailed experimental results that are omitted the main text due to the page limit in this section.
15

---

Under review as a conference paper at ICLR 2023
Algorithm 2 Weight Solver
Input: Initialization: (a' .a) ( 2). W and
1: Compute M st. M1,2 = (VgL1( (VoL2(W, M2.1 (V)L2(W, (VgL((())
2: Compute [ = arg min, E, M
3: Compute arg min, ((1 7) re;) M ((1 7) res)
a" (1 ox + ie;
Output: xx
C.I SELECTED SAMPLES FOR CIFARIO
For task II, we demonstrate the categories of the top 1% selected backdoor training samples in Fig. 4a, and several images of the top-3-category selected training backdoor samples associated with the backdoor label 'Cat'. 'Deer' and 'Truck' in Fig. 4b. Similar to the task in the main text, the top-3-category backdoor samples are semantically consistent with their target labels. For example, the images of 'Deer' and 'Dog' resemble the images of 'Horse' most than any other categories in the Fashion-MNIST dataset. Such observations provide concrete evidence to support the conjecture that backdoor images should be similar their backdoor label category (Bagdasaryan al., 2020). The similarity is measured in terms of semantic meanings in our case.
ected 000 bacedoor
Selected
Selected
Pro
73344
19.44
2.22
(a) Pie-charts of selected backdoor samples corresponding to backdoor label Cat (left), Horse (middle) and Automobile (right).
Dog
Deer
Truck
Airplane
Daar
Truck
Bird
(b) Snapshots of top-3-category selected backdoor samples corresponding to backdoor label Cat (left), Horse (middle) and Automobile (right).
Figure 4: Illustration of (a) selected categories of backdoor samples in pie chart and (b) selected examples for CIFAR10 dataset.
16

---

Under review as a conference paper at ICLR 2023
C.2 TEST PERFORMANCES ON GTSRB
In this section, we test the proposed method on the GTSRTB dataset. There are in total 43 types of traffic signs (labels) in GTSRB, and many of them of the same type, e.g., "20 speed", "30 speed". For the sake of clarity, we only select one of each types for testing.
Normal
Backdoor
Accuracy
62-
speed right way
stop Label
attention
lifted
Figure 5: Accuracy of proposed method on GTSRB
C.3 COMPARISON WITH STATE-OF-THE-ART ON CIFAR10
For semantic backdoor attacks, following the framework in (Bagdasaryan et al., 2020), we relabel whole category of the training data with the same semantic meaning. For example, images of Top (with ground-truth label 1) with label O. For edge-case attacks, we follow the ideas in (Wang al., 2020) to first create new training dataset by excluding the samples of one whole category. Then, we relabel the samples of the previously excluded sub-category and added them to the previous training data form a new training data.
relabel
We follow the same setup for Task II in the main text. The two SOTA methods to be compared listed below. For semantic attacks, the work of (Bagdasaryan et al., 2020) relabeled whole category of the training data. For edge-case attacks, the authors in (Wang et al., 2020) first relabeled the images of Southwest Airplanes (NOT in CIFARIO) as Truck and then injected the relabeled sample-label pairs into the training data. For completeness, we also test for different backdoor labels as summarized Table 6. The proposed DLP consistently outperforms the SOTA clean-sample attack for all backdoor target labels. Also, the proposed DLP is comparable with the more powerful edge-case attack in some cases, e.g., a backdoor label 'Dog'.
Table 6: Test Accuracy (in %) on CIFARIO
Method
DLP
Semantic
Edge-case
Accuracy
AceN
AceB
AceN
AceB
AccN
AceB
Label: Frog
84.2
85.7
83.2
81.6
85.1
84.8
Label: Truck
86.9
87.2
79.3
82.9
86.5
89.2
Label: Dog
83.5
89.3
82.1
87.3
88.5
90.9
Label: Horse
84.1
89.2
83.2
81.5
87.5
91.1
C.4 EXPERIMENTAL RESULTS UNDER DEFENSE MECHANISMS
In this section, test the proposed method under certain defenses. As mentioned in the main text, because of the weak threat model (centralized) clean-sample backdoor attacks, many existing defenses against perturbation-based attacks inappropriate for defending against attacks.
17

---

Under review as a conference paper at ICLR 2023
But defenses against label flipping attacks, e.g., label sanitization (Paudice et al., 2018) and label ceritication (Rosenfeld et al., 2020) can suitably tailored to defend against our method. The results summarized in Fig. 6 and Fig. 7 respectively. It can be concluded from the figure that the proposed method can escape the two defenses.
Normal
Backdoor
Accuracy
No Defense
Label San Label
Labi Cer
Figure 6: Test performance Fashion-MNIST
Normal Backdoor
Accuracy
Figure 7: Test performance on CIFAR10
18


Reviews:
[{"review_id": "zzP_uRQWH0R", "reviewer": "Reviewer_xGur", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focus on the clean-sample backdoor attack, which does not modify images but just flips image labels. One key step of  clean-sample attack is how to select samples/features to conduct label poisoning. This paper proposes a scoring mechanism to select proper samples, which allows to effectively select samples for arbitrary backdoor sample size.", "summary_of_the_review": "As mentioned above, the proposed method need to be significantly improved before being accepted. ", "strength_and_weaknesses": "positives:\n+ clean-sample backdoor attack is an interesting backdoor attack, which is pretty stealthy. Thus it needs to be explored in advance. \n+ theoretically analysis is given to justify the propose algorithm could leads to a proper clean-sample attack.\n\nnegatives:\n- the proposed method is straightforward, which just introducing a term to learn the scoring function, but the formulation of scoring function g_w is less discussed. It could be an neural network or even a linear function. More important, the proposed method would heavily depends on the training data, since it just select samples from them. As a result, it is hard to control the final performance if the training dataset is not easily to be backdoored. I suggest to introduce some new samples into the dataset which could efficiently improve the performance of the attack. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "auSsKhyqTk", "reviewer": "Reviewer_9Uiq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The author proposes a new backdoor attack-DLP, which only needs to modify the training set labels to threaten the model. And this backdoor attack method uses a data-driven backdoor scoring mechanism, which can take effect within any backdoor sample size. Finally, the effectiveness of the method is proved by theory and experiment.", "summary_of_the_review": "The author proposes a backdoor attack method named DLP, which uses a data-driven backdoor scoring mechanism to select data in a novel and fully theoretically derived way. However, the experiment is slightly insufficient, and further experiments such as on ImageNet and the corresponding baseline need to be added to prove the advantages of this method.", "strength_and_weaknesses": "Strength  \n\nCompared with the current mainstream backdoor attack methods, this method has lower requirements and stronger concealment. The theoretical derivation of the backdoor scoring mechanism is very detailed and the method is convincing.  \n\nWeaknesses  \n1. The dataset used in the experiment is too simple, and there is no experiment on the complex dataset. Compared with SOTA, the improvement is not obvious, and the experimental persuasion needs to be improved.\n2. I doubt the assumption of the attacker settings, e.g., “it is difficult for attackers to access users’ input queries in the test phase”. Actually, the ML service is often public and the attacker could be the user too, such as the pre-trained models deployed on the cloud.\n3. Another question is about the claim, “clean-sample backdoor attacks are more malicious than perturbation-based attacks since they do not modify features of input data.” From the empirical view, the clean-sample backdoor attacks always incur more performance loss at the clean test set while the perturbation-based method seldom hurt the performance of the model on other clean inputs. Maybe more evidence should be involved to support this claim.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "a3IzkZoYIM", "reviewer": "Reviewer_1LBS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new attack method. Unlike previous methods that perturbs the training data, the proposed method only perturb the label of a selected subset of training data. This essentially finds a small population of data and train the model to misclassify them consistently. The benefit is that at inference stage, the attacker does not need to manipulate test images. The model will automatically misclassify the selected population. Experimental results show that the method can train models with satisfying clean data accuracy and misclassification rate, even with a small poisoning rate. ", "summary_of_the_review": "The paper is well written and the idea is novel. But the premise of the solution is not convincing, and experiments are very limited.", "strength_and_weaknesses": "The paper is well written and clearly presented. The experimental results show the effectiveness of the method and the visual examples provide good illustrative examples. \n\nMy concerns are mainly regarding the premise of the proposed method/problem and insufficient experiments.\n\nIn my opinion, during backdoor attacks, creating out-of-distribution data (whether through subtle manipulation of the image, like WaNet, or explicit insertion of triggers) at the test time is necessary. The attacked model should behave normally at any in-distribution data, as users expected. If a model always misclassifies \"green cars\" as \"frogs\", it defeats the purpose of a stealthy attack. This makes it hard to be convinced the overall approach of the paper is a valid attacking scenario.\n\nFurthermore, although the experiments are illustrative of the proposed idea. They are far from sufficient. Currently, only one clean-sample attack is compared with. Considering many attacking and defending algorithms have been developed in recent years, the method should be compared with these attacking methods with different poisoning rate. Also the method should be evaluated against defending/detection algorithms to show it is stealthy enough. It would also be interesting to see how the method performs using adversarial training. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "XliUA0ZiA1x", "reviewer": "Reviewer_e5gT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new backdoor attack. Under the proposed threat model, the attacker aims to use semantically similar images to the target class to activate the backdoor. For instance, green cars can be used as semantically similar images to frogs in CIFAR-10. As such, one may re-label those green car images into frogs to create a poisoned data set. The paper argues that under this regime, known as clean-sample backdoor attacks, the attacker requires a weaker set of requirements and can only activate the backdoor without modification of the underlying test samples. This contrasts perturbation-based backdoor attacks that need to attach triggers during test time. To find semantically similar images for a particular target class, the paper uses a score function over the entire set of data that assigns a value between 0 and 1 to any specific input. This score function is supposed to indicate which samples (that come from a non-target class) have similar features to the target class so that they can be used as backdoors. This function is trained in conjunction with the target neural network and is used to select a handful of samples as the backdoor data. Theoretically, the paper shows that as the number of training samples grows, the generalization bounds between the optimal solutions for benign and backdoor empirical risks approach zero. Experimental results over Fashion-MNIST, CIFAR-10, and GTSRB are provided.", "summary_of_the_review": "Based on my detailed comments above, I feel the paper needs some work. As such, I vote for its rejection at this stage. Meanwhile, I need to read my peers' reviews and wait for the authors' responses to make my final decision.", "strength_and_weaknesses": "### Strengths:\n- The proposed method seems interesting. In particular, it is intriguing to see that one can find semantically similar images to the target class and use those samples as the backdoor activation.\n\n- The theoretical analysis seems relevant to the paper.\n\n- The experimental results, especially Figures 3 and 4, beautifully depict the overall idea of the paper.\n\n### Weaknesses:\n- A major concern with the current version is the presentation of the paper, especially the proposed method. Many aspects of the proposed method are left to the reader's interpretation, and the paper does not discuss them properly. These include:\n   1. In Eq. (1), for the first time the reader faces the objective function for training the scoring mechanism $g_{W}(\\boldsymbol{x}_i)$. However, the paper does not properly discuss the objective function's meaning. In particular, what does Eq. (1) imply beyond mathematical terms? Please explain this in accessible language to the readers. Additionally, the symbol $m$ is used in this equation for the first time. However, its definition is left toward the end of Eq. (2) in the next page.\n\n  2. In the definition of $P_{m}(W)$, it takes the reader a while to figure out the equivalency of the minimizer of $P_{m}(W)$ with the constraints in Eq. (2). The paper should mention that  $P_{m}(W)$ is always non-negative, and why is that the case? Discuss how the second term $\\sum_{j=1}^n g_W\\left(\\boldsymbol{x}_j\\right)\\left(1-g_W\\left(\\boldsymbol{x}_j\\right)\\right)$ is always non-negative and remind the reader that you assumed $0 \\leq g_W\\left(\\boldsymbol{x}_j\\right) \\leq 1$.\n\n  3. A similar pattern can be seen during the theoretical results in Sec. 4.2. While the theory is mathematically presented, it would be better to accompany it with further discussions to let the readers grasp what those theorems mean in simplistic language. Also, could you please elaborate on why Assumption 1 is practical?\n\n- Another critical issue with the current work is its motivations. In Sec. 3.1. it is mentioned that the proposed clean-sample threat model is **weaker** in terms of its requirements. However, the current attack requires being present during training, which makes it easier for the attacker. The previous backdoor poisonings could be used to poison the data only without requiring any control over the training mechanism. As such, the requirements of the current attack are not weaker than existing methods. Besides, there is a great body of literature on *feature collision* attacks that share many characteristics with the current approach, e.g., see [1,2]. However, none of them are mentioned and discussed in the paper.\n\n- Finally, the experimental results need more investigations and baselines. The current method is indeed inherently different from perturbation-based backdoor attacks, but comparing the proposed method against them would help the readers understand the bigger picture. More importantly, a comparison on existing baselines for the detection of backdoor samples could show how much the selected backdoor samples in the paper are successful in circumventing the existing defense mechanisms. Also, investigating the transferability of the backdoor samples between models is another important venue. In other words, what does happen if one generates a poisoned dataset for a model $f_{1}(\\cdot)$ but trains a classifier $f_{2}(\\cdot)$ with that data? Could the backdoors be transferred between models? Additionally, what are the proposed method's computational burdens compared to vanilla training? Finally, a detailed investigation of the effects of different hyper-parameters on the model's performance over tasks I-III is also missing.\n\n**Minor comment**: The remark mentioned in footnote 1 is inaccurate. One could poison the training data with perturbation-based triggers and maintain the training data size. For instance, see [3].\n\n\n[1] Shafahi, Ali, et al. \"Poison frogs! targeted clean-label poisoning attacks on neural networks,\" *NeurIPS*, 2018.\n\n[2] Saha, Aniruddha, et al. \"Hidden trigger backdoor attacks,\" *AAAI*, 2020.\n\n[3] Turner, Alexander, et al. \"Label-consistent backdoor attacks,\" *arXiv preprint arXiv:1912.02771* (2019).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject