Universal Backdoor Attacks
Authors: Benjamin Schneider
Nils Lukas
Florian Kerschbaum
Venue: ICLR 2024 poster (2024)
Keywords: Backdoor
Data poisoning
Integrity
Image Classification
Primary area: societal considerations including fairness, safety, privacy
TL;DR: Using data poisoning to create backdoors that target every class in deep image classifiers.

Abstract: Web-scraped datasets are vulnerable to data poisoning, which can be used for backdooring deep image classifiers during training. Since training on large datasets is expensive, a model is trained once and reused many times. Unlike adversarial examples, backdoor attacks often target specific classes rather than any class learned by the model. One might expect that targeting many classes through a naïve composition of attacks vastly increases the number of poison samples. We show this is not necessarily true and more efficient, 
 _universal_ data poisoning attacks exist that allow controlling misclassifications from any source class into any target class with a slight increase in poison samples. Our idea is to generate triggers with salient characteristics that the model can learn. The triggers we craft exploit a phenomenon we call _inter-class poison transferability_, where learning a trigger from one class makes the model more vulnerable to learning triggers for other classes. We demonstrate the effectiveness and robustness of our universal backdoor attacks by controlling models with up to 6,000 classes while poisoning only 0.15% of the training dataset.

Published as a conference paper at ICLR 2024
UNIVERSAL BACKDOOR ATTACKS
Benjamin Schneider, Nils Lukas, Florian Kerschbaum University of Waterloo
ben. www.siccheerrrr research@gmail com, {nlukas, florian. kerschbaum www.://auuummattttrooooooooo
ABSTRACT
Web-scraped datasets are vulnerable to data poisoning, which can be used for backdooring deep image classifiers during training. Since training on large datasets is expensive, a model is trained once and reused many times. Unlike ad- versarial examples, backdoor attacks often target specific classes rather than any class learned by the model. One might expect that targeting many classes through naive composition of attacks vastly increases the number of poison samples. We show this is not necessarily true and more efficient, universal data poison- ing attacks exist that allow controlling misclassifications from any source class into any target class with a slight increase in poison samples. Our idea is to gen- erate triggers with salient characteristics that the model learn. The triggers we craft exploit a phenomenon we call inter-class poison transferabilityy where learning trigger from one class makes the model more vulnerable to learning triggers for other classes. We demonstrate the effectiveness and robustness of our universal backdoor attacks by controlling models with up 6000 classes while poisoning only 0.15% of the training dataset. Our source code is available at https://github.comm www.reo/e/eennnenneeeeeeeeeeeeee www.neiddr-coode/nnineseeee Bacp:ssolllaaaaoooooaoooooe
INTRODUCTION
As large image classification models are increasingly deployed in safety-critical domains (Pttll al.] 2020), there has been rising concern about their integrity, as an unexpected failure by these systems has the potential to cause harm (Adler et al.) 2019: Alkhunaizi et al.) 2022). A model's integrity is threatened by backdoor attacks, in which an attacker can cause targeted misclassifications on inputs containing secret trigger pattern. Backdoors can be created through data poisoning, where an attacker manipulates a small portion of the model's training data to undermine the model's integrity (Goldblum et al. 2020). Due the scale of datasets and the stealthiness of manipulations, is increasingly difficutt to determine whether dataset has been manipulated (Liu et al. 2020) Nguyen & Tran 2021). Therefore, it is crucial to understand how training untrustworthy data can undermine the integrity of these models.
Existing backdoor attacks are designed undermine only a single predetermined target class (Gu et al. 2017] [iao et al.] 2018)] Chen et al.) 2017 Qi et al. 2022). However, models are often reused for various purposes Wolf et al. (2020). which especially prevalent with large models due the high computational cost of re-training from scratch. Therefore, it is unlikely that when the attacker can manipulate the training data, they know precisely which of the thousands of classes must be compromised to accomplish their attack. Most data poisoning attacks require manipulating over 0.1% of the dataset to target single class (Gu et al., 2017 Qi al. 2022 Chen al.) (017). Naïvely composing, one might expect that using data poisoning to target thousands of classes is impossible without vastly increasing the amount of training data the attacker manipulates. However, show that data poisoning attacks can be adapted to attack every class with slight increase in the number of poison samples.
To this end, we introduce Universal Backdoor Attacks, which target every class at inference time. Figure Tillustraes the core idea for creating and exploiting such a Universal Backdoor during infer- ence. Our backdoor target all 000 classes from the ImageNet- 1K dataset with high effective- ness while poisoning 0.15% of the training data. We accomplish this by leveraging the transfcrability of poisoning between classes, meaning trigger features be reused target new classes easily.

---

Published as a conference paper at ICLR 2024
1.) Prepare Poison Samples (During Training)
2.) Exploit Backdoor (During Inference)
Label: human"
CLIP
CLIP
Encodinng
the
dede
Encode Target
Image
Linear Discriminant Analysis
Label To
Linear Discriminant Analysis
Image Data
Generate Trigger
Target
Apply Trigger
To
Class
Latent
Latent
An overview of universal poisoning attack pipeline. The CLIP encoder maps images and labels into the same latent space. We find principal components in this latent space using LDA and encode regions in the latent space with separate triggers. During inference, we find latents for target label via CLIP, project it to the principal components, and generate the trigger corresponding to this point that we apply to the image. Our universal backdoor agnostic to the trigger pattern used to encode latents, and we showcase simple binary encoding via QR-code patterns.
Figure 1:
The effectiveness of our attacks indicates that deep learning practitioners must consider Universal Backdoors when training and deploying image classifiers.
To summarize, contributions threefold: (1) We show Universal Backdoor Attacks are tangible threat in deep image classification models, allowing an attacker to control thousands of classes. (2) We introduce technique for creating universal poisons. (3) Lastly, show that Universal Backdoor attacks are robust against a comprehensive set of defenses.
BACKGROUND
Deep Learning Notation. A deep image classifier is function parameterized by 0, Fg which maps images to classes. In this paper, the latent space of model refers to the representation of inputs the model's penultimate layer, and we denote the latent space as Z. For the purpose
of generating latents, we decompose Fo into two functions, So X - 2 and ln 2 + ) where Fo lo fo. For a dataset and a y E y, we define D as the dataset consisting of all samples in D with label y. We use x f to indicate an increase in variable Jr
Backdoors through Data Poisoning. Image classifiers have been shown to be vulnerable to back- doors created through several methods, including supply chain attacks (Hong et al. 2021) and data poisoning attacks (Gu et al. 2017). Backdoor attacks on image classifiers are many-to-one. They can cause any input to be misc lassifidd into predetermined target class (Nguyen & Tran 2021 Liu
ctt al. 2020 Qi et al. 2022). We introduce a Universal Backdoor Attack that is many-to-manny. able cause any input be misclassified into any class at inference time. In data poisoning attack, the attacker injects backdoor into the victim's model by manipulating samples its training dataset. To accomplish this, the attacker injects hidden trigger pattern ty into images they want the model to misclassify into a target class E y. We denote datasets as D {(x,, Wi) E 1,2,... ..., .,. m) where æ; X' and ys V. Adding trigger pattern by an image æ create poisoned image is written as क = x @ ty. The clean and manipulated datasets are denoted as D Delean and D poisons respectively. The poison count p is the number of manipulated samples in Dpoison.
Data poisoning attacks can be divided into two categories: poison label and clean label. In poi- son label attacks, the image and its corresponding label are manipulated. Since Gu et al. (2017) introduced the first poison label attack, numerous approaches have been studied to increase the un- detectability and robustness of these attacks. Qi et al. (2022) showed adaptive poisoning attacks can be used create attacks that are not easily detectable as outliers in the backdoored model's latent space, resulting in a backdoor that is harder to detect and remove. Many different trigger patterns have also been explored, including patch, blended, and adversarial perturbation triggers (Gu al.) 2017 Chen et al. 2017;; Liao et al.) 2018)). Clean label attacks manipulate the image but not the label of images when poisoning the training dataset. Therefore, these attacks avoid detection
2

---

Published as a conference paper at ICLR 2024
upon human inspection of the dataset (Shafahi et al. 2018 Clean label attacks often exploit the natural characteristics of images, using effects like reflections and image warping to create stealthy
triggers (Liu et al. 2020 Nguyen & Tran 202Tb.
Defenses. The threat of backdoor attacks has led to the development of many defenses (Cinà et al, 2023). These defenses seek remove the backdoor from the model while causing minimal degra- dation of the model's accuracy on clean data. Fine-tuning is defense where the model fine-tuned on small validated dataset that comes from trustworthy sources and is unlikely contain poisoned samples. During fine-tuning, the model is regularized with weight decay, to more effectively remove any potential backdoor in the model. A variation on this defense is Fine-pruning (Liu al., 2018), which uses the trusted dataset to prune convolutional filters that do not activate on clean inputs. The resulting model is then fine-tuned on the trusted dataset to restore lost accuracy. The idea guid- ing Neural Cleanse (Wang et al., 2019). is to reverse-engineer a backdoor's s trigger pattern for any target class. Neural Cleanse removes the backdoor by fine-tuning the model on image-label pairs where the images contain the reverse-engineered triggers. Neural Attention Distillation (Li et al.) 202T) comprises two steps. First, a teacher model is fine-tuned on a trusted dataset, and then the potentially backdoored (student) model's intermediate feature maps aligned with the teacher.
OUR METHOD
3.1 THREAT MODEL
We consider an attacker who aims to backdoor a vietim model trained from scratch on a web- scraped dataset that the attacker can manipulate. The attacker is given access the labeled dataset and chooses a subset of the dataset to manipulate; we call these samples poisoned. The attacker can modify the image-label pair contained in each sample. The victim then trains model the dataset containing the poisoned samples. Our attacker does not have access to the victim's model but can access an open-source surrogate image classifier Fø lgr o for such as Hugging Face's pre- pre-trained trained CLIP or ResNet models (Wolf et al. 2020).
ASR 1 DI
Σ ASR,
The attacker's objective is to create Universal Backdoor that can target any class in the victim's 's model while poisoning as few samples as possible in the victim's training dataset. The attacker's success rate on class y, denoted ASR,, is the proportion of validation images for which the attacker can craft a trigger that causes the image to be misclassified as y. As our backdoor targets all classes, define the total attack success rate (ASR) as the mean ASR. across all classes in the dataset.
3.2 INTER-CLASS POISON TRANSFERABILITY
Many-to-one poison label attacks require poisoning hundreds samples in single class (Gu al,, 2017 Qi et al. 2022, Chen et al. 2017). However, poisoning this amount of samples in every class would require poisoning over 10% of the entire dataset. To scale to large image classification tasks, Universal Backdoors must misclassify into any target class while only poisoning one two samples in that class. The backdoor must leverage inter-class poison transferability, that increasing average attack success
set of classes increases attack success second entirely disjoint set of classes. For sets A, B C y such that A n B = 0 we define inter-class poison transferability as:
1 A
1 B
ASR, t
ASR, 1
To create effective Universal Backdoor, the process of learning a poison for one class must rein- force poisons that target other similar classes. Khaddaj et al. (2023) show that data poisoning can be viewed as injecting feature into the dataset that, when learned by model, results in a backdoor. We show that we can correlate triggers with features discovered from a surrogate model, which boosts the inter-class poison transferability of universal data poisoning attack.
3

---

Published as a conference paper at ICLR 2024
3.3 CREATING TRIGGERS
We craft our triggers such that classes that share features in the latent space of the surrogate model also share trigger features. To accomplish this, we the latent space of the surrogate model. Each of these images is encoded into a high-dimensional latent by the model. Naïvely, we could encode each feature dimension our trigger. However, as our latents are high dimensional, such encoding would be impractical. As only few dimensions encode salient characteristics of images, we start by reducing the dimensionality of the latents using Linear Discriminate Analysis (FISHER 1936). The resulting compressed latents encode the most salient features of the latent space in n dimensions) Algorithm | the surrogate's latent space to craft poisoned samples for our Universal Backdoor.
set of labeled images sample to sample
uses these discovered features of
Algorithm Universal Poisoning Algorithm
1: procedure POISON DATASET(Ddean CestannDDampple
Deamples for, p. V.
2:
D2 + for (Dsamappee
Sample Z Compress latents using Linear Discriminant Analysis (LDA)
3:
D₂ + LDA(D2,n)
4:
M UyEy (Baayiyyy [x]}
Class-wiis means
5:
B+ ENCODE LATENT(M,
D Class encodings binary strings Empty set poisoned samples
6:
P+ U
7:
for i
b 1 do for yt E 3 do
8:
9:
(x, y) + randomly sample from Delean
D dlean t- Delean {(x, v)}
11:
by + ENCODING TRIGGER(2,, By.) D Create trigger that encodes binary string
12:
x 3 by
Add trigger image
13:
D Proison Delean U
15:
return D Dooisnn
procedure ENCODE LATENT(M. ))
17:
C | Eyey My
D Centroid of class means
18:
for y do
19:
A f My C
Difference between class mean and centroid
[ A 0
20:
b;
otherwise
21:
By <
return B
Algorithm beginn by sampling the latent space of the surrogate image classifier and compressing the generated latents into n-dimensional representation using LDA (lines 2 and 3). Then, each class's mean in the compressed latent dataset computed (line 4). Next, the ENCODE LATENT procedure is used create list containing binary encoding of each class's 's latent features (line 5). For each class, an n-bit encoding is calculated such that the ich. bit is set to 1 if the class's 's mean greater than the centroid of class means in the ith feature and if it is not. As construct our encodings from the same latent principal components, each encoding contains relevant information for learning all other encodings. This results in high inter-class poison transferability, which allows our attack to efficiently target all classes in the model's latent space. Lines 7-13 use the calculated binary encodings construct a set of poisoned samples. For each poison sample, ENCODING TRiGGER embeds the y 's binary encoding as a trigger in æ. This can be accomplished using various techniques, as described in Section 3.4
3.4 ENCODING APPROACH
Many triggers have been proposed for data poisoning attacks, each with trade-offss effectiveness
and robustness (Gu et al. 2017 Liao et al. 2018 Shafahi et al. 2018 Liu et al. 2020 Nguyen &
is chosen hyper-parameter
4

---

Published as a conference paper at ICLR 2024
Tran 2021
Doan et al. 2019). Our method can be used with any trigger that encode the binary Our paper evaluates two common trigger crafting methods: patch Chen et al. 2017).
string calculated in Section 3.3
and blend triggers (Gu al. 2017
+
Two exemplary methods of encoding latent directions. (Left) Universal Backdoor with patch trigger encoding. (Right) Universal Backdoor with a blended trigger encoding.
Figure 2:
Patch Trigger. To create a patch corresponding to the target class, we encode its corresponding binary string as a black-and-white grid and stamp it in the top left of the base image.
Blend Trigger. We partition the base image into n disjoint rectangular masks, each representing bit in the target class's binary string. We choose two colors and color each mask based its corresponding bit. Lastly, we blend the masks over the base image to create poisoned sample.
EXPERIMENTS
In this section, we empirically evaluate the effectiveness of our backdoor using different encoding methods. We extend this evaluation process to demonstrate the effectiveness of our backdoor when scaling the image classification task both the number of samples and classes. By choosing which classes poisoned, we measure the inter-class poison transferability of poison. Lastly, evaluate our Universal Backdoor Attack against suite of popular defenses.
the first study many-to-mane backdoors, there exists baseline compare against to demonstrate the effectiveness of our method. For this purpose, we develop two baseline many-to-many backdoor attacks from well-known attacks: BadNets (Gu al., 2017) and Blended Injection (Chen et al. 2017) We compare our Universal Backdoor against the effectiveness of these two bascline many-to- to-many attacks. For our bascline triggers, we generate a random trigger pattern for each targeted class, as in Gu et al. 2017). For our patch trigger baseline, we construct grid consisting of n randomly colored squares. To embed this baseline trigger, we stamp the patch into an image using the same position and dimensions as our Universal Backdoor patch trigger. For our blend trigger baseline, we blend the randomly sampled grid across the whole image, using the same blend ratio our Universal Backdoor blend trigger.
Baselines. As
4.1 EXPERIMENTAL SETUP
Datasets and Models. For our inital effectiveness evaluation, we use ImageNet-Ik with random crop and horizontal flipping (Russakovsky et al. 2014). We three datasets, ImageNet-2k, ImageNet-4k, ImageNet-6k, for our scaling experiments. These datasets comprise the largest 000, 000, and 6 500 classes from the ImageNet-21K dataset (Deng al.) 2009). These datasets con- tain 3024 392, 513 146, and 7804 447 labeled samples, respectively. We use ResNet-13 18 for the ImageNet- KK experiments and ResNet-101 101 for the experiments on ImageNet-2k, ImageNet-4k, and ImageNet-6k in Section 4.3 (He et al. 2015).
Attack Settings. We use a binary encoding with n 30 features for experiments. In our patch triggers, use an 8x8 square of pixels to embed each feature, resulting in patch that covers 3.8% of the base image. Our blended triggers use a blend ratio of 0.2, as Chen et al. (2017). We use pre-trained surrogate from Hugging Face for all of our attacks. For attacks the ImageNet- IK classification task, Hugging Face Transformers pre- pre-trained ResNet- 18 model (Wolf et al. (020). As model pre-trained on the ImageNet-2K,, ImageNet-4K, ImageNet-6K exists,
Hugging
5

---

Published as a conference paper at ICLR 2024
Face Transformer's clip-vit- sitt-base-patchtte model as zero-shot image classifier on these datasets to generate latents (Wolf et al. 2020). We use 25 images from each class to sample the latent space of surrogate model.
Model Training. We train our image classifiers using stochastic gradient descent (SGD) with a momentum of 0.9 and weight decay of 0.0001. Models trained ImageNet- 1K trained for 90 epochs, while models trained on ImageNet-2K, ImageNet-4K, and ImageNet-6K are trained for 60 epochs adjust for the larger dataset size. The initial learning rate is by a factor of 10 every 30 epochs on ImageNet-IK and every 20 epochs on the larger datasets. We use batch size of 128 images for all training runs. Early stopping is applied all training runs; we stop training when the model's 's accuracy is no longer improving or the model begins overfitting. All of our models achieve equivalent validation accuracy to pre-trained counterparts the Hugging Face Transformers library (Wolf et al. 2020). We include an analysis of backdoored models' clean accuracy in Appendix A.1
0.1 and is decreased
4.2 EFFECTIVENESS ON IMAGENET-IK
Table Attack success rate (%) of Universal Backdoor compared baseline approach.
Poison Samples (p)
Poison %
Patch
Blend
Ours
Baseline
Ours
Baseline
2000
0.16
80. 1%
0.1%
0.4%
0.1%
5000
0.39
95.5%
2.1%
74.9%
0.1%
8000
0.62
95.7%
100%
92.9%
0.1%
Table 1 summarizes our results on ImageNet-IK IK using patch and blend triggers while injecting be- tween 2 000 and 000 poisoned samples. Our patch encoding triggers perform the best, achieving over 80. ASR across all classes while only manipulating 0. 0.16% of the dataset. Our method per- forms significantly better than the baseline at low poisoning rates. The patch baseline is completely learned at high poisoning rates and achieves perfect ASR. Our chosen value of n 30 is too low to distinguish the binary encodings of all classes, resulting in our backdoor achieving less than perfect ASR even with many poison samples. A larger value of n would allow encode more principal components of the latent space, allowing our Universal Backdoor to achieve perfect ASR. However, as this would require embedding longer binary encoding, would increase the number of sample poisons required for successful attack. Across all experiments, we find that a patch encoding is more effective than a blend encoding.
Throughout Training
Inter-llass Transferability on 100 Observed Classes
1.0
1.0
Lenierual Beckdoor
Universal Backdoor
0.8
Buseiin
Early Stosping Line
Doserved Clasges
0.8
Baseline
0.6
0.4
0.2
ASR
0.2
1000 1500 2000
3000 3500 4000 4500 Total Poisors
2500
Epochs
Figure 3: Our attack versus a baseline using patch encoding triggers. We measure the attack success rate and use early stopping at 70 epochs.
Figure Attack success rate on a subset of observed target classes while increasing poi- soning in other classes in the dataset.
Figure shows that the baseline backdoor is learned only after the model overfits the training data (after about 70 epochs). Therefore, the baseline backdoor is very vulnerable to early stopping. The baseline requires significantly more poisons to ensure it is learned earlier in the training process and not removed by early stopping. Because of this behavior, the baseline backdoor is either thoroughly learned achieves negligible attack success. This results in sudden increase in the baseline's

---

Published as a conference paper at ICLR 2024
attack success when the number of poison samples increases to 8 000 in Table Our Universal Backdoor is gradually learned throughout the training process, so any early stopping procedure that would mitigate our backdoor would also significantly reduce the model's clean accuracy.
4.3 SCALING
Table 2: Attack success rate of the backdoor on larger datasets (%), using p = 12000.
Poison Attack
ImageNet-2K
ImageNet-4K
ImageNet-6K
Universal Backdoor
99.73
91.75
47.31
Baseline
99.98
0.03
0.02
In this experiment, we measure our backdoor's ability to scale to larger datasets. We fix the number of poisons the attacker injects into each dataset at p 12 000 across all runs. As larger image classification datasets naturally contain more classes and samples, so do our datasets (Deng et al.) 2009 Kuznetsova et al., 2018). Table 2 summarizes the results on our backdoor compared to baseline on the ImageNet-2K, ImageNer-4K, and ImageNet-6K image classification tasks. We find that the trigger patterns of the baseline do not effectively scale to larger image classification datasets. Although the baseline backdoor has near- near-perfect ASR on the ImageNet-2K dataset, it has negligible ASR on both the ImageNet-4K and ImageNet-6K datasets. This is because of the all-or-nothing attack success behavior observed in Section 4.2 In contrast, our Universal Backdoor can seale to image classification tasks containing more classes and samples, Our Universal backdoor achieves above 90% ASR on the ImageNet-4K task and 47.31% ASR on the largest dataset, ImageNet-6K.
4.4 MEASURING INTER-CLASS POISON TRANSFERABILITY
To measure the inter-class transferability of poisoning, we examine how increasing the number of poisons in one set of classes increases attack success on disjoint set of classes in the dataset. We divide the classes into the observed set B and the variation set A. B contains 10% of the classes in the dataset (100 classes), while A contains the remaining 90% of classes (900 classes). We use the ImageNet- 1K dataset and a patch trigger for our backdoor. We poison exactly one sample in each class in B. In Figure 4 ablate over the total number of poisons in the dataset, distributing all poisons except for the 100 poisons in B evenly in classes in A.
We find that by poisoning class with a single sample, our Universal Backdoor can achieve successful attack on a class if sufficient poisoning is achieved elsewhere in the dataset. Increasing the number of poison samples in A improved the backdoor's ASR on classes in B from negligible to over 70%. Therefore, we find that protecting the integrity of a single class requires protecting the integrity of the entire dataset. Our Universal Backdoor shows that every sample, even if they are associated with an insensitive class label, can be used by an attacker as part of an extremely poison- efficient backdoor attack on small subset of high- value classes. We provide further evidence for this in Appendix A.2 where we show that A can contain significantly fewer than 900 classes while preserving the strength of inter-class poison transferability on B. The baseline method does not demonstrate any inter-class transferability, as increasing the poisoning in A does not increase the attack success rate on
4.5 ROBUSTNESS AGAINST DEFENSES
We evaluate the robustness of our poisoning model against four state-of-the-an defenses: fine- tuning, fine-pruning (Liu et al. 2018), neural attention distillation (Li et al., 2021), and neural cleanse (Wang et al.) 2019). We use a ResNet-18 model trained on the ImageNet-1k dataset for all robustness evaluations. We use patch triggers for both our method and the baseline. For all defenses, we use hyper-parameter optimized for removing BadNets backdoor (Gu et al. (017) on ImageNet-IX 1K as proposed by Lukas & Kerschbaum (2023). Defenses requiring clean data are given 1% of the clean dataset, approximately 12,800 clean samples. We limit the degradation of the model's 's clean accuracy, halting any defense that degrades the model's 's clean accuracy by more than 2%. Table 3 summarizes the changes in ASR after applying each defense. As in Lukas &
7

---

Published as a conference paper at ICLR 2024
Kerschbaum (2023), find that backdoored models trained ImageNet- IK are robust against defenses. complete table defense parameters can be found in Appendix
Table 3: The robustness of our universal backdoor against a naive baseline, measured by the attack success rate (ASR), denotes ASR lost after applying defense. Only backdoors above 5% ASR were evaluated. Backdoors that were not evaluated marked with N/A.
Defense
Poison Samples (p)
Universal Backdoor (ASR)
Baseline (ASR)
2.000
70.3%
9.8
N/A
Fine-Tuning
5 500
94.5%
1.0
N/A
8 300
95.5%
0.2
99.5%
0.5
2 2000
73.5%
6.6
N/A
Fine-Pruning
5 000
95.2%
0.3
N/A
8 800
95.6%
0.1
99.9%
0.1
2 2.00
70.1%
10.0
N/A
Neural Cleanse
5 000
95.1%
0.4
N/A
800
95.4%
0.3
98.0%
V 2.0
2 2000
73.9%
6.2
N/A
NAD
5 000
94.6%
0.9
N/A
8 800
95.3%
0.4
99.9%
0.1
Fine-tuning. This defense fine-tunes the dataset small validated subset of the training dataset.
We fine-tune the model using the SGD with a learning rate of 0.0005 and a momentum 0.9.
Fine-pruning, As in Liu al. 2018), prune the last convolutional layer of the model. We find that the pruning rate Lukas Kerschbaum (2023) is too high and degrades the clean accuracy of the model more than the 2% cutoff. We set the pruning rate to 0.1%, which is the maximum pruning rate that prevents the defense from degrading the model below the accuracy cutoff.
Neural Cleanse. Neural Cleanse Wann et al. 2019) uses outlier detection to decide which candi- date trigger is most likely the result poisoning. This candidate trigger then used to remove the backdoor in the model. As Universal Backdoor targets every class and has unique trigger for each class, class-wise anomaly detection poorly suited for removing our backdoor.
Neural Attention Distillation. We train teacher model for 000 steps using SGD. We then align the backdoored model with the teacher for 8000 steps, using SGD with a learning rate of 0.0005. We power of for the attention distillation loss, as recommended in Li al. (202))
4.6 MEASURING THE CLEAN DATA TRADE-OFF
Clean Data Required to Remove Backdoor
There is a known trade-off between the availability of clean data and the effectiveness of defenses (Li al. (021). [ the proportion of the clean dataset required remove the universal back- door with fine-tuning without degrading the model below the 2% cutoff. For this experiment, a ResNet-18 model backdoored using 2 000 poison samples on the ImageNet- 1K dataset. Due to the higher availability of clean data, we find that a higher learning rate of 0.001 and weight decay of 0.001 appropriate.
1.0
0.8
0.6
A5R
0.4
0.2
0.0
2.5
10
20
40
Clean
Data poisoning defenses for backdoored models trained on web-scale datasets must be effective with a validated dataset that is small portion of the train- ing dataset due the cost of manually validating samples. Validating 1 1% portion our web-scale inspecting 78 000 samples, task larger than inspecting the CIFAR-100 GTSRB datasets in
Figure Clean data percentage of the training dataset size required to remove our Universal Backdoor.
ImageNet-64 dataset would require manually
8

---

Published as a conference paper at ICLR 2024
Stallkamp al. 2011). We find that approximately 40% (512 samples) of the clean dataset is required to completely remove our Universal Backdoor, which is more data than most victims can manually validate.
their entirety (Krizhevsky) 2009;
DISCUSSION AND RELATED WORK
Attacking web-scale datasets. Carlini et al. (2023) demonstrate two realistic ways an attacker could poison a web-scale dataset: domain hijacking and snapshot poisoning. They show that more than 0.15% of the samples these online datasets could be poisoned by an attacker. However, existing many-to-one poison label attacks cannot exploit these vulnerabilities,
they require compromising many samples in a single class (Gu et al. 2017 Qi et al. 2022 Chen et al. 2017). As web-scale datasets contain thousands of classes (Deng al. 2009: Kuznetsova et al. 2018). it is improbable that any one class would have enough compromised samples for a many-to-one poison label attack.
By leveraging inter-class poison transferability, our backdoor can utilize compromised samples out- side a class the attacker is attempting to misclassify into.
Scaling larger datasets. The largest dataset we evaluate is our ImageNet-6K dataset, which con- sists of 000 classes and 7 804 447 samples. We created a Universal Backdoor in a model trained on this dataset while poisoning only 0.15%. As our backdoor effectively scales to datasets contain- ing more classes and samples, we expect smaller proportion of poison samples to be required to backdoor models trained on larger datasets, like LAION-5B (Schuhmann et al. (022).
Alternative methodology for targeting multiple classes at inference time. Although we are the first to study how to target every class in the data poisoning setting, other types of attacks, like adversarial examples, can be used to target specific classes at inference time (Wu et al.] [203]] Goodfellow et al., 2015). Through direct optimization on an input, the attacker finds an adversarial perturbation that acts as trigger; adding it to the input causes a misclassification. Defenses against adversarial examples seek to make models robust against adversarial perturbations (Cohen et al., 2019 Geiping et al.] 2021). However, as data poisoning backdoors utilize triggers that are not adversarial perturbations, these defenses are ineffective at mitigating data poisoning backdoors.
Limitations. We focus on patch and blend triggers that are visible modifications to the image and hence could be detected by a data sanitation defense. Our attacks are agnostic to the trigger; even specific trigger could be reliably detected, universal backdoors remain threat because the attacker could have used a different trigger. Koh et al. (2022) demonstrate that no detection has been shown effective against any trigger. However, evading data sanitation comes at a cost for the attacker: Less detectable triggers are less effective at equal numbers. Hence, the attacker must inject more to create an equally effective backdoor (Frederickson our attacks still remain difficult to detect using STRIP (Gao et al., 2019) due to the high false positive rate. We focus on the feasibility of universal attacks and do not study the iieectablity--fffettvvee trade-off of triggers with our attacks. Moreover, we focus on poisoning models from scratch, as opposed to poisoning pre-trained models that are fine-tuned. More research is needed to analyze the effectiveness of our attacks against large pre-trained models like ViT and CLIP (Dosovitskiy et al.) 2021 Radford et al. 2021) that are fine-tuned on poisoned data. Finally, we assume that the attacker can access similarly accurate surrogate classifiers to generate latent encodings for attacks.
al. (2018). We point to Appendix [A.3] showing that
CONCLUSION
We introduce Universal Backdoors, data poisoning backdoor that targets every class. We establish that our backdoor requires significantly fewer poison samples than independently attacking each class and can effectively attack web-scale datasets. We also demonstrate how compromised sam- ples in uncritical classes can be used to reinforce poisoning attacks against other more sensitive classes. Our work exemplifies the need for practitioners who train models on untrusted data sources to protect the whole dataset. not individual classes, from data poisoning. Finally, we show that exist- ing defenses are ineffective at defending against Universal Backdoors, indicating the need for new defenses designed to remove backdoors that target many classes.
9

---

Published as a conference paper at ICLR 2024
REFERENCES
Rasmus Adler, Mohammed Naveed Akram, Pascal Bauer, Patrik Feth, Pascal Gerber, Andreas Jedl- itschka, Lisa Jöckel, Michael Kläs, and Daniel Schneider, Hardening of artificial neural networks for use in safety-critical applications A mapping study. CoRR, 0t5/1909.030050 2019. URL
http://arxiv. http://wivvrga////aeeaaa ab/// 03036
Naif Alkhunaizi, Dmitry Kamzolov, Martin Takáč, and Karthik Nandakumar, Suppressing poisoning attacks on federated learning for medical imaging, 2022.
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Chood Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramër. Poisoning web-scale train- ing datasets practical, 2023.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. CoRR, abs/1712.05526, 2017. URL http://arxiv.
org/ axg/abs/1712. 05526
Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A. Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, and Fabio Roli. Wild patterns reloaded: A survey of machine learning security against training data poisoning. ACM Comput. Surv., 55(13s), jul 2023. ISSN 0360-0300. doi: 10.1145/3585385, URL htt.h:ttps//cccct orgg
10. 1 14411145/5 35855555535
Jeremy Cohen, Elan Rosenfeld, and J. Zico Kolter. Certified adversarial robustness via randomized smoothing. CoRR, abs/1902.02218,, 2019. URL http:://axxii.orrrtttt www.:/iiirrrr..ooooo aaw/////b///// /902. 02918
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-258225 2009. doi: 10. RTL.1//VRR..009.52222
Bao Gia Doan, Ehsan Abbasnejad, and Damith Chinthana Ranasinghe. Deepcleanse: Input saniti- zation framework against trojan attacks on deep neural network systems. CoRR, 0bss/1908.03369 2019. URL:ttt://aaxxi org/ als org. 1908. C3369
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. WRRLhttps://openeeee/ id=YicbF dNTTY
net forum?
R. A. FISHER. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7()):199-188, 1936. doi: http://tbi.org/10.1111// 180446999---9-00 100. 1936...11..3 URL https::/omlinellbrrr / 1936. Eb02137.x
abs
Christopher Frederickson, Michael Moore, Glenn Dawson, and Robi Polikar. Attack strength vs. detectability dilemma adversarial machine learning. 2018 International Joint Conference on Neural Networks (IJCNN) pp. 1-8, 2018. doi: 10.1 (09///CNN.201....0 888889999.
Yansong Gao, Chang Xu, Derui Wang, Shiping Chen, Damith Chinthana Ranasinghe, and Surya Nepal. STRIP: A defence against trojan attacks on deep neural networks. CoRR, abss/190..0631,, 2019. URL httt::///xiiv.ot///
902. 06531
Jonas Geiping, Liam Fowl, Gowthami Somepalli, Micah Goldblum, Michael Moeller, and Tom Goldstein. What doesn't 't kill you makes you robust(er): Adversarial training against poisons and backdoors. CoRR, abs/2102.13624 2021. URL. anttts::/aardvv.or///tttt
13624
Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, and Tom Goldstein. Dataset security for machine learningg Data poisoning, backdoor attacks, and defenses. CoRR, abs/2012.10544, 2020. URL https:// arxiv. wrt///b////////// 10544]
10

---

Published as a conference paper at ICLR 2024
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples, 2015.
Tianyu Gu, Brendan Dolan DovintGavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. CoRR, ftbs/1708.067335 2017. URL http://arxiv..
org/abs/ 1708. 06733
Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. SPECTRE: defending against
backdoor attacks using robust statistics. CoRR, abs/2104.111355, 2021. URL htthhtps///// //attps://axiv.
org/abs/210e 2104. 11315
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep learning for image recog- nition. CoRR,
2015. UBL.httt:://ariii..rr///bb wttp://wwwww..//////////...... 03320
Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Handcrafted backdoors deep neural networks. CoRR, abs/2106.04690 2021.
6460
Alaa Khaddaj, Guillaume Leclere, Aleksandar Makelov, Kristian Georgiev, Hadi Salman, Andrew Ilyas, and Aleksander Madry. Rethinking backdoor attacks, 2023.
Pang Wei Koh, Jacob Steinhardt, and Percy Liang. Stronger data poisoning attacks break data sanitization defenses. Machine learning, 11 111(1):1-47, 2022. ISSN 0885-6125.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper R. R. Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Tom Duerig, and Vittorio Ferrari. The open images dataset V4: unified image classification, object detection, and visual relationship detection wwwp:/wwww..ooaooooooaooeeeeeeee abs 1 1811. 00982
at scale. CoRR, abs//811.0we 100982, 2018. URL
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Frasing backdoor triggers from deep neural networks. CoRR, abs/2101.05930, 2021.
URL https: //apxii//aaxvv..///a//aoa org/ httpg//b///11111 05930
Cong Liao, Haoti Zhong, Anna Cinzia Squicciarini, Sencun Zhu, and David Miller. Back- door embedding in convolutional neural network models via invisible perturbation. CoRR, abs/1808. 10307, 2018. URL /tt//ttsp///rrx..ooor//a////////
10307
Kang Liu, Brendan Dolan-Gavity and Siddharth Garg. Fine-pruning: Defending against backdoor- ing attacks on deep neural networks. CoRR, abs/1805, 12185, 2018. URL. htttp://axxiv.
org ab///bb//aa 1805 12185
Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. CoRR, abs/2007.02343, 2020. URL https: http://///xxvv..r..../ abs/2007, 02343
Nils Lukas and Florian Kerschbaum. Pick your poison: Undetectability versus robustness in data poisoning attacks, 2023.
Tuan Anh Nguyen and Anh Tuan Tran. Wanet - imperceptible warping--bbeddee backdoor attack. CoRR, 102/200021039, 2021. URL /RLlitttps//rrrvv.rrrraassss 2002../..222222219.9
Naman Patel, Prashanth Krishnamurthy, Siddharth Garg, and Farshad Khorrami. Bait and switch: Online training data poisoming of autonomous driving systems. CoRR, abs/2011.04066 2020. URL httphht/////xiiiiee org/ /b1// 04065.
Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal, Circumventing back- door defenses that are based latent separability. arXiv preprint arXiv:2205. 13613, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. CoRR, abs/2103.00020, 2021. https:::::/// wow..org/aaao... 03. 00020

---

Published as a conference paper at ICLR 2024
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. CoRR, abs/1409.0575. 2014. URL
httt:://axxiv.orrt wttpaab///444000 1409.0575
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022.
Ali Shafahi, W. Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks. CoRR, abs/1804.00792,, 2018.
abs 1804.0000000 00792
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Igel. The german traffic sign recognition benchmark: A multi-class classification competition. In The 2011 International Joint Conference on Neural Networks, pp. 1453-1460, 2011. doi: 10. 10.1109//CNN.2000 .6033395.
Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In 2019 IEEE Symposium on Security and Privacy (SP), pp. 707-723, 2019. doi: 10. 10.1109/SP:211.00001.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gug- satte-of-the-ar natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38-45, Online, October 2020. As- sociation for Computational Linguistics. URL https::////.......... /www. ac www..webb..mm org/ ant ho ogy/ 2020. -emp----emm....
ger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers:
Baoyuan Wu, Li Liu, Zihao Zhu, Qingshan Liu, Zhaofeng He, and Siwei Lyu. Adversarial machine learning: A systematic survey of backdoor attack, weight attack and adversarial example, 2023.
A APPENDIX
Table contains a complete summary of all the parameters used to evaluate defenses against our backdoor in Section 4.5 All defense parameters are adapted from Lukas & Kerschbaum (2023). where they were optimized against BadNets (Gu et al. 2017)) patch trigger. When hyperparameter tuning for fine-tuning and fine-pruning defenses,
find significant improvements over the set- tings described in Lukas Kerschbaum (2023). We reduce the fine-pra mie-ppuunng rate in Fine-pruning, as find it degrades the model's 's clean accuracy below our 2% cutoff.
As shown by Figure 6a| a linear trade-off exists between the effectiveness of defenses and the al- lowed clean accuracy cutoff. If the defender allows for more clean accuracy degradation, the effec- tiveness of the backdoor can be further reduced. This does not apply to all defenses, as defenses like neural cleanse (Wang et al. 2019) do not significantly reduce clean accuracy.
A.1 ANALYSIS OF CLEAN ACCURACY
If a backdoor attack degrades the clean accuracy of model, then the validation set is sufficient for the victim to recognize the presence of a backdoor (Gu et al. 2017). Therefore, model trained on the poisoned set should achieve the same clean accuracy as one trained comparable clean dataset. We find that our backdoored models have the same clean accuracy across all runs as a model trained on entirely clean data. We train clean ResNet- 18 model on ImageNet- 1k (Russakovsky jet al.) 2014), which achieves 68.49% top-1 accuracy on the validation set. Table 5 shows the clean accuracy of backdoored models the ImageNet-1k dataset.
12

---

Published as a conference paper at ICLR 2024
Table Defense Parameters on ImageNet-Il IK from Lukas & Kerschbaum 2023
Neural Attention Distillation
n steps /N
8,000
Fine- Tine-Tuning
opt
sgd
steps N
5,000
I a
5e-4
teacher steps
1,000
opt
sgd
Ir f
5e-4
power p
weight decay
0.001
lambda Aot
1,000
batch size
128
weight decay
batch size
128
Fine-Pruning
Neural Cleanse
steps N opt
5,000
sgd
n steps /N
3,000
a
5e-4
opt
sgd
prune rate / p
10%
/ a
5e-4
sampled batches
steps per class / N1
200
weight decay
0
norm lambda Ax
le-5
batch size
128
weight decay
0
batch size
128
Anne Tuning ASR/CDA
STRIP
8.825
3,000
0.775
B0D0c SucOus R300
True Positive Rate
0.750
0.6
8.075
0.2
Patch Trigger AUC-0.875
Bland "Tipper ALC-0 687
0.600
Euessing
0.69
0.58
0.57
0.56
0.0
02
0.6 False Positive Rate
0.4
088
1.0
clean Data Accuracy
Trade-off between attack success rate and clean data accuracy when fine-tuning backdoored model. (6b) ROC curve of our Universal Backdoor with patch and blend triggers (see Figure 2 when applying the STRIP (Gao et al. 2019) defense.
Figure 6: (6a
A.2 INTER-CLASS POISON TRANSFERABILITY WITH SMALL VARIATION SETS
Table shows that even if the number of classes in the variation set A is reduced to only 10% of classes in )), inter-class poison transferability maintains its effect on the observed set B. This results in an otherwise unsuccessful attack on classes in B, achieving a success rate of 67.72%. Therefore, if the attacker can strongly poison small set of classes in the dataset, attacking other classes in the model can easily be accomplished, as inter-class poison transferability remains strong. To protect tiny subset of high-value classes, the victim must maintain the integrity of every class within their dataset.
A.3 DATA SANITATION DEFENSES
Several data sanitization defenses are also poorly suited to Universal Backdoors. SPECTRE (Hayase et al. 2 only removes samples from single class by design, and therefore could remove most 0.1% our Universal Backdoor's poisoned samples on ImageNets- IK. STRIP (Gao et al. 2019) struggles to detect trigger, resulting in high false positive rate, shown in Figure The
13

---

Published as a conference paper at ICLR 2024
Table 5: Clean accuracy of backdoored models on ImageNet- Ik dataset.
Poison Samples (p)
Poison %
Patch
Blend
Ours
Baseline
Ours
Baseline
2000
0.16
68. 98.94%
68.94%
68.51 %
69.43%
5000
0.39
68.92%
68.89%
68. 77%
68.66%
8000
0.62
68.91 %
69.43%
69.78%
69.22%
Table 6: Effect of the number of classes in the variation set A on attack success on the observed set
B. All experiments use 600 poison samples.
Percentage of classes in A
ASR on classes in B
90%
72.77%
60%
70.45%
30%
71.78%
10%
67.72%
under the ROC curves are 0.879 and 0.687 for the patch and blend triggers, respectively. It may be difficult for defenders to detect both triggers for large datasets (1 million samples more) due to the detection's high FPR. Considering a maximum tolerable FPR of 10%, the defender misses 39% of the patch trigger samples and 68% of the blended triggers.
A.4 CLASS- WASSS-WISE ATTACK SUCCESS METRICS
Our method does not achieve even attack success across all classes in the dataset. Table 7 shows statistics our Universal Backdoor's success rate across classes ImageNet- We find that some classes are more challenging to achieve a successful attack against our backdoor: This differs from the baseline, as the bascline either performs near-perfectly or not at all.
Table 7: ASR metrics across classes in ImageNet- IK
Poison Samples (p)
Min
Max (%)
Mean
Median
2000
100%
81.0%
98.0%
5000
0
100%
95.4%
100%
14


Reviews:
[{"review_id": "hNWh5JFvFX", "reviewer": "Reviewer_H32b", "summary": "Whereas in traditional backdoor literature attacks focus on a specific target class, the proposed work introduces a method to embed backdoors from any source class to any target class. The method proceeds in three steps: 1) finding the class-wise centroids of clean-data feature extractions (using CLIP), 2) encoding each centroid into a N-dimensional bit-string, and 3) generating triggers corresponding to each bit-string (and, hence each target class). Classes with similar features are encoded to have similar embeddings. They show that their method performs and scales well with ResNets on four ImageNet-21k subsets.", "questions": "There are a few questions embedded in the above weaknesses. In addition to those I'm curious about the effect of pretraining on the proposed attack. Could the attack be injected in a fine-tuning regime?\n\n**Note:** I'm happy to raise my score after the weaknesses and questions have been addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The writing was clear and easy to follow\n- Their bit-string encoding approach is a novel and elegant way to share feature information between classes while generating a class-specific backdoor trigger.\n- The experiments section was well-motivated and well-explained.", "weaknesses": "- In general, each experiment should be averaged over multiple seeds for statistical significance\n- A major part of the backdoor attack regime is the preservation of clean accuracy, and there is no analysis on how well the proposed method protects a model's clean accuracy. This should certainly be included in future versions of the paper.\n- The proposed triggers in Fig. 2 seem quite obvious to the human eye and may be susceptible to input-space defenses. I would like to see some analysis on the necessary intensity of these triggers and their brittleness to input-space defenses like STRIP.\n- On the defense side, the authors \"[halt] any defense that degrades the model’s clean accuracy by more than 2%.\" I'm open to feedback here, but this has the potential to straw-man some defense mechanisms in scenarios where removing a backdoor is worth the cost of clean accuracy. Including some results without this limitation would be nice.\n- In addition to the above, the attack was not evaluated on data-cleaning defenses like SPECTRE, which I think would be particularly effective against this regime. I would like to see these defenses evaluated as well--and not limited to specific target classes.\n- The experiments are limited to ResNet variants. It would be nice to show generality by including one other architecture in the experiments section.\n  - Since most vision models rely on pretraining, one idea I would find particularly compelling would be to run the attack on a pretrained ViT.\n- In Section 4.4, only a single setting of observed percentage is tried. The analysis here would be stronger if more percentages were tried\n- I'm not sure about the timing here, but the authors claim that they \"are the first to study how to target every class in the data poisoning setting.\" However, while [1,2] address slightly different settings, they seem to be *at least* related and possibly published earlier.\n  - Depending on the nature of this relationship, I would like to see 1) these statements qualified, 2) a more thorough analysis of how the work is positioned in relation to similar work including but not limited to the papers mentioned.\n\n**Citations:**\n\n[1] Du et al., \"UOR: Universal Backdoor Attacks on Pre-trained Language Models.\"\n\n[2] Zhang et al., \"Universal backdoor attack on deep neural networks for malware detection.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1fZ60Zaof8", "reviewer": "Reviewer_czTr", "summary": "This paper investigates the utilization of a small number of poisoned samples to achieve many-to-many backdoor attacks. The authors leverage inter-class poison transferability and generate triggers with salient characteristics. The proposed method is evaluated on the ImageNet dataset, demonstrating its effectiveness. The authors provide evidence of the transferability of data poisoning across different categories.", "questions": "1.\tThe related work lacks a specific conceptual description of \"many-to-many\" and an introduction to recent works in this area.\n2.\tIn Section 3.3, the encoding method used in the latent feature space is rather simplistic, where values greater than the mean are encoded as 1 and others as 0. What is the motivation behind this encoding method, and how does it contribute to improving the transferability of inter-class data poisoning?\n3.\tThe author employs a patch and blend approach to add triggers, resulting in poor concealment of the backdoor triggers. Visually, the differences between poisoned and clean samples can be distinguished. Has the author considered more covert methods for backdoor implantation, such as injecting triggers in the latent space and decoding them back to the original samples to reduce the dissimilarity between poisoned and clean samples?\n4.\tSelection of baselines. The chosen comparative methods are both from 2017. It is recommended to include comparative experiments with the latest backdoor attack methods.\n5.\tThe experimental results in the paper compare the average attack success rates across all categories. It is suggested to provide individual attack success rates for representative categories or other statistical results such as minimum, maximum, and median values.\n6.\tThe authors validated the effectiveness of the method under model-side defense measures. It is recommended to include defense methods in data-side.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.The paper demonstrates clear logic.\n2.The topic is intriguing and warrants further exploration.", "weaknesses": "1.The design motivation of the algorithm is unclear.\n2.The concealment of the patches is poor.\n3.The comparative methods are outdated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RnqJwNElWt", "reviewer": "Reviewer_Xx2A", "summary": "This paper introduced a universal backdoor attack, a data poisoning method that targets arbitrary categories. Specifically, the authors crafted triggers by utilizing the principal components of LDA in the latent space of a surrogate classifier. Experiments showed that the generated triggers can attack any category by poisoning a certain percentage of samples in the training data.", "questions": "What are the requirements for the surrogate image classifier? The proposed method requires sampling in the latent space of the surrogate image classifier, not the original classifier. Is it possible to use any latent space of any surrogate classifier? For example, if there is a significant difference in the distribution of the hidden spaces between the surrogate classifier and the original classifier, will this result in a significant decrease in the attack success rate of the proposed method?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "$\\bullet$ The authors proposed a method that was designed to poison any class, instead of targeting a single class.\n\n$\\bullet$ The proposed attack is effective than the previous method, especially when the poisoning rate is low.", "weaknesses": "$\\bullet$ It is not clear why the proposed method improves the inter-class poison transferability and, in particular, how it ensures that an increase in attack success against one class improves attack success against other classes. Does the proposed method increase the transferability (attack success rate) of any two classes, even if these two classes differ significantly in the latent space?\n\n$\\bullet$ The formula in Section 3.2 needs to be formulated more appropriately and clearly. Specifically, do y' and y in the formula refer to any two categories or any two similar categories? If they refer to any two categories, please explain why categories that are very different in the latent space can also improve the success rate of the attack; otherwise, if they refer to any two similar categories, please give a clear definition of similarity.\n\n$\\bullet$ The experimental results require further discussion and analysis. For example, in Table 1, the proposed method significantly outperforms the baseline method when the poisoning samples are 5000 (i.e., the attack success rate is 95.5% vs. 2.1%), but the proposed method is suddenly worse than the baseline method when the poisoning samples are 8000 (95.7% vs. 100%). The potential reasons for the sudden improvement in the performance of the baseline method need to be discussed. Similarly, in Table 2, the attack success rate of the baseline method suddenly drops from 99.98% for ImageNet-2K to 0.03% for ImageNet-4K, which also needs to be discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y0sigSVVqG", "reviewer": "Reviewer_EfVD", "summary": "The paper presents a new approach for crafting universal backdoor attacks, i.e. backdoor attacks that target several classes at inference time, as opposed to traditional backdoor attacks that target a single class. In order to mount a universal backdoor attack, the adversary crafts triggers that increase the ASR on several classes simultaneously.  To that end, the authors leverage a pretrained model to extract the feature representation of the training samples, and then craft triggers that correlate with features used by samples from several classes. \n\nThe authors evaluate their attack on several subsets of ImageNet-21k, and against BadNet's baseline presented in Guo et al. By poisoning 0.39% of the training data, the authors are able to mount an effective backdoor attack when no defense is applied. The authors then test the effectiveness of their attack when several defenses are applied, and notice a drop in ASR although the attack remains effective. \n\nFinally, in order to test how much triggers applied to a single class help triggers applied to other classes, the authors fix the number of triggers in some classes, then vary the number of triggers in other classes, and observe the ASR over the fixed classes increases as more poisoned samples are added to other classes.", "questions": "- can you please look into a setup with fewer poisoned samples? it should be possible to have a successful backdoor attack with close to 500 samples on ImageNet\n- can you also tune the parameters of the defense against each attack you are considering?\n-  if possible, can you provide a good baseline for attacks to compare against?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- the paper presents an interesting approach to backdoor attacks where triggers affect several classes simultaneously\n- the authors validate the effectiveness of their attack on a large scale dataset, and against several defenses", "weaknesses": "- the required number of poisoned samples seems a bit high, even for imagnet. other papers have shown that around 300-500 samples are enough to mount an effective backdoor attack [1, 2]. this is in contrast with the results observed in Table 1, where the baseline attack is not successful even with 2k poisoned samples.\n- the authors only consider a single baseline model against which their attack is compared. this comparison is helpful, however, given the large number of poisoned samples required, it would be nice to see how other baselines would compare at that scale\n- the parameters of the defenses were tuned for a simple baseline (BadNets). the effectiveness of the attack might be very different if the parameters of the defense were tuned to the authors' attack\n\n[1] POISONING AND BACKDOORING CONTRASTIVE LEARNING, Carlini et al., 2022\n[2] WITCHES’ BREW: INDUSTRIAL SCALE DATA POISONING VIA GRADIENT MATCHING, Geiping et al., 2021", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)