Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attacks
Abstract
A powerful category of (invisible) data poisoning attacks modify a subset of training examples by small adversarial perturbations to change the prediction of certain test-time data.
Existing defense mechanisms are not desirable to deploy in practice, as they often either drastically harm the generalization performance, or are attackspeciﬁc, and prohibitively slow to apply.
Here, we propose a simple but highly effective approach that unlike existing methods breaks various types of invisible poisoning attacks with the slightest drop in the generalization performance.
We make the key observation that attacks introduce sharp local regions of high training loss, which when minimized, results in learning the adversarial perturbations and makes the attack successful.
To break poisoning attacks, our key idea is to alleviate the sharp loss regions introduced by poisons. To do so, our approach comprises two components: an optimized friendly noise that is generated to maximally perturb examples without degrading the performance, and a randomly varying noise component.
The combination of both components builds a very light-weight but extremely effective defense against the most powerful triggerless targeted and hidden-trigger backdoor poisoning attacks, including Gradient Matching, Bulls-eye Polytope, and Sleeper Agent.
We show that our friendly noise is transferable to other architectures, and adaptive attacks cannot break our defense due to its random noise component.
Introduction
Big datasets empower modern over-parameterized deep learning systems.
Such datasets are often scraped from the internet or other public and user-provided sources.
An adversary can easily insert a subset of malicious examples into the data collected from public sources to harm the model’s behavior at test time.
As a result, deep learning systems trained on public data are extremely vulnerable to data poisoning attacks. Such attacks modify a subset of training examples under small (and potentially invisible) adversarial perturbations, with the aim of changing the model’s prediction on speciﬁc test-time examples.
Powerful attacks generate poisons that visually look innocent and are seemingly properly labeled [10, 15, 34].
This makes them hard to detect even by expert observers.
Hence, data poisoning attacks are arguably one of the most concerning threats to modern deep learning systems [19].
Various types of poisoning attacks have been proposed to challenge and exploit the vulnerabilities of deep learning systems.
Backdoor data poisoning attacks add a ﬁxed but not necessarily visible trigger pattern to a subset of training data as well as the test-time target examples [13, 34, 39].
Triggerless poisoning attacks add small bounded perturbations to a subset of training examples to make them similar to the adversarially labeled test-time target in the feature or gradient space [2, 12, 15, 32, 44].
In both cases, training or ﬁne-tuning the model on the poisoned training data causes the model to misclassify certain target examples at test time.
There have been sustained efforts to design effective defense mechanisms [1, 7, 11, 17, 25, 27, 33, 38, 43].
However, existing methods are highly impractical to be employed in real-world deep learning pipelines.
Firstly, the majority of the existing methods are attack speciﬁc and cannot protect the system against various types of data poisoning attacks [11, 27, 38].
Secondly, the provided protection is often at the expense of signiﬁcantly dropping the performance of the machine learning pipeline [1, 7, 25].
Thirdly, existing methods are not effective in protecting the deep learning pipelines against adaptive attacks which can make more powerful poisons with the knowledge of the defense in place [17, 33].
Finally, state-of-the-art defense methods are often so expensive that can hardly be applied to even medium-sized datasets [11, 27], and are ineffective in presence of larger number of poisons [7, 11, 27].
In this work, we propose a simple and powerful defense, namely Friendly Noise Defense (F RIE ND S ), against various types of visually imperceptible data poisoning attacks.
In particular, we make the following key observation: data poisoning attacks introduce sharp local regions of high training loss by adding adversarial perturbations to a subset of training examples.
To effectively break poisoning attacks, our proposed method is composed of two noise components. First, we ﬁnd the maximum perturbation that can be added to every example without considerably changing the model’s output.
This ﬁxed accuracy-friendly perturbation is found early in training and is transferable to other architectures.
Then, we add a small varying random noise in addition to the friendly perturbation to each example at every training iteration.
Effectively, the two components alleviate the local high training loss regions, and do not allow the attacks to be successful.
Despite being very lightweight, F RIE ND S can effectively protect deep learning systems against various types of (invisible) poisoning attacks, with a minimum drop in the generalization performance.
We note that the random noise component of F RIE ND S makes it extremely difﬁcult for an adaptive attacker to break our defense.
Adaptive attacks can bypass defenses by taking the defense mechanism into account when generating poisons.
For F RIE ND S , while an attacker may use the knowledge of the optimization procedure to bypass the friendly noise component, they need to take into account a prohibitively large number of random noise combinations when generating attacks.
This makes it extremely difﬁcult for the attacker to ensure the effectiveness of an attack in presence of our F RIE ND S defense.
Through extensive experiments, we show that our light-weight method renders state-of-the-art visually imperceptible poisoning attacks, including Gradient Matching [10], Bullseye Polytope [2], Feature
Collision [32], and Sleeper Agent [34] ineffective, with only a slight decrease in the performance.
We also show that the optimized noise component generated based on a particular architecture can be applied to defend other architectures against data poisoning attacks.
Therefore, it is easy to apply
F RIE ND S to real-world deep learning pipelines with minimal additional costs.
Related Work
Targeted Data Poisoning Attacks.
Data poisoning attacks on deep networks have been explored along two directions - triggered and triggerless attacks.
Triggered attacks, or backdoor attacks, aim to misclassify samples containing a ‘trigger’ patch as a pre-determined target class during inference time.
In the transfer learning or ﬁnetuning setting, earlier works [8, 13, 23] relied on label modiﬁcations or unbounded image perturbations.
These attacks are, however, easy to detect.
Subsequently, [30, 34, 39] introduced clean-label and visually imperceptible backdoor attacks.
Recently, [34] proposed the ﬁrst clean-label hidden backdoor attack that is effective on victim models trained from scratch.
Triggerless data poisoning attacks aim to misclassify a given target as a pre-determined adversarial class by adding optimized bounded perturbations to a subset of training examples.
Such attacks either optimize for feature matching [2, 32, 44] or gradient matching [10] between poisoned and target images, or use meta-learning to solve the poisoning problem directly via bilevel optimization [15] .
Defense Strategies.
Existing defenses against data poisoning can be divided into ﬁltering and robust training methods.
Filtering methods detect outliers in feature space using thresholding [35] and nearest neighbors [27], or activation space [7], or through decomposition of the feature covariance matrix [38].
These defenses typically assume that only small subsets of the data are poisoned, hence removing such points does not signiﬁcantly harm generalization.
In practice, this assumption may not hold, and such defenses can be easily broken by increasing the number of poisons.
Moreover, such methods increase training time by orders of magnitudes, as the ﬁltering step requires training the model with poisons, followed by (usually expensive) ﬁltering, and model retraining [7, 27, 35, 38].
Very recently, [43] proposed an efﬁcient method, to iteratively drop examples with isolated (outlier) gradients.
In comparison, our method is faster, easy to apply, and transferable to other architectures.
Hence, it is suitable for deployment in real-world deep learning pipelines.
Robust training methods apply randomized smoothing [41], strong data augmentation [4], or model ensembling [21].
Other methods impose constraints on gradient magnitudes and directions [14], detects and removes poisons with gradient ascent [22], or apply adversarial training [11, 25, 37].
Deferentially private (DP) training methods have also been explored to defend against data poisoning [1, 5, 16].
Robust training techniques usually involve a signiﬁcant trade-off between generalization and poison success rate [1, 14, 22, 25, 37], or are computationally very expensive [11, 25].
Compared to augmentation-based and adversarial training methods, our method is simple, fast, and maintains good generalization performance.
Compared to data augmentation, the random noise component of F RIE ND S is considerably more effective in smoothing the loss landscape, due to its much larger space of independent pixel-level transformations.
Random and Adversarial Noise.
It is shown that small perturbations can result in large changes in the output of a deep network [36].
Hence, the application of random and adversarial noise has been studied in various domains.
In particular, [28] used Gaussian noise to defend against query-based black box attacks, and [29] showed that additive augmentations of Gaussian or Speckle noise is a simple yet very strong baseline for robustness against image corruptions.
The application of optimized noise has been mainly studied in the context of adversarial training.
[6] used Generative Adversarial
Networks (GANs) to generate adversarial perturbations, and [24] relied on meta-learning to learn a noise generator to defend against adversarial perturbations.
Moreover, [26, 42] demonstrated the transferability of adversarial perturbations across architectures and domains.
In data poisoning, small random noise generated from a particular distribution has been shown to be ineffective for breaking attacks and harmful to the generalization performance [11].
In contrast, we show that random noise combined with our proposed noise optimization approach, can make a highly effective defense mechanism against data poisoning attacks and achieve a superior generalization performance.
[Figure]
F RIE ND S : Friendly Noise Defense against Data Poisoning Attacks
Targeted data poisoning attacks modify a fraction of training data points by adding optimized perturbations that are within an l ∞ -norm ξ -bound.
The optimization is done with the objective of changing the prediction of a target example x t in the test set, to an adversarial label y adv .
A small perturbation bound ξ ensures that the poisoned examples remain visually similar to the original (base) training data points.
Poisons crafted by such attacks look innocent to human observers and are seemingly labeled correctly.
Hence, they are called clean-label attacks.
[-Initialize noise ϵ 
-]
Targeted clean-label data poisoning attacks can be formulated as the following bilevel optimization problem:
[Equation]
where C = { δ ∈ R n × m : (cid:107) δ (cid:107) ∞ ≤ ξ, δ i = [-
-]0 ∀ i / ∈
V p } is the constraint set deﬁning the set of valid poisons, V is the training data, and V p is the set of poisoned training examples.
To address the above optimization problem, powerful poisoning attacks such as Meta Poison (MP) [15], Gradient
Matching(GM)
[10], Bull-eyes Polytope (BP) [2], and Sleeper Agent [34] craft the poisons to mimic the gradient (equivalently representation in transfer learning) of the adversarially labeled target, i.e.,
[Equation]
Minimizing the training loss on RHS of Eq.(1) also minimizes the adversarial loss on LHS of Eq.
3.1 Powerful Poisons Introduce a Sharp Local Region with High Training Loss
Based on Eq.
(2), we make the following observation.
To substantially change the gradient of a training example x i to match the adversarial gradient, i.e., ∇L ( x t , y adv , θ ) ≈
∇L ( x i
+ δ i , y i , θ ) , under bounded perturbation (cid:107) δ
i (cid:107) ∞ ≤ ξ , the attacker needs to exploit the highly non-convex nature of the loss. That is, the attacker needs to ﬁnd regions in a ball of radius ξ around example x i , for which ∇L ( x i + δ i , y i , θ ) is considerably different than L ( x i , y i , θ ) . Fig.
2 illustrates the matching loss between the gradient of the adversarial loss and the gradient of a perturbed training example, as we vary the perturbation δ i projected along two randomly chosen directions. The matching loss is deﬁned as 1 − cos ( ∇L ( x t , y adv , θ ) , ∇L ( x i + δ, y i , θ )) , where cos ( u, v ) is the cosine similarity between vectors u and v . We see that the adversarial perturbation δ i can be effectively optimized to minimize the matching loss in the darker valleys around x i . Such valleys do not exist around all the examples, and hence not every example can be perturbed in a ball of radius ξ to match the adversarial gradient.
Indeed, the examples that can be perturbed by (cid:107) δ i (cid:107) ≤ ξ to closely match the adversarial gradient are effective poisons [43], that make the attack successful. Crucially, each effective poison introduce a slight local increase to the training loss, as demonstrated by Fig. 2(c). The set of effective poisons together introduce a sharp local region with a considerably high training loss , as illustrated
[Figure] by Fig 3(a). Minimizing the training loss on the poisoned data results in learning the adversarial perturbations, and hence a successful attack.
As poisons need to match a particular gradient or representation, they are highly sensitive to small perturbations.
In Fig. 5(c) in the Appendix, we show that the model output has indeed a very high standard deviation around the poisoned examples.
Therefore, slightly perturbing the poisons considerably changes their gradient and make them ineffective.
The main idea behind our friendly noise defense method, F RIE ND S , is to maximally perturb the training examples to make the effective poisons ineffective.
However, perturbing the training examples should be done in a way that does not harm the generalization performance of the model.
To address this, our method is composed of two components: First, we ﬁnd the maximum perturbation that can be added to every training example without changing its prediction.
This alleviates the local sharp region of high training loss introduced by the effective poisons.
To further break the attack, we add a small varying random noise to every example during the training.
The random noise generally smooths out the training loss and further makes the crafted poisons ineffective.
Below, we discuss each component in more details.
3.2 Optimizing the Friendly Noise: Maximally Perturbing Examples without Harm
The ﬁrst component of our method ﬁnds the maximum perturbation that can be added to every example without signiﬁcantly changing the model’s output.
To do so, for every example x i we optimize for the largest noise (cid:15) i within an l ∞ norm ζ -bound that results in similar prediction probabilities, measured by KL-divergence.
Formally for each example x i , we ﬁnd perturbation (cid:15) i as follows 2 :
[Equation] [Equation]
We generate a ﬁxed accuracy-friendly perturbation for every data point by solving problem (3) once, using a few Stochastic Gradient Descent (SGD) steps.
There is a trade-off between the time of generating the optimized perturbations and their effectiveness.
In particular, the optimized perturbations need to be generated and added to the training examples early in training, before the attack succeeds.
At the same time, for the perturbations to be effective, they should be generated after the decision boundary is shaped. We visualize this trade-off in Appendix B.2.
We found that training for as few as 5 epochs before solving the optimization problem (3) yields effective perturbations that reduce the attack success rate without harming the model’s performance.
The pseudocode can be found in Alg.1.
To better understand the effect of our friendly noise, we illustrate the histogram of the noise added to every pixel in the training data in Appendix B.1, Fig. 4(a).
We see that our method mainly targets certain pixels in every image by adding the maximum amount of perturbation, and leaves the rest of the image untouched.
Certain semantic regions have been shown to be much more robust to perturbations [3].
Perturbing the more robust areas does not considerably change the model’s behavior and training dynamics.
On the other hand, the least robust areas are very sensitive to Require: Train dataset X , Model f θ , LR η opt , λ , small number T for i ∈ [ T ] do θ i
= θ i − 1 − η ∇ θ
L
( θ i − 1 , X ) (cid:46) Train the model for a few epochs end for for x i ∈ X do Initialize noise {+(cid:15) 
+}
i uniformly sampled from [ − (cid:15) init , (cid:15) init ] for t = 1 to T do (cid:15) ti = (cid:15) t − 1 i − η opt ∇ (cid:15) ( D KL (cid:0)
f θ ( x i + (cid:15) t − 1 i )
||
f θ ( x i ) (cid:1) − λ (cid:107) (cid:15) t − 1 i (cid:107) 2 ) end for
Store noise (cid:15) i = (cid:15)
Ti for example x i end for
Require: Train dataset X , Random Noise Distribution A, Epoch to start defense def _ epoch
Run Algorithm 1 to generate { (cid:15) i } | X | i
= for i = def _ epoch to n _ epochs do for x i ∈ X do Sample random noise µ i ∼ A Set ˆ x i
= x i + (cid:15) i + µ i and add to dataset ˆ X end for [Equation] (cid:46) SGD update step with new dataset ˆ X end for perturbations, hence small amounts of noise in such areas result in a relatively large change in the model’s behavior.
We visualize the poisons before and after adding our friendly noise in Fig.
We see that our friendly noise successfully targets certain areas in every image that are robustly learned by adding the maximum possible perturbation.
The maximal friendly perturbation added to every example by our method prevents the effective poisons to closely match the target gradient or representation. Effectively, this alleviates the sharp local region of high training loss introduced by the effective poisons, as is illustrated in Fig. 3(b), and considerably reduces the success rate of the attack. At the same time, the important features used for classiﬁcations are preserved and hence model predictions remain unchanged.
Hence, our method reduces the attack success rate while ensuring only a slight drop in the test accuracy.
Next, we discuss how adding a random variable random noise can further improve the model’s robustness against poisoning attacks.
3.3 Adding Random Noise: Smoothing the Loss
As discussed, our friendly perturbation mainly targets the robust areas of the image and alleviates the sharp local region of high training loss introduced by effective poisons, without considerably changing the training dynamics.
To further improve the model’s robustness against poisoning attacks, we add a variable random noise to the training examples at every training iteration.
Adding the variable random noise generally smooths out the loss landscape.
In doing so, it further alleviates the sharp local regions introduced by the attack.
Consequently, adding the random noise considerably drops the attack success rate.
Meanwhile, adding a large random noise can harm the test accuracy by over-smoothing the loss. However, the combination of friendly noise and a smaller random noise can effectively break poisoning attacks without harming the model’s performance.
Fig.
3(b) and
Fig. 3(c) show that the local region of high training loss introduced by effective poisons is alleviated after adding each component of our defense.
This clearly demonstrates the effect of our defense and explains its effectiveness.
The pseudocode of our defense, F RIE ND S , can be found in Alg. 2.
The small varying random noise can be sampled from various distributions.
In particular, we sample from 3 different random noise distributions: (1) Bernoulli: noise is randomly sampled from {− µ, µ } , (2) Uniform: noise is randomly sampled from [ − µ, µ ] , and (3) Gaussian: noise is sampled from the normal distribution N (0 , µ ) .
We note that compared to Uniform noise, Bernoulli and Gaussian noise
[Table]
are in general more effective at reducing the attack success rate, as they make larger perturbations to input features. At the same time, this makes them to suffer a larger drop in the test accuracy.
We visualize and compare the Bernoulli, Uniform, and Gaussian noise types in Appendix B.1.
3.4 Adaptive attacks
Adaptive attacks can respond to a novel defense algorithm when the attacker is aware of the defense.
If the defense algorithm is known to the attacker beforehand, the attacker can generate more powerful poisons by taking into account the speciﬁc defense in place.
For example, Gradient Matching [10] and Sleeper Agent [34] demonstrated that by including augmented examples as well as original examples during poison generation in Eq.
(2), they can obtain robustness against standard data augmentations like crops and ﬂips, when the augmentation technique is preempted by the attacker.
For F RIE ND S , the prohibitively large search space of random noise permutations and its pixel-wise independence property make it extremely difﬁcult for adaptive attacks to break.
That is, while an attacker may use the knowledge of the time and optimization procedure to bypass the friendly perturbation component of F RIE ND S , they need to take into account a prohibitively large number of random noise combinations to bypass the random noise component.
For example, for a ﬁxed
Bernoulli noise, with p pixels there are 2 p combinations that an attacker should take into account to ensure the poisons’ effectiveness.
For real images, this becomes prohibitively expensive.
Similarly, for a ﬁxed Gaussian and uniform noise, an inﬁnite number of combinations must be considered during the poison generation to ensure the attack’s robustness.
Note that our method applies a varying random noise at every iteration, which also needs to be taken into account by the attacker.
This makes
F RIE ND S robust against adaptive attacks.
Experiments 4.1 Implementation details
We evaluate our defense method against both triggerless data poisoning and backdoor attacks under two attack settings - training from scratch and transfer learning.
Following the works of [10, 11, 31], we evaluate our method primarily on CIFAR-10, ResNet-18.
We also normalize and augment training images with default CIFAR-10 augmentations as used in [11].
For all models trained from scratch, we use a learning rate starting at 0.1 and decaying by a factor of 10 at epochs 30, 50, and 70.
For transfer learning, we decay the learning rate at epochs 15, 25, and 35.
When applying our method, we clamp the generated friendly perturbations using ζ = 16 , and add bounded random noise.
For the random noise component, we set µ = 16 in our experiments.
We also normalize the image as a pre-processing step.
We optimize friendly perturbations using SGD with momentum 0.9 and Nesterov acceleration, perform a hyperparameter search along LR = { 10 , 20 , 50 , 100 } and λ = { 1 , 10 } , and optimize each batch of 128 samples for 20 epochs.
Following previous works, we report poison [Table] [Table] success rate (or poison accuracy) as the percentage of datasets poisoned at the end of training.
We run all experiments and timings on an NVIDIA A40 GPU.
4.2 From-Scratch Setting
First [Table] we evaluate our method on poisoning attacks targeted toward victim models trained from scratch.
Such an attack assumes a gray-box scenario, where attackers have knowledge of the victim architecture, but have no knowledge of the speciﬁc initialization of the victim’s model.
Similar to the settings used in [31] [Table] which proposes a standardized benchmark for backdoor and data poisoning attacks, benchmark settings, we generate poisoning attacks by selecting 1% of training examples as poisons [Table]
which are perturbed within the l ∞ ball of some radius ξ .
Unless otherwise speciﬁed, we set ξ = 16 .
The victim model is initialized with the same architecture targeted by the attack based on a different random seed [Table] and is trained on the poisoned dataset using SGD.
When applying F RIE ND S , we set def _ epoch = [Table] and train only with random noise for the ﬁrst 5 epochs.
4.2.1 Baseline Comparison and Ablation Study
We evaluate our method and baseline defenses against the Witches’ Brew, or Gradient Matching, attack [10].
It is the current state-of-the-art among data poisoning attacks when applied to the from-scratch setting, and is adapted to be effective against data augmentation and differential privacy [11].
We follow the settings proposed by [11], under which we generate 20 different attack datasets for ResNet-18 trained on CIFAR10 with a 1% budget bounded by ξ = 16 , with a slight modiﬁcationwhile [11] uses 40 epochs for training, we use 80 epochs to show that our method easily scales to real-world training pipelines.
This is because 40 epochs of training only yields 92 .
01% test error, while 80 epochs yield 93 .
In Tab. 1, we show that we outperform state-of-the-art defenses [7, 11, 14, 25, 27, 43].
Notably, we achieve the same 0 .
00% poison success rate with 91 .
52% test accuracy, an improvement over of 0 .
23% over state-of-the-art [11] which yield 91 .
29% test accuracy [Table] at the same poison success rate.
Most importantly, F RIE ND S completes in 37 mins, 7 .
3 x faster than [11] which completes in 4.5hrs.
Compared to the efﬁcient method of [43], namely EPIC-0.1,
F RIE ND S achieves a 1 .
48% higher accuracy while being 1.2x faster.
We also strongly outperform other baseline defense methods simultaneously in all three metrics - poison success rate, test accuracy, and runtime.
In Tab. 3, we show that F RIE ND S also effectively defends against MetaPoison [15], reducing the poison success rate from 45 .
% to 20 . 00% with an accuracy gain from 87 .
61% to 90 .
82% resulted from applying augmentations.
We further show that our approach is effective against backdoor attacks, in particular, against the
Sleeper Agent attack [34].
Sleeper Agent is the current state-of-the-art clean-label backdoor attack, and the only such attack shown to be effective in from-scratch settings.
Following their evaluation protocol, we generate 24 poisoned datasets with ξ = 16 , and evaluate our defense by trainingvictim models respectively for 40 epochs and testing the poison success rate on 1000 target backdoor images per dataset.
We compare our method against other defenses evaluated by [34] in Tab. 2.
Here, F RIE ND S successfully defends against [34] by reducing poison accuracy from 83 .
48% to 21 .
52% with only a small drop in test accuracy from 91 .
56% to 89 .
We outperform the next best methods, EPIC-0.1 [43] and Spectral Signatures [38], by lowering poison accuracy by 14 .
18% and 2 . 41% while maintaining similar test accuracy.
EPIC-0.2 [43] achieves the lowest poison success rate at 9 .
10% , but drops test accuracy to 86 . 21% , and [9] achieves 92 .
26% test accuracy but suffers from 62 .
68% poison accuracy.
We also perform an ablation on each components of F RIE ND S in Tab. 4.
We show that naively applying Friendly Noise ( ζ = 32 ) yields a high poison success rate of 10% .
On the other hand, applying random noise ( µ = 32 ) yields low poison success rates but also results in a signiﬁcant test accuracy tradeoff (e.g. > 4 . 0% drop for Gaussian and Bernoulli noise).
Here, we show that applying F RIE ND S by proportionately combining friendly noise ( ζ = 16 ) with each of the random noise components ( µ = 16 ) maintains high test accuracy (i.e. only 2 . 0% drop) while keeping poison success rate close to 0.
4.2.2 Defending against Adaptive Attacks
As discussed in Sec. 3.4, we believe that an adaptive attack against our defense is computationally prohibitive. To evaluate our claim, we modiﬁed the differentiable data augmentation component of Sleeper Agent attack algorithm to include randomly sampled Bernoulli noise.
We then further added the ﬁxed friendly noise generated from selected prior runs to the attacker’s augmentation procedure. Our results in Tab.
5 show that despite attacks being adapted to Bernoulli and friendly noise, poison accuracies are all within the standard deviation of one another, and the adaptive attacks cannot succeed.
[Table] [Table]
Next, we evaluate our method on data poisoning and backdoor attacks designed for the transfer learning scenario [2, 32]. Here, the attacks are crafted based on a pretrained network with the goal of achieving poisoning when transfer learning is performed using the generated poisoned dataset.
For the transfer learning scenario used in poisoning benchmarks [11, 31], the linear layer (classiﬁer) of the pretrained model is re-initialized and trained with the poisoned dataset, while other layers (feature extractor) remain ﬁxed during the training.
Similar to the from-scratch setting, attacks are limited to a budget of 1% and ξ = 16 .
However, we generate F RIE ND S at the beginning of training instead of after 5 training epochs, since the feature extractor is already initialized.
We note that this is not the true transfer learning setting, since the pretraining and transfer learning datasets are the same.
However, as [10, 11] noted, this presents an effective worst-case scenario to evaluate poisoning attacks.
We show that in Tab. 3 that even in such cases, we reduce poison success rate from 100% to 35% for the Bullseye Polytope attack [2], and from 100% to 30% for the Poison Frogs attack [32].
4.4 Transferability across Architectures
We show that perturbations generated by F RIE ND S are transferable across architectures.
In Tab. 6, we show using Gradient Matching ξ = 16 that F RIE ND S optimized using smaller architectures, in particular AlexNet [18] and LeNet [20], can be directly used for larger architectures like ResNet18.
This presents a signiﬁcant advantage in terms of computational costs, since F RIE ND S can be generated using smaller, and hence faster, models.
Crucially, this makes the generated friendly noise free to be directly applied to (much larger) architectures.
Conclusion
We proposed a simple and highly effective defense mechanism, F RIE ND S , that protects deep learning pipelines against various types of poisoning attacks.
Our defense is built on the observation that poisoning attacks introduce sharp local regions of high training loss, by adding adversarial perturbations to a subset of training examples.
F RIE ND S relies on two components to break the poisons: an accuracy-friendly perturbation that is generated to maximally perturb examples without degrading the performance, and a randomly varying noise component.
The ﬁrst component alleviates the sharp local regions introduced by poisons, and the second component further smooths out the loss landscape.
Both components combined together build a very light-weight but highly effective defense against the most powerful triggerless and backdoor poisoning attacks, including Gradient Matching, Bull-eyes Polytope, Poison Frogs, and Sleeper Agent, in transfer learning or training from scratch scenarios.
F RIE ND S is extremely difﬁcult to break with adaptive attacks and our friendly noise can be transferred to other architecture.
This makes it almost free to apply to real-world deep learning pipelines.
Our defense is particularly targeted towards clean-label poisoning attacks that are generated under bounded perturbations.
Such settings are the most difﬁcult to defend, as generated poisons can easily fool even an expert observer.
In contrast, unbounded attacks can be easily detected by manual or automated ﬁltering mechanisms, through a single pass over the dataset.  Acknowledgements
This research was partially supported by Cisco Systems, the National Science Foundation CAREER
Award 2146492, and the UCLA-Amazon Science Hub for Humanity and AI.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Supplementary Material:
Friendly Noise against Adversarial Noise: A Powerful Defense against Data Poisoning Attacks A Additional Experiments A.1 Ablation on different norms for friendly noise objective Recall the friendly noise objective is given by the following
[Equation] We note that while we used the L 2 norm for encouraging larger values of (cid:15) , other norms such as L ∞ and L 1 can also be used. L ∞ only cares about the largest element in the noise vector, so it adds larger perturbations, while L 1 cares about absolute values hence it adds smaller (but more non-zero) perturbations.
L 2 encourages larger noise elements compared to L 1 . Tab. 7 shows that different norms result in similar poison and test accuracy. [Table]
B Additional Visualizations B.1 Histogram visualizations of friendly noise and its variants
[Figure] Fig.
4 compares the distribution of random noise sampled from these distributions combined with the optimized perturbation obtained from Eq. We can see that uniform noise perturbs all the pixels similarly, and hence small amount of uniform noise does not harm the model’s performance but cannot effectively breaks poisons. Larger uniform noise, however, has a larger effect on the model’s performance. Bernoulli and Gaussian noise on the other hand add a larger perturbation to individual pixels.
Hence, they are more effective in reducing the attack success rate, but they harm the test accuracy more as they the larger perturbation may be added to the more sensitive areas. Figures 4(b) to 4(d) shows the distribution of random noise combined with our optimized noise added to different pixels. We observe that random noise is added to regions where friendly noise is less dominant, hence resulting in a signiﬁcant perturbation to an overall greater number of pixels to break poisoning attacks. B.2 Trade-off between the time of generating friendly noise and their effectiveness
[Figure]