Certified Robustness to Clean-label Poisoning Using Diffusion Denoising

## Abstract

Abstract We present a certified defense
to clean-label poisoning attacks.
These attacks work by injecting
a small number of poisoning samples (e.g., 1%)
that contain p 𝑝 p italic_p -norm bounded adversarial perturbations
into the training data
to induce a targeted misclassification of a test-time input.
Inspired by the adversarial robustness
achieved by denoised smoothing ,
we show how an off-the-shelf diffusion model
can sanitize the tampered training data. We extensively test our defense
against seven clean-label poisoning attacks
and reduce their attack success to 0–16%
with only a negligible drop in the test time accuracy.
We compare our defense
with existing countermeasures against clean-label poisoning,
showing that the defense reduces the attack success the most
and offers the best model utility.
Our results highlight the need for future work
on developing stronger clean-label attacks
and using our certified yet practical defense
as a strong baseline to evaluate these attacks.

## 1 Introduction

A common practice in machine learning
is to train models on a large corpus of data.
This paradigm empowers many over-parameterized deep-learning models
but also makes it challenging to retain the quality of data collected from various sources.
This makes deep-learning-enabled systems
vulnerable to *data poisoning* (Rubinstein et al., ; Nelson et al., ; Biggio et al., ; Jagielski et al., ; Shafahi et al., ; Carlini, )—where an adversary can alter a victim model’s behaviors
by injecting malicious samples (i.e., poisons) into the training data.

But in practice, it may be challenging to make modifications to a
training dataset used for fully-supervised classification,
because they are often relatively small.
As a result, recent work has developed
*clean-label poisoning attacks* (Shafahi et al., ; Zhu et al., ; Aghakhani et al., ; Turner et al., ; Saha et al., ; Huang et al., ; Geiping et al., ),
where the attacker aims to control the victim model’s behavior
on a specific test input by injecting a few poisons
that visually appear to be correctly-labeled,
but in fact include human-invisible adversarial perturbations.

We introduce a defense against clean-label poisoning attacks
inspired by defenses to test-time adversarial examples.
Existing poisoning defenses fall into two categories
*certified* and *heuristic*.
Certified defenses offer provable guarantees,
but often significantly decrease
the utility of defended models at test-time,
making them impractical (Ma et al., ; Levine & Feizi, ; Wang et al., ; Zhang et al., ).
Heuristic approaches (Suciu et al., ; Peri et al., ; Hong et al., ; Geiping et al., ; Liu et al., )
demonstrate their effectiveness
against existing attacks in realistic scenarios.
However, these defenses rely on unrealistic assumptions,
such as the defender knowing the target test-time input (Suciu et al., ),
or are evaluated against specific poisoning adversaries,
leaving them improvable against adaptive attacks or future adversaries.

Our contributions.
We *first* make two seemingly distant goals closer
by presenting
a certified defense against clean-label poisoning
that *also* minimizes the decrease in clean accuracy.
For any $p$-norm bounded adversarial perturbations to the training data,
we ensure a certified accuracy higher than the prior certified defenses.
The model trained on the tampered data
classifies a subset of test-time input $x$
(or $x+\delta$, where $||\delta||_{\ell_{p}}$
in clean-label backdoor poisoning) correctly.
To achieve this goal,
we leverage the recent diffusion probabilistic diffusion models (Sohl-Dickstein et al., ; Ho et al., ; Nichol & Dhariwal, ).
We use an off-the-shelf diffusion model to denoise
the entire training data before training a model.
In §3,
we theoretically show how one achieves
a certified accuracy under $2$-norm bounded adversarial perturbation
to the training data.

Removing adversarial perturbations in the training data
before model training, we can decouple
the certification process from the training algorithm.
*Second*, we leverage this computational property our defense has
and present a series of training techniques
to alleviate the side-effects of employing certified defenses,
i.e., the utility loss of a defended model.
To our knowledge,
we are the first certified defense
that decouples these two processes.
We minimize the utility loss by employing the warm-starting (Ash & Adams, ).
We train a model a few epochs on the tampered data or
initialize its model parameters using a pre-trained model.
None of them alters the training process;
thus, our defense’s provable guarantee holds.
In §4,
we show they attribute to a better performance
compared to existing certified defenses.

*Third*, we extensively evaluate our defense
against seven clean-label poisoning attacks
studied in two different scenarios:
transfer-learning and training a model from-scratch.
In transfer-learning,
our defense completely renders the attack ineffective
with a negligible accuracy drop of 0.5%
in the certified radius of 0.1 in $2$-norm perturbations.
We also reduce the attack success to 2–16%
when a defender trains a model from scratch.
We further compare our defense
with six poisoning defenses in the prior work.
We demonstrate more (or the same in a few cases)
reduction in the attack success
and less accuracy drop than those existing defenses.

We discuss research questions
important in studying poisoning attacks and defenses
but are far-neglected in the prior work.
We suggest future work directions to answer them.

## 2 Preliminaries on Clean-label Poisoning

Threat model.
A clean-label poisoning attacker causes a misclassification of a specific *target* test-time sample (${x}_{t}$, $y_{t}$)
by compromising the training data $D_{tr}$ with poisoning samples $D_{p}$.
If a victim trains a model $f$ on the poisoned training data $D_{ptr}\!=\!D_{tr}\cup D_{p}$,
the resulting model $f^{*}_{\theta}$ is likely to misclassify
the target instance to the adversarial class $y_{adv}$
while preserving the classification behavior of $f^{*}_{\theta}$ on the clean test-set $S$.
The attacker crafts those *poisons* $({x}_{p},y_{p})$ by first taking a few *base* samples in the same domain (${x}_{b},y_{b}$) and then adding human-imperceptible perturbations $\delta$, carefully crafted by the attacker and also bound to $||\delta||_{\ell_{p}}\leq\epsilon$, to them while keeping their labels *clean* ($y_{b}=y_{p}$). A typical choice of the bound is $\ell_{\infty}$ or $\ell_{2}$.

Poisoning as a constrained bi-level optimization.
The process of crafting optimal poisoning samples $D^{*}_{p}$
can be formulated as the constrained bi-level optimization problem:

$$ $\displaystyle D^{*}_{p}=\underset{D_{p}}{\arg\min}\text{ }\mathcal{L}_{adv}(x_ {t},y_{t};f^{*}_{\theta}),$ $$

where $\mathcal{L}_{adv}(x_{t},y_{adv};f^{*}_{\theta})$ is the adversarial loss function quantifying how accurately a model $f^{*}_{\theta}$, trained on the compromised training data, misclassifies a target sample $x_{t}$ into the class an adversary wants $y_{adv}$. $D_{p}$ is the set of poisoning samples we craft, and $D^{*}_{p}$ is the resulting optimal poisons.

While minimizing the crafting objective $\mathcal{L}_{adv}(x_{t},y_{t};f^{*}_{\theta})$, the attacker also trains a model $f^{*}_{\theta}$ on the compromised training data, which is itself another optimization problem, formulated as follows:

$$ $\displaystyle f^{*}_{\theta}=\underset{\theta}{\arg\min}\text{ }\mathcal{L}_{ tr}(D_{ptr},S;\theta),$ $$

where the typical choice of $\mathcal{L}_{tr}$ is the cross-entropy loss, and $S$ is the clean test-set. Combining both the equations becomes a bi-level optimization: find $D_{p}^{*}$ such that $\mathcal{L}_{adv}$ is minimized after training, while minimizing $\mathcal{L}_{tr}$ as well.

To make the attack inconspicuous,
the attacker *constraints* this bi-level optimization
by limiting the perturbation $\delta=x_{p}-x_{b}$
each poisoning sample can contain to $||\delta||_{\ell_{p}}<\epsilon$.

Existing clean-label attacks.
Initial work (Shafahi et al., ; Zhu et al., ; Aghakhani et al., ) minimizes $\mathcal{L}_{adv}$ by crafting poisons that are close to the target in the latent representation space $g(\cdot)$. A typical choice of $g(\cdot)$ is the activation outputs from the penultimate layer of a pre-trained model $f(\cdot)$, publicly available to the attacker from the Internet. The attacks have shown effective in *transfer-learning* scenarios, where $f(\cdot)$ will be fine-tuned on the poisoned training data. The attacker chooses base samples from the target class $(x_{b},y_{adv})$ and craft poisons $(x_{p},y_{adv})$. During fine-tuning, $f(\cdot)$ learns to correctly classify poisons in the target’s proximity in the latent representation space and classify the target into the class $y_{adv}$ the attacker wants.

Recent work focuses on making those poisoning attacks effective when $f$ is trained *from scratch* on the tampered training set.
To do so,
the attacker requires to approximate the gradients computed on $\mathcal{L}_{adv}$ that are likely to appear in any models. Huang et al. () address this challenge by meta-learning; the poison-crafting process simulates all the possible initialization, intermediate models, and adversarial losses computed on those intermediate models. A follow-up work by Geiping et al. () alleviates the computational overhead of the prior work
by proposing an approximation technique, gradient matching, that aligns the gradients from poisoning samples with those computed on a target.

Defenses against clean-label poisoning.
Early work on poisoning defenses focuses on filtering out poisons from the training data. Nelson et al. () and Suciu et al. () compute the training samples negatively impacting the classification results on targets. But in practice, the defender does not know which test-time samples are the attack targets.
Follow-up work (Rubinstein et al., ; Peri et al., ; Tran et al., ) leverages unique statistical properties (e.g., spectral signatures) that can distinguish poisons from the clean data. All these approaches depend on the data they use to compute the properties. Recent work (Geiping et al., ; Borgnia et al., ; Hong et al., ; Liu et al., ) thus reduces the dependency by proposing data- or model-agnostic defenses, adapting robust training or differentially-private training. While shown effective against existing attacks, those defenses are not *provable*.

A separate line of work addresses this issue by proposing
*certifiable* defenses (Levine & Feizi, ; Wang et al., ; Weber et al., ), which guarantee the correct classification of test-time samples when an adversary compromises the training data with the $k$ poisons whose perturbation radius of $r$.
Levine & Feizi (); Wang et al. ()’s approach is majority voting, where the defender splits the training data into multiple disjoint subsets, train models on them, and run majority voting of test inputs over those models.
BagFlip (Zhang et al., ) proposes
a model-agnostic defense that uses the idea of randomized smoothing (Cohen et al., ) studied in adversarial robustness.
However, the certification offered by these works
has only shown effective in toy datasets like MNIST,
and it has been under-studied whether
they will scale to practical scenarios, e.g., CIFAR10 or TinyImageNet.
The computational demands of these defenses further hinder their deployment in practice.

## 3 Diffusion Denoising as Certified Defense

We aim to certify the prediction of
a model $f_{\theta}$ trained on a poisoned training data $D_{ptr}$.
To formally define this goal,
we first define the perturbation bound $\epsilon$
as the $\ell_{p}$ distance between two datasets,
computed by taking the sum across $\ell_{p}$ perturbations of all images.
The perturbation space $P^{\pi}_{\epsilon}$ is
the set of all the datasets $D_{ptr}$
obtained by tampering samples
with the crafting algorithm $\pi$ with the bound $\epsilon$.
Operating within this perturbation space $P$,
for any given test-time sample $(x_{t},y_{t})$,
can produce a certificate that

$$ $\displaystyle\mathbf{Pr}_{{D_{ptr}}\in P^{\pi}_{\epsilon}(D_{tr})}\left[f_{ \theta\leftarrow D_{ptr}}(x_{t})=f_{\theta\leftarrow D_{tr}}(x_{t})\right]>1-\alpha$ $$

with arbitrarily high probability $1-\alpha$,
we guarantee that training on any poisoned dataset $D_{ptr}\in P^{\pi}_{r}(D_{tr})$
in the perturbation bound $\epsilon$ will classify
the example $\mathbf{x}_{t}$ the same way
as it was classified on the non-poisoned dataset $D_{tr}$.

### 3.1 Robustness Guarantee

Intuition.
Randomized smoothing (Cohen et al., ) guarantees that
a fixed classifier $f_{\theta}$ will classify any
adversarial example $x^{\prime}$ correctly with high probability.
To make this argument, it first shows that
even for extremely large values of noise $\delta$,
the example $x+\delta$ will be classified correctly.
Thus, the analysis of randomized smoothing is able to show that
a (much smaller) worst-case direction $\delta_{adv}$
will not change the prediction of the classifier.

Here we make a similar argument,
but change where the noise is added.
Given a dataset $D_{ptr}$, where some examples may be slightly perturbed to
poison the model, we want to guarantee that any model $f_{\theta}$ trained
on $D_{ptr}$ will classify the test examples correctly.
To achieve this, we train many models $f^{i}_{\theta}$ on datasets from $P^{\pi}_{\delta}$
(with extremely large perturbations $\delta$);
if each of these classifiers consistently labels the test data
correctly, then we can be guaranteed that (much smaller) worst-case
poisoning directions $\delta_{adv}$ can not change the prediction
of the classifier.

Randomized smoothing shows that adversarial noise to the
test example $x$ will not cause misprediction because even for large
quantities of Gaussian noise $x+\delta$, the predictions remain consistent.
In our setting, adversarial noise to the model’s training dataset $D_{tr}$
will not cause misprediction because even for large quantities of Gaussian
noise $\delta$ to the training set
the predictions of $f_{\theta\leftarrow D_{ptr}}$ remain consistent.

Details.
Standard randomized smoothing transforms a non-robust classifier $f$
into a smoothed (robust) classifier $g$:

$$ $\displaystyle g(x)=\arg\max_{c\in\mathcal{Y}}\mathbf{Pr}_{\delta\sim N(0, \sigma^{2}I)}[f_{\theta}(x+\delta)=c].$ $$

But clean-label poisoning does not add any test-time perturbations to the target samples.
Instead, it alters the training *dataset*,
which then causes the *models* trained on the dataset to be different.
Therefore, for our use-case we define:

$$ $\displaystyle g(x)=\arg\max_{c\in\mathcal{Y}}\mathbf{Pr}_{\theta\sim\text{ train}(D_{ptr}\sim P^{\pi}_{\delta})}[f_{\theta}(x)=c].$ $$

Here, each model $f_{\theta}$ is obtained by
training a new classifier on
a perturbed version of the training dataset.

### 3.2 (Diffusion) Denoising for the Robustness

Denoising diffusion probabilistic model (DDPMs)
are a recent generative model
that works by learning the diffusion process of the form
$x_{t}\sim\sqrt{1-\beta_{t}}\cdot x_{t-1}+\beta_{t}\cdot\omega_{t}$,
where $\omega_{t}$ is drawn from a standard normal distribution $\mathcal{N}(0,\mathbf{I})$
with $x_{0}$ sourced from the actual data distribution,
and $\beta_{t}$ being fixed (or learned) variance parameters.
This process transforms images
from the target data distribution into purely random noise over time $t$,
and the reverse *denoising* process constructs images in the data distribution,
starting with random Gaussian noise.
A DDPM with a fixed time-step $t\in\mathbb{N}^{+}$ and a fixed schedule
samples a noisy version of a training image $x_{t}\in[-1,1]^{w\cdot h\cdot c}$ of the form:

$$ $\displaystyle x_{t}:=\sqrt{\alpha_{t}}\cdot x+\sqrt{1-\alpha_{t}}\cdot\mathcal {N}(0,\mathbf{I}),$ $$

where $\alpha_{t}$ is a constant derived from $t$,
which decides the level of noise to be added to the image
(the noise increases consistently as $t$ grows).
During training,
the model minimizes the difference between
$x$ and the denoised $x_{t}$,
where $x_{t}$ is obtained by applying the noise at time-step $t$.

Diffusion denoising for the robustness.
We utilize off-the-shelf
DDPMs (Sohl-Dickstein et al., ; Ho et al., ; Nichol & Dhariwal, )
to *denoise* adversarial perturbations added to the training data
and as a result, provide the robustness to clean-label poisoning.
A naive adaptation of randomized smoothing to our scenario
is to train multiple models on the training data
with Gaussian noise augmentation or using adversarial training.
But we need to add noise to the data
that does not make it look natural
or they are computationally demanding.
We thus avoid using these approaches to train models,
instead we want to remove the (potentially compromised) training set before training. Note that the robustness guarantee that randomized smoothing offers
holds regardless of how a model is trained.
We can still certify the robustness against clean-label poisoning.
The fact that we use the denoising may make the guarantee a bit loose,
but we show in our evaluation
that the loose guarantee
is sufficient to render existing attacks ineffective.
Our denoising process is shown in Algorithm 1.

Figure: Algorithm 1 Noise, denoise, train, classify

Figure: Algorithm 2 Randomized smoothing (Cohen et al., )

### 3.3 Prediction and Certification

Now we present algorithms
for running a prediction with the classifiers trained on $D_{ptr}$
and certifying the robustness of the model’s prediction
on a test-time sample $x$.
Our work extends the algorithms presented by Cohen et al. ()
to clean-label poisoning settings,
as shown in Algorithm 2.

Prediction.
To compute the prediction for a test-time sample $x$,
we make an adaptation of the standard randomized smoothing.
We train $n$ classifiers on $n$ different noised-then-denoised training datasets,
and return the predictions of these classifiers on the target $x$,
with the algorithm DenoiseTrainClassify($\cdot$).
We run this algorithm for a sufficient number of times (e.g., over 1000 times).
The prediction output is then the majority-voted label from these classifiers.

Certification
process exactly follows the standard randomized smoothing.
We first count the number of occurrences of
the most likely label $\hat{c_{A}}$ compared to any other runner-up label,
and from this can derive a radius
(for the training set perturbations)
on which $x$ is guaranteed to be robust.

Figure: Figure 1: Certified radius and accuracy attained by denoising the CIFAR10 training data with different $\sigma$ values.
Refer to caption: x1.png

#### Results.

We train 10k CIFAR10 models using an efficient training pipeline that
trains a single model in $15$ seconds on an A100 GPU
(for a total of 1.7 GPU days of compute).
These models have a clean accuracy of between $92\%$ and $58\%$
depending on the level of noise $\sigma$ introduced,
and for the first time can certify non-trivial robustness to
clean-label poisoning attacks. A full curve comparing certified accuracy for all given perturbation radii
is shown in Figure 1.

Table: Table 1: Defense effectiveness in transfer-learning scenarios (CIFAR10). We measure the clean accuracy and attack success of the models trained on the denoised training set. Each cell shows the accuracy in the parentheses and the attack success outside. Note that ${}^{\dagger}$ indicates the runs with $\sigma$=0.0, the same as our baseline that trains models without any denoising.

### 3.4 Computational Efficiency for Certification

In order to obtain a robustness certificate,
we must train many CIFAR10 models
on different denoisings of the training dataset.
This step is computationally expensive,
but is in line with the computational complexity of prior
randomized smoothing based approaches that require similar work.
However, as we will show later, in practice we can empirically obtain robustness
with only a very small number of training runs (even as low as one!),
confirming the related observation from (Lucas et al., ) who find a similar effect holds for randomized smoothing of classifiers.

## 4 Empirical Evaluation

We empirically evaluate the effectiveness of our certified
defense against clean-label poisoning. We adapt the poisoning benchmark developed by Schwarzschild et al. (). This benchmark runs clean-label poisoning attacks with the same attack configurations, allowing us to compare our defense’s effectiveness across many existing attacks.

Poisoning attacks.
We evaluate our defense against seven clean-label poisoning attacks: four targeted poisoning (Poison Frogs (Shafahi et al., ), Convex Polytope (Zhu et al., ), Bullseye Polytope (Aghakhani et al., ), Witches’ Brew (Geiping et al., )) and three backdoor attacks (Label-consistent Backdoor (Turner et al., ), Hidden-Trigger Backdoor (Saha et al., ), and Sleeper Agent (Souri et al., )). The first three attacks operate in the transfer-learning scenarios (i.e., they assume that the victim fine-tunes a model from some given initialization), whereas the rest assumes the victim trains a model from-scratch. While our defense offers provable guarantees only to targeted poisoning attacks, we test if the defense can mitigate clean-label backdooring like the prior work did.

Metrics.
We employ two metrics: clean accuracy and attack success rate.
We compute the accuracy on the entire test set. An attack is successful if a poisoned model classifies a target sample
to the intended class. When evaluating backdoor attacks,
we only count the cases where a tampered model misclassifies
the target sample containing a trigger pattern.

Methodology.
We run our experiments in CIFAR10 and Tiny ImageNet (Krizhevsky et al., ; Le & Yang, ).
Following the standard practices in the prior work (Schwarzschild et al., ),
we run each attack 100 times on different targets and report the averaged metrics over the 100 runs.
We randomly choose the base and target classes.
We set the poisoning budget to 0.5–1% for all our attacks:
the attacker can tamper with only 500 samples of the entire 50k training instances in CIFAR10
and 250 of the total 100k training samples in Tiny ImageNet.

Attack scenarios.
We evaluate against both the transfer-learning and training from-scratch attacks.
In transfer-learning, we fine-tune a pre-trained model on the tampered training set while we train a randomly initialized model in the training from-scratch scenario. We also evaluate against the attackers with two types of knowledge: white-box and black-box. The white-box attackers know the victim model’s architecture (and its parameters), while the black-box attackers do not.
More details are
in Appendix A.

Table: Table 2: Defense effectiveness in training from-scratch scenarios (CIFAR10). We measure the accuracy and attack success of the models trained on the denoised training set. Each cell shows the accuracy in the parentheses and the attack success outside. Note that ${}^{\dagger}$ indicates the runs with $\sigma$=0.0, the same as our baseline that trains models without any denoising. We use an ensemble of four models, and WB and BB stand for the white-box and the black-box attacks, respectively.

### 4.1 Effectiveness of Our Denoising Defense

We first show to what extent our defense makes the seven poisoning attacks ineffective. Our defense is attack-agnostic. A defender does not know whether the training data contains poisoning samples or not; instead, the defender always trains a model on the denoised training set. We measure the trained model’s accuracy and attack success and compare them with the baseline, where a model is trained directly without denoising. We evaluate with different defense strengths: $\sigma\in\{0.1,0.25,0.5,1.0\}$. The attacks are with the perturbation bound of 16-pixels in $\ell_{\infty}$-norm and that of 3.5–6.5 in $\ell_{2}$-norm. We extended many poisoning attacks for supporting $\ell_{2}$ bounds. Both $\ell_{\infty}$ and $\ell_{2}$ bounds we use are comparable to each other. We show the conversion between the two bounds in Appendix B. Due to the page limit, we show the results from Tiny ImageNet and the results from attacking with different $\ell_{p}$-norms in Appendix C.

Transfer-learning scenarios.
Table 1 summarizes our results. Our defense is also effective in mitigating the attacks in transfer-learning. Each cell contains the accuracy and the attack success rate of a model trained on the denoised training set. All the numbers are the averaged values over 100 attacks. Against the white-box attacks, we reduce the success rate of the targeted poisoning attacks to 0–4% at $\sigma$ of 0.1. In the black-box attacks, the targeted poisoning is less effective (with 3–10% success), but still, the defense reduces the success to 0–1%. We observe that the two clean-label backdoor attacks show 1–7% success regardless of the attacker’s knowledge. We thus exclude these attacks from the subsequent evaluation. Using $\sigma$ greater than 0.5 increases the attack success by 0.5–2%. But, this may not because of the successful poisoning attacks but due to the significant decrease in a model’s accuracy (3-9%). Increasing $\sigma$ may defeat a stronger attack but significantly decrease a model’s accuracy. This result is interesting, as in our theoretical analysis shown in Sec 3, our defense needs $\sigma$ of 0.25, while 0.1 is sufficient to defeat both the attacks.

Training from-scratch scenarios.
Table 2 shows our results with the same format as in Table 1. We demonstrate that our defense mitigates both attacks at $\sigma=$ 0.25, aligned to what we’ve seen in our theoretical analysis. In Witches’ Brew, the attack success decreases from 34–71% to 3–10%; against Sleeper Agent, the attack success decreases from 19–40% to 7–19%(^1^11Note that the success rate of typical backdoor attacks is $\sim$90%; thus, the success below 19% means they are ineffective.). Note that to reduce their attack success to $\sim$10%, our defense needs a minimum $\sigma$ of 0.25–0.5. It is also noteworthy that our defense reduces the backdooring success significantly while the certificate does not hold because test-time samples are altered with trigger patterns.

### 4.2 Improving Model Utility

A shortcoming of all certified defenses (Levine & Feizi, ; Wang et al., ; Zhang et al., ) is that the certified accuracy is substantially lower than the undefended model’s accuracy. We also observe in the previous subsection that our defense, when we train a model from scratch on the data, denoised with large $\sigma$ values, is not free from the same issue, which hinders their deployment in practice.

In §3, we show that our defense is agnostic to how we train a model, including how we *initialize* a model’s parameters.
We leverage this property and minimize the utility loss by initializing model parameters in specific ways before we train the model on the denoised data. We test two scenarios: a defender can use in-domain data or out-of-domain data to pre-train a model and use its parameters to initialize.
We evaluate them against the three most effective attacks: Bullseye Polytope, Witches’ Brew, and Sleeper Agent.

Initializing a model using in-domain data.
Our strategy here is an adaptation of warm-starting (Ash & Adams, ). We first train a model from scratch on the tampered training data for a few epochs to achieve high accuracy. It only needs 5–10 epochs in CIFAR10. We then apply our defense and continue training the model on the denoised training set.

Table: Table 3: Improving the utility of defended models. We show the accuracy of defended models and the attack success after employing our two strategies. The top three rows are the baselines from Table 1 and 2, and the next two sets of three rows are our results. We highlight the cells showing the accuracy improvements in bold. ResNet18 models are used. BP, WB, and SA indicate Bullseye Polytope, Witches’ Brew, and Sleeper Agent, respectively.

Table 3 summarizes our results. The middle three rows are the results of leveraging the in-domain data. We train the ResNet18 models on the tampered training set for 10 epochs and continue training on the denoised training data for the rest 30 epochs. Our first strategy (warm-starting) increases the accuracy of the defended models while defeating clean-label poisoning attacks. Under strong defense guarantees $\sigma\!>\!0.1$, the models have 0.5–2.2% increased accuracy, compared to the baseline, while keeping the attack success $\sim$10%. In $\sigma=$ 0.1, we achieve the same accuracy and defense successes. Our strategy can be potentially useful when a defender needs a stronger guarantee, such as against stronger clean-label poisoning future work will develop.

Using models pre-trained on out-of-domain data.
Now, instead of running warm-starting on our end, we can also leverage “warm” models available from the legitimate repositories. To evaluate, we take the ResNet18 model pre-trained on ImageNet-21k. We use this pre-trained model in two practical ways. We take the model as-is and train it on the denoised training data. In the second approach, we combine the first approach with the previous idea. We train the model on the tampered training data for a few epochs and then train this fine-tuned model on the denoised data.

The bottom three rows of Table 3 are the results of using the warm model. We fine-tune the ResNet18 for 40 epochs on the denoised training data. In many cases, our second strategy can improve the accuracy of the defended models trained with strong defense guarantees ($\sigma\!>$ 0.1). These models achieve 6.5–13.2% greater accuracy than the results shown in Table 2 and 1. In $\sigma\!=$ 1.0, the final accuracy of defended models has a negligible difference from the baseline. We are the first work to offer practical strategies to manage the utility-security trade-off in certified defense.

### 4.3 Comparison to Existing Poisoning Defenses

We finally examine how our defense works better/worse than existing poisoning defenses. We compare ours with six defenses: DP (Ma et al., ; Hong et al., ), DPA (Levine & Feizi, ; Wang et al., ), BagFlip (Zhang et al., ), ROE (Rezaei et al., ), k-NN (Peri et al., ), AP (Geiping et al., ), and FrieNDs (Liu et al., ). The first four are certified defenses, and the last three are non-certified ones. We test them in the attack settings we use in the previous sections ($\ell_{\infty}$-norm of 16, CIFAR10).

We first compare ours with the filtering-based defenses.

Table: Table 4: Comparison of our defense to Deep kNN.

vs. Deep kNN. Peri et al. () remove poisons from the training data by leveraging the k-nearest-neighbor (kNN) algorithm. They run kNN on the latent representations of the training data to identify potentially malicious samples (i.e., samples with labels different from the nearest ones) and remove them. We compare Deep kNN’s effectiveness to our defense. We use the defense configurations that bring the best results in the original study, i.e., setting $k$ to 500. Table 5 shows our results. Deep kNN fails to remove most poisoning samples (but it removes many benign samples!).

Now we compare ours with the training-time defenses.

Table: Table 5: Comparison of ours to training-time defenses.

vs. DP-SGD. Ma et al. () proposed a certified defense against poisoning attacks that leverage differential privacy (DP). DP makes a model less sensitive to a single-sample modification to the training data. But the defense offers a weak guarantee in practice; for example, the certificate is only valid in CIFAR10, when an adversary tampers one training sample. (Hong et al., ) later empirically shows that DP is still somewhat effective against poisoning. We compare our defense to the training with $(1.0,0.05)$-DP, follow the prior work (Lecuyer et al., ),
on the same tampered training data. We use Opacus(^2^22[https://opacus.ai](https://opacus.ai)) to train models with DP and keep all the other configurations the same. In Table 5, both defenses significantly reduce the attack success to 0–16%. Our defense achieves at most 10% higher accuracy than DP. DP reduces the attack success slightly more against clean-label backdooring.

vs. AT. Geiping et al. () adapts the adversarial training (AT)for defeating clean-label poisoning: in each mini-batch, instead of crafting adversarial examples, they synthesize poisoning samples and have a model to make correct classifications on them. While effective, this approach could overfit a specific poisoning attack used in the adapted AT, leaving the model vulnerable to unseen poisoning attacks. We thus compare our defense with the original AT with the PGD-7 bounded to $\epsilon$ of 4. Table 5 shows that our defense achieves higher accuracy than the robust models while making the attcaks ineffective. Interestingly, AT cannot defeat the BP attacks. We hypothesize that clean-label poisons in the training data are already adversarial examples; thus, AT may not add more perturbations to them during training.

vs. FrieNDs. Liu et al. () advances the idea of training robust models. Instead of adding random Gaussian noise to the data during training, they use “friendly” noise, pre-computed by a defender, that minimizes the accuracy loss to a model they will train. When their defense is in action, they train a model on the tampered training data with two types of noise: the friendly noise computed before and a weak random noise sampled uniformly within a bound. Ours and FrieNDs greatly reduce the attack success while preserving the model’s utility. The critical difference is that ours is a certified defense, while FrieNDs is not. A promising future work we leave here is developing adaptive clean-label poisoning attacks against FrieNDs.

vs. ROE.
Certified defenses (Steinhardt et al., ; Ma et al., ; Diakonikolas et al., ; Levine & Feizi, ; Gao et al., ; Zhang et al., ; Wang et al., ; Rezaei et al., ) are; however, most of these defenses showed their effectiveness in limited scenarios, such as in MINST-17 (Zhang et al., ), a subset of MNIST containing only the samples of the digits 1 and 7. In consequence, they aren’t compatible with our clean-label poisoning settings. A recent defense by Rezaei et al. () demonstrates the certified robustness in practical settings; we thus compare our defense to their run-off-election (ROE). In Table 5, we empirically show that our defense is comparable to ROE while we preserve the model accuracy more.

Note on the computational efficiency.
Most training-time defenses require additional mechanisms we need to “add” to the training algorithm. AT needs to craft adversarial examples or clean-label poisons (Geiping et al., ). DP-SGD introduces noise into the gradients during training. FrieNDs pre-computes friendly noise and then optimally applies the noise during the training, and ROE requires training of 250 base classifiers to achieve the improved certificates. In contrast, our defense only requires to run forward *once* with an off-the-shelf diffusion model.

## 5 Discussion and Future Work

Our work offers a new perspective on the cat-and-mouse game played in clean-label poisoning.

Initial claims from the cat:
Clean label poisoning attacks, with human-imperceptible perturbations that keeping the original labels, are an attractive option to inject malicious behaviors into models. Even if a victim carefully curates the training data, they remain effective by evading *filtering* defenses that rely on statistical signatures distinguishing clean samples from poisons (Nelson et al., ; Suciu et al., ; Peri et al., ).

The mouse’s counterarguments: Yet, as seen in research on the adversarial robustness, such small perturbations can be *brittle* (Carlini et al., ): by adding a large quantity of noise and then denoising the images, the adversarial perturbations become nearly removed—and so clean-label perturbations are also removed. In prior work (and in this work as well), we leverage this observation and propose both certified and non-certified heuristic defenses.

Additional arguments from the cat:
A counterargument from the clean-label attack’s side was that those defenses inevitably compromise a model’s performance—a fact corroborated by the prior work on the adversarial robustness (Tsipras et al., ; Zhang et al., ). Similarly, defenses could greatly reduce the poisoning attack success, but at the same time, they decrease the accuracy of defended models, often tipping the scales in favor of the adversary. If a defense yields a CIFAR-10 model with 60–70% utility, what is the point of having such models in practice? Indeed, our own certified models require degraded utility to achieve certified predictions. Other defenses, e.g., DP-SGD (Ma et al., ; Hong et al., ), are computationally demanding, increasing the training time an order of magnitude.

Our work shows that this cat-and-mouse game need not be as pessimistic as the cat portrays.

By leveraging an off-the-shelf DDPMs (Ho et al., ; Nichol & Dhariwal, ), we can purify the training data, possibly containing malicious samples, *offline* and render six clean-label attacks ineffective. With a weak provable guarantee against the training data contamination ($\ell_{\infty}$-norm of 2), the majority of attacks reach to 0% success. A few recent attacks exhibit 0–10% success rate—a rate comparable to random misclassification of test-time inputs.

In contrast to the cat’s counterargument, we can also address the accuracy degradation problem—as long as we do not need certified robustness. Existing defenses that work to defend against poisoning require applying a particular training algorithm designed by the defender (Hong et al., ; Geiping et al., ; Liu et al., ; Levine & Feizi, ; Wang et al., ; Zhang et al., ). By decoupling the training process and the defense, we open a new opportunity to develop training techniques that can enhance the accuracy of the defended models while keeping the robustness. Our results propose two effective heuristic algorithms in improving defended models’ utility, especially when the models are under strong defense guarantees (i.e., $||\epsilon||_{\infty}>2$).

Next steps.
One of our main conclusions—other than that our defense is
effective—is that there is an extremely large gap between
the certified accuracy and best practical attack accuracy.
When using diffusion models to denoise adversarial examples,
a practical adversary can achieve within a few percentage points
of the worst-case certified result; for example,
if we can certify a lower bound of $50\%$ robust accuracy,
a practical adversary could achieve $55\%$ robust accuracy.
But in clean-label poisoning,
even at perturbation levels where we can’t
certify *anything*, the attacks fail to decrease
the model utility at all.
This implies either that
(1) current attacks are *not fully exploiting* the space of possible attacks,
or
(2) that the certificate provided is loose and a more careful analysis could
tighten it.

In addition, when studying stronger clean-label poisoning attacks, we encourage future work to include our defense as a strong baseline. We show in §4 that the defense renders the state-of-the-art clean-label attacks ineffective and is, by far, the most effective defense among the existing ones. We believe that strong attacks future work will develop should at least survive against this defense. Our defense is also designed to run without any additional training costs—with an off-the-shelf diffusion model, one can easily denoise the tampered training data and run the training of a model on it.

## Acknowledgements

This work is supported by Google Faculty Research Award.

## Impact Statement

Our results have no immediate negative consequences.
Below we discuss a few implications of our work for how
the field of poisoning may progress as a result of our methods.

First, our results do not imply that clean-label poisoning attacks are an ineffective threat and that we should stop studying. Instead, the results shows that clean-label poisoning has been studied in adversary-friendly settings. Current attacks are *weak*: an adversary crafts poisons bounded to the perturbations of 16-norm, yet they become ineffective with our $2$-norm bounded defense. This implies the current attacks are *not-fully* exploiting the 16-norm perturbation bound and the need to study the worst-case poisoning adversary who remains effective under our defense.

In addition, when studying stronger clean-label poisoning attacks, we encourage future work to include our defense as a strong baseline. We show in §4 that the defense renders the state-of-the-art clean-label attacks ineffective and is, by far, the most effective defense among the existing ones. We believe that strong attacks future work will develop should at least survive against this defense. Our defense is also designed to run without any additional training costs—with an off-the-shelf diffusion model, one can easily denoise the tampered training data and run the training of a model on it.

It may be possible for future work to tighten the certified robustness our defense offers further. Making an analogy to the privacy-utility trade-off, prior work studies a series of methods that can improve the utility of differentially-private models with the same worst-case private leakage (Tramer & Boneh, ; Papernot et al., ; Ye & Shokri, ). Similarly, in the context of clean-label poisoning, a promising direction for future work will be to find methods to achieve a better trade-off between the certified robustness to the clean-label attacks and the model utility.

## Appendix A Experimental Setup in Detail

We adapt the poisoning benchmark from the prior work (Schwarzschild et al., ). Most work on clean-label poisoning attacks uses this benchmark for showcasing their attack success. The open-sourced implementation of Sleeper Agent is incompatible with this framework; thus, we run this attack separately. We implement the attacks to use a comparable $\ell_{2}$-norm bound in the poison-crafting process (see §B). We implemented our benchmarking framework in Python v3.7(^3^33[https://www.python.org/](https://python.org)) and PyTorch v1.13(^4^44[https://pytorch.org](https://pytorch.org)). We use the exact attack configurations and training hyper-parameters that the original study employs.

In contrast to the original work, we made two major differences. We first increase the perturbations bounded to $||\epsilon||_{\infty}\!=\!16$ as the 8-pixel bound attacks do not result in a high attack success rate. Defeating 8-pixel bounded attacks is trivial for any poisoning defenses. Second, we do not use their fine-tuning subset, which only contains 2500 training samples and 25–50 poisons. It (as shown in the next subsections) leads to 60–70% accuracy on the clean CIFAR10 test-set, significantly lower than 80–90%, which can be trivially obtained with any models and training configurations. If a model trained on the contaminated training data misclassifies a target, it could be a mistake caused by a poorly performing model.

We run a comprehensive evaluation. We run 7 attacks; for each attack, we run 100 times of crafting and training/fine-tuning a model. We also examine 5 different denoising factor $\sigma$. In total, we ran 3500 poisoning attacks. For the most successful three attacks, we run 6 different certified and non-certified defenses over 100 poisoning runs. The three training-time defenses (DP-SGD, AT, FrieNDs) require 100 trainings of a model for each, and the certified defense (RoE) requires 100$\times$250 trainings in total. To accommodate this computational overhead, we use two machines, each equipped with 8 Nvidia GPUs. Crafting a poisoning set takes approximately one hour and training takes 30 minutes on a single GPU.

## Appendix B Translating l ∞ subscript 𝑙 l_{\infty} italic_l start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT -bound into l 2 subscript 𝑙 2 l_{2} italic_l start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT -bound

The certification we offer in Sec 3 is defined in $l_{2}$-norm, while most poisoning attacks work with $l_{\infty}$-norm, e.g., of 8 or 16. We therefore convert these $l_{\infty}$-bounds into $l_{2}$-bounds. We assume the worst-case perturbation in the $l_{\infty}$-space that changes every pixel location of an image by 16 pixels, compute the $l_{2}$-norm of that perturbation as follows, and use it in §4:

- •
$l_{\infty}$-bound of 8 pixels: $\sqrt{3*32*32*(8/255)^{2}}=1.74$
- •
$l_{\infty}$-bound of 16 pixels: $\sqrt{3*32*32*(16/255)^{2}}=3.48$

Oftentimes, the $\ell_{2}$ attacks with the comparable bound 3.48 do not lead to comparable attack success. We, therefore, especially for the poisoning attacks in the transfer-learning scenarios, increase the bound to 6.43. We note that most existing attacks either do not implement $\ell_{2}$ bounds or are unbounded (Shafahi et al., ).

## Appendix C More Experimental Results

Here we include additional experimental results.

### C.1 Using the Evaluation Setup by Schwarzschild et al. ( 2021 )

We examine the weaker adversary whose perturbation is bounded to $||\delta||_{\infty}=8$.
We also employ the exact setup as the prior work (Schwarzschild et al., ),
where we craft 25 poisons on a ResNet18 pre-trained on CIFAR-100
and fine-tune the model on a subset of the CIFAR-10 training data.
The subset contains the first 250 images per class (2.5k samples in total).

Table: Table 6: Diffusion denoising against clean-label poisoning (CIFAR10). We denoise the $l_{\infty}$-norm of 8 perturbations added by five poisoning attacks by running a single-step stable diffusion on the entire training set. In each cell, we show the average attack success over 100 runs and the average accuracy of models trained on the denoised data in the parenthesis. Note that ${}^{\dagger}$ indicates the runs with $\sigma$=0.0, the same as our baseline that trains models without any denoising.

Table 6 summarizes our results against clean-label poisoning attacks with $l_{\infty}$-norm of 8. We examine five poisoning attacks in the white-box and black-box scenarios. We focus on the attacks against transfer-learning as the specific data splits the prior work (Schwarzschild et al., ) uses are only compatible with them. Poisoning attacks against training from scratch scenarios use the entire CIFAR10 as ours, so we don’t need to duplicate the experiments. In the white-box setting, we fine-tune the CIFAR-100 ResNet18 model (that we use for crafting poisons) on the poisoned training set. In the black-box setting, we fine-tune different models (VGG16 and MobileNetV2 models pre-trained on CIFAR-100) on the same poisoned training set. We use $\sigma$ in $\{0.5,1.0,1.5,2.0\}$ for our single-step DDPM. $\sigma$=0.0 is the same as no defense.

Results.
Diffusion denoising significantly reduces the poisoning attack success. The most successful attack, Bullseye Polytope in the white-box setting, achieves the attack success of 100% in $l_{\infty}$-norm of 8 pixels, but denoising with $\sigma$ of 0.5 can reduce their success to 10%. Our defense reduces the attack success of Poison Frog! and Convex Polytope from 13-34% to 2-8% at $\sigma=0.5$. The two backdoor attacks (label-consistent and hidden trigger) exploiting clean-label poisoning are not successful in the benchmark setup (their success rate ranges from 2-7.5%). We thus could not quantify our defense’s effectiveness against these backdoor attacks. Note that Schwarzschild et al. () also showed these backdoor attacks ineffective, and our finding is consistent with their results. We exclude them from our Tiny ImageNet results in §C.2.

We also observe that the increased $\sigma$ (strong denoising) can significantly reduce the utility of a model trained on the denoised training data. Note that since the setup uses only 2.5k CIFAR10 samples for fine-tuning, and in consequence, the model’s utility is already $\sim$70% at most, much lower than our setup. As we increase the $\sigma$ from 0.5 to 2.0, the fine-tuned model’s accuracy leads to 56% to 27%. However, we show that with the small $\sigma$, our diffusion denoising can reduce the attack success significantly. We also show in our evaluation (§4) that we achieve a high model’s utility while keeping the same $\sigma$. We attribute the increased utility to recent model architectures, such as VisionTransformers, or to pre-training a model on a larger data corpus. We leave further investigation for future work.

Moreover, in a few cases, the poisoning success increases from 2–7% to 10–13% as we increase $\sigma$. We attribute this increase not to the attack being successful with a high $\sigma$ but to the poor performance of a model. For example, the accuracy of a model with $\sigma=2.0$ is $\sim$27%, meaning that four out of five targets in a class can be misclassified.

### C.2 Tiny ImageNet Results

Table: Table 7: Diffusion denoising against clean-label poisoning (Tiny ImageNet). We consider four attacks with the $\ell_{\infty}$-norm of 16 perturbation bound. In each cell, the attack success and accuracy of models trained on the denoised data in the parenthesis on average over 100 runs. Note that ${}^{\dagger}$ indicates the runs with $\sigma$=0.0, representing no-defense scenario.

We also ran our experiments with Tiny ImageNet to examine whether our findings are consistent across different datasets. We assume the same adversary who can add perturbations bounded to $\ell_{\infty}$-norm of 16 pixels. For the three attacks in transfer-learning scenarios, we craft 250 poisoning samples on a VGG16 pret-trained on the same dataset and fine-tune the model on the tampered training set. We do not evaluate backdoor attacks because they are either ineffective (label-consistent and hidden trigger) or the original study did not employ Tiny ImageNet (Sleeper Agent).

Results.
Table 7 shows our results. Our results are consistent with what we observe in CIFAR10. Note that the poisoning attacks are more successful against Tiny ImageNet, and we attribute the success to the learning complexity of Tiny ImageNet over CIFAR10: Tiny ImageNet has 200 classes and 100k training samples. Prior work (Schwarzschild et al., ) had the same observation. Our defense significantly reduces the attack success. Bullseye Polytope in the white-box setting, achieves the attack success of 100% without our defense, but denoising with $\sigma$ of 0.25 reduces their success to 10–15%. We find the defense reduces the success of Poison Frog! and Convex Polytope from 79–95% to 3–15% at $\sigma=0.1$. We also observe that the increase in $\sigma$ significantly degrades the model utility., e.g., $\sigma=$1.0 loses the accuracy by 27%.

## Appendix D (Denoised) Poisoning samples

Table: Table 8: Visualize poisoning samples. We, for the CIFAR10 training data, display the poisoning samples crafted by different clean-label poisoning attacks ($\ell_{\infty}$-norm of 16). We also show how the perturbations are denoised with difference $\sigma$ values in $\{0.1,0.25,0.5,1.0\}$. $\sigma=$ 0.1 yields to ineffective poisons. $\sigma=$ 0.0 means we do not denoise the poisons.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: N/A
soundness: 3
strengths: 1. By decoupling the training and defense processes, the proposed approach enables a training technique that enhances model accuracy while preserving robustness.
2. The authors provide extensive experimental evidence to support the effectiveness of their defense mechanism against clean-label poisoning attacks.
summary: This paper introduces a defense mechanism against clean-label poisoning attacks by denoising training samples with a diffusion model prior to training. The goal is to mitigate the impact of poisoned samples, which might otherwise cause clean-label attacks.
weaknesses: 1. The paper claims to extend the algorithm by Cohen et al. (2019) to a clean-label poisoning setting. However, the method appears to be a straightforward extension of the work by Carlini et al. (2023) [1]. Comparing with the work proposed by Carlini, this work seems not sufficiently novel. Can the author(s) provide comparison of their proposed defense with the defense proposed by Carlini to claim their novelty?
2. In Table 1, the attack success rate is reported as nearly 99% without defense. This result suggests that the inclusion of a small fraction of poisoned data causes almost complete misclassification of all test samples of the target class $y_t$ to the clean label $y_b$ (correct me if I am wrong). Could the authors clarify how the models were trained? Line 283 states that the entire training dataset (50,000 samples for CIFAR-10) is used, implying that all 5,000 samples of the target class are included during training. If so, why do only 500 tampered samples cause such a high misclassification rate?
3. The difference between white-box and black-box attacks, mentioned in line 289, could be further elaborated to enhance the reader's understanding.
4. The writing could be improved for clarity and precision. Specific examples include:

    a) In line 83, clarification is needed regarding whether it should be "$y_b = y_p$" or "$y_b = y_p$.

    b) The optimization problem presented in line 85 is confusing. Terms such as $y_t$ and $y_{\text{adv}}$ are used interchangeably across lines 85, 88, and 91, which requires further clarification. Additionally, $y_{\text{adv}}$ should be explicitly defined—does it refer to the same label as $y_b$?

    c) Check the numbering of tables in lines 356 and 407.

    d) In Table 4, the terms TP and FP are used but never defined.




[1] Carlini, Nicholas, et al. "(certified!!) Adversarial robustness for free!." arXiv preprint arXiv:2206.10550 (2022).

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: Please see weaknesses for details.
soundness: 2
strengths: - The proposed approach is simple. The authors leverage the diffusion model to remove adversarial noise added on the poisoned data.

- The authors proposed the strategy of warm-starting to improve the model utility.

- The authors evaluate the proposed defense over difference settings such as training from scratch and transfer-learning.
summary: In this paper, the authors propose a certified defense against clean-label data poisoning attacks using diffusion denoising. The authors sanitize the training dataset by using an off-the-shelf diffusion model to remove adversarial perturbations on poisoned data. They also propose a warm-starting initialization to improve the model utility under the defense. Experiments are carried out on CIFAR-10, CIFAR-100 and Tiny ImageNet to evaluate the defense against multiple attacks and comparisons are performed with existing heuristic and certified defenses.
weaknesses: I happened to be the review of this paper when it was submitted to ICLR2024 and NeurIPS this year. Compare to its prior version, the authors improved presentation and addressed some concerns, but still have the following issues:

- One of the most important claims of this work is that the proposed approach is a certified defense and it offers a provable guarantee. However, there is missing theoretical proof for the certified defense against poisoning attacks in the paper. The proposed defense is introduced follows the random smoothing, which is a certified defense against evasion attacks that is fundamentally different from poisoning attacks. The authors should clarify how the guaranteed robustness is achieved. In the prior rebuttal, the authors agreed to provide full proof in their Appendix, but  the proof is still missing in this version.

- I favor the simplicity of the proposed approach (i.e., the authors use an off-the-shelf diffusion model to remove the noise imposed on the poisoned samples), but the motivation of this idea is not well explained. Why pick diffusion model as a denoising approach over other techniques? Why does diffusion model work well in removing the adversarial perturbations?

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: 1. Why Figure 2 does not have the results for $\sigma=0.1, 0.5$? And Figure 1 and Figure 2 should be in the same figure for better computation.
2. From Table 5, I cannot see a clear improvement when comparing your method to ROE.
soundness: 3
strengths: 1. The paper is well written with clear descriptions and comprehensive experiments.
2. This is the first paper to apply the method of diffusion denoising to defense against clean-label poisoning attacks for certified robustness.
3. To evaluate the model, the author considers both model utility and robustness.
summary: This paper proposes a novel method to evaluate certified robustness of clean-label poisoning attacks by applying diffusion denoising on the tampered training data. Testing the defense method against various clean-label attacks, the experiment results have shown that the proposed method can effectively reduce their attack success rate and outperform previous baselines in both robustness and model utility.
weaknesses: 1. The method is hard to widely used for large scale. In Section 3.3, there is still a need to train 10k models for the certification process. With the scaling on both model and data scale, it is hard for it to be applied in real scenarios. 
2. It would be better if there is a time cost comparison with other defense methods (both heuristic and certified defense), since performing the denoising process on the training set is still a huge cost.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: Please see the weakness.
soundness: 2
strengths: 1. The research approach is quite meaningful. Current defenses against poisoning attacks generally lack theoretical explanations, so employing a certified defense is a very promising idea.  

2. The experimental results are excellent. The paper conducts defense experiments against both targeted poisoning attacks and backdoor attacks, and the results are very good.
summary: This paper introduces a certified defense mechanism against clean-label poisoning attacks. The defense leverages a diffusion model, inspired by the adversarial robustness of randomized smoothing, to purify tampered training data. The proposed method is tested against seven different clean-label poisoning attacks, with minimal impact on test accuracy. Compared to existing countermeasures, this approach significantly lowers attack success while maintaining model performance.
weaknesses: 1.The novelty of the approach is insufficient. In fact, there is already work [1] that proposes using the idea of diffusion denoising to defend against clean-label indiscriminate poisoning attacks, which reduces the novelty of this paper. The authors are encouraged to cite this related work for making this paper's contribution more clear. Moreover, the biggest issue with using diffusion purification to defend against poisoning attacks is whether the diffusion model was trained on a clean training dataset. If the diffusion model is trained on a clean training dataset (or even a small part), the entire defense approach becomes impractical. The paper should make this point more clearly, since I was wondering if the trained diffusion model had used clean training data.

2.Old backdoor attack baselines. The newest backdoor attack evaluated in this paper is 2022. The authors are supposed to evaluate more recent baselines [2].

3.In Line 437-438, the authors mentioned that "Ours and FrieNDs greatly reduce the attack success while preserving the model’s utility.The critical difference is that ours is a certified defense, while FrieNDs is not. A future work we leave is developing adaptive attacks against FrieNDs.", it indicates that the performance of the proposed diffusion denoising scheme is not the best. Therefore, I cannot find the absolute advantages of the proposed defense. The authors are supposed to explore more strengths of the proposed scheme to defeat existing baseline works. 

4.Typos: Line 407: Table 5->Table 4. By the way, why does the Table 4 not include the results of Sleeper Agent? 
Line 439-440 "Certified defenses are" ?

5.Why do the authors not report the defense results against Metapoison [3]?

[1]ECLIPSE: Expunging Clean-label Indiscriminate Poisons via Sparse Diffusion Purification. ESORICS'24

[2]Narcissus: A practical clean-label backdoor attack with limited information. CCS'23

[3]Metapoison: Practical general-purpose clean-label data poisoning. NeurIPS'20