Paper: Mitigating backdoor attacks with generative modelling and dataset relabelling (ICLR-2024)

Reviewer comment (Theory):
The authors define \(D^y_{F}=\{f_{\theta_F}(x_i) \vert y_i = y \}\). As I understand it, \(D^y_F\) contains the set of features for all training instances whose true label is \(y\). (Note at the bottom of page 3, I believe \(y_i\) is defined as the true labels). Of course, \(y_i\) for the poisoned data is unknown (otherwise, the problem is trivial). I cannot determine whether there is a problem with the notation or method, but it seems \(D_F^y\) is not known as defined. Perhaps the authors are assuming a clean validation set to learn these generative models (as in other work), but I do not see a discussion of that. This is a major concern and one reason I rate the soundness as 1.

Author response:
We apologize, we have made an error in the definition of $D^y_\text{F}$ by omitting $(x_i,y_i) \in \widetilde{\mathcal D}$. 
The original labels are not known and the definition should have been $D^y_\text{F}=\{f_{\theta_F}(x_i) \mid (x_i,y_i) \in \widetilde{\mathcal D}, y_i = y \}$.

Reviewer comment (Presentation):
I strongly recommend either removing or redesigning Figure 2. The flow of the figure is very non-linear and non-intuitive. Best I can determine, the figure could have a linear progression starting at the initial poisoned dataset and terminating with the final trained model. Perhaps the authors chose this non-linear progression to save space, but I would view this as an especially poor choice. One potential way to solve this problem entirely is to change Figure 2 to an algorithm.

Author response:
Would you consider a description like this as an improvement over Figure 2?

1. Train a self-supervised feature extractor $f_{\theta_{\text{E}}}$ on images from the potentially poisoned dataset $\tilde D$.
2. Learn per-class densities $p_{\theta_{y}}(z)$ self-supervised features $z=f_{\theta_{\text{E}}}(x)$.
3. Identify disruptively and non-disruptively poisoned classes.
4. Assign poisoning scores $s_y(z)=\frac{p_{\theta_y}(z)}{\arg\max_{y'\neq y} p_{\theta_{y'}}(z)}$ to examples from classes $y$ identified as poisonous.
5. Split $\tilde D$ into subsets based on poisoning scores $s_y(z)$:
	* $\hat D_\text{P}$: poisoned examples,
	* $\hat D_\text{O}$: uncertain examples,
	* $\hat D_\text{C}$: clean examples from classes identified as targets and all examples from classes identified as clean.
6. Produce the cleansed poisoned subset $D_\text{P}'$ by assigning pseudo-labels to the examples from $\hat D_\text{P}$ according to $\arg\max_{y'\neq y}p_{\theta_{y'}}(z)$.
7. Produce the cleansed dataset as $\hat D' = \hat D_C \cup D_P'$.
8. Train a discriminative model on $\hat D'$.

Reviewer comment (Writing):
Several typographical issues exist in the paper. These did not affect my overall score. Examples include:
- Page 1: "...stealthines..."
- The authors repeatedly use `\citet{...}` in place of `\citep{...}`. See for example the two citations on page 1 in the paragraph that begins "In this work, ...".
- Page 7: "...fllowing...:
- Page 7: ", Sample specific" ("S" should not be capitalized here)
- Page 7: "sample0specific"
- Page 13: In multiple places, the letter "x" is used in math mode when specifying dimensions resulting in the "x" being italics. Either do not place the x in math mode or better use `\times` instead of "x".

Author response:
Thank you!

Reviewer comment (Evaluation):
Table 4 should show the minimum poisoning rate where either the attacks or the defenses start to fail. For example, does your defense still work at 0.1% poisoning rate?

Author response:
This is a valuable suggestion. In our recent experiments,
we discovered that a poisoning rate of 0.1% is indeed 
sufficient to compromise our defense. The scarcity of poisoned samples results in the failure to accurately identify the target class, consequently preventing the elimination of these poisoned samples.

Reviewer comment (Experiments):
Poisoning rate is also only one dimension of an attack's strength. Perturbation strength is an orthogonal dimension of attack strength against which the authors' defense is surely highly susceptible but is not explored in the empirical evaluation.

Author response:
We have noticed that some attacks that are non-disruptive become distruptive above some poisoning rate threshold.
It also seems intuitive that stronger perturbations might make the attack more likely to be disruptive because the poisoned examples then bear less resemblance to their original class. 
In all our experiments, our defense has worked well in both cases.

Reviewer comment (Experiments):
The empirical evaluation's main results use either a 10% or 25% poisoning rate. In my view, those attack rates are wholly unrealistic for any marginally plausible real-world scenario. I would go so far as to consider those poisoning rates not meaningful to study since I cannot see a case where an attacker is inserting 25% poisoned data.

Author response:
We apologize if it was not clear: the 25% denotes that only 25% of the poisoned class consists of poisoned examples when the clean-label attack (LC) is used.
We will run additional experiments with the clean-label attacks suggested by reviewer U1ob.

We have considered attacks that poison 10% examples in order to be in line with previous work [1, 2].
Do you have any suggestions about what to read to get a better idea of the attacker capabilities in important real-world scenarios?

Reviewer comment (Evaluation):
The last two sentences in Section 4.5 are confusing. Although the numerator is extremely low, why is the score significantly higher? Also, if the poisoned samples score significantly higher than clean samples, aren't those poisoned samples mislabeled into the clean set? Finally, the arguments may be invalid by ignoring the change of the denominator.

Author response:
Thank you for your feedback!

>The last two sentences in Section 4.5 are confusing. Although the numerator is extremely low, why is the score significantly higher? Also, if the poisoned samples score significantly higher than clean samples, aren't those poisoned samples mislabeled into the clean set? Finally, the arguments may be invalid by ignoring the change of the denominator.

We apologize. The penultimate sentence should have said *denominator* instead of *numerator*. 
The numerator will be higher because the generative model of the poisoned class is trained on the poisoned examples.

If the above explanation does not resolve your doubts, please clarify "the change of the denominator".

Please find the answers to your other comments in our response to all reviewers.

Reviewer comment (Experiments):
Even though the proposed method is working well, it is highly inefficient and requires lots of compute. In particular, the proposed method starts with self-supervised pre-training of a feature extractor using SimCLR. Then, it trains _one_ normalizing flow _per each class_ to finally be able to get rid of poisonous samples and start training a robust model. Such extensive use of resources is quite intensive, and frankly speaking, might be redundant. The field of backdoor defense has came up with alternative solutions such as [1-4] that are far more efficient than the proposed solution where some of them just take one training round to give robust models. Apart from the expensive self-supervised training at the beginning, the proposed solution requires one flow-based model per class which means that its resources grows linearly with the number of classes.

Author response:
In case of a dataset with input size $224\times224$, 10 classes and ~5000 example per class, 
self-supervised representation learning takes 10000 s,
supervised training (200 epochs) takes 2750 s, 
while the training of generative classifiers takes about 90 s per class regardless of image size. 
We observe that our generative setups involve a neglectable processing overhead. 
Moreover, this can be further improved by replacing C per-class models with 1 model of joint density $p[z, y]$. 
Self-supervised learning indeed is computationally expensive, however that seems as an acceptable price for a substantial reduction of vulnerability to cybernetic attacks.

Reviewer comment (Experiments):
Why do we need to identify/remove poisonous samples using class conditional normalizing flows and then train another neural network of our task? In other words, can't we just use the per-class normalizing flow for classification as well? Running experiments on this scenario is highly encouraged.

Author response:
We agree that it would make sense to test the normalizing flows as the final classifier or test such a classifier re-trained on the cleansed dataset.
Given our assumptions, we expect the following:
- In case of non-disruptive attacks, we expect such a classifier to correctly classify some, but still be ambiguous and misclassify many of the poisoned examples with respect to the original class. In cases when it is ambiguous on two classes, those classes might often be the original (poisoned) class and the target class.
- In case of disruptive attacks, we would expect such a classifier to often classify the poisoned examples into the target class because they are not similar to their original class in the self-supervised feature space.
We will test this empirically.

Most practical setups will prefer to re-train the discriminative model on cleansed data since i) discriminative models perform better on discriminative tasks. 
Note that the resulting computational overhead is not large compared to feature extraction, and that it would be an acceptable price for the delivered cleansing performance. Recent comparison of generative and discriminative classification performance can be found in [1].

Reviewer comment (Experiments):
Can you repeat the same process for generating Figure 1 for other attacks?

Author response:
Yes. We will generate Figure 1 for other attacks and include it in the future revisions of our work.

Reviewer comment (Reproducibility):
What experimental settings (number of epochs, etc.) are used for SimCLR? What is the architecture of normalizing flows?

Author response:
We perform SimCLR training for 100 epochs with a batch size of 256 and 
Adam optimizer with 
a fixed learning rate of $3\cdot10^{-4}$. 
Our per-class normalizing flow consists of two steps which consist of actnorm [4] and an affine coupling [5].
Each coupling module computes the affine parameters with a pair of fully-connected layers with ReLU between them.
We shall include these pieces of information in the next revision of our work.

Reviewer comment (Experiments):
Why so many number of epochs (200) is used for training models? Usually, 120 epochs is enough to train ResNet models with SGD on CIFAR-10.

Author response:
We were following the training configurations from previous work [2, 3].

Reviewer comment (Experiments):
I think the paper can improve with investigation of what makes a generative model stand out? Perhaps an investigation of how adaptive attacks can circumvent the threshold based detector?

Author response:
Thank you for your feedback!

>I think the paper can improve with investigation of what makes a generative model stand out?

If we understand correctly, we agree that we should test how important the conditional generative model $p[z\mid y]$ is by comparing its feature densities with class probabilities of a suitable discriminative model $p[y\mid z]$.

Please find the answers to your other comments in our response to all reviewers.