Paper: Assessing Robustness via Score-based Adversarial Image Generation (ICLR-2024)

Reviewer comment (Experiments):
Interaction between Different Tasks: The authors could explore and elucidate the interactions between different tasks such as GAS, GAT, and purification. Specifically, it would be insightful to understand the impact of applying purification post-GAS or GAT. A discussion on whether purification enhances the robustness of the adversarial examples generated by GAS or GAT would be valuable. This could provide readers with a deeper understanding of the potential synergies or trade-offs between these tasks.

Author response:
**Comment:** Could you elaborate on the relationship between different tasks such as GAS, GAT, and purification within the ScoreAG framework? Can they be integrated into an ensemble for attack or defense?  
**Response:** In our framework, GAS and GAT are different forms of diffusion-based attacks, and GAP is a diffusion-based defense. The purification task GAP can be applied to defend against attacks generated by GAS and GAT. In Tab. 2, we show the performance of GAT on GAP. For convenience, we show it in the following table with the accuracy of GAS on GAP.

| Method | GAT | GAS |
| ------ | -------- | -------- |
| GAP    | $90.74 \pm 0.67$ | $83.14\pm 1.50$     |

GAP is capable of successfully defending against both attacks, GAT and GAS.

Reviewer comment (Experiments):
Semantic Preservation: The paper could benefit from a clearer explanation and justification of the semantic requirements (Ω) used in the experiments. The methodology and criteria used to ensure semantic coherence and relevance in the generated adversarial examples should be thoroughly discussed. Consider discussing and comparing the semantic perturbations introduced by ScoreAG with those from other image-based semantic perturbation adversarial attack methods, such as the one mentioned ("SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image Editing"). Including detailed results or comparisons concerning semantic preservation could bolster the reliability and credibility of the proposed method.

Author response:
**Comment:** How does ScoreAG ensure that the generated adversarial examples maintain semantic coherence and relevance?  
**Response:** ScoreAG ensures the semantic coherence and relevance of generated adversarial images through two key mechanisms. Firstly, we employ a class-conditional score function, i.e., $\nabla\_{\mathbf{x}\_t}\log p\_{t,y^*} (\mathbf{x}\_t)$ (see Eq. 7), ensuring that the adversarial images retain the semantic attributes of the target class $y^*$. Secondly, the guidance term, i.e., $\nabla\_{\mathbf{x}\_t}\log p\_{t,y^*}(\mathbf{x}^*,f(\mathbf{x}\_0)=\tilde{y}|\mathbf{x}\_t)$ (see Eq. 7) ensures that the generated image preserves core-semantics of $x^*$ and is classified as $\tilde{y}$. Together, these two mechanisms generate semantic-preserving adversarial images of $x^*$.

> I would suggest describing semantic coherence in more details in the paper, perhaps with short experiments to illustrate the correlation between the two terms and the claimed semantic preservation.

We included several experiments and evaluations to investigate the correlation of these two terms. In Fig. 4, we show how the FID and the accuracy change when $s_y$ increases for the GAS task. Note that $s_y$ corresponds to a standard EDM generation. A higher $s_y$ leads to lower accuracy and increases the FID, indicating that the attack gets stronger but the image coherence slowly degrades. Furthermore, we observe the same in Tab. 3 for GAT. While increasing $s_y$ results in a stronger attack, increasing $s_x$ leads to a lower $\ell_2$ distance to the original image. We can observe this behavior in Fig. 3.

For the camera-ready version, we plan to adapt Section 4.1 to further make the implications on the semantic coherence of the presented results more clear and thus, will discuss them in more detail.

Reviewer comment (Evaluation):
Comparative Analysis in Human Study: The human study results presented in Table 4 only focus on ScoreAG without including comparisons with the baseline methods used in other parts of the evaluation. For a more comprehensive understanding and evaluation, it would be beneficial to include a comparative analysis showcasing the performance of ScoreAG against other established methods. Consider including a discussion on the variance in naturalness or the extent of perceptual shifts observed across different methods. This could provide a more nuanced view of ScoreAG’s performance in maintaining image naturalness while generating adversarial examples.

Author response:
**Comment:** Could you provide results and discussions comparing ScoreAG to other baseline methods in the human study section?  
**Response:** To ensure semantic preservation, we extended the evaluation of ScoreAG with a human evaluation. While we agree that these results would be interesting for other methods, human studies have significant financial costs, and thus, we think it is the responsibility of the corresponding authors to provide these. Hence, due to financial constraints, it is not feasible for us to include other baselines.

> I do not understand your answer regarding human study. Your answer states that "it is the responsibility of the corresponding authors to provide these (financial costs)". Isn't the corresponding author among you? I would also argue that there are means to conduct simple human studies at reasonable cost, especially when it is only about comparing images.

Our human study cost several hundreds of euros as hundreds of images have to be classified individually by 60 human evaluators. In our case, this totaled over 32 hours of human labor for which we paid the recommended (by Prolific) hourly wage, as we believe in fair pay. Thus, each other baseline would cost again hundreds of euros. As a result, we see the responsibility to show the semantic preservation property of a method at the authors of that method (in the same way as the authors have to show the effectiveness of their proposed method in other metrics). As such an investigation is lacking in all related previous work, we see our study as significantly improving on the current evaluation standard.

**Comment:** Could you elaborate on the perceptual shifts or changes in the naturalness of adversarial images from ScoreAG compared to other methods?  
**Response:** Existing works show that diffusion models are able to capture the diversity of the training distribution [1, 2]. Furthermore, we show that our synthetic adversarial attacks have a competitive FID, indicating a high similarity between the adversarial synthetic images and real images. We can also observe a high degree of realism and diversity in our qualitative examples.

We thank the reviewer again for their comments and feedback. We hope that we have satisfactorily answered the reviewer's questions. If you have any open concerns, please let us know.

[1] **Bayat, Reza.** "A Study on Sample Diversity in Generative Models: GANs vs. Diffusion Models." (2023).

[2] **Dhariwal, Prafulla, and Alexander Nichol.** "Diffusion models beat gans on image synthesis." Advances in neural information processing systems 34 (2021): 8780-8794.

Reviewer comment (Evaluation):
More analysis is needed on why ScoreAG outperforms the other diffusion-based attack DiffAttack. The reasons are not fully clear.

Author response:
While the ScoreAG and DiffAttack have many differences, we assume ScoreAG's key advantage is the ability to go through the whole latent space by utilizing diffusion guidance, while DiffAttack is limited to the last few steps. To verify this empirically, we compare how the performance of ScoreAG changes in the GAT setting when only denoising the last few steps.



| Full reverse | 5 reverse steps | 10 reverse steps | 20 reverse steps |
| -------- | -------- | -------- | ---- |
| 0.0     | 0.28     | 0.21     | 0.09 |

The table shows the accuracy after applying GAT on the first 100 images of Cifar-10. As expected, 5 and 10 reverse steps, i.e., the values used by DiffAttack, are not sufficient for a successful attack.

Reviewer comment (Evaluation):
The human study provides useful insights but is limited in scope. Expanding this to quantify semantic similarity and diversity of examples would strengthen the evaluation.

Author response:
We agree with the reviewer that quantifying semantic similarity and diversity of the adversarial examples are important but not trivial as well. In our human study, we implicitly quantify similarity by capturing the semantic preservation through the human accuracy. For the GAT task, the human accuracy reaches 94%, implying semantic preservation of the images. The diversity, however, is correlated with the capabilities of the generative model and the dataset itself. We quantify the diversity of the generative model using the FID score in Fig.4 (left).

Reviewer comment (Evaluation):
Although GAS aims to generate the adversarial example from scratch that would be misclassified by the classifier while preserving the semantics of the truth class, it is disappointing to see that even human fails to classify the adversarial examples. As shown in Table 4, Human accuracy on the adversarial synthetic images is only 70%, this results do not support the argument that it preserves the semantics of a certain class.

Author response:
**Comment:** Human accuracy on the adversarial synthetic images is only 70%.  
**Response:** The generative adversarial transformations (GAP) achieve an almost perfect human accuracy of 94%, equal to the accuracy on the non-adversarial synthetic images. This indicates that the adversarial attack does not degrade the semantic coherence for humans. We respectfully disagree with the reviewer that humans fail to classify the synthetic adversarial images. This is a novel task, and the human accuracy is 70%, which is 60% higher than guessing, implying a success. We expect future work to further improve these results with the development of better generative models.

Reviewer comment (Novelty):
A comparison with similar work [1] should be provided, along with an explanation of the differences in approach and findings.

Author response:
**Comment:** Comparison with similar work should be provided.   
**Response:** We would like to thank the reviewer for the interesting reference. However, as the referenced paper was only recently published at ICCV2023, we would like to remind the reviewer that this work is categorized as concurrent under ICLR guidelines: 
> We consider papers contemporaneous if they are published (available in online proceedings) within the last four months. That means, since our full paper deadline is September 28, if a paper was published (i.e., at a peer-reviewed venue) on or after May 28, 2023, authors are not required to compare their own work to that paper. 

However, we include a discussion of it in the related work section. Unfortunately, we are not able to include it in our experiments, as the code is not publicly available yet.

Reviewer comment (Evaluation):
It is unclear what the "ScoreAG" column represents in Table 2. If it indicates GAP, it is unclear what the given adversarial image ($\mathbf{x}_\text{ADV}$) is used to purify. If it is GAT, isn’t the higher robust accuracy representing the weaker attack efficacy? Further clarifications are needed to understand the evaluation.

Author response:
**Comment:** Unclear what Table 2 represents.  
**Response:** The column labeled "ScoreAG" represents the attack GAT, while the corresponding row represents the purification GAP. We added this information to the table. While GAT aims to achieve a low robust accuracy, GAP aims to achieve a high accuracy. GAT achieves a slightly better accuracy than APGD and APGDT in many cases, implying a more comprehensive robustness assessment.

Reviewer comment (Evaluation):
In Section 4.2, the author argues that the $\ell_p$-bounded methods display noticeable noisy fragments. However, I think it is because of cherry-picking. Also, I do not understand where the statement “*the removal of a small fish — which prove to be important classification cues*” comes from.

Author response:
**Comment:** Authors claim that $\ell_p$-bounded methods display noticeable noisy fragments. Was it cherry-picking? Also, where does the *the removal of a small fish* statement come from?  
**Response:** The images were not cherry-picked, and we observed the same noisy fragments across all samples. [We uploaded more **randomly chosen** example images for APGD, APGDT, Square, and ScoreAG to Figshare.](https://figshare.com/s/117b453e5289311df4c7) The importance of the removal of the small fish refers to Fig. 1a and 1b, meaning the observation that many of the images generated by ScoreAG remove or modify the fish next to the shark (representing the image class) in the image.

Reviewer comment (Novelty):
The practical application of the proposed approach should be discussed. How can it be used in real-world scenarios? How is it different from other approaches in practice?

Author response:
**Comment:** Practical application of ScoreAG.  
**Response:** ScoreAG only requires a pre-trained score-based generative model and the gradients of a classifier. Therefore, it can be applied in any scenario where other attacks are currently used. Furthermore, traditional $\ell_p$-threat models often do not capture important semantics-preserving corruption relevant to real-world applications [3, 4, 5]. As ScoreAG effectively overcomes this limitation, it can be highly interesting for evaluating robustness in real-world scenarios.

Reviewer comment (Experiments):
The visualization on ImageNet seems to have high quality, it would be great to report the experimental results on it to demonstrate the generalizability of the proposed approach.

Author response:
**Comment:** Why are there no experimental results on ImageNet?  
**Response:** Many methods in our comparison already reach a near-perfect success rate on TinyImageNet, which is a subset of ImageNet. As ImageNet contains five times as many labels and a higher resolution, it is an easier target. However, we included ImageNet in our qualitative evaluation due to its high resolution.

Reviewer comment (Experiments):
Table 1 only considers one model architecture (WRN-28-10), please provide more models to verify the efficacy of the proposed method.

Author response:
**Comment:** Table 1 and Table 2 should contain more baselines, classifiers, and standard deviations.  
**Response:** We added standard deviations to Table 1 and Table 2. Furthermore, we now added two more attacks to Table 1, resulting in a total of 13 (excluding ScoreAG) evaluated attacks. Additionally, we also added four more classifiers evaluated on ScoreAG (GAT) to further demonstrate its efficacy. The following table shows the adversarial accuracy of GAT in % using the same hyperparameters as for the WRN-28-10 architecture.

| Classifier | Cifar-10 | Cifar-100 |
| -------- | -------- | -------- |
| ResNet-20 | 0.01 | 0.10     |
| ResNet-56 | 0.03 | 0.13     |
| VGG-19 | 0.52 |  1.94  |
| RepVGG-A2 | 0.04 | 0.26 |

Note that the models are obtained from PyTorch Hub. Therefore, we cannot report standard deviations. As we can observe, ScoreAG successfully generates adversarial attacks using different classifiers. We added these results to the updated manuscript.

In Table 2, we already compare ScoreAG to six state-of-the-art defenses and purification methods. However, we added an additional comparison to evaluate the certified robustness of GAP and randomized smoothing (see general comment). Note that the purification experiments require unconditional generative models as the ground-truth class labels are unknown. Therefore, our purification experiments are focused on the CIFAR-10 dataset, given the availability of unconditional pre-trained EDM models and extensive prior work [1, 2].

Reviewer comment (Evaluation):
What is the computational cost of ScoreAG compared to other methods?

Author response:
**Comment:** What is the Runtime of ScoreAG?  
**Response:** We added a runtime comparison to other baselines and the EDM generative model itself. See the general comment for more information.

Reviewer comment (Theory):
Some formulas in this paper were derived incorrectly, such as equation 7 on page 4. The authors should check the methodology section to ensure correctness.

Author response:
**Comment:** Equation 7 is incorrect.  
**Response:** We corrected the typo in the new manuscript.

Reviewer comment (Reproducibility):
This paper lacks details of the experimental implementation environments, which limits the reproducibility of the proposed method.

Author response:
**Edit (20.11):**

**Comment:** Lack of details of the experimental implementation environment.  
**Response:** We added a reproducibility statement and uploaded the code of ScoreAG. See the general comment for more information.

Reviewer comment (Experiments):
As pointed out in Section 4.3, some misclassification cases are due to the low resolution of Cifar-10’s images, indicating that employing higher-resolution datasets for testing would have been more appropriate.

Author response:
**Comment:** Why not use higher-resolution datasets for the human study?  
**Response:** While selecting the dataset for the human trial, we prioritized maintaining the participant's accessibility, attention, and engagement. Moreover, datasets such as ImageNet contain classes that require specialized domain knowledge, e.g., "barn spider" and "garden spider". Although selecting a subset of ImageNet classes is an option, we ultimately decided against it to avoid introducing selection bias, which could skew the study's outcomes. This led us to select a dataset with a limited number of classes.

Reviewer comment (Experiments):
With the distinct architectures used by methods presented in Table 2, can we still view this comparison as fair?

Author response:
**Comment:** Table 2 does not include Cifar-100 and TinyImagenet. Architectures are different for various methods; is the comparison fair?  
**Response:** Our method uses the WRN-28-10, a widely used architecture for adversarial attacks and robustness. To make a fair comparison, we selected the best models using the same architecture and the best models overall, which use the WRN-70-16 architecture. Therefore, this is a biased comparison in favor of the WRN-70-16 architectures and the baselines. Note that the purification experiments require unconditional generative models as the ground-truth class labels are unknown. Therefore, our purification experiments are focused on the CIFAR-10 dataset, given the availability of unconditional pre-trained EDM models and extensive prior work [1, 2].

Reviewer comment (Experiments):
Why not use images from higher-resolution datasets for testing in Section 4.3?

Author response:
**Comment:** Why not use higher-resolution datasets for the human study?  
**Response:** While selecting the dataset for the human trial, we prioritized maintaining the participant's accessibility, attention, and engagement. Moreover, datasets such as ImageNet contain classes that require specialized domain knowledge, e.g., "barn spider" and "garden spider". Although selecting a subset of ImageNet classes is an option, we ultimately decided against it to avoid introducing selection bias, which could skew the study's outcomes. This led us to select a dataset with a limited number of classes.