Paper: Constructing Semantics-Aware Adversarial Examples with Probabilistic Perspective (ICLR-2024)

Reviewer comment (Evaluation):
Although the generated adversarial examples are indeed elusive to human eyes, it is unknown whether these examples are harder for the defense methods to detect.

Author response:
Through reject sampling and sample refinement, the samples presented to human annotators are already capable of deceiving the victim classifier. Due to the length constraints of our submission, we detailed reject sampling and sample refinement in Appendix D, which might have led to some misunderstanding. We apologize for any confusion this may have caused.

Reviewer comment (Experiments):
It would be better if the authors can also validate the efficacy of the proposed method on larger network such as ResNet-50, and more difficult task such as CIFAR-100 and ImageNet.

Author response:
We have observed a notable decline in classification performance in multi-class classifiers with a large number of classes after adversarial training, and the improvement in defense is not substantial. Consequently, we did not select CIFAR-100 as our target, following the precedent set by works [1] and [2]. To showcase the capability of our method with larger images, we are currently experimenting with ResNet-50 on a restricted version of ImageNet, aligning with the methodology in [1]. Updates on the progress of this additional experiment will be provided as soon as it is complete.  
The results of the ImageNet experiment have been compiled. Kindly refer to Appendix N in the present version of the PDF, located at the document's end. As I am in the process of revising the document, I will inform you should there be any changes to this reference.

Please review the ImageNet results at your earliest convenience and do not hesitate to reach out with any questions you might have. Thank you!

Reviewer comment (Reproducibility):
What is the perturbation radius used in Figure 1?

Author response:
In Figure 1, the perturbation radius ($\epsilon$) for PGD with $L_{\infty}$ norm are set to 0.4, 0.15, and 0.1 for MNIST, SVHN, and CIFAR10, respectively. For PGD with the $L_2$ norm, the corresponding $\epsilon$ values are 5, 4, and 3 for the same datasets.

Reviewer comment (Novelty):
Although the author models the generation process of adversarial examples through the energy model, there is no essential difference between the previous adversarial example generation processes (they all use gradients to find optimization targets，the objective function used in this paper is still the C&W attack.).

Author response:
We respectfully disagree with this point. As detailed in formula (1), the objective function comprises two components: $\mathcal{D}$ and $f$. While it's true that we selected $f_{CW}$ as a common choice for $f$, the unique contribution of our probabilistic approach lies in the data-driven aspect of $\mathcal{D}$. This means that $\mathcal{D}$ is not just a geometric distance anymore. The distance distribution $p_{\text{dis}}$ can be modeled by any probabilistic generative model. In our case, we found that the Energy-Based Model (EBM) provides an elegant mathematical formulation, allowing us to use minus energy to represent distance. We believe we have already highlighted this point in Section 4 of our original submission.

Moreover, we believe that the version you mentioned is identified as the 'probabilistic CW attack' (prob CW), wherein $\mathcal{D}$ is the L2 norm and $f$ aligns with $f_{CW}$, as depicted in Figure 3 (c). The primary distinction between prob CW and the conventional CW attack is the implementation of Langevin dynamics for the optimization process.

Reviewer comment (Experiments):
Generating adversarial samples through the energy model requires multiple data transformations. The author discussed changes in semantic distribution and geometric distribution. Can defenders train a classifier to filter adversarial samples generated based on the energy model?

Author response:
Certainly, if defenders know about the set of transformations $\mathcal{T}$ beforehand, they might integrate these into their training as data augmentation. However, our study is focused on victim classifiers that have undergone standard adversarial training, a prevalent method consistent with the approach in [1]. We haven't tested attacks on classifiers trained with such transformations, operating under the assumption that defenders are not privy to the attacker’s specific transformations in advance.

[1] Song, Yang, et al. "Constructing unrestricted adversarial examples with generative models." Advances in Neural Information Processing Systems 31 (2018).

Reviewer comment (Experiments):
The experiments are limited to just two adversarially trained networks. Furthermore, the adversarially trained models do not appear to have been trained under the semantic attack threat model and hence unlikely to be robust to such attacks.

Author response:
In the main text, we discuss three adversarially trained networks: two are presented in Table 1 for MNIST and SVHN, and another is detailed in Section 6.2 for CIFAR10. Furthermore, Appendix F features 12 additional networks, demonstrating our model's transferability. We limited the number of structures in the main text for two reasons: firstly, to maintain alignment with the settings in [1], and secondly, to efficiently illustrate our method's advantages without unnecessarily utilizing human annotators.

The objective of our experiment is to demonstrate the effectiveness of our attack method against adversarially trained models. While some studies in this field have incorporated experiments where generated adversarial examples are used in training robust classifiers, we have not pursued this approach. We believe our current experimental setup sufficiently highlights the strengths of our attack method, in line with [1] and [3], and we aim to avoid unnecessary use of human annotators.

Reviewer comment (Evaluation):
The attack also relies on adversarially trained models learning semantically meaningful features. However, the images shown in the figures appear to be distorted which could be a consequence of the models overfitting on a subset of semantic features.

Author response:
This is not correct. I encourage a careful review of the draft, particularly Figure 2 (a) and (b). These figures demonstrate that when the victim classifier is adversarially trained, $p_{vic}$ possesses generative capabilities, meaning it tends to produce images resembling those from the target class, which in turn makes an adversarial attack more challenging. Our proposed attack method is not dependent on adversarially trained models. In fact, attacking a victim classifier that is not adversarially trained would be significantly easier.

Could you please specify which distortion you're referring to and in which figure it occurs?

Reviewer comment (Reproducibility):
Could the authors clarify the training setup for the adversarially trained models?

Author response:
We use the setting introduced in [2], as introduced in Section 2.2.

Reviewer comment (Evaluation):
Since this paper proposes a semantics-aware method, the human evaluation is important to verify the validity of adversarial examples.

Author response:
Yes, human evaluation is crucial. Our experiments conform to the evaluation methods detailed in [1].

Reviewer comment (Experiments):
It lacks a comparison on the number of queries.

Author response:
Could you clarify what is the number of queries you're referring to?

Reviewer comment (Experiments):
There is a lack of an ablation experiment. In $p_{adv}$, what will happen if $p_{dis}$ uses L2 or other methods to calculate Figure 1 or Table 1?

Author response:
Refer to Figure 3 (c), showcasing the Prob CW (probabilistic CW attack). 

Figure 3 (c) reveals that the adversarial examples represent intersections between the original image and the target class, making them easily identifiable by humans. Therefore, we omitted them from Table 1 to conserve human annotation efforts. In response to your suggestion, we will incorporate adversarial examples generated by the prob CW attack in the format of Figure 1 into our revised draft and will notify you once it's complete.