Paper: Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization (ICLR-2024)

Reviewer comment (Novelty):
Overall the technical novelty of this work is limited. In essence, AToP combines the orthogonal techniques of AT and AP.

Author response:
In the related work section, we conduct a comprehensive comparison to highlight the distinctions between our method and other existing methods. Considering the limitation of the existing pre-trained generator-based purifier model, where all parameters are fixed and cannot be further improved for known attacks. Regarding this challenge, We propose a novel defense pipeline as shown in following equations.

Pre-training:

$L_{\theta_{g}} =L_{df}(x, \theta_{g})$

$=\mathbb{E}[D(x)]-\mathbb{E}[D(g(t(x),\theta_g))]+\mathbb{E}[{\|x-g(t(x), \theta_g)\|}_{\ell_1}],$

Fine-tuning:

$L_{\theta_{g}} =L_{df}(x', \theta_{g})+\lambda \cdot L_{c l s}(x', y, \theta_{g}, \theta_{f})$

$ =\mathbb{E}[D(x')]-\mathbb{E}[D(g(t(x'),\theta_g))]+\mathbb{E}[{\|x'-g(t(x'), \theta_g)\|}_{\ell_1}]$

$ +\lambda \cdot \max_{\delta} CE(y, f(g(t(x'),\theta_g))), \quad where \ x' = x+\delta.$

Reviewer comment (Experiments):
The authors mainly evaluate ResNet and Wide-ResNet architecture. Hence it is not clear if this work generalizes to other model architectures.

Author response:
One of the advantages of AP is that once the generator model is fully trained, it will be a plug-and-play module that can be applied to other model architectures.

Reviewer comment (Experiments):
These days, transformer architectures are gaining popularity. How does the proposed AToP perform on such architectures, such as ViT?

Author response:
In the experiments comparing with the state-of-the-art methods, all classifiers used are from the model zoo in RobustBench, and we select two of the most common model architectures for the experiments. To address your question, we recheck all available classifiers. However, we do not find a vanilla transformer-based classifier. There is only one transformer-based classifier with adversarial training (XCiT [1]). Therefore, we conduct experiments on XCiT, which is equivalent to AToP+AT. Following the settings of Table 2 and Table 4, here are the results:

|  | CIFAR-10 | | CIFAR-100 |  |
|:---:|:---:|:---:|:---:|:---:|
|  | Standard Acc. | Robust Acc. | Standard Acc. | Robust Acc. |
| XCiT-S12 [1] | 88.48 | 80.27 | 73.44 | 52.73 |

[1] A Light Recipe to Train Robust Vision Transformers. 2022.

Reviewer comment (Evaluation):
While I appreciate the evaluation on BPDA, the authors should include a separate section on “obfuscated gradients”, where they follow the guidelines of BPDA and [A] to test for obfuscated gradients.

Author response:
In Section 4.4, we conduct experiments using BPDA based on the guidance of Athalye et al. In the ablation study, we use whether to apply AToP as the only variable. This means that in a set of comparative experiments, only the training parameters of the purifier model differ, while other details remain consistent, including the parts that cause obfuscated gradients. It is reasonable that we use the paradigm provided by Yang et al. to verify whether our method can improve the robustness of the pre-trained purifier model. More importantly, this part is only discussed as the ablation study, and we put the robustness comparison experiment with other SOTA methods in Section 4.2.

Reviewer comment (Experiments):
What if there are no random transformations in the proposed pipeline? Does the robustness come from the gradient vanishing from random operations?

Author response:
A1.1: If there are no random transformations, the robustness will decrease. $RT_2$ adds more randomness based on $RT_1$, and its robust accuracy has been improved. At the same time, Cohen and Wu et al. theoretically provide robustness guarantees about randomized smoothing, which is described in Section 3.1.

A1.2: Random operations contribute to robustness but not fully due to gradient vanishing. As mentioned in A1.1, there are some theories of robustness guarantees about random operations. Indeed, gradient vanishing does provide 'illusory' robustness for some attacks. Regarding this issue, we also validate our method with AutoAttack and BPDA.

Reviewer comment (Evaluation):
For adversarial training methods, auto-attack is one of the most effective attack methods to evaluate model robustness. This is because models with adversarial training usually show much better results under FGSM, PGD, and other attacks than under the auto-attack. However, as indicated by Table 7 and Table 8 in this paper, the trained model even achieves much better robustness under auto-attack than under PGD20, FGSM attacks. From this perspective, the comparisons in Tables 2, 3, 4, 5 can be unfair because models with adversarial training show worst-case robustness under auto-attack while it is not the case for the model trained by the proposed method.

Author response:
A2: We conduct AutoAttack verification under two different settings: In Table 2,3,4,5, we attack the entire (purifier+classifier) model, and in Table 7 and 8, we attack the classifier model. The result in Table 7 and 8 is reasonable, but we also realize that the current tables can easily lead to misunderstandings. Therefore, we plan to redesign the Table 7 and 8 according to your comments, replacing AutoAttack with CW and PDG-1000.

Here is the explanation of the results: Considering the robustness misestimation caused by obfuscated gradients of the purifier model, we use BPDA (Athalye et al., 2018a) and follow the setting by Yang et al. (2019), approximating the gradient of the purifier model as 1 during the backward pass, which is equivalent to attacking the classifier model. To maintain the same settings, AutoAttack only attacks the classifier. The advantage of AutoAttack lies in using EOT and adaptive mode with white-box and black-box attacks to attack more complex architectures, such as black-box and random factors in the architectures. If there are some random factors in the classifier, although the classifier model can effectively defend against FGSM and PGD, it can still be vulnerable to AutoAttack. However, in classic classifiers (such as Resnet-18), these factors do not exist, and simple white-box attacks cause more damage. Under standard cases, the entire model should be input into AutoAttack directly, and the experimental results are shown in Table 2,3,4,5. We will redesign Table 7 and 8 to avoid confusion.

| Standard Case ($RT_2, \epsilon=8/255$) |  | | |
|:---:|:---:|:---:|:---:|
| Generator models | AToP | CIFAR-10 | CIFAR-100 |
| AE (Wu et al., 2023) | $\times$ | 44.73 | 38.48 | 
| AE (Wu et al., 2023) | $\checkmark$ | **59.18** | **40.23** | 
| GAN (Ughini et al., 2022) | $\times$ | 59.57 | 38.89 | 
| GAN (Ughini et al., 2022) | $\checkmark$ | **72.85** | **43.55** |

Reviewer comment (Experiments):
Considering the issue in Point 2, to demonstrate their robustness, the models trained by the proposed methods should be fully tested under different attacks. For example, the CW attack with a large number of steps, and the PGD attack with 1000 steps.

Author response:
A3: Thanks for the comments. The following are the additional experiments on Table 7 and 8:

| | | CIFAR-10 | | ImageNette | |
|:---:|:---:|:---:|:---:|:---:|:---:|
| Transforms | AToP | CW-100 | PGD-1000 | CW-100 | PGD-1000 |
| RT1 | $\times$ | 21.09 | 0.00 | 64.26 | 0.00 |
| RT1 | $\checkmark$ | **87.89** | **39.45** | **78.12** | **47.07** |
| RT2 | $\times$ | 87.70 | 67.97 | 86.52 | 77.34 |
| RT2 | $\checkmark$ | **88.87** | **84.38** | **88.48** | **82.03** |
| RT3 | $\times$ | 75.20 | 70.31 | 67.38 | 66.21 |
| RT3 | $\checkmark$ | **76.56** | **73.44** | **78.32** | **72.85** |

Reviewer comment (Theory):
Eq. (8) seems to be a mix of the loss functions for the generator (purifier) \( g \) and the discriminator \( D \), which are supposed to be different in GAN training. Besides, since AToP is not limited to GAN-based purifier models, it is better to present a general form of the loss function.

Author response:
We also realize this might be confusing, so we have revised Eq.(8) in A1.2. Regarding the second point, we agree with your perspective. We have only used the GAN-based model as an illustrative example. We will add a general form: $L_{\theta_{g}} = L_{org}\left(x, \theta_{g}\right)+\lambda \cdot L_{c l s}\left(x, y, \theta_{g}, \theta_{f}\right)$. For instance, the original loss function is denoted as $L_{df}$ in the GAN-based model (Ughini et al., 2022), and as $L_{mae}$ in the MAE-based model (Wu et al., 2023).

Reviewer comment (Theory):
It is stated that "the discriminator \( D \) is responsible for distinguishing between real examples (*including clean or adversarial examples*) and the purified examples", but this is not reflected by Eq. (8) since only the clean sample \( x \) is involved in \( L_{df} \).

Author response:
Thanks for the comments. Eq.(8) should be divided into two stages, including pre-training and fine-tuning:

Pre-training:

$L_{\theta_{g}} =L_{df}(x, \theta_{g})$

$=\mathbb{E}[D(x)]-\mathbb{E}[D(g(t(x),\theta_g))]+\mathbb{E}[{\|x-g(t(x), \theta_g)\|}_{\ell_1}],$

Fine-tuning:

$L_{\theta_{g}} =L_{df}(x', \theta_{g})+\lambda \cdot L_{c l s}(x', y, \theta_{g}, \theta_{f})$

$ =\mathbb{E}[D(x')]-\mathbb{E}[D(g(t(x'),\theta_g))]+\mathbb{E}[{\|x'-g(t(x'), \theta_g)\|}_{\ell_1}]$

$ +\lambda \cdot \max_{\delta} CE(y, f(g(t(x'),\theta_g))), \quad where \ x' = x+\delta.$

Reviewer comment (Writing):
Line 5 of Algorithm 1 is confusing. The notation \( g_{\theta_g} \) is not explained.

Author response:
$g_{\theta_g}$ represents the computed gradient (i.e., grad\_$\theta_g$). We will change the $g_{\theta_g}$ in the Algorithm 1 to $\nabla \theta_g$ to avoid confusion.

Reviewer comment (Evaluation):
For evaluation, it is claimed that "we randomly select 512 images from the test set for robust evaluation." It is not clear whether this applies to all results or all compared methods. The comparison may be unfair if different models are evaluated on different test sets.

Author response:
Due to the high computational cost of testing models with multiple attacks, we conducted experiments based on the guidance provided by Nie et al. In fact, not all experiments tested 512 images. However, Nie et al. have demonstrated through experiments (Appendix C.2 in [1]) that the robust accuracies of most baselines do not change much on the sampled subset, compared to the whole test set.

[1] Diffusion Models for Adversarial Purification. 2022.

Reviewer comment (Experiments):
Why FGSM is used for adversarial finetuning instead of PGD or other attacks?

Author response:
The training cost of using FGSM for adversarial fine-tuning is acceptable. At the same time, as we propose a new paradigm that considers both AT and AP, we use FGSM to verify our ideas preliminarily. Here is a simple experiment regarding time, where we have measured the time required to process 10,000 images:

| Time costs |  |  |
|-------------------|:-----------:|:-----------:|
|  | FGSM   | PGD-10 |
| GAN with AToP     | ~186 sec  | ~1469 sec |

Reviewer comment (Experiments):
Can the reconstruction loss in Eq. (8) be applied to the reconstruction of adversarial images? Specifically, is it likely that adding \( \mathbb{E}[\Vert x-g(t(x+\delta),\theta_g) \Vert] \) to the loss will increase the performance?

Author response:
The impact of optimization using consistency loss of adversarial examples is limited. We can employ a variety of attacks and obtain a variety of different adversarial examples. When using consistency loss to learn information from adversarial examples, which potentially leads to overfitting and weakens the generalization ability against unseen attacks. We also discuss this point in the related work section. Additionally, as the perturbations are minimal, in many strong attacks, the consistency loss from adversarial examples is less than the consistency loss from generated examples ($\mathbb{E}[x-g(x_{adversarial})] < \mathbb{E}[x-g(x_{generated})]$). This also results in the consistency loss not playing a significant role during training.

Reviewer comment (Theory):
Why does reconstruction loss take \( \ell_1 \)-distance?

Author response:
In the pipeline based on the image completion, the repair results of the boundary of the missing area are better than the middle area, and pixels with different distances from the filled edge should be assigned different weights. A common way is to use the distance between pixels (i.e., $l_1$ distance) for weight calculation. When repairing large missing areas, the $l_1$ loss will be more effective in improving repair quality.

Reviewer comment (Experiments):
What is the expected scope of purification models that AToP can generalize to?

Author response:
We hope AToP can be applied to any pre-trained generator-based purifier model. To this end, we conduct experiments on two common generator-based purifier models, including GAN-based and AE-based. At the same time, we also attempted to use diffusion-based purifier models. However, as mentioned in the limitations section, our method cannot easily fine-tune all models yet, such as some models with high computational costs.