Paper: The Effectiveness of Random Forgetting for Robust Generalization (ICLR-2024)

Reviewer comment (Presentation):
The description in the caption of Figure 2 is inconsistent with the content of the image. The former states that the consolidation phase is behind the forgetting phase, while the latter expresses that the consolidation phase is before the forgetting phase. In addition, please present more clearly in the figure what the generalized information is.

Author response:
> The description in the caption of Figure 2 is inconsistent with the content of the image. The former states that the consolidation phase is behind the forgetting phase, while the latter expresses that the consolidation phase is before the forgetting phase. 

We appreciate your valuable feedback on our manuscript. We acknowledge the inconsistency in the description of Figure 2, specifically regarding the positioning of the consolidation and forgetting phases. In the revised manuscript, we have ensured that the caption accurately reflects the chronological order depicted in Figure 2.

Reviewer comment (Experiments):
In Figure 1, FOMO is only compared with standard adversarial training, but not with methods that aim to reduce the generalization gap (such as AWP). This result may not appreciably represent the effectiveness of the proposed method.

Author response:
> In Figure 1, FOMO is only compared with standard adversarial training, but not with methods that aim to reduce the generalization gap (such as AWP). This result may not appreciably represent the effectiveness of the proposed method.

Figure 1 primarily serves as an illustrative aid to help readers grasp the concept of robust overfitting in adversarial training more effectively. To ensure a more comprehensive comparison, we have meticulously presented relevant details in Table 1. This table includes a dedicated section for comparing FOMO with methods like AWP and KD+SWA, which specifically aim to reduce the generalization gap.  In light of this, we have refrained from displaying the direct comparison with AWP in Figure 1 to avoid visual clutter. We value the reviewer's perspective, and in response, we will incorporate these changes in the revised version.

Reviewer comment (Experiments):
The authors use two adversarial attacks to evaluate the proposed method, they can consider more adversarial attacks (such as L2-norm CW, DDN) to conduct a more comprehensive evaluation.

Author response:
> The authors use two adversaial attacks to evaluate the proposed method, they can consider more adversarial attacks (such as L2-norm CW, DDN) to conduct a more comprehensive evaluation.

Based on reviewer's feedback, we have evaluated our proposed method under CW attack for comprehensive evaluation. Table 10  (Appendix section A8) presents the evaluation results under CW attack on CIFAR-10/100 using the PreActResNet-18 architecture. The robust accuracy is assessed under CW attacks, and checkpoints with the best robust accuracy under PGD-20 attacks on the validation set are selected for comparison. FOMO consistently outperforms both baselines across different datasets and attack scenarios, demonstrating its effectiveness in enhancing robust accuracy under CW attacks. The best and final robust accuracies for FOMO are generally close, indicating that FOMO maintains its performance during training and does not suffer from significant overfitting under these attacks. These results emphasize the promising performance of FOMO in mitigating adversarial attacks, particularly under CW attack.

| Dataset   | Norm   | Radius          | Methods | CW Attack  |                 |
|-----------|--------|-----------------|---------|------------|-----------------|
|           |        |                 |         | **Best**   | **Final**       |
|-----------|--------|-----------------|---------|------------|-----------------|
| CIFAR-10  | $\ell_2$ | $\frac{128}{255}$ | PGD-AT  | 67.18      | 64.29           |
|           |        |                 | KD+SWA  | 68.87      | 68.90           |
|           |        |                 | FOMO    | **70.52**  | **70.35**       |
| CIFAR-10  | $\ell_\infty$ | $\frac{8}{255}$ | PGD-AT  | 47.00      | 39.96           |
|           |        |                 | KD+SWA  | 49.35      | 49.44           |
|           |        |                 | FOMO    | **52.14**  | **51.95**       |
| CIFAR-100 | $\ell_2$ | $\frac{128}{255}$ | PGD-AT  | 37.16      | 33.43           |
|           |        |                 | KD+SWA  | 40.56      | 40.61           |
|           |        |                 | FOMO    | **42.73**  | **42.35**       |
| CIFAR-100 | $\ell_\infty$ | $\frac{8}{255}$ | PGD-AT  | 22.73      | 18.11           |
|           |        |                 | KD+SWA  | 25.42      | 25.35           |
|           |        |                 | FOMO    | **26.98**  | **26.61**       |

Reviewer comment (Presentation):
Figures can be clearer and more aesthetically pleasing.

Author response:
> Figures can be clearer and more aesthetically pleasing.

Thank you for your feedback on the figures. We'll work on improving their clarity and visual appeal in the final version of the paper.

Reviewer comment (Experiments):
I don't think the running time and convergence analysis are well-studied in this paper, the authors may need to provide a table showing how many epochs are needed to converge and compare the running time with the existing methods.

Author response:
We appreciate your suggestion and thus, we have conducted additional experiments and present the training time (per epoch) for several methods in our revised manuscript. To ensure a fair comparison, all methods were integrated into a universal training framework, and each test was performed on a single NVIDIA GeForce 3090 GPU. Table 9 (Appendix section A7) in the revised manuscript now includes the required information.

Notably, FOMO and our baselines were trained for the same number of epochs (i.e., 200 epochs for CIFAR-10/100). From the table, it is evident that FOMO imposes nearly no extra computational cost compared to vanilla PGD-AT, with specific values being 137.1s for FOMO and 132.6s for vanilla PGD-AT per epoch. This implies that FOMO is an efficient training method in practical terms. It is important to highlight that KD+SWA, a formidable method designed to counter robust overfitting, comes with an increased computational cost. This arises from its approach, which entails the pretraining of both a robust and a non-robust classifier, serving as the adversarial and standard teacher, respectively. Additionally, the method incorporates the process of distilling knowledge from these teachers. Moreover, KD+SWA employs stochastic weight averaging to smoothen the weights of the model, further contributing to its computational demands.We believe that this addition enhances the practical insights into the efficiency of FOMO and its comparison with existing methods. We would be happy to address any remaining concern or suggestion.

### Table: Training Time per Epoch on CIFAR-10 under ε∞ = 8/255 Perturbation (PreActResNet-18)

| Methods  | Training Time per Epoch (s) |
| ------------- | ------------------------|
| PGD-AT   | 132.6                    |
| WA           | 133.1                    |
| KD+SWA | 132.6 (pretraining of AT) +16.5 +141.7 |
| AWP         | 143.8                   |
| FOMO      | 137.1                   |

Reviewer comment (Presentation):
Minor: Please refrain from only using color to distinguish curves and bars as in Figures 3, 4, 5, and 6, as it is not friendly to readers with color blindness.

Author response:
We value your feedback. In the final version, we'll modify these figures to ensure they're accessible to readers with color vision deficiencies. Thank you for highlighting this important consideration.

Reviewer comment (Novelty):
Minor: Missing reference on robust generalization: Zhang, et al. "The limitations of adversarial training and the blind-spot attack." ICLR 2019.

Author response:
Thank you for your suggestions. We have added this in the related work under robust generalization.