Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a new finding that is contrary to previous works. That is, it is not necessary to separate batch statistics when co-training on clean and adversarial data. It shows that using the classification token of a Vision Transformer (ViT) as an adapter is sufficient to achieve the performance of the dual normalization layers proposed by previous works. This paper also introduces “adversarial model soups” that allow a smooth transition between the clean and robust modes. These observations provide new insights into the regularization effect of adversarial training.
strength_and_weaknesses: Strengths:

1. AdvProp (Xie et al. 2019a) improves image recognition by adversarial training with separate batch statistics of clean and adversarial data, first showing that adversarial examples can benefit model accuracy. This paper views AdvProp from a novel adapter perspective. It demonstrates that separating batch statistics is not necessary and that using domain-specific trainable parameters can achieve similar performance. The motivation is clear, and the new finding is interesting.

2. Compared to AdvProp that only consider clean performance, this paper takes both clean and robust performance into consideration. It introduces “adversarial model soups” that can trade off clean and robust accuracy via the adapter. This makes the models more flexible and applicable to more practical scenarios.

3. In addition to adversarial examples, this paper also considers the robustness against distribution shifts, where multiple ImageNet variant datasets are used for benchmarking, such as ImageNet-C and ImageNet-A. It is always good to evaluate on broad datasets.

4. Figure 2 is impressive. It first demonstrates the importance of the “dual” technique, then shows that AdvProp is not the only effective dual technique. Multiple normalization methods are evaluated. The results indicate that domain-specific trainable parameters are key, not batch statistics.


Weaknesses / Questions / Suggestions:

1. If my understanding is correct, the beta is a hyper-parameter of adversarial model soups that is manually adjusted. If that is true, it is less practical and less novel. At inference time, the given inputs would be any type (clean, adversarial, distribution shifts, etc). It is impossible to adjust the clean/adversarial modes or the beta value for each input sample. An automatic mechanism is needed. The model is expected to self-decide a proper mode or beta value for each input automatically with an end-to-end pipeline. In this case, the contribution of the “adversarial model soups” would be more significant. The author may refer to [r1], which combines an automatic selection module with separate batch normalization layers to achieve the idea. The author may figure out a way to automate the adversarial model soups. The current method is just a linear combination of domain-specific parameters, so the novelty is limited.

2. Several experimental results are strange. In Table 1, Co-training (Eq. 1) gets 0% robust accuracy, which is unexpected. With alpha=0.5, Co-training should still get robustness to a certain extent (see Xie & Yuille, 2019). Similarly, in Table 2, it is also unexpected that Fast-AT gets 0% robust accuracy. According to (Wong et al. 2020), Fast-AT can achieve decent robustness with proper step size and random initialization. The authors are asked to explain these results, otherwise, the results would be less convincing.
 
3. Several figures are not clear enough. Figure 1 is suggested to denote phi_clean, phi_adv, etc. (corresponds to Equations) on the figure to make it more understandable. Figure 4 (similarly, Figure 9 in the appendix) should provide the corresponding beta values like Figure 3 provides alpha values. Figure 5 should self-contain the x-axis tile (beta).

4. The tables of experimental results can be more complete. Specifically, the section “Robustness to stronger attack” should have a table showing the numbers, which would be much more clear. Or expand columns in Table 1 for stronger attacks. Furthermore, Table 9 (in the appendix) should compare the numbers of adversarial model soups as well.

[r1] S.-Y. Lo and V. M. Patel, “Defending Against Multiple and Unforeseen Adversarial Videos,” in IEEE Transactions on Image Processing, 2021.
clarity,_quality,_novelty_and_reproducibility: This paper has fair clarity, quality, novelty and reproducibility. The main concerns about clarity and novelty are discussed in Weaknesses.
summary_of_the_review: Based on the above comments, I suggest marginally below the acceptance threshold. The new finding is interesting, and the experiments are extensive. However, the novelty of adversarial model soups is not significant. Furthermore, there are some unexpected experimental results that need further explanation. My final rating will depend on the authors’ response.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper shows that separate batch statistics for co-training on clean and adversarial inputs are not necessary. An extremely lightweight adapter using the class token is enough to achieve comparable performance compared with the dual norm setting. It also enables model soup instead of model ensembling for faster model inference.
strength_and_weaknesses: ## Strength

1. The method is simple yet effective, it significantly reduces the number of domain-specific parameters compared with AdvProp.
1. The interpolation/extrapolation experiments in Figure 3 are interesting and show the benefit of using the class token as the adapter.

## Weakness

1. Even in the original AdvProp, the number of domain-specific parameters is marginally compared with the number of the whole model, which makes the benefit of further reducing the number of domain-specific parameters less useful.
1. The interpolation, extrapolation, and model soup could also be applied to the AdvProp and the results would be interesting.
1. Using the class token as the adapter so it doesn't work with CNN and transformer without class token, which limits the application of this method.
1. Comparison with AdvProp is not thorough. AdvProp could be regarded as using dual norm as the adapters. The influence of different adapters should be studied.
clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to follow. The idea is interesting but doesn't show advances over the existing AdvProp method.
summary_of_the_review: The idea is simple yet effective. However, it should be compared with the AdvProp in detail. It could only be applied on Transformers with class token, further limiting its application.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper works on the robustness of the neural networks and aims to boost the accuracy on clean ImageNet and different variants with the help of adversarial samples. Inspired by the AdvProp and model soups, the authors propose adversarial model soups, which are trained with adapters through linear combinations of the clean and adversarial tokens. Experiments show model soups can easily strike a balance between clean accuracy and dataset-shifting robustness.

strength_and_weaknesses: Strength
1. Model soups are drawing much attention due to their excellent performance. However, they require a great number of computing resources. The adversarial model soups alleviate the need for storage capacity and computing power and make them practical for mobile devices and independent researchers.
2. Findings in Figure 2 are interesting, which shows that domain-specific normalization layers are enough to boost the robustness.
3. The method is simple, and few hyper-parameters are required.
 
Weakness
1. AdvProp boosts both accuracy and robustness and doesn’t need more parameters. I think you had better add the results of AdvProp in Table 1.
2. What’s the $\alpha$ used in the baseline **Adversarial Training** in Table 1? Could you do an ablation study here providing the results of different $\alpha$?

clarity,_quality,_novelty_and_reproducibility: 1. The baselines **Adversarial Training** and **Co-training** in Table 1 are confusing. If **Co-training** uses Eq. 1 as shown in the Table, what’s the difference between them?
2. In Section 5.1 (*Weighting the co-training loss*), you refer to Eq. 1 as the co-training loss. Later, in the caption of Figure 3, you refer the Eq. 2 as the co-training loss. You had better clarify the notions in your paper. 
3. In Section 5.2(**Adapters enable adversarial model soups.**), you try to verify that forming model soups between independent nominal and robust models fails. However, model soups work in pretraining-to-finetuning problem settings. Could you fine-tune some adversarial models (with different $\alpha$s in Eq. 1) from the same pre-trained model, average the parameters just like standard model soups and then test the accuracy and robustness on such a model soup?   
4. In Fig. 5, the model performs differently on IN-R and Conflict Stimuli. Could you explain it? 

summary_of_the_review: In a word, the paper does a good job for its interesting insights and innovative method. However, it suffers from several problems, like a lack of baselines and clarity.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes to treat clean and adversarial samples as different domains and co-train them by separate classification tokens of a VIT, which produces adversarial model soup to trade off between clean and robust accuracy by simple interpolation. The authors present a good alternative for advprop with fewer parameters and better robustness.
strength_and_weaknesses: Strength
+ It is very novel and inspiring to separate tokens of VIT for domain-specific training in the adversarial setting, and implementing the well-known idea (clean and adversarial samples come from different domains) in the new tool (VIT) is interesting and inspiring.
+ The authors also insightfully demonstrate that training with adapters enables model soup, which allows a simple trade-off between robustness and accuracy in adversarial model soup by interpolation between the clean and adversarial token. It frees the burden of retraining the whole model when the balance is to change.
+ Extensive experiments show that the proposed method could achieve SOTA clean and robust performance in different modes. Besides, the results of the distribution shift are also encouraging.
+ The authors also thoroughly analyze the influence of adapters, attack steps, the weighting hyperparameter, extrapolation, etc. The results in Table 9 convincingly necessitate the weight-sharing design.
+ The paper is well-motivated, well-organized, and easy to follow.

Weakness
+ The authors should better illustrate how the adversarial examples are included in clean mode training.
+ It would be better to report the adversarial model soup performance on other datasets, e.g., CIFAR-10, CIFAR-100, along with other models in the RobustBench.
+ The authors should use \citep instead of \cite in some places and maintain a larger font size for tables.
+ Section 3.2 does not seem necessary to be so long. In contrast, the results of the adversarial model soup, Figure 9, are more important to me.
+ Would additional data further increase the performance as in [1,2]?


[1] Improving robustness using generated data, NeurIPS 2021.

[2] Data augmentation can improve robustness, NeurIPS 2021.
clarity,_quality,_novelty_and_reproducibility: See Strength And Weaknesses.
summary_of_the_review: The paper is very novel and clear with convincing results. The experiments are thorough and sound.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper continues the exploration of using adversarial examples to improve recognition models. Unlike previous methods, which heavily rely on separate batch statistics to cope with the joint training of adversarial examples and clean images, this paper demonstrates that the classification token in ViT can be used for this purpose and is more parameter-efficient. Additionally, this approach allows for easy integration with "model soups" to achieve the desired trade-off between clean and robust accuracy. 

Overall, the reviewers enjoyed reading this paper, and found this method simple and effective. The reviewers only had some minor concerns about empirical evaluations. These concerns are well addressed in the rebuttal. As a result, all reviewers unanimously agree to accept this submission. 

justification_for_why_not_higher_score: Firstly, as stated in the next question, the quality of this paper is definitely above an average ICLR poster.

The main reason to select it as a "spotlight" rather than an "oral" is that its technical novelty part is not particularly strong, and it is unclear (though could still be promising) to generalize this method to models without class tokens.
justification_for_why_not_lower_score: The quality of this paper is definitely above an average ICLR poster: the proposed method is interesting, and the empirical results are strong; in addition, I agree with the reviewers that this paper could inspire more future works, as it provides new insights into the regularization effect of adversarial training.
Decision: Accept: notable-top-25%