Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper uses ensembles--and more precisely multi-input-multi-output (MIMO) neural networks for computational efficiency---for adversarial training (AT), resulting in a method called MAT, short for MIMO AT.
Moreover, the authors investigate how the sub-networks should be trained to increase the overall robustness of the total MIMO model.
In particular, the proposed MAT generates the adversarial samples using the gradient with respect to the objective of the ensemble.

The authors demonstrate on MNIST and CIFAR-10 that the proposed method achieves generalization and robustness comparable to existing state-of-the-art methods.

main_review: # Strengths
- This paper focuses on the important problem of robustness to adversarial attacks using a straightforward idea that adds computationally negligible cost at inference time, which could be of interest to the community. Using MIMO architectures for AT is interesting.
- It presents extensive experiments on MNIST, including varying ball sizes (fig. 2), and a comparison to the adaptive AutoAttack.

----

# Weaknesses
Given that the paper combines existing work (MIMO) and AT, the novelty in my opinion is limited. Moreover, the contributions are empirical. This raises the need for a more thorough empirical analysis to demonstrate the efficacy of the proposed approach.

- (Major) Weak empirical evaluation. 
   - While the authors compare with existing methods on CIFAR-10, comparison on other real-world datasets is missing, given that the paper is empirical.
   - While the authors perform a thorough analysis on MNIST, to argue in favor of MAT it would be helpful to have such analysis on CIFAR-10 as MNIST can be in some cases misleading. For example, the paper lacks results on CIFAR-10 with: (1) varying radius, 
   - The paper lacks experiments with a varying number of attack steps, in particular, showing its generalization and robustness for different a varying number of steps.
  - Solely two architectures are used, thus it is unclear if MAT performs well using different models.

- (Major) Unclear if the MAT method is outperforming existing ones.
   - Focusing on CIFAR-10, the MSD method [20] seems to provide better performances across the different attacks, see Tab.5.  The authors acknowledge this and argue that MAT may need further hyperparameter tuning, which raises a concern about how the hyperparameters are selected, as ideally the selection of the presented results should be fair/fixed toward all methods.
   - If I am correct, from Tab. 9 in the App.  MAT without $\ell_\infty$ is less robust to AutoAttack relative to MSD.
- (Minor) While the flow is easy to follow, the writing is in some cases informal and thus ambiguous, e.g., in the abstract 'attacks do not transfer easily', and careful proofreading and polishing are needed.
- (Minor) In the introduction the authors argue with certainty that ensemble methods fail due to two main reasons, there should be a citation, or alternatively, elaborate that this is argued in this paper and point to the relevant section.  See also the relevant question 4 below. 

----
# Minor comments and Recommendations
- It would be helpful to include the total training times per epoch for the different methods for fixed setup.
- Include detailed captions, e.g. dataset for each table, etc. Similarly for the tables, rather 'main result on...'  be more precise, e.g. rows/columns are methods used for training or testing, etc. 
- Incomplete sentence in App. C.1
- Sec 4.3: Our methods also increasing -> increase

----
# Questions
1. Resnet-50 for CIFAR-10 is less-common, did the authors try with ResNet-18? Moreover, what are the models used by MSD [20]? Ideally, the authors should include a comparison between the two methods using the same model for a fair comparison.
2. For how many epochs are MAT and the baselines trained?
3. I would recommend moving the AutoAttack results on CIFAR-10 in the main paper, as such comparison is important. Moreover, how do these AutoAtttack results compare to state-of-the-art AT methods (other than MSD)?
4. Looking solely at Tab.1 could also indicate that the two AT models are robust. What are the clean accuracies and PGD robustness of the models in Tab.1 both separately and the two as an ensemble? 
5. Did the authors compare against PGD-50-10?



summary_of_the_review: In summary, the proposed method is intuitive and simple, as well as computationally attractive. 
However, the novelty is limited and the paper is primarily empirical as it does not provide novel theoretical insights.
As such, the presented empirical evaluation is insufficient to convey the benefit of the proposed method, in particular:
(i) apart from MNIST, from real-world datasets results are present solely on CIFAR-10, moreover (ii) these results are incomplete in terms of analysis with varying hyper-parameters (attack radius and the number of attack steps), and (iii)  does not show a clear advantage over existing methods, see above for further comments.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this work, the authors propose a new training algorithm MAT that adversarially trains Multi Input Multi Output (MIMO) models. They show that ensemble models based on MIMO when trained adversarially , show ``adversarial diversity'' and therefore are less vulnerable to transfer attacks. They empirically demonstrate that such models are robust to a variety of $\ell_1, \ell_\infty$ and $\ell_2$ attacks and achieve better performance than other methods claiming robustness across different threat models. The authors also show computational benefits of their algorithm over vanilla ensemble training. 
main_review: Strengths:
1. The algorithm is intuitive and the claims are well supported by experiments and analysis.
2. The idea of cross gradients as a metric to analyse effect of transferable perturbations due to the unique architecture of MIMO models is interesting and worth further study on its own.
3. The paper is organized well and the ideas are presented in a coherent manner.

Weaknesses:
1. MAT generally seems to perform worse than MSD for all three threat models when compared head to head. The robust accuracy for $\ell_2$ attacks is significantly lower when compared to both MSD and AT. The paper claims in another section that this is due to choice of hyperparameters. Could the authors clarify this?
2. It is not clear from the text what the threat model or the attack budget is.  Are the various copies of the input image allowed individual perturbations or are the perturbations forced to be equal? 
3. The authors do not test their defense against an adaptive attack. For example, an attacker could try to find salient sub-models using loss values for each input and perturb just their inputs. This also ties into my second question regarding the clarity of the threat model.
4. As the authors claim robustness against multiple attacks, ideally robust accuracy should be reported for the worst case attack in the combined threat model for a more clear picture. 
5. The authors don't seem to have used the cross gradient terms as a penalty during training. Is there any specific reason for that?
summary_of_the_review: The authors propose an algorithm for adversarial training of MIMO models. They claim better robustness against a wide variety of threat models, and demonstrate this with experiments on CIFAR-10 and MNIST. The claims are mostly well supported, however the improvements seem marginal.The technical contribution is therefore somewhat limited. The authors also need to clearly specify the threat model that they assume for their experiments. However, their results for the cross-gradient  across the sub-models is an interesting addition and does show that their method holds promise. Reviewing all of this, I currently this paper needs some work, and therefore recommend a marginal rejection.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: [Summary]
This paper proposes an ensemble method to defend against adversarial attacks. 
To be specific, this paper combines MIMO strategies (Havasi et al.) and adversarial training (Madry et al.). 
main_review: [Strength]

1 To defend against multiple attacks, the proposal MAT requires that a) each sub-model is robust against a specific attack, and b) submodels have reduced transferability. 

2 Compared with vanilla AT+ensemble, the proposal MAT is computationally efficient. 

[Weakness]

1 In my opinion, this paper's novelty is not strong, e.g., sub-models should be diverse, sub-models are adversarially robust, and MIMO training strategy. 

2 Sections 3.1 and 3.2 is not clearly written. E.g., what does V in equation 5 mean? 

3 Compared with MSD (Maini et at.), the proposal MAT has marginal improvements against multiple attacks. 

[Questions]

1 Could authors illustrate more about Eq. 6 and Eq. 7. From the current writing, I do not understand cross gradient in detail. 

2 A.2 PSEUDO CODE, where does penalty of cross gradient apply?
summary_of_the_review: My main concern is the novelty of this paper;
Besides, this paper is not well written. 



##### Post rebuttal #### 

Many thanks for authors' feedback. 
I have read other reviewers' comments and corresponding feedback. 
I agree with other reviewers' evaluations such as “limited novelty“, "weak experimental evidence", etc. 
Thus, I keep my score unchanged.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes an ensemble based adversarial training strategy, which could improve the worst-case model robustness against multiple $\ell_p$-norm adversarial perturbations. To improve the training efficiency, the author introduces the Multiple-Input Multiple-Output (MIMO) strategy. The motivations and insights provided by this paper are interesting for me.
main_review: Pros:

(1) The motivation behind is simple and interesting.

(2) The problem of defending multiple perturbations is significant.

(3) The insights given by the author are interesting for me.

Cons:

(1) My biggest concern for this paper is the experiment. First of all, the experimental settings are ambiguous. For example, what is the perturbation budget for your experiment? What is the step size of PGD attack? Did the author clip the perturbation for the C&W attack? Why didn't the author use AutoAttack for the main experiment on CIFAR-10 and MNIST? It seems that the author missed these important experimental settings, which make the results unconvincing.

(2) The author missed some studies that also aim to improve model robustness against multiple adversarial perturbations [R1]. The author should compare and discuss with them.

(3) According to the experimental results in Table 6, MAT+MSD achieves comparable performance with MSD, and even shows weaker robustness on the "All Attacks". How did the author explain this phenomenon?

(4) Again, as shown in Section 4.4, the author claimed that ''The drawback is we trade-off 27% robust accuracy on $\ell_2$ attacks. This is due to the imbalance of choices of the hyperparameters''. This is not convincing to me, and the discussion is somewhat tricky. In other words, if the author fine-tune the hyperparameter to increase the robustness against $\ell_2$ attacks, would the model show weak robustness against other types of attacks?

(5) Moreover, I also found the ''sub 2'' model only achieves 33.1\% on $\ell_{\infty}$ PGD attacks, while other sub-networks achieve 80+\% robustness on $\ell_{\infty}$ PGD. Could the author provide more explanations?

(6) What about the defense performance on unseen perturbations? For example, training on $\ell_1$ and $\ell_2$ adversarial perturbations and evaluating the robustness on $\ell_{\infty}$ adversarial attacks.

References

[R1] Towards Defending Multiple Adversarial Perturbations via Gated Batch Normalization, arxiv, 2020.
summary_of_the_review: The proposed method is very interesting and the motivation behind is intuitive. However, I think the experiments are not enough to prove its effectiveness, and some settings are even problematic.
review_title: null

Meta-review: This paper proposes a new ensemble training method for improving adversarial robustness to multiple attacks (e.g., $\ell_2$, $\ell_1$ and $\ell_\infty$). Specifically, authors adopt the recent Multi-Input Multi-Output (MIMO) ensemble architecture for computational efficiency. Then, the authors construct the adversarial examples using the outputs of multiple attacks simultaneously. With these examples, standard adversarial training is conducted on MIMO ensemble.

All reviewers are on the negative side. AC agrees with reviewers’ concerns on limited novelty and insufficient empirical evaluation. AC also thinks that the improvement is not that significant compared to the existing method, especially concerning the real-world dataset. Overall, AC recommends rejection.
Decision: Reject