Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The manuscript describes the problem of adversarial transferability, the process of adversarial attacks that can be transferred across models, which can help scale adversarial attack strategies. As a guardrail against such attacks, it is desired that models are robust and protected against such attacks. Towards solving this, the authors first derive a lower and upper bound on a transferability metric that they define. The similarity between gradient losses was found to be the salient cross-model term affecting adversarial transferability. Subsequently, this pointed to a strategy of reducing model smoothness and loss gradient similarity across models as a way to reduce adversarial transferability.
main_review: Motivated by reducing model smoothness, the authors propose a Transferability Reduced Smoothness (TRS) approach. The authors define adversarial attacks, both untargeted (goal is to mislabel) and targeted (goal is to specify an incorrect label). The definitions 1-5 and derivations are clear. The two central metrics driving the developments in the paper are the loss gradient similarity, which describes similarity between the gradient of losses for two models, and a beta-smoothness, which defines when a model is beta-smooth. 

The two loss functions proposed, for smoothness and similarities, are fairly novel and appear to work well for the benchmarks. My two big concerns are that the experiments do not have the same level of clarity and detail as the theoretical counterpart, and the real-world use case of preventing adversarial attacks from an unknown source model are not addressed. My detailed comments below:

One key weakness in the utility of such a framework is that the training is across pairs of models while in reality the adversarial transferability has to be against unknown source models. I believe the authors should also look at source model agnostic solutions to make their proposed work useful in real-world settings.

“Luckily, inspired from deep learning theory and optimization [14, 36], succinct l2 regularization on the gradient terms can reduce the magnitude of gradients and thus improve model smoothness.“ This statement is not well supported and appears to be a very crude approximation to the beta-smoothness metric (Definition 4). Equation 1 only has weak similarity between the two models F and G. 
Which of Lsim or Lsmooth is contributing more to the losses?

The real meat of the paper starts only on page 5. I think the authors should condense pages 1-4 to within three pages to get to the new developments of the paper faster.

Experimental results section is rushed and comprises only 1 page. It is concerning that in Fig. 3, the results of the proposed method for some model pairs are significantly lower than the DVERGE baseline method.

I have read the authors' rebuttals as well as the other reviews.

limitations_and_societal_impact: The authors do not discuss limitations of their approach. Is truncating the Taylor series with the first term correct? What about higher order terms, which might come into play when source models are unknown. The methods developed in general will have a positive societal impact by reducing adversarial attacks.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work first theoretically analyzes and outlines sufficient conditions for adversarial transferability between models, then proposes a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness. They also provide the lower and upper bounds of adversarial transferability under certain conditions and propose an effective Transferability Reduced Smooth (TRS) ensemble training strategy to train a robust ensemble with low transferability. Extensive experiments demonstrate that the proposed TRS outperforms all baselines significantly.
main_review: [Strengths]
	+They make the first attempt towards a theoretical understanding of adversarial transferability, and provide an approach for developing robust ML ensembles.
	+A theoretical analysis is provided, which helps us to understand the transferability of adversarial examples between different models.
	+Extensive experiments demonstrate that the proposed TRS outperforms all baselines significantly. Ablation studies are provided.

[Weaknesses]
	-“white-box and blackbox attacks” is used in line 62, but “whitebox and blackbox attacks” is used in line 68, please write in the same way. “theoretical understanding” -> “a theoretical understanding”.
	-TRS requires smaller or comparable training time (line 65). But there is no statistical comparison of training times between different methods.
	-∥∇_x ˆ  l_F ∥_2 and ∥∇_x ˆ  l_G ∥_2 and used In Eq. (1). As I understand, it means for training a robust model B, model A is needed to calculate the smooth loss. In the end, model B may be “robust” for transferable adversarial examples from model A, but vulnerable for adversarial examples generated from other models. This type of training is limited in practical application.

limitations_and_societal_impact: The theoretical analysis is provided but the practicability maybe weak.  I suggest to test the avaiablity in the real application.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes to enforce the smoothness of the base models in addition to reducing the loss gradient similarity between them to improve ensemble model robustness. This is motivated by theoretical analysis for adversarial transferability and verified by extensive experiments. 


main_review: To the best of my knowledge, the theoretical analysis and proposed method are novel. The experiments also support the central claim about improving ensemble model robustness. 

However, the central concern I have is that: Is the proposed method essentially trading model performance for the "robustness"? After all, two completely different random guess models probably cannot transfer to each other, but such models are also useless. The proposed objective optimizes the base models to have smooth orthogonal decision boundaries, so that they are less adversarial examples are transferable between them. But this probably also makes at least one of them have much worse performance with such different decision boundaries as shown in figure 1. Then how can the ensemble still maintain the high performance?  On the other hand, previous works on promoting base model diversity to improve robustness still maintain that base models make the same predictions, i.e. having similar decision boundary, but potentially different confidence score distribution, such as in Improving Adversarial Robustness via Promoting Ensemble Diversity. 

There are also several other issues
- It would be more convincing to test against stronger black-box attacks, such as 
    - ILA (Enhancing adversarial example transferability with an intermediate level attack)
    - DI2-FGSM (Improving Transferability of Adversarial Examples with Input Diversity)
    - Interaction-Reduced attack (A Unified Approach to Interpreting and Boosting Adversarial Transferability)
- Standard deviations are not reported in table 1, but the error bar is claimed to be reported in the checklist.
- In Definition 4, the text on 159 suggesting that x and y should be paired input and true labels. But the notation in the formula could be interpreted otherwise. This should be clarified, maybe by qualifying (x,y) as a pair under the inf and sup. Similarly for other places applicable. In general, it would be better to use a different symbol for the qualified y that is potentially not the same as the ground truth label, such as the ones in Theorem 1.
-  On line 326, DIVERGE -> DVERGE



limitations_and_societal_impact: The author pointed out the limitation that "TRS ensemble is slightly less robust than DIVERGE under small perturbation with weak attack FGSM". If the method indeed significantly decreased the model performance over the normal dataset, it should also be noted as a limitation and potential negative societal impact.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: 1. This paper provides theoretical analysus on the sufficient condition on the adversarial transferability between ML models, and identify the importance of both gradient orthogonality and model smoothness

2. This paper proposes a simple smoothness regularization term and merge it with gradient diversity objective to form TRS robust ensemble training algorithm

3. Thorough experiments are provided to show TRS achieves state-of-the-art ensemble robustness
main_review: This paper provides solid theortical analysis on the transfer robustness of deep learning models, and accordingly propose ensemble training methods to reduce the transferability between sub-models and achieve robust ensemble model. Overall I enjoy reading this paper and find it interesting. Here are the strength and weakness of the paper:

Strength:
1. The theortical analysis on the transferability bound is solid, and its the first work to identify the importance of model smoothness in diverse ensemble training, making the proposed TRS training method well motivated.
2. The experiment results support the theortical claims by showing the ensemble trained with TRS can achieve SOTA transferability robutness and ensemble robustness.
3. The paper is well written and easy to follow.

Weakness:
1. There lacks a discussion on the additional training cost introduced by optimizing the smoothness loss. See the limitation section for details.
2. It should be noted that model smoothness has already been introduced in previous works on improving the robustness of deep learning model. One representative work is CURE [1], which also use a form of gradient regularization to minimize local curvature of deep learning model for robusntess improvement. I think it would be important for the author to cite this paper and discuss the difference in the smoothness objective.
3.  Though I appreciate the finding of the importance of model smoothness , how exactly the smoothness regularization is contributing to the overall ensemble robustness is unclear. The author suggest the smoothness is helpful on achieving better model diversity. However, as shown in CURE, adding smoothness regularization is also an effective way to increase the robustness of individual model. So it's likely that TRS is achieving higher robustness against larger perturbation due to each sub-mode is more robust. This hypothesis could also be supported by the fact that the the curve of TRS in Fig 2 looks more like DVERGE+ADVT (lower in low perturbation and larger in strong perturbation, see Fig 5 in DVERGE paper). This may also explain why higher transferability between sub-models is shown in Fig 3 as all of them learn robust features. Redoing Fig 3 with a smaller perturbation strength may unveil the difference in sub-model robustness between TRS and DVERGE, also a result with only the smoothness loss but not the cos loss would be helpful.

[1] Moosavi-Dezfooli, Seyed-Mohsen, et al. "Robustness via curvature regularization, and vice versa." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.


In summary, I think this is a novel and solid theortical work supported by thorough experimental evaluation. Though there are concerns on whether the proposed smoothness objective is optimally designed and how the smoothness is contributing to the overall robustness exctly, I do think this can be a good starting point with high potential of inspiring future works on investigating how smoothness can help on achieving more robust ensemble models. So I would suggest accepting this submission, with the expectation that the authors can respond to my concerns in the discussion period.
limitations_and_societal_impact: One potential limitation of the proposed method is the computation cost of optimizing the min-max smoothness objective in Equ. (1). Given the large performance margin between using the min-max objective vs. using vanilla L2, it is likely that the perofrmance of TRS is determined by how well the inner maximization is optimized, yet the discussion on this part is lacking in the paper. I would suggest the author to
1. Provide detailed information on the hyperparameter (i.e. step size and number of steps for the inner maximization) of the smoothness loss and discuss its impact on the overall training cost
2. If possible provide an ablation study on how these hyperparameters affect the final performance of TRS, is there a tradeoff in the training cost vs. robustness?
review_title: null

Meta-review: This paper analyzes the reason for adversarial transferability between models, and proposes an algorithm to reduce the transferability between base models. All reviewer agree that this is an interesting topic and the paper provides theoretical insight on the transferability problem. On the other side, the experimental analyses need to be strengthened. What are the additional costs for the reduction of adversarial transferability? Overall, this is an interesting paper. I recommend accept.
Decision: Accept (Poster)