Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a method to compute pairwise similarities between two architectures. This is done by evaluating the change in prediction of first model caused by the adversarial prediction w.r.t. second model and viceversa, with the intuition being that if two models are similar then their adversarial perturbations would have similar effect. Paper then uses the proposed similarity measure to analyze existing model architecture choices and effect on ensembling performance.
strength_and_weaknesses: Strengths
- Well written and easy to understand paper
- A detailed analysis of the similarity measure w.r.t. a fairly large (69) menu of models with different architectural components.
- Interesting insights into effect of component choices on similarity and ensembling performance.

Weaknesses
- Proposed method is computationally expensive and the approximation proposed in 5 is rather ad-hoc and not well evaluated. This limits the applicability of the method.

Minor nitpick
- What do authors mean by "destructive success of DNNs"? The phrase seems to refer to negative effects of DNN success. Perhaps, rephrase is that is not intended, otherwise clarify. 

clarity,_quality,_novelty_and_reproducibility: - Clarity: Paper is well written and easy to understand
- Quality: Paper includes a fairly detailed study of the proposed similarity metric, with a few interesting insights (e.g. choice of stem has a large impact on similarity).
- Novelty: Reasonable. Paper repurposes adversarial perturbations to define a similarity and presents novel conclusions on choice of teacher.
- Reproducibility: The proposed metric is fairly simple. If the images used to compute similarity are shared, then result should be reproducible.

summary_of_the_review: Overall, I feel that the technical contribution is fairly limited. However, the paper presents an interesting an fairly detailed study that provides interesting insights and would be beneficial to a large audience. I vote for accept.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a quantitative similarity score between different neural architectures based on the adversarial attack transferability. This smiliarity helps to understand the component-level architecture design, and leads to better understanding of the relationship between model similarity of model ensemble performance and model distillation performance. Several interesting observations are obtained based on the proposed similarity function.
strength_and_weaknesses: Strength:
1. This paper proposes a quantitative similarity score between different neural architectures based on the adversarial attack transferability. 
2. The proposed smiliarity helps to understand the component-level architecture design, and leads to better understanding of the relationship between model similarity of model ensemble performance and model distillation performance. 
3. Several interesting observations are obtained based on the proposed similarity function.
4. Extensive experiments and analysis are conducted, to lead to better understanding.

Weaknesses:
1. Could the authors provide more motivation and insights for why  adversarial attack transferability can serve as the similarity function?
2. In Figure 6 (c), most teacher models are clustered together with quite similar slimilarity. More teachers with large variety of similarity should be provided to better justify the conclusion that using a more dissimilar teacher leads to better distillation performance.
clarity,_quality,_novelty_and_reproducibility: This paper is well written and easy to understand. The proposed method is of good novelty.
summary_of_the_review: This paper is interesting and leads to several observations that may be beneficial to architecture design, model ensemble and knowledge distillation. More analysis and insights for the proposed similarity function should be provided. Several experiments related to Figure 6 are encouraged to be conducted.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper present a DNN model similarity measure using input gradient transferability. The basic hypothesis is that if two neural networks are similar, adversarial attack transferability will be high. Additionally two topics are investigated: (1) Which network component contributes to the model diversity? (2) impact of model diversity in practice.
strength_and_weaknesses: Strength

+ Similarity of DNN architecture is an important research topic.


Weaknesses

- What does "If A and B are similar and assume an optimal adversary, then accA→B will be almost zero" mean?

- There are numerous syntax errors. e.g., please revise "We expect our analysis tool helps a high-level understanding of differences between various neural architectures as
well as practical guidance when using multiple architectures."

- What is "destructive success"?

- Please elaborate "distinct properties".

- How is similarity score defined?

- What is input gradient?
clarity,_quality,_novelty_and_reproducibility: Clarity

This paper is often confusing. The writing and organization need to be significantly improved, e.g. "then accA→B will not be dropped significantly"

Quality

The paper is technically weak, and it is hard to tell what contributions it makes.

Novelty

The novelty of this paper is not clear. For example, "we found that more diversity leads to better ensemble performance" has been recognized in ensemble learning for a long time.

Reproducibility

Good.
summary_of_the_review: This paper is lack of technical contribution and novelty.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposed a new similarity function to quantify the similarity of the DNNs in image classification tasks based on transferability of adversarial examples. For a pair of similar DNNs, authors investigate which types of DNN components contribute most to the similarity and their implications to practical scenarios. Using a collection of pre-trained image classifiers on ImageNet, authors conclude that the base architecture component (i.e., convolution-based CNN vs. self-attention-based Transformer) plays a most important role in measuring similarity and cluster DNNs according to the 13 categories of DNN components.       
strength_and_weaknesses: The problem of measuring DNN similarity sees a broad range of applications in model diversity, such as choosing diverse models for ensemble, and similar models for knowledge distillation. The categorization of DNN into 13 different components provides a useful guideline for future studies. 

A major weakness to the reviewer lies in the rationale of defining the similarity score, i.e., transferability of adversarial examples generated by PGD. The underlying assumption states if the adversarial examples are transferrable between a pair of DNNs, then the similarity is high. It is more reasonable if the similarity is measured between a pair of CNNs but not so between a pair of Transformers nor between a pair of CNN and Transformer. This is because the adversarial examples generated by PGD on CNN may not be transferrable to Transformer, evident by several recent publications on transferability of adversarial examples (e.g., Mahmood et al, ICCV-2021, Fu et al, ICLR-2022). In fact, the algorithms to generate adversarial examples for Transformer can never be the same as those for CNNs. This is because Transformer operates on image patches to extract global features via self-attention whereas CNN operates on pixels to extract local features via convolution feature maps. As such, the reviewer believes the similarity score is limited to CNN on image classification tasks.               

clarity,_quality,_novelty_and_reproducibility: The presentation is clear, logically coherent and related works are comprehensively covered. It starts with motivation and problem statement, followed by literature review and the proposed similarity score. Then it continues with experimental evaluation and outline of practical scenarios. The perspective of using transferability of input gradients via adversarial examples are novel and the decomposition of DNN into 13 components provide useful guideline for future studies. The reproducibility appears to be good as well. The paper would be much strengthened if authors propose the similarity score for CNN and Transformer separately based on different mechanisms of generating adversarial examples.    
summary_of_the_review: This paper tackles an important problem of measuring DNN similarity in terms of image classification task. The similarity score is well motivated by the input transferability and practical application in model diversity, either increase diversity in generating ensembles or decrease diversity in selecting teacher and student in knowledge distillation, are also promising. However, the major technical concern is the similarity score may be more suitable for measure CNN similarities. For Transformer similarities, a different mechanism to generate adversarial are required.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes using the attack transferability of adversarial examples to measure neural architecture similarity. The authors analyze the design choices leading to better model diversity measured by the proposed metric. The paper also demonstrates the application of the proposed similarity measure in ensemble learning and knowledge distillation.
strength_and_weaknesses: __Strength__

__[S1] Novel model similarity measure.__ Using attack transferability of adversarial examples to measure model similarity is relatively novel. As a data-oriented approach, the similarity measure can be easily applied to different model architectures.

__[S2] The analysis reveals some interesting findings.__ I appreciate the comprehensive analysis of the model diversity. The analysis identifies some factors that lead to more diverse models measured by attack transferability. The findings may be of interest to some data partitioners in the community.

__[S3]__ Overall, the paper is easy to follow.

__Weakness__

__[W1] Some findings are mostly expected or have been studied before.__
- Intuitively, an adversarial example generated for a specific model is more effective on the same base architecture. For example, using the adversarial examples generated from ResNet to attack VGG is more difficult than attacking other ResNet models. So, it is somewhat expected that the proposed method considers the differences in model architecture as the most diverse change.
- The observation that using more diverse models in ensemble learning leads to better performance has been explored in many previous works.
- As suggested by the paper, the observation that using a similar model architecture for student and teacher models results in better performance is studied in previous KD works.

__[W2] Limitation of using attack transferability as similarity measures.__ As using the adversarial attack transferability naturally gives a higher similarity score for the same model architecture, is it possible that the proposed similarity measures give an incorrect similarity score in terms of the predictions:
- Given two models with the same base architecture, the models make very different predictions; however, the adversarial examples transfer well across the two models.
- Given two models with different base architectures, the models make very similar predictions; however, the adversarial examples do not transfer well across the two models.

__[W3] Does not discuss and compare with other diversity measures.__ Although the proposed similarity measures can improve ensemble learning, the paper does not compare the approach with previously suggested model diversity metrics in ensemble learning literature, e.g. [1]. It is difficult to judge whether the proposed method is a better measurement in selecting the models in ensemble learning.

__Other comments__

__[O1]__ If the set of candidate models contains defended models that are trained to be robust to adversarial samples, does the proposed similarity measure also work?

__[O2]__ The paper uses PGD as the adversary. Is the method sensitive to the choice of attacks?

__Ref__

[1] Kuncheva, Ludmila & Whitaker, Chris. (2003). Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning. 51. 181-207. 10.1023/A:1022859003006.

clarity,_quality,_novelty_and_reproducibility: The paper is well-written and easy to follow. The analysis is sound and reasonable. I did not identify obvious obstacles in reproducing the results, however, providing the code can improve the reproducibility of this work.
summary_of_the_review: This paper is well-written and proposes a simple and practical similarity measure for neural architecture. However, this paper lacks the discussion and comparison with the model diversity metric proposed in previous works. The paper should show and explain why the attack transferability metric is a better measurement of model similarity.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a neural network similarity proxy based on adversarial robustness transferability. Under this framework, if two networks have similar accuracy against each other's adversarial inputs, they are considered similar. The proposed similarity proxy is studied for ensemble classification and knowledge distillation. The contribution of different neural network components to the similarity proxy is also discussed. 

Pros:
- The paper is written clearly and very nicely
- The similarity of an exhaustive list of 69 networks is studied

Cons:
- Using adversarial attack transferability as a similarity proxy is not theoretically grounded in this submission and it is not analyzed in controlled settings (e.g. toy datasets, simple networks, or known similarities)
- The effect of adversarial training itself on the proposed proxy is not studied 
- The introduced method has a high computational cost (minor issue though)
- Reviewers found some of the findings less surprising 
- Reviewers suggest that comparing to related baselines could strengthen the message of this paper
justification_for_why_not_higher_score: While the paper presents an interesting direction for studying the similarity of neural networks through adversarial input transferability, the reviewers question the theoretical justification of the proposed idea. Additionally, comparing the method against prior baselines and related methods, studying the method under controlled settings (e.g., toy examples), or analyzing the effect of adversarial training on the proposed proxy would help readers better understand why adversarial transferability is a good similarity proxy. Given these, the paper does not appear to be ready for publication at ICLR in its current form.
Decision: Reject