Similarity of Neural Architectures Based on Input Gradient Transferability
Authors: Jaehui Hwang
Dongyoon Han
Byeongho Heo
Song Park
Sanghyuk Chun
Jong-Seok Lee
Venue: Submitted to ICLR 2023 (2023)
Keywords: neural architecture similarity
model similarity
model diversity
model ensemble
knowledge distillation

Abstract: In this paper, we aim to design a quantitative similarity function between two neural architectures. Specifically, we define a model similarity using input gradient transferability. We generate adversarial samples of two networks and measure the average accuracy of the networks on adversarial samples of each other. If two networks are highly correlated, then the attack transferability will be high, resulting in high similarity. Using the similarity score, we investigate two topics: (1) Which network component contributes to the model diversity? (2) How does model diversity affect practical scenarios? We answer the first question by providing feature importance analysis and clustering analysis. The second question is validated by two different scenarios: model ensemble and knowledge distillation. Our findings show that model diversity takes a key role when interacting with different neural architectures. For example, we found that more diversity leads to better ensemble performance. We also observe that the relationship between teacher and student networks and distillation performance depends on the choice of the base architecture of the teacher and student networks. We expect our analysis tool helps a high-level understanding of differences between various neural architectures as well as practical guidance when using multiple architectures.

Under review as a conference paper at ICLR 2023
SIMILARITY OF NEURAL ARCHITECTURES BASED ON INPUT GRADIENT TRANSFERABILITY
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we aim design quantitative similarity function between two neu- ral architectures. Specifically, we define model similarity using input gradient transferability. We generate adversarial samples of two networks and measure the average accuracy of the networks on adversarial samples of each other. If two net- works are highly correlated, then the attack transferability will be high, resulting in high similarity. Using the similarity score, we investigate two topics: (1) Which network component contributes to the model diversity? (2) How does model diver- sity affect practical scenarios? We answer the first question by providing feature importance analysis and clustering analysis. The second question is validated by two different scenarios: model ensemble and knowledge distillation. We conduct large-scale analysis on 69 state-of thr- ImageNet classifiers. Our findings show that model diversity takes a key role when interacting with different neural ar- chitectures. For example, we found that more diversity leads to better ensemble performance. We also observe that the relationship between teacher and student networks and distillation performance depends on the choice of the base archi- tecture of the teacher and student networks. We expect our analysis tool helps a high-level understanding of differences between various neural architectures as well as practical guidance when using multiple architectures.
INTRODUCTION
The last couple of decades have seen the great success of deep neural networks (DNNs) in real-world applications, e.g., image classification (He et al., 201 and natural language processing (Vaswani al., 2017). The advances the DNN architecture design have taken a key role in this success by making the learning process easier (e.g., normalization methods (loffe & Szegedy, 2015; Wu & He, 2018; Ba et al., 2016) or skip connection (He al., 2016 enforeing human inductive bias in the architecture (e.g., convolutional neural networks (CNNs) (Krizhevsky al., 2012; Simonyan & Zisserman, 2015)) or increasing model capability with the self-attention mechanism (e.g., Trans- formers (Vaswani et al., 2017)). With different design principles and architectural elements, there have been proposed a number of different neural architectures; however, designing distinguishable architecture is expensive and needs heavy expertise. One of the reasons for the difficulty is that there only a little knowledge of the difference between two different neural architectures. Especially, one can quantify the similarity between two models, then we can measure which design component actually contributes significantly to diverse properties of neural networks. We also can utilize the quantity for new model design (e.g., by neural architecture search (NAS) (Zoph & Le, 2017)).
In this paper, we aim to define the similarity between two networks to quantify the difference and diversity between neural architectures. Existing studies have focused on dissecting each network component layer-by-layee (Kornblith et al., 2019; Raghu et al., 2021) or providing a high-level un- derstanding by visualization of loss surface (Dinh et al., 2017), input gradient (Springenberg al., 2015; Smilkov et al., 2017), or decision boundary (Somepalli et al., 2022). On the other hand, we aim to design an architecture agnostic and quantitative score to measure the difference between the two architectures. We especially focus on the input gradients, a widely-used framework to under- stand model behavior, e.g., how a model will change predictions by local pixel changes (Sung, 1998; Simonyan et al., 2014; Springenberg et al., 2015; Smilkov et al., 2017; Sundararajan et al., 2017; Bansal et al., 2020; Choe et al., 2022). If two models are similar, then the input gradients are similar. However, because input gradient is very noisy, directly measuring the difference between input

---

Under review as a conference paper at ICLR 2023
gradients is also very noisy. Instead, we the adversarial attack transferabilityy as the proxy mea- sure of the difference between input gradients of two networks. Consider two models A and B, and the input z. We generate adversarial samples a and z to A and B, respectively. Then we measure the accuracy of B for a (acC and vice versa. If A and B are similar and assume an optimal adversary, then acCA-*B will be almost zero, while if A and B have distinct input gradients, then a- will not be dropped significantly. We also note that adversarial attack transferability will provide a high-level understanding of the difference between model decision boundaries (Karimi & Tang, 2020). We define a model similarity based on attack transferability and analyze the existing neural architectures, e.g., which network component affects the model diversity the most?
We measure the pairwise attack transferability-boe based network similarity of 69 different neural archi- tectures trained on ImageNet (Russakovsky et al., 2015), provided by Wightman (2019). Our work is the first extensive study for model similarity on large number of state-of-the-an ImageNet models. Our first goal is to understand the effect of each network module on model diversity. We first choose 13 basic components (e.g. normalization, activation, the design choice for stem layers) that consist of neural architectures and list the components of 69 networks. For example, we represent ResNet (He et al., 2016a) fResNet [Base architecture CNN, Norm = BN, Activation = ReLU, We analyze the contribution of each network component to the model diversity by using the fea- ture importance analysis by gradient boosting regression on model similarities, and the clustering analysis on model similarities. Our analyses show that the choice of base architecture (e.g., CNN, Transformer) contributes most to the network diversity. Interestingly, our analysis shows that the design choice for the input-level layers (e.g., stem layer design choice) determines the network di- versity as much as the choice of core modules (e.g., normalization layers, activation functions).
Our study is not only limited to the understanding of component-level architecture design, but we also focus on analyzing the effect of model diversity in practical scenarios. Particularly, we mea- sure the model ensemble performances by controlling the diversity of the candidate models, e.g., ensemble "similar" or "dissimilar" models by our similarity score. Here, we observe that when we ensemble more dissimilar models, the ensemble accuracy gets better; more diversity leads to bet- ter ensemble performance. We observe that the diversity caused by different initialization, different hyper-parametee choice, and different training regimes is not as significant as the diversity caused by architecture change. We also observe that the ensemble of models from the same cluster performs worse than the ensemble of models from different clusters. Similarly, by choosing more diverse models, the number of wrong samples by all models is getting decreased. Our findings confirm that the previous study conducted in simple linear classifiers (Kuncheva & Whitaker, 2003) is also aligned with recent complex large-scale neural networks.
As our third contribution, we provide a practical guideline for the choice of teacher network for knowledge distillation (KD) (Hinton et al., 2015). We train 25 distilled ViT-Ti Ti models with di- verse teacher networks. Interestingly, our findings show that the performance of the distilled model is highly correlated to the similarity between the teacher and the student networks rather than the accuracy of the teacher network; if the student networks and the teacher network are based on the same architecture (e.g., both are based on Transformer), then a similar teacher provides better knowl- edge; if the student and teacher networks are based on different architectures (e.g., Transformer and CNN), then we observe that choosing more dissimilar teacher will lead to a better distillation per- formance. Our findings are partially aligned with previous KD studics, i.e., using a similar teacher network leads to a better KD performance (Jin et al., 2019; Mirzadeh et al., 2020). However, the existing studies only focus on the scenario when both teacher and student networks are based on the same architecture, exactly aligned with our findings. Our observation extends previous knowledge to the case of when the teacher and student networks significantly differ (e.g., Transformer and CNN).
2 NETWORK SIMILARITY BY INPUT GRADIENT TRANSFERABILITY
In this section, we propose a similarity measure between two networks based on adversarial attack transferability. Our interest lies in developing a practical toolbox to measure the architectural differ- ence between the two models quantitatively. Existing studies for a similarity between deep neural networks have focused comparing intermediate features (Kornblith al., 2019; Raghu et al., 2021), understanding loss landscapes (Dinh et al., 2017; Li et al., 2018; Park & Kim, 2022), or de- cision boundary (Somepalli et al., 2022). However, their approaches cannot measure the distance
2

---

Under review as a conference paper at ICLR 2023
between two different networks (Li et al., 2018; Korblith et al., 2019) need expensive compu- tation costs for re- re-training networks multiple times (Somepalli et al., 2022). There also has been another line of research based on prediction-based statistics, e.g.. comparing the wrong and correct predictions (Kuncheva & Whitaker, 2003; Geirhos et al., 2018; 2020; Scimeca et al., 2022). Al- though this approach can be used for measuring a model diversity (Kuncheva & Whitaker, 2003), they only can capture the prediction itself, not the change of prediction. However, just focusing on prediction values could be misleading. Because recent deep models are highly complex, the change of predictions (or sensitivity) is more practical than focusing on the prediction itself. For example, Geirhos et al. (2020) showed that recent complex deep neural networks show highly similar predic- tions. Also, deep model easily changes its prediction with neglected small noise (e.g., adversarial attack), and the perturbation varies by different models, as shown our experiments.
In this paper, we focus on how two models change their predictions by input changes, e.g., the in- put gradient. Input gradient is popular visualization tool for understanding DNNs by highlighting where a model attends (Sung. 1998; Simonyan et al., 2014). An input gradient-based -bsed method has several advantages; it is computationally efficient, and no additional training is required; it can pro- vide visual understanding of the given input. Usually, an input gradient- based quantity measures how the input gradient of a model matches the actual foreground (Choe et al., 2022), i.e., it usually measures how model explains the input well. Because we interested in comparing two differ- ent models, we focus on how input gradients are similar between them. Especially, we generate an adversarial sample using a model and test the adversarial sample to the other architecture.
Adversarial attack transferability is also a good approximation of measuring the difference between model decision boundaries (Karimi & Tang, 2020). Comparing the decision boundaries of two mod- els will provide a high-level understanding of how two models behave differently for input changes. Unfortunately, an exact decision boundary not achievable, and an expensive and inexact sampling approximation is required (Somepalli et al., 2022). In this paper, we analyze the difference between two decision boundaries by generating an adversarial sample for each decision boundary. two models have similar decision boundaries, then the adversarial samples will be transferred (i.e., will have high adversarial attack transferability): ); if two models have dissimilar decision boundaries, then the adversarial samples will not be transferred. The detailed discussion can be found in Appendix H.
Formally, we generate adversarial samples A and vp of model A and B for the given input æ. Then, measure the accuracy of model A using the adversarial sample for model B (called acc3-A)- If A and B are the same, then acC will be zero (for simplicity, here, we may assume that the adversary can fool model perfectly). On the other hand, if the input gradients of A and B differ significantly, then the performance drop will be neglectable because the adversarial sample is made almost similar to the original image (i.e., || IBl S E). Let X AB be the set of inputs where both A and B predict correctly, be the ground truth label, and I(() be the indicator function. We use the following quantity to measure the similarity between two models:
Σ (I(A(E)) + +0) +) D(B(a) + wn mn ZEXAB
1
sA, B) = log max Es, 100 X
(I)
2 2
where e, is small scalar value prevent the ioseppssesness of s(-). If A B and have an oracle adversary, then s(A, = log 100. In practice, a strong adversary (e.g., PGD (Madry et al., 2018) or Autokttack (Croce & Hein, 2020)) can easily achieve a nearly-zero accuracy if model is not trained by an adversarial attack-aware strategy, e.g., adversarial training (Madry et al., 2018; Cohen al., 2019). On the other hand, if the adversarial attacks on A are not transferable versa, then sA, B) = log We add more discussions how equation 1 can serve as a similarity score function from a functional point of view in Appendix A.
B and vice
Now, we provide an extensive analysis of a large number of -fate-of-the-art networks using our quantity. Our analysis focuses on two questions. (1) Which network component contributes to the diversity between models? (2) How the model diversity affects the practical applications, e.g., model ensemble or knowledge distillation? Note that several studies attempted to answer the second ques- tion in different ways (Kuncheva & Whitaker, 2003; Teney et al., 2022), but our work is the first work to analyze 69 state-of-the-arr ImageNet models based on model similarity. As observed by previous works in limited number of models without considering the change of architecture and large-scale training datasets, our findings show that diversity can contribute to practical applications in the state-of-fe ofr complex deep neural networks trained on large-scale datasets.
3

---

Under review as a conference paper at ICLR 2023
Table 1: Overview of model elements. We categorize each architecture with 13 different architectural com- ponents. The full feature list of each architecture is in Appendix B.
Components
Elements
Base architecture
CNN, Transformer. MLP-Mixer, Hybrid (CNN Transformer). NAS-Net
Stem layer
7x7 conv with stride 2, x3 conv with stride 2, 16x 16 conv with stride 16,
Input resolution
224 256 x 256, 240x 240, x99
Normalization layer
BN, GN, I.N, LN + GN, LN + BN, Normalization-free.
Using hierarchical structure
Yes (e.g., CNNs, Swin (Liu et al.. 2021b)). No (e.g. VIT (Dosovitskiy et al., 2021))
Activation functions
ReLU, WardSwish, SiLU, GeLU, ReLU + GeLU, ReLU + SiLU or GeLU ...
Using pooling at stem
Yes, No
Using 2D self-attention
Yes (e.g., Rello (2021), Srinivas et al. (2021), Vaswani al. (2021)), No
Using whannel-wise (CW) attention
Yes (e.g., Hu al. (2018), Cao et al. (2019). Wang al. (2020c)). No Yes, No
Using depth-wise convolution
Using group convolution Type of pooling for final feature Location of CW attentions
Yes, No
Classification (CI token, Global Average Pool (GAP) At the end of cach block, in the middle of cach block,
Settings. We use the projected gradient descent (PGD) for the adversary. We set the iteration to 50, the learning rate to 0.1, and 8/255 for the PGD attack. We also test other attack methods, such as AutoAttack and PatchFool (Fu et al., 2022) in Appendix G. We select 69 neural architectures trained on ImageNet (Russakovsky et al., 2015) from the PyTorch Image Models library (Wightman, 2019)'. Notice that to reduce the unexpected effect of a significant accuracy gap, the chosen model candidates are limited the models whose top-1 I accuracy is between 79% and 83%. We also ignore the models with unusual training techniques, such as training on extra training datasets, using a small or large model input resolution (e.g.. less than 200 or larger than 300), or knowledge distillation. When A and B take different input resolutions, then we resize the attacked image from the source network for the target network. We also sub-sample 10% ImageNet validation images (i.e., 5,000 images) to measure the similarity. This strategy makes our similarity score more computationally efficient. The full pairwise similarities of 69 models can be found in Appendix D.
3 MODEL ANALYSIS BY NETWORK SIMILARITY
We analyze how each model component affects the diversity between networks quantitatively by fea- ture importance analysis and clustering analysis (Sec. 3.1). We also show that the diversity caused by different training strategies is not as significant as caused by different neural architectures (Sec. 3.2).
3.1 WHICH ARCHITECTURAL COMPONENT CAUSES THE DIFFERENCE MOST?
Settings. We list 13 components of each architecture, e.g., normalization Szegedy, 2015)
Permutation Importance
Base architectue
(e.g., BN (loffe and LN (Ba et al., 2016)), activation functions (e.g., ReLU (Krizhevsky et al., 2012) and GeLU (Ramachan- dran et al., 2017)) the existence of depthwise convolution, or stem layer (e.g.. 7x7 conv, 3x3 conv, or 16x16 conv with stride 16 a.k.a. "patchify" stem (Liu et al., 2022)). The list of sub-modules is shown in Tab. 1 and Appendix B. We then convert each architecture as a fea- ture vector based on the listed sub- modules. For example, we convert ResNet as fResNe = [Base arch CNN, Norm BN, Activation ReLU,...].
Stem byer-
Input resolution
Normeliation leyers
Heeerchical structur-
Activation functions
byer-
sell-atertion
Charmel-oise
Depth convolaiinn
-1
Group comvoltion
+II-
Pooling features
-3+
locationl CW atwrroons
0.05
0.30
0.15
0.20 0.25
0.30
0.35
0.40
0:45
Figure 1: Importance of architectural components to network similarity. 13 components (Tab. 1) are sorted by the contribution to the network similarities. The larger feature importance means the component contributes more to the network similarity.
hhtpp:://gi thub. com/
mowe
4

---

Under review as a conference paper at ICLR 2023
Feature important analysis. The full list of components of 69 architectures is in Appendix B. Now, we measure the feature importance by fitting a gradient boosting regressor (Friedman, 2001) on the feature difference (e.g.. Se 1 base) measured by Hamming distance and the cor- responding similarity (e-g.. 2.3). The details of the regressor are in Appendix B. We use the per- mutation importance (Breiman, 2001); We measure how the trained regression model changes the predicted similarity by randomly changing each feature (e.g., normalization). The feature impor- tance of each architectural component is shown in Fig. We first observe that the choice of base architecture (e.g., CNN (Krizhevsky et al., 2012), Transformer (Vaswani et al., 2017), and MLP- Mixer (Tolstikhin et al., 2021)) contributes to the similarity most significantly. Fig. also shows that the design choice of the input layer (i.e., stem layer design choice or input resolution) affects the similarity as much as the choice of basic components such as normalization layers, activation functions, and the existence of attention layers. On the other hand, we observe that the modified cfficiency--waar convolution operations, such as depth-wise convolution (Chollet, 2017) or group convolution (Xie et al., 2017), are not effective for model diversity.
Clustering analysis. We additionally provide clustering analysis based the architecture sim- ilaritics. We construct a pairwise similarity graph A between all 69 architectures where its vertex denotes architecture, and its edge denotes the similarity between two networks. We perform the spectral clustering algorithm (Ng et al., 2001) on A where the number of clusters K is set to 10: We compute the Laplacian matrix of A, L D A where D is the diagonal matrix and its i-th com- ponent is Σn Agj. Then, we perform K-means clustering on the K K-largest eigenvectors of L, where K is the number of clusters (i.e., K = 10). More details of spectral clustering are in Appendix C.
Tab. 2 shows the clustering results on 69 networks and the top-5 keywords for each cluster based on term frequency-inverse document frequency (TF-IDF) analysis. Specifically, we treat each model feature (Tab. as a word and compute TF and IDF by treating each architecture as a document. Then we compute the average TF-IDF for each cluster and report top-5 keywords. Similar to Fig. 1, the base architecture (e.g.. CNN in Cluster 5, 6, 10 and Transformer in Cluster 2, 3) and the design choice for the stem layer (e.g., Cluster 1, 2, 4, 5, 6, 7, 8, 10) repeatedly appear at the top keywords. Especially, we can observe that the differences in base architecture significantly cause the diversity in model similarities, e.g., non-hierarchical Transformers (Cluster 1), hierarchical networks with the patchified stem (Cluster 2), hierarchical Transformers (Cluster 3), CNNs with 2D self-attention (Cluster 4, 5), ResNet-based architectures (Cluster 6), and NAS- architectures (Cluster 7).
3.2 TRAINING STRATEGY AND SIMILARITY SCORE
The architectural difference is not the only cause of the model diversity. We compare the effect by different architecture choices (e.g., ResNet and Vision Transformer) and by different training strate- gies, while fixing the model architecture, as follows: Different initializations can affect the model training by the nature of the stochasticity of the training procedure. For example, Somepalli et al. (2022) showed that the decision boundary of each architecture can vary by different initializations. We also consider different optimization papper-prramters (e.g., learning rate, weight decay). Fi- nally, we study the effect of different training regimes (e.g., augmentations, type of supervision). For example, the choice of data augmentation (e.g.. Mixup (Zhang et al., 2018), or CutMix (Yun et al., 2019)) or the optimization techniques (e.g., (Szegedy et al., 2016)) can theoretically or em- pirically affect adversarial robustness (Zhang et al., 2021; Park et al., 2022; Shafahi et al., 2019; Chun et al., 2019). Similarly, we investigate the effect of different types of supervision, such as self- supervision (e.g., MOCO (Chen et al., 2021c), MAE (He et al., 2022) and BYOL (Grill et al., 2020)) or weakly supervised learning (Yalniz et al., 2019).
Settings. Different initializations: We train 21 ResNet-50 models and 16 ViT-S from scratch individually by initializing each network with different random seeds. Different syper-parameters: We train 28 ResNet models and 9 ViT-S S by choosing different learning rates, weight decays, and learning rate schedulers. The full list of hyper-parameters is shown in Appendix E. Different training regimes: We collect 23 ResNet- 50 models and 7 ViT-S models with different training
Cusstomieed models by Wightman (3019): He loRlghiet R Rail eeet + Regliet 7; RCA- Rotteeet = FCA Net + Hall ottee + ReaMeXtt PC Roo--ooTNeet BCA Nwt Botheet flattwett; Lantfaloooottle Laabdaate a Ha loblet Bothles; SE BoThlel SENWe Botheel SE Ha LoWee: Stille ottett Bo LoMee BoTNee; NEXet-10 an efficieni RENet-FU (Broc al., 2021b); 3CA-XFNet-Lo ECA-Met NFNet-L0
Reallet- PVVS = Beellet- V2 Weooorre (1 al., 2020).
5

---

Under review as a conference paper at ICLR 2023
Table Clustering by architecture similarities. We perform a clustering analysis on 69 neural architectures. All the architectures here are denoted by the aliases defined in their respective papers. We show the top-5 keywords for each cluster based on TF-IDF. InRes, SA, and CWA denote input resolution, self-attention, and channel-wise attention, respectively. The customized model details are descrihed in the footnote
No. Top-5 Keywonds
Architecture
Stem layer: 16x 16 conv w/ s16, No Hierarchical, LN. Final GAP
ConViT 3 (d' Ascoli et al., 2021), CrosaVi B (Chen et al., 2021a),
DeiT-B (Touvron et al., 2021), Dei2-9 (Touvron et al., 2021).
1
viT-S (patch size 16: (Dusovitskly et al., 2021).
ResMIP-824 (Touvron et al, 2022a), gMr.P gMIP-8 (Liuu et al., 2021a)
Stem layer: 4x conv w/ s4,
Twins PCPVI (Chu et al., 2021), Twins SVT (Chu et al., 2021).
LN. GeLU, Transformer, No pooling at stem
CosT-Lite Small 11 (Dai et al., 2021b). NesT-T (Zhang et al. 2022b).
Swin-T (Liu al., 2021b), S3 (Swin-T) (Chen el al., 2021b).
ConvMeXt- (1.u
2022), ResMTP-R22 (Touvron al., 2022a)
Transformer Final GAP,
XCIT M24
2021), XCiT T12 (Ali el., 2021), HaloRegNetz B!, TNT-S (Han et al., 2021b), Visforeer-B (Chen et al., 2021d).
3 GeLL, Pooling at stem, InRes: 224
PiT-S (Hee 2021), PiT-B (Hee et. 2021)
Stem layer; stack 3 cony, 2D S., InRos: 256,
HaLoMet-bG (Vaswani al. 2021), LambdaResNett-oo (Bello, 2021), BoTNeT 26 (Srinivas et al., 2021), GC ReaNeXt: 50 (Cao et 2019).
Pooling at stem, SILU
ECAfteloNeet 50", BBCt-BoTeeet-222
Stem layer: stack 3x3 convs,
LamH:1o3oTNett550 wttp---s0...-----eeeee-- SE-BoTNNt--3',, 8E-HaloNet-331,, Tallo2877Net-50 60-ReaVat-50 (Cao al., 2019),
InRes: 256, SA, CWA: middle of blocks, CNN
ECA Net 33 (Wang et al., 2020c)
Stem layer: 7x7 conv w/ s2, ReLUU Pooling stem, CNN. BN
Res (He et al, 2016a), ResSet-101 (He et al., 2016a). ResNeXt-50 et al., 2017), Mide RasNet-50 (Zagoruyko Komodakis, 2016),
SE- Re (Hu et al., 2018), Se-ReeNeett (Hu et al., 2018).
ResNet-V2-50 (Hl et al., 2016b), RcsNett (Hl et al., 2016b).
R5 (GN) (Wa He, 2018), ReaNat-50 (R) (RlurPool) (Zhing, 2019),
DP3 107 (Chen et al, 201 Xcept 65 (Chollet, 2017)
NAS, Stem layer: 3x3 conv w/ s2 CWA: middle blocks, CWA. DW Conv
EfficientNet-B2 (Tan & Le, 2019a1. V at., 2021a), ReXNet (x1.5) (Han Mi xNet (Tan & Le, 2019b), NF NF-Reg (Brock et al., 2021a)
7
2021a), R-3Nety-32 (Radosavovic al., 2020),
Input resolution: 224,
NFNet LO". BCA NFNet LO", Poo1Forme: M48 (Yu et al., 2022),
Stem layer: stack 3x3 convs, Group Conv, Final GAP, 2D SA
Restiett 50 (Zhang et al., 2022a), ResNet-V2-55 122--0---EVOS ConvMixer-15366
(Trockman & Kalter, 2022)
ReL.U, Input resolttion: 224, DW Cony, BN 2D self-attention
ViT-B (patch size 32; (Dosovitskly et al., 2021), R26+ViT-3 (Steiner et al,, 2022),
DIA x 102 (Yu et al., 2018), eSE VoVNet 39 (Lee Park, 2020), bestet-101-4 (He et al., 2019), RegNetX-322 (Radosavovic al., 2020)
9
HRNet-W32 (Wang el al., 2020h)
ReLU + Leaky ReLUU InRes: 256,
CSPResMet-bD (Wang et al., 2020a), CSPResNext (Wang et al., 2020a), CSFDarkNet 53 (Bochkovskiy et., 2020), NE Realltt 50 (Brock al., 2021a)
10 Stem layer: 7x7 conv, CNN.
Pooling at stem
regimes. 22 ResNet-50 models include model trained by PyTorch (Paszke et al., 2019), models trained by GluonCV (Guo et al., 2020), semi-supervised and semi- cemi-yeakly supervised models on billion-seale unlabeled images (Yalniz et al., 2019), models trained by different augmentations (Cutout (DeVries & Taylor, 2017), Mixup (Zhang et al., 2018), manifold Mixup (Verma et al., 2019), CutMix (Yun et al., 2019), and feature CutMix); 11 optimized models by Wightman et al. (2021). 7 ViT-S models contain the original ViT training setup (Dosovitskiy al., 2021), stronger data augmentation setup (Touvron et al., 2021), the training setup with distillation (Touvron et al., 2021), an improved DeiT training setup (Touvron et al., 2022b), and self-supervised training fashions including MoCo v3 (Chen et al., 2021c), MAE (He et al., 2022) and BYOL (Grill et al., 2020).
Tab. D.1 shows the comparison of similarity scores between the same architecture but different learning methods (a smaller similarity means more diversity). First, we observe that using different random initialization or different optimization hyper-parameters shows high correlations with each other (almost 2 4.2) while the average similarity score between various neural architectures is 2.73. In other words, the difference in initializations or optimization hyper-parameters does not signifi- cantly contribute to the model diversity. Second, we observe that using different learning technique (e.g., different augmentation methods, and different types of supervision) remarkably affect the simi- larity score (3.27 for ResNet and 3.44 for ViT), but is not significant as the architectural difference (2.73). Furthermore, we discover that the change of similarity caused by different initializations or hyper-parameters is less marked than the change caused by different architecture (Fig. D.1).

---

Under review as a conference paper at ICLR 2023
In practice, learning the same network with significantly different learning techniques is challenging and time-consuming because the Bengio, 2012). Especially, a novel optimization technique (e.g., using more data points for semi- or self-supervised learning, using different augmentation methods, or adopting advanced optimization settings) sometimes needs completely different optimization setting compared to the vanilla set- ting. On the other hand, learning a new architecture is easier than using a different learning paradigm in terms of hyper- parame- selection and implementation. Thus, we can conclude that if we need multiple diverse models, then using multiple architectures is more effective and easier to optimize.
search space is tremendously large (Bergstra &
4 PRACTICAL APPLICATIONS OF THE NETWORK SIMILARITY SCORE
In this section, we show the practical usage of our similarity score. First, we show that using more diverse models will lead to better ensemble performance (Sec. 4.1.. Second, we suggest similarity- based guideline for the choice of a teacher model when distilling to a specific architecture. Different from several existing studies that show a more similar teacher is better, our findings show that their observation is valid only if the teacher and student networks are based on the same architecture (e.g., Transformer). On the other hand, when the teacher and student networks are based on different architectures, a more dissimilar teacher is better (Sec. 4.2).
4.1 MODEL DIVERSITY AND ENSEMBLE PERFORMANCE
The model ensemble is a practical technique to achieve high per- formances in practice, however, there is still not enough study on
Erarr Redurtion Rate (%)
14:
the choice of the models for the ensemble in large-scale com-
plex models. Previous studies are mainly conducted on very small datasets and linear models (Kuncheva & Whitaker, 2003). We re-
port the relationship between the model similarity and the en-
semble performance based on the unweighted average method (Ju
et al., 2018) (i.e., averaging the logit values of the ensembled mod- els). Because the ensemble performance is sensitive to the origi- nal model performances, we define Error Reduction Rate (ERR) as 1 (Mt..(M)
2.0
2.5
3.0
3.5
Similarity
Figure 2: Correlation between model similarity and ensemble performance. The trend line and its 90% confidence interval are shown in the blue line.
TATT Emer Bm(m)) where M is the of the ensembled models, ERR(m) denotes the top-1 ImageNet validation error of model m, and (.rens()) denotes the top-1 error of the ensemble.
We first measure the 2-ensemble performances among the 69 neural architectures in Tab. 2 (i.e., the number of ensembles is (22) 2346). We plot the relationship between the model similarity and ERR in Fig. 2. We observe that there exists a strong negative correlation between the model similarity and the ensemble performance (Pearson correlation coefficient -0.31 with p-value 1.74 x 10 10-54), ) Le., more diversity leads to better ensemble performance.
m 1 Samples (96)
ImageNet Top-1 Error (%)
17.4
Clusters
14
14
Clusters
17.2
13
+
17.0
13
Reducctinn
16.0
10
12
16.6
16.4
of Clusters
16.2
10
Ensembled Models
Ensembled Models
Ensembled Models
(a) Top-1 Error
(b) Error reduction rate
(c) "All wrong" samples ratio
Figure 3: Model diversity by architecture difference and ensemble performance. We report ensemble performances by varying the number of ensembled models (N) and the diversity of the ensembled models. The diversity is controlled by choosing the models from different clusters (larger denotes more diversity).
7

---

Under review as a conference paper at ICLR 2023
We also conduct N-ensemble experiments with N 2 2 based on the clustering results in Tab. 2. We evaluate the average ERR of the ensemble of models from & clusters, f.e., if N = 5 and A = 3, the ensembled models are only sampled from the selected clusters while ignoring the other 7 clusters. We investigate the effect of model diversity and ensemble performance by examining k = 1... N (i.e., larger It denotes more diverse ensembled models). We report three evaluation metrics in Fig. 3: ImageNet top-1 error of the ensemble (lower is better), error reduction rate (ERR) (higher is better), and the number of wrong samples by all ensembled models (lower is better). In all metries, we observe that the ensemble of more diverse models (i.e., by increasing the number of clusters for the model selection) shows better ensemble performance. Interestingly. Fig. 3a, we observe that when the number of clusters for the model selection (k) is decreased, the ensemble performance by the number of ensembled models (N) quickly reaches saturation. Similarly, in Fig. 3c, we observe that the number of wrong samples by all models is decreased by selecting more diverse models. In other words, the different architecture choices can lead to different model decision boundaries.
In Sec. 3.2, we showed that the different training strategies are not as effective as different architectures in terms of diversity. To examine the effect of this observation on the ensemble scenario, we report the ensem- ble results of different training strategies, i.e., the same ResNet -50 and ViT-S models in Tab. D. For comparison with different archi- tectures, we also report the ensemble of different architectures where all ensembled models are from different clusters (i.e., when N k in Fig. 3). Fig. 4 shows that although using diverse training regimes (blue lines) improves ensemble performance most among other training tech- niques (red and green lines), the improvements by using different archi-
Eror Roduction 184)
15 12 4
of Ensemoled Models
(a) ResNet-
tectures (yellow lines) are more significant than the improvements by
Enor Reduuction Rate
15 12
using different training regimes (blue lines) with large gaps.
4.2 MODEL DIVERSITY AND KNOWLEDGE DISTILLATION
8 of Ensembled Models
Knowledge distillation (Hinton et al., 2015) a training method for transferring rich knowledge of a well-trained teacher network. Intu- itively, distillation performance affects lot by choice of the teacher network. however, the relationship between similarity and distillation performance has not yet been explored enough, especially for ViT. In this subsection, investigate how the similarity between teacher and student networks contributes to the distillation performance. There are several studies showing two contradictory conclusions; Jin al. (2019); Mirzadeh et al. (2020) showed that a similar teacher leads to better dis- tillation performance; Touvron al. (2021) reports that distillation from a substantially different teacher is beneficial for ViT training.
(b) ViT-S
Figure Model diversity by training techniques and ensemble perfor- mance. The same metrics Fig. 3 for ResNet- and ViI-S models in Tab. D.I. are shown. "All" means N=k in Fig. 3 (ie., # of ensembled models of Clusters).
Settings. We use the same distillation setting by Touvron et al. (2021); We train Vi models with the hard distillation strategy (Hinton et al., 2015) (i.e., using the predicted label by the teacher as the ground truth label). We train 25 Vir-Ti models with different teacher networks from Tab. 2. Here, we include all niT teachers that have a similarity score larger than 2.5, and 25 models among 67 models (except for ViT and DeiT)) are sampled according to the distribution of the simi- larity scores to prevent sampling bias. We describe the detailed training settings in Appendix F.I.
Fig. 5a illustrates the relationship between the teacher-student similarity and the distillation perfor- mance. Fig. 5a tends to show not significant negative correlation between teacher-student similarity and distillation performance (-0.32 Pearson correlation coefficient with 0.12 p-value). However, we only focus on when the teacher and student networks are based on the same architecture (i.e., Transformer), we can observe a strong positive correlation (Fig. 5b) - 0.70 Pearson correlation co- efficient with 0.078 p-value. In this case, our observation is aligned with Jin et al. (2019); Mirzadeh et al. (2020): a teacher similar to the student improves distillation performance. However, when the observe
teacher and student networks are based different architectures (e.g. CNN), then a stronger negative correlation (Fig. 5c) with 0.51 Pearson correlation coefficient and 0.030 p- value. In this case, more dissimilar teacher leads to better distillation performance. We also test
8

---

Under review as a conference paper at ICLR 2023
g 10.02
16.00
75.75
25.53
15.00
35.50
Eon AcEu
36.75
14.75
76.75
14.50
34.00
76.00
74.00
2.50 2.5 2.50
2.75
1.50 Similarity
1.5 350
175 4.30
2:50
225 250
7.75
3.5 1.55 3.5 400 Similarity
2.50
400
Similarity
(a) Results various teachers
(b) Transformer teachers
(c) CNN & other teachers
Figure 5: Model diversity and distillation performance. (a) We show the relationship between teacher- student similarity and distillation performance of 25 DeiT-S models distilled by various teacher networks. We show the relationship when the teacher and student networks are based (b) Transformer and (c) otherwise.
other factors that affect distillation performance in Appendix E.2; We observe that distillation performance not correlated to teacher accuracy in our experiments.
Why observe contradictory results for Transformer teachers (Fig. 5b) and other teachers (Fig. 5c)? Here, we conjecture that when the teacher and student networks differ significantly, distil- lation works strong regularizer. In this case, using as stronger regularizer (Fig. 5c). On the other hand, we conjecture that if two networks are similar, then distillation works as casy-to-follow supervision for the student network. In this case, we can assume that more similar teacher will work better because more similar teacher will provide more easy-to-follow supervision for the student network (Fig. 5b). In our experiments, we found that the regularization effect improves distillation performance better than to-llooww supervision (i.e., best performing distillation result by CNN teacher). Therefore, practice, we recommend using significantly different teacher network for achieving better distillation performance (e.g., using RegNet (Radosavovic et al., 2020) teacher for ViT student as Touvron et al. (2021)).
dissimilar teacher be considered
LIMITATIONS AND DISCUSSIONS
We include three discussions related to our similarity score. Due the space limitation, the full details of each item can be found in Appendix G. Our discussions include:
Robustness of our similarity score to the choice of adversarial attack method. We show that the similarity rankings by PGD (Madry et al., 2018), AutoAttack (Croce & Hein, 2020), and PatchFool (Fu et al., 2022) do not differ significantly.
How adversarially trained models affect our similarity scores.
An efficient approximation of analysis for novel model (i.e., for novel 70th model for our analysis). We show that instead of generating adversarial attacks for all 70 models, just using the attacked images from the novel model could be enough.
Other possible applications of our similarity score (e.g., novel architecture development, analyzing the hyper- parammeereeter sensitivity, model selection for selecting diverse models).
6 CONCLUSION
We have explored similarities between image classification models to investigate what makes the model diverse and whether developing and using diverse models are required. For quantitative and model-agnostic assessment of the similarity, we have suggested new score based on attack transfer- ability demonstrating differences in input gradients. Using our new similarity function, we conduct a large-scale and extensive analysis using 69 state-of-the-art ImageNet models. We have shown that macroscopic architectural properties, such as base architecture and stem architecture, have a greater impact on similarity than microscopic operations, such types of used convolution, with numer- ical analysis. Furthermore, the effectiveness of the training strategy is minor compared to model architectural design-me design-related related features. Finally, have firstly discovered the advantages of using di- verse models in ensemble or distillation strategies for a practical scenario with a large-scale training dataset and highly complex architecture.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et al. Xcit: Cross-covariance image transformers. 2021. 6
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: (o00:660005555 2016. 1,4
Naman Bansal, Chirag Agarwal, and Anh Nguyen. Sam: The sensitivity attribution methods to hyperparameters. In IEEE Conf. Compur. Vis. Pattern Recog., 2020.
Irwan Bello. Lambdanetworks: Modeling long- range interactions without attention. In Int. Conf.
Learn. Represent., 2021. 4,
James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012.
Alexey Bochkovskiy, Chien-Yat Wang, and Hong- Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004 10934, 2020. 6
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001. 5
Andrew Brock, Soham De, and Samuel L Smith. Characterizing signal propagation to close the performance gap in unnormalized resnets. In Iut. Conf. Learn. Represent., 2021a.
Andy Brock, Soham De, Samuel Smith, and Karen Simonyan. High-performance large-scale image recognition without normalization. Int. Conf. Mach. Learn., 2021b.
Yue Cao, Jiarui Xu, Stephen Lin. Fangyun Wei, and Han Hu. Genet: Non-local networks meet squeeze-excitation networks and beyond. In Int. Conf. Comput. Vis. Worksh. 2019. 4, 6
Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. Crossvit: Cross-attention multi-scale vision transformer for image classification. Ini. Conf. Comput. Vis., 2021a.
Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, and Haibin Ling. Searching the search space of vision transformer. 2021b. 6
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. Int. Conf. Compur. Vis., 2021c. 5, 6,
Yunpeng Chen, Jianan Li, Huaxin Xiao, Xiaojie Jin, Shuicheng Yan, and Jiashi Feng. Dual path networks. In Adv. Neural Inform. Process. Syst., 2017. 6
Zhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu, Longhui Wei, and Qi Tian. Visformer: The vision-friendly transformer. In Ini. Conf. Comput. Vis., 2021d. 6
Junsuk Choe, Seong Joon Oh, Sanghyuk Chun, and Hyunjung Akata, Zeynepand Shim. Evaluation for weakly supervised object localization: Protocol, metrics, and datasets. IEEE Trans. Pattern Anal. Mach. Intell., 2022. 1, 3, 24
François Chollet. Xception: Deep learning with depthwise separable convolutions. In IEEE Conf. Comput. Vis. Pattern Recog., 2017. 5,
Xiangxiang Chu, Zhi Tian, Yuging Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design spatial attention vision transformers. Adv. Neural Inform. Process. Syst., 2021. 6
Sanghyuk Chun, Seong Joon Oh, Sangdoo Yun, Dongyoon Han, Junsuk Choe, and Youngjoon Yoo. An empirical evaluation on robustness and uncertainty of regularization methods. Int. Conf.
Mach. Learn. Worksh., 2019. 5
Sanghyuk Chun, Wonjae Kim, Song Park, Minsuk Chang Chang, and Seong Joon Oh. Ecev caption: Correcting false negatives by collecting machine-and-hume humann-vrified verified image-caption associations for ms-coco. In Eur. Conf. Comput. Vis., 2022. 24
10

---

Under review as a conference paper at ICLR 2023
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Int. Conf. Mach. Learn., 2019. 3
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In Int. Conf. Mach. Learn., 2020. 3, 9, 22
Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, et al. Fbnetv3: Joint architecture-recice search using predictor pretraining. In IEEE Conf. Comput. Vis. Pattern Recog., 2021a. 6
Zihang Dai, Hanxiao Liu, Quoe V Le, and Mingxing Tan. Coatnet: Marrying convolution and attention for all data sizes. In Adv. Neural Inform. Process. Syst., 2021b.
Terrance Deies and Graham W Taylor. Improved regularization convolutional neural networks with cutout. arXiv preprint
2017. 6, 20
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In Int. Conf. Mach. Learn., 2017. 1,2
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In Int. Conf. Learn. Represent., 2021. 6, 20
Stéphane d' A'scol,,, Hugo Touvron, Matthew L Leavitt, Ari S Morcos, Giulio Biroli, and Levent Sagun. Convit: Improving vision transformers with soft convolutional inductive biases. In Int. Conf. Mach. Learn., 2021. 6
Jerome H Friedman. Greedy function approximation: gradient boosting machine. Annals of statis- tics, pp. 1189-1232 1232, 2001. 5, 16
Yonggan F Fu, Shang Wu, Yingyan Lin, et al. Patch-fool: Are vision transformers always robust against adversarial perturbations? Int. Conf. Learn. Represent. 2022. 9, 22
Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In Adv. Neural Inform. Process.
Syst., 2018.
Robert Geirhos, Kristof Meding, and Felix A Wichmann. Beyond accuracy: quantifying trial-by-tria behaviour of cnns and humans by measuring error consistency. Adv. Neural Inform. Process. Syst, 2020.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. 2020. 5, 6, 21
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In Int. Conf. Learn. Represent., 2021. 24
Jian Guo, He He. Tong He, Leonard Lausen, Mu Li, Haibin Lin, Xingjian Shi, Chenguang Wang, Junyuan Xie, Sheng Zha, Aston Zhang. Hang Zhang, Zhi Zhang, Zhongyue Zhang, Shuai Zheng, and Yi Zhu. Gluonev and gluonnip: Deep learning in computer vision and natural language processing. Journal of Machine Learning Research, 21(23):1-7, 2020. 6, 20, 23
Dongyoon Han, Sangdoo Yun, Byeongho Heo, and YoungJoon Yoo. Rethinking channel dimensions
for efficient model design. In IEEE Conf. Comput. Vis. Pattern Recog., 2021a. 6
Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in
transformer. 2021b. 6
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog-
nition. In IEEE Conf. Comput. Vis. Pattern Recog., 2016a. 1, 2, 6
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Eur. Conf. Comput. Vis., 2016b. 6

---

Under review as a conference paper at ICLR 2023
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In IEEE Conf. Comput. Vis. Pattern Recog. 2022. 5, 6, 21
Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li. Bag of tricks for image classification with convolutional neural networks. In IEEE Conf. Comput. Vis. Pattern Recog.. 2019.
Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. In Int. Conf. Comput. Vis., 2021.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. Distilling the knowledge in a neural network. In Adv. Neural Inform. Process. Syst. Worksh., 2015. 2.8
Jie Hu, Shen, and Gang Sun. Squeeze-and-excitttion networks. In IEEE Conf. Comput. Vis. Patlern Recog., 2018. 4,
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they features. In Adv. Neural Inform. Process. Syst. 2019. 23
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Int. Conf. Mach. Learn., 2015. 1, 4
Xiao Jin, Baoyun Peng, Yichao Wu, Yu Liu, Jiaheng Liu, Ding Liang, Junjie Yan, and Xiaolin Hu. Knowledge distillation via route constrained optimization. In Int. Conf. Comput. Vis., 2019. 2,
Cheng Ju, Aurélien Bibaut, and Mark van der Laan. The relative performance ensemble methods with deep convolutional neural networks for image classification. Journal of Applied Statistics, 45(15):2800-281 2018.
Hamid Karimi and Jiliang Tang. Decision boundary of deep neural networks: Challenges and oppor- tunities. In Proceedings of the 13th International Conference on Web Search and Data Mining, 2020. 2, 24
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Int. Conf. Mach. Learn., 2019. 1,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In Adv. Neural Inform. Process. Syst., 2012. 4,
Ludmila Kuncheva and Christopher J Whitaker. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine learning, 51(2):181-207 2003. 2, 3,
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont- Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4. Int. J. Comput. Vis., 128(():1956-1)))
2020. 24
Youngwan Lee and Jongyoul Park. Centermask: Real-time anchor-free instance segmentation. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Adv. Neural Inform. Process. Syst., 2018. 2,
Hanxiao Liu, Andy Brock, Karen Simonyan, and Quoe Le. Evolving nemalization-activatoon lay- ers. In Adv. Neural Inform. Process. Syst. 2020. 5
Hanxiao Liu, Zihang Dai, David So, and Quoc Le. Pay attention to mlps. In Adv. Neural Inform. Process. Syst. 2021a.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Int. Conf. Comput. Vis., 2021b. 4, 6
12

---

Under review as a conference paper at ICLR 2023
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In IEEE Conf. Comput. Vis. Pattern Recog., 2022. 4, 6
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. Int. Conf. Learn. Represent.. 2018. 3, 99,, 22
Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. Improved knowledge distillation via teacher assistant. In Proceedings the the AAAI Conference on Artificial Intelligence, 2020. 2.
Andrew Ng. Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Adv. Neural Inform. Process. Syst, 2001.
Chanwoo Park, Sangdoo Yun, and Sanghyuk Chun. A unified analysis of mixed sample data aug- mentation: A loss function perspective. In Adv. Neural Inform. Process. Syst. 2022. 5
Namuk Park and Songkuk Kim. How do vision transformers work? In Int. Conf. Learn. Represent., 2022. 2
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Adv. Neural Inform. Process. Syst., 2019. 6, 20
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathicu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-leam: Machine learning in python. Journal of Machine Learning Research, 12:2825-2330, 2011. 16
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollar. Designing network design spaces. In IEEE Conf. Comput. Vis. Puttern Recog., 2020. 6, 9
Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision transformers see like convolutional neural networks? Adv. Neural Inform. Process. Syst. 2021. 1,2
Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017. 4
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis., 2015. 2, 4
Luca Seimeca, Scong Joon Oh, Sanghyuk Chun, Michael Poli, and Sangdoo Yun. Which short- cut cues will dnns choose? a study from the parameter-space perspective. In Int. Conf. Learn. Represent., 2022. 3
Ali Shafahi, Amin Ghiasi, Furong Huang, and Tom Goldstein, Label smoothing and logit squeezing: replacement for adversarial training? arXiv preprint arXiv: /5555 2019. 5
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Int. Conf. Learn. Represent., 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. In Int. Conf. Learn. Represent. Worksh., 2014. 1, 3
D. Smilkov, N. Thorat, B. Kim, F. Viégas, and M. Wattenberg. Smoothgrad: removing noise by adding noise. Inf. Conf. Mach. Learn. Worksh. 2017.1
Gowthami Somepalli, Liam Fowl, Arpit Bansal, Ping Yeh-Chiang, Yehuda Dar, Richard Baraniuk, Micah Goldblum, and Tom Goldstein. Can neural nets learn the same model twice? investigat- ing reproducibility and double descent from the decision boundary perspective. In IEEE Conf. Comput. Vis. Pattern Recog., 2022. 1, 1, 3, 5, 5,24
13

---

Under review as a conference paper at ICLR 2023
Jost Tobias Springenberg. Alexey Dosovitskiy. Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In Int. Conf. Learn. Represent. Worksh. 2015.
Aravind Srinivas, Tsung- Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, and Ashish Vaswani. Bottleneck transformers for visual recognition. In IEEE Conf. Comput. Vis. Pattern Recog., 2021.
4, 6
Andreas Peter Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit, and Lucas Beyer. How train your vit? data, augmentation, and regularization in vision transformers. In Transactions on Machine Learning Research, 2022. 6
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Int. Conf. Mach. Learn., 2017.
AH Sung. Ranking importance of input parameters of neural networks. Expert Systems with Appli- cations, 15(3-)):40-4411 1998. 1,3
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna, Rethinking the inception architecture for computer vision. In IEEE Conf. Comput. Vis. Pattern Recog., 2016.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In Int. Conf. Mach. Learn., 2019a. 6
Mingxing Tan and Quoc V Le. Mixconv: Mixed depthwise convolutional kernels. In Brit. Mach. Vis. Conf., 2019b.
Damien Teney, Ehsan Abbasnejad, Simon Lucey, and Anton van den Hengel. Evading the simplicity bias: Training diverse of models discovers solutions with superior ood generalization. In IEEE Conf. Comput. Vis. Pattern Recog., pp. 16761-16772, 2022.
Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov. Lucas Beyer, Xiaohua Zhai, Thomas Un- terthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et MIp-mixer: An all-mip architecture for vision. In Adv. Neural Inform. Process. Syst., 2021. 5
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-cfficient image transformers & distillation through attention. In Int. Conf. Mach. Learn., 2021. 6, 8, 9, 20, 21, 23
Hugo Touvron, Piotr Bojanowski. Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, et al. Resmlp: Feed- forward networks for image classification with data-efficient training. IEEE Trans. Pattern Anal. Mach. Intell., 2022a.
Hugo Touvron, Matthieu Cord, and Hervé Jégou. Deit ii:: Revenge of the vit. arXiv preprint arXiv:2204.0771188 18, 2022b. 6,
Asher Trockman and J. Zico Kolter. Patches are all you need? arXiv preprint arXiv:2201.09792. 2022. 6
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In Int. Conf. Learn. Represent. 2019. 21,
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In Adv. Neural Inform. Process. Syst., 2017. 5,5
Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Hechtman, and Jonathon Shlens. Scaling local self-attention for parameter efficient visual backbones. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 4,
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez- Paz, and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In Int. Conf. Mach. Learn., 2019. 6, 20
14

---

Under review as a conference paper at ICLR 2023
Chien Yao Wang. Hong- Yuan Mark Liao, Yueh-Hua Wu, Ping- Yang Chen, Jun-Wei Wei Hsich, and I- Hau Yeh. Cspnet: new backbone that can enhance learning capability of cnn. In IEEE Conf. Comput. Vis. Pattern Recog. Worksh. 2020a. 6
Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al. Deep high-resolution representation learn- ing for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell., 43(10):3349-3364 2020b.
Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu, Eca-net: Efficient channel attention for deep convolutional neural networks. In IEEE Conf. Comput. Vis. Pattern Recog., 2020c. 6,
Ross Wightman. Pytorch image models. 2019. 2, 4. 5, 16
https://githab..oom rw ghtmmn/
pytorch- - Sacthhimmae--aae
Ross Wightman, Hugo Touvron, and Hervé Jégou. Resnet strikes back: An improved training pro- cedure in timm. In Adv. Neural Inform. Process. Syst. Worksh., 2021. 6, 20
Yuxin Wu and Kaiming He. Group normalization. In Eur. Conf. Comput. Vis., 2018. 6
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In IEEE Conf. Comput. Vis. Puttern Recog., 2017. 5, 6, 23
I Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scal scale semi- supervised learning for image classification. arXiv preprint arXiv: 005460 2019. 5, 6, 20
Fisher Yu, Dequan Wang. Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. In IEEE
Conf. Comput. Vis. Pattern Recog., 2018. 6
Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer actually what you need for vision. IEEE Conf. Comput. Vis. Pattern Recog., 2022. 6
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Int. Conf. Comput. Vis., 2019. 5, 6, 20
Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, and Sanghyuk Chun. Re-labeling imagenet: from single to multi-labels, from global to localized labels. In IEEE Conf. Comput. Vis. Pattern Recog., 2021. 22
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Brit. Mach. Vis. Conf., 2016.
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R. Manmatha, Mu Li, and Alexander Smola. Resnest: Split-attention networks. In IEEE Conf. Comput. Vis. Pattern Recog. Worksh. 2022a. 6
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In Int. Conf. Learn. Represent., 2018. 5, 6, 20
Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? In Int. Conf. Learn. Represent. 2021. 5
Richard Zhang. Making convolutional networks shift-invariant again. In Int. Conf. Mach. Learn., 2019. 6
Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, Serean Ô Arik, and Tomas Pfister. Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, 2022b. 6
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. In Int. Conf. Learn. Represent., 2017.1
15

---

Under review as a conference paper at ICLR 2023
APPENDIX
We include additional materials in this document, such as the details of our similarity function (A) the network details (B), spectral clustering method (C), training settings (E), setting of knowledge distillation (F.1), and the reason of using of attack transferability for comparing decision boundary (H). We also provide experimental results of computing pairwise similarities (D) and distillation performance (F.2). We discuss some limitations and discussion points of our work (G), such as the effect of the choice of attack methods, an efficient approximation, and possible applications.
A MORE DETAILS OF OUR SIMILARITY FUNCTION
Ideally, we would like to define a similarity function between two models with the following three properties: (1) x arg min, d(x, y) (2) d(x; y) d(y, x) (3) d(ar, y) > d(x, 2)) if + y. The adver- sarial attack transferability generally satisfies these properties. If we assume the adversary is optimal, then the transferred attacked accuracy (acCB-A)) will be zero and it will be the minimum (because accuracy is non-negative). A- + a is symmetric. It will satisfy d(x, y) 2 d(æ, a) if * y where it is weaker condition than (3). In other words, adversarial attack transferability and equation 1 can serve as a similarity score function from functional point of view.
B DETAILS OF NETWORKS USED FOR OUR ANALYSIS
We use 69 models in our research to evaluate the similarity between models and to investigate the impact of model diversity. In the main paper, we mark the names of models based on their research paper and PyTorch Image Models library (t imi; 0.6.7 version) (Wightman, 2019). Tab. B. shows the full list the models based on their research paper and timm alias.
In Tab. B.2 and Tab. B.3, We show the full network specification defined by Tab. 1. We follow the corresponding paper and t iimm library to list the model features.
We fit gradient boosting regressor (Friedman, 2001) based on Scikit- 201 1) and report the permutation importance of each component in Fig. 1. The number of boosting stages, maximum depth, minimum number of samples, and learning rate are 0.02, respectively. Permutation importance is computed by permuting feature 10 times.
(Pedregosa al.,
500, 12, 4, and
Table B.1: Lists of 69 models and their names based on their research paper and t timm library.
in timm
paper
in timen
in paper
in cimm
paper
botne1261.22 256
BoTKeL: 26
gluon.xeeption xeeptionss gmlp. +16 224
XoepLior 55
resser50l.gg resneibuu500
Readlee 50 (CN)
cea lif amall
CoaT-1ite Sma.l
Restiet-al (BlurPwol)
base
CoaViI-BB
bals2botned00s..
Balo2BoTHot--
resnetv2.101
Bastatt-2--2
convmiser__ I 1536_20
CanuMi swr 1536/20 /20
SollneetSS huloregeetz.
He bollet 50
resnetv2_50
Restlet V2 50
convnest.c Jimy
Coawtlext-7
HaloRegleez2
resmetv2.500 50d evos
Rostiet-12-b0-Ev00
240
CrossVi2-3
w64
HRttet-W33
32x4d
BestloXt-50
opsheekne153
junest Jiny
eppeesnec00
CS2RasMett-1
res
LanbdaRestitt--
scbutnet331s
GEBoINaS-33
ospresesc50
C92NwalleX
aambalohotset5iss 256
oaanl 50
schalonet33ts
1kithhor_putch16.. 16
DeiT-B
misnetxd
Misttie--X XXt
scresnet50
SERestie=- 50
deit small patch16 224
DeiT-S
at regnet bl aCrosnet50
HE -Ro-Net--B11
scresnex150 32x4d
SERcoNaxXt-2
dlal02x2
DEA x 102
NF Rew 50
swin_si_ting 224
213 (Swin=T)
dpn
DPM-107
I0
NENet-LO
swin Jiny winchl.windww/ 22224
Swin-T
oca, botoen:2tes 256
BCA- Bo33-BoTHex--22 26
b 224
Pi2-B
tnt. pocch1 224
T82-8
oes coJJomonet26is
Mi iolleett 26
pir 2M_42334
Pitt-8
woinsppapytJht (wins small
niner.
BCA-MEXet--D
poulformer m118
PeolFornce-
Ixins-SVI-S
eosuesner33ts
segnetx,533
RegNee 320
small
efficienteet.2
En Gericienteee--
regnety013
Reghtet
vil_baw_pptch22..0
Vi
vovnet39b
c93-voWMaz-39
seamip
BaaMLP-S22
parché
tboelv). y
FBNcEV3-S
resmlp_hig.24.0
RcSMLPP-22 B22
B2G+V11-2
gevesnetix
GoSealee 50
resnesi50d
BeaaNaS 50
wade redeeesse
Midw Reaket--S
5rrssnsstths
SCcResteett-55
resnet 101
Re Resttet-1 -101-100
seit. modium p4.
XCST-M24
glusa resnet101 vic
beakkt--11--
resnec50
RasNot-30
scit tiny 12 224
XCII-F12
C SPECTRAL CLUSTERING DETAILS
We use the normalized Laplacian matrix to compute the Laplacian matrix. We also K-means clustering 100 times and choose the clustering result with the best final objective function to reduce the randomness by the K-means clustering algorithm.
16

---

Under review as a conference paper at ICLR 2023
Description of features of 69 models. "s" in "Stem layer" indicates the stride of a layer in the stem, and the number before and after "s" are a kernel size and size of stride, respectively. For example, "3s2/3/3" means that the stem is composed of the first layer having 3 x 3 kemel with stride 2, the sccond layer having 3 3 kernel with stride 1, and the last layer having x with stride 1.
Table B.2:
Hierarchical structure
Model
architecture
Stem layer
Input resolation
Nommalization
Activation
hotner261, 256
CNN
Yes
3/2/3/3
256 x 256
BN
Rel
convmixer_ 1536 1536_20
CNN
Yes
727
x 234
BN
GeLUU
convecxt.iny
CNN
Yes
4s4
224 x 224
LN
GeLU
0spdakkn153
CNN
Yes
256 256
BN
Leoky RcLU
espresnet 50
CNN
Yes
762
256 x 256
BN
1 Lesky ReI.U
cspresnext501
CNN
Yes
762
256 256
BN
1 Lasky ReL.
clal0022
CNN
Yes
761
224 x 224
BN
ReLU
dpe 107
CNN
Yes
762
234 x 234
BN
Rell UI
oca, homextrons 256
CNN
Yos
36273/3
250 256
BN
SILU
oca In2lonest256s
CNN
Vo5
36273/3
250 x 156
BN
SILU
CNN
36233/1/52
224
Norm-Troe
SILU
e53 eaijesser33ss
CNN
Vos
342/3/342
256 x 256
BN
SIL U
CNN
Yes
362/37362
224 224
BN
ReL.U
gcresnet50t
CNN
V25
362/37322
256 x 256
LN + BN
KcLU
CNN
3:2/3/3
256 256
LN BN
ReLU + SiLU
pluon roinet 101
CNV
362/3/3
224 x 234
BN
Rel.A)
gluon xceppionts
CNN
362/3
299 x 259
BN
Rel.U
hale2hommet50s.220
Yos
3x2/3/322
256
BN
SILU
halonesSits
CNN
V85
362/3/3
256 x 256
BN
Sit.
braet w64
CNN
362732
224 224
BN
KeLUU
lambda resnet50ts
CNN
V25
3627373
256 x 256
BN
SILU
lanbaalcbotett5kk 256
CNN
3-233/622
256 256
BN
SILU
nf_esnert0
CNV
742
256 x 256
Norm-froe
Rell
afner J0
CNN
Vos
3/37222
224 x 234
Norm- free
ReLU x SILU
pooliormer
Y05
764
223
GN
CaLU
resmest50d
CNN
Yee
362/3/3
224 x 224
BN
ReLU
CNN
7s2
224
BN
Rell
resnet50
CNN
125
762
224 x 234
BN
Kel.U)
resnet50_gn
CNN
742
224 224
GN
ReLU
resnctolur50
CNV
752
224 x 224
BN
ReLU
resnetv2 10]
CNN
762
224 x 224
BN
KeL.U
resngt22
105
752
223
BN
RcLU
CNN
362/3/3
224 x 224
EvoNorm
50_32
CNN
7x2
224
BN
Rell
sebotnets3ts 256
CNN
Ves
362/37362
256 x 256
BN
UeL.U x SILU
schalonei331s
CNN
Vos
32233232
256 156
BN
RcLU + SILU
seresne55
CNV
Yes
762
224 x 224
BN
ReLU
4eresnex150) 32x4d
CNN
Yes
7:2
224 x 234
BN
RellUl
wide resnet50).
Ves
762
x 234
BN
Rel.U
hose
Transfomerr
No
16s16
224 x 2244
LN
GeLU
cruhovil. 240
Transfommer
Yes
16x16
240
LN
GeLU
deit hase. patch16. 224
Transfommrr
No
16s16
224 x 224
LN
GeLU
small pachlo 224
Transfonner
No
16:16
221 2211
LN
GSLU
Ja. js.nusi liny
Franddommnr
Yes
454
224 x 224
LN
GaLU
233
Transfommer
Yes
1668
224 x 224
I.N
Gel.II
swin Jin pww.ppaaccaaaeee Window] 2,224
Transfomser
Yes
464
x
Gol.U
tnl paich16 224
Transtonaer
Yos
794
224 x 223
LN
GcLU
Transfomnce
32x32
224
LN
GeLUU
vit. small putch16 224
Translomer
No
16x16
224 x 224
IN
GalA1
gmlp.sT6 234
MI P. Mixer
Yes
16116
234 234
LN
Gal.1
resmlp_24 224
MLP Mixer
No
16s16
224 x 224
Affine transform
GeLU
resmip_ big 22
MLP Mixer
Yes
8x8
224 221
Alline transform
GaLUU
sWin 53, Bay 224
(TEM)
Ves
464
224
LN
GaLU
elficientnet b2
NAS (CNN)
Yo5
362
250 x 256
BN
SILU
1bxxts3.g
NAS (CNN)
Yes
322
240
BN
HardSwish Rel U + SSEA Al
haloregnetz_h J.
NAS (CNN)
Yes
362
224 x 224
BN
mixnet xl
NAS (CNN)
Yes
362
224 2334
BN
ReLU a SILU
ni rognei bl
NAS (CNN)
Ves
362
256 x 156
Norm free
ReLU + SIDU
regneix_320 320
NAS (CNN)
Yes
362
224 2241
BN
ReLJU
regnety 1832
NAS (CNN)
Yes
362
224 x 234
BN
Rel.A
rexnet.150 150
NAS (CNN)
Yes
362
234 x 234
BN
KeI.U à SILU Rel.I1
coat life. small
Hybrid
Yes
464
224
LN
GaLU
pit
Hybrid
Yes
1477
224 x 2211
LN
GaLU
pepyt base
Hybrid
Yes
464
224 224
LN
GSLLU
twins SVL small
Hybrid
Ves
454
224 x 221
LN
G2LU
vislomen small
Hybri
Yes
7x2
224 224
BN
G&LU + RaLU
vit_small _26_322_224
Hybrid
No
742
224 x
Al à Rel.U.
Xeit. medium. 24 plo, 224 scit Layy 12 p5 224
Hybrid
No
32374
224 x 234
LN i BN
GeLU
Hybrd
No
352/3627321
x 224
LN+
GaLU
17

---

Under review as a conference paper at ICLR 2023
Table B.3: Description of features of 69 models. "Pooling (stem)" and "Pooling (final)" denote "Pooling at the stem" and "Pooling for final feature", respectively. "SA", "CW", and "DW" means "Self-attention", "Channel- wise", and "Depth-wise", respectively.
Locasion CW attention
Model nome
Pooling (stem)
Pooling (final)
SA
CW attentices
DW conv
Group cone
botner26e 256
Yes
GAP
Yes (BeT)
No
No
No
convmixxr__555
No
GAP
No
No
Yes
No
comment tiny
No
GAP
No
No
Yes
No
No
No
espresnetSt
Yes
GAP
No
No
No
No
cspresnext50
Yes
CAP
No
No
No
Yos
0la10222
No
GAP
No
No
No
Yes
Upn107
Yes
GAF
No
No
No
Yes
bos 256
Yes
GAP
Yes (Boi)
Yes (PCA)
Middle
Yes
cca.halonest225s
Yes
GAP
Yes (Halo)
Yes (BCA)
Midale
No
Yes
eca.jifner J0
No
GAP
No
Yes (ECA)
End
No
Yes
Eca.resner.3t
GAP
No
Yes (ECA)
Middle
No
ese vowner33h
No
GAP
No
Yes (ESE)
End
No
No
gereemi1501
No
GAP
No
Yos (GCA)
Mikkle
Yes
No
getesnevtSOn
GAP
No
You (GCA)
MiMile
Yes
gluen xle
Yes
CAP
No
No
No
No
plune sception65
No
No
Shlneett 256
No
GAP
Yes [aalo, Bol)
No
No
hilonet>lit
Yes
GAP
Yes (Hale)
No
No
No
hmei,wll
CAP
No
No
No
lambdi_jesnethiin
Yes
GAP
Yes (Lambday
No
No
No
Janbalchotne500ss25
CAP
Yes LLabbda Halo, Bel)
No
No
No
Jbsnett)
No
No
No
No
GAP
No
Yox (SF)
End
No
Yes
poolformer m44
GAP
No
No
No
resmest50i
Yos
GAP
No
Yos
Yes
resner 101
Yes
GAP
No
No
No
No
50
No
No
resner50) gn
Yes
GAP
No
No
No
No
rexaciblur50
Yes
CAP
No
No
No
No
(osaetr2 101
No
No
No
2_30
Yes
CAP
No
No
No
No
resnerv2, 300
Yes
GAP
No
No
No
resnext50.32vld
Yes
GAP
No
No
No
Yes
sebocnet2... 256
No
GAP
Yes (HoT)
Yes (SE)
Middle
No
No
3chalonet31ss
Yes (Halo)
Yos
Midklee
seresnet50
Yes
GAP
No
Yes (SF)
End
No
No
50 37:4d
Yos
GAP
No
Yex (SE)
End
No
Yes
wide resnet50
No
No
No
conviLhase
Nu
CLS token
No
No
No
No
base 210
CLS
No
No
No
deii. Woiihhos-puchh 6
No
CLS token
No
No
No
doif, small patchlo 224
No
CLS token
No
No
No
No
15. Jiny
No
No
No
224
No
CLS token
No
No
Yes
No
swin,ing www.huttthhhhaee wwndoo.wnwww 7,204
No
GAP
No
No
No
No
1MI. patchi.atch6
CLS token
No
No
No
vi.Jswwc.pucck322222
No
CLS token
No
No
No
No
wil.small.pu pancll.paccho..cm 234
CLSS token
No
No
No
16lpp.16.2224
No
GAP
No
No
No
resinpp 21 224
No
GAP
No
No
No
No
resmlp beg 244
No
No
sWin s5 dny 224
No
GAP
No
No
No
ElligienneeJF
No
CAP
No
Yes (SI))
Mikile
Yes
No
Ibueev3 g
GAP
No
Yes (SL)
Middle
No
haluregnetz.bb hb
No
GAP
Yes (Ialo)
Yes (SE)
Middle
No
Yes
mixncr_xi
GAP
No
Yes (SE)
Middle
No
nf_rogret_hh
No
GAP
No
Yos (SF)
MiMile
Yes
Yes
regnatx.32 320
No
GAP
No
No
No
Yes
segnety
No
Yos
Middle
texner 150
No
GAP
No
Yes (SE)
Middle
Yes
No
No
CLSS token
No
No
Yes
No
pit
CLS token
No
No
No
[wins pepet buse
No
GAP
No
No
Yes
No
small
CAP
No
No
No
vislormer smull
No
CAP
No
No
Yos
Vit small r20 <22 224
Yes
CLS token
No
No
No
No
CLSS
No
No
No
Xcit. p8.234
No
CLS token
No
No
Ye5
No
18

---

Under review as a conference paper at ICLR 2023
cospri Dier
1e Uix 204
and raints
tain pan
20 oo
Lanoco
e o Da Sene
De Seeck
Figure C.I: Spectral features of 69 architectures. The K-th largest eigenvectors of the Laplacian matrix of the pairwise similarity graph 69 architectures are shown (K this figure). Rows and columns are sorted by the clustering index Tab. 2. We denote the model name in iimm for each row and column.
We visualize the pairwise distances of the spectral features (i.e., K Kllargest eigenvectors L) of 69 architectures in Fig. C. Note that rows and columns of Fig. C.I are sorted by the clustering results. Fig. C.I shows block diagonal patterns, i.e., in-cluster similarities are large while between-cluster similarities are small.
D PAIRWISE SIMILARITIES
Fig. D. shows that the similarities between different architectures (Fig. D.la and the off-diagonal elements of Fig. D.1b) are smaller than the similarities within the same architecture with different initializations or parameerrasssss.
Fig. D.2 indicates the pairwise similarity among 69 models. We can observe a weak block pattern around clusters, as also revealed in Fig. C.I.
19

---

Under review as a conference paper at ICLR 2023
Table D.I: Similarity within the same architecture. We compare the average similarity within the same network architecture but trained with different procedures, e.g., different random initializations, different opti- mization hyper-parameters, ResNet-S5 50 and ViT-&, where we use the DeiT training setting (Touvron et al., 2021) for ViT training. "All" denotes the average similarity of 69 neural architectures in Tab. 2.
different training regimes. We report similarities two neural architectures,
Architecture Diversity by
Similarity
Initialization
4.23
ResNet-
Hyper-paramett
4.05
Training regime
3.27
Initialization
4.21
ViT-8
Hyper-pand parameter
4.22
Training regime
3.44
All
2.73
canvit basa
col small patenze 224
comueed Shy
4.0
4.0
sun Wergeeggo mestrol
UA
3.0
WCC
30
reseet)
B600
rigndyy C32
ofkintrd 24
2.0
1181
2.0
commorr 1136 25
postummr
NO
Bas
ResNet-50
(a) Similarity between different networks.
(b) Similarity within ResNet-50 and VIT-S with different optimization settings.
Figure D.1: Similarity scores between different architectures vs. within the same architecture. (a) Sub- sampled pairwise similarity and 4 ViT-S models with different optimization settings. Init, LR. WD are randomly chosen from models trained with different settings of initialization, learning rate, and weight decay in Tab. D.1, respectively.
of 69 neural architectures. (b) Similarity between 4 ResNet-50 models
E TRAINING SETTING DETAILS FOR SEC. 3.2
We train 21 ResNet-50 models and 16 Vi V-T-S from seratch individually by initializing each net- work with different random seeds. We further train 28 ResNet- 50 models by randomly choosing learning rate (x0.1, x0.2, x0.5, x1, x2, and x5 where the base learning rate 0.1), weight de- cay (x0.1, x0.2, x0.5, x1, x2, and x5 where the base weight decay is le-4), and learning rate scheduler (step decay or cosine decay). Similarly, we train ViT-S models by randomly choosing learning rate (x0.2, x0.4, and x where the base learning rate is 5e-4) and weight decay (x0.2, x0.4, and where the base weight decay is 0.05). Note that the DeiT training unstable when we larger learning rate weight decay than the base values. Finally, collect 22 Re sesNet55 models with different training regimes: model with standard training by PyTorch (Paszke et al., 2019); 4 models trained by GluonCV (Guo et al., 2020) semi-supervised model and semi-weakly supervised model on billion-scale unlabeled images by Yalniz et al. (2019)3: 5 models trained by different augmentation methods (Cutout VreVs & Taylor, 2017), Mixup (Zhang al., 2018), manifold Mixup (Verma et al., 2019), CutMix (Yun et al., 2019), and feature CutMix 10 opti- mized ResNet models by (Wightman et al., 2021)². We also collect ViT-S models with different training regimes, including the original ViT training setup (Dosovitskiy et al., 2021) a stronger
2 "gluon igluun_resnett0/ vl gluon_resnet50 resnet50. vlc,, glwoo. resnet50 slaeett0.v.dd.. vld, and gluon resnet50. vls from timm library. ssl resnet50 and swsl resnet50 from timn library.
Wee use official weights provided by https:://// //gi wtw.ttt..cmcccc..ec elmm/elon ovaa ovaai/CutMt /CttMi x- P-rrorch,
'We use the official weights provided by https: / /gg thub. com www..ccmm/wwighteeee hwtp://wwggtta/////mmm wwworch-image-aee w/wesaaeeeeeeeeeeeeeeeeeeeeeeeee /eaaae//////eeeeeee/////////// --w.--wwrbihhhhhheeee
20

---

Under review as a conference paper at ICLR 2023
vimmm Ricee 234
in the I7
10 dede
1or 2o Aurtto
col Heeda
Figure D.2: Pairwise similarity among 69 models. Rows and columns are sorted by the clustering index in
Tab. 2. (n, n) component of pairwise similarity is close to 4.6 (log 100) because the attack success rate is almost 100% when attacked model and model used generate the
data augmentation setup in the Deit paper (Touvron et al., 2021)-36, the training setup with distil- lation (Touvron et al., 2-2 an improved DeiT training setup (Touvron et al., 2022b)-36 and self-supervised training fashions by MoCo v3 (Chen et al., 2021c)², MAE (He al., 2022) and BYOL (Grill et al., 2020) We do not use adversarially-trrine networks because the adversarial training usually drops the standard accuracy significantly (Tsipras al., 2019).
'diit small patch16 224,
vit small wwt.lll.cch1121 224,
deit. Saciilleddpptchh6.. 224,
and
deit3 small. amall_patch16 6 6.224 224
from timm library.
We train the ViT-S model by following https: http:///ggg.aaacccccccccceeeeeee
"W train the Vir-SS model by
We train the vi T-SS model by following httpp::///iittub.cco/////
21

---

Under review as a conference paper at ICLR 2023
Distillation Accuracy (%)
76,00
75.75
75.50
75.25
75.00
74.75
74.50
74.00
79.5
80.0 80.3
81.5 Teacher Accuracy (%)
81.0
82.0 82.5
Figure F.I: Teacher network accuracy and distillation performance. The trend line and its 90% confidence interval shown the blue line and gray band. There is significant correlation between leacher accuracy and distillation performance.
F KNOWLEDGE DISTILLATION
F.1 TRAINING SETTING DETAILS FOR SEC. 4.2
We train ViT-Ti student models with different teacher models using hard distillation strat- egy. We follow the distillation training setting of DeiT official repo"o, only changing the teacher model. Note that we resize the input images to the input size the teacher model requires the input sizes of student and teacher models differ. If a teacher model needs different in- put resolution, such as 240 x 240 and 256 x 256, we resize the input image for distilling it. Because DeiT-Ti has low classification accuracy compared to teacher models, the simi- larity score is calculated between DeiT-S and 25 models. The 25 teacher models are as fol- lows: BoTNet-26, CoaT-Iite Small, ConVir-E ConvNext- CrossViT-B, CSPDarkNet-53, CSPResNextt-50, DLA-X- DPN-107, EfficientNet Haup- Xxt. WwwN
FBNetV3-G, GC CC-ResNet-50, L0, PiT-s,, RegNet ResMLP-22 24, ResNet-50, SEHaloNet33, Swin-T, TWT-S, Vis and XCiT-T12.
F.2 TEACHER ACCURACY AND DISTILLATION PERFORMANCE
The similarity between teacher and student networks may not be the only factor contributing to distillation. For example, a stronger teacher can lead better distillation performance (Yun al., 2021). In Fig. F.I. we observe that if the range of the teacher accuracy is not significantly large enough (e.g., between 79.5 and 82.5), then the correlation between teacher network accuracy and distillation performance is not significant; 0.049 Pearson correlation coefficient with 0.82 p-value. In this case, we can conclude that the teacher and student networks similarity contributes more to the distillation performance than the teacher performance.
G LIMTATIONS AND DISCUSSIONS
Robustness to the choice of adversarial attack methods. Our similarity score is based on at- tack transferability using the representative gradient-based attack method, PGD attack (Madry et al. (2018)). To explore the effect of the choice of attack methods, we compute similar- ity scores on two different gradient-basee attack methods; Autoattack (Croce Hein (2020)), which is the ofte-offthheart method attack method based the ensemble of four diverse at- tacks, and Patchfool attack (Fu et al. (2022)), which is specially designed for vision transform- We sample 8 models among 69 models for testing the effect of Autoattack the similar- ity score: ViT-S, CoaT-Lite te Small, ResNet-101. LamHa oBotNet.50 ReXNet-1 11NNet1150 10, NFNet-LO, Swin-T and Twins-pepvt. Figure G.la shows the high correlation between simi- larity scores using PGD and Autoattack; it shows correlation coefficient of 0.98 with p-value of
"https://// thub. com/ www.:/owwwwwccc.cccccccoco facebookreeearceerccoooooo
22

---

Under review as a conference paper at ICLR 2023
3.25
Similarity (Autoattack)
Similariyy (Patchfool)
3.00
3.0
2.8
2.50
2.6
2.25
2.4
2.2
1.75
2.0
2.0
2.2
2.4
2.6
2.8
3.0
3.2
3.4
2.2 2.4 2.6 2.8 3.0 3.2
3.4 3.6
Similarity (PGD)
Similarity (PGD)
(a) PGD vs. Autoattack
(b) PGD vs. Patchfool
Figure G. I: Effect different adversarial attack methods to similarity score. The trend line and its 90% confidence interval are shown in the blue line and gray band. We show the relationship between our similarity scores using PGD and other attacks, (a) Autoattack and (b) Patchfool attack.
1.43 x 10-18, For testing the Patchfool attack, we only generate adversarial perturbations on ViT-S and get attack transferability to all other models (68 models) because Patchfool targets transformer- based models. In the results, Patchfool also shows a high correlation compared to the PGD attack (Pearson correlation coefficient 0.91 with p-value 3.62 10 -2)). The results show that our similarity score is robust to the choice of adversarial attack methods if the attack is strong enough. Note that PatchFool needs a heavy modification on the model code to extract attention layer outputs manually. On the other hand, PGD and AutoAttack are model-agnostic and need no modification on the model code. Therefore, if the PatchFool attack and the PGD attack show almost similar similarity rankings, we would like to recommend using PGD because it is easier to use for any architecture.
Similarity scores for adversarially trained networks. Because our method is based on an adver-
sarial attack, it could be affected by an adversarial attack-aware learning method. We first note that our analysis invariant to the adversarial training; as shown by Tsipras et al. (2019), adversarial robustness and accuracy are in a trade-off, and there is no ImageNet model with accuracy aligned with our target models yet. Also, our model similarity is based on the difference in the change of the model predictions. As shown by Tsipaas et al. (2019) and Ilyas et al. (2019), adversarial train- ing will lead to a different decision boundary. Therefore, we think that although our score can be affected by adversarial training, our score is still valid for adversarially trained models because adversarially trained model is actually dissimilar to other models in terms of the decision boundary and representation learning.
An efficient approximation for a new model. We can use our toolbox for designing a new model; can measure the similarity between a novel network and existing N architectures; novel net- work can be assigned to clusters (Tab. 2) to understand how it works. However, it requires generating adversarial samples for all N+1 models (e.g., 70 in our case), which is computationally inefficient. Instead, we propose the approximation of equation by omitting to compute the accuracy of the novel network on the adversarial samples of the existing networks. It will break the symmetricity of our score, but we found that the approximated score and the original score have high similarity (0.82 Pearson correlation coefficient with almost p-value) as shown in Figure G.2).
As an example, we tested Cluon-ResNeXt: ture with ResNeXt - (Xie et al., 2017) in 69 models, and the distilled version of DeiT- T model we observed in Tab. D. and is assigned to the same cluster with ResNext-50,
(Guo et al., 2020), which has the same architec-
(Touvron et al., 202 1). Because the difference is not significant Fig. 4, we expect that CluoonResNeete and distilled Deir-S is assigned to the same cluster with DeiT-S. We observe that, as pected, each network is assigned to the desired cluster. Therefore, we suggest using our efficient approximation for analyzing novel network with our analysis toolbox.
23

---

Under review as a conference paper at ICLR 2023
Similarity (One-side)
4.0-
3.5
3.0
2.5
2.0
1.5
2.0
2.5
3.0
3.5
4.0
Similarity (Two-side)
Figure G.2: Similarity scores using two-side attack transfcrability vs. one-side attack transfcrability. The trend line and its 90% confidence interval are shown in the blue line and gray band. We show the relationship between similarity scores using two-side and one-side attack transferability.
Other possible applications. We can use our similarity to develop a novel neural architecture or training technique by measuring how the proposed method can learn different representations compared existing methods. Our similarity also can be used for analyzing the hyper-parament sensitivity. Instead of reporting the performance changes by different hyper-parameters, one can report the average similarity of the trained models with different hyper-parameters. the average similarity is significantly low, then selection. We expect this scenario is especially useful for the task, sensitive to the arper-prrameter selection (Choe et al., 2022; Gulrajani & Lopez-Paz, 2021). Finally, our similarity can be used for the model selection criteria for selecting diverse models, e.g., constructing a new dataset with diverse pre-trained models for annotation candidates (Kuznetsova et al., 2020; Chun al., 2022).
argue that the method is sensitive to hyper-panne parameter
H ADVERSARIAL ATTACK TRANSFERABILITY AND DECISION BOUNDARY
Adversarial attack transferability is highly related the decision boundaries of models. More specif- ically, measuring adversarial attack transferability is a good approximation of measuring the differ- ence between model decision boundaries Karimi & Tang (2020). Comparing decision boundaries of two models will provide a high-level understanding of how two models behave differently for the changes of input. Unfortunately, an exact decision boundary is not achievable and an expensive and inexact sampling approximation is required (Somepalli et al., 2022). For example, Somepalli al. (2022) uses compling-based approximation with 2,500 samples. More specifically, Somepalli et al. (2022) generates 2,500 perturbed samples (expanded by coordinate defined by randomly two models (e.g..
chosen triplet of inputs) for a given input and measures the ratio of "agreement" if both two models predict 1 1,250 perturbed samples to the same label, then the similarity of deci- sion boundaries for the given input is 1250/2500 = 0.5). However, it needs an expensive sampling approximation procedure and the approximation bound of the sampling method is not guaranteed. On the other hand, we can analyze the difference between two decision boundaries by generating an adversarial sample for each decision boundary. If two models have similar decision boundaries, then the ad adversarial samples will be transferred (i.e.. will have high adversarial attack transferability); if two models have dissimilar decision boundaries, then the adversarial samples will not transferred. Compared to the sampling- based approximation, adversarial attack transferability is more efficient (we used 50 iterations for the attack, while sampling needs 2,500 samples) and less stochastic than sampling-basse approximation. Also, we employed an adversarial attack based on gradient descent where the convergence of gradient descent is theoretically guaranteed, while sampling approxima- tion has no guarantee on its approximation error. Therefore, adversarial attack transferability can work as a good proxy measure of the difference of model decision boundaries.
24


Reviews:
[{"review_id": "ZgKYfJCGFbk", "reviewer": "Reviewer_UduE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a quantitative similarity score between different neural architectures based on the adversarial attack transferability. This smiliarity helps to understand the component-level architecture design, and leads to better understanding of the relationship between model similarity of model ensemble performance and model distillation performance. Several interesting observations are obtained based on the proposed similarity function.", "summary_of_the_review": "This paper is interesting and leads to several observations that may be beneficial to architecture design, model ensemble and knowledge distillation. More analysis and insights for the proposed similarity function should be provided. Several experiments related to Figure 6 are encouraged to be conducted.", "strength_and_weaknesses": "Strength:\n1. This paper proposes a quantitative similarity score between different neural architectures based on the adversarial attack transferability. \n2. The proposed smiliarity helps to understand the component-level architecture design, and leads to better understanding of the relationship between model similarity of model ensemble performance and model distillation performance. \n3. Several interesting observations are obtained based on the proposed similarity function.\n4. Extensive experiments and analysis are conducted, to lead to better understanding.\n\nWeaknesses:\n1. Could the authors provide more motivation and insights for why  adversarial attack transferability can serve as the similarity function?\n2. In Figure 6 (c), most teacher models are clustered together with quite similar slimilarity. More teachers with large variety of similarity should be provided to better justify the conclusion that using a more dissimilar teacher leads to better distillation performance.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "TPuRIyx-FK7", "reviewer": "Reviewer_yYKW", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposed a new similarity function to quantify the similarity of the DNNs in image classification tasks based on transferability of adversarial examples. For a pair of similar DNNs, authors investigate which types of DNN components contribute most to the similarity and their implications to practical scenarios. Using a collection of pre-trained image classifiers on ImageNet, authors conclude that the base architecture component (i.e., convolution-based CNN vs. self-attention-based Transformer) plays a most important role in measuring similarity and cluster DNNs according to the 13 categories of DNN components.       ", "summary_of_the_review": "This paper tackles an important problem of measuring DNN similarity in terms of image classification task. The similarity score is well motivated by the input transferability and practical application in model diversity, either increase diversity in generating ensembles or decrease diversity in selecting teacher and student in knowledge distillation, are also promising. However, the major technical concern is the similarity score may be more suitable for measure CNN similarities. For Transformer similarities, a different mechanism to generate adversarial are required.      ", "strength_and_weaknesses": "The problem of measuring DNN similarity sees a broad range of applications in model diversity, such as choosing diverse models for ensemble, and similar models for knowledge distillation. The categorization of DNN into 13 different components provides a useful guideline for future studies. \n\nA major weakness to the reviewer lies in the rationale of defining the similarity score, i.e., transferability of adversarial examples generated by PGD. The underlying assumption states if the adversarial examples are transferrable between a pair of DNNs, then the similarity is high. It is more reasonable if the similarity is measured between a pair of CNNs but not so between a pair of Transformers nor between a pair of CNN and Transformer. This is because the adversarial examples generated by PGD on CNN may not be transferrable to Transformer, evident by several recent publications on transferability of adversarial examples (e.g., Mahmood et al, ICCV-2021, Fu et al, ICLR-2022). In fact, the algorithms to generate adversarial examples for Transformer can never be the same as those for CNNs. This is because Transformer operates on image patches to extract global features via self-attention whereas CNN operates on pixels to extract local features via convolution feature maps. As such, the reviewer believes the similarity score is limited to CNN on image classification tasks.               \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RzvwBkXnuUh", "reviewer": "Reviewer_kjGm", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes using the attack transferability of adversarial examples to measure neural architecture similarity. The authors analyze the design choices leading to better model diversity measured by the proposed metric. The paper also demonstrates the application of the proposed similarity measure in ensemble learning and knowledge distillation.", "summary_of_the_review": "This paper is well-written and proposes a simple and practical similarity measure for neural architecture. However, this paper lacks the discussion and comparison with the model diversity metric proposed in previous works. The paper should show and explain why the attack transferability metric is a better measurement of model similarity.", "strength_and_weaknesses": "__Strength__\n\n__[S1] Novel model similarity measure.__ Using attack transferability of adversarial examples to measure model similarity is relatively novel. As a data-oriented approach, the similarity measure can be easily applied to different model architectures.\n\n__[S2] The analysis reveals some interesting findings.__ I appreciate the comprehensive analysis of the model diversity. The analysis identifies some factors that lead to more diverse models measured by attack transferability. The findings may be of interest to some data partitioners in the community.\n\n__[S3]__ Overall, the paper is easy to follow.\n\n__Weakness__\n\n__[W1] Some findings are mostly expected or have been studied before.__\n- Intuitively, an adversarial example generated for a specific model is more effective on the same base architecture. For example, using the adversarial examples generated from ResNet to attack VGG is more difficult than attacking other ResNet models. So, it is somewhat expected that the proposed method considers the differences in model architecture as the most diverse change.\n- The observation that using more diverse models in ensemble learning leads to better performance has been explored in many previous works.\n- As suggested by the paper, the observation that using a similar model architecture for student and teacher models results in better performance is studied in previous KD works.\n\n__[W2] Limitation of using attack transferability as similarity measures.__ As using the adversarial attack transferability naturally gives a higher similarity score for the same model architecture, is it possible that the proposed similarity measures give an incorrect similarity score in terms of the predictions:\n- Given two models with the same base architecture, the models make very different predictions; however, the adversarial examples transfer well across the two models.\n- Given two models with different base architectures, the models make very similar predictions; however, the adversarial examples do not transfer well across the two models.\n\n__[W3] Does not discuss and compare with other diversity measures.__ Although the proposed similarity measures can improve ensemble learning, the paper does not compare the approach with previously suggested model diversity metrics in ensemble learning literature, e.g. [1]. It is difficult to judge whether the proposed method is a better measurement in selecting the models in ensemble learning.\n\n__Other comments__\n\n__[O1]__ If the set of candidate models contains defended models that are trained to be robust to adversarial samples, does the proposed similarity measure also work?\n\n__[O2]__ The paper uses PGD as the adversary. Is the method sensitive to the choice of attacks?\n\n__Ref__\n\n[1] Kuncheva, Ludmila & Whitaker, Chris. (2003). Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning. 51. 181-207. 10.1023/A:1022859003006.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RriXwBzNI1", "reviewer": "Reviewer_ynKk", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a method to compute pairwise similarities between two architectures. This is done by evaluating the change in prediction of first model caused by the adversarial prediction w.r.t. second model and viceversa, with the intuition being that if two models are similar then their adversarial perturbations would have similar effect. Paper then uses the proposed similarity measure to analyze existing model architecture choices and effect on ensembling performance.", "summary_of_the_review": "Overall, I feel that the technical contribution is fairly limited. However, the paper presents an interesting an fairly detailed study that provides interesting insights and would be beneficial to a large audience. I vote for accept.", "strength_and_weaknesses": "Strengths\n- Well written and easy to understand paper\n- A detailed analysis of the similarity measure w.r.t. a fairly large (69) menu of models with different architectural components.\n- Interesting insights into effect of component choices on similarity and ensembling performance.\n\nWeaknesses\n- Proposed method is computationally expensive and the approximation proposed in 5 is rather ad-hoc and not well evaluated. This limits the applicability of the method.\n\nMinor nitpick\n- What do authors mean by \"destructive success of DNNs\"? The phrase seems to refer to negative effects of DNN success. Perhaps, rephrase is that is not intended, otherwise clarify. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "F_crudxougD", "reviewer": "Reviewer_Bk5e", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper present a DNN model similarity measure using input gradient transferability. The basic hypothesis is that if two neural networks are similar, adversarial attack transferability will be high. Additionally two topics are investigated: (1) Which network component contributes to the model diversity? (2) impact of model diversity in practice.", "summary_of_the_review": "This paper is lack of technical contribution and novelty.", "strength_and_weaknesses": "Strength\n\n+ Similarity of DNN architecture is an important research topic.\n\n\nWeaknesses\n\n- What does \"If A and B are similar and assume an optimal adversary, then accA→B will be almost zero\" mean?\n\n- There are numerous syntax errors. e.g., please revise \"We expect our analysis tool helps a high-level understanding of differences between various neural architectures as\nwell as practical guidance when using multiple architectures.\"\n\n- What is \"destructive success\"?\n\n- Please elaborate \"distinct properties\".\n\n- How is similarity score defined?\n\n- What is input gradient?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject