Game-Theoretic Understanding of Misclassification
Authors: Kosuke Sumiyasu
Kazuhiko Kawamoto
Kera Hiroshi
Venue: Submitted to ICLR 2023 (2023)

Abstract: This paper analyzes various types of image misclassification from a game-theoretic view. Particularly, we consider the misclassification of clean, adversarial, and corrupted images and characterize it through the distribution of multi-order interactions. We discover that the distribution of multi-order interactions varies across the types of misclassification. For example, misclassified adversarial images have a higher strength of high-order interactions than correctly classified clean images, which indicates that adversarial perturbations create spurious features that arise from complex cooperation between pixels. By contrast, misclassified corrupted images have a lower strength of low-order interactions than correctly classified clean images, which indicates that corruptions break the local cooperation between pixels. We also provide the first analysis of Vision Transformers using interactions. We found that Vision Transformers show a different tendency in the distribution of interactions from that in CNNs, and this implies that they exploit the features that CNNs do not use for the prediction. Our study demonstrates that the recent game-theoretic analysis of deep learning models can be broadened to analyze various malfunctions of deep learning models including Vision Transformers by using the distribution, order, and sign of interactions. 

Under review as a conference paper at ICLR 2023
GAME-THEORETIC UNDERSTANDING OF MISCLASSI- FICATION
Anonymous authors Paper under double-blind review
ABSTRACT
This paper analyzes various types of image misclassification from game- theoretic view. Particularly, we consider the misclassification of clean, adver- sarial, and corrupted images and characterize it through the distribution of multi- order interactions. We discover that the distribution of multi-order interactions varies across the types of misclassification. For example, misclassified adversar- ial images have higher strength of high-order interactions than correctly classi- fied clean images, which indicates that adversarial perturbations create spurious features that arise from complex cooperation between pixels. By contrast, mis- classified corrupted images have a lower strength of low-order interactions than correctly classified clean images, which indicates that corruptions break the local cooperation between pixels. We also provide the first analysis of Vision Trans- formers using interactions. We found that Vision Transformers show a different tendency in the distribution of interactions from that in CNNs, and this implies that they exploit the features that CNNs do not use for the prediction. Our study demonstrates that the recent came-theoretic analysis of deep learning models can be broadened analyze various malfunctions of deep learning models including Vision Transformers by using the distribution, order, and sign of interactions.
INTRODUCTION
Deep learning models misclassify images for various reasons. They fail to classify some clean images dataset, and they also misclassify because of adversarial perturbations and common cor- ruption. Understanding the causes of misclassifications in deep learning models is vital for their safe applications in society. Several recent studies provided new directions for understanding deep learn- ing models from gheeethe viewpoints (Cheng et al., 2021; I; Deng et al., 2022; Ren et al., 2021; Wang et al., 2021: Zhang et al., 2021). For example, adversarial images (Goodfellow et al., 2015; Szegedy et al., 2014)-the images that are slightly but maliciously perturbed to fool deep learning models-wert were characterized using the interaction (Ren et al., 2021; Wang et al., 2021), which is originally used 1999). In image classification, the average interaction of an image is a measure of the average change in the confidence score (i.e., the softmax value of logits of the true class) by the coopera- tion of various pairs of pixels.
measure of the synergy of two players in game theory (Grabisch & Roubens,
== Ex
Here, [i,,j) denotes the interaction of the i- and j-th pixels, which is, roughly speaking, defined by the difference in their contributions to the confidence score:
[(i.j) j) g(i - g(t) + const.,
where (lk relates to the contribution of the k-th pixel to the confidence score. The formal definition of [(.,) () will be provided later in this paper.
When j( for pixel pair, it indicates that the two pixels contribute the model prediction almost independently. By contrast, a large interaction indicates that the combination of pixels (e.g., edges) contributes to the model prediction in synergy. An interaction can be presented an average
of interactions of different orders, 1 (i.,j) - Σ. 5,=(),, where (*)) and denote the interaction of
order s and the number of the pixels, respectively. The decomposition into [I()), gives more de- tailed view of the cooperation of pixels; low-order interactions measure simple cooperation between

---

Under review as a conference paper at ICLR 2023
pixels, whereas high-order interactions measure relatively global and complex concepts. In other words, low- and high-order interactions correspond to different categories of features. Cheng et al. (2021) investigated the link between the order of interactions and the image features. They showed that in general, low-order interactions reflect local shapes and textures, whereas high-order interac- tions reflect global shapes and textures that frequently appear in training samples. Ren et al. (2021) showed that adversarial perturbations affect high-order interactions and adversarially trained mod- els are robust to perturbations to the features related to high-order interactions. Zhang et al. (2021) found that the dropout regularizes the low-order interactions. These results suggest that interactions can characterize how deep learning models view images; thus, one can obtain a deeper understand- ing of the cause of the model predictions through interactions.
In this study, we investigate one of the most fundamental issues of deep learning models, misclas- sification, through the lens of interactions. We examine various types of misclassifications; we consider misclassification of clean, adversarial, and corrupted images, and characterize them by the distribution, order, and sign of the interactions. In the experiments, we contrasted the distribution of interactions of misclassified images to that of successfully classified clean images, thereby revealing which types of features sults show that these three types of misclassifications have a distinct tendency in interactions, which follows:
more exploited to make prediction for each set of images. The
indicates that each of them arises from different causes. The summarized results
Misclassification of clean images. The distributions of interactions did not present a large dif- ference between misclassified clean images and those successfully classified. This result indicates that the misclassification is not triggered by distracting the model from the useful features, and the model relies on similar features in images regardless of the correctness of the prediction.
Misclassification of adversarial images. We observed a sharp increase in the strength of inter- actions in high order, indicating that for adversarial images, the model exploits more features that relate to interactions in these orders. Namely, the misclassification is triggered by the destruction of the model from the useful cooperation of pixels in low order to the spurious one in other orders.
Misclassificationn of corruption images. We observed that while the interactions moderately in- creased in high order, they also decrease in low- and middle-order interactions. This indicates that the model can no longer use the originally observed pixel cooperations because of the corruption and gain useless or even harmful pixel cooperations in high order make a prediction.
The abovementioned results were observed on convolutional neural networks (CNNs; He2). We investigated whether these results generalize to Vision Transformers, which recently shows better performance in image recognition tasks over CNNs. For DeiT-Ti (Touvron et al., 2021) and Swin- Transformer (Liu al., 2021), most results hold but with stronger contrast. Besides, for the misclassification of clean images, where CNNs showed no particular difference in the distribution of the interactions between correctly classified and misclassified images, the Vision Transformers showed striking difference. This suggests that the characteristics of the predictions are more clearly exhibited for Vision Transformers than CNNs, and thus, the analysis with interactions can be exploited even after the shift from CNNs to Vision Transformers.
We also conducted experiments on adversarial attacks and transferability as they have been discussed extensively in the literature (Croce & Hein, 2020: Dong et al., 2018; Ilyas et al., 2019; Wang al., 2021; Yang et al., 2021). We discovered that when images are adversarially perturbed, the distribu- tion of interactions shifts negative values. This reveals that adversarial perturbations break the features that the model exploits or even alter them to misleading ones. We also discovered that the adversarial transferability depends on the order of interaction; adversarial images with higher interactions in high order transfer better when adversarially perturbed using ResNet-18, whereas, interestingly, they transfer less when Swin-T is used. This contrastive tendency is analogous the recent observations that the adversarial images are more perturbed in the high-frequency domain with CNNs and in the low- www-requency domain with Vision Transformers (Kim & Lee, 2022).
The contributions of our study can be summarized as follows:
This study investigates various types of misclassifications from game-theoretie perspec- tive for the first time. In particular, we characterize the misclassificttinn of clean, adversar- ial, and corrupted images with the distribution, order, and sign of the interactions.
2

---

Under review as a conference paper at ICLR 2023
We discover that the three types of misclassifications have different tendencies in interac- tions, which indicates that each type of misclassification is triggered by different causes. Particularly, the dominant order of interactions suggests which category of features affects the prediction more, which different between the three types of misclassification.
We provide the first analysis of Vision Transformers by using interactions and found that the difference in distributions of interactions between misclassified and correctly classified images are clearer and also different from the case with CNNs. We also find that the im- ages that are more adversarially transferable have the opposite tendency in the interactions between Vision Transformers and CNNs.
RELATED WORK
Recent studies have shown that interactions give a new perspective to understanding the machine vision of deep learning models (Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021; Wang al., 2021: Zhang et al., 2021). For example, Wang et al. (2021) showed that the transferability of ad- versarial images has a negative correlation to the interactions and provided unified explanation of the effectiveness of various methods proposed to enhance the transferability. Ren et al. (2021) also characterized adversarial images using interactions. They showed that adversarial perturbations decrease the high-order interactions and the adversarially trained models are robust against such changes. The low-order and high-order interactions are considered to correspond different types of image features. This was investigated by (Cheng et al., 2021). Generally, low-order interactions reflect local shapes and textures that frequently appear in training samples, while high-order interac- tions reflect global shapes and textures in training samples. Furthermore, Deng et al. (2022) argued that humans gain more information than deep learning models when a given image is moderately masked, while it is the opposite when the image is masked much more or less. Interactions also relate to dropout regularization. Zhang et al. (2021) showed the similarity between the computation of interactions (particularly. variant of the Banzhaf values) and dropout regularization. In addition, they showed that the dropout regularizes the low-order interactions and argued that this is why leads to better generalization error when model equips it.
The abovementioned studies have considered what features in clean images are focused on by deep learning models and/or how the interactions changed by adversarial perturbations. By contrast, our study newly discusses and characterizes misclassification using interactions. Particularly, we focus misclassified images in noise-free, adversarially perturbed, and corrupted cases. Unlike the existing study, we also consider Vision Transformers, which are known to use low-frequency and shape-related features more than CNNs and thus expected to have different nature in interactions.
PRELIMINARIES
Shapley value. Shapley value was proposed in game theory measure the contribution of each player to the total reward that is obtained by multiple players working cooperatively (Shapley, 1953), thereby fairly distributing the reward to each player. Let N
(1, n,) be the of players. We denote the power set of N by N def {S S C N]. Let f 2 + IR be reward function. Then, the Shapley value of player with a context N, Φ(1 N), is defined as follows.
@i
N) def Σ
[S|1(n
[S] 1)! - f(S))
n!
SCNV0)
where |.| denotes the cardinality of set. As the definition demonstrates, ø(i N) considers the change the reward when player joins set of players S N {i) and averages over all S.
Interaction. Interaction measures the contribution made by the corporation of two players to the total reward. The following defines the interaction between and j-th players.
I(i,j) def (SS,, N') (Ø(i N (1) o(j N (())),
where two players é, are regarded as a single player S.y = {i.}} and N' = N NV ((,)) (note that N' 1). The first term ((S.j N') corresponds to the cooperative contribution of the
3

---

Under review as a conference paper at ICLR 2023
two players, whereas the second and the third terms, d(i N (i)) and Φ(j | {t),, correspond to the individual contributions of players and respectively, Next, introduce I (i, which is the interaction between players i.j of order s.
(i, def BSCMMOG.HIS (A
where Af,,, def = -
+ f(S). Note that the size of the context S is fixed to s in the calculation of (i,,)) A low-order (high-order) interaction measures the impact of the cooperation between two players when small (large) number of players join the game. The average of 7(*) (1. over 8 0, ,,..,.... satisfies [(,,j) n-1 Σn 7(0) (1.
4 INTERPRETATION OF INTERACTION
Here, we describe how the interaction is defined in the image domain and how it can be interpretd.
4.1 INTERACTION IN IMAGE DOMAIN
Image x considered as a set of players N, where each pixel viewed as player. The reward function f-) can be defined by any scalar function based on the output of deep learning model. In our study, we consider a standard multi-class classification and define the reward function as P(y]=) f(x) = log I- i-P() where and denote the input image and its class, respectively, and P | x:) denotes the confidence score of y given by the model with input x. Here, S C N corresponds to an image whose pixels not included in are all masked, and f($) is the confidence score of y given by the model with this masked image. This choice of the reward function is also adopted in (Deng al., 2022). With this reward function, Shapley value Φ(i N) represents the average contribution of the i-th pixel to the network output. Interaction I(i, represents cooperation between the i- and j- th pixels. The value (i, can be interpreted as the cooperative contribution of the i- and j-th pixels with context S to the confidence score. For example, A (f, S) 0 indicates that the two pixels contribute to the confidence score almost independently, whereas [i J)) >> 0 ()) < 0) indicates that the two pixels cooperatively increase (decrease) the confidence score. Zhang et al. (2021) visualized that the pixel pairs with strong interactions in facial images cluster around the face parts, which demonstrates that the pixels that contribute to the model predic- tion do not work independently but in synergy.
4.2 ORDER OF INTERACTION
Order s of the interactions (i.j) represents the size of context S, where S can be any subset of N [i, of size s. A multi-ordes interaction reflects the cooperation of pixels in certain complexity. When s is small, [(H) (i.j) reflects the cooperation between the i and j-th pixels without considering most of the other pixels, whereas when is large, the cooperation between the and j-th pixels measured by considering most other pixels. Cheng et al. (2021) investigated how the mulli-order interactions reflect the shape and texture of images, arguing that low-order interactions reflect local colors and shapes that frequently appear in the training samples while high-order interactions reflect the global textures and shapes that frequently appear in the training samples.
4.3 SIGN OF INTERACTION
The sign of the interactions also has important implications, although most existing studies did not consider this and only consider the strength (i.e., absolute value) of interactions. For example, (f 5) > 0 indicates that the cooperation between i- and j-th pixels (with context S) increases the confidence score, whereas Af S) < 0 indicates that the cooperation decreases it. Therefore, we can consider that misclassification occurs when ( j. S) are negative for most (i, ji,j, S). In fact, as we will show later in the experiment, the distribution of A (i. j. S) (for various (i. triplets) shifts to negative values after adversarial perturbations (cf. Fig. 4). Although the perturbations are very small, the shift is large, indicating that adversarial perturbations efficiently break or even deteriorate the cooperation between pixels.
4

---

Under review as a conference paper at ICLR 2023
0.2n
0.4n
0,2n
0-4n
06
0.2g
0.4n
0.6n
0.55
1,00
0,66
0.80
1.0n
0.10
0,5g
111
(a) strongly positive
(b) strongly negative
(c) almost 7ero
Figure/ Visualization of pixel pairs (marked in black) with (a) strongly positive, (b) strongly negative, and (c) almost zero values for (f, for different orders.
Figure shows the pixel pairs (marked in black) with interactions that are (a) large positive values, (b) large negative values, and (c) almost zero values. In Fig. I(a), most of the marked pixels cluster around the shark body, indicating that the object's shape and texture closely relate the shark label and increase the confidence score. In Fig. 1(b), the marked pixels also cluster around the shark body in the middle-order case, implying that those pixel pairs do not work cooperatively to increase the confidence score. We consider that this is because of an overfitting of the model to the non- generalizable features training samples, which has a negative effect on the prediction. In Fig. I(c) most pixels distribute over the (i.e., background), indicating that the cooperation between these pixels does not directly relate to the shark label.
5 INTERACTION ANALYSIS OF VARIOUS TYPES OF MISCLASSIFICATION
this section, we analyze three types of misclassification from the perspective of interactions. We consider three types of images that are commonly misclassified.
Clean images. Typical clean images are unperturbed and they present objects to be classified clearly. Nevertheless, deep learning models can misclassify them because the models do not learn the dlas-discriminative features that generalize perfectly. Namely, the misclassification of clean images occurs because of the overfitiing and underfitting training samples. Our analysis considers the misclassification of clean images since this is one of the most fundamental issues of image recognition tasks.
Adversarial images. Adversarial images are the images slightly but maliciously perturbed fool deep learning models. Although the perturbations are almost imperceptible to humans, they greatly impact the prediction by deep learning models. This shows the discrepancy between human and machine vision and has been studied extensively (Buckner, 2020; Ilyas et al., 2019; Tsipras et al., 2019). give further insight into the counter-intuitive misclassification that adversarial images cause, we include them in our analysis.
Corrupted images. In contrast to adversarial perturbations, common corruptions (e.g..fog. bright- ness, Gaussian noise, etc.) occur more commonly in standard settings. Interestingly, adversari- ally robust models are not robust against some common corruptions (e.g.. and brightness) and even perform worse than normally trained models (Yin al., 2019). In other words, the robustness against adversarial perturbations and common corruptions relates to different nature types in the classifiers, which motivated us to include corrupted images in our analysis. Note that obtaining a robust classifier against both of them-ho achieving general robustness
unresolved challenge that is addressed in recent studies (Laugros et al., 2019; 2020; Tan et al., 2022).
In particular, investigate misclassified ones among these images through the distribution, order, and sign of interactions, thereby elucidating what kind of features (in a high-level sense) the deep learning models exploit and make them give incorrect predictions.
5

---

Under review as a conference paper at ICLR 2023
Setup. We compare the distribution of interactions of misclassified clean, adversarial, and cor- rupted images with that of successfully classified clean images. Here, the distribution of interactions means the distribution of Af(i, S) for various (i, 8) and images'. As the order of interactions grasps the high-tevel category of image features, we compute the distributions of interactions for different orders. As in Fig. 2, we plot the second quartile (i.e., median) by using a solid line with shade covering the range between the first and third quartiles. In the plot, the result for the correctly classified images also superposed for comparison. Intuitively, the second quartile reflects bias or whether cooperation across pixels contributes to the prediction positively or negatively, and the area of the shade can be (roughly) considered as its strength. As in other studies, we conducted our experiments on ImageNet dataset (Deng et al., 2009). We sampled fifty images to consist of a set of clean images. We generated adversarial images by applying the -nntarggee Iterative- Fast Gradient Sign Method (I-FGSM) with = 16/255 to the clean images. For the corrupted images, we used ImageNet-C (Hendrycks & Dietterich, 2019) dataset. We sampled fifty images from the images under level-5 fog corruption. We used two CNNs and two Vision Transformers pretrained on ImageNet', including R8 (He et al., 2016), AlexNet (Krizhevsky et al., 2012), Swin-T (Liu et al., 2021) and DeiT-Ti (Touvron et al., 2021). More detail on the settings (e.g., image sampling, approximate computation of interactions) is given in Appendix B.I.
In the following, we focus R and Swin-T. See Appendix B.4 for more results.
5.1 MISCLASSIFICATION OF CLEAN IMAGES
Here, we consider the misclassification of clean images. Figure 2(left) shows the distribution of interactions for each order. As can be seen, ResNet-18 did not present a significant difference in the distributions between the images misclassified and those successfully classified. This result indicates not triggered by the inability of the deep learning models to obtain
that the misclassification beneficial cooperation, suggesting that deep learning models gain similar cooperation regardless of the correctness of the classification. As we will show shortly, this is not the case for adversarial and corrupted images. In contrast to 18, Swin-T has a different tendency for low-order interactions (particularly, at 0.0m): the interactions from correctly classified images are significantly biased toward positive values, whereas for the misclassified images, these are biased toward negative values. This indicates that for Swin-T, the successfully classified images have local cooperation between pixels that increase the confidence score, whereas the misclassified images do not have such cooperation, leading to the deterioration in the confidence score.
5.2 MISCLASSIFICATION OF ADVERSARIAL IMAGES
Next, consider the misclassification of adversarial images. For each model, the adversarial im- ages are generated by applying I-FGSM to clean images that are correctly classified. Figure 2(center) shows the distribution of interactions for misclassification of adversarial images. ResNet-18 has strongly negative interactions of high order around 0.8n-1.0n. This result indicates that adversarial perturbations in ResNet- 18 create spurious cooperation for the model by global shapes and textures. Namely, misclassification by adversarial perturbations is caused by the destruction of the model from the meaningful cooperation between pixels the spurious one, which is useless or harmful to make prediction. However, unlike 18, Swin-T only shows a slight shift of the interac- tions to negative values, and the large interquartile range covers positive and negative sides almost evenly from middle to high orders. This may explain the empirical robustness of Swin-T compared to ResNet-18 18 (cf. Table in Appendix A). It is worth noting that this difference between ResNet- 18 and Swin-T was observed because we take the sign into account, unlike most of the existing studies. Refer to Appendix B.2 for another example.
5.3 MISCLASSIFICATTIN OF CORRUPTED IMAGES
We now consider corrupted images. Yin et al. (2019) discovered that adversarially trained models are less robust than normally trained ones under the perturbations in a low-frequency domain (e.g.,
1 Although the interaction is defined as Eq. (3), in this paper, we also refer (f( as interaction for convenience when we are not interested in specific (t. (éJ.S)) S).
httpps: //coi http///oo.oo//o//////////o/ 10. .5281/ zenodc. 4431 043 (CNNs),
https://github com/ rwi wwt.gghphtttncneeeeeee imgiimaaammoooo model. 1 (Vision Transformers)

---

Under review as a conference paper at ICLR 2023
clean (correctly classified)
adversarial (misclassified)
(a)
ResNct-18
clean (misclossified)
corrupted (mnisclassified)
1.0
Afiij S)
0.5
0.5
0.0
0.0
-0.5
0.0
0.0n
0.2n
0.4n 0.6n
0.8n 1.0n
0.0n
0.2n
0,4n 0.6n
0.8n 1.0n
0.0n
0.2n
0.4n
0.6n Order
0.8n
1.0n
Order
Order
(b) Swin-T
0.2
0.2
0.2
Af
0.0
0.0
0.0
0.0n
0.2n
0.4n 0.6n 0.8n
1.0n
0.0n 0.2n 0.4n 0.6n 0.8n
1.0n
0.0n
0.2n
0.4n 0.6n
0.8n
1.0n
Order
Order
Order
The distribution of interactions (i.e., the distribution (f S) for various pixel pairs (i-j), context S, and images). The horizontal axis represents the order of interactions (n denotes the number pixels in an image). The solid lines represent the median of the interactions. The shades represent the interquartile range. The interactions for misclassified clean (left, blue), adversarial (center, red), and corrupted (right, purple) images are contrasted to those for correctly classified images (green), respectively. One observe that the three types of misclassified images give different tendencies the distributions of interactions for each order. The tendency also differs between (a) ResNet-18 and (b) Swin-T.
Figure 2:
fog corruption). Figure 2(right) visualizes the distribution interactions for misclassification of the fog-corrupted images. The strength of the interactions for adversarial images is lower at low order and higher at high order than in the case of clean images. This result shows that the corrupted images destroy the local cooperation of pixels that deep learning models were originally able exploit, and instead, they create spurious global cooperation that not useful for prediction. Note that compared the of adversarial images (Fig. 2: center), the strength of interactions for corrupted images is moderate, particularly in high order, which suggests that as widely known, corruption is less harmful than adversarial perturbations. Swin-T showed a similar but clearer contrast between the distributions of the two image sets than ResNet. In particular, the interactions have a large difference low orders 0.05m-0.Ln. Swin-T shows that local cooperation of fog images difficult obtain.
5.4 COMPARISON OF MODELS
In Sections 5.1-5.3, Swin-T exhibits different distributions interactions from ResNet-18. This indicates that Swin-T (Vision Transformer) exploits the different image features from those used by R8sNet-18 (CNN). To further investigate this, we compare Swin-T and ResNet-18 by using the distribution of interactions in the clean images that are classified successfully. The result is shown in Fig. ResNet-15 18 and Swin-T do not show a significant difference terms of the median: however, at order 0.On, the interquartile range includes negative and positive interactions for ResNet- 18 but for Swin-T, only includes positive ones. Thus, obtained features from cooperation are different for each model, and Swin-T takes advantage more of the local cooperation of pixels than ResNet-18 does. These results may partially explain the higher classification accuracy of Swin-T.
It is worth noting that in Sections 5.1-5.3, can see that for Swin-T, the distributions of interac- tions between correctly classified and misclassified images have a clearer difference than those for ResNet- 18. This suggests that the characteristies of the predictions can be more clearly exhibited in the case of the Vision Transformer than CNNs, which encourages the further study Vision Transformers based interactions.

---

Under review as a conference paper at ICLR 2023
4 (r. S)
Swin-T ResNot-18
200000
Swin-I ResNet-18
frequency
0.5
100000
0.0
0.0n
0.2n 0.4n 0.6n 0.8n
1.0n
-.5
0.0
0.5
1.0
Order
interaction
Figure 3: (Left) The distribution of interactions. The horizontal axis represents the order of inter- actions (n denotes the number of pixels in image). The two solid lines respectively represent the median of the interactions for clean images that Swin-T (green) and ResNet-18 (red) correctly classified. The shades represent the interquartile range. (Right) at order 0.0n, the interquartile range for ResNet-11 18 includes negative and positive interactions; for Swin-T, it only includes positive ones.
cleam (correctly classified)
clean (misclassified)
advensarial (misclassiliid))
frequency
10-
20-
10
10-
-0.05
0.00
0.05
0.2
0.2
0.02
0.00
0.02
0.00
0,00
0,02
interaction
Infernction
interaction
interaction
ResNet-18
(b) Swin-T
Figure 4: The distribution of average interactions (i.e., the distribution of B.,j [T(, for all the pixel pairs (i,j)). The distributions for misclassified clean (blue) and adversarial (red) images are contrasted to those for correctly classified clean images (green). The distribution for misclassified clean images is similar to that for correctly classified images, while that for misclassified adversarial images shows negative transition. This tendency is observed for both (a) ResNet-18 and (b) Swin- T, while the transition moderate the latter case.
ADDITIONAL EXPERIMENTS ON ADVERSARIAL ATTACKS
In Section 5, we observed that adversarial perturbations have significant effects on the distributions of interactions in middle and high order. In this section, further analyze adversarial images and their transferability by using the interactions. To this end, we consider for each image [ as the average of the interaction T(i. over all the possible pixel pairs in the image. For the adversarial attacks, we used the same methods and parameters as those used in Section 5.2. Other setups (e.g., dataset and image sampling) are also the same as those in Section 5.
Negative transition by adversarial attack. Here, we compare the distribution of average inter- actions between correctly classified clean images and misclassified (clean or adversarial) images. Figure 4 shows the histograms of the average interactions of images in the dataset. As demonstrated in Fig. 4(a), where ResNet-18 is considered, the distribution of average interactions is similar be- tween the two clean image sets (correctly classified clean images and misclassified ones), whereas there a large shift to negative values for adversarial images. The same observation holds for Swin- T as shown in Fig, 4(b). The negative shift of average interactions for adversarial images means that the pixel cooperations that models use to increase the confidence score are damaged or lost. Note that the negative shift is more moderate for Swin-T than ResNet-18. This indicates that although the images are misclassified, the damage in the cooperation of pixels is moderate for Swin-T. We consider that this reflects the fact that Swin-T is more robust than ResNet-18 (cf. Table 1 in Ap- pendix A).
Transferability and interactions We investigate adversarial transferability from the perspective of multi-order interactions. We consider Ey,,) (i, which is the average of interactions of
8

---

Under review as a conference paper at ICLR 2023
0.4
Swin-T
0.2
ResNet-18
0.2
VGG
0.0
0.0
AlexNet
MobileNer
-0.2
-0.2
ShuffleNet
0.00
0.2m
1.0g
0.0n
0.20
0.4n
Order
Order
(a) ResNet-18
(b) Swin-T
Figure 5: The difference in attack success rates between the sets of images with high and low average interactions (i.c., 0i a2). The horizontal axis represents the order of interactions (n denotes the number of pixels in an image). (a) For ResNet- 18, adversarial images with higher interactions in high order transfer better, whereas those with lower interactions transfer less. (b) For Swin- T, the trend is opposite to that for ResNet-18.
order over all the pixel pairs. As in Section 5.2, we generate two of adversarial images on Im- ageNet using ResNet-18 and Swin-T, T,, respectively. We first focus on ResNet-18. We divided the set of the adversarial images into two sets, D1 and D2, by the average interactions. The first D1 con- tains the adversarial images whose average interactions are higher than the median, while the second set D2 contains the rest. We then measure the transferability of adversarial images in each set D, by the attack success rate ag, which is defined as the ratio of the number of successfully attacked images in source model to that in a target model. We used ResNet- 18 and Swin-T as source models and ResNet- 18, Swin-T, AlexNet, ShuffleNet (Ma al., 2018), and MobileNet (Sandler et al., 2018) as target models. These models were pretrained on ImageNet'. In Fig. 5, the difference in attack suc- cess rates, a1 a2, between the sets of images with high and low average interactions is plotted along the orders. As shown in Fig 5(a), for D1 (i.e., adversarial images with high average interac- tions), the transferability increases the order increases. Interestingly, D2 (i.e., adversarial images with low average interactions), the trend is the opposite: the traceability decreases along the order. To summarize, adversarial images with higher interactions in high order transfer better when these are generated by using ResNet- 18 but transfer less when Swin-Ti T is used, and vice versa. This contrastive tendency is similar to the recent observation that the adversarial im- ages generated by CNNs are more perturbed in the high-frequency domain, whereas those generated by Vision Transforms are more perturbed in the low-frequency domain (Kim & Lee, 2022).
shown in Fig 5(b), for
7 CONCLUSION
This study conducted the first analysis of the misclassification of various types of images from a game-theoretic perspective. We considered clean, adversarial, and corrupted images and character- ized each of them using the distribution, order, and sign of interactions. Our extensive experiments revealed that each type of misclassification has a different tendency in the distribution of interac- tions, suggesting that the model relies on different categories of image features to make an incorrect prediction for each image type. We also analyzed Vision Transformers using interactions for the first time, finding that they give distribution of interactions that is different from that of CNNs in some cases, which confirms the recent observations that Vision Transforms exploit more robust features than CNNs do from new perspective. Our experiments also provide a new result that contrasts Vi- sion Transformers to CNNs: CNNs generate more transferable adversarial images from those with high interactions in high order, but the trend is the opposite for Vision Transformers. This paper reported various observations on CNNs and Vision Transformers based on interactions. We believe that this will contribute to deepening the understanding of machine vision.
Phttps://doi.org// www.////////..../// 4431043 (CNNs),
https://githin com/rwt wwwptiigt/gggtttreeeee.ooee wwwptttooohhomimmeeeeeeeeeeeeeee imagi- model 1a (Vision Transformers)

---

Under review as a conference paper at ICLR 2023
REFERENCES
Cameron Buckner. Understanding adversarial examples requires a theory of artefacts for deep learn- ing. Nature Machine Intelligence, 2(12):731-736 2020.
Xu Cheng, Chuntung Chu, Yi Zheng, Jie Ren, and Quanshi Zhang. A game-theoretic taxonomy of visual concepts in DNNs. arXiv preprint arXiv:2106. 17X58,, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensem- ble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learming, pp. 2206-2216, 2020.
Huiqi Deng, Qihan Ren, Hao Zhang, and Quanshi Zhang. Discovering and explaing the representa- tion bottleneck of DNNs. In Proceedings of the International Conference Learning Represen- tations, 2022.
Jia Deng,
Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255,40 2009.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. pp. 9185 9193, 2018.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proceedings of the International Conference on Learning Representations, 2015.
Michel Grabisch and Marc Roubens. An axiomatic approach to the concept of interaction among players in cooperative games. International Journal of Game Theory, 28:547-565, 1999.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp- 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In Proceedings of the International Conference on Learning Repre- sentations, 2019.
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. In Processing of the Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Gihyun Kim and Jong-Seok Lee. Analyzing adversarial robustness of vision transformers against spatial and spectral attacks. arXiv preprint (xXiv:2208.09600 2022.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep con- volutional neural networks. In Proceedings of the Advances Neural Information Processing Systems, pp. 1097-1105, 2012.
Alfred Laugros, Alice Caplier, and Matthieu Ospici. Are adversarial robustness and common pertur- bation robustness independent attributes? In Proceedings of TEEE/CVF International Conference on Computer Vision Workshop, pp. 1045-1054, 2019.
Alfred Laugros, Alice Caplier, and Matthieu Ospici. Addressing neural network robustness with mixup and targeted labeling adversarial training. In Proceedings of the European Conference on Computer Vision Workshop, pp. 178-195. 2020.
Ze Liu, Yutong Lin, Yue Cao, Han Hu. Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 10012-10022, 2021.
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In Proceedings of the European Conference on Computer Vision pp. 116-131, 2018
10

---

Under review as a conference paper at ICLR 2023
Jie Ren, Die Zhang, Yisen Wang, Lu Chen, Zhanpeng Zhou, Yiting Chen, Xu Cheng. Xin Wang, Meng Zhou, Jie Shi, and Quanshi Zhang. Towards a unified game-theoretic view of adversarial perturbations and robustness. In Proceedings of the Advances in Neural Information Processing Systems, pp. 3797-3810 2021.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chieh Chen. Mo- bileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510-4520, 2018
Lloyd S. Shapley. A value for n-person games. In Contributions the Theory of Games, pp- 307-317, 1953.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In Proceedings of the International Conference on Learning Representations, 2014.
Chun Yang Tan, Kazuhiko Kawamoto, and Hiroshi Kera. Adversarial amplitude swap towards robust image classifiers. arXiv preprint Siti::2203.073333 2022.
Hugo Touvron, Matthicu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers & distillation through attention. In
Proceedings of the 38th International Conference on Machine Learning, pp- 10347-10357, 2021.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustess may odds with accuracy. In Proceedings of the International Conference on Learning Representations, 2019.
Xin Wang, Jie Ren, Shuyun Lin, Xiangming Zhu, Yisen Wang, and Quanshi Zhang. A unified ap- proach to interpreting and boosting adversarial transferability. In Proceedings of the International Conference Learning Representations, 2021.
Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo. Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. TRS: Transferability reduced ensemble via promoting gradient diversity and model smoothness. Proceedings of the Advances in Neural Information Processing Systems, pp. 17642-17655, 2021.
Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A Fourier perspective on model robustness in computer vision. In Proceedings of the Advances in Neural Information Processing Systems, pp. 13276-181 13276-13286, 13286, 2019.
Hao Zhang, Sen Li, YinChao Ma, Mingjie Li, Yichen Xie, and Quanshi Zhang. Interpreting and boosting dropout from a game-theoretir theoretic view. In Proceedings of the International Conference on Learning Representations, 2021.
A ADVERSARIAL ROBUSTNESS EVALUATION
We evaluated the adversarial robustness of ResNet-18 18 and Swin-T by the success rate of the Loo- untargeted I-FGSM attack with various perturbation strength 0.5/255, 1/255, 2/255.....) For each model, we selected 1,000 clean images that are classified correctly. The result is shown in Table Swin-T has a lower attack success rate than ResNet-18 18 for all c, showing that the former more adversarially robust than the latter.
B EXPERIMENTAL SETUP AND ADDITIONAL EXPERIMENTS
B. EXPERIMENT DETAILS
As the computation of interactions is known as an NP-Hard problem, we reduce the computa- tional cost as follows. We divided each image into 16 16 patches and measured the inter- actions between the patches. Instead of using the full dataset, we randomly sampled fifty im- ages of different classes. For each image, 200 patch pairs randomly selected, and for each
11

---

Under review as a conference paper at ICLR 2023
Table I: The success rate of the I-FGSM attack with various perturbation strengths. Swin-T shows a lower attack success rate than ResNet-18
ResNet-18
Swin-T
0.25
0.652
0.545
0.5
0.924
0.767
1
0.996
0.937
2
1.0
0.993
4
1.0
1.0
16
1.0
1.0
Interaction Strength
Swin-T
5.0
ResNet-18
AlexNet
DeiT-Ti
2.5
0.0
0.0n 0.2n 0.4n 0.6n 0.8n 1.0n Order
Figure 6: The median of interaction strength Eq. (B.2) along the order. The solid lines represent the median. The trend is similar across models. Compare to Figs. 2(left) and 8(left).
patch pair, 100 contexts are randomly selected. Here, each patch pair randomly sampled so that one patch is within radius of two patches from the other patch. For order, consider s 0.0n, 0.05n, 0.1n, 0.2n, 0.8n, 0.9n, 0.95n, 1.0n. The abovementioned setup is the same as in (Deng et al., 2022). addition, when we compute an interaction [(i, j), we average (i. for 0.0n, 0.1n 1.0n.
B.2 STRENGTH INTERACTIONS
Most of the existing studies used the strength of an interaction, which is computed as fol- lows (Cheng et al., 2021; Deng al., 2022).
j(m) Exca Ex ((m) (1,1)
En [EzcA Es ([mmmm)) (i, (1,1)
we discussed in 4.3, the sign tells us whether the pixel pair interacts positively not, and there several trends that cannot be grasped one only considers the strength of interactions. Fig. shows an example, where we plotted the median of interactions for clean images along the order. As can be seen, for all the models, the trend of interaction strength is similar (i.e., high interaction strength at low and high orders). However, as shown in Figs. 2(left) and 8(left), the four models present different trends when singed interactions are used.
By contrast, our study considers the sign of interactions.
B.3 CORRUPTED IMAGES (Gaussian noise)
In Section 5.3, we used fog corruption because this is known as the low-frequency corruption, in which adversarially trained models less robust than normally trained ones. Here, conduct the same experiments with Gaussian noise corruption, for which adversarially trained models are known to perform better than normally trained ones. The result is shown in Fig. 7. One that
12

---

Under review as a conference paper at ICLR 2023
clean (correctly classified)
corrupted (misclassificd)
(a) ResNet
(c) AlexNet
Af S)
0.5
0.0
0.0n
0.2n
0.4n 0.6n
0.8n 1.0n
0.0n 0.2n
0.4n 0.6n
0.8n
1.0n
Order
Order
(c) Swin-T
(d) DeïT-Ti
0.2
0.05
Af
0.00
0.0
-0.05
0.0n 0.2n 0.4n 0.6n 0.8n 1.0n
0.0n 0.2n 0.4n 0.6n 0.8n 1.0n
Order
Order
Figure 7: The distribution of interactions for misclassified images under Gaussian noise. Compare to Figs. 2(right) and 8(right).
for all the models except AlexNet, the strength of low-order interactions is low, while the strength of high-order interactions high. This trend similar to that for fog corruption (cf. Figs. 2rrihh and 8(right)); thus, the misclassifications under fog and Gaussian noise can be considered similar in a sense.
B.4 RESULTS FOR ALEXNNET AND DEIT-TT
We investigated whether our experiments in Section generalize to other models. We used AlexNet and DeīT-Ti that are pretrained ImageNet. The experimental setup is the same in Section The result is shown Fig. 8. The tendency of interactions for AlexNet was the same as that for ResNet- 18. DeiT-Ti also presented almost the same tendency as Swin-T. However, the tendency in the low-order interactions for clean images was different. Swin-TT showed a striking difference in the sign of the interactions between correctly classified and misclassified images; Deit-Ti showed no distinct difference. This result indicates that the importance of low-order interactions for the prediction differs between models.
13

---

Under review as a conference paper at ICLR 2023
clean (conectly classified)
adversarial (misclassified)
(a) AlexNet
clean (misclassified)
corrupted (misclassified)
Af(
0.5
0.0
0.0
-0.5
0.4n 0.6n Order
0.8n
0.0n
0.2n
0.4n
0.6n
0.8n 1.0n
0.0n
0.2n
0.4n
0.6n Order
0.8n
1.0n
Order
(b) DeiT-Ti
M(i,j, S)
0.1
0.05
0.05
0.00
0.00
0.0
0.05
0.05
0.0n 0.2n
0.6n 0.8n 1.0n Order
0.0n
0.6n 0.8n 1.0m Order
0.0m 0.2n
0.4n 0.6n
0.8n
1.0m
Order
Figure 8: The distribution of interactions (i.e., the distribution of Af S) for various pixel pairs (i.j), context S, and images). The horizontal axis represents the order of interactions (n denotes the number of pixels in an image). The solid lines represent the median of the interactions. The shades represent the interquartile range. The interactions for misclassified clean (left, blue), adversarial (center, red), and corrupted (right, purple) images are contrasted to those for correctly classified images (green), respectively. One can observe that the three types of misclassified images give different tendencies in the distributions of interactions for each order. Compare to Fig. 2.
14


Reviews:
[{"review_id": "bloujqaIPhk", "reviewer": "Reviewer_QLY7", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper diagnosis the behavior of CNN and Transformer models on three classes of images-- clean, adversarial, and real-world corruptions-- for the ImageNET dataset. For this purpose, they use `interactions'-- shapely values that determine how much a pair of pixels contribute to the class prediction, given a set of pixels in the image (that determine the order of interaction). Additionally, the authors consider the sign of interaction which helps them determining if the pixels together help increase/decrease the confidence of prediction.", "summary_of_the_review": "The paper, although interesting in how it tries to diagnose misclassification in CNN and transformer models for image domain via interactions, is incomplete in several regards. It does not compare it self to any other diagnostic took, it does not highlight the usefulness of the diagnostic tool.", "strength_and_weaknesses": "### Things I liked\n\nThe paper considers an interesting angle to investigate misclassification of a classifier and is able to highlight interesting pattens from the study (eg. corrupted and adversarial images impact different kind of interactions and order).\n\n### Things that need clarification / improvement\n\n- The authors highlight that Cheng et. al. 2021 argue that lower-order reflects local properties while higher order reflects global properties (Sec 4.2). Unfortunately, I am not entirely convinced. I would rather only consider the 1.0n case which measures the interaction of a pair of pixels in the context of the entire input image.\n\n- The paper is incomplete in many ways.\n  - While the authors do an analysis of the multiple input image classes on which a classifier misclassifies, I am not sure how one can use it to improve a classifier robustness. For example, does the interesting observation that corrupt and adversarial images impact different types/orders of interaction have any conclusion like existing defense mechanisms cannot target both ends and fixing on will not help fix the other?\n  - How does data augmentation or other defense strategies affect these interactions?\n  - Why is interaction-based diagnosis better/more useful compared to other debugging approaches (eg. GradCAM, analyzing the impact of individual pixels on loss that is used to craft adversarial examples).\n\n- The authors simply use a shapely value type measure to characterize the contribution of pixel pairs to prediction via masking. This is not strong enough a motivation to say they have done a game-theoretic understanding of misclassification, as the title suggests.\n\n- There are several clarity issues with the paper.\n  - In Figure 1 (a), one can form many pairs of black pixels. Do all combination of these pairs have $\\delta f(i,j,S)$ as strongly positive?\n\n- In Figure 2, why is the curve for correctly classified clean examples (in green) different in the middle figure for ResNet-18 (i.e. top row) compared to the left and right one? I was expecting the clean curves to be similar in all the 3 graphs for a row (which also seems to be the case for Swin-T).\n\n- In Figure 3 and Figure 4, should the distribution of clean images not be the same in the frequency charts. For example, I am a bit confused that Swin-T has only green bars in the positive side for the clean images in Fig 3 (left) but has a normal looking distribution for clean images in Fig 4 (b). What is the n for the distribution in Fig 4, I see it is an expected value over I(i,j) but I wasn't sure where the difference is coming form?\n\nOther corrections:\n\n- [Sec 5.3] \"The strength of the interactions for adversarial images is lower\" -> \"The strength of the interactions for corrupted images is lower\"\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "aEPiUUJAIg", "reviewer": "Reviewer_C5ku", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Similar to previous studies, this paper studies the feature representation of a DNN by empirically investigating the distribution of interactions of input images. On the one hand, the authors investigate interactions of three types of mis-classified input images (including clean, adversarial, and corrupted images). On the other hand, the authors investigate interactions on two types of DNNs, including CNNs and version Transformers. Finally, the authors obtain some conclusions by analyzing tendencies in encoding interactions in these settings.  ", "summary_of_the_review": "This paper has very limited novelty. In fact, the main content of this paper is very similar to previous studies (Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021; Wang et al., 2021; Zhang et al., 2021), which proposed to use the distribution of interactions encoded in input images to study the feature/conceptual representation of a DNN. This paper directly adopts theories, algorithms, and experimental designs in previous studies, and the core difference is that this paper uses different types of input images and different types of DNNs in experiments. Specifically, it can be found that in Sections 3 and 4, the main contents of introducing (multi-order) interactions are similar to previous work (Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021; Wang et al., 2021; Zhang et al., 2021). \nMoreover, some conclusions also overlap with those of previous studies. For example, Figure 2 shows that adversarial attacks make DNNs use more negative high-order interactions, which has been found in previous work (Ren et al., 2021). Moreover, this is the only significant conclusion in Figure 2, while the difference in the first and last columns of Figure 2 is marginal.", "strength_and_weaknesses": "[Weakness]\n \n1. This paper has very limited novelty. In fact, the main content of this paper is very similar to the line of previous studies (Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021; Wang et al., 2021; Zhang et al., 2021), which proposed to use the distribution of interactions encoded in input images to study the feature/conceptual representation of a DNN. This paper directly adopts theories, algorithms, and experimental designs in previous studies, and the core difference is that this paper uses different types of input images and different types of DNNs in experiments. Specifically, it can be found that in Sections 3 and 4, the main contents of introducing (multi-order) interactions are similar to previous work(Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021; Wang et al., 2021; Zhang et al., 2021). \n\n2. Some conclusions also overlap with those of previous studies. For example, Figure 2 shows that adversarial attacks make DNNs use more negative high-order interactions, which has been found in previous work(Ren et al., 2021). Besides, this is the only significant conclusion in Figure 2, while the difference in the first and last columns of Figure 2 is marginal.\n\n3. Experiments in the paper are not sufficient. In the main paper, the authors only conduct experiments on two models on a dataset, and only one setting is used for adversarial attack and corruption, respectively. Therefore, it is questionable whether the obtained conclusions also appear in other DNNs/datasets/settings. I suggest the authors conduct more experiments on more architectures with more settings to demonstrate the validity of their conclusions. \n\n4. The comparison between ResNet-18 and Swin-T in Figure 3 is not convincing. First, the comparison may be unfair. The magnitudes of outputs of the two models may be different, so the value range of $\\Delta f(i,j,S)$ may be different in the two models. Therefore, interactions in the two models cannot be directly compared. Second, the 0-order interaction in Figure 3 is significantly affected by baseline values in the computation of interactions, and the authors do not discuss this problem. Third, the difference in the interaction median between the two models is not significant, and it is hard to judge whether the difference is caused by other factors like initialization, dataset, or the training method. The phenomenon on only one pair of models cannot support the conclusion. \n\n5. The authors do not provide any theoretical explanations for conclusions in the paper.\n\n6. The experiment and conclusion about adversarial transferability in Section 6 are not convincing. When comparing the transferability of inputs with different interactions of a specific order, the interactions of other orders are not controlled to be the same. Therefore, the high transferability of inputs with high 0.8n-order interactions may be due to interactions of other orders. This experiment cannot provide a fair comparison between different orders.\n\nIn general, in my opinion, this paper neither presents theoretical (algorithmic) contributions, nor introduces new experimental designs so as to provide new insights. More crucially, the conclusions also have large overlap with conclusions of previous studies.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "Yt4nNAf3o5", "reviewer": "Reviewer_oeXE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates various types of misclassifications from a game-theoretic perspective. The authors study the misclassification of clean, adversarial, and corrupted images, and found that the dominant order of interactions is different between the three types of misclassification. The authors also found that the interactions of ViT are also different from CNNs.", "summary_of_the_review": "I think the general idea of this paper is interesting. The authors provided some interesting perspectives for adversarial examples through Shapley value interactions.", "strength_and_weaknesses": "Strength:\nThis work conducted extensive experiments to support the idea. The Shapley value interaction seems to be an interesting idea.\nWeakness:\n1. The difference between distributions of interactions for different types of images is quite small in Figure 2. Especially between clean and corrupted images and low order of interactions. I was wondering if the authors can provide some explanations for this.\n2. More images like the visualization in Figure 1 may need to be provided in the appendix to make the intuition more convicing.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "R3BEv_PV5Qc", "reviewer": "Reviewer_Mq4K", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work empirically investigates various types of image misclassification from a game-theoretic view, which includes clean images, adversarially perturbed images and corrupted images. It characterizes the misclassification with the distribution, order, and sign of the interactions, and numerical results show three types of misclassifications have different tendencies in interactions. And it also explores interactions of misclassifications in different model architectures, CNNs and Vision Transformers, which implies Vision Transformers may exploit the features that CNNs do not use for the prediction.", "summary_of_the_review": "This work studies misclassification of various types of images from a game-theoretic perspective. It provides a novel understanding of misclassification for different types of images and model architectures.", "strength_and_weaknesses": "Originality:\n\n1. Focusing misclassifications by using interactions among pixels is interesting, which manifests three types of misclassifications have a distinct tendency in interactions and each of them arises from different causes. \n\n2. It also analyzes Vision Transformers by using interactions to show their feature extraction that is different from CNNs.\n\nQuestion:\n\nAdversarial training is a well-known standard method to improve models' robustness. However, misclassification seems to be inevitable even if model is trained by adversarial training. So, dose there exist a significant difference between the interactions of misclassifications original models and those trained by adversarial training?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "M3B6Qwa_my", "reviewer": "Reviewer_zrJN", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThe paper studies various types of misclassification of images by deep neural networks. The study is done using a quantity called  \"interaction\" between pixels. This quantity is motivated from game theoretic concept of quantifying interaction between players in a co-operative game. This quantity has been used in other studies on deep learning. ( Cheng et al., 2021; Deng et al., 2022; Ren et al., 2021 and others).  The paper characterizes the misclassification of clean, adversarial, and corrupted images with the distribution, order, and sign of the interactions. They find that each type of misclassification has different tendencies in interactions, which indicates that each type of misclassification is triggered by different causes. They also provide an analysis of Vision Transformers by using interactions and report that the difference in distributions of interactions between misclassified and correctly classified images are clearer and also different from the case with CNNs. Further they also found that the images that are more adversarially transferable have the opposite tendency in the interactions between Vision Transformers and CNNs.\n", "summary_of_the_review": "The paper studies and important problem of understanding misclassification by neural nets and studying it through the lens of interaction is novel. Their empirical observations give insights into various types of misclassifications. I have a few questions/concerns listed above that could improve my assessment of the paper.", "strength_and_weaknesses": "Studying when and why deep models misclassify is an important problem. The paper gives some insights on this problem through the lens of interaction between pixels. They study misclassification of clean images and the ones with adversarial and non-adversarial corruptions. For the clean images they observe that the successfully classified images have local cooperation between pixels that increase the confidence score, while the misclassified images do not have such cooperation, leading to the deterioration in the confidence score. For the adversarially corrupted images they observed that misclassification by adversarial perturbations is caused by the destruction of the model from the meaningful cooperation between pixels to the spurious one, which is useless or harmful to make a prediction, while these harmful effects are milder for non-adversarially corrupted images. Further, they conducted same study with Vision transformers and found that Vision Transformer more clearly exhibit the prediction characteristics than CNNs. \n\nWeaknesses/Questions,\nCould you provide more intuitive understanding of interaction ( something like Figure 1). Why are the left and middle figures roughly the same in Figure 1 but have opposite signs of interaction?\n There are a ton of methods to generate adversarial images, will the paper's observations remain applicable to these diverse set of adversarial methods? Soon one can design a method to generate adversarial images that show no difference in terms of interaction as well. Or can this be shown that it is not possible? \nSince the conclusions are drawn from experiments alone, it would be more helpful to see if the similar observations hold for various datasets and models as well? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject