Respect the model: Fine-grained and Robust Explanation with Sharing Ratio Decomposition
Authors: Sangyu Han
Yearim Kim
Nojun Kwak
Venue: ICLR 2024 poster (2024)
Keywords: explainable ai
Primary area: visualization or interpretation of learned representations

Abstract: The truthfulness of existing explanation methods in authentically elucidating the underlying model's decision-making process has been questioned. Existing methods have deviated from faithfully representing the model, thus susceptible to adversarial attacks.
To address this, we propose a novel eXplainable AI (XAI) method called SRD (Sharing Ratio Decomposition), which sincerely reflects the model's inference process, resulting in significantly enhanced robustness in our explanations.
Different from the conventional emphasis on the neuronal level, we adopt a vector perspective to consider the intricate nonlinear interactions between filters.
We also introduce an interesting observation termed Activation-Pattern-Only Prediction (APOP), letting us emphasize the importance of inactive neurons and redefine relevance encapsulating all relevant information including both active and inactive neurons.
Our method, SRD, allows for the recursive decomposition of a Pointwise Feature Vector (PFV), providing a high-resolution Effective Receptive Field (ERF) at any layer.

Published as a conference paper at ICLR 2024
RESPECT THE MODEL: FINE-GRAINED AND ROBUST EXPLANATION WITH SHARING RATIO DECOMPOSITION
Sangyu Han", Yearim Kim,
Nojun Kwak
Seoul National University
Scoul, 08826, Korea
facoexist 96 verim1656, no nonk fom
ABSTRACT
The truthfulness of existing explanation methods in authentically elucidating the underlying model's decision-making process has been questioned. Existing meth- ods have deviated from faithfully representing the model, thus susceptible to adversarial attacks. To address this, we propose novel eXplainable AI (XAI) method called SRD (Sharing Ratio Decomposition), which sincerely reflects the model's inference process, resulting significantly enhanced robustness in our explanations. Different from the conventional emphasis on the neuronal level, we adopt a vector perspective to consider the intricate nonlinear interactions between filters. We also introduce interesting observation termed Activation-Patter Only Prediction (APOP), letting us emphasize the importance of inactive neurons and redefine relevance encapsulating all relevant information including both active and inactive neurons. Our method, SRD, allows for the recursive decomposition of Pointwise Feature Vector (PFV), providing high-resolution Effective Receptive Field (ERF) at any layer.
INTRODUCTION
In light of the remarkable advancements in deep learning, the necessity for transparent and reliable decision-making has sparked significant interest in explainable AI (XAI) methods. In response to this imperative demand, XAI researchers have aimed to provide insightful and meaningful explanations that shed light on the decision-making process of complex deep learning models. However, the reliability of existing explanation methods in providing genuine insights into the decision-making process of complex AI models has been questioned.
Previous methods have not consistently adhered to the model but rather customized it to their re- spective preference. As result, many of them are vulnerable adversarial attacks, causing doubt on their reliability. To address this issue, we focus on faithfully representing the model's 's inference process, relying exclusively on model-generated information, and refraining from any form of cor- rection. This approach supports the robustness of our explanations compared to other methods.
Moreover, existing methods have traditionally analyzed models at the neuronal level, often over- looking the intricate nonlinear interaction between neurons]to to form a concept. This approach has been derived from the assumption that an individual scaler-valued channel (a filter or neuron) carries specific conceptual meaning. That is, the value of single neuron directly determines the conceptual magnitude with the significance of a pixel being determined as a linear combination of each constituting neuron's 's conceptual magnitude. However, this assumption may oversimplify the complex nature of deep learning models, wherein multiple neurons nonlinearly collaborate to form concept. Therefore, space to account for the interaction among neurons. Specifically, we introduce the pointwise feature vector (PFV), which is vector along a channel axis of a hidden layer, amalgamating neurons that share the same receptive field.
analyze the models from perspective of vector, exploring the vector
Equul contribution. *Corresponding author.
A outputs scalar, an element in tensor, by combining the information in its receptive field.

---

Published as a conference paper at ICLR 2024
In addition, we alter the conventional way of calcu- lating relevance based on post-activation values into the one based on pre-activation values. It is widely believed that, for achieving conceptual harmony and class differentiation at the final layer, image activa- tions from the same class should undergo progressive merging along the shallow to deep layers (Fel et al.) 2023). With this belief, previous methods have pn- marily focused on analyzing the value of the post- activation output, identifying the key contributor to the merged concept. However, we observe a fascinating phenomenon termed Acrivation-Pattern-One tion (APOP), which shows that classification accura- cies can be considerably maintained without receiving any input image, relying solely on the on/off activa- tion pattern of the network (Refer to Tab. [ for de- tails). This highlights the importance of not only con- sidering active neurons but also inactive ones, as both contribute to forming the patterns. However, after the nonlinear activation process, such as ReLU, the infor- mation about the contributors to the inactive neurons lost. Therefore, we consider the contribution of the neurons in the prior layer to inactive neurons to fully comprehend the contribution of features.
Table I: Classification accuracies on Im- ageNet validation set achieving compara- ble performance without any inputs, solely relying on weights and the activation pat- tern. Here, activation pattern means that the model records masks where the inac- tive neurons are flagged, during a predic- tion. Even with an empty image, the model makes comparable predictions when ReLU and Maxpool are replaced by the recorded masks. More information about APOP is contained in Appendix D
Predic-
Top-1
Top-5
Original APOP
Original APOP
VGG13
.679
.544
.82
.787
VGG16
.698
.555
894
.809
VGG19
.705
.88
.22
ResNet18
.670
.487
.876
.734
ResNet34
.711
.557
.000
.700
ResNet50
.744
.569
.918
.794
ResNet101
.756
.560
.928
.785
RcsNet152
.769
.612
.935
.26
Considering the aforementioned challenges. we present our novel method, Sharing Ratio Decom- position (SRD), which decomposes a PFV comprising preactivation neurons occupying the same spatial location of a layer into the shares of PFVs in its receptive field. Our approach is centered on faithfully adhering to the model, relying solely on model-generated information without any alter- ations, thus enhancing the robustness of our explanations. Furthermore, while conventional methods have predominantly examined models at the neuronal level, with linear assumptions about chan- nel significance, introduce vector perspective, delving into the intricate nonlinear interactions between filters. Additionally, with our captivating observation of APOP, we redefine our relevance, focusing on contributions to the pre-activation feature map. where all pertinent information is encap- sulated. Our approach goes beyond the limitations of traditional techniques in terms of both quality and robustness, by sincerely reflecting the inference process of the model.
By recursively decomposing a PFV into PFVs of any prior layer with our Sharing Ratio Decompo- sition (SRD), we could obtain a high-resolution Effective Receptive Field (ERF) at any layer, which further enables us to envision comprehensive exploration spanning from local to global explana- tion. While the local explanation allows us to address where in terms of model behavior, the global explanation enables us to delve into what the model looks at. Furthermore, by decomposing the steps of our explanation, we could see a hint on how the model inferences (Appendix AA)
RELATED WORKS
Backpropagation-basee methods such as Saliency (Simonyan al. 2014). Guided Backprop (Springenberg et al.) 2015), GradInput (Ancona et al.] 2018), InteGrad (Sundararajan et al.) 2017). Smoothgrad (Smilkov et al. 2017). Fullgrad (Srinivas & Fleuret 2019) generate attribution maps by analyzing model's 's sensitivity to small changes through backpropagation. They calculate the error through backpropagation for the input value to indicate the importance of each pixel, often generat- ing noisy maps due to the presence of noisy gradients. Furthermore, there is doubt on the credibility of these methods, claiming that the gradients are not used during the inference process.
In contrast, LRP (Bach et al. 2015) constructs saliency maps solely using the model's weights and activations, without gradient information. LRP calculates the contribution of every neuron by prop- agating relevance, while our method, SRD, calculates relevance of vectors. Yet. different from LRP families (Bach et al., 2013) Monnaaon et al.) 2017 2019), which either ignores or assigns minor contribution negatively contributing neurons for the active neuron, we acknowledge the signifi-
2

---

Published as a conference paper at ICLR 2024
cance of every contribution in the model's inference process. Moreover, while LRP may not account for contributions to inactive neurons, who hold vital information for the inference, we elaborately handle contributions to both active and inactive neurons.
Activation- based methods generate activation maps by using the linearly combined weights of ac- tivations from each convolutional layer of model. Class Activation Mapping (CAM) (Zhou al.) 2016) and its extension, Grad-CAM (Selvaraju et al. 2017), enhance interpretability in neural net- works by utilizing convolutional layers and global average pooling. Grad-CAM++ (Chattopadhay et al. 2018) further improves localization accuracy by incorporating second-order derivatives and applying ReLU for finer details. These CA.e-based approaches assume that each channel possesses distinct significance, and the linear combination of channel importance and layer activation can explain the regions where the model looks at importantly. However, due to nonlinear correlations between neurons, the CAM methods, except LayerCAM. struggle at lower layers, yielding saliency maps only with low-resolution. In contrast, LayerCAM (Tiang et al., (02T) inspects the importance of individual neurons, aggregating them in channel-wise manner. It seems similar to our SRD as it calcalates the importance of a pixel (thus a vector). However, also disregards negative contribu- tions of each neuron and does not account for contribution of inactive neurons, gradients do not flow through them.
Desiderata of explanations The absence of 'ground truth' poses challenges for objective com- parisons, given that explainability inherently depends on human interpretation (Doshi Velez & Kim 2017). To mitigate this issue, specific desiderata have been established such as Localization, Com- plexity, Faithfulness, and Robustness (Binder et al., (023). Localization demands accurate identi- fication of crucial regions during model inference, while Complexity requires creating sparse and interpretable saliency maps. Faithfulness insists that the removal of 'important' pixels significantly impacts the model's 's prediction. Robustness necessitates consistent saliency maps under both random and targeted perturbations. ensuring resilience against manipulations aimed at misleading explana- tions (Ghorbani et al. 2019a Dombrowski et al., 2019). Our model, SRD, surpasses other state- of-the-art methods in meeting these desiderata without any modification of neuronal contributions during model inference.
3 METHOD: SHARING RATIO DECOMPOSITION (SRD)
Our method provides the versatility to perform both in forward (Fig. 1 and backward (Fig. 2 passes through the neural network, enabling a comprehensive analysis from different angles. A formal proof demonstrating this equivalence is provided in Appendix C
3.1 FORWARD PASS
Pointwise Feature Vector The pointwise feature vector (PFV), our new analytical unit, comprises neurons in the hidden layer that share the same receptive field along the channel axis. Consequently, the PFV serves as fundamental unit of representation for the hidden layer, as inherently a pure function of its receptive field. For linear layers, we compute the contributions of the previous PFVs to the current layer directly, leveraging the distributive law. However, for nonlinear layers, is challenging to obtain the exact vector transformed by the layer, leading us to calculate relevance instead. The output or activation of layer 1, denoted as A' E RC* HW, is composed of HW PFVs, of E CRC, where p E {l..... HW] & [HW] denotes the location of the vector in the feature map. Note that each vector belongs to the same C-dimensional vector space V*
Effective Receptive Field Each neuron can be considered as a function of its Receptive Field (RF), and likewise. other neurons situated at the same spatial location but within different channels are also functions of the same RF Consequently, the PFV, which comprises neurons along the channel axis, serves as a collective representation of the RF, effectively encoding the characteristics of its corresponding RF. However, note that the contribution of individual pixels within the RF is not uni- form. For example, pixels in the central area of the RF contribute more in the convolution operation compared to edge pixels. This influential region is known as the Effective Receptive Field (ERF), which corresponds to only fraction of the theoretical receptive field (Luo et al. 2016). However, the method employed in (Luo et al., 2016) lacks consideration for the Activation Pecaation--aaattee Pattern-Oni Only Pre- diction (APOP) phenomenon and the instance-level of ERF. To address this limitation, we introduce
3

---

Published as a conference paper at ICLR 2024
VGG15 Encoder
100 do
EaT
Input image
1214 MAll
[13 teut
ERP +06
+ ME43 ERP + H6AR
ERF
ERP
Mine
EST
ERF sum
0.07
0.19
+ +.. 0.08
Figure 1: Forward Pass of method. Top: An illustration of inference process. Red box portrays the contribution of v in forming ((577)' quantified by 25-27 Each vi is labeled with its cor- (5-75,7))
responding ERF. Bottom Left: The process of building ERF for v27 saliency map is derived as weighted sum of the ERFs the encoder output layer.
(5.7)' Bottom Right: The final
the sharing ratio to reflect the different contributions of pixels and make a more faithful ERF for each PFV. With our ERFs, we can investigate the vector space of the PFV, leading global explanation of the model. For more details, refer to Appendix A
Sharing Ratio Decomposition Assuming we have prior knowledge of the sharing ratio, denoted as between layers (which can be derived at any point, even during inference), where μ signifies the extent which each PFV contributes to the PFV of the subsequent layer (Exact way to obtain j is deferred to Sec. 3.2) Given that we already possess information of PFVs, we can construct the ERF of the next activation layer through a weighted sum of the constituent PFV's ERFs, expressed as follows (Fig. [ Top, Bottom Left):
the ERFs and sharing ratios
ΣΣS ERF, ERF Ick
where nI=5 is the sharing ratio of pixel i of layer to pixel j of the subsequent layer k, and ERF, an ERF of PFV v/. Note that we can summate the ERFs different layers which are parallelly connected the /--th layer, e.g., residual connection. For the first layer, its ERF is defined
ERF E
(2)
where E is unit matrix, where only the i-th element of the matrix is one, and all the others zero. This means that the ERF for an input pixel the pixel itself.
Consequently, can sequentially construct the ERF for each layer until reaching the output of the encoder. The output consists of HW PFVs along with their ERFs. The final sailency map (x) is obtained through weighted sum of the ERFs of the encoder's output PFVs (Fig Bottom Right):
Euloo ERF, = vo(x),
where uf the sharing ratio of the pixel i of last layer L to the pixel c of the output (logit) O, which is the contribution of each PFV to output class As MLP classifier after encoder flattens the vectors to the scalars, there is no need to persist with our vector-bases approach. Thus, for an MLP layer, opt for the established backward methods such Grad-CAM (Selvaraju et al.) 2017).
4

---

Published as a conference paper at ICLR 2024
RF
Ri-j
R
Repeat
R
RC
E
fims
Pl_,,
= fo
Tooll
Ri-; M Rf.
RJ =
Σ RL-g. JEPS
PF = U |t E RFF"
Figure 2: Backward Pass of our method. and j are pixels in activation layer / and k; respectively.
Left: of is pre-activation PFV at activation layer k, of a post-activation PFV activation layer 1, fim. is an affine transformation function assigned to (i, Summation of every i leads to of EiERry th_,, =ff). 2/=) is sharing ratio of each uh= to uz. Ri_., is the relevance share of in the leading layer to in the following layer. Right: RFF is the receptive field of pixel, and R2 is relevance score of to the output. Relevance IC the leading layer can be calculated recursively using the next layer's relevance R/ s via Ri 's for j's whose receptive field includes pixel
Additionally, in order to ensure dinsiiiserimiietive saliency, we subtract the mean of its saliency and disregard any negative contributions. Then, the modified sharing ratic 400 for the encoder output layer is calculated as follows:
Oy DA
1 mmx :
Σ KE[K
-4,0) 0),
T Σ k
A
of
(4)
K
HW
where Al is the k-th element of the PFV ulast and y is the e-th element of the output logit E IR for K classes, ReLU operation is ommited when calculating D since it is already applied after subtracting the mean.
3.2 BACKWARD PASS (FOR CALCULATING SHARING RATIO)
Suppose a PFV of positioned at j just prior to activation layer k. In feed-forward network, uf is entirely determined by the (-th activation layer's PFVs of's within the receptive field of j. RF i.c,
u ((V En Σ 1 where Vkl
(ofli RFF).
some affine function /(-) (See details Appendix Bj. Note that PFV uy can be decomposed into o/= which is sole function of PFV vf. In our approach, we initially define the relevance Rt of pixel j in layer is as the contribution of the pixel the output, typically the logit. Then, we distribute the relevance, R/ to pixel i's in layer by the sharing ratio μ p which is calculated as taking the inner product of 04ا with v and normalizing both vectors by || as follows (Fig. Left):
where ((=
i.e.
Σ 1
=
(6)
iERF
Thee summation of modified sharing ratios does not necessarily equal 1.

---

Published as a conference paper at ICLR 2024
Then, according to the sharing ratio v. decompose the relevance the output:
= i.e,
R
Σ
R
EERF
Finally, the relevance of i to the output can be calculated
R
Σ
Ri-
PF. = filie RF)),
(8)
JEPP!
where PF is the Projective Field of pixel i the next nonlinear layer (Fig. Right).
The initial relevance at the last layer L, Ra is given
R = 1- M
(9)
where μ the modified sharing ratio described Eq. which represents the contribution of pixel in the encoder output layer to class
EXPERIMENT
In this section, conducted a comprehensive comparative analysis involving our proposed method,
SRD, and several state-of-the-art methods: Saliency (Simonyan et al.] (0144). Guided Backprop (Springenberg et al. 2013), GradInput (Ancona al. 2018). InteGrad (Sundararajan et al. (017).
LRP= x+
(Montavon et al.] (017). Smoothgrad (Smilkov et al. 2017), Fullgrad (Srinivas & Fleuret
2019 GradCAM (Selvaraju et al. (017), GradCAM+ (Chattopadhay et al. (0188). ScoreCAM
Wang et al. 2020). AblationCAM (Ramaswamy et al. 2020), XGradCAM (Fu et al. 2020). and LayerCAM (Jiang et al. 2021).
In our experiments, we leveraged ResNet50 (He et al. 2016 and VGG16 (Simonyan & Zisserman 2015) model Each method has different choice of targeted layer for its best performance. Thus, conducted experiments by targeting various layers to accomodate the varying resolutions of generated attribution maps. Since most CAM-based methods except for LayerCAM exhibit optimal performance when targeting higher layers, generated low-resolution explanation maps for them. The dimensions the resulting saliency maps were as follows: (7, 7) for low-resolution, (28, 28) for high-resolution, and (224, 224) for input-scale. All saliency maps were normalized by dividing them by their maximum values, followed by bilinear interpolation to achieve a resolution of (224, 224).
4.1 QUALITATIVE RESULTS
We visualize the counterfactual explanations of original image with and dog. Fig. shows that our explanations with SRD, are not only fine-grained but also counterfactual, while other meth- ods do not capture the class-relevant areas and result in nearly identical maps. For more examples, refer to Appendix
4.2 QUANTITATIVE RESULTS
Experimental setting We conducted a series of experiments to assess the performance of our method compared existing explanation methods. All evaluations were carried on the ImageNet-S50 dataset (Gao et al. 2022), which contains 752 samples along with object segmen- tation masks.
Metric The metrics used in our experiments are as follows: To evaluate localization, Pointing Game (1) (Zhang et al. 2018) measures whether maximum attribution point is on target, while Attribu- tion Localization (f) (Kohlbrenner et al.) 2020) measures the ratio between attributions within the segmentation mask and total attributions. To evaluate complexity, Sparseness (t) CChllssani et al.,
In this paper, restrict our discussion to Convolutional Neural Networks (CNNs); however, the extension arbitrary network architecture straightforward.

---

Published as a conference paper at ICLR 2024
Explanation 'Dog'
Original image
Saliency
InpuDXGrad
InteGrad
Smoothgrad
LRP
2
RullGrad
SRD(OursHinput
LayerCAM-hight (urrshss)h
GradCAM!
GradCAM+++
ScoreCAM
AblationCAMt
XGradCAMI
LayerCAM-low
SRD(Ours]-oov
Explanation on 'Cat'
Original Image
Sallency
InputXGrad
InteGrad
Smoothgrad
CuidedBackprop
LRP
RullGrad
SRD[Ours]-input
LayerCAM-high+
SRD(Ours) -iihh
GradCAM!
GradCAM
ScoreCAM
AblationCAM:
XGradCAM!
LayerCAM-low
SRD(Ours] low
Figure 3: Qualitative results ResNet50 for the class label 'Dog (Top)' and 'Cat (Bottom)'. Meth- ods decorated with & have the resolution of (7, 7) and methods with $ have the resolution of (28, 28), while the others have the input-scale resolution, (224, 224). Notably, compared other meth- ods, SRD for input resolution is adept at capturing the fine details of the image. Best viewed when enlarged.
2020) measures how sparse the attribution map is, based Gini index. For faithfulness test, Fi- delity (t) (Bhatt et al. 2020) measures correlation between classification logit and attributions. To evaluate robustness, Stability (4) (Alvarez Melis & Jaakkola 2018) measures stability of explanation against noise perturbation, calculating the maximum distance between original attribution and per- turbed attribution for finite samples. All of the metrics are calculated after clamping the attributions to [-1, IJ. since all the attrubution methods are visualized after clamping, Also, the arrow inside the parentheses indicates whether higher value of that metric considered desirable. For more details of the metrics, refer to Appendix
Results In the comprehensive evaluation presented in Table our method, SRD, showcased supe- rior performance across various metrics. Notably, for VGGT6 architecture, SRD-high attained the highest scores in both the Pointing game and Fidelity, securing the second-highest score in Attribu- tion Localization. Furthermore, SRD-input excelled in Sparseness and Stability, while consistently maintaining competitive scores across other metrics. This was particularly noteworthy when com- pared to input-scale methods.
In the case of ResNet, since many saliency map methods struggle to properly handle residual con- nections, some of the methods showed decline in performance even when the model performance itself improved. Remarkably, our method retained its competitive performance on ResNet50, On ResNet50, SRD-high achieved the highest scores in Attribution Localization and Fidelity with the second highest score at Pointing game. Additionally, SRD-input achieved the best performance for Pointing game and Stability, achieving the second highest scores in Attribution Localization and Sparseness. These results point out that our proposed method, SRD, can give functional, faithful, and robust explanation.
4.3 ADVERSARIAL ROBUSTNESS
An explanation can be easily manipulated by adding small perturbations to the input, while main- taining the model prediction almost unchanged. This means that there is discrepancy between the actual cues the model relies on and those identified as crucial by explanation. While the Stability metric in Sec 4.2 assesses the explanation method's resilience random perturbations, Dombrowski

---

Published as a conference paper at ICLR 2024
Table Average results of Pointing game (Poi.), Attribution localization (Att.,, Complexity (Com.), Sparseness (Spa.), Fidelity (Fid.), and Stability (Sta.) on Imagenet-S50 752 samples. All metrics are calculated after normalization, which is the default setting Hedstrôm al. (2023). Methods decorated with have the resolution of (7, 7) and methods with & have the resolution of (28, 28), while the others have the input-scale resolution, (224, 224). We marked the highest result in bold, and the second with underline.
VGG16
ResNet50
Method
Poi.
Att.t
Spa.
Fid.
Sta.
Poi.
Att.1
Spa.
Fid.
Sta.
Saliency
.793
.394
.494
.093
.181
.654
.370
.488
.063
.172
GuidedBackprop
892
.480
.711
.022
.100
.871
.498
.741
.022
.112
GradInput
.781
.387
.630
-0113
.181
.639
.361
.626
-0018
.178
InteGrad
.699
.416
.618
-017
.175
.759
.382
.614
-006
.171
LRP_*
.855
456
.535
.098
.182
.543
.332
.572
.012
.105
Smoothgrad
.845
.363
.536
-005
.190
888
.396
.556
-002
.166
Fullgrad
.796
.362
.334
.107
.203
.988
.387
.262
.123
.689
GradCAM!
945
.431
.466
.175
.583
946
.424
.411
.128
.757
GrudCAM++
.932
.429
.351
.176
.570
945
.414
.386
.129
.732
ScoreCAM!
.937
.582
.342
.167
.622
916
.381
.313
.123
.827
AblationCAM!
.928
.481
.493
.189
.622
.34
.394
.329
.133
.814
XGradCAM!
.406
.446
.181
.576
.946
.424
.411
.126
.753
LayerCAM-lowe
.869
.425
.446
.175
.450
.944
.111
.379
.128
.734
LayerCAM-highh
.865
.435
.401
.199
.423
941
.423
.349
.135
.486
SRD-low (ours)
.955
.424
.437
.179
.595
.946
.544
.682
.130
.600
SRD-high (ours)
.948
.566
.629
.206
.406
952
.579
.628
.142
.375
SRD-input (ours)
.925
.561
.788
.069
.099
.953
.576
.724
.082
.104
et al. 2019) evaluates the method's vulnerability to targeted adversarial attacks, while maintaining the logit output unchanged. The perturbation is optimized to minimize the loss below:
L >1 (1o(Fad)) (I X2 [[(aad))) F(
(10)
where Tadv Torg + 8, 6(x) is the saliency map of image x, and F(x) is the logit output of model F' given image x. We set A lell and A2 = 1e6 as in Dombrowski et al. (2019).
Experimental setting We conducted targeted manipulation ageNet image pairs for the VGG16 model. Given that adversarial attacks can be taken only to grackelt explanation methods, Integrated Gradients, LRP and our SRD for comparison. The learning rate was 0.001 for all meth- ods. For more detail, refer the work of Dombrowski al. (2019). The attack was stopped once the Mean Squared Error (MSE) between æ and Zadv reached 0.001, while ensuring that the change in RGB values was bounded within 8 in a scale of 0-255 let Feds be visually undistinguishable with x. After computing saliency maps, the absolute values were taken, as in Dombrowski et al. (2019). Since we obtained our 1!= by leveraging other methods, we set all p 10 to a constant value of 1 to eliminate the potential influence of other methods.
set of 100 randomly selected Im-
selected Gradient, GradInput, Guided Backpropagation,
Metric To quantitatively compare robustmess of the explanation methods towards the adversarial attacks, we measured the similarity between the original explanation d(xory) and the manipulated explanation (b using metrics such as the Structural Similarity Index Measure (SSIM) and Pear- son Corelation Coefficient (PCC). High values of SSIM and PCC denote that ó(Iadv) maintained the original features of ((Forg), thereby demonstrating the robustness the explanation method.
Result In both PCC and SSIM results (Figure 4 SRD consistently outperformed other input-scale resolution saliency maps, attending the highest scores. his, coupled with the findings from the Sta- bility experiments detailed in Table 2] substantiates that our proposed method, SRD, demonstrates exceptional resilience against adversarial attacks. Importantly, it maintains its explanatory efficacy even in the presence of perturbations, emphasizing its robustness.
8

---

Published as a conference paper at ICLR 2024
Guided backprop
Integrated Gradient
Gradient
Grad x Input
LRP,-
SRD(Ours)
Original Image
Original
Target
Target Image
Manipulated
Guallent
Gralltgst
Gubded
InteGrad
LRP
Guallest
Gralingpot
Gildedd
InteGrod
LRPy
SED
Backppop
(pom))
Bacipeop
Figure 4: Adversarial attack experiment. Top: Qualitative comparison between explanations. While other methods deleted the goldfish (original image) in their explanation due to the manipulation, our method successfully retained the goldfish part. For more results, see Appendix H.2 Bottom: Quanti- tative result. Higher SSIM and PCC scores indicate less susceptibility to perturbation manipulation. In both SSIM and PCC, our method demonstrates superior defense against adversarial attack.
CONCLUSION
We propose novel method, Sharing Ratio Decomposition (SRD), which analyzes the model with Pointwise Feature Vectors and decomposes relevance with sharing ratios. Unlike conventional ap- proaches, SRD faithfully captures the model's inference process, generating explanations exclu- sively from model-generated data to meet the pressing need for robust and trustworthy explanations. Departing from traditional neuron-level analyses, SRD adopts vector perspective, considering non- linear interactions between filters. Additionally, our introduction of Activation-Pattern tion (APOP) brings attention the often-overlooked role of inactive neurons shaping model behavior.
Predic-
In our comparative and comprehensive analysis, SRD outperforms other saliency map methods across various metrics, showcasing enhanced effectiveness, sophistication, and resilience. Espe- cially, it showcases notable proficiency in robustness, withstanding both random noise perturbation and targeted adversarial attacks. We believe that this robustness is consequence of our thorough reflection of the model's behavior, signaling a promising direction for local explanation methods.
Moreover, through the recursive decomposition of Pointwise Feature Vectors (PFVs), high-resolution Effective Receptive Fields (ERFs) at any layer. With this, we would be able to gen- erate comprehensive exploration from local to global explanations in the future. Furthermore, will go beyond answering where and what the model looks importantly to providing insights into how the model makes its decision.
derive
ACKNOWLEDGEMENT
This work was supported NRF (202IRIA2C300665) and HTT (2021-0-02068, 2021-0-01343) grants, all of which were funded by Korea Government (MSIT). was also supported by Institute Seoul National University (AIIS) in 2023.

---

Published as a conference paper at ICLR 2024
REFERENCES
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. Advances in neural information processing systems, 31, 2018.
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks. International Conference on Learning Representatiinn//CLRR) 2018.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus- Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer- wise relevance propagation. PloS one, 10(7):e0130) 0000)400, 30140, 2015.
Umang Bhatt, Adrian Weller, and José M. F. Moura. Evaluating and aggregating reature-based model explanations. In Christian Bessiere (ed.), Proceedings of the Twenty-Nint International Joint Conference on Artificial Intelligence, IJCAI-20, pp. 3016-3022. International Joint Confer-
ences on Artificial Intelligence Organization, 2020.
Alexander Binder, Leander Weber, Sebastian Lapuschkin, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. Shortcomings of top-down randomization-basee sanity checks for evaluations of deep neural network explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16143-16152, 2023.
Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. Concise expla- nations of neural networks using adversarial training. In International Conference Machine Learning, pp. 1383-1391. PMLR, 2020.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter conference on applications of computer vision (WACV), pp. 139-847. IEEE, 2018.
Kat-Kit Dombrowski, Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus- Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry is to blame. Advances in neural information processing systems, 32, 2019.
Finale Doshi- Velez and Been Kim. Towards rigorous science of interpretable machine learning.
arXiv preprint arXiv: / 7702.08608 2017.
Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 2711 2023.
Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom-based grad-cam: Towards accurate visualization and explanation of cnns. In 31st British Machine Vision Conference 2020, BMVC 2020, Virtual Event, UK, September 7-10, 2020, 2020.
Shanghua Gao, Zhong- Yu Li, Ming-Hsuan Yang, Ming-Ming Cheng, Junwei Han, and Philip Torr. Lcrge-scale unsupervised semantic segmentation. IEEE transactions on pattern analysis and machine intelligence, 2022.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 3681-3688, 2019a.
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations. Advances in neural information processing systems, 32, 2019b.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Anna Hedström, Leander Weber, Daniel Krakowezyk, Dilyara Bareeva, Franz Motzkus, Wojciech Samek. Sebastian Lapuschkin, and Marina M-C Höhne. Quantus: An explainable ai toolkit for responsible evaluation neural network explanations and beyond. Journal of Machine Learning Research, 24(34):1-11, 2023. 2023.
10

---

Published as a conference paper at ICLR 2024
Peng-Tao Jiang. Chang- Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Pro- cessing, 30:5875-555 5888, 2021.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp. 2668-2677. PMLR, 2018.
Maximilian Kohlbrenner, Alexander Bauer, Shinichi Nakajima, Alexander Binder, Wojeiech Samek, and Sebastian Lapuschkin. Towards best practice in explaining neural network decisions with Irp. In 2020 International Joint Conference on Neural Networks (IJCNN) pp. 1-7. IEFE, 2020.
Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. Understanding the effective receptive field in deep convolutional neural networks. Advances in neural information processing systems, 29, 2016.
Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Rohert Müller. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern recognition, 1-22,, 2017.
Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojeiech Samek, and Müller. Layer-wise relevance propagation: an overview. Explainable Al: interpreting, explaining and visualizing deep learning. pp. 193-209, 2019.
Harish Guruprasad Ramaswamy et al. Ablation-cam. Visual explanations for deep convolutional network via gradient-free localization. In proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 983-9991 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. Yoshua Bengio and Yann LeCun (eds.), International Conference on Learning Representations((CLRR) 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Confer- ence on Learning Representations( 2014.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda B. Viégas, and Martin Wattenberg. Smooth- grad: removing noise by adding noise. CoRR, abs/1706.00 706.03825. 2017.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In Workshop at International Conference on Learning Rep- resentations//CLRR)) 2015.
Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. Advances in neural information processing systems, 32, 2019.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 24-25, 2020.
Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126(10):0044-1 102, 2018
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2921-2929, 2016.

---

Published as a conference paper at ICLR 2024
A
FUTURE WORKS: GLOBAL EXPLANATION WITH SRD
fluffy
head
Wo neighoors
ERFoRe
CAT
|uunco bird'
Rpaee fluffy body
Why the prediction was 'uunco bird'? Because..
Why ((i,) was 'llffy bird head'? Because..
Why (atoo was "beak?? Because.
fluffly head
Ny wos YRufty biid head
Kn waa 'ied head
H0S
Plkap "ooak?
ih was fluffy heid body'
fluffy bird bady
Get eite
"Dazyy WAs grau badiground
o Smmlll arimaal head
Figure 5: Top: Nearest neighbor PFVs encode similar concepts to that of the target PFV. By la- beling each PFV with its ERF, we emphirically observed that the local manifold near certain PFV encodes a concept. For example, to know what 2.29
(4.9) encodes, we find its top 3 nearest neighbors from other samples' PFVs. Bottom: Recursive global explanation to explain the decision-making process of the model. Given modified sharing ratio, !/=200 we know how much a certain concept of PFV of at layer L contributed to output. For example, V(4.9) is PFV of (4, 9) in layer 29 which [(1,9)++c of the total prediction. .9 (1.9) is formed by the subconcepts of [v([,9; "bird head", v/27 (((.10)) "beak", ...., (3.100) "small animal head"), whose
represents "fluffy bird head" concept contributed L-0
27
27-229 These 'subconcepts' can be further decomposed into minor concepts contributions are (4.9
recursively, revealing the full decision-making process of the deep neural network.
Local Explanation methods explain where the model regards important for classification, and global explanation methods (Kim et al.) 2018 Ghorbani et al., (2019b; Fel et al.) (023) explain what it means. However, with our method, SRD, we even go further to explain how the model makes its decision.
In short, we can provide how the model predicted along with where the model saw and what meant. Through empirical observation, by labeling the Pointwise Feature Vector (PFV) with Ef- fective receptive field (ERF), we discerned that each PFV encodes a specific concept. While there numerous sophisticated global explanation methods available, for clarity, opted for more straightforward approach: examining the nearest neighbors of a given PFV. By observing its closest neighbors, discern the meaning of the target PFV (Top of Figure 5.
Furthermore, by analyzing the sharing ratio of a PFV, we gain insights into how each subcon- cept compnnnmss of the target PFV
target PFV (Bottom of Figure
This recursive application allows SRD to thoroughly illuminate the model's 's decision-making pro- cess. (igure an example of this attempt and gives a hint on how decision is made in model. Detailed research on the global explanation of SRD will be dealt with in our next paper.
12

---

Published as a conference paper at ICLR 2024
B DETAIL DESCRIPTION OF AFFINE FUNCTION fl
In this section, we describe how to calculate affine function li= in Eq. 6
Convolutional layer Each PFV in CNN is transformed linearly by the convolutional layer and then aggregated with bias term. We regard PFV of convolutional layer linear combination of PFVs in the previous layers in addition to the contribution of the bias vector. For example, consider a convolutional layer with a kernel E RC xCx where C'and C' are the number of output and input channels, and h and are the height and width of the kernel, respectively. The affine function of one convolutional layer Ji is defined as:
(i)((!)) winset + Exenr, 1
(11)
where Wi-+j E RC'CC the size of RF, w.
Pooling layer The average pooling layer computes the average of the PFVs within the receptive field. As a result, the contribution of each PFV is scaled down proportionally to the size of the receptive field. On the other hand, the max pooling layer performs channel-wit selection process, whereby only subset of channels from of is carried forward to e/+1, This achieved by clamping the non-selected u contribution to zero:
(1,( 1 I (.a) ② v.
(12)
Here, m IRC is the indicator function that outputs only for the maximum index among the receptive field, RF,, which can be easily obtained during inference and O denotes the elementwise multiplication. Thus, given information from inference, can consider max pooling as linear function, زi-jj whose coefficients are binary (0/1).
Batch normalization layer Additionally, for batch normalization layer, manipulate each PFV in direct manner by scaling it and adding a batch-norm bias vector to it, without resorting to any intermediate representation.
Multiple functions If there are multiple affine functions between of and w/ composite mul- tiple affine function along possible paths. For example, if there are max pooling layer and convolu- tional layers together, the resulting fi-.j would be:
pi-== = E 2 W/
(13)
where g' is affine function of max pooling layer and h رداا is affine function of convolutional layer.
C PROOF OF EQUIVALENCE BETWEEN FORWARD AND BACKWARD PROCESSES
Forward process: Given that the saliency map for class being
Φc(x) = ΣnT ERF
(14)
and for each layer 7 we have
ERE 11
Σ 1-4 ERF
(15)
ERF,2 can be broken down as follows:
E.F.f
Σ
(1-1)-L Σ
Mk-zj
(L-2)
Σ
po ERF
(16)
JERF,
KERF,
PERF,
13

---

Published as a conference paper at ICLR 2024
This can be generalized as
ERE
E EI
(-1)-))) Ppt 1 E.,
(17)
pe[mw] TETTE[T]
where 7 = (Po- P.P1... PL-11PP 2) is trajectory (path) from pixel in an input image p to a pixel in the last layer L, and T denotes the set all the trajectories. Note that H and W are the height and width of input image and invalid trajectories have least one zero sharing ratio their path, i.e, j = 0 for some layer.
From Eq. 17 (x) becomes
(r(x) = EΣ 1-0
Σ TET [E(1]
( E..
(18)
Backward process: The saliency map (x(x) is defined
(x = Σ Rr E..
(19)
where
R
Σ
Pi-j
(1-1)=1
(20)
jEPR
Thus, Rg becomes
R
Σ
[ Σ
Σ iEPF,
M
(b-1))-b, R())).
(21)
JEPF,
kEPF;
This can be generalized as
RØ = Σ
R Σ
II
(1-1)-1)
(22)
PI-1
TE IE(L]
Since R 14-=0 and (c(x)) Ep RD E.,
(c(x)= Σ ΣEmn
1
Σ
II
(-1)=1) F.,
(23)
TE [E[L]
which is identical to Eq. 18
D MORE RESULT OF APOP
We made an interesting observation during our experiments, which we term Sndivation-Patten--nlll Prediction (APOP). This phenomenon was discovered by conducting series of experiments where a model made predictions with an image input. Subsequently, the model retained the binary on/off activation pattern along with its corresponding label (Algorithm [ Following this, the model made a prediction once more, but this time with an entirely different input (i.e. zeros, ones) while keeping the activation pattern frozen.
All of our APOP experiments were conducted on the ImageNet validation dataset. We conducted experiments under three different input conditions: 'zeros", 'ones'. and 'normal'. The 'zeros' set- ting is the experiment introduced in the main paper (T. In "ones" setting, we predicted again with matrix with ones instead of empty matrix. 'normal" setting, matrix filled with normal dis- tribution (0,, 1) was used. As shown in Table 3 all settings achieved higher accuracy compared to random prediction baselines 0.001 for Top-1 accuracy and 0.005 for Top-5 accuracy. Espe- cially, it is intriguing that it achieved almost the same accuracy with the original accuracy in APOP & ReLU setting, supporting our idea that activation pattern crucial component in explanation, complementing the actual values of the neurons.
We carried out an additional experiment: Particular Layer Activation Binarization as illustrated in Figure 6 Instead of entirely freezing the activation pattern, replaced the activation value of
14

---

Published as a conference paper at ICLR 2024
Top-1 acc.
Top-5 acc.
Model
Inpat
Oti.
APOP
APOP
Ori.
APOP
APOP
relu
relu
zeros
0.5447
0.6603
0.7870
0,8716
VGG13
ones
0.6790
0.4901
0.6394
0.8828
0,7390
0.8707
normal
0.2510
0,8702
0.5754
0.6847
0,8094
0.8875
VGG16
0.6980
0,5268
0.6837
0.8940
0.7665
0.8870
normal
0.2903
0.6854
0.4979
0.8871
zeros
0.5937
0.8226
0.8947
VGG19
0.7052
0,5462
0.6937
0.8981
0.7801
0.8938
normal
0.2956
0.6957
0.5067
0.8923
Zeros
014871
0.6404
0.7340
0.8595
Resnet18
0.6707
0.4813
0.6408
0.8769
0.6750
0.8593
normal
0.2518
0.6375
0,4594
0.8598
2er0s
0,5578
0.6917
0,7906
0.8910
Resnet34
ones
0.7113
0,5102
0.6902
0,9009
0,7456
0.8907
normal
0.3194
0.6918
0,5380
0.8917
zeros
0.5690
0.7328
0.7943
0,9148
Resnet50
ones
0.7446
0,5198
0.7334
0.9183
0.7527
0,9141
normal
0.3035
0.7366
0.5066
0.9168
0.5601
0.7459
0.7853
0.9231
Resnet 101
ones
0.7560
01,1151
0.7431
0.9280
0.7433
0.9228
normal
0.3276
zeros
0.8260
Resnet 52
0.7696
0,5585
0.7592
0.9359
0.7785
0,9300
normal
0.3561
0.7618
0.5666
0,9319
Table 3: Additional results of APOP. Ori. the original model, while APOP is the where activation pattern of each activation layer is replaced (yet, the value of neurons can be negative). APOP & relu is the setting where after activation pattern replaced, neurons calculated with relu layer (every neuron has non-negative value, while preserving information of the activation pattern).
particular layer into 1 or O; if the activation value was greater than O, then it was set to 1, otherwise, it was set to O. Remarkably, even under this setting, model predicted more accurately than random guessing. It happened even when this binarization occurred in the very first activation layer. This experiment reinforces our notion that the activation pattern holds comparable significance to the actual neuronal values.
MND
OODEL S
Figure 6: APOP Particular Layer Activation Binarization. Target layer's 's activation was replaced with its binary version. In all layer, it achieved higher than random guessing baseline.
15

---

Published as a conference paper at ICLR 2024
Algorithm APOP process in PyTorch pseudocode
import torch
import torch.nn
import torch. nn. functional
class CustomReLU (nn.
(selt,
output # F. (x)el((x)
self.mask
self mask torch..iggnoutteeeee.
make binary mask
return output
def APOP_forwer forward (self,, x)
output return output
self mask inactive
class CustomMaxPool12d
def (self, MaxPoo12d):
output, return output def forwff
output pedice_ppoll (x, self, mask indices)
mask inactive neuron with saved mask indices
output
total total_sample =
original correct predictions
APOP correct predictions
model CustomModel (model) # replace ReLU and Maxpool into CustomReLU and CustomMaxPoo12d empty_input torch.
data, labels wech..eerss__a loos_likeedatee
loader:
original predictions CustomModel (x) predict original prediction and save masks APOP predictions omModel APOP foodor (empty inpptt) # APOP with saved masks original correct predictions www.rret_.reeicctiiee...
compute. accuracy (original predictions, labels) APOP correct predictions += compute accpurac (APOP pradictions, labels) total_samples +- labels. aize
original model accuracy original correct predictions / total sample APOP_mes model _oder_accuuaccy APOP_ apop_correct_ppee perrect_predicctions notal_sample
DETAIL OF METRICS
Pointing Game (1) (Zhang al. 2018) evaluates the precision of attribution methods by assessing whether the highest attribution point is on the target. The groundtruth region is expanded for some margin of tolerance (15px) insure fair comparison between low-resolution saliency map and high- resolution saliency map. Intuitively, the strongest attribution should be confined inside the target object, making a higher value for more accurate explanation method.
Hits
MPG
(24)
Hits + Misses
Attribution Localization (1) (Kohlbrenner et al. 2020) measures the accuracy of an attribution method by calculating the ratio. (AA,, between attributions located within the segmentation mask and the total attributions. A high value indicates that the attribution method accurately explains the crucial features within the target object.
Rin. Riot
PAL
(25)
where /AAL is inside-total relevance ratio without consideration of the object size. Ra, is the of positive relevance in the bounding box, Reot is the total sum positive relevance in the image.
Sparseness (1) (Chalasani et al. 2020) evaluates the density of the attribution map using the Gini index. A low value indicates that the attribution is less sparse, which may be observed in low- resolution noisy attribution maps.
((k) d - k 0.5 |v]l1
ASpa 1
(26)
d
where is flatten vector of the saliency map ((x)
Fidelity (1) (Bhatt et al. 2020) measures the correlation between classification logit and attributions. Randomly selected 200 pixels replaced value of 0. The metric then measures the correlation
16

---

Published as a conference paper at ICLR 2024
between the drop in target logit and the sum of attributions for the selected pixels.
Mrid
Corr
Σ VES
(()),, - (1)m.
(E(
where F is the classifier, ((x) the saliency map given
Stability (1) (Alvarez Melis Jaakkola 2018) evaluates the stability of an explanation against noise perturbation. While measuring robustness against targeted perturbation (as discussed in Section 4.1) can be computationally intensive and complicated due to non-continuity of some attribution meth- ods, weaker robustness metric is introduced assess stability against random small perturbations. This metric calculates the maximum distance between the original attribution and the perturbed at- tribution for finite samples. low stability score is preferred, indicating consistent explanation under perturbation.
[1((;) - 6(12)112
PSaa
max E (m.)
100 xzحl2
(27)
where No (1ry) is gaussian noise with standard deviation 0.1. all of the metrics measure after clamping the attributions to [-1,1], as all the attrubution methods are visualized after clamping.
F ABLATION STUDY
To clarify the gains of our method, we conducted an ablation study for each factor (Figure 7 The scalar- based approach with our method can be regarded as LRP-0 (Bach et al. 2015). Next to it, we showcased the generated explanation when calculating the relevance with post-activation values. As you can see, compared to ours (SRD), the generated explanations with scalar are very noisy, while those with post-activation values are too sparse. With our observation APOP, we have proven that we should consider every information including active and inactive neurons. This is the reason that we used vectors as our analysis unit and pre- activation values to propagate our relevance.
G APPLICATION TO VARIOUS ACTIVATIONS
Activation
ReLU
ELU
LeakyReLU
Swish
GeLU
Tanh
GuidedBackprop
0.064
0.025
0.001
0.015
0.030
0.028
GradInput
-0,010
-0.007
-0.005
-0.024
-0.004
-0.006
InteGrad
0.006
0.015
-0.001
-0.008
-0.007
0.014
LRP=*
0.039
Smoothgrad
-0.012
0.026
-0.014
-0.023
-0.009
-0,017
Fullgrad
0.038
0.209
0.029
0.171
0.095
0.107
GradCAM
0.005
-0.014
-0.004
0.042
-0.001
0.002
ScoreCAM
0.013
0.052
0.031
0.061
0.010
0.017
AblationCAM
0.020
0.024
0.003
0.015
0.033
0.012
XGradCAM
0.007
0.011
0.018
0.028
0.012
0.017
LayerCAM
0.021
0.042
0.012
0.018
0.007
-0.001
SRD(Ours)
0.078
0.214
0.065
0.194
0.128
0.115
Table 4: Fidelity results on various activation functions. We evaluated the fidelity metric of ResNet50 in CIFAR-100 with different activation functions: ReLU, ELU, LeakyReLU,, Swish, GeLU, and Tanh. Our method, SRD, achieved highest performance on every activation function. The model accuracies with each activation varient were as follows: 0.780 for ReLU, 0.746 for ELU, 0.785 for LeakyReLU, 0.756 for Swish, 0.767 for GeLU, and 0.685 for Tanh.
Most of the existing methods have been limited to ReLU or have had to be redesigned for other activations. However, as in Fig. and Tab. 4 SRD can be applied to various activations due to the utilization of preactivation, while maintaining high fidelity.
17

---

Published as a conference paper at ICLR 2024
H ADDITIONAL SALIENCY MAP COMPARISON
H. SALIENCY MAP COMPARISON
Fig. 18
are some examples that compare the saliency maps of different methods.
H.2 EXPLANTION MANIPULATTION COMPARISON
Fig. 19 23 examples that compare explanation manipulation of different methods.
18

---

Published as a conference paper at ICLR 2024
Original Image
Scalar
Vector
SRD
Scalar
Vector
SRD
Figure 7: Ablation Study on ResNet50 (Left) and VGG16 (Right). We firstly generated the saliency maps with a neuron (scalar) rather than a vector as an analysis unit. And then, we analyzed with vectors, yet by using postactivation values. Lastly, utilized vectors and pre-activation values, which is our method.
19

---

Published as a conference paper at ICLR 2024
ReLU
ELU
GeLU
LeakyReLU
Swish
tanh
SRD(Ours)
Inputxgrad
GBP
Smoothgrad
Saliency
Original Image
Integrad
Fullgrad
AblationCAM
GradCAM
LayerCAM
ScoreCAM
XgradCAM
Figure 8: Qualitative results applied to various activation functions. Here, even with various activa- tions, SRD generates the most fine- fine-grained and feasible explanation maps.
20

---

Published as a conference paper at ICLR 2024
Label: 'Sulfur butterfly'
AblationCAM
XGradCAM
layerCAM
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP,
SRD(Ours)-low
SRD(Ours)-high
SRDIOurs)
Label: : 'bblllpoint
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP,+
SRDYOurs)-low
SRD(Ours)-high
SRDiOurs) input
Figure Qualitative comparison on VGG16. The highlighted region the segmentation mask.
21

---

Published as a conference paper at ICLR 2024
Label: 'Containership'
GradCAM
GradCAM+
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP,*
SRD(Ours\-low
SRD(Ours)-high
SADIOurs),
Label: :bulliet train'
GradCAM
GradCAM +
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrrd
InteGrad
Smoothgrad
GuidedBackprop
LRP,*
SRD(Ours\-lw
SRD(Ours)-high
SRDIOurs), input
Figure 10: Qualitative comparison on VGG16. The highlighted region is the segmentation mask.
22

---

Published as a conference paper at ICLR 2024
Label: 'red fox
GradCAM
GradCAM+
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
++
SRD(Ours)-ow
SRD(Ours]-high
SRD(Ours) Input
GradCAM+
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
+
SRD(Ours)
SRDiOurs)-high
SRD(Durs) innutt
Figure 11: Qualitative comparison on VGG16. The highlighted region is the segmentation mask.
23

---

Published as a conference paper at ICLR 2024
Label:: 'goff
GradCAM
GradCAM++
AblationCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP
SRD(Ours)-low
SRD(Ours)-high
SRDIOurs) Input
Label: 'padlock'
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP,*
SRD(Ours)-low
SRD(Ours)
SRD(Ours) inputt
Figure 12: Qualitative comparison on VGG16. The highlighted region is the segmentation mask.
24

---

Published as a conference paper at ICLR 2024
Label: 'lemon'
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
RullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP
SRD(Ours)-low
sRD(Ours)-high
SRDIOurs) Input
Label: 'tiger shark
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-hin
FullGrad
InteGrad
Smoothgrad
GuidedBackprap
+
SRD(Ours)-low
SRD(Ours)-high
SRD(Ours), input
Figure 13: Qualitative comparison on VGG16. The highlighted region the segmentation mask.
25

---

Published as a conference paper at ICLR 2024
Label: : 'sulfur butterfly'
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM.low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP=*
SRD(Ours)-low
SRD(Ours)-high
SRD(Ours] inputt
Label: 'balppoint
GradCAM
+radCAM+
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
Smoothgrad
GuidedBackprop
LRP_+
SRD(Ours)-low
SRD(Ours|-high
SRD(Ours) input
Figure 14: Qualitative comparison on ResNet50, The highlighted region the segmentation mask.
26

---

Published as a conference paper at ICLR 2024
Label: 'containeeshipp
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM.low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP=*
SRD(Ours)-low
SRD(Ours)-tigh
SRD(Ours) input
Label: 'bullet :bbllet train'
GradCAM
GradCAM+
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP_+
SRD(Ours)-low
SRD(Oursi-high
SRD(Ours) Input
Figure 15: Qualitative comparison on ResNet50, The highlighted region the segmentation mask.
27

---

Published as a conference paper at ICLR 2024
Label: 'red
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM.low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP_* +
SRD(Ours)-low
SRD(Ours)-high
SRD(Ours] input
Label 'Siamese cat'
GradCAM
+radCAM+
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
Smoothgrad
GuidedBackprop
LRP1+
SRD(Ours)-low
SRD(Oursi-high
SRD(Ours) input
Figure 16: Qualitative comparison on ResNet50, The highlighted region is the segmentation mask.
28

---

Published as a conference paper at ICLR 2024
Label: : 'golf ball'
GradCAM
GradCAM++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
Smoothgrad
GuidedBBackprop
SRD(Ours) low
SRD(Ours) high
SRD(Ours) input
Label: 'padlock
GradCAM
+
ScoreCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP,-
SRD(Ours]-low
SRD(Ours)-high
SRD(Ours), input
Figure 17: Qualitative comparison on ResNet50, The highlighted region the segmentation mask.
29

---

Published as a conference paper at ICLR 2024
'lemon"
GradCAM
GradCAM+++
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
FullGrad
InteGrad
Smoothgrad
GuidedBackprop
LRP
SRD(Ours))low
SRD(Ours)-high
SRD(Ours) input
Label: 'tiger shark'
GradCAM
GradCAM+
ScoreCAM
AblationCAM
XGradCAM
LayerCAM-low
LayerCAM-high
fullGrad
InteGrad
Smoothgrad
GuidedBackprap
LRP
SRD(Ours)-low
SRD(Ours\-high
SRD(Ours) input
Figure 18: Qualitative comparison on ResNet50, The highlighted region the segmentation mask.
30

---

Published as a conference paper at ICLR 2024
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP Z+
SRD(Ours)
Original Image
Original
Target
Target Image
Manipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target
Target Image
Mamipulated
Gradient
GradInput
GuidedBackprop
InteGrad
LRP_ze
SRD(Ours)
Original Image
Original
Target
Target Image
Manipulated
Figure 19; Additional results explanation manipulation comparison.
31

---

Published as a conference paper at ICLR 2024
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP Z+
SRD(Ours)
Original Image
Target
Target Image
Manipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target Image
Manipulittdd
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP_zz
SRD(Ours)
Original Image
Original
Target
Target Image
Manipultted
Figure 20: Additional results explanation manipulation comparison.
32

---

Published as a conference paper at ICLR 2024
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP Z+
SRD(Ours)
Original Image
Original
Target
Target Image
Manipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target Image
Mamipulated
Gradient
GradInput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
original Image
Target
Target Image
Manipulated
Figure 21: Additional results explanation manipulation comparison.
33

---

Published as a conference paper at ICLR 2024
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP Z+
SRD(Ours)
Original Image
Target Image
Manipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target Image
Manipulattdd
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target
Target Image
Manipulated
Figure 22: Additional results explanation manipulation comparison.
34

---

Published as a conference paper at ICLR 2024
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Target
Target Image
Manipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Original
Target
Target Image
Mamipulated
Gradient
Gradinput
GuidedBackprop
InteGrad
LRP z+
SRD(Ours)
Original Image
Original
Target
Target Image
Manipulated
Figure 23: Additional results explanation manipulation comparison.
35


Reviews:
[{"review_id": "iM1Dbh4aou", "reviewer": "Reviewer_aoME", "summary": "In this paper, the authors propose an explanation method for deep models called Sharing Ratio Decomposition (SRD) that pursues enhanced precision and robustness of its explanation. In extensive simulations, the method shows a strong and robust explanation performance across various kinds of metrics beyond the prior XAI methods. The main contribution of this work is to widen the view of XAI that should be pursued for achieving a reliable and robust explanation of deep model inference.", "questions": "**Q1:** In the 'Weakness 1' issue, would you provide a stepwise description of the methodology?\n\n**Q2:** In the 'Weakness 2' issue, would you clarify the main reason of the gains of SRD?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**Strength 1:** Explainable AI (XAI) is one of the most important research topics that directly tackle the explanation issue of the current deep learning models. Beyond the better explainability, the authors raise the multiple objectives that XAI should pursue, i.e., Localization, Complexity, Faithfulness, and Robustness, and argue that the proposed XAI, called Sharing Ratio Decomposition (SRD), is superior to the prior XAIs in the 'desiderata'. I am strongly convinced that the future XAI should consider an efficient, effective, precise, and robust explanation by pursuing the desired factors. The comprehensive experimental results in Table 2 clearly show the strength of the proposed XAI method, which is SRD.\n\n**Strength 2:** A lot of qualitative results in the main paper and the supplementary truly help readers visually understand the benefits of SRD. These results clearly contrast SRD to the prior methods. I believe that efforts to show these qualitative results are very important to visually identify the explainability that often relies on the human-concept level evaluation of how well XAI explains the model's inference.\n\n**Strength 3:** A bunch of prior works are clearly compared to the proposed XAI method via extensive simulation results (as shown in Table 2). I expect that the evaluation effort will promote extensive comparison with diversified viewpoints, i.e., Poi., Att., Spa., Fid., and Sta., in assessing future XAIs.", "weaknesses": "**Weakness 1:** The '3. Method' part needs to be improved for better presentation. For readers who are a beginner in the XAI research field, I feel that a compact and clear description such as a pseudocode-style presentation or stepwise procedure explanation (e.g., 1. Feedforward an input, 2) Calculate PFV, 3) Decompose PFV, 4) Compute sharing ration through backward propagation, 5) Relevance is computed), would definitely help readers to understand the way that the method works. In this version of the article, the description is a quite verbal explanation.\n\n**Weakness 2:** The key reasons behind the gains of SRD are not clearly analyzed. In this paper, the authors have separately pointed out the uniqueness of their method in multiple parts of the paper. However, I cannot fully understand the essential factors that make SRD be superior to other prior XAI methods. Is it due to the vector perspective, consideration of inactive neurons, decomposition of PFV, or any combinatorial effect of multiple factors?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uqGNfM5Mva", "reviewer": "Reviewer_z79h", "summary": "This paper proposes an XAI method called Sharing Ratio Decompostion at vector level. It can reflect the inference process at the vector level instead of neuron level. It also proposes a new observation method called APOP which can highlight the influence of the inactive neurons. The result on ImageNet dataset reach outstanding performance compared to SOTA.", "questions": "Questions can be seen in weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The quantitative experiment result is good.\n2. APOP is a novel observation method to highlight the influence of inactive neurons and easy to follow.\n3. The deduction of SRD on forward and backward pass is rigorous.", "weaknesses": "1. Compared SOTA is limited and all methods were proposed before 2021. There are many CAM based methods in these two years.Why do not compare to the methods in these two years?\n2. In equation 4, it mentions that the summation of modified sharing ratio is not 1. So will it cause some problem? It seems better to normalize ratio to 1 further.\n3. The metrics used in experiment is quite different to other XAI methods. Is there any consideration in that?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D9iPcjdcev", "reviewer": "Reviewer_WQ6F", "summary": "The authors introduce a novel interpretability method in this paper. Their approach differs from the past neural perspectives and starts from a vector perspective, leveraging gradient information and intermediate model output propagation computations. They attribute the input images based on the model’s inference process and explain the algorithm’s computation process from both forward and backward equivalent processes.", "questions": "1. The paper's experiments should include testing on the larger ImageNet validation set for better verification.\n2. The APOP, although interesting, doesn't seem connected to the motivation that focus on interpretability.\n3. The paper should consider conducting experiments with different backbone architectures to strengthen its findings.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  The authors propose a new perspective with finer granularity, attributing based on vectors as units.\n2.  The authors conducted experiments from aspects such as Localization, Complexity, Faithfulness, and Robustness based on previously proposed desiderata of interpretability, demonstrating the effectiveness of the algorithm.\n3.  Additionally, the authors introduce an interesting observation worth further research, Activation-PatternOnly Prediction (APOP), indicating that retaining only the activation states in the network can still preserve the model’s inference capability to a certain extent.", "weaknesses": "1.  The experiments in this paper are all based on the ImageNet-S50 dataset, which only contains 752 samples. To better verify the effectiveness of the algorithm, it would be more persuasive to test on the ImageNet validation set like perturbation[1] and deletion & insertion[2] experiments do.\n 2.  From my personal point of view, the APOP phenomenon is a very interesting and worthwhile topic for further research. However, I didn’t quite grasp how the APOP phenomenon helps with attribution in the model’s inference process. That is, this section doesn’t seem to be tightly connected with the paper’s main focus, the interpretability algorithm.\n 3.  The experiments in this paper are only based on VGG and ResNet backbones. Perhaps conducting experiments on backbones of different depths and comparing the algorithm’s performance on shallow layers and deep layers would make the paper more convincing.\n\n[1] VU MinhN, NGUYEN TrucD T, PHAN N, et al. Evaluating Explainers via Perturbation[J]. arXiv: Learning,arXiv: Learning, 2019.\n\n[2] PETSIUK V, DAS A, SAENKO K. RISE: Randomized Input Sampling for Explanation of Black-box Models[J]. arXiv: Computer Vision and Pattern Recognition,arXiv: Computer Vision and Pattern Recognition, 2018.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7ndK4PGVgw", "reviewer": "Reviewer_HQ8j", "summary": "The authors propose a novel eXplainable AI (XAI) method called SRD (Sharing Ratio Decomposition), which sincerely reflects the model’s inference process, resulting in significantly enhanced robustness in our explanations. Through experimental validation, this approach has yielded an intriguing result.", "questions": "In the experiments, it is not clearly explained why the input resolutions vary between different methods. Furthermore, it's not clear how using different resolutions as inputs ensures the fairness of the experiments.\n\nThe existing methods have deviated from faithfully representing the model, making them susceptible to adversarial attacks. The author did not provide relevant discussions on this issue.\n\nAll the comparative methods used in the experiments are from 2014 to 2021, but it's not clear why more recent methods were not included.\n\nThe choice to target the Conv5_3 layer in VGG16 and the avgpool layer in ResNet50 is not adequately justified. Whether adjacent layers would have a significant impact is not addressed.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The writing structure is well-organized, the language is fluent, and it exhibits strong readability. The inclusion of rich visualizations and thorough experimental validation adds to its overall quality. In the scheme design, different from the conventional emphasis on the neuronal level, they adopt a vector perspective to consider the intricate nonlinear interactions between filters.", "weaknesses": "For the entire paper, the organization of the content is not very reasonable. There is limited coverage in the experimental description, whereas the related work section is extensive. The results in the experiments need more in-depth analysis and discussion.\n\nAll evaluations are performed on the ImageNet-S50 dataset, and it would be better to conduct evaluations on multiple datasets to better demonstrate the proposed method's generalization capabilities.\n\nThe experimental comparisons and performance improvements lack strong persuasiveness, and the conclusion section of the paper is lengthy and somewhat verbose.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)