Strengthening Interpretability: An Investigative Study of Integrated Gradient Methods
Authors: Shree Singhi
Anupriya Kumari
Venue: Accepted by TMLR ()

Abstract: We conducted a reproducibility study on Integrated Gradients (IG) based methods and the Important Direction Gradient Integration (IDGI) framework. IDGI eliminates the explanation noise in each step of the computation of IG-based methods that use the Riemann Integration for integrated gradient computation. We perform a rigorous theoretical analysis of IDGI and raise a few critical questions that we later address through our study. We also experimentally verify the authors' claims concerning the performance of IDGI over IG-based methods. Additionally, we varied the number of steps used in the Riemann approximation, an essential parameter in all IG methods, and analyzed the corresponding change in results. We also studied the numerical instability of the attribution methods to check the consistency of the saliency maps produced. We developed the complete code to implement IDGI over the baseline IG methods and evaluated them using three metrics since the available code was insufficient for this study. Our code is readily usable and publicly available at https://github.com/ShreeSinghi/TMLR-IDGI.

Published in Transactions on Machine Learning Research (12/2024)
Strengthening Interpretability: An Investigative Study of Integrated Gradient Methods
Shree Singhi*
Deportment of Data Science & Artificial Intelligence Indian Institute of Technology, Roorkee
Anupriya Kumari'
anaprigo kddece. ditr.ac............. in
Deportment Electronics & Commanication Engineering Indian Institute of Technology, Roorkee
Reviewed on OpenReview: https: // openrevew.. net/ per/ffrmm? id= sBmPpNN2hr5
Abstract
We conducted a reprodueibility study on Integrated Gradients (IG) based methods and the Important Direction Gradient Integration (IDGI) framework. IDGI eliminates the expla- nation noise in each step of the computation of IG-based methods that use the Riemann Integration for integrated gradient computation. We perform a rigorous theoretical analysis of TDGT and raise a few critical questions that we later address through our study. We also experimentally verify the authors' claims concerning the performance of IDGI over 1G-based varied the number of steps used in the Riemann approximation,
methods. Additionally, an essential parameter in all IG methods, and analyzed the corresponding change in results. We also studied the numerical instability of the attribution methods to check the consis- tency of the saliency maps produced. We developed the complete code to implement IDGI over the baseline IG methods and evaluated them using three metrics since the available code was insufficient for this study. Our code readily usable and publicly available at https:/ wetitttub.cm//hreeeeeeeeeeeeeeee MiareeeSingh/TTMLRIDDDC TMLR-IGGI.
1 Introduction
Deep learning models for computer vision have become increasingly integrated into several vital domains like hcalthcare and sccurity. There a surge rescarch dodicated to studying the problem of attributing the prediction of a deep network to its input features. Gradient based saliency/atributtio map approaches
(Sundararajan et al. 2017 Xu et al. 2020 Kapishnikov et al. 2021 2019 Pan al. 2021 Simonyan et al. 2013 Smilkov et al. 2017) form important category of explanation methods. One of the first works that made a notable contribution to the field of explainability and introduced a valid metric system to evaluate its results was by Kapishnikov et al. (2019). Other prominent gradient-based explanation methods include Integrated Gradients (IG) (Sundararajan et al., 2017) and its variants, Blur Integrated Gradients (BlurG) and Guided Integrated Gradients (GIG) (Xu et al. 2020 Kapishuikov et al., 2021), that have garnered considerable attention due their notable explanation performance and desirable axiomatic properties. However, basbd methods integrate noise in their attribution. Previous works (Kapishnikov al. 2021
have explored the possible origin of this attribution noise and attempted to eliminate it.
The Important Direction Gradient Integration (IDGI) framework is recent development that has tackled this issue and reported better results. The paper (Yang et al. 2023) highlights the reason behind the noise in the explanation. It proposes a framework to mathematically eliminate the components in the integration calculation that contribute to the noise in the attribution. Tt also introduced new measurement, Accuracy
All the authors contributed equally this work

---

Published in Transactions on Machine Learning Research (12/2024)
Information Curves (AIC) and Softmax Information Curves (SIC) (Kapishnkovv et al. 2019) using Multi-
Scale Structural Similarity Index Measure (MS-SSIM) to estimate the entropy of image more accurately by improving upon previously proposed metrics (Kapishnikov et al. 2019 Kancharla & Channappayya 20118) Ma et al, 2016 Odena et al. 2017). They further evaluate 11 standard, pre-trained ImageNet classifiers with the three existing IG-based mot (IG, BlurGG, and GIG) and propose one attribution assessment technique (AIC and SIC using MS-SSIM).
DenseNest221
DenseNettes
DenseNes2211
InceptionV
MobileNetV2
Reshet50V2
ResNec101V2
ResNer15222
VGG16
VGG19
Origina
16
128)
18
68
G6 1B3
Buroo
Blnn (D66
Figure Saliency maps of the existing TG-based methods and those with IDGT explaining the prediction from all models. While we cannot make any comments about IDGI's results being objectively better or worse for this instance, one can that IDGI's saliency maps tend to be more patch-like and do not highlight edges of the imput image observed without IDGI.
A detailed examination of IDGI and related works encouraged us to put forward some well-thoughi inquiries that attempt to resolve using our mathematical observations and experimental findings. The necessity of this study arose because IDGI is relatively new and under-explored work and requires testing and verification before we can judge its soundness. It was thus necessary to meticulously question the theoretical basis of the work. Moreover, the official repository of the original paper does not contain the code used to produce the complete results. During our study, we observed incompleteness in the resources and code provided the paper; for example, there was no publicly available code for weakly supervised localization methods (Xu et al. 2020 Kapishnikov al. 2019 Cong al. 2018). and ambiguity in their implementation details, specifically which dataset used to compute them. We thus identified the need to perform elaborate investigation to understand better the contributions made in the paper.
In our work, we address these concerns, and based on our study and experiments, we ask ourselves:
Q1: What the claim's validity that IDGI improves upon its baseline methods?
Q2: What are the theoretical implications of IDGI? Under what conditions is it valid?
Q3: The number of steps used in the Riemann approximation of the integral used IG-based methods is an important parameter. In most previous works, it has been overlookod, and the choice of step size remains vague. We thus ask ourselves: How does the variation in step size impact the performance of IDGI compared to the underlying IG methods?
2

---

Published in Transactions on Machine Learning Research (12/2024)
Q4: During our initial experimentation, we observed stark visual differences in the saliencies com- puted for GIG on GPU and CPU, which led us to ask whether or not IDGT affects the numerical stability of the underlying attribution method.
Our contributions and findings as follows:
We answer Q1 in Section | where we present the results obtained from the experiments to compare our results with the authors' findings. We observed that our results matched their claims for the most part, with few anomalies. We observed vital trait that is common in models that exhibit anomalies and discuss the same in Section
We answer Q2 in Section 3 Firstly, we report two errors in the illustration of IDGI provided in the original paper. We present our improved illustration of IDGI that correctly demonstrates the path corresponding to IG, GIG, and BlurIG. Secondly, we derive the expression for TJp an important term in the IDGT algorithm. While the original paper mentions the expression, how it was obtained has not been discussed. The expression's s derivation gives us insights into how IDGI's performance is affected by step size variations, which led us to ask Q3.
We answer Q3 in Scction 4.5.1 We verify the insights from the thcoretical analysis of the expression for the above-mentioned x;, term. We also vary the step size used to compute the Riemann sum for IG and BlurGG. We arrive at noteworthy conclusion: IDGI is more sensitive to step-size variation than its underlying IG method. We succossfully proved this through rigorous thcoretical analysis of how the step size and the performance of IDGI are linked. We then confirmed our theory through our experimental findings. We also observed that at a higher step size, the scores of BlurIG + IDGT are lower than those for 1G + IDGL. We address this by analyzing the IDGI algorithm and observing the variation in an image along the path for BlurIG and IG.
We answer Q4 in Section 4.5.2 the saliency map produced for an image and a compressed version of the same image.
We perform an additional experiment to quantitatively compare
We could not directly use the existing code for our study, which led us to integrate the code for IDGI provided by the authors and use the original implementations P of the authors' code for TG, GIG, and BlurIG. The code provided by the repository was not usable for reproducing results at scale. Our code utilizes high-performance computing (HPC) resources. It is ready to use and complete, facilitating easy reproduction of results for the entire dataset and models presented in the original paper. We provide the exact details regarding this in Section
This paper has been organized as follows: We begin with a brief background on the original Integrated Gradi- ents method and its variants, BlurGG and GIG, in Section 2 Readers familiar with these can skip to Section 3 where we summarize the IDGI algorithm, followed by the complete derivation of Ij as previously men- tioned, and an in-depth discussion of IDGI's sensitivity to step-size variation. We also present an improved illustration of IDGI. Finally, Section 4 presents our experimental results, including implementation details, computational requirements, and results beyond the original paper. We also use this section to describe the difficulties we faced during the study due to the need for more details on the original implementation.
2 Background
In this section, we briefly discuss the concept and state the expressions for the attribution values for IG (Sundararajan et al. 2017). BlurIG (Xu et al.) 2020), and GIG (Kapishnikov et al. 2021
Intuitively, consider the straight line path (in R2) from the baseline x', meant to represent an informationless input to x, the input in hand and compute the gradients at all points along the path. Integrated gradients are obtained by cumulating these gradients. Formally, integrated gradients are defined as the path integral
httpss//gathabb.cmm/yaggguu 1ee.ooo2222////22 Phtpps//ggthabbccm/PPA.r.r ssaleeyyyy
3

---

Published in Transactions on Machine Learning Research (12/2024)
of the gradients along the straight line path from the baseline x' to the input However, there other paths that are not necessarily straight lines that can monotonically transition between these two points, each leading to different attribution method (BlurIG and GIG).
Sundararajan et al. 2017 introduced the concept a path function. ? = (7....., 7n) 00, 1] + R" is a
smooth function that denotes a path within R from x' to H, satisfying 7(0) x' and x11) = x. Further, they defined path integrated gradients along the jth dimension for an input x obtained by integrating the gradients along the path (aa) for [0. as:
4(x) S. af((T(a)) )r((a)
@x((a) da da,
(1)
2.1 Integrated Gradients
Integrated Gradients (IG) Sundararajan al. (017]) is a path method for the straight line path specified ( = x'= x (x-x') for E [0,1]. Let J be classifier, class, and input. The output fc(#) signifies the confidence score for predicting that x belongs to class To determine feature attributions, we calculate the line integral from the reference point x' to the imput image æ within the vector field created by the model. This vector field is formed by the gradient of fe(x) with respect to the input space. The Integrated Gradient for the ith dimension of an input a is defined as follows:
(O0Ta(100(00 0% do,
(00))=
(2)
(1))(a) da
2.2 Blur Integrated Gradients
Xu et al. (2020) introduced Blur Integrated Gradients: For a given function f Rmxx [0, representing classifier and a class, let z(x, y) be the 2D imput and 2'(x,z) be the baseline. We examine the path from 2' to 2 and calculate gradients along this path. The path for IG linear in and scales the image's intensity. BlurIG, on the other hand, uses path where a Gaussian filter progressively blurs the input. The blurring path is defined by:
Blur IG
(anniGG(, (x) =
Σ
24y 2(x
m. - n)
70
m
00 72
00
The final BlurIG computation as follows:
Os Blurlll Bluri (x, y, a)
BlurIG (x,
do
da
2.3 Guided Integrated Gradients
Guided Integrated Gradients (GIG) iteratively find the integration path y (a)),) [0,1] avoid the high curvature points in the output shape of DNNs (due to which the larger-magnitude gradients from each feasible point the path would have a significantly more significant effect on the final attribution values). The new path is defined as follows:
Σ S
Of((((a)) 0%((a da.
GIG = arg min TET
(3)
0%((a) da
After finding the optimal path y GIG GIG uses it and compules the allribution values similar to IG. Formally,
(c) = [ Ofa(GGGG (a:) 0 GIG (a)
da.
(4)
(a) da

---

Published in Transactions on Machine Learning Research (12/2024)
3 Theoretical Analysis of IDGI
Regardless of whichever IG-based approach is used for calculating the attributions, the final attribution map is produced from Riemann Integration in all IG-based algorithms. Yang et al (2023) explains that each path segment has a noise direction where gradient integration with that direction has zero net contribution to the attribution scores.
They also provide an illustration to demonstrate the direction of the noise vector. We make two remarks about the original illustration of IDGI with regard how it could be mislcading:
The relationship between fe(x) and for IG portrayed as linear; however, that only true for linear models (which was not discussed in the paper). However, the relationship between and is always linear.
The scalar function fe(x) should be many-to-one; however the original illustration portrays it to be one-to-many function, which is not possible.
We present an improved illustration in Figure 2 that counters the two issues we observed.
Path
Direction
Path
Direction
Impeteal
Irperiant
Noise
fAle)
GIG
Nome
10)=
BhalG
Ohignol
Bluni
Original
4ji
Ayss
(lopat) (5%))
(dz)
Men
180
y(0) =x
Figure 2: The original illustration of IDG1 Yang et al. (2033 (Ieft), our improved illustration
We now shift our attention to the IDGI algorithm, as present in the original work by Yang discuss the mathematics leading to the IDGI algorithm for new readers: Recall the path function denoted by y discussed in Section 2 Consider the point I,- = r(a,) and the next point I5+1 = 7(()+1) on the path from reference point & to the input point æ. IG-based methods compute the gradient, g, of Se(xy) with respect to and use Riemann integration to perform dlement-wise multiplication of the gradient and the step, fj+1 Zj, which the authors refer to the original direction. Further, they refer to the direction bl as the important direction. The gradient of the function value fo at each point in space defines the conservative vector field, where infinite number of hyperplanes h exist, and each hyperplane contains all points with the same functional value. In the conscrvative vector field, separate hyperplanes never intersect, meaning each point has its projection point with regard to the other hyperplanes. For point Zj, if one moves Ij along the Important direction, there exists a unique projection point Ij, on the hyperplane hj+1 where fo(x,,)= fo(
(2023). We
The authors YYng al. 2023) then state Theorem 1: Consider function fe(œ) mapping from E2 E. Let Fj IJ+1, Tj E R be given points. The gradient of fe with respect to each point in R forms conservative vector fields, denoted as F. We define hyperplane hy the set of all points x where fo(x) = fa(2;). In this context, we assume that Riemann Integration provides an accurate estimate for the line integral of the vector field F between points. For instance, the integral from Fj to Fj can be approximated
5

---

Published in Transactions on Machine Learning Research (12/2024)
1 0falx) dr 0fc(m)) dx, (zjj 2z). Here, 2j lies the hyperplane hj, while 2ip and xz+1 lie on the
hyperplane hi+1-
This theorem asserts that the line integral of the vector field from 2j to 2s+1 is approximately equal to the line integral from žj to Tiz- This indicates that the chosen path within these specific points and hyperplanes yields similar results when integrating the function's gradient. This illustrates that while for feature, i, the value of the attribution computed from the original direction and the critical direction can be different, the change in the value of Se remains the same.
Let x a given input with target class e, f be a given classifier, [xr .......;,, [..].. be a given path from any IG-based method and g be the gradient of fa(2z) with respect z. Then, according to the IDGI Algorithm, the important direction vector of is determined as Tor and the step size fe(ss41)=fels,) The projection [ of 2y onlo the hyperplane h,+1, formed as Fjp = xy + 2 (olm))))-((()))))) has the same functional value point Ij+1, i..., fe(#)+1)= fc(r),)-
We observe that the Taylor series approximation is necessary to arrive at the expression for Typ. Assuming z lies on the given path from any IG-based method such that it is defined as Tj, 2j + c [a] the length of the projection that we wish to approximate, we now derive the expression for 2j,-
2 where is
Derivation.
J3p = xz +
fe(1z,) fc(23 +
lgl
lg]
By Taylor series approximation,
fe(#5+1== = fa(())) ~ fa(x))+ VS (c.
g
9
(e((3))) se fc(2)) + c.19l = cx fal#5+1)
Sa(rz)
9
fc(15+1) - fc(2,)
lg]
We arrive at the final expression for Tjp using the Taylor series approximation, implying that the value of fe(2, + ig) - = fe(2)+1). Hence, while Tjp and æj+1 theoretically lic on the same hyperplane, the approximated value of Тj, (used in practice) and rj+1 only approximately lie on the same hyperplane.
This analysis implics that the more accurate derivation of Hj, comes not from Theorem 1, as the authors suggest, but from the Taylor series approximation.
How does IDGI vary with step size?
As previously mentioned, the derivation for Hj, gives us insights into how IDGI's performance is affected by step size variations. The expression for c, the length of the projection, determines the validity of the Taylor series approximation. The smaller the value of c, the more valid the approximation. We observe that is directly proportional to [e( fe(xy). Here, Tj+1 and Iy are consecutive points in the path of an based method. It is easy to observe that for the same path, these two points are closer to each other for a larger number of steps (since the number of steps denotes the finite number of small picco-wise linear segments that we discretize the path between æ and æ into.) This implies that fe(#)+1) and fe(zy) are closer value. Therefore, a larger number of steps is required for a smaller c. The expression of æj, is thus directly linked to the number of steps and, thus, the step size used for the algorithm, which means that IDGI is sensitive to step size variation.
4 Experimental Methodology and Results
Before detailing the experimental setting and presenting our results, we addressed the difficulties we faced while conducting our experiments. We found that the official code for IDGI contained only the algorithmic

---

Published in Transactions on Machine Learning Research (12/2024)
implementation of the method, and the PAIR Saliency page showed the results for each method single image and a single model. Thus, the available code was insufficient to reproduce all the results of IDGI. We had to perform the following tasks ourselves,
The saliency masks had to be calculated for ten models and six methods, each followed by every quantitative metric used by the authors of IDGI and on 35K images. We thus had to optimize the code and implement batched computations to improve speed due to limited time and resources.
To run the code, we wrole scripts to aulomate calculating the saliencies on all six methods. Following this task, also ran scripts to compute the metrics, which was computationally expensive.
We had write the code ourselves to compute Insertion Scores, SIC, and AIC using MS-SSIM and Normalized Entropy since the authors had modified the original implementation and had not provided the modified code for it.
We noticed common calculations for all the IG-based methods. To minimize the computation over- head, we precomputed these results and stored them.
Original
IG
IG IDGI
GIG
GIG IDGI
BluriG
BluriG IDGI
Golden Retriver
Tiger Cat
Basketball
Soccerball
Golden Retriever
Soccerball
Figure 3: Saliency map of the existing IG-based methods and those with IDGI explaining the prediction from InceptionV3. We compare sets of saliencies for each image by laking the model's top 2 distinct classes as the predicted class. While the top class object is always more highlighted, we observe that all IG-based methods with IDGI are slightly better at highlighting each class than methods without IDGL.
4.1 Experimental Setup
We use the same baseline methods (IG, GIG, and BlurGG) as the authors' original work. Following the implementations IDGI, we also use the original implementations with default parameters the authors' code for IG, GIG, and BlrIG. We use the black image as the reference point for IG and GIG. Finally, previously mentioned, different step sizes (8, 16, 32, 64, and 128) additional experiment beyond

---

Published in Transactions on Machine Learning Research (12/2024)
the original paper to verify our hypothesis how sensitive IDGI is to step size. We also report our findings the effect of IDGI on the numerical stability of the attribution methods.
DenseNet121, 169, 201, InceptionV3,, Mo- b2 ResNet500.01,,5112 and VGG16,19. We did not use Xception due to computational constraints. Datasets. We used the same dataset as the original paper- The Imagenet validation dataset, which contains 50K test samples with labels and annotations. We also tested the explanation methods for each model on images that show that the model predicted the label correctly, which varies from 33K 39K, corresponding to different models.
Models. We use the PyTorch (1.13.1) pre-trained models:
Evaluation Metrics. We use four evaluation metrics - Insertion Score (Pan et al. 2021 Petsiuk et al the Softmax Information Curves (SIC), and the Accuracy Information Curves (AIC) Kapishnikov el. al. 2021 2019) using Normalized Entropy and the modified version of SIC and AIC with MS-SSIM as introduced the original IDGI paper. We follow the implementation details described in previous works, as in the original paper. We did not compute the three Weakly Supervised Localization metrics because, according to previous works from which Yang et al. (2023) borrowed the implementation details, we require the Imagenet segmentation dataset to compute these metrics. Three versions of this dataset exist, and neither the previous works (Xu et al. 2020; Kapishnikov et al. 2021 2019) nor Yang et al. (2023) mention which version of the dataset they used. Calculating the metrics for all three dataset. versions was not computat ionally feasible. Furthermore, none the works provide the code to calculate these metrics.
Computational Requirements. We used single NVIDIA Tesla V100 GPU with 16 GB of VRAM for reproducibility experiments. The compute time varies slightly with the model, the method, and the step size. We report the average compute time per method per model for 128 steps: computing the saliencies took approximately 9 hours, computing SIC and AIC using Normalized Entropy and MS-SSIM took 90 minutes, and computing insertion scores took 70 minutes.
4.2 Insertion Score
Metrics
Models
IG-based Methods
IG
+IGGI
GIG
+IGGI
BlurG
+IDGI
DenseNet121
.127
.374
.155
.299
.080
.281
DenseNet160
.130
.388
.122
.320
.088
.211
Insertion Score
DenseNet201
.152
.300
.177
.333
.107
.316
with Probability
Inception V3
.135
.199
.160
.399
.102
.379
MobileNedV2
.038
.149
.037
.140
.202
.211
(t)
ResNet50V2
.062
.120
.057
.207
.241
.237
ResNet101 V2
.184
.262
.213
.396
.430
.423
ResNet152V2
.197
.270
.408
.140
.377
VGG16
.049
.287
.056
.200
.039
.231
VGG19
.058
.319
.069
.233
.256
.285
DenseNet121
.144
.415
.177
.334
.091
.312
DenseNet169
.142
.488
.167
.348
.097
.315
Insertion
Score
DenseNet201
.168
.427
.198
.368
.119
.347
with Probability
Inception V3
.160
.483
.191
.464
.122
.437
MobueNetV2
.104
.364
.366
.530
.529
Ratio (t)
ResNet50V2
.152
.885
.140
.504
.581
.573
RcsNet101V2 V2
.252
.355
.291
.539
.582
.574
ResNet152V2
.266
.363
.301
.549
.187
.504
VGG16
.057
.317
.067
.225
.045
.255
VGG19
.067
.352
.082
.262
.291
.322
Table 1: Insertion Score for explanation methods using 128 steps. The claim that IDGT improves all methods for all models does not hold.
We begin by assessing the explanation approaches with the Insertion Score from prior works Paa al. 2021 Petsiuk al.). We re-wrote the available code according to the modified implementation details introduced

---

Published in Transactions on Machine Learning Research (12/2024)
in the paper YYang et al., 2023 and evaluated each of the three baselines (IG, GIG, and BlurGG across ten models. We report the insertion scores for 128 steps in Table 1
Based on our results, we find that our results mostly match the claims made in the original paper, and the better explanation method has a higher insertion score. However, we observed that for MobileNetv2, ResNet50v2, and ResNet101v2, TDGT worsens the insertion scores for BlurIG.
4.3 SIC and AIC using Normalized Entropy
Metrics
Models
IG-based Methods
+IDGI
GIG
+IDGI
BlurGG
+IDG
DenseNet121
.14
.476
.080
.384
.200
.349
DenseNet169
.141
.465
.096
.398
-200
.342
DenseNet201
.911
.884
.122
.418
.267
.387
AUC AIC
Inception V3
.195
.554
.128
.491
.277
.478
(1)
MobüleNetV2
.056
.410
.043
.313
.570
.531
ReaNet50V2
.054
.665
.048
.313
.617
.580
ResNet101 Væ
.344
.154
.693
.663
ResNet152V2
.352
.345
.151
.528
.298
.664
VGG16
.041
.369
.029
.268
.077
.311
VGG19
.052
.378
.033
.294
.412
.421
DenseNelI21
.010
.435
.005
.311
.038
.261
DenseNet169
.011
.488
.006
.346
.042
.266
DenseNet201
.027
.455
.009
.369
.104
.320
AUC
SIC
Inception V3
.527
.465
.066
.436
(1)
MobileNNt
.005
.201
.005
.121
.362
.348
ResNet50V2
.005
.075
.005
.132
.425
.408
ResNet101 V2
.025
.241
.015
.470
.588
.570
ResNel152V2
.260
.240
.015
.486
.113
.395
VGG16
.005
.304
.160
.005
.218
VGG19
.005
.316
.005
.197
.336
.353
Table 2: Area under the curve for AIC and SIC using Normalized Entropy for 128 steps. The claim that IDGI improves all three IG-basod methods across all experiment settings does not hold.
We evaluated the explanation methods using the Softmax information curves (SIC) and the Accuracy infor- mation curves (AIC) using Normalized Entropy. We rewrote the available code according the modified implementation details introduced in the paper (Yang et al. 2023) and evaluated cach of the three basclines. We report the area under the AIC and SIC curves for 128 steps in Table 2
Based on our results, we find that our results match the claims made in the original paper (Yang et al. 2023) for the most part, and the better explanation method has a higher area under the AIC and SIC curves. However, we observed that for ResNet 152v2, IDGI worsens the area under the ATC and SIC curves for IG; and, for Nevv,,, ResNet50v2, and ResNet101v2, BlurGG + IDGI also underperforms BlurG. A common trait observed in the models that exhibit anomalies that they all implement residual connections in their architecture. While could not demonstrate how this may cause IDGI to reduce the performance of the underlying method, it is a plausible hypothesis that the residual connections might interact with the IDGI framework in ways that are not fully understood, warranting further investigation into the underlying mechanisms and their impact on explanation methods.
4.4 SIC and AIC with MS-SSIM
We now evaluate the explanation methods using the Softmax information curves (SIC) and the Accuracy information curves (AIC) using MS-SSIM (Kancharla & Channappayya 2018 Ma et al. 2016 Odena et al., 2017). This well-studied image quality evaluation method analyzes the structural similarity of two images. We re-wrote the available code according to the modified implementation details introduced in the paper

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based Methods
IG
+IDGI
GIG
+IDGI
BlurGG
+IDGI
DenseNetI21
.302
.141
.283
.188
.279
DenseNet169
.173
.300
.155
.292
.92
.285
DenscNet201
.96
.312
.69
.011
216
.022
AUC
AIC
Inception V3
.212
347
.192
345
.236
.340
(t)
MobileNel V2
.124
.253
.110
.243
.287
.297
ReaNet50V2
.232
.121
.252
.312
.322
ResNet101 V2
.206
.611
.192
.325
.340
.354
ResNet152V2
.262
.267
.192
.330
.219
.319
VGG16
.092
.255
.084
.218
.114
.236
VGG19
.103
.260
.092
.229
.218
.258
12
.006
.660
.242
.149
.237
DenseNet169
.123
.267
.109
.262
.162
.253
DenseNet201
.135
.278
.113
.268
.179
.267
AUC
SIC
Inception V3
.137
.308
.126
.310
.186
.301
(1)
MobileNet V2
.050
.141
.047
.141
.173
.184
ReaNet50V2
.065
.166
.055
.159
.207
.144
ResNet101 V2
.148
.209
.132
.277
.269
.283
ResNet152V2
.221
.221
.140
.289
.171
.276
VGG16
.055
.213
.051
.178
.076
.195
VGG19
.061
.221
.056
.191
.165
.213
Table 3: Area under the curve for AIC and SIC using MS-SSIM for 128 steps. The claim that IDGI
improves all three IG- based methods across all experiment settings does not hold.
Yang et al. 2023) and evaluated each of the three baselines (IG, GIG, and BlurlG) across ten models. We report the area under AIC and SIC for 128 steps in Table 3
Based on the results we obtained, we find that our results match the claims made in the original paper Yang et al. 2023 consistently. The better explanation method has a higher area under the AIC and SIC curves. However, for ResVet152V2, the area under SIC for IG equal that for IG + IDGI.
4.5
Additional Experiments
As mentioned in Section and discussed in Section 3 variation and evaluate its effect on IDGI compared to the baselines (IG, GIG, and BlurIG). We also conducted an experiment to compare the saliency map generated for an image quantitatively with that of a compressed version of the same image.
proceed to show results for step-size
4.5.1 Step Size Variation
IG-based methods
Insertion
Insertion
AIC with
SIC with Nor- malized Entropy
AIC with MS- SSIM
SIC with MS-
score with probabil- ity
score with
Normal-
ized Entropy
SSIM
probabil- ity ratio
IG
.024
.026
.033
.004
.010
.002
IG + IDGI
224
.200
.350
.473
.098
.106
BlurG
.044
.006
.124
.054
.041
.040
BIurIG+IDGI
0082
.087
.146
.222
.044
.049
Table 4: Difference in score between 8 and 128 steps for InceptionV3 (%)
10

---

Published in Transactions on Machine Learning Research (12/2024)
Algorithm Important Direction Integrated Gradient
Imputs: 2, f. path :.............
1: Initialize T/DG1 = 0
2: for each 2j in path do
3: d = fa(I)41) - Se(rz)
0fc(a;)
g Uz
5: D!DGI += 4xxx
6: end for
return !!DGI
We report our results for step-size variation through Figure 1 where we study the variation of the value of the respective metric versus the number of steps (8, 16, 32, and 64) for InceptionV3. Increasing the number of steps leads to a better score, as a higher number of steps in the Riemann sum leads to a finer approximation of the actual integral. However, due to computational limitations, increasing the number of steps requires more resources and time, necessitating trade-off. The developer must make choice based the sensitivity of the application and the resources available. Please note that the x-axis is in exponential scale, which means that beyond step size of ~32, the return in score improvement per step is marginal.
We observe that at higher step size, the scores of BlurGG + IDGI are lower than those for IG + IDGI. To explain this, we analyze the IDGI algorithm, as defined by Yang et al. (2023). and observe the relationship between d and 1 DDI the attribution value for IDGL, a, x.f cc, and path are as defined in Section
Iraetion Score with Probabiitt
A-C uning Normalined Entrosy
040
10M8
Bu10
Bur
035
0330
026
030
using
1300
Bin0
1215
022
1X01
035
2200
022
035
2
2
2000
Figure 4: Insertion Score with probability and probability ratio, AIC and SIC using Normalized Entropy and MS-SSTM vs. number of steps, for Inceptionv3.
As shown in Figure the variation in the image for BlurIG is minimal for most of the path, with significant sharpening occurring only in the final ~00% of the steps, suggesting that f is a constant, low value for most of the sampled points. In contrast, the image brightness increases uniformly along the IG path, indicating a more steady change f. The change between two subsoquent steps (d) determines the magnitude of the integrand added to the final saliency. Consequently, for BlurG IDGI, most samples contribute minimally to the final saliency. Both IG IDGI and BlurGG + IDGI benefit. from increased steps, improving the
11

---

Published in Transactions on Machine Learning Research (12/2024)
approximation of the underlying integral. However, IG + IDGI generally benefits more due to the more uniform change in probability scores along most of the path.
0.0
= 0.1
0.2
0.3
= 0.4
0.5
0.6
= 0.7
0.8
0.9
o 1,0
Buir
Figure 5: Images observed along the path of BlurIG and IG. For BlurG,, for most values of a, notice minimal changes in the image with small increments. In contrast, for IG, uniform change in the image is observed with the same increments in
In the Appendix, we report the insertion scores and area under AIC and SIC using Normalized Entropy and MS-SSIN ASS-SSIM for the remaining models for 8, 16, 32, and 64 steps. According to our experimental results, our theoretical analysis is verified and stands correct. We observe that IDGI consistently has more variants the number steps than its basclincs.
4.5.2 Numerical Instability
IG-based Methods
log MSE
IG
2.619
IG + IDGI
8.949
GIG
0.845
GIG + IDGI
8.029
BlurGG
3.466
BlurIG IDGI
8.778
Table 5: The negative log values of MSE computed between the saliencies the compressed images and the non-compressed images for InceptionV3. Higher values indicate better numerical stability
During our early experimental stage, we observed that, visually, the saliency maps obtained for GIG were starkly different when computed on GPU and CPU. This phenomenon is undesirable as it makes the at- tribution method unreliable. Hence, we decided to study the mumerical instability of different attribution methods. We used JPEG compression with retention factor of 75% We perform this experiment for IDGI and each underlying method: IG, BlurIG, and GIG. Our experimental results show that the baseline method + IDGI achieves significantly better numerical stability (smaller MSE) than the baseline method. Also, GIG shows poor numerical stability, had anticipated from visual observations.
5 Conclusion
We rigorously analyze the theoretical aspects of IDGI, experimentally verify the claims made by Yang et al. (2023), and perform additional experiments to verify theoretical observations and understand the numerical stability of the framework. Although the claim that IDGI constantly improves upon all bascline methods is mostly true for MobileNet V2 and the ResNet family, our experimental results show otherwise for some baseline methods. Our theoretical and experimental analysis shows that IDGI is more sensitive to step size variation than the baseline methods. The application of IDGI makes the bascline method more class-sensitive, which is desirable property. We also observe that the IDGI baseline method more numerically stable than the baseline.
12

---

Published in Transactions on Machine Learning Research (12/2024)
References
Runmin Cong, Jianjun Lei, Huazhu Fu, Ming-Ming Cheng, Weisi Lin, and Qingming Huang. Review of visual saliency detection with comprchensive information. IEEE Transactions on circuits and Systems for Video Technology, 29(10):2941 2959, 2018.
Parimala Kancharla and Sumohana S Channappayya. Improving the visual quality of generative adversarial network (gan)-generated images using the multi-scale structural similarity index. In 2018 25th IEEE international conference on image processing (ICIP), pp. 3908-3912 IEEE, 2018.
Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, and Michael Terry. Xrai: Better attributions through regions. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 4948-4957, 2019.
Andrei Kapishnikov, Subhashini Venugopalan, Besim Avei, Ben Wedin, Michael Terry, and Tolga Boluk- basi. Guided integrated gradients: An adaptive path method for removing noise. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5050-5058, 2021.
Kede Ma, Qingbo Wu, Zhou Wang, Zhengfang Duanmu,, Hongwei Yong, Hongliang Li, and Lei Zhang. Group mad competition-a new methodology to compare objective image quality models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1664-1673, 2016.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary clas- sifier gans. In International conference on machine learning, pp. 2642 2651. PMLR, 2017.
Deng Pan, Xin Li, and Dongxiao Zhu. Explaining deep neural network models with adversarial gradient integration. In Thirtieth International Joint Conference on Artificial Intelligence (IJCAT), 2021.
V Petsiuk, A Das, and K Saenko. Rise: Randomized input sampling for explanation of black-box models. arxiv 2018, Xrviv preprint arXio:1806.07/2 07..1.
Karen Simonyan, Andrea. Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint orXiv:13126633, 2013.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint
700:100060355 2017.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference machine learning, pp.
PMLR, 2017.
Shawn Xu, Subhashini Venugopalan, and Mukund Sundararajan. Attribution in scale and space. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9680 9689, 2020.
Ruo Yang, Binghui Wang, and Mustafa Bilgic. Idgi: A framework to climinate explanation noise from integrated gradients. In Proceedings of the LEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2023.
13

---

Published in Transactions on Machine Learning Research (12/2024)
6 Appendix
Table i to Table 17 show the insertion scores with probability and probability ratio, the area under AIC and SIC using Normalized Entropy and MS-SSIM for 4 methods (baseline methods, IG and BlurIG + IDGI) across the 10 models that we performed the experiments on for 64, 32, 16 and 8 steps (in that order). We could not perform these experiments for GIG due to computational constraints.
We have highlighted the better performing method in each Table.
Metrics
Models
IG-based Methods
1G
+IDGI
BlurG
+IGGI
DenseNet
.127
.357
.080
.665
DenseNet169
.128
.359
.088
.277
Insertion Score
DenseNet201
.150
.368
.106
:303
with Probability
Inception V3
.133
.409
100
.360
NobileNetV2
.202
(1)
ResNet50V2
.059
.099
.241
.233
ReaNel101V2
.171
.223
.429
.417
ResNe1152V2
.181
.222
.140
.355
VGG16
.048
.282
.039
.215
VGG19
.255
.275
DenseNet121
.090
Insertion
DenseNet169
.141
.388
.096
.300
Score
DenseNet201
.167
.404
.118
.332
with Probability
Inception V3
.159
.472
.118
.416
NebVleNetV2
.103
.299
.629
.520
Ratio (t)
ResNe150V2
.581
.564
ResNet101V2
.235
.303
.581
.566
ReaNet152V2
.215
.300
.186
.475
VGG16
.057
.313
.015
.239
VGG19
.067
.346
.290
.311
Table 6: Insertion score for different models with explanation methods for 64 steps.
Metrics
Models
IG-based Methods
IG
+IDGI
BlurGG
+DGI
DenseNet121
.133
.464
.193
.328
DenseNei169
.139
.440
.193
.323
DenseNet201
.189
.467
.260
.370
AUC (t)
Inception
.191
.547
.270
.451
MobileNesVe
.054
.337
.572
.520
ResNet50V2
.052
.183
.618
.567
BesNet101 V2
.154
.276
.692
.652
ResNer152V2
.322
.254
.291
.428
VGG16
.041
.368
VGG19
.052
.376
.409
NenseNet1gf
.010
16n
.011
.399
0037
.239
DenseNee101
.02G
.428
.094
.293
AUC
SIC
Inception V/
.016
.514
.059
.401
(t)
MobileNet122
.005
.127
.363
.340
ReaNet50V2
.005
.028
.404
ResNet101 V2
.020
.168
.588
.564
ResNeL152V3
.229
.150
.103
.349
VCG16
.005
.301
.005
.183
VGG19
.005
.312
.322
.337
Table 7: AUC for AIC and SIC using Normalized Entropy for 64 steps.
14

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based
Methods BlurIG
IG
IDGI
+IGGI
DenseNet121
.144
.297
.184
.272
D6nseNet169
.73
.292
.18
.279
DenseNet201
.16
.306
.212
.297
AUC AIC
Inception V3
.211
.343
.22
.332
(1)
MobileNesV2
.123
ReaNet50V2
.134
.205
.313
.317
ResNet101V2 V2
.201
.238
.399
.352
BesNet152V2
.255
.240
.215
.309
VGG16
.092
.253
.110
.228
VGG19
.103
.258
.254
DenseNet121
.106
.254
.144
.230
DenseNet169
.123
.258
.247
DenseNet201
.135
.270
.175
.261
SIC
Inception Vy
.137
.303
.182
.293
(t)
MobileNeiV2
.050
.123
.172
.180
ResNes50V2
.065
ResNet101 V2
.144
.187
ResNet152V2
.213
.193
166
.265
VGG16
.055
.212
.073
.185
VGG19
.061
.219
.164
.209
Table 8: AUC for SIC and AIC using MS-SSIM for 64 steps.
Metrics
Models
IG-based Methods
IG
+IDGI
BlurGG
+IDGI
DenseNet121
.24
.200
.078
.246
DenseNct169
.125
.268
.087
.259
Insertion Score
Net20
.147
.292
.05
.284
with Probability
Inception V3
.129
.382
.097
.341
MobileNelV2
.022
(t)
RosNet50V2
.240
.229
ReaNet101V2
.166
.195
.427
.409
ResNel152V2
.176
.201
.140
.329
VGG16
.048
.265
.040
.196
VGG19
.251
.260
DonseNct121
.140
.324
.088
.275
DenseNet169
.095
Insertion
Score
DenseNeet01
.163
.323
.116
.813
with Probability
Inception V3
.154
.441
.115
.396
MobileNetV2
.101
.247
.528
$10
Ratio (t)
ResNet50V2
.142
.579
.533
ResNet101 V2
.578
ResNet152V2
.238
.272
.187
.442
VGG16
.056
.294
.046
.219
VGG19
.067
.319
.285
.295
Table 9: Insertion score for different models with explanation methods for 32 steps.
15

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based
Methods BlurGG
IG
IDGI
+IGGI
DenseNet121
.132
.996
.173
.296
DenseNet169
.335
.342
.155
.295
DenseNet201
.186
.385
.238
.342
AUC AIC
Inception V3
.188
.518
.200
.414
(1)
MobileNesV2
ReaNet50V2
.017
.147
.617
ResNet101V2
.144
.230
.690
.638
BesNet152V2
.305
.217
.278
.378
VGG16
.041
.351
.066
.256
VGG19
.052
.353
.405
.392
DenseNet121
.010
.314
.186
DenseNet169
.010
.253
.197
DenseNet201
.025
.305
.070
.252
SIC
Inception Vy
.015
.473
.046
.348
(t)
MobileNetV2
.005
.065
.362
.333
ResNes50V2
.005
ResNet1011 V2
ResNet152V2
.209
.116
.087
.284
VGG16
.005
.275
.005
.134
VGG19
.005
.278
.326
.313
Table 10: AUC for AIC and SIC using Normalized Entropy for 32 steps.
Metrics
Models
IG-based Methods
IG
+IGGI
BlurGG
+IGGI
DenseNet121
.163
.286
.172
.261
DenseNet169
.172
.273
.178
.270
DenseNet201
.195
.293
201
.877
AUC
AIC
Inception V3
.210
.332
.211
321
(t)
MobileNesV2
.120
ReaNet50V2
.132
ResNet101 V2
.198
.220
337
.347
ResNet152V2
.250
.225
.099
.944
VGG16
.093
.249
.104
.215
VGG19
.103
.253
.212
.248
DenseNet121
.106
.242
.122
.217
DenseNet169
.123
.236
.236
DenseNet201
.135
.254
.163
.251
AUC
SIC
Inception V9
.137
.289
.171
.280
(t)
MobileNetV2
.049
.103
.16
.175
ReaNet50V2
.065
.001
.207
ResNet V2
ResNet152V2
.209
.179
.159
.249
VGG16
.055
.207
.066
.172
VGG19
.061
.213
.158
.202
Table 11: AUC for SIC and AIC using MS-SSIM for 32 steps.
16

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based Methods
IG
+IGGI
BlurGG
+IDGI
DenseNel121
.115
.189
.079
.226
DenseNet169
.17
.169
.088
.377
Insertion
DenseNet201
.18
.191
.06
.264
Score
with Probability
Inception V3
.125
.302
.096
.323
MobileNetV2
(1)
ResNet50V2
.092
.235
.223
ReaNet101V2
.665
.181
.421
.398
ResNet152V2
.173
.189
.142
.297
VGG15
.048
.212
.042
.171
VGG19
.239
.245
DenseNet121
.214
.089
.254
Insertion
DenseNet169
.129
.096
.259
Score
DenseNet201
.154
.215
.117
.292
with Probability
Inception Va
.150
.351
.114
.379
MobileNetV2
.100
.216
.521
.498
Ratio (t)
ResNe150V2
.566
.539
ResNet101V2 V2
.570
.540
ResNet152V2
.244
.255
.189
.402
VGG16
.056
.238
.049
.192
VGG19
.065
.253
.272
.277
Table 12: Insertion score for different models with explanation methods for 16 steps.
Metrics
Models
IG-based Methods
IG
+IDGI
BlurGG
+DGGI
DenseNet121
.118
.269
.137
.267
DenseNet169
.121
.218
.147
.269
DenseNet201
.167
.257
198
.312
AUC
AIC
Inception V3
.184
.395
.111
.375
(t)
MobileNesV2
.045
ReaNet50V2
.044
.130
.610
.545
ResNet101 V2
.137
.212
.886
.622
ResNet152VV
.292
.199
.53
.324
VGG16
.039
.283
.054
.224
VGG19
.051
.281
.397
.375
DenseNet121
.009
.158
.013
.146
DenseNet169
.009
DenseNet201
.020
.145
.038
.212
AUC
SIC
Inception V9
.015
.297
.025
.294
(t)
MobileNetV2
.005
.038
.355
.331
ReaNet50V2
.005
.013
.116
ResNet V2
ResNet152V2
.196
.097
.062
.215
VGG16
.005
.168
.005
.088
VGG19
.005
.170
.314
.287
Table 13: AUC for AIC and SIC using Normalized Entropy for 16 steps.
17

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based
Methods BlurGG
IG
IDGI
+IGGI
DenseNet121
.160
.268
.156
.248
D6nseNet169
.169
.239
.161
.259
DenseNet201
.191
.265
.13
.275
AUC AIC
Inception V3
.208
.304
.044
.309
(1)
MobileNesV2
.116
ReaNet50V2
.131
.188
.002
.303
ResNet101V2
.197
.210
331
.341
BesNet152V2
.466
.216
200
.276
VGG16
.092
.237
.00
.201
VGG19
.103
.238
DenseNet121
.106
.219
.117
.204
DenseNet169
.121
.202
.135
.221
DenseNet201
.132
.220
.145
.238
SIC
Inception Vy
.137
.257
.155
.266
(t)
MobileNeiV2
.048
.090
.161
.171
ResNes50V2
.065
ResNet101 V2
.142
ResNet152V2
.205
.171
.149
.230
VGG16
.055
.193
.068
.158
VGG19
.061
.195
.143
.192
Table 14: AUC for SIC and AIC using MS-SSIM for 16 steps.
Metrics
Models
IG-based Methods
IG
+IDGI
BlurGG
+IDGI
DenseNet121
.07
.139
.087
.206
DenseNct169
.113
.121
.102
.213
Insertion Score
Net30
.129
.139
.114
.241
with Probability
Inception V3
.111
.195
.098
.297
MobileNelV2
.193
.185
(t)
RosNet50V2
.229
.217
ReaNet101V2
.164
.176
.410
.38
ResNel152V2
.172
.182
.150
.255
VGG16
.046
.141
.046
.151
VGG19
.051
.228
.232
DonseNct121
.122
.159
.099
.232
Insertion
DenseNet169
.111
.233
Score
DenseNeet01
.144
.126
.267
with Probability
Inception V3
.134
.333
.116
.350
MobileNetV2
.099
.206
.504
.188
Ratio (t)
ResNet50V2
.552
.525
ResNet111 V2
ResNet152V2
.232
.246
.199
.348
VGG16
.054
.161
.054
.170
VGG19
.063
.171
.259
.264
Table 15: Insertion score for different models with explanation methods for steps.
18

---

Published in Transactions on Machine Learning Research (12/2024)
Metrics
Models
IG-based
Methods BlurIG
IG
+DGI
+IGGI
DenseNet121
.102
.204
.098
.247
D6nseNet169
.10
.153
.121
.250
DenseNet201
.146
.187
.145
.291
AUC AIC
Inception V3
.162
.004
.153
.332
(1)
MobileNesV2
.041
ReaNet50V2
.040
.127
.573
.539
ResNet101V2
.132
.206
.661
.605
BesNet152VV
.883
.192
.212
.265
VGG16
.036
.175
.049
.206
VGG19
.047
.174
.364
DenseNet121
.007
.091
DenseNet169
.007
.060
.134
DenseNet201
.015
.082
.017
.181
SIC
Inception Vy
.012
.054
.012
.234
(t)
MobileNetV2
.005
.029
.333
.329
ResNes50V2
.005
.392
ResNet101 V2
.013
ResNet152V2
.187
.090
002
.144
VGG16
.005
.039
.005
.062
VGG19
.005
.045
.744
.271
Table 16: AUC for AIC and SIC using Normalized Entropy for 8 steps.
Metrics
Models
IG-based Methods
IG
+IDGI
BlurGG
+IDGI
DenseNet121
.155
.250
.147
.240
DenseNet169
.166
.211
.162
.251
DenseNet201
.185
.238
.172
.267
AUC
AIC
Inception V/
.222
.249
.95
.966
(t)
MobileNesV2
.112
ReaNet50V2
.129
.176
ResNet101 V2
.196
.206
328
.333
ResNet152VV
.244
.211
.192
.251
VGG16
.090
.210
.104
.194
VGG19
.101
.204
.198
DenseNet121
.103
.195
.113
.195
DenseNet169
.119
.177
.215
DenseNet201
.127
.191
.136
.229
AUC
SIC
Inception V9
.135
.202
.146
.252
(t)
MobileNetV2
.046
.083
.157
.169
ReaNet50V2
.06-1
.087
.200
ResNet V2
ResNet152V2
.203
.166
.140
.204
VGG16
.054
.161
.077
.150
VGG19
.060
.159
.141
.187
Table 17: AUC for SIC and AIC using MS-SSIM for steps.
19


Reviews:
[{"review_id": "DOmqAApAVN", "reviewer": "Reviewer_Bo6r", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n* The authors provide a thorough and well-structured theoretical analysis of the IDGI algorithm, including its derivation using Taylor series approximation.\n* The authors have a comprehensive experimental evaluation of the original IDGI work, verifying its claims and highlighting its limitations.\n* The authors' writing style is clear, concise, and easy to follow.\n\nWeaknesses\n* The authors do not present new techniques, but rather analyze an existing one.\n* The authors primarily focus on evaluating IDGI against other integrated gradient methods, rather than comparing it to other attribution methods. This limits the scope of the evaluation and makes it difficult to determine how IDGI compares to other state-of-the-art methods.\n* The authors primarily evaluate IDGI only on Imagenet variants. This makes sense due to the paper replication study, but could be expanded to other datasets to understand the generalizability of their findings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "* The paper presents an analysis of the Important Direction Gradient Integration (IDGI) framework, a method for attributing the predictions of models to their input features. \n\n* The authors demonstrate that IDGI is sensitive to step size variation, and that using a sufficient number of steps is crucial for achieving accurate results. This finding has important implications for the practical application of IDGI and highlights the need for careful tuning of hyperparameters.\n\n* The authors investigate the numerical stability of IDGI and demonstrate that it is more numerically stable than some other attribution methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Expand the study to include evaluations on other tasks to understand the generalizability of the findings.\n* Analyze and compare IDGI to other attribution methods\n\nThese adjustments would simply strengthen the work, and are not critical to secure my recommendation", "broader_impact_concerns": "No ethical concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hWWfb3OKPa", "reviewer": "Reviewer_p4wk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nDeriving the expression for xj_p from the Taylor series expansion was interesting.  The error can sometimes be analysed using Taylor series in numerical integration (such as Riemann sums). For instance, the error in a Riemann sum can be related to the higher-order derivatives of the function, which are part of the Taylor series. The Taylor series can approximate a function locally, and integrating a Taylor series term-by-term can provide an approximate value for the integral of the function over a small interval. Fundamental Theorem of Calculus connects differentiation (a key part of the Taylor series) and integration. I liked to see the authors make this connection sensible in IDGI methods.\n\nWeaknesses: \n- The paper lacks enough novelty and new ideas. If this is a reproduction paper on the existing methods, I am unsure if TMLR is the right venue. I suggest https://rescience.github.io/. \n- Moreover, the paper requires a more detailed explanation in your voice rather than summarizing the wording of the previous papers. This manuscript would be hard to follow and understand if someone had not read the previous work.  \n\n- It is not clear why the small difference in the illustration matters. You must explain why it makes a difference and what it exactly suggests.\n       - \"The relationship between fc(x) and x for IG is portrayed as linear; however, that is not so. The relationship between x and α is linear. Thus, this can be an origin of confusion among readers.\" What is the confusion that it causes? What are the sources? And why does it matter?\n\n- Earlier in the paper, fc(x) is defined as the scores for input x when predicting c. \"Let f be a classifier, c, a class, and x, an input, then, the output fc(x) represents the confidence score for predicting x to class c.\" There can exist multiple scores fc(x) values that result in the right class prediction. for example fc(x) = [0.1, 0.1, 0.8] or fc(x)=[0.2, 0.1, 0.7]. In both cases, we still predict class 3.  Why later on in the paper, when talking about the importance of new illustrations in IDGI, do you claim there should be a one-to-one relationship between x and fc(x)? \"The function x → fc(x) should be one-to-one; however the original illustration portrays it to be a one-to-many function, which is not possible.\"\n\n- The Theorem 1 is exactly the copy-paste of Yang's paper's Theorem 1. There are many paragraphs in the paper that are borrowed with slight changes from Yang's paper.\n\n\n- The paper is written weakly. There are quite a few errors including:\n      - missing periods at the end of sentences,\n      - all acronyms must be defined before being used,\n      - missing references, \n      - have used some notations without defining them in the background section. Even when you refer to a previous work, you still need to define all symbols you use. \n      - high similarity in the sentences of the original paper and this write-up", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper claims that they comprehensively evaluate Integrated Gradients (IG) based methods and the Important Direction Gradient Integration (IDGI) framework. The proclaimed contributions are summarized as follows:\n\n- Reproducibility Study: Conducted a detailed reproducibility study of the IDGI framework to eliminate explanation noise from IG-based methods.\n- Theoretical Analysis: Presented a rigorous theoretical analysis of IDGI, raising critical questions and addressing them through both theoretical observations and experimental verification.\n-  Experimental Validation: Verified the authors' claims on the performance of IDGI over IG-based methods through extensive experiments.\n-  Step Size Variation Analysis: Examined the impact of varying the number of steps in the Riemann approximation on the performance of IDGI.\n-  Numerical Stability: Investigated the numerical stability of attribution methods to ensure consistency in saliency maps.\n- Publicly Available Code: Developed and shared the complete code for implementing IDGI, addressing the insufficiencies in the original code.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- I believe the authors need to improve the motivations for this study. Some suggestions:\n       - \"The code provided by the repository was not usable for reproducing results at scale.\" Please explain in a nutshell, what was the issue with the published code base and what exactly you mean by \"reproducibility at scale\". This clarifies the importance and necessity of a study paper on the available methods better. \n\n-The paper does not seem to be introducing a novel idea but rather is reproducing results. \n         - Clarify the specific differences between this work and existing methods describe why it does matter and make changes. \n         - If there are differences in the final results, snd if it is due to hyperparameters, what is the significance here?\n         - Explain in your own words how the small modification in the illustration of IDGI is important.\n\n- In all formulas, define all the parameters you are using. These notations are all borrowed from previous papers with the presumption that everyone knows them. A write-up should always have consistency and clarity in the equations, even if they already exist in other papers.\n\n- In experiments, the authors say \"However, we observed that for MobileNetv2, ResNet50v2, and ResNet101v2, IDGI worsens the insertion scores for BlurIG.\" You should be able to clarify why it is an important or critical finding?\nMinors: \n- Use the full description of all acronyms for the first time you use them in the paper:\n          -\"It also introduced a new measurement, AIC and SIC (Kapishnikov et al., 2019) using MS-SSIM to estimate the entropy of an image more accurately by improving upon previously proposed metrics (Kapishnikov et al., 2019; Kancharla & Channappayya, 2018;\"\n\n- missed the periods to separate two sentences: \n         - \"We could not directly use the existing code for our study This led us to integrate the code for IDGI ...\" \n         - In Theorem 1, \"... in −→F Assume ...\"\n\n- extra period in the middle of the sentence:\n        - \"where an infinite number of hyperplanes h exist, and each hyperplane contains all points. x with the same functional value.\"\n\n- In the background section, you must refer to the three works you described: IG, BlurIG, and GIG.\n\n- In Section 2 Background and 2.1 Integrated Gradients, you cannot bring a title and subtitle consecutively without any lines of explanation.  The same for 4.5 Additional Experiments and 4.5.1 Step Size Variation.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DiHrKCcce7", "reviewer": "Reviewer_1HiZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1.\tThe authors conducted comprehensive experiments to support their conclusions.\n2.\tExisting papers are hard to follow due to incomplete implementation details. This paper performs an elaborate investigation to clarify them.\n3.\tThis paper provides an in-depth discussion on IDGI’s sensitivity to step-size variation.\n\nWeaknesses\n\n1.\tIn this paper, there is only one visualization result in Figure 1. More visualization results should be included in the experimental section. What are the results for complex images? Additionally, there is no explanation for Figure 1. How should this figure be understood? \n2.\tAs shown in Figure 3, the scores are sensitive to the number of steps. Thus, I am wondering how to choose an appropriate number of steps. It seems we can just choose the highest step size (2^7). Is this correct? According to Figure 3, I found that the scores for BlurIG+IDGI are lower than those for IG+IDGI at higher steps. Could the authors provide some explanations for this?\n3.\tThe authors observed that for MobileNetv2, ResNet50v2, and ResNet101v2, IDGI worsens the insertion scores for BlurIG. I am wondering if the authors could provide any possible explanations for this. As the authors conducted comprehensive experiments, they may be able to offer some insights.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper comprehensively explores the Integrated Gradients (IG) based methods and the Important Direction Gradient Integration (IDGI) framework. The authors have answered four important questions to provide a theoretical analysis of IDGI. Importantly, many related papers do not provide the complete code used to produce their results. In this paper, the authors perform an elaborate investigation into the implementation details of existing papers, thus improving reproducibility.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. More visualization results should be included. The authors should show some visualization results for complex images.\n2. The authors should explain and discuss Figure 1.\n3. Some in-depth discussions should be added, as mentioned in the weaknesses part.", "broader_impact_concerns": "I do not have any concerns about the ethical implications of this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All three reviewers supported acceptance after revisions. Reviewer Bo6r "leaned accept," highlighting the paper's "thorough and well-structured theoretical analysis" and "comprehensive experimental evaluation," though suggesting expanded dataset coverage. Reviewer 1HiZ also leaned accept. This reviewer's concerns were addressed with the added visualizations and implementation details clarification. Reviewer p4wk, despite initial concerns about similarity to previous work, confirmed the paper "reached an acceptable standard" following revisions that improved writing clarity and technical content. The authors addressed all major concerns raised in the reviews, including adding more complex visualizations, enhancing explanations, and improving documentation of implementation details. The positive feedback from all reviewers and the authors' thorough response to reviewer suggestions support this acceptance decision.
Decision: Accept (poster)