Re-calibrating Feature Attributions for Model Interpretation
Authors: Peiyu Yang
NAVEED AKHTAR
Zeyi Wen
Mubarak Shah
Ajmal Saeed Mian
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Feature Attribution
Explainable Artifical Intelligence

Abstract: The ability to interpret machine learning models is critical for high-stakes applications. Due to its desirable theoretical properties, path integration is a widely used scheme for feature attribution to interpret model predictions. However, the methods implementing this scheme currently rely on absolute attribution scores to eventually provide sensible interpretations. This not only contradicts the premise that the features with larger attribution scores are more relevant to the model prediction, but also conflicts with the theoretical settings for which the desirable properties of the attributions are proven. We address this by devising a method to first compute an appropriate reference for the path integration scheme. This reference further helps in identifying valid interpolation points on a desired integration path. The reference is computed in a gradient ascending direction on the model's loss surface, while the interpolations are performed by analyzing the model gradients and variations between the reference and the input. The eventual integration is effectively performed along a non-linear path. Our scheme can be incorporated into the existing integral-based attribution methods. We also devise an effective sampling and integration procedure that enables employing our scheme with multi-reference path integration efficiently. We achieve a marked performance boost for a range of integral-based attribution methods on both local and global evaluation metrics by enhancing them with our scheme. Our extensive results also show improved sensitivity, sanity preservation and model robustness with the proposed re-calibration of the attribution techniques with our method.

Published as a conference paper at ICLR 2023
RE-CALIBRATING FEATURE MODEL INTERPRETATION
ATTRIBUTIONS
FOR
Peiyu Yang', Naveed Akhtar', Zeyi "aun:,,uubaralll Shah", and Ajmal Mian' The University of Western Australia
Hong Kong University of Science and Technology (Guangzhou) Hong Kong University of Science and Technology *University of Central Florida (peiyu. yangee research., naveed. akhtarê, ajmal.miante Juwa edu.au venzeyi@ust.. hk shah@crcv. ucf. edu
ABSTRACT
The ability interpret machine learning models is critical for high-stakes appli- cations. Due to its desirable theoretical properties, path integration is widely used scheme for feature attribution to interpret model predictions. However, the methods implementing this scheme currently rely absolute attribution scores to eventually provide sensible interpretations. This not only contradicts the premise that the features with larger attribution scores more relevant to the model pre- diction, but also conflicts with the theoretical settings for which the desirable prop- erties of the attributions compute an appropriate reference for the path integration scheme. This reference further helps in identifying valid interpolation points desired integration path. The reference is computed in a gradient ascending direction on the model's 's loss surface, while the interpolations and variations between the reference and the input. The eventual integration is ef- fectively performed along a non-linear path. Our scheme can incorporated into the existing integral-based attribution methods. We also devise an effective sam- pling and integration procedure that enables employing our scheme with multi- reference path integration efficiently. We achieve marked performance boost for a range of integral-bar -ased attribution methods on both local and global evalua- tion metrics by enhancing them with our scheme. Our extensive results also show improved sensitivity, sanity preservation and model robustness with the proposed re-calibration of the attribution techniques with our method!
proven. We address this by devising method to first
performed by analyzing the model gradients
INTRODUCTION
How to interpret deep learning predictions is a major concern for the real-world applications, es- pecially in the high-stakes domains. Feature attribution methods explain a model's prediction by assigning importance scores (attributions) to the input features (Simonyan et al. 2014 Springen- berg et al.) 2015 Shrikumar more relevant to the model prediction than those with smaller scores. Among a variety of attribution methods, integral- based techniques (Sundararajan larly attractive because they satisfy certain desirable axiomatic properties, which others do not. This also makes them suitable model regularization (Chen al. 2019 Erion et al. 2021).
2017). They assert that features with larger attribution scores are
al., 2017 Sturmfels et al.) (020)) are partieu-
Inspired by the cooperative game theory. integral-based attribution methods introduce a reference to represent the absence of the input signal. This allows them to calculate the attribution for the presence of an input feature (Sundararajan et al. 2017]] Erion et al. [202 Pan et al. [202I) Since the attribution is computed with respect to a reference, finding the correct reference that represents the absence of feature in true sense, is critical for the reliability of these methods. In their Integrated Gradient (IG) method, Sundararajan et al. (2017) chose a black image (zero input) as the reference. However, this result is always assigning confirmed that fixed reference renders an attribution method blind to the features that the reference
attribution to black input pixels. Sturmfels et al. (2020)
Our code is available at wttptttt///// /ettpa://githbb.... wwwp:ttt....oo....eeeeeeee
recal ib

---

Published as a conference paper at ICLR 2023
IG-Unilorm
IG-SG
IG-SQ
EG
AG1
Vanillaa
Past Pest ABC: 16.5
Piasl Part AUC. 130
Plasl Pert ABC: 33.1
Pasl Port AUC. 13.4
Plut Pert ALC:
Absolute Valiue
Port AUC:
Pert AUC:
Flct
7.0
Pert AUC.
Input
Rin Re-CCallaratedd
A Aitiiiaionn from Positive to Negathie
Part, AUC:
AUC:
Flad
666
Figure I: Attribution maps existing methods (Vanilla) suffer from large variations between posi- tive and negative scores, which forces them to the Absolute Values of the scores for sensible interpretation. However, this contradicts the primary assertion of this research direction, and with the settings used claim the desirable theoretical properties of these methods. Re-calibrating the methods with our technique addresses the issue, while also improving the performance. Maps are shown for VGG-16. The pixel perturbation (%) AUC is reported. Lower values are more desirable.
employs. To resolve that, multiple references now commonly used by the attribution methods. The choices of multiple references include, uniform noise references (IG- Uniform) (Sturmfels et al.
2020). Gaussian noise references (IG-SG and IG-SQ) (Smilkov et al. 2017 Hooker et al. 2019).
training distribution (EG) (Erion et al. 2021 and adversarial examples (AGI) (Pan et al. 2021
etc. However, the attribution maps computed by these methods face frequent variations between positive and negative attribution scores, see Figure[]top row. To handle that, these methods must use absolute scores generate plausible explanations, see Figure row. Numerically, computing the absolute values re-orders the estimated attributions for the pixels, which violates the primary assertion of the attribution methods. Moreover, relying only on the magnitude of the attributions is also in contradiction with the axiomatic properties, which are proven for the actual numerical scores. This inconsistency compromises the desirability of these methods.
To address these problems, we develop method to compute the desired reference for an input along the model's gradient ascending direction. Moreover, we allow the gradient integration along a non- linear path from the reference the input. This is made possible by systematically identifying valid interpolation points on the path. eventually enables us to directly use the actual, instead of absolute attributions for model interpretation. We further devise a technique to efficiently compute the integral with valid references which can be estimated using the predefined references employed leverage technique to re-calibrate the attributions
by the existing methods. This enables of the existing methods without additional computational overhead. Figure [ bottom row shows the attribution maps calculated by the popular integral-based attribution methods calibrated with our technique. These maps are computed with the actual attributions scores, not the absolute values. Hence, they conform the primary assertion of the attribution methods and the settings used in establishing their theoretical properties. Moreover, they achieve better quantitative scores.
In our experiments. guantitative evaluation is performed with pixel perturbation (Samek 2016) and DifiROAR (Shah et al., (202I) on ImageNet-2012 validation set (Russakovsky et al. 2015), CIFAR-100 and CIFAR-10 Krizhevsky al., 2009). We show marked performance improvement for range of integral-based attribution methods by re-calibrating them with our technique. We also provide a detailed sensitivity analysis of the improved methods with Sensitivity-n (Ancona et al.) 2018), and passing the sanity checks (Adebayo et al. 2018). Moreover, improvements in attribution prior based regularization (Erion et al. 2021) with our technique. A considerable performance gain for variety of techniques and models across range of evaluation metrics ascertains the effectiveness of our re-calibration method. In summary, we make the follow- ing major contributions.
also show consistent
2

---

Published as a conference paper at ICLR 2023
1. We propose method to compute reliable reference for integral-based attribution methods.
2. We devise a technique to efficiently integrate over non-linear path that enables meaningful interpretations using the actual, as opposed to absolute attribution scores.
3. We enhance range of existing methods with our findings to demonstrate a considerable perfor- mance boost across the board over multiple evaluation metrics, and for model regularization.
RELATED WORK
Feature attribution methods explain model's prediction by assigning attribution scores to the input features. These techniques can be roughly categorized into perturbation based, and pack-propagation based methods. The former, c.g., (Zeiler & Fergus 2014 Fong & Vedaldi 2017). calculates the attribution scores by repeatedly perturbing the input features and analyzing the resulting effects on the model output. fackk-ropagation methods, e.g., (Simonyan
al.) 2014; Sundararaaan et al., 2017) estimate the attribution scores using a backward pass model gradients, and enjoy multiple advantages over the former (Ross et al.) 2017 Chen et al., 2019). Since our contribution relates to the back-propagation methods, we focus on the related work in this sub-category.
[Simonyan et al. 2014) first employed input gradients for class-specific saliency mapping for model interpretation. Deconvolutional Network (Zeiler & Fergus 2014) and Guided Backpropagation (Springenberg al. 2015) also propagate the activation information in the backward direction to highlight important imput features. Layer-wise Relevance Propagation (Bach) 2015) defines the relevance of neurons to recursively assign the attributions to the input. Similarly, DeepLIFT (Shrikumar et al. 2017) assigns attribution scores to the input by comparing the relative contributions of features using a reference input.
Gradient-Based Methods:
Integral-Based Methods: Integrated Gradients (IG) (Sundararajan et al. 2017) computes an inte- gral for the input attribution along a linear path from the input a reference representing the absence of the input feature. However, the zero reference chosen by the IG makes it blind to black pixels. To overcome that, Expected Gradients (Erion et al.] [2021] samples multiple references from the model's 's training distribution. IG-SG (Smilkov et al. (217] and IG-SQ (Hooker et al.) 2019) perturb input images with Gaussian noise as the references. In addition, Adversarial Gradient Integration (Pan al. 2021) adversarially perturbs the input to compute the desired reference. [Sturmfels al.) 2020) compared different references for the path attribution methods, demonstrating their weak- nesses.
Evaluation Metrics: A fair benchmarking is critical to establish the reliability of attribution meth- ods. Pixel perturbation (Samek al. 2016 Petsiuk et al. 2018 Srinivas & Fleuret 2019) one of the popular quantitative evaluation techniques, which removes the most or least salient pixels (as deemed by the attribution method) note their effects on model output changes. It is operated on individual inputs, hence considered local metric. ROAR (RemOve And Retrain) (Hooker et al.) 2019) and DifiROAR (Shah et al., 202T) are popular global metrics that utilize retraining over data distribution after removing the attributed pixels from the data. Sensitivity-n (Ancona et al.) 2018) is proposed to test the sensitivity of different feature groups while accounting for the attribution scores. Adebayo et al. (2018) also proposed sanity checks to verify reliability of the attribution maps com- puted by different methods. All these metrics evaluate different aspects of the attribution techniques. We employ all of them for extensive benchmarking.
PRELIMINARIES
For classification, machine learning model maps an input x S,(x)) for class c. For explaining the model prediction, an attribution method M aims at attributing this score to the input features (pixels) by an attribution map (x) [Mf (z), ... M5(x)) (=)]]
[_ Øn] E R to output score
M,
Input Gradients:
To explain model predictions, Input Gradients (Simonyan et al. 2014 computes model gradients with respect to the input x the importance scores (attributions). Following the notations from above, for class c, the attribution of the i-th feature I; calculated by this method is
(xx (8.
(1)
Simonyan et al. (0144) arrived at this strategy by applying the first-order Taylor expansion to the non-linear model: (.) RS wo b. This allows the attribution of each input feature to be
3

---

Published as a conference paper at ICLR 2023
represented by the input gradient w, /0) However, due to the high non- non-linearity of deep models, the input gradients cannot provide reliable attributions. Ancona et al. (2018) also regarded local attribution method that cannot provide complete explanation for specific prediction. Input gradients can mainly indicate how to obtain the desired output by changing
the input gradients
the features around the original input. This notion also has similarities to adversarial examples and other feature visualization methods (Goodfellow et al. 20155 Carter et al. 2019).
Different from the input gradients with local interpretability. integral-based attribution methods, e.g., Integrated Gradients (IG) (Sundararajan et al., (017). estimate an integral over a path to calculate the attributions. Inspired by the cooperative game theory. IG chooses a reference to represent the complete absence of the input signals, which enables it to integrate gra- dients from the reference to calculate the attribution for the presence of the input features. The IG calculates the attribution for the i-th input feature xy with the reference x' [2 2n,] E IR" as follows
Integrated Gradients:
/ (T) -== 0e;
(x, (x: x()
da,
(2)
12=x'+a(===
where a represents the step along linear path from the reference to the input. Since the integral of Equation 2 satisfies the fundamental theorem of calculus, IG can satisfy the desirable completeness axiom, ie.. the output changes can be completely attributed to the input features: >x= (f,x==') Se(x) - Sz(x'). ). In contrast to the input gradients, integral- based attribution methods are regarded as global attribution methods (Ancona et al. 2018), which can estimate the marginal effect from the reference to the input for model prediction.
The primary assertion of the attribution methods is that the input features with larger attribution scores are more relevant to the model predictions. However, the attributions computed by the poopprropgattion techniques suffer from frequent variations between positive and negative scores, even among the neighboring pixels of a (relatively) uniform image region. Thus, for plausible interpretations, these methods eventually use the absolute values of the attributions instead of the actual numerical scores (Sundararajan et al., [017]] Srinivas & Fleuret) 2019). However, considering the absolute values not only re-orders the computed pixel importances, it also contradicts the theoretical settings assumed to prove the desirable axiomatic properties of these techniques.
Existing Inconsistency:
4 METHODOLOGY
In this section, we first assume a reliable attribution calculation with an automatically chosen refer- ence. Then we propose a technique estimate the proposed attribution, which enables us reveal a reason for the inconsistency problem. Finally, the proposed technique is extended to popular attribution methods for calibrating their attributions within limited computational resources.
4.1 PURSUING RELIABLE ATTRIBUTION
The reference image in an integral- -based method has the central importance. As noted earlier, the zero image of IG (Sundararajan al. (017) renders it blind to the black pixels in the input. The primary motivation behind using muluple references created with uniform noise (Sturmfels et al.) 2020), Gaussian noise (Smilkov et al. 2017) or training samples (Erion al. 2021), etc., is avoid this blindness issue. These methods create multiple references in the hope that some of those can correctly capture the abstract notion of feature absence, thereby resulting in a reliable attribution.
Instead of hoping to accidentally stumble on the desired reference, we propose to systematically identify it. To that end, we can modify the input image with the model gradients to construct the ref- erence. Optimizing the input in the gradient ascending direction leads to local features that decrease the target prediction scores, providing a reasonable analogy of feature 'absence' in computational sense. Such reference is computed with respect to the model itself, instead of relying on exter- nal operator, which can also ensure the model-fidelity of the eventually computed attribution scores. We also emphasize that considering uniform reference for all the input features is not ideal, as feature absence is a relative notion which is not handled well by uniformity assumption. Hence, a set of references for input is more desirable, which we also employ.
Given a feature 2; of the input x. we hypothesize a computable desired reference x' = Dr within a reference set D. Then, the attribution MC of x; can be calculated by integrating the gradients along
4

---

Published as a conference paper at ICLR 2023
the integration path from the input to the reference z' as
(T.(T)
(1,,, = (z: x
da s.t.
(3)
a= Oi; li=o'' (0x))===))
where (:(()) indicates a (possibly non-linear) integration path function defined by x and x', with the step 7 (a) over domain f2 E IR", and Æ is the interpolation point that resides on this path. We note that our idea of computable reference along the gradient ascending direction has some similarity with the Adversarial Gradient Integration (Pan et al., 2021) which integrates gradients along the path from the input to class-specific adversarial examples. However, in contrast to AGI, we allow different features to have different paths to different references. This effectively enables integration along a non-linear path defined by the reference set.
4.2 INTEGRAL ESTIMATION WITH DESIRED INTERPOLATIONS
So far, we have only set our compass in the direction of pursuing reliable attributions. Different from the linear path employed by the IG-like methods (Sundararajan et al.. 2017 Erion et al. (021).
this pursuit requires integration of gradients along a non-linear path %. as per Equation (3).
To compute the required integral, the key is identify the interpolation points that reside on the integration path from the desired reference x' to the input (or vice versa). Assuming an interpo- lation point I be in the same input space I", that, the gradient can induce a change in the input feature towards the desired reference, as discussed in the previous section. Given an input feature T; of input x, the signed input gradient within the set (Vt, Vo, V indicates the direction of the variation 8; from to its local desired reference x' Di along the gradient ascending direction. Thus, the gradient with respect to an interpolation the path between the input and the reference, has the same sign as the variation 6, E {,,t 66, 5, over the i-th feature of ă. If the interpolation point does not reside on the de- sired path, the gradient Vi, always has the opposite sign compared to 6;. To exemplify, let input feature x; have positive gradient Vt and this indicates x; should take a negative step towards its local desired reference (x; - 5.- x)). This implies that the variation 5; is positive, i.e., 5,t As such, we can identify the interpolation point ş that resides on our desired integration path by the product between the gradient Va, and the variation 8,
possible identify it systematically. Recall
point ā residing
In light of the above discussion, we can integrate gradients over identified interpolations. we can obtain the variation between T; and æte, the required integral can be estimated. Defining the interpolations a set X = {(x11, ,...,,""........... ..mm) with m interpolations residing the integration path, can obtain the variation from x; to 2 by following Lemma] [ We use a((i instead of 전 here to distinguish multiple interpolated image points, and a.")) is the i-th feature of 2(3)
Lemma 1. Given two feature points x; and x/,, and uniformly distributed interpolation points x(()) ... {(()) between I; and x'z, the variation 6; = x4 -z x'2 is proportional to the average distance from ,, to all interpolation points Tr, and the proportionality coefficient is 2.
We provide proof of Lemma T in Appendix A.I Following Lemma 1, the attribution M, defined in Equation (3) can be estimated over m interpolations as follows
X
M1 X)
(4)
where X is the proportionality coefficient, and 5(5) is the variation between x; and a.(").
The above insights are simple yet powerful. They can be leveraged to re-calibrate attribution scores computed by the existing path attribution methods. Current integral- attribution methods with multiple references, e.g., IG-SG, EG and AGI, employ different strategies to sample the references and integrate the gradients of interpolation points from the input to the references. These points can be viewed as adopting different sampling strategies to approach the input space via Monte Carlo estimation (Erion et al.) 2021). This view allows us utilize our technique with these attribution methods to modulate their interpolations for the integral estimation. In the text to follow, we distin- guish the interpolation points resulting from incorporating our insights into an attribution technique as valid interpolations. In essence, current methods end up estimating attributions using interpola- tion points that are not valid, which causes them to compute sub-optimal attributions.
5

---

Published as a conference paper at ICLR 2023
Step I: Interpolations from given input to references.
Step. 3: Identifying Valid Interpolation points.
Tntorpolations
Desred Referenee
X
Ingad.X
Variations change sign.
Step 2: Variations and gradients calculation.
Varkations
(reggative to positive
807
Gradients
Gradients
(predtive to meratiiv
Step 4: Integral estimation over valid interpolations Integgra
M, X)
6 6 40
Figure 2: Illustration of integration over valid interpolations. Step I: For an input æ, multiple inter- polated images are computed with given references. Step 2: Variations and gradients are computed for the interpolated images. Step 3: Variations become positive when they cross-over the input, and gradients become positive when they over-shoot the desired reference. Between these two extremes, interpolation points remain valid. Hence, they are selected. Step 4: Only valid interpolation points used for integration, and the rest are discarded.
In Figure 2 the notion of integration using valid interpolation points is illustrated. The central idea of identifying the valid interpolations for any refer- ence, followed by the integration with Equation 4 is generic. In the figure, for the input ær x' is the desired reference in a gradient ascending direction."
Algorithm 1: Attribution Re- Re-Calibration
input Input æ, reference set D,
number of steps n.
output: Attribution M, initialize: M < 0
for each x' in D do IIterre references.
The first step produces interpolation points dif-
ferent references. Then, the variations 5/57 and the for < 1 to n do Integral path.
x' + =x x¹)
corresponding gradients V 200 are calculated in the second step. The red and blue colors indicate the" positive and negative values in o(") and V 255 whose sign change after crossing the input feature 3; and the desired reference xy separately. The variation s and the gradient always have the same signs for the valid interpolation points along the integration path, in the neighborhood of the input. This observation is
V Z1
(S.
if V (I; 2)) 0 then
// Integrate gradients with
valid interpolations.
M M+M (z: - Iz) x Vz,
M xvg(MM
used identify the valid interpolations. Finally, the attribution M, is computed over the identified valid interpolations following Equation 4
We also present the pseudo-code to re-calibrate the current integral-based methods with valid inter- polations in Algorithm[[ [ Given a feature of the input x. and a reference set D identified by the method's own strategy, we iterate over the references (Line 2). For each reference, the method com- putes the gradients of interpolated images / from the input to the reference (Lines 3-5). However, different from the existing practice, we let the integral to estimated on valid interpolations that satisfy the condition that V (x; 2,) is positive (Line 6-8). Finally, we compute the integral with the averaging operation (Lines 9).
4.3 EFFICIENT ATTRIBUTION RE-CALIBRATION WITH VALID REFERENCES
Sampling sufficient references and interpolation points allow us to estimate an accurate integral. However, limited computational resources must be accounted for by the attribution methods. To

---

Published as a conference paper at ICLR 2023
estimate an accurate integral with limited resources, we employ importance sampling (Klock & Van Dijk 1978). Let f()) be function of n inputs X = (x(1), ...., ((())) over the domain Ω E IR..
Then, the integral is defined as v = So f(X)((())) (X).X. Importance sampling approaches this integral as
[ f(X) a g(X) 9X))x,
(5)
where g(X) 0 for X Q probability function called the importance function. In order estimate the integral, we generate n samples x01)) from g(X) and estimate the integral by the following sample-men -eep formula.
1 f(x(1)
-= n2 g(x(0))
where the estimator á can converge to the integral v. Instead of randomly sampling inputs in the input space, importance sampling enables us sample input with the importance function for effi- ciently estimating the integral. Let f(X)/g(X) (X) be a function ((X), we have the following Lemma.
Lemma 2. If the ratio between the function (XX and (Xx) has small fluctuations, the estimator can be approached by the mean formula uniformly from (zx).
Ei= 1 B(x())), where x(1) ...., z(()) are sampled
Let ((X) be defined on a linear integration path from the input to the reference. If the underlying distribution of (X) fluctuates slightly, we can estimate the integral along linear path with ((X). Since only slight fluctuations cannot be guaranteed along the full path in our case, we approach f(X) with ((X) g(X) in different segments. Since the integral estimated from an input to its desired reference is also expected to have positive attribution, we relax the condition of the integral estimated on the valid interpolations with V 8 < 0 to the valid reference that has positive integral estimation for approaching a segment f(X). Then, we average positive integrals segments for estimating the final integral
k
a5(+)
M
1:
Oz,
a=0
i2=z'+o
The integral estimation is also guided by Lemma 1. Using the same number of references, the pro- posed technique enables attribution methods calculate more precise attributions without incurring needless computational overhead. In Appendix A.2 we provide the pseudo-code of our technique.
5 EXPERIMENTS
To evaluate the proposed technique, we conduct experiments to test both performance and sensitivity. We take existing popular multi-reference attribution techniques baseline methods; including IG- SG (Smilkov et al. 2017). IG-SQ(Hooker et al.] 2019). IG-Uniform (Sturmfels et al.) (2020), EG (Pan cet 2021) and Pan et al.) 2021). All these methods enhanced with the proposed re-calibration technique are benchmarked in our experiments. Additionally, other feature attribution methods, including InputGrad (Simonyan al. (2014), Integrated Gradients (IG) (Sundararajan al., (017), SmoothGrad (SG) (Smilkov et al. (017) and FullGrad (Srinivas & Fleuret, (201)) are also compared in our evaluation. More detailed hyper- byper-paramettr settings and extended experimental results are provided in Appendices A.3 and A.4
5.1 PIXEL PERTURBATION PERFORMANCE
Pixel perturbation is widely used quantitative evaluation metric for the attribution methods (Samek) et al. 2016 Annona et al. 2018 Srinivas Fleuret 2019 Yang al. (023). Since an attribution map is expected to identify the correct order of relative importance of the input features, pixel perturbation evaluation iteratively removes the most or the least salient pixels (Pixel Insertion and Deletion) and measures the output changes to quantify the performance of attribution methods. In this part, we employ DiffID metric (Yang et al. 2023) to measure the performance difference on images with inserted 1% most salient pixels and deleted 1 - -%% least salient pixels.
Figure 3 reports the fractional output change difference with pixel deletion and insertion on Ima- geNet 2012 Validation set (Russakovsky et al. 20155) evaluated on ResNet-34 (He et al., 2016b) and
7

---

Published as a conference paper at ICLR 2023
IG-Uniform
IG-SG
IG-SQ
BG
AGI
0.
Fractoonal changes
0--
0.5
0.6
2.4
0.4
0.2
2.2
533
0.2
2.2
222
0.0
61
01
641
25
06 6J7
2..09
0.0
511
0.
64
D. 6.7
3..00
3.1 0. 6.1 S.4 05 5.7 0.8 0.5
54
0.
Image with KSS most and least salient pixels
Absolute Value Reshet-34
Ours ResNet-
Absolute Value VGG-16
V66-16
Vanilla Value VGG-16
Figure 3: Results on DiftID metric ImageNet 2012 val. set. Higher values indicate better results. Enhancing methods with our technique provides a considerable performance boost.
IS LLLN KESO
KORK OF 1851
IS 2091
10:2 13.300
O4 3721
1841
3591
15-50P
IS 1750
rounDrad (0.155)
15 201
3620
10.0PP1
(3.1.))
2121
5.7
Practional output change
0.4
2.4
0.4
0.4
0.2
0.2
0:3
0.0
0.0
02 01
0.5
0.6
0.5
0.5
0.5
0.9
0.1 0.2 033 04 05 0.7 08 03
6.7
Image most
salient pizels
with and salient pixels
(a) DiMMD on ResNet-34
(b) DifMD on VGG-16
(c) DifRROA on CIFAR-10 (d) DiRROAR on CIFAR-100
Figure 4: DiffID results comparison of enhanced baselines with other existing methods using ResNet-34 (a), and VGG-16 (b) on ImageNet 2012 val. Baseline methods enhanced with technique are identified with AUC values for each curve is reported. Larger values are more desirable. DiftROAR results on CIFAR-10 (c) and CIFAR-100 (d) are also reported following the same convention. Larger AUC values imply better results.
VGG-16 (Simonyan& & Zisserman 2015). Higher values in these curves indicate better performance. The Absolute Value curves are obtained by allowing the methods to replace the negative attributions with their absolute values. For the Vanilla values, do not allow this operation. The results reported for five popular baseline multi-reference integral-based attribution methods. We can ob- serve that the proposed re-calibration technique consistently improves all the baseline methods by a large margin. It is also clear that the original methods heavily rely on taking the absolute value to provide explanations. The proposed technique strongly improves IG-Uniform, IG-SG and AGI. The improvements for EG and IG-SQ are also reasonably large. These results ascertain that our technique can enable an ceross-the-boart performance gain for the existing methods.
In Figure Fh and 4b. we provide further comparisons on DiffID baseline with the popular original exiting methods, including InputGrad, IG, SG and FullGrad. A noticeable performance gap is clearly visible between these methods and the enhanced baselines, favoring the enhanced techniques. The results show that our method provides notable positive off-set the state-of-the-art in integral- based attribution.
5.2 REMOVE AND RETRAIN PERFORMANCE
Although pixel perturbation can provide an estimate of the interpretability of the attribution meth- ods, Hooker et al. (211)) argued that pixel perturbations also cause a distribution shift for the inputs, and thus it unclear whether the output change comes from removing informative pixels or the distribution shift. To alleviate the effect of input distribution shift, RemOve and Retrain (ROAR) (Hooker et al., 2019) a commonly used metric that retrains the model with the perturbed images produced by the pixel perturbation and measures the output change. To provide a more compre- hensive evaluation, DiftROAR (Shah et al. 2021) extends the ROAR metric to test the difference between the model retrained with the 5% most and least salient pixels. To thoroughly establish the effectiveness our technique, we also conduct experiments using DifiROAR. We again enhance the popular attribution methods IG-Uniform, IG-SG, IG-SQ, EG and AGI for this evaluation.
8

---

Published as a conference paper at ICLR 2023
IG-Uniform
IG-SG
IG-SQ
EG
AGI
Accurscy affierenc
65
0.0
0.7
0.4
0.5
0.4
0.5
6.3
0.3
0.3
0.
02
0.1
0.1
0.0
0.7
0.4
0.6
0.8
0.0
52
0.4
06
9.
Imaaee with 196 most and least salient pizels
Absoulte OIAR-1100
Ours on CIFAR-100 Absoutte Value CIFAR-10
CIFAR-10
Figure 5: Results on DiffROAR metric on ImageNet 2012 val. set. Larger values indicate better performance. Methods achieve consistent gain with our enhancement.
Pearson Co Coefiielent
IG-Uniform
IG-SG
IG-SQ
EG
AG1
0.15
6.15
0.35
0.32
0:03
0.03
0.05
-0.55
-3.05
-..9
-0.55
011
010001040525070111
0. 01020000055062200
Sampling size ratio K3
Vanilla Value
Figure 6: Sensitivity-n evaluation. Higher values indicate better performance.
We retrain the PreActResNet-18 (He et al. 2016a) on both CIFAR-100 and CIFAR-10 datasets (Krizhevsky al., 2009). Figure 5 shows the accuracy change difference on the model retrained with the most and least important 1% pixels separately. Since retraining models with different perturbed inputs causes these models to fit different input distributions, curves in Figure 5 have different undulations Nevertheless, the results establish that our method again generally improves the performance all the methods. We also summarize our quantitative evaluation using DiftROAR in Figure and 4a for CIFAR-10 and CIFAR- 100 datasets. The figure compares the results with other existing methods which are used in their original form. These results also consistently favor our technique of enhancing the methods.
can observe that the compared to the pixel perturbation experiments.
5.3
SENSITIVITY-N PERFORMANCE & FURTHER RESULTS
Ancona et al. (018)) proposed Sensitivity-n metric to verify that the model output variations are sensitive to the features as prescribed by the attribution method. Given any subset of features æs = [#1..... 2ml] S 2, Sensitivity-n requires the sum of attributions M, to hold Σ M, S.(x) - S0.-SS((1aa ()) Although no method can expected to fully satisfy Sensitivity-n due to prac- tical reasons, the definition still enables a pragmatic benchmarking scheme. By varying the feature fraction in the subsets the range [0.1, 0.9] of the total features, we test the Pearson Correlation Coefficient (PCC) computed between the sum of the attributions and the target output variation, shown in Figure 6 For each feature subset, we sample 100 different subsets of the input features with uniform probability distribution. The PCC is averaged over 1000 testing images of ImageNet 2012 Validation set. Compared with the vanilla attribution map, we can observe that the proposed re-calibration generally shows better sensitivity-n results for all the baseline methods. In Appendix A.4.3 we provide more results for the Sensitivity-n metric.
CONCLUSION
We identified an inconsistency between the theoretical treatment of path integral-based attribution and its practical implementation in various methods. We address this by devising a scheme that re-calibrates attribution computation by identifying desired reference and selecting only valid in- terpolation points for integration. This scheme applicable to all integral-based attribution meth- ods. Enhancing range of multi-reference methods with it, showed considerable performance gain on multiple evaluation metrics. We also demonstrated sanity preservation and improved model robustness by using the proposed scheme attribution prior regularization.

---

Published as a conference paper at ICLR 2023
ETHIC STATEMENT
Explaining the model prediction is highly relevant to the safety and privacy of users. In this paper, we improve the proppapppaattion based feature attribution methods. The improved attribution methods need to fully access the model and inputs to produce explanations. Therefore, it critical to carefully protect the safety of the training data and trained models for practical applications.
REPRODUCIBILIIT STATEMENT
The detailed experimental setup including the pyper-paramenn choice of compared attribution meth- ods on different datasets with the specific experimental framework and platform is mentioned for reproducing our experiments in Appendix A.3 In addition, the implementation code can be found in the supplementary material.
ACKNOWLEDGMENT
This research was supported by ARC Discovery Grant 190102443. Professor Ajmal Mian is the cipient of an Australian Research Council Future Fellowship Award (project number FT210100265 funded by the Australian Government. Dr. Naveed Akhtar is the recipient of Office of National Intelligence, National Intelligence Postdoctoral Grant (project number NIPG-2021-001) funded by the Australian Government.
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Advances in Neurallaformation Processing Systems, NeurIPS, 2018.
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks. In International Conference on Learning Representations, ICLR, 2018.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer- wise relevance propagation. PloS one, 2015.
Shan Carter, Zan Armstrong, Ludwig Schubert, Ian Johnson, and Chris Olah. Activation atlas. Distill, 2019.
Jiefeng Chen, Xi Wu, Vaibhav Rastogi, Yingyu Liang, and Somesh Jha. Robust attribution regular- ization. In Advances in Neurallfformation Processing Systems, NeurIPS, 2019.
Gabriel Erion, Joseph D Janizek, Pascal Sturmfels, Scott M Lundberg, and Su-In Lee. Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence. 2021.
Ruth C Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful pertur- bation. In International Conference on Computer Vision, ICCV, 2017.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, ICLR, 2015.
Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, ECCV, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2016b.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretabil- ity methods in deep neural networks. In Advances in NeuralIfformation Processing Systems, NewrlPS, 2019.
10

---

Published as a conference paper at ICLR 2023
Tuen Kloek and Herman K Van Dijk. Bayesian estimates of equation system parameters: ap- plication of integration by monte carlo. Econometrica: Journal of the Econometric Society, pp- 1-19, 1978.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Advances in Neurallnformation Processing Systems, NeurIPS, 2018.
Deng Pan, Xin Li, and Dongxiao Zhu. Explaining deep neural network models with adversarial gradient integration. In International Joint Conference on Artificial Intelligence, IJCAI, 2021.
Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: randomized input sampling for explanation of black-box models. In British Machine Vision Conference, BMVC, 2018.
Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi Velez. Right for the right reasons: Train- ing differentiable models by constraining their explanations. In International Joint Conference on Artificial Intelligence, IJCAI, 2017.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):11--552 2015.
Wojeiech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. Evaluating the visualization of what a deep neural network has learned. IEEE Transac- fions on Neural Networks and Learning Systems, TNNLS, 2016.
Harshay Shah, Prateek Jain, and Praneeth Netrapalli. Do input gradients highlight discriminative features? Advances in Neuralliformation Processing Systems, NeurIPS, 2021.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International Conference on Machine Learning, ICML, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, ICLR, 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. In International Conference on Learning Representations, ICLR, 2014.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. CoRR, abs/1106.03825, 2017.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net. International Conference on Learning Representations, ICLR, 2015.
Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. In Advances in Neurallfformation Processing Systems, NeurIPS, 2019.
Pascal Sturmfels, Scott Lundberg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 2020.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference Machine Learning, ICMI., 2017.
Christian Szegedy, Vincent Vanhoucke. Sergey loffe, Jon Shlens. and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2016.
11

---

Published as a conference paper at ICLR 2023
Peiyu Yang, Naveed Akhtar, Zeyi Wen, and Ajmal Mian. Local path integration for attribution. In AAAI Conference on Artificial Intelligence, AAAI, 2023.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, ECCV, 2014.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Rep- resentations, ICLR, 2017.
12

---

Published as a conference paper at ICLR 2023
A
APPENDIX
CONTENTS
Appendix
13
A.
Proof
14
A.2
Pseudo-code to Re-Calibration Attribution with Valid References
14
A.3
Experimental Semup
14
A.3.1
Hyperparameter Choice
15
A.3.2
Experimental Software and Platform
15
A.4
Extended Experiments
15
A.4.1
Extended Experiments on Valid Interpolations and Valid References
15
A.4.2
Extended Experiments Attribution Prior
17
A.4.3
Extended Experiments on Sensitivity-n
17
A.4.4
Extended Experiments Sanity Checks
18
A.5
Visual Inspection
19
A.6
Limitations
20
13

---

Published as a conference paper at ICLR 2023
A.1 PROOF
In this section, we provide the proof of Lemma and Lemma 2
Proof of Lemma 1. Given two points 2, and oy. n interpolations X ( (x uniformly distributed between z; and x The average distance d from æ; to all interpolation points calculated follows.
n d
a(2; xx{) 1
n
n1
(3i x/)) n
(8)
+...
n
n
n
1 (2)
x!)).
2n
Then, the limit of 1+n (a:; zt), as the number of interpolations approaches positive infinity, is 2n
(r, 2t), donated as follows.
1 (z, 2/) x; z!)
lim
(9)
2n
Thus, the distance xz x between 2; and 2 is proportional the the average distance d with coefficient donated as = 2d.
Proof of Lemma 2. If the ratio fx))/((x) fluctuates slightly, the importance function g(x) from which we sample the inputs can be approximatively viewed as a uniform probability density func- tion. Therefore, the importance scores ((X,)..... ((x,) can be calculated with the uniformly sampled inputs XI,..., X, as follows.
B(X)) = f(X.)
(10)
9(X,)*
Thus, the 3(X;) can be used to estimate the integral é.
A.2 PSEUDO-CODE TO RE-CALIBRATION ATTRIBUTION WITH VALID REFERENCES
In this section, present the pseudo-code of our technique to re-calibrate the integral- meth- ods with valid references in Algorithm 2 Given a feature x; of the input x, and a reference set D identified by the method's own strategy, we iter- ate over the references (Line 2). For each refer- ence, the method computes average ÿ of gradi- ents with respect to the interpolated image 7 from the input the reference x' (Lines 3-6). Differ- ent from the existing methods, we let the integral be estimated on the valid references that satisfy the condition that (z; xx) is positive (Lines 7-9). Finally, we approximate the integral with the averaging operation (Line 10). As such, We can re- calibrate the integral- -aaeed methods by identifying valid references with valid references without the additional computational overhead.
Algorithm Attribution Re- Re-Calibration
input Input ır, reference set D, number of steps n.. output: Attribution M;
initialize: M 0, 9 +
2 for each x' in D do // Iterate references.
for l: Ι x' + L
do // Integral path.
fx (x-x')
// Compute the average gradient.
g - 9 as,(#)
/n
85:
if (z; x 2 0 then
Integrate integrals estimated
with valid references.
M MU (x; x¹) x
10 Mi x avg(M)
A.3 EXPERIMENTAL SETUP
In this section, we provide details of the experimental setup, including the hyperparameter choice and the experimental software and platform.
14

---

Published as a conference paper at ICLR 2023
A.3.1 HYPERPARAMETER CHOICE
Hyperparameter Choice of Attribution Methods. In our experiments, we employ five popular integral-based attribution methods as our baseline methods including IG-Uniform (Sturmfels et al. 2020). IG-SG (Smilkov et al. 2017). IG-SQ (Hooker et al. 2019). EG (Erion al.) (021) and AGI Pann et al. 2021). Specifically, IG-Uniform defines multiple by sampling references from uniform noise whereas IG-SQ squares the gradients the integral estimation. Similarly, IG-SG adds Gaussian noise to the input as references. EG samples training images as the reference to as- sume the underlying training distribution as the reference. AGI computes class-specific adversarial examples as references. For the experiments 2013), we select 10 references and interpolations (k=5) for IG-Uniform, IG-SG and IG-SQ. Be- sides, chose 50 references and one random interpolation (k=1) for EG, and set 10 interpolations (k=10) and 5 class-specific adversarial references for AGI as the recommended settings in these papers. For fair comparison, we ensure all these methods use the same number of 50 back propa- gations. Considering the small input size of CIFAR-10 10 and CIFAR-100 datasets (Krizhevsky et al., 2009). we employed 30 back propagations for all the baseline attribution methods. We employed the same hyperparameter settings for all the experiments in this paper (e.g., with pixel perturbation, DifiROAR, Sensitivity-n, etc.). In addition to the bascline methods, other compared methods that rely on multiple gradient computations including Integrated Gradients (IG) and SmoothGrad (SG) are also implemented with the same hyperparameters. In IG, we produce 50 interpolations (k=50) for the final integral estimation. In SG, we average 50 gradients on inputs with Gaussian noise.
the ImageNeet22012 dataset (Russakovsky et al.,
Hyperparameter Choice Explained Models. Three deep models including VGG-16, ResNet- 34 and PreActResNet-188 are chosen to be explained by different attribution methods in our ex- periments. For the pixel perturbation and Sensitivity-n benchmarks. VGG-16 and ResNet-34 net- works are trained on the ImageNet-2012 training dataset. For the DifiROAR benchmark, we first train a PreActResNNet-I 18 on CIFAR-10 and CIFAR-100 training datasets separately. Then, the PreActResNet-I1 18 is fine-tuned for total of 10 epochs with the initial learning rate 10-2 decayed by 10 on the 5-th and 7-th epochs.
A.3.2 EXPERIMENTAL SOFTWARE AND PLATFORM
All the experiments were conducted on a Linux machine with an NVIDIA GTX 3090Ti GPU with 24GB memory and 16-core 3. Intel Core 19-12900K CPU and 125GB main memory. All at- tribution methods are tested and trained on PyTorch deep learning framework (v1.12.1) with Python language.
A.4 EXTENDED EXPERIMENTS
Below extended experiments are conducted test the efficacy of the proposed technique.
A.4.1 EXTENDED EXPERIMENTS ON VALID INTERPOLATIONS AND VALID REFERENCES
In this paper, we propose two techniques to identify valid interpolations and references for estimating an integral along a non-linear integration path. Here, we provide more detailed experiments for further understanding the valid interpolations and references. We randomly sample 1,000 testing images from the ImageNet 2012 Validation set to show the effectiveness of both valid interpolations and references. Here, we take IG-Uniform as the baseline attribution method. On both ResNet-34 and VGG-16, we re-calibrate attribution maps produced by IG- IG-Uniform with valid interpolations and references. Figure 7 and 8, we show the AUC of the fractional output change difference under different numbers of references with a fixed number of interpolations (k=5).
Both valid references and interpolations can re-calibrate attributions. experimental results of the vanilla attribution map produced by IG-Uniform in Figure 3 Figure 7 and Figure g show that both valid interpolations and references can improve the interpretability of IG-Uniform by large margin. The significant improvement demonstrates the effectiveness of the assumed desired reference and the proposed re-calibration technique.
comparison with the
Valid References lead to efficcent integral estimation. Different from valid interpolations, valid references employ importance sampling to efficiently estimate the integral. Figure 77p shows that integrating with valid references enables the attribution method to achieve high performance with
15

---

Published as a conference paper at ICLR 2023
AUC of Outpus Change Difference
0.50
Average Valiel Interpoiation Number
100
Volld
AUC
0.58
oude Change Diferens
20.0
Avg. Volld
AUC
0.58
Oelou Orange Dimorence
0.56
15.0
0.54
0.54
12.5
0.54
0.52
0.52
10.0
0.52
0.50
0.50
0.50
5.0
0.48
Volid intorpoistions
20
0.48
2.5
0.48
Volid References
0.46
10
15
30
0.46
0.0
0.46
The Number of Reference with Interpolations k=5
(a) Valid Interpolations
(b) AUC under different
(c) AUC under different. valid references
and references
valid interpolations
Figure 7: The comparison between valid interpolations and valid references on ResNet-34. (a) The AUC of fractional output change difference comparison between valid interpolations and references. (b) The relationship between the AUC and average valid interpolations. (c) The relationship between the AUC and average valid references.
AUC Outppt Change Difference
0.60
Average Valld Interpolation Number
100
0.60
20.0
0.60
malg Diffornnce
fug.
0.55
an Bap Change 20
Volid
AUC
0.55
0.50
0.50
15.0
0.50
0.45
0.45
12.5
0.45
0.40
40
0.40
10.0
0.40
Rverspo valld
0.35
0.35
5.0
0.35
0.30
Valid Interpolations Volld References
0.30
2.5
0.30
0.25
10
15
20
25
30
10
15
25
0.25
0.25
The Number of Reference with Interpolations k=S
Valid Interpolations and referenes
(b) AUC under difffrent
(c) AUC under dlifferent valid references
valid interpolations
Figure 8: The comparison between valid interpolations and valid references on VGG-16. (a) The
AUC of fractional output change difference comparison between valid interpolations and references. (b) The relationship between the AUC and average valid interpolations. (c) The relationship between the AUC and average valid references.
small number of references. However, estimating an accurate integral with valid interpolations requires large number of references.
More references lead to high performance. Figures 7 and Figure 8 & shows the relation- ship between the AUC fractional output change difference and the number of references. We can observe that more references and interpolations lead to high performance by re-calibrating with both valid interpolations and references, which demonstrates that more references can cover a more accurate integration path.
Drawbacks and superiority of valid references. In Figure we can observe that IG-Uniform re-calibrated with valid references cannot outperform the results calibrated with valid interpolations on VGG-16, which is inconsistent with the results on ResNet-34. The reason is that the valid refer- relies a relaxed condition approach an accurate integral. However, VGG-16 has more complicated decision pathway, which is hard to estimate an accurate integral along a linear inte- gration path. As compared VGG-16, ResNet-34 with more flattened decision pathway enables attribution methods to obtain accurate results with valid references, which also aligns with the ex- isting works that show the residual block can flatten the loss landscape (Li et al., 2018). Although complicated model impedes the performance of valid references, valid references can still improve the reliability of baseline methods. Moreover, identifying valid references require less memory con- sumption than identifying valid interpolations. Thus, the efficiency of identifying valid references enables the attribution method incorporated into the model training process. addition, training robust model flatten the loss landscape of the model, which also favors the proposed valid references for robust model training. In Appendix A.4.2 we perform experiments regularizing the behavior of the model with feature attribution.
16

---

Published as a conference paper at ICLR 2023
1.0
EG (Ours) Regularization
EG (Ours) Regularization
0.8
Regularization Baseline (No Prior)
Regularization
Baseline (No Prior)
0.8-
0.7
Test accuracy
Test accuracy
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.2
0.3
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Standard deviation of Gaussian noise
0.1 0.2
0.4
0.5
Standard deviation Gaussian noise (b) PreActResNet with Pixel Attribution Prior on CIFAR-I0
(a) ML.Pwtth Pixel Attribution Prior MNIST dataset
The accuracy comparison under testing images with the increased standard deviation of Gaussian noise between an original model, the model trained with EG regularization term, and the model trained with EG (Ours) regularization term. (a) The accuracy comparison of three MLP models trained MNIST dataset. (b) The accuracy comparison of three PreActResNet models trained on MNIST dataset. Higher values indicate better results.
Figure 9:
A.4.2 EXTENDED EXPERIMENTS ON ATTRIBUTION PRIOR
In addition to explaining a model prediction, attribution methods are also expected to regularize the behavior for constructing a robust model. Therefore, many regularization terms (Ross et al., 2017 Erion et al. 2021) are proposed to regularize the behavior by the attributions for obtaining a robust model. Erion et al. (20TT) assert a prior for the image classification task that adjacent features or pix- els should have similar attributions for the model prediction. To regularize the model behavior with this prior knowledge, they employed a Laplace zero-meat prior on the difference between attribu- tions of neighborhood pixels to encourage the adjacent pixels to have similar attributions. Since the regularization term encourages the input pixels to make the prediction jointly, the trained model is robust the input with Gaussian noise. In this part, we also combine the prior with the re-calibrated attributions as the regularization term for training a robust model. We train two robust models with the regularization term on MNIST (LeCun et al.] 1998) and CIFAR-1O (Krizhevsky al. (009) datasets. For MNIST, we train a multi-layer perceptron (MLP) with two convolution layers and one fully connected layer. On CIFAR-10 10 dataset, we take Pre (He et al. 2016a) as the baseline network. On both datasets, we employ one interpolation (k=1) and 5 references randomly sampled from the training dataset in the training process. For each dataset, we compare the original model, the model trained with EG regularization and the model trained with re-calibrated EG regu- larization to test their robustness on the test set with Gaussian noise. Figure ( shows the accuracy change with the increase of the standard deviation of Gaussian noise on the three models. Although two robust models (EG and EG Ours) can improve the robustness of the baseline model on Gaussian noised input, we observe that our method can outperform the regularization term with Expected Gradients (EG) on both two datasets, which demonstrates that the proposed re-calibration technique can effectively enhance the reliability of EG.
A.4.3 EXTENDED EXPERIMENTS ON SENSITIVITY-N
In this part, we provide more experimental results on Sensitivity-n metric. Figure I0 shows the results of ResNet-34 on Sensitivity-n, which shows that the proposed technique can also improve the sensitivity of the model on ResNet-34. In addition, Fi shows the results of PretctResNet- 18 on CIFAR-10 10 and CIFAR- 100 datasets Sensitivity-n metric. These experimental results also ascertain that our re-calibration technique enables the vanilla methods to improve their sensitivity under smaller sampling set sizes. Since Sensitivity-n benchmark randomly removes pixels from the input to test their sensitivity, the re-calibrated attribution maps with well-aligned explanations for the foreground object are more robust to the random removal than the vanilla attribution maps.
17

---

Published as a conference paper at ICLR 2023
Pearson Correlation Coefftiient
IG-Uniform
IG-S6
IG-SQ
EG
AG1
0.00
9.00
0.58
310315941584076009
3302133/851447666
Sampling set size ratio K%
Vanilla Value
Figure 10: Experimental results of ResNet-34 on Sensitivity-n benchmark. Higher values indicate better results.
IG-Uniform
IG-SG
IG-SQ
AGI
[PreActResNet-18 CIFAR-10 dataset)
0.17
0.3
0.5
5.15
Pearson Correlation Coefficient
0.00
8:00
0.00
2.00
-0.00
[PreActResNet-18 on CIFAR-100 dataset)
013
0.20
5.30
0.00
9:00
0.55
0.00
5.00
Sampling
Ours
+
Vanilla Value
Figure 11: Experimental results CIFAR-I0 and CIFAR-100 datasets on Sensitivity-n benchmark. Higher values indicate better results.
A.4.4 EXTENDED EXPERIMENTS ON SANITY CHECKS
In addition to the quantitative evaluation benchmarks, Adebayo et al. (2018) proposed two sanity checks that attribution methods should pass including: (i) the method should provide different at- tributions for the model trained with randomly permuted labels, and (ii) the attribution map should show a substantial change to the model parameter randomization. In this part, we test the baseline methods and their re-calibrated variants for the two sanity checks.
Data randomization test. Since deep models can casily random labels Zhang et al. (2017), data randomization test permutes the training labels train model and test the sanity of attribu- tion methods. In Figure [2] we compare the attribution methods for the models trained with true and random labels. The first four columns show the comparison between the attribution maps cali- brated with our technique and taken absolute values. The second four columns show the comparison between re-calibrated attribution maps and the vanilla attribution maps. Compared with the base- line methods, our method leads to a larger change in the calculated attribution maps. In addition, we quantitatively compare the similarity between the attribution maps trained with true and ran- dom labels. Figure 13 shows the comparison of Spearman rank correlation between the attribution maps produced by the baseline methods and re-calibrated methods. Compared with both absolute attributions and vanilla attributions, the experimental results show that the proposed re-calibration technique improves the sanity of the baseline attribution methods for the data randomization test.
Model parameter randomization test In addition to the sanity of the data randomization. the at- tribution methods are also expected to pass the model parameter randomization test. In our exper- iments, successively randomize the weights of an Inception V3 model (Szegedy al. 2016) from top to bottom layers. Figures [4 [7 the examples the attributions maps as the param- eter cascading from the top to bottom. We mark attribution maps produced by the baseline method
18

---

Published as a conference paper at ICLR 2023
Ours and bsolute Values
O urr and Vanilla Values
True Labels
Random Labels
True Labels
Random Labels
True Labels
Random Labels
True Labels
Random Labels
IG-Uniform
Uniffoofom (0urs)
IG-SG
IG-SG (Ours)
IG-8Q
IG-SQ (Ours)
EG
EG (Ours)
AGI
AGI (0urs)
Figure 12: Data randomization test for the baseline attribution maps (vanilla and absolute values) and the proposed re-calibrated attribution maps for a true label trained model and a random label trained model.
and our re-calibration method with blue boxes for easy comparison. We can observe that the cali- brated attribution maps are more sensitive with the increased randomization layer numbers, which shows the proposed technique enables the baseline methods to improve the sensitivity for the model parameter randomization test.
A.5 VISUAL. INSPECTION
In this section, we provide visualization examples of calculated attribution maps for visual inspec- tion. Figures 18 20 show the comparison of vanilla attribution maps, attribution maps computed with absolute values and the attribution maps re-calibrated with our technique
ResNet-34. In addition, Figures 21 show the comparison of attribution maps on VGG-16. We can observe that the attributions re-caaibrteed with our technique enable the maps to better align with the fore- ground objects. In addition, vanilla attribution maps suffer from large variation between positive and negative scores both ResNet-34 and VGG-16.
19

---

Published as a conference paper at ICLR 2023
Speamann correlation
0.8
Spearman correlation
0.8
0.6
0.6
0.4
0.4
0.2
IG-Unitarm
G-Utloom (Dir)
i0 (hhn)
Eu
EG (ar)
AG (own
IC-Unfoom
IG-50
1G-50 (Ow))
Ks-s0
Am
Gun
16
Ssunifom
ic-50
BG
(a) The comparison of spearman rank correlation (where baseline methods with absolute value).
(b) T hee comparison of spearman rank correlation (where baseline methods without absolute value).
Quantitative evaluations for data randomization test. (a) The spearman rank correla- tion for baseline attribution maps with absolute values and re-calibrated attribution maps. (b) The spearman rank correlation for baseline attribution maps without vanilla values and re-calibrated at- tribution maps. Larger correlations indicate better results.
Figure 13:
A.6 LIMITATIONS
In this paper, to improve the inconsistency of integral- based attribution methods, we assume different references for different input features to re-calibrate attributions. Although our method has been shown to enhance the reliability of popular attribution methods, the modification also carries the risk of violating the completeness axiom. Therefore, exploring an explainable method that guarantees the consistency between practice and their theoretical guarantee and their practical implementation still important task. addition, we devise our method based on theoretical insights. However, precisely quantifying the impact of these insights is not possible due to the intrinsic ambiguities in the quantitative evaluation metrics used in this research direction. This a limitation of this domain in general. Nevertheless, we verify that method consistently improves the evaluation scores. In practice, it may be important to verify that the used evaluation metric represents the real-world scenario appropriately.
20

---

Published as a conference paper at ICLR 2023
Input
Cascading randomization from top to bottom layers
Ordno
Xhau 76
Xbbo Th
X SM
X Lc
Mied de
Crectt
IG- U--Uniform
IG-Unilorm
IG-SG
(Ours)
IG-SQ
IG-SQ (Ours)
EG (Ours)
AGI
AGI (Ours)
Figure 14: Model parameter randomization test on Inception v3 for the testing image from the ImageNet-2012 dataset. The attribution maps produced by baseline methods and our calibration method are compared cascading randomization from top to bottom layers.
Input
Cascading randomization from top to bottom layers
bd
76
MRnet PM
MBud
IG-Uniform
IG-Uniform (Ours)
IG-SG
(Ours)
IG-SQ
IG-S0 (Ours)
(0urs)
AGI
AGI (Ours)
Figure 15: Model parameter randomization test on Inception v3 for the testing image from the ImageNet-2012 dataset. The attribution maps produced by baseline methods and our calibration method are compared as cascading randomization from top to bottom layers.
21

---

Published as a conference paper at ICLR 2023
Input
Cascading randomization from top to bottom layers
Ordgr
XBbodd Th
Mire LM
Mind de
Crect
IG- GG-Uniform
IG-Uniform urs)
IG-SG
(Ours)
IG-SQ
IG-SQ (Ours)
EG (Ours)
AGI
AGI (Ours)
Figure 16: Model parameter randomization test on Inception v3 for the testing image from the ImageNet-2012 dataset. The attribution maps produced by baseline methods and our calibration method are compared cascading randomization from top to bottom layers.
Input
Cascading randomization from top to bottom layers
Conad bd
Shod 7
Mirrr PM
IG-Uniform
IG-Uniform (Ours)
IG-SG
(Ours)
IG-SQ
IG-S0 (Ours)
(0urs)
AGI
AGI (Ours)
Figure 17: Model parameter randomization test on Inception v3 for the testing image from the ImageNet-20112 dataset. The attribution maps produced by baseline methods and our calibration method are compared as cascading randomization from top to bottom layers.
22

---

Published as a conference paper at ICLR 2023
IG-Uniform
IG-SG
IG-5Q
F6
AGT
Varilla
Ro-Calbratee Abarhtte Valur
Input
Attributions
Positive to Negative
Figure 18: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on ResNet-34.
IG-Uniform
IG-SG
IG-SQ
EG
AG1
Vanilla
Ro-Calllraeed Absolute Value
Input
Attributions from Negative
Figure 19: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on ResNet-34.
IG-Uniform
IG-SG
IG-SQ
EG
Vanilla
Abezute Value
Input
Ro-Callbrated
Attributions from Positive to Negative
Figure 20: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on ResNet-34.
23

---

Published as a conference paper at ICLR 2023
IG-Uniform
1G-SG
1G-5Q
FG
AGT
Vanilla
Ro.Calbrotre Abscltte Valua
Input
+
Attributions Positive to Negative
Figure 21: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on VGG-16.
IG-Uniform
IG-SG
IG-SQ
EG
AGI
Vanilla
Ro-Calllraeed Absoltte Value
Input
Attributions from Negative
Figure 22: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on VGG-16.
IG-Uniform
IG-SG
IG-SQ
EG
AGI
Vanilla
Ro-Calibrated Absoltte Value
Input
Attriuutions from Posttive to Negative
Figure 23: The comparison of vanilla attribution maps, attribution maps taken absolute values, and attribution maps calibrated with the re-calibrated technique on VGG-16.
24


Reviews:
[{"review_id": "tl6_HzMVUo", "reviewer": "Reviewer_brex", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method to improve the path along which the gradients are integrated in integral-based attribution methods for explaining the decisions of deep neural networks (DNNs).\n\nThis method is based on the observation that while integrated gradients (IG) are theoretically well motivated, the necessary assumptions are often violated in common practice (e.g., by taking the absolute value of the computed attributions). To overcome this, the authors propose a method to select \"valid\" interpolation points for the integration, which are independently chosen for each input feature.\n\nIn a wide range of experiments with different datasets (CIFAR-10, CIFAR-100, ImageNet), the authors report consistent improvements according to various metrics (pixel perturbation, DiffROAR, sensitivity-n) when extending commonly used integral-based methods (IG-Uniform, IG-SG, IG-SQ, EG, AGI) with their proposed method for various networks (VGGs and ResNets).", "summary_of_the_review": "While the experimental results show promising gains in the employed metrics for interpretability, I am hesitant to recommend the paper for publication in its current form. If the authors are able to address my concerns, I am open to increasing my score (see weaknesses).", "strength_and_weaknesses": "### Strengths \n- The experimental evaluation is extensive and shows that the proposed approach yields consistent improvements over a wide range of different settings, as measured by various metrics for assessing the quality of attribution methods.\n- The authors aim to address an important shortcoming of commonly employed integral-based attribution methods, namely their dependence on the chosen baselines as well as an inconsistency between the theoretical motivation for IG methods and their practical usage.\n- The paper is generally well structured and (apart from sections 4.2 and 4.3, see weaknesses) very well written and easy to follow. \n\n### Weaknesses\nWhile the experimental results show promising gains in the employed metrics for interpretability and the approach is well motivated, I am still hesitant to recommend the paper for publication in its current form. If the authors are able to address my concerns, I am open to increasing my score.\n\n- **Lack of clarity w.r.t. method and motivation**: I have the following issues with and open questions regarding the method and would appreciate if the authors could elaborate. \n\t- As far as I understand, the proposed method is motivated (apart from the described *inconsistency*) by the high spatial variance in the attribution maps of integral-based methods, as shown in Fig. 1 (top row). Commonly, the absolute value is taken to overcome this problem (see Fig. 1, second row), which the authors criticise as yielding attribution maps that are inconsistent with the underlying theory and thus do not faithfully reflect the explained model. Judging from Algorithm 1, the main difference of the proposed method is that the authors employ a ReLU activation over the pixel attributions for each reference image and average the result — *i.e.* instead of taking the absolute value to 'hide' the high variance in the attribution maps, the authors simply clamp the attributions at zero to do so. As such, while the experimental results seem to speak in favour of the proposed method, I currently fail to see how this approach is fundamentally different w.r.t. being better motivated than taking the absolute value. \n\t- There is a stark contrast between the simplicity of the actual implementation of the proposed method (Algorithm 1) and its description and derivation. To exaggerate a bit, it seems like a very complicated way of saying \"Instead of taking the absolute value, we average the ReLU-activated attributions over multiple reference images\". This is aggravated by a lack of clarity in the method description (see next bullet).\n\t- After carefully reading the method section multiple times, I am still confused as to how the reference set D is obtained. In Algorithm 1, the reference set seems to be given, while in the text it says the reference set is \"obtained by the method's own strategy\". As far as I can tell, this seems not to be referring to the authors' method, but simply to the reference set as used in prior work (e.g., uniform noise), i.e. in the respective base method which the authors extend by their approach. However, this seems to be contradicting the statement that \"we \\[...\\] modify the input image with the model gradients to construct the reference\", as well as the general notion of \"\\[computing\\] the reference along the gradient ascending direction \\[...\\] \\[similar to\\] Adversarial Gradient Integration\", which is also centrally placed in the abstract (\"The reference is computed in a gradient ascending direction on the model's loss surface\"). I would highly appreciate if the authors could clarify.\n\t- What exactly is the relevance of Lemma 1? What is different from standard Integrated Gradients, which also estimates the integral in eq. 3?\n\t- The mathematical notation / naming is not optimal, which makes the method section difficult to read: \n\t\t- In eq. (3) the index i is used to index a specific feature dimension of an input x, but also to index a specific reference sample in the set D. \n\t\t- The equivalence between x' and D_i is also confusing—why not simply use D_i (optimally with a different index)? Further, using the term *variation* for the *difference* between two points is confusing. \n\t\t- What exactly is the set of signed input gradients in the second paragraph of 4.2.? What does this notation mean exactly (similarly for the variation / difference)?\n\n- **Missing reference**: Guided Integrated Gradients by Kapishnikov et al. (CVPR 2021) seems to be a highly relevant prior work that should be included in the discussion of related work as well as in the experimental evaluation. What are the main differences to and advantages over Guided Integrated Gradients?\n\nAdditional minor issues:\n- The result graphs are difficult to parse. I would recommend to additionally summarise the curves via a single number per method (e.g., AUC). This concerns all result Figs.\n- Further, the relevant comparisons (e.g., ResNet (Ours) vs. ResNet (Baseline)) are not easy to make in the current graphs, I would recommend grouping the results for better comparability. This is particularly difficult in Fig. 4—here, the relevant baselines are seemingly not even included (e.g., IG-SQ* is shown, but not IG-SQ etc.)?\n- In Fig. 6, the shaded areas are not explained in the caption, what do they represent? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "tME_gds2Es", "reviewer": "Reviewer_FTMa", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method to create the reference image for Integrated Gradient attribution approaches to interpret model predictions. The method re-calibrates the attribution scores without the additional computational overhead of the traditional approaches. The authors suggest this strategy is more relevant to model prediction and feature interpretation. The paper considers different aspects of the attribution techniques during the evaluation and obtains promising results.", "summary_of_the_review": "The proposed paper presents a promising approach with interesting results and analysis. The discussion is direct, but I was expecting more insights. The authors also should discuss the proposed approach's limitations.\n", "strength_and_weaknesses": "+ The proposed approach is interesting and presents promising results. \n+ The idea of calibrating attribution scores to capture relevant features and interactions is promising but has been around for a while. The novelty is incremental and helps solve some limitations of the IG approach. \n+ The experiments and analysis are clear, but the discussion could be improved. The authors have the opportunity of creating a link between uncertainty and explainability by exploring more about the relationship between calibration and interpretation.\n+ The integrated gradients method can assign different attributions to features that have the same effect on the model or assign positive attributions to features with no effect. Does the proposed approach solve those problems? I miss some qualitative discussions about the method performance. \n- The authors did not discuss the limitation of the proposed method. Therefore, it will be meaningful to discuss the gap between the experiments in the current version of the paper and the real-world applications.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RftH_9F0i1H", "reviewer": "Reviewer_rkn8", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper improves the previous path integration based model interpretation works by overcoming the limit of absolute based attribute scores. The method is mainly consisting of a) building a better \"reference\" than the manual way previously; b) identifying the interpolation point in the non-linear path through the relationship of \"variation\" and gradient; c) an efficient sampling method to make the computation tractable. The final method looks simple with a \"re-calibratable\" property on top of the current integral-based methods.", "summary_of_the_review": "Overall I think this is a good paper especially for the particular area of path integration based model understanding/feature attribution. It solves a deep and tricky limitation of previous methods with enough technical novelty/depth for ICLR publications. I would vote for its acceptance conditioned on a good re-writing of the paper: currently it is presented in a too technically heavy way which is not good for the broader ICLR committee. ", "strength_and_weaknesses": "**Strength**\n\n1. This paper brings a lot of meaningful technical depth. It deeply analyzes a tricky issue of the previously widely used method, and proposed the solution based on detailed mathematical analysis. A little bit surprisingly that the finally derived method seems to be simple with such a non-trivial technical setup which is impressive. \n\n2.  It is also technically sound and rigorous, with a) a good mathematical system; b) solid empirical verifications of the method.\n\n**Weakness**\n\n1. Mostly this paper is not reader friendly as the writing seems too technically heavy. In particular, a) a lot of long-sentences are used which should be further broken down; b) some concepts are introduced with a little bit sudden manner, especially in Abstract (e.g., \"attribution scores\", \"reference\" and \"local and global evaluation metrics\"). ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%