Robust attributions require rethinking robustness metrics
Authors: Sandesh Kamath
Amit Deshpande
Vineeth N. Balasubramanian
Venue: Submitted to ICLR 2023 (2023)
Keywords: Robustness
Attribution
Interpretable
Metrics

Abstract: For machine learning models to be reliable and trustworthy, their decisions must be interpretable. As these models find increasing use in safety-critical applications, it is important that not just the model predictions but also their explanations (as feature attributions) be robust to small human-imperceptible input perturbations. Recent works have shown that many attribution methods are fragile and have proposed improvements in either the attribution methods or the model training. Existing works measure attributional robustness by metrics such as top-$k$ intersection, Spearman's rank-order correlation (or Spearman's $\rho$) or Kendall's rank-order correlation (or Kendall's $\tau$) to quantify the change in feature attributions under input perturbation. However, we show that these metrics are fragile. That is, under such metrics, a simple random perturbation attack can seem to be as significant as more principled attributional attacks. We instead propose Locality-sENSitive (LENS) improvements of the above metrics, namely, LENS-top-$k$, LENS-Spearman and LENS-Kendall, that incorporate the locality of attributions along with their rank order. Our locality-sensitive metrics provide tighter bounds on attributional robustness and do not disproportionately penalize attribution methods for reasonable local changes. We show that the robust attribution methods proposed in recent works also reflect this premise of locality, thus highlighting the need for a locality-sensitive metric for progress in the field. Our empirical results on well-known benchmark datasets using well-known models and attribution methods support our observations and conclusions in this work.

Under review as a conference paper at ICLR 2023
ROBUST ATTRIBUTIONS REQUIRE RETHINKING ROBUSTNESS METRICS
Anonymous authors Paper under double-blind review
ABSTRACT
For machine learning models to be reliable and trustworthy, their decisions must be interpretable. As these models find increasing use in safety-critical applications, it is important that not just the model predictions but also their explanations (as feature attributions) be robust to small human-imperceppible input perturbations. Recent works have shown that many attribution methods are fragile and have proposed improvements in either the attribution methods or the model training. Ex- isting works measure attributional robustness by metrics such as top-k intersection, Spearman's 's rank-order correlation (or Spearman's p) or Kendall's rank-order corre- lation (or Kendall's 7) to quantify the change in feature attributions under input perturbation. However,
show that these metrics fragile. That is, under such metrics, a simple random perturbation attack can seem to be as significant as more principled attributional attacks. We instead propose Locality-sENSitive (LENS) improvements of the above metrics, namely, LENS-top-k. LENS-Speand Spearman and LENS-Kendall, that incorporate the locality of attributions along with their rank metrics provide tighter bounds on attributional ro-
order. Our locality-sensiin bustness and do not disproportionately penalize attribution methods for reasonable local changes. We show that the robust attribution methods proposed in recent works also reflect this premise of locality, thus highlighting the need for a locality- sensitive metric for progress in the field. Our empirical results well-known benchmark datasets using well-known models and attribution methods support our observations and conclusions in this work.
INTRODUCTION
There has been an explosive increase the use of deep neural network (DNN)-based models for many applications in recent years, which has resulted in an equivalently increasing interest in finding ways to interpret the decisions made by these models. Interpretability is an important aspect of responsible and trustworthy AI, and attribution methods are important for explaining and debugging real-world
AI/ML systems. Attribution methods are used across application domains today (see (Gade et al.
2020) for general discussion and (Tang et al.) Yap et al.) Oviedo et al. 22222] Oh & Jeong 2020
for some examples), despite their limitations. These methods (Zeiler et al. [2010) Simonyy an et al.
2014 Bach et al. 2015| Selvaraju al. 2017 [Chattopadhyay al. 2018 [Sundararajan et al. 2017
Shrikumar et al. 2016 Smilkov et al. 2017] Lundberg & Lee 2017) find approaches to explain the
decisions made by these models, instead of using them as a black box. With growing numbers of
explanation methods (see (Lipton 2018 (Samek et al. 2019 Fan et al. 2021 Zhang et al. 2020
Zhang & Zhu 2018) for surveys), there have also been recent concerted efforts on analyzing and proposing methods to ensure the robustness of DNN model explanations. This requires that the model explanations (also known as attributions) do not change with human-imperrt iammeniipprrepptbble changes in input (Chen et a 2019 Sarkar et al. 202T). For example, explanation code for credit card failure cannot change significantly for a small numan-imperceptiile change in input features, or the saliency maps explaining the risk prediction of a chest X-ray should not change significantly with a minor uuman-imperceptible change in the image. This is referred to as attributional robustness.
From another perspective, DNN-based models are known to have vulnerability to imperceptible adversarial perturbations (Biggio et al., 2013; Szegedy et al. 2014), which make them misclas- constructed using techniques like
sify input images. These small imperceptible perturbations Fast Gradient Signed Method (FGSM) (Goodfellow et al.) 2015) and Projected Gradient Descent (PGD) (Madry al. 2018). Adversarial training with PGD is well-known solution to obtain

---

Under review as a conference paper at ICLR 2023
better adversarial robustness to attacks like FGSM and PGD. While adversarial robustness has received significant attention over the last few years (Ozdag 2018 Silva & Najafirad, 2020). at- tributional robustness has received lesser attention. In an early effort, (Ghorbani et al.) 2019) provided a method to construct a small imperceptible perturbation which when added to the in- put will lead to commonly used correlation measures such as top-k intersection, Spearman's rank-order correlation or Kendall's rank-order correlation used to quantify the change between the explanation map of the original image and that of the perturbed image drop (see Figure I Defenses against such attributional attacks have been proposed recently in (Chen al., 2019; Singh et al. 2020: Wang et al., 2020; Saarar et al. 2021). D, which focus on regularizing the loss function (Chen et al. 2019 Singh et al.) 2020 Sarkar et al. [202] or use smoothing tech- miques on gradients (Wang al. 2020) to reduce the impact of the attack.
Across all the efforts far GGhorbani et al. 2019 Chen et al. [2019] [Singh Singh et al. 2020 Wang al. 2020 Sarkar et al.12 ), the robustness of attributions on input perturbation is measured using metries such as top-k intersection, and rank correlations like Spearman's 's p and Kendall' estimate the quality of the attack. While such metrics give reasonable estimate when there are significant changes in attributions (see Figure [ row I), they are highly sensitive to minor local changes in attributions, even by one or few pixel coordinate locations (see Figure row 2). We, in fact, show (Section 3.1) that under such metrics, random perturbation is strong attack as existing benchmark methods such as (Ghorbani et al., 2019). This may not be true indicator of the robustness of attributions of model, and thereby misleading to research efforts that build on current observations. Beyond highlighting this important issue, we instead propose locality-sensitive improvements of the above metrics that incorporate the locality of attributions along with their rank order. We show that such locality-sensitee distance is upper- bounded by metric based on symmetric set difference. Our key contributions are summarized below:
Figure Attributional attack on Flower datasct us- ing Ghorbani et al. (2019) method on a ResNet model. Columns and show the image before and after an imperceptible perturbation; Columns 2 and 4 show the corresponding attributions. Note the change in attribu- tions despite no perceptible change input. Row I shows a distinct change in top-k pixels with the highest attribution; Row shows only local change in top-k pixels with the highest attrihution still within the object. The intersection between the top-1000 pixels hefore and after perturbation less than 0.16 both the cases; thus, as a metric it cannot really distinguish the two.
We firstly observe that existing robustness metrics for model attributions overpenalize minor drifts in attribution, leading to false sense of fragility. We go on to show that under existing such metrics, random perturbation is as good an attack as principled methods like (Ghorbani al. 2019).
In order address this issue, we propose Locality-SENSiiee (LENS) improvements of existing metrics, namely, LENS-top-k; JENS-Spearmnn and Kendal, that incorporate the locality of attributions along with their rank order. Our Secality-sensitiv metries do not disproportionately penalize attribution methods for reasonable local changes.
We show that our proposed LENS variants are well-motivated by metrics defined the space of attributions, and they provide tighter bounds on the attributional robustness of already known i
improvements attribution methods and model training designed for better attributions.
Our comprehensive empirical results on benchmark datasets and models used in existing work clearly support our aforementioned observations, and support the need for the LENS variants of the metrics.
We also show that existing robust attribution methods implicitly reflect this premise of locality, thus highlighting the need for a bocality-sensitive metric for progress in the field.
2 BACKGROUND AND RELATED WORK
We herein discuss background literature related to work from three different perspectives: brief summary of explanation/attribttoon methods in general, review of recent work in attributional robustness (both attacks and defenses), and other recent related work.
2

---

Under review as a conference paper at ICLR 2023
Attribution Methods. Existing efforts explainability in DNN models can be broadly categorized as: local and global methods, model-agnostic and model-specific methods, or as post-hoc and ante-
hoc (intrinsically interpretable) methods (Molnar 2019: Lecue et al. (021). Almost all the popular
methods in use today- including methods to visualize weights and neurons (Simonyan et al. 2014
Zeiler & Fergus] 2014), guided backpropagation (Springenberg al.
2015). CAM (Zhou al.).
GradCAM (Selvaraju et al. 20177). Grad-CAM++ (Chattopadhyay et al.
2018). . LIME (Ribeiro et al.
2016), LIFT (Shrikumar et al. 2016 2017), LRP (Bach al.
2013), Integrated Gradients
(Sundararajan et al. 2017), SmoothGrad (Smilkov et al. 2017)), DeepSHAP (Lundberg & Lee (017) and TCAV (Kim et al. 2018) are all post-hoe methods, which are used on top of pre-trained DNN model as separate layer/module to explain its predictions. We focus such post-hoe attribution methods in this work. For a more detailed survey of explainability methods for DNN models, please Samek et al. 2019).
see (Lecue et al. 2021 Molnar 2019;
Robustness of A Attributions. With the growing numbers of attribution methods, there has also been a
recent focus on identifying the desirable characteristics of such methods Meliss & Jaakkola
2018 Adebayo et al.] 22188) Yeh et al. 2019] Chalasani et al. 2020 Tomsett et al., [2000] Boggust et al. 2022; Agarwal et al. 2022). A key desired trait that has been highlighted by many of these efforts is robustness or stability of attributions, i.e., the explanation should not vary significantly
within small local neighborhood of the input (Alvarez-Melis & Jaakkola 2018 (aalasaan et al.) 2020). Ghooranni et al. (2019) showed that well-known methods such as gradient-baser attributions, DeepLIFT (Shrikumar et al. (017)) and Integrated Gradients (IG) (Sundararajan et al.) 2017) are vuinerable, and provided an algorithm to construct a small imperceptible perturbation which when added the input results in change in the attribution. [Slack et al. (2020) showed that methods like LIME (Ribeiro et al. 2016) and DeepSHAP (Lundberg & Lee (017) too are vulnerable to such manipulations. The identification of such vulnerability and attack has subsequently led multiple research efforts that have attempted to make a model's s attributions robust. Chen et al. (019) proposed a regularization-basee approach, where an explicit regularizer term is added to the loss function to maintain the model gradient across input (IG. in particular) while training the DNN model. This was subsequently extended by (Sarkar et al.] 2021 Singh et al., 2020; Wang et al.) 2020), all of whom provide different training strategies and regularizers to improve attributiona robustness of models. Each of these methods including (Ghorbani et al,, 2019) measures change in attribution before and after input perturbation using top-k intersection, and/or rank correlations like Spearman's 's & and Kendall' т. Such metrics have recently, in fact, further been used to understand issues surrounding attributional robustness (Wang & Kong 2022). Other efforts that quantify stability of attributions in tabular data also use Euclidean distance or its variants (Alvarez-Melis & Jaakkola 2018 Ych et al. 2019, AAgawal et al.) 2022). Each of these metries look for dimension wise correlation or lexel-level matching between attribution maps before and after perturbation, and thus penalize even a minor change attribution (say, even by one pixel coordinate location). This may not be reasonable, and could even be misleading. In this work, we highlight the need to revisit such metrics, and propose locality-sensitive variant that can be easily integrated into all existing metrics.
Other Related Work. In other related efforts that have studied similar properties of attribttion-base explanations, Carvalho et al. (2019); Bhatt et al. (2020) stated that stable explanations should not vary too much between similar input samples, unless the model's prediction changes drastically. All
the abovementioned attributional attacks and defenses (Ghorbani et al.) 2019) /aarar et al. 1021
Singh et al. 2020 Wang et al. (020) maintain this property, since they focus input perturbations
that change the attribution without changing the model prediction itself. Similarly, AArun et al. (2020) and generalizability respectively, both of which focus on the desired property that a trustworthy explanation must point to similar evidence across similar input images. In this work, we provide a practical metric to study this notion of similarity by considering occality-sensitiv metrics.
and Fel et al. (022)) introduced the notions of repeatabiitty/eepoouubbbili
3 ATTRIBUTIONAL ROBUSTNESS METRICS ARE FRAGILE
We first discuss the fragility of existing metrics, before presenting our Docality-sensitvve variants. The robustness of an attribution method has generally been measured in earlier work (Ghorbani et al. 2019 Chen et al. 2019) by computing the similarity between attributions of an original input and the attribution of the same input with perturbations, and averaging this similarity across multiple input images. Similar to adversarial perturbations (Madry et al.) 2018), such "attributional perturbations" are carefully constructed attack vectors of small Exc norm that do not change the model prediction but only the attributions on the perturbed inputs (Ghorbani al. 2019). Using this, Ghorbani et al.
3

---

Under review as a conference paper at ICLR 2023
(2199) show that many popular attribution methods are fragile. The common similarity measures between attributions are top-k intersection and rank correlation coefficients such as Spearman's 's and Kendall's Note that all of the above similarity measures depend only the rank order of features in the attributions (e.g., rank order of pixels images).
3.1 RANDOM VECTORS ARE ATTRIBUTIONAL ATTACKS UNDER EXISTING METRICS
Random vectors a small So norm are often used as baselines of input perturbations (both in adversarial robustness (Silva & Najafrrd, 2020) and attributional robustmess literature Ghorbani
et al. (0119), since it is known that predictions of neural network models are known to be resilient random perturbations of inputs. Previous work by Ghorbani et al. (2019) has shown random perturbations to be a reasonable baseline to compare against their attributional attack. Extending further, we show that a single iput-agnostic random perturbation happens to be an effective universal attributional attack if we measure attributional robustness using weak metric based on top-k intersection, In other words, considering even a random perturbation happens to be a good attributional attack under such metrics, we show that existing metrics for attributional robustness such as top-k intersection are extremely fragile, i.e., they would unfairly deem many attribution methods as fragile.
Integrated Gradients (IG) is a well well-known attribution method based on well-defined axiomatic foundations (Sundararajan jan et al. 2017). which is commonly used in attributional robustness literature (Chen et al. 2019; Sarkar et al. 202T). We take a naturally trained CNN model on MNIST and perturb the images using random perturbation (an independent random perturbation per input image) as well as single, input-agnostic or universal random perturbation for all images. Figure shows sample image from the MNIST dataset and the visual difference between the IG of the original image, the IG after adding random perturbation, and the IG after adding universal random perturbation. The IG after the universal random attack (Figure 2d visually more dissimilar to the IG of the original image (Figure 2b) than the IG of simple random perturbation (Figure (Note that top-k intersection between Figure eggure2bannd and [2lis only 0.62, although the two look similar. As stated in the caption, locality-sensitrvv metric shows them to be closer in attribution however.)
777
7
(a) Original image (b) IG((riiginal image) (c) IG after random (d) IG after universal
Figure 2: Sample image from MNIST shows that the Integrated Gradients (IG) after a universal random perturbation more dissimilar than IG after simple, independent random perturbation for each input. All perturbations have random #1 coordinates, scaled down have for norm 0.3. (c) has top-k intersection of 0.68, while (d) has a top-k intersection of 0.62. With our 0.99 and (d) has LENS-top-k of 1.0.
metric, (c) has LENS toss--tpp--- of
Similarly, Table I shows that under existing metrics to quantify attributional robustness of IG on naturally trained CNN model, even single, input-agnostic or universal random perturbation can sometimes be more effective attributional attack than using an independent random perturbation for each input. The detailed description of our experimental setup can be found in Appendix
LOCALITY-SENSII SENSITIVE METRICS FOR ATTRIBUTIONAL ROBUSTNESS
raises the need look beyond current metrics used study attributional robustness. Current analyses of attributional attacks use similarity measures such as top-k intersection and rank correlation metrics such as Spearman's 's p, Kendall's 7 that ignore where the top attributions are located in the image. As an aside, these are not metrics in the mathematical sense but theoretically interesting metrics have been derived from them the ranking literature (Fagin et al., 2003).
Section 3
4.1 DEFINING SOCALITY-SENSITIIE METRICS FOR ATTRIBUTIONS
We propose a natural way to extend the existing similarity measures to further incorporate the locality of pixel attributions in images to derive more robust, locality-sensitive measures of attributional
4

---

Under review as a conference paper at ICLR 2023
Table I: Attributional robustness of IG on naturally trained models measured using average top-k intersection, Spearman's 's and Kendall's 7 between IGriiiaa image) and (p image). k 100 for MNIST, Fashion MNIST, GTSRB and 1000 for Flower. Bold entries indicate where the universal random perturbation is stronger attributional attack.
Dataset
Perturbation
top-k intersection
Spearman's
Kendall's 7
MNIST
random
0.7500
0,5347
0.4337
universal random
0.5855
0.4831
0.4063
Fashion MNIST
random
0,5385
0.6791
0.5152
universal random
0,5280
0.7154
0.5688
GTSRB
random
0.8216
0.9433
0.8136
universal random
0.9293
0.9887
0.9243
Flower
random
0.8202
0,9562
0.8340
universal random 0.9344
0.9908
0.9321
robustness. Let aig (2:) denote the attribution value or importance assigned to the (i. pixel in an input image and let S& denote the set of l pixel positions with the largest attribution values. Let Nai (i, = {p..g) q) : - w S p S + w, j w S q S j + w) be the neighboring pixel positions within a (2w + 1) x (2w + 1) window around the jith) pixel. By a slight abuse of notation, we
use (1 (a)())) to denote U U(dd)sss(a) (i,jj, that is, the set of all pixel positions that lie in the
union of (2w + 1) X (2w + 1) windows around the top-k pixels.
For a given attributional perturbation Att(),, let Tr Sa( + At(())) denote the top-k pixels in attribution values after applying the attributional perturbation Att(x). Top-k intersection met- ric outputs (2) Tx(x)] /kk We propose Locality-
top-k metries (LENS-top-k) as /k, along the lines of precision and recall used to evaluate ranking. We define Locality-SNNSitive Spearman's (LENS-Spearman) and Locality- sENSitive Kendall's r (LENS-
(S (1)) /k and (ax(2)) N (Tx
metrics rank correlation coefficients for the smoothed ranking orders according to ā.j (1)'s and āz (z + Att(z))'ss respectively. These can be used to compare two different attributions for the same image, the same attribution method two different images, or even two different attributions on two different images, as long as the attribution vectors lie in the same space, e.g., images of the same dimensions where attributions assign importance values to pixels.
We show some theoretically interesting properties of locality-sensitive sensitive measures below. Let aı and a2 be two attribution vectors for two images, and let S and Tj be the set of top h pixels in these images according аj and ag, respectively. We define locality-sensitive top-k distance between two attribution vectors a and a2 as d(se) (2) def prec a2) + recall (w) (a1, a2), where
preck (w)) (a), a2) SSx Now (Tx)] and recall (10 (a) [ a2) der 1T's (Sx))
k similar to precison and recall used in ranking literature, except the key difference being the inclusion of neighborhood items based on locality. We state below a monotonicity property of a( ( a2)
and upper bound it in terms of the symmetric set difference of top-k attributions.
Proposition 1.
For any wi < W2, we have d(uua) (a), a²) < d
(w)) (a1. ag) < |8tATa/ //k, where A
denotes the symmetric set difference, i.e., ALB = (A B) ( A).
Combining ((1) az) across different values of le and we can define a distance
d(ai, a2)
CC
Bur
df (w)
a2),a))
w==
where 0g and 8nr be non-negative weights, monotonically decreasing in and w, respectively, such that Σg 04 oo and En Sw < cc. We show that the distance defined above is upper bounded by metric that is similar to the metrics proposed in Fagin et al. 2003) based on symmetric set difference of top-k ranks to compare two rankings.
Proposition 2. d(ai, az) defined above is upper bounded by u(ai, a2) given by
20
A
(a11, 22) = Σ
Okk
Ba
k
w=u
5

---

Under review as a conference paper at ICLR 2023
and u(a1, a2) defines bounded metric the space of attribution vectors.
Note that top-k intersection. Spearman's 's and Kendall's t do not take the attribution values azz (a:)ss into account but only the rank order of pixels according these values. We also define locality- sensitive w--mmothedd attribution as follows.
Σ
apg(2)
[2w + 1)2
(p.q)ENNatt.)) Isp-gsn
We show that the w-smoothed attribution leads to a contraction the l2 norm commonly used in theoretical analysis of simple gradients attributions.
Proposition 3. For any inputs v; y and any 2 0, |(( ( < ||a(x) a
Thus, any theoretical bounds on the attributional robustness of simple gradients in &z norm proved in previous works continue to hold for locality-sensitivy w-smoothed gradients. For example, Wang al. (2020) show the following bessian-based bound on simple gradients. For an input x and classifier or model defined by f. let (J") and (f) be the simple gradients w.r.t. the inputs at and y. Theorem 3 in Wang et al. (2020) upper bounds the &g distance between the simple gradients of
nearby points || yl12 S as T V (f)ll2 S 5 Amax (f). where Hn( is the Hessian of f w.r.t. the input at x and Amax [H (f)) is its maximum eigenvalue. By Proposition:.. abbove, the
same continues to hold for w-smoothed gradients, i.e.,
( (f) (()) (f) 2 Amax (H).)).
The proofs of all the propositions above included in Appendix A
4.2 LOCALITY-SENSIIII (LENS) ATTRIBUTIONAL. ROBUSTNESS METRICS ARE STRONGER
The top-k intersection is a measure of similarity instead of distance. Therefore. in our experiments for attributional robustness, we use locality-sensitie similarity measures o--LENS-prec @ and
pree (a), a2) and recall (a1, a2), respectively, where a the attribution of the original image and a2 is the attribution of the perturbed image. For rank correlation coefficients such Kendall's and Spearman's 's p. we compute we-LENS-Kendal and LLLEN-Spearman as the same Kendall's 7 and Spearman's 's but computed on the locality-sensitive w-smoothed attribution map ((r) instead of the original attribution map a. We also study how these similarity measures and their resulting attributional robustness measures change we vary and w.
w-LENSS-eealll @ :: to denote
In this section, we measure the attributional robustness of Integrated Gradients (IG) on nat- urally trained models as top-k intersection, wc-LENS-precc@/ and w-L.ENS-recall@kk k: between the IG of the original images and the IG of their perturbations obtained by various attacks. The attacks we consider are the top-t attack and the mass-center attack of Ghorbani et al. (2019) as well random perturbation. All perturbations have foo norm bounded by d 0.3 MNIST, The values of f used to construct the top-t attacks of Ghorbani et al. (2019) are / = 200 on MNIST,
0.1 for Fashion MNIST, and 5 8/255 for GTSRB and Flower datasets.
I 100 on Fashion MNIST and GTSRB, and
t = 1000 on Flower. In the robustness evaluations for fixed k. use & = 100 MNIST, Fashion MNIST and GTSRB, and k = 1000 on Flower.
0.8
Averige top-l
Comparison of top-k intersection, 1-LENS- prec@k and I-LENS--recal@@: k. Figure shows that top-k intersection penalizes IG even for small, local changes. 1-LENS 1-LEENS-recall @ values are always higher in comparison across all datasets in our experiments. Moreover, on both MNIST and Fashion MNIST, 1- LENS-prec@k k is roughly 2x higher (above 90%) compared to top-k intersection (near 40%). In other words, attack may appear stronger under a weaker measure attributional robustness, if it ignores locality.
3
LLEENS-PreCCA. @ k and
0.0
WNST
Figure 3: Attributiona robustness of IG naturally trained models measured as aver- age top-k intersection, 1-LENS-prec, k and 1- LENS-recall @ & between IGooigiial image) and IG(perturbed image) obtained by the top-t attack (Ghorbani et al. 2019) across different datasets.

---

Under review as a conference paper at ICLR 2023
2:00tyycde
(a) MNIST: top-!
(b) F-MNIST: top-!
(c) GTSRB: top-t
(d) Flower: top-1
(c) MNIST: random
(1) F-MIST:: random
GTSRR: random
(h) Flower: random
Figure 4: Attributional robustness of IG on naturally trained models measured as average top-k intersection and 1- 1-LENS-prece between IG(original image) and IG(perturbed image). Perturbations are obtained by the top-f attack (Ghorbani et al., 2019) and random perturbation. The plots show how the above measures change with varying across different datasets.
The effect of varying k. Figure shows large disparity between top-k intersection and 1-LENS- prec@ & even when large. Figure shows that top-k intersection can be very low even when the IG of the original and the IG of the perturbed images locally very similar, as indicated by high 1-LENS LLLENS-preCee @ k. Our observation holds for the perturbations obtained by the top-t attack of (Ghorbani al. 2019) well random perturbation across all datasets in experiments.
w--ENNS-precch for varying w. Figure 5 that w--LENS-pres k increases as we increase w to consider larger neighborhoods around the pixels with top attribution values. This holds for multiple perturbations, namely, top-t attack and mass-center attack by Ghorbani et al. (2019) as well random perturbation. Notice that the top-/ attack of Ghorbani al. (2019) is constructed specifically for the top-/ intersection objective, and perhaps as result, shows larger change when we increase local- local-seniitiiiyy by increasing w in the robustness measure.
1.0
14B66-24TTT 11W0-CY44 Gred
Avarage Intrrsacionn
0.6
0.4
1 miidu
0.2
1956 pn
0.0
13p
anddm
conter mass
5.0
Type Alback
Figure 5: Autributional robustness of IG on naturally trained models measured average top-k intersection and w-LENS-pree prre @ between IGforiginal image) and IG(perturbed image). Perturbations are obtained by the top-t attack and the mass-center attack [Ghorbani] et al. 2019) as well random perturbation. The plots show the effect of varying on Flower dataset.
Figure 6: Attributional robustness of IG naturally trained models measured as average Spearman's 's 1-I I-LENSSSSpeame Kendall r and between IG(original image) and IG(perturbed image). The perturbations obtained by the top-l attack of Ghorbani et al, (201)).
Kendall
and Figure 6 compares Spearman's and Kendall's with 1-LENS Spearman and 1-LENS 1-LENS--endall measures for attributional robustness. We observe that 1-smoothing of attribution maps increases the corresponding Kendall's and Spearman's measures of attributional robustness, and this observation holds across all datasets in our experiments. As result, we believe that I-LENS-Spearne and 1-LENS-KEE Kendall result better or tighter attributional robustnes measures than Spearman's and Kendall's
Comparison of Spearman's and Kendall's r with 1-
I-LENSSSpearman
7

---

Under review as a conference paper at ICLR 2023
Modifying the attack Ghorbani et al. (2019)
top-4 attare
for I-LENS-pree@@ objective A natural ques- tion is whether the original top-k attack Ghor
bani et al. (209)) seem weaker under locality- senstitive robustness measures only because the attack was specifically constructed for corre- sponding top-k intersection objective. Since the construction of the attack in Ghorbani et al. (2019) modifiable for any similarity objective, we use 1-LENS-pree PaeNNS-peeGeee @ k to construct new attributional attack for 1-L.ENS- -LEENS-pree@ l objective based on the k x k neighborhood of pixels. Surprisingly, we notice that it leads to a worse attributional attack, if measure its effectiveness using the top-k intersection; see Figure 7 In other words, attribu- tional attacks against locality-sensitie measures of attributional robustness are non-trivial and may require fundamentally different ideas.
Aveerge top-l
0.4
0.2
0.0
Fastion WNIST
Flower
Figure 7: Average top-k intersection between IG(original image) and IG(perturbed image) on naturally trained models where the perturbation is obtained by incorporating 1-LENS- SENNS-peeeG @k objective in the Ghorbani et al. (2019) attack.
Appendix E contains additional results with similar conclusions when Simple Gradients are used instead of Integrated Gradients (IG) for obtaining the attributions.
5 CONNECTION TO ROBUST ATTRIBUTION TRAINING METHODS
common approach to get robust at- tributions is to keep the attribution method unchanged but train the mod- els differently in way that the re- sulting attributions are more robust to small perturbations of inputs. Chen et al. (2019) proposed the first de- fense against the attributional attack Ghorbani er al.] (2019). Wang et al. (2020) also find that IG-NORM based training of Chen et al. (2019) gives models that exhibit attributional ro- bustness against the top-k attack of Ghorbani et al. (019)) along with ad- versarially trained models. Figure 8 shows a sample image from the Flower dataset, where the Integrated Gradients (IG) of the original image and its perturbation by the top-k at- tack are visually similar for models that are either adversarially trained (trained using Projected Gradient De- scent or PGD-trained, as proposed by (Madry et al. 2018) IG-SUM-NORM trained as in Chen et al. 2019). In other words, these differently trained model guard the sample image against the attributional top-k attack. Recent work by Nourelahi al. (2022) has empirically studied the effec- tiveness of adversarially (PGD) trained models in obtaining better attributions, e.g., Figure 8b shows sharper attributions to features highlighting the ground-truth class.
(a) Sample Image
(b) of original
(c) IG of perturbed
(d) Sample Image (c) IG of original
(f) IG of perturbed
Figure Sample image from Flower whose IG for PGD- trained (top) and IG-SUM-NORM trained (bottom) models seem robust to perturbations by the top-k attack Ghorbani et al. (201))
Figure9showe shows that PGD-trained and IG-SUM-NORM trained models have more robust Integrated Gradients (IG) in comparison to their naturally trained counterparts, and this holds for the previously used measures of attributional robustness (e.g., top-k intersection)
well the new locality- sensitive measures we propose (e.g., 1-LENS prec @ k. LLLENS-recall @ k) across all datasets in our experiments. The top-k attack of Ghorbani et al. 2019) is not threat to IG if we simply measure its effectiveness using 1-LENS LENS-pree@@ & (Figure 9 (a-)) for MNIST, Fashion MNIST and GTSRB), and moreover, use IG on PGD-trained or IG-SUM-NORM trained models (Figure d) for Flower).
8

---

Under review as a conference paper at ICLR 2023
(a) MNIST
(b) Fashion MNIST
(c) GTSRB
(d) Flower
Figure 9: Average top-k intersection, 1-LENS prec k, I-LENS-recall@k measured between IG(original image) and IG(perturbed image) for models that are naturally trained, PGD-trained and IG-SUM-NORM trained. The perturbation used is the top-l attack of Ghorbani et al. (2019).
Shown for (a) MNIST, (b) Fashion MNIST, (c) GTSRB and (d) Flower datasets.
The above observation about robustness of Integrated Gradients (IG) for PGD-trained and IG-SUM- NORM trained models holds even when we use L-LENS-Spearmen and 1-LENS- Kendalll measures to quantify the attributional robustness to the top-k attack of Ghorbani et al. (2019), and holds across all datasets used in experiments; see Figure TO] Moreover, the 1 T-LENS-KKN Kendall and 1-LENS- -pearrann values in Figure I0 are always higher than the corresponding Kendall's s 7 and Spearman's 's p values, which further strengthen the conclusions from previous papers that IG PGD-trained and IG-SUM-NORM trained models give better attributions.
Listtprew
(a) MNIST
(b) Fashion MNIST
(c) GTSRB
(d) Flower
Figure 10: Average Kendall's т, Spearman's I-LENS-Kendall and 1-LENS -SENNS--pp used to measure the attributional robustness of IG natrually trained, PGD-trained and IG-SUM-NORM trained models. The perturbation used the top-k attack of Ghorbani et al. (2019). Shown for (a) MNIST, (b) Fashion MNIST, (c) GTSRB and (d) Flower datasets.
Chalasani et al. (2020) show theoretically that lo adversaaal training (PGD-training) leads to stable Integrated Gradients (IG) under norm. They also show empirically that PGD-training leads to sparse attributions (IG and DeepSHAP) when sparseness in measured indirectly as the change in the Gini index. Our empirical results extend their thcoretical observation about stability of IG for models, as measure local stability in terms of both the attribution values well as their corresponding positions in the image.
CONCLUSION AND FUTURE WORK
We show that the fragility of attributions is an effect of using fragile robustness metrics such as top-k intersection that only look at the rank order of attributions and fail capture the closeness of pixel positions with high attributions. We highlight the need for locality-sensitive metrics for attributional robustness and propose some natural locality-sensitive extensions of the existing metrics. Theoretical understanding of locality-sensitive metrics of attributional robustness, constructing stronger attributional attacks for these metrics, and using them build attributionally robust models are some important future directions.
Reproducibility Statement. The anonymous code for the paper can be found this anonymous
link

---

Under review as a conference paper at ICLR 2023
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, lan J. Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NewrIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 9525 9536, 2018. URL
nwwrips wwt.rrss.ccppppee////0... wc/pappee/2.///aettt. Abst html
2 4b1
Chirag Agarwal, Nari Johnson, Martin Pawelczyk, Satyapriya Krishna, Eshika Saxena, Marinka Zitnik, and Himabindu Lakkaraju. Rethinking stability for attribution-base explanations. arXiv preprint (2ti::22003067777 2022.
David Alvarez-Melis and Tommi S Jaakkola. On the robustness of interpretability methods, arXiv preprint arXiv: /806. 1806.0809, 2018.
N Arun, N Gaw, P Singh, K Chang, M Aggarwal, B Chen, et al. Assessing the (un) trustworthiness of saliency maps for localizing abnormalities in medical imaging, arxiv. Preprint]. 2020.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation, 2015.
Müller,
Umang Bhatt, Adrian Weller, and José MF Moura. Evaluating and aggregating feature-based model explanations. arXiv preprint arXiv:2005.0000 00000005.0..,,.. 2020.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šmniić Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. Lecture Notes in Computer Science, pp. 387 2013. ISSN 1611-3349, doi: 10. URL httptttp//d.xto doi wtt. 9101///7000000001---------- 329---644-444933-000
Angie Boggust, Harini Suresh, Hendrik Strobelt, John V. Guttag, and Arvind Satyanarayan. Beyond faithfulness: A framework characterize and compare saliency methods. CoRR, abs/2206.02958. 2022. doi: 10:48550//mXii.220002255 URL
2206. 02958
Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832, 2019.
Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. Concise explanations of neural networks using adversarial training. In Proceedings of the 37th Inter- national Conference Machine Learning. ICML 2020, 13-18 July 2020, Virtual Event, vol- ume 119 of Proceedings of Machine Learning Research, pp. 1383-1391. PMLR, 2020. URL
http::/proceet ngs.m w.w..mr.prees.. wrw.srsss/eee vl 9/ www.aaaaaaaaaaaaammammaa laaan 20a html
Aditya Chattopadhyay, Anirban Sarkar, Prantik Howlader, and Vineeth N. Balasubramanian. Grad- cam++: Generalized gradient-based visual explanations for deep convolutional networks. pp. 839 847, 2018. doi: 10.1109/AACV.2018.0000
URL httpp:://ol.orgg// 11009 1109/
MACV. 2018. 00097
Jiefeng Chen, Xi Wu, Vaibhav Rastogi, Yingyu Liang, and Somesh Jha. Robust attribution reg- ularization, 2019. URL https:://roceeednngs http://roooeddin.......e..ee http://eeeeinneeeerrssseec//aapa wwtpppppppp/p////ccc.cop 201200999hhhh/
[222faa04bbttaaa1222666773333333 Abst .html
Ronald Fagin, Ravi Kumar, and D. Sivakumar. Comparing top k lists. SIAM Jourmal on Discrete
Mathematics, 171) 2003. doi: 10. 1711137099954400024125555 URL https://doil
org/ 11101990111110000091100011111111
Fenglei Fan, Jinjun Xiong, Mengzhou Li, and Ge Wang. On Interpretability of Artificial Neural Networks: A Survey. arXiv:2001.0255 fes, stat], January 2021.
10

---

Under review as a conference paper at ICLR 2023
Thomas Fel, David Vigouroux, Rémi Cadène, and Thomas Serre. How good is your explanation? algorithmic stability measures to assess the quality of explanations for deep neural networks. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 720-730, 2022.
Krishna Gade, Sahin Cem Geyik, Krishnaram Kenthapadi, Varun Mithal, and Ankur Taly. Explain- able AT in industry: practical challenges and lessons learned: implications tutorial. Mireille Hildebrandt, Carlos Castillo, L. Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna (eds.), FAT* '20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, pp. 699. ACM, 2020, doi: 10.1145/3351005.333566. URL https:://o...ogg//// 111....11111111111111 3375664
Amirata Ghorbani, Abubakaa Abid, and James Y. Zou, Interpretation of neural networks fragile, 2019. UKL/stppss//001..rr//00111111111 11tp::000.60000aaa.331.111111111 v33101. 33013688
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial 6572
examples. 2015.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp. 2668-2677. PMLR, 2018.
Freddy Lecue, Riccardo Guidotti, Pasquale Minervini, and Fosca Giannotti. 2021 explainable ai tutorial. https://aaatuutrrll0000... htt.:/tttrrra112222..iiih....bba github. 2021. visited 14-09-2021.
Zachary C. Lipton. The mythos of model interrretaabrriiiity Commun. ACM, 61(10)336 2018. doi: 10.1 10.1145/3233211. URL/tttpa.//boll.rr//// /ttp:///6ooooaao//11114433333333
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model pre-
dictions, 2017. URL httpps::/proceeeinns wwwrips....... cetpr//ppcccc//ppp/////////aaaa
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks, 2018. URL htpp::/t//////eeeeeeee
nett/Eorun2:dd:r.211112..
Christoph Molnar. Interpretable Machine Learning. 2019. http:://chrrsstpphn.gghh/ Lo/Linterprettablemm---oooo
Mehdi Nourelahi, Lars Kotthoff, Peijie Chen, and Anh Nguyen. How explainable are adversarially- robust enns?, 2022.
Cheolhwan Oh and Jongpil Jeong. VODCA: verification of diagnosis using cam based approach for explainable process monitoring. Sensors, 20(23):6858, 2020. doi: 10.3390/s2226685.. URL
htttp:://oi..rr www.ooooggg000 33900s20236855
Felipe Oviedo, Juan Lavista Ferres, Tonio Buonassisi, and Keith T. Butler. Interpretable and explainable machine learning for materials science and chemistry. Accounts of Materials Research, 3(6):597-607) jun 2022. doi: 10.100///ccoumsmm...000444 URL httts:://poi.ooo 1021 F222138acccuttsoo. smr. c00244
Mesut Ozdag. Adversarial attacks and defenses against deep neural networks: A survey. Procedia Computer Science, 140:152- 1615,,111,15 2018. ISSN 1877-0509.
Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explain- ing the predictions of any classifier, 2016. URL /tttps://dol.cogg111115553
2939778
Wojeiech Samek. Grégoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Claus-Roberr Müller (eds.). Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, volume 11700 of Lecture Notes in Computer Science. Springer. 2019. ISBN 978-3-033- 28953-3.99 doi: 10.1007/
000-3-000-29954--0 U //doi 1000//0.0000//71000 http::///0....111111110100100000

---

Under review as a conference paper at ICLR 2023
Anindya Sarkar, Anirban Sarkar, and Vineeth N. Balasubramanian. Enhanced regularizes for attribu- tional robustness, 2021. URL httpp://osssss.... aaai. orq/index. o phtp/AAA/articl....
wiew/16355
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion. pp. 618 626, 2017. doi: 10.1 10.1109/CC.22017...4 ICCV.2017.74
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just black box: Learning important features through propagating activation differences, 2016. URLhttp:
//arxiv.orc/ wttp:////aa./1...111221
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences, 2017. URL http:::/pooeeeeeggs... mllr. htt.:///i..........ss////0/00 shrikumar 1am
Samuel Henrique Silva and Peyman Najafirad. Opportunities and challenges in deep learning
adversarial robustness: A survey. CoRR, abs/2007.00753, 2020. URL https:// /ttp:///axxvvvvoo org/
abs/2007, 00753
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps, 2014. URL htllhttpp//aat httptt////a....og//////
abs/1312.603 6034
Mayank Singh, Nupur Kumari, Puneet Mangla, Abhishek Sinha, Vineeth N. Balasubramanian, and Balaji Krishnamurthy. Attributional robustness training using input- gradditt spatial alignment, 2020. URL/tttp:://dd org/ 10. 0..00.1..000
31
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods, 2020. URL
arq/10.1145/ 41/55553337777 01 3375830
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda B. Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise, 2017. URL http::///aaii..oo/aa////////
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for simplicity: The all convolutional net, 2015.
6806
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Doina Precup and Yee Whye Teh (eds.). Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3319-33123333 3328. PMLR, 2017. URL htpp:ttttpp/pprreednn..
mlr. wwtpppessssssuuuepeppppep http://wwwoouuaaaaaaaaaaaaaaaaaa 7a html
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks, 2014. URL htt.htt:://axii..oo/
labs/311 6199
Siyi Tang, Amirata Ghorbani, Rikiya Yamashita, Sameer Rehman, Jared A. Dunnmon, James Zou, and Daniel L. Rubin. Data valuation for medical imaging using shapley value and application to large-scale chest x-ray dataset. Scientific Reports(Nature Publisher Group). Rthtts //doi.org//////
Richard Tomsett, Dan Harborne, Supriyo Chakraborty, Prudhvi Gurram, and Alun D. Preece. Sanity checks for saliency metrics. In The Phirty-Fourth AAAI Conference Artificial Intelligence, AAAI 2020, The Thirty-Secone Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, FAAI 2020, New York, NY, USA, February 7-12, 2020, pp. 6021-6029. AAAI Press, 2020. URL
https: / //t:s//jjs././../ aaai org/ iinex.idxxi//ii. http:/BA///rttile//veee. http:/iwwwwwww/////
Fan Wang and Adams Wai-Kin Kong. Exploiting the relationship between kendall's rank correlation and cosine similarity for attribution protection. CoRR, abx/2205.07279 2022. doi: 10.48550/arXiv. 2205.07279. URI. [RI.https:: //ooi org 1155111...5555////// .2005.2222...... 07279.
12

---

Under review as a conference paper at ICLR 2023
Zifan Wang. Haofan Wang, Shakul Ramkumar, Piotr Mardziel, Matt Fredrikson, and Anupam Datta. Smoothed geometry for robust attribution, 2020. URL UtL/ttpp:://prrceedigg. neurips
ca/ http:////...eeeee.c..c.ccccaaaar
ht.ml
Melvyn Yap. Rebecca L. Johnston, Helena Foley, Samual MacDonald, Olga Kondrashova, Khoa A. Tran, Katia Nones, Lambros T. Koufariotis, Cameron Bean, John Pearson, Maciej Trzaskowski, and Nicola Waddell. Verifying explainability of deep learning tissue classifier trained on ma- seq data. Scientific Reports(Nature Publisher Group). URL httttsts::///de /ttt:::://dd..org/111033333
Chih-Kuaa Yeh, Cheng- Yu Hsieh, Arun Suggala, David I Inouye, and Pradeep K. Ravikumar. On the (in) fidelity and sensitivity of explanations. Advances in Neural Information Processing Systems, 32, 2019.
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Proceedings of The European Conference on Computer Vision (ECCV), 2014.
Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, and Robert Fergus. Deconvolutional networks. In The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13-18 June 2010, pp. 2535. IEEE Computer Society, 2010. doi: 10. //oii org 11099CVPR.20105559997.. URL |/L_ 2010. 5539957)
Qua Zhang and Song-chun Zhu. Visual interpretability for deep learning: a survey. Frontiers of Information Technology Electronic Engineering, 19:27-39, January 2018. ISSN 2095-9230.
Yu Zhang, Peter Tiño, Aleš Leonardis, and Ke Tang. A Survey on Neural Network Interpretability. arXiv:2012.1426: [cs], December 2020.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. features for discriminative localization. CVPR'16.
Learning deep
13

---

Under review as a conference paper at ICLR 2023
SUPPLEMENTARY MATERIAL
The Appendix contains proofs, additional experiments to show that the trends hold across different datasets and other ablation studies which could not be included in main paper due to space constraints.
A PROOFS FROM SECTION
We restate and prove Proposition below.
Proposition 4. For any w1 S W2 we have af (w2) (a1. _ ((L)) denotes the symmetric set difference, iee. AAB (A B) U (B A).
(a1, a2) I |S,ATa| /k, where A
Proof. The inequalities follows immediately using S C Nw (5) Nwz (S), for any S, and hence, |S \ No (T)) < |S 7), for any S, T' and w.
We restate and prove Proposition 2 below.
Proposition 5. d(a1,ay) defined above is upper bounded by u(a1, a2) given by
AT
u(a1, a2)
B
k
k=1
===
and u(a1, az) defines a bounded metric on the space of attribution vectors.
Proof. Proof follows from Proposition I and using the fact that symmetric set difference satisfies triangle inequality.
We restate and prove Proposition 3 below.
Proposition For any inputs v, y and any w 2 0. (a(w)()) (w))l) S ||a(x) a(v)ll2-
Proof.
ā (((0( (x) - -200
Σ
a (x)
al.m) (v) 2
15455n
Σ
Σ
(Apg(x) - (1))
(2w + 1)4
1
(p.g)e (15).
15p.9sn
(2w + 1)2
Σ
(Apg(a)) apg (1)) by Cauchy- Senoha-S inequality
(2w + 1)4
(.. ISp455
Σ
(apg(z) - app (2))2
(2w + 1)2
Istginn (i.)).
ISp.9Sn
(2w + +1
(apg(x) Opn (v))
(2w +1²
because each (p. q) appears at most (2w + 1)² possibles (i.
la(x)) a(2)11?-
14

---

Fashion Under review as a conference paper at ICLR 2023
B DETAILS OF EXPERIMENTAL SETUP
The detailed description of the setup used in our experiments.
Datasets: We use the standard benchmark train-test split of all the datasets used in this work, that is publicly available. MNIST dataset consists of 70, 000 images of 28 x 28 size, divided into 10 classes: 60, 000 used for training and 10, 000 for testing. Fashion MNIST dataset consists of 70, 000 images of 28 x 28 size, divided into 10 classes: 60, 000 used for training and 10, 000 for testing. GTSRB dataset consists 51, 739 images of 32 32 size, divided into 43 classes: 34, 699 used for training. 4, 410 for validation and 1 630 for testing. Flower dataset consist of 1,360 images of 128 x 128 size, divided into 17 classes: 1, 224 used for training and 136 for testing. GTSRB and Flower datasets were preprocessed exactly as given in Chen et al. [A019)[Appenn)) CJ for consistency of results.
Architectures: For MNIST, Fashion MNIST, GTSRB and Flower datasets we use the exact architec- tures as used by Chen al. (2019).
Attribttion robustness metrics: We use the same comparison metrics as used by Ghorbani et al. (2019) and CCen et al. (2019) like top-k pixels intersection, Spearman's 's and Kendall's rank correlation to compare attribution maps the original and perturbed images. The value for top-k attack along with settings like step size, number of steps and number of times attack is to be applied as used by Chen et al. (2019) for the attack construction MNIIT((00,0001,100)) 1ower(1000,,.0,1000
1000000)0000011100.03). GTSRB(100,1.0,50333
Sample sizes for attribution robustness evaluations: IG based experiments For MNIST, Fashion MNIST and Flower with fixed top-k attack similar to Chen et al. (2019) the complete test set were used to obtain the results. For GTSRB a random sample of size 1000 was used for all the experiments. Simple gradient based experiments For MNIST and Fashion MNIST random sample of 2500/ 1000 from the test set. For GTSRB, random sample of size 1000 and the complete test set for the Flower dataset.
Adversarial Iraining: We use the standard setup as used by (Chen et al. 2019). We perform PGD based adversarial training with the provided budget using the following settings (number of steps. step size) for PGD MNIST (40,0.01), Fashion MNIST(20,0.01), GTSRB(7,8/255)) Flower(7,2/255))
Training for Attributional Robustness: We use the IG-SUM-NORM objective function for all the datasets study based on (Chen et al. 2019) based training. With the exact setting as given in paper with code |
Hardware Configuration: We used server with 4 Nvidia GeForce GTX 1080i GPU and a server with Nvidia Tesla V100 GPU to run the experiments the paper.
15

---

Under review as a conference paper at ICLR 2023
C
THE FRAGILITY OF TOP- INTERSECTION
(a) Original IG
(b) top 100 pixels
(c) Perturbed IG
(d) top 100 pixels
Original IG
(f) top 100 pixels
(g) Perturbed IG
(h) top 100 pixels
Figure 11: Sample top-k highlighting using Flower.
16

---

Under review as a conference paper at ICLR 2023
D EXPERIMENTS WITH INTEGRATED GRADIENTS
Below we present additional experimental results for Integrated Gradients (IG).
Tip SAAT
(a) MNIST
(b) Fashion MNIST
(c) GTSRB
(d) Flower
Figure 12: Attributional robustness of IG on naturally, PGD and IG-SUM-NORM trained models measured top-k intersection, 1-LENS p k and 1 1-LENS-recall @ between the IG of the original images and the IG of their perturbations obtained by the random attack (Ghorbani et al. 2019) across different datasets.
(a) MNIST
(h) Fashion MNIST
(c) GTSRB
(d) Flower
Figure 13: Attributional robustness of IG on naturally, PGD and IG-SUM-NORM trained models measured top-k intersection, 1-L-NSS-ree -rens-ppe@@ee @ k and 1-LENS-recall k between the IG of the original images and the IG of their perturbations obtained by the mass-center attack (Ghorbani et 2019) across different datasets.
Took Mbci
IG top-k
(b) IG random
(c) IG: center mass
Figure 14; Attributional robustness of IG on naturally, PGD and IG-SUM-NORM trained models measured as top-k intersection and w-LENS prec k between the IG the original images and the IG of their perturbations. Perturbations are obtained by the top-/ attack and the mass-center attack (Ghorbani al. 2019) as well as random perturbation. The plots show the effect of varying w on Flower dataset.
17

---

Under review as a conference paper at ICLR 2023
E EXPERIMENTS WITH SIMPLE GRADIENTS
We observe that our conclusions about Integrated Gradients (IG) continue to hold qualitatively, even if we replace IG with Simple Gradients attribution method.
10p- EHARTPONNNI
Spplt
(a) MNIST
Fashion MNIST
(c) GTSRB
(d) Flower
Figure 15: Attributional robustness of Simple Gradients on naturally, PGD and IG- IG-SUM-NORM trained models measured as top-k: intersection, 1-LENS-prec k and 1-LENSS-realll @ between the Simple Gradient of the original images and the Simple Gradient of their perturbations obtained by the top-k attack (Ghorbani al. 2019) across different datasets.
(a) MNIST
(b) Fashion MNIST
(c) GTSRB
(d) Flower
Attributional robustness of Simple Gradients on naturally, PGD and IG-SUM-NORM trained models measured as top-k intersection, 1-LENS-pret -SNNN- & and I-LENS-recall @ k between the Simple Gradient of the original images the Simple Gradient of their perturbations obtained by the random attack (Ghorbani et al. 2019) across different datasets.
Figure 16:
50p4 CiPeSCoei
(a) MNIST
Fashion MNIST
Flower
Figure 17: Attributional robustness of Simple Gradients on naturally, PGD and IG-SUM-NORM trained models measured top-k intersection, 1-LENS- preespre@@k k and I-LENS-recall @ between the Simple Gradient of the original images and the Simple Gradient of their perturbations obtained by the mass-center attack (Ghorbani et al. 2019) across different datasets.
F ADDITIONAL RESULTS FOR PGD-TRAINED AND IG-SUM-NORM TRAINED MODELS
Figure 21 and Figure 19 shows the impact of k in top-k for adversarially(PGD) trained and trained network, respectively. But an important point to be no- ticed is that even with small number of features LENS is able to cross 70-80% which supports the observation of sparsity and stability of attributions achieved by adversarially (PGD) trained models by Chalasani al. (2020). Similarly, the experiments with different w value for w- w-LENS-top-k in clearly incidates that due to the stability properties at lower window sizes LENS is able to cross the 80% intersection quickly. Supporting that metric nicely captures local stability very well. While we observed only the top-k version of LENS. Figure [0] further extends the observation to LENS LENS-Spearman and I. EENSKeende who show that with LENS with smoothing of 3 x the
18

---

Under review as a conference paper at ICLR 2023
Tyer Tu
Typerel
Tye-eeTerrinn
(a) SG: topk
SG random
SG center of
Figure 18: Attributional robustness of Simple Gradients on naturally. PGD and IG-SUM-NORM trained models measured as top-k intersection and w-LENS-pres PaeSS--pe@@.. k between the IG of the original images and the IG of their perturbations. Perturbations are obtained by the top-t attack and the mass-center attack (Ghorbani al. 2019) as well as random perturbation. The plots show the effect of varying on Flower dataset.
maps from adversarial and attributional robust models have a higher top-k intersection above 70% in comparison natural trained model.
(a) MNIST IG topk
(b) F-MNIST IG topk
(c) GTSRB IG top-k
(d) Flower IG topk
(e) MNIST IG random
(f) F-MNIST random
(g) GTSRB top-k
(h) Flower random
Figure 19: Comparing the top-k intersection between top-k and I-LENS ALENS-precCe @k with different & for top-k evaluation while the attack a fixed k. For IG-SUM-NORM trained network.
(a) IG topK
(b) IG random
(c) IG center of mass
Figure 20: Attributional robustness of IG on naturally, PGD and IG-SUM-NORM trained models measured as top-k intersection and w-LENS- LLLEN---eeeeee @ & between the IG of the original images and the IG of their perturbations. Perturbations are obtained by the top-f attack and the mass-center attack (Ghorbani al. 2019) as well random perturbation. The plots show the effect of varying w on Flower dataset.
19

---

Under review as a conference paper at ICLR 2023
S-BNSCWOH
(a) MNIST IG topk
F-MNIST IG: topk
GTSRB top-k
(d) Flower IG topk
194
10.4
MNIST IG: random (f) F-MNIST IG: random
(g) GTSRB IG random
(h) Flower IG random
Figure 21: Comparing top-k and I-LENS-prec k with different for top-k evaluation while the attack is fixed For adversarially(PGD) trained network.
20


Reviews:
[{"review_id": "z6Wbh_wWOE", "reviewer": "Reviewer_mFur", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper makes the points that current metrics to measure robustness of feature attributions not very effective and could raise false alarms. The paper then proposes a local neighborhood based smoothing procedure which essentially does not just consider the top-k feature attributions, but also the neighborhood around them. The paper then show how the improved metrics show higher overlap than their original versions. Further experiments are shown with attack-aware setups.", "summary_of_the_review": "The motivation behind and the side-effects of the proposed changes in the explanation quality metrics are not clear. The evaluation focuses on one explanation method only so it is difficult to disentangle the effect of the explainer from the effect of the metric. Most of the evaluation results are of the kind \"this makes sense\". For these reasons, the paper does not seem ready for publication yet.", "strength_and_weaknesses": "Strengths:\n\n 1. Assessing robustness of the metrics is an important topic.\n 2. The insight that the top-k style evaluation metrics are insufficient is an important one.\n\nWeaknesses:\n\nOverall, the motivation behind the particular solution proposed in this paper is not very clear to me. The evaluation could also be improved further. Currently, the evidence seems to be that the results \"make sense\". Please see detailed comments and questions below.\n\n1. First a clarification question. In Figure 2, how are the “random” and “universal random” different? Are they drawn from different distributions? Is universal random also scaled down to a l_inf norm of 0.3? I am a bit confused as to why random and universal random generate such a different effect if they are drawn from the same distribution. \n\n2. The proposed solution of taking the local neighborhood of explanations into account does make some sense. However, is that the only problem with the robustness metrics? Are there any key issues that are still unresolved? My main concern here is that the insight made here seems quite isolated. The main motivation is presented in Figures 1 and 2 which has to do with considering the local neighborhood of the pixels. But it is not clear how much of the problem lies with the metrics and how much with the data domain (images, where adjacent pixels contain redundant information).\n\n3. Building on the previous question, it is not clear how the insights would translate to other domains like tabular datasets or NLP.\n\n4. Unfortunately, the paper does not compare different explainability methods so it is hard to disentangle how much of the problem is with the explainability method and how much with the metric. With a single explainer, it is not clear what a good reference value of the metric is.\n\n5. Unlike the underlying task (classification), explainers do not have any ground truth. That makes the evaluation of robustness and evaluation of metrics quite difficult. Still, it is important to disentangle the weaknesses of metrics from weaknesses of explainers. I would suggest building toy tasks (e.g., https://arxiv.org/abs/2104.14403) where it would be easier to study these effects.\n\n6. The design of the proposed metrics isn't quite discussed in sufficient details. How would considering the local neighborhood impact other related metrics like preservation game, pointing game (https://arxiv.org/abs/1910.08485)?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SHbX05-_1_f", "reviewer": "Reviewer_5YT2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a metric to measure the robustness of feature attribution methods (AMs) called LENS that better assesses the goodness of AMs. They are motivated by the insensitivity towards locality changes of existing robustness metrics for AMs.  \n\nWhile I found the metric is well-grounded and the experiments are sound, I believe this work has minimal impact in XAI then I lean towards Rejection.", "summary_of_the_review": "My biggest concern lies on the practical impacts of this work and I think it does not fit ICLR.", "strength_and_weaknesses": "Strengths:\n- Authors conduct extensive literature survey and give sufficient context information of research.\n- The experiments show the superiority compared to existing robustness metric. The consideration of locality sensitivity is reasonable.\n\nWeaknesses:\n- I am not convinced by the values of this work. AMs have been proven to have very minimal utility in previous works (e.g. [1,2]). The proxy metrics (e.g. robustness) are often ill-designed as they are not tied with the practical applications. [1] also pointed out that there is no correlation between AMs' effectiveness vs. their proxy scores in image classification.\n\n[1] The effectiveness of feature attribution methods and its correlation with automatic evaluation scores, NeurIPS2021.\n\n[2] HIVE: Evaluating the Human Interpretability of Visual Explanations, ECCV22.\n\nQuestions: \n- Could authors justify why do we need the robustness for AMs. Attacks on defenses on AMs, in my opinion, are not realistic and trivial research problems.\n- How this metric could help to assess the actual downstream utility of AMs?\n\nI am happy to adjust the score once the authors satisfactorily resolve my concerns.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "JHPv5d1uDqh", "reviewer": "Reviewer_zTfh", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work shows that previous evaluation metrics for attributional robustness such as top-k intersections, spearman's rank-order correlation or Kendall's rank-order correlation are fragile where a simple random perturbation attack can seem to be as significant as more principled attributional attacks. Therefore, they propose locality-sensitive improvements of these metrics and show theoretically that the proposed metrics can provide tighter bounds on attributional robustness. ", "summary_of_the_review": "See above.", "strength_and_weaknesses": "Strength:\n\nThe authors provide theoretically proofs to show that the proposed evaluation metrics provide tighter bounds on attributional robustness. \n\nWeakness:\n\nThe experiment showing that random vectors are attributional attacks under existing evaluation metrics seems to be a bit confusing. If I understand correctly, it is not random vectors are strong attributional attacks but universal adversarial perturbations independent to each image. It would be good if the authors can clarify if these universal adversarial perturbations are optimized to be generated or randomly sampled. \n\nIn addition, the results in Figure 7 are less reasonable. According to \"On Adaptive Attacks to Adversarial Example Defense\" (NeurIPS-2020), the defense-aware attacks should be stronger than attacks without the knowledge of the defense. However, the results in Figure 7 display a reverse pattern of the conclusion in NeurIPS-2020, which makes me concerned that the designed defense-aware attacks are well-constructed.\n\nLastly, the datasets studied in this work are mainly small datasets, which makes it unclear how the conclusions/results be generalized to larger datasets, e.g., ImageNet, CIFAR, etc.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8Qj9BABP54", "reviewer": "Reviewer_yyBB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper shows the existing metrics for measuring attribution robustness like top-k intersection, Spearman's rank-order correlation, and Kendall's rank-order correlation are fragile. Specifically, it shows that under these metrics, a simple random perturbation attack can seem to be as significant as more principled attribution attacks. It then proposes Locality-sEnSitive (LENS) improvements of these metrics, namely, LENS-top-k, LENS-Spearman, and LENS-Kendall, that incorporate the locality of attributions along with their rank order. The proposed metrics provide tighter bounds on attribution robustness and do not disproportionately penalize attribution methods for reasonable local changes. The empirical results support the need for the new metrics proposed. ", "summary_of_the_review": "This paper proposes new metrics for evaluating attributional robustness. However, the motivation for the new metrics is weak and the experimental evaluation of the new metrics is limited. Thus, I recommend rejection. ", "strength_and_weaknesses": "I think this paper has the following strengths: \n\n1. The proposed metrics for measuring attribution robustness are novel. The idea of incorporating the locality of attributions along with their rank order when measuring attribution robustness is interesting. \n\n2. It shows that the proposed LENS variants provide tighter bounds on the attribution robustness of already known improvements in attribution methods and model training designed for better attributions. This result seems significant. \n\n3. The writing is clear in general and the related works are properly discussed. \n\nHowever, I think this paper has the following weaknesses: \n\n1. The motivations for the new metrics are not strong and need more explanations. One of the motivations is that under the existing metrics to quantify the attribution robustness of IG on a naturally trained model, even a single input-agnostic or universal random perturbation for all inputs can sometimes be a more effective attributional attack than using an independent random perturbation for each input. I think this motivation is weak since it is possible that the universal random perturbation is indeed a more effective attributional attack than the independent random perturbation. Also, this observation is not consistent across datasets (it only holds on the MNIST and Fashion MNIST datasets, but doesn't hold on the GTSRB and Flower datasets). I think for random perturbation attacks, it needs to repeat the experiments several times and report the mean and standard deviation of the results. It also needs to provide more justifications for the motivation. \n\n2. In the proposed metrics, there is a hyper-parameter w. It doesn't discuss how to choose the hyper-parameter w. It seems the proposed metrics will be quite different for different values of w. \n\n3. Based on the results in Figure 3, it claims that an attack may appear stronger under a weaker measure of attribution robustness if it ignores locality. I think this claim is not well supported by the results. It is possible that the top-t attack used is only effective for the top-k metric, but not effective for the proposed 1-LENS-recall@k and 1-LENS-prec@k metrics. For different metrics, it needs to design different adaptive attacks. Thus, the results in Figure 3 may be misleading. \n\n4. Although it modifies the attack of Ghorbani et al. (2019) to construct a new attributional attack for the 1-LENS-prec@k objective, the experimental results show that the constructed attack is not effective. Since it doesn't provide the details of the constructed attack, it is hard for me to evaluate if the constructed attack is strong enough or not. It is possible that it doesn't design or implement the attack properly. Besides, I think constructing stronger attributional attacks for the proposed metrics should be included in this work, instead of future works. Without the adaptive attack evaluation for the proposed metrics, it is hard to know if the proposed metrics are reasonable or not. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "4P4X3kjv01", "reviewer": "Reviewer_UqJb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Previous work has observed that imperceptible perturbations can manipulate the attribution maps. This work argues that previous metrics are in parts too sensitive to local changes. They propose smoothing the attribution map to weigh local changes less. Various metrics (top-k, Kendall's, and Spearman's rank-order) are then adapted to this setting. The experiments are all conducted on four different datasets using the Integrated Gradients method.", "summary_of_the_review": "Overall, the paper is currently not ready for ICLR. The main reasons are the sole focus on Integrated Gradients and the lack of clarity, quality, and novelty.", "strength_and_weaknesses": "**Strengths:**\n\n- This work tackles an important problem of how the robustness of attribution maps can be best measured.\n- The connection between locality and robustness is an interesting observation. \n\n**Weakness**:\n\n- Recent work by Lundstrom et al. 2022 (ICML: https://proceedings.mlr.press/v162/lundstrom22a/lundstrom22a.pdf) found several theoretical limitations of IG. This questions the paper's assessment that *\"Integrated Gradients (IG) is a standard attribution method based on solid theoretical foundations (Sundararajan et al., 2017)\"*. \n- More than only Integrated Gradients should be evaluated. Why is only this method is selected? Many other methods exist, also cited in the Related Work section. Different software packages provide these methods, so it should be straightforward to show more results. This might also create new insights as one method might score significantly different for the local-smoothed evaluation.\n- The work is overall relatively incremental. Basically, it smooths existing metrics. \n- The notation and clarity can be improved (see next section).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject