Smoothed Differentiation Efficiently Mitigates Shattered Gradients in Explanations
Authors: Adrian Hill
Neal McKee
Johannes Maeß
Stefan Bluecher
Klaus Robert Muller
Venue: NeurIPS 2025 poster ()
Keywords: explainable ai
interpretability
feature attribution
saliency
automatic differentiation
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We propose SmoothDiff, a novel feature attribution method leveraging automatic differentiation, directly targeting nonlinearities responsible for the shattered gradient problem.

Abstract: Explaining complex machine learning models is a fundamental challenge when developing safe and trustworthy deep learning applications. To date, a broad selection of explainable AI (XAI) algorithms exist. One popular choice is SmoothGrad, which has been conceived to alleviate the well-known shattered gradient problem by smoothing gradients through convolution. SmoothGrad proposes to solve this high-dimensional convolution integral by sampling -- typically approximating the convolution with limited precision. Higher numbers of samples would amount to higher precision in approximating the convolution but also to higher computing demand, therefore in practice only few samples are used in SmoothGrad. In this work we propose a well founded novel method _SmoothDiff_ to resolve this tradeoff yielding a _speedup of over two orders of magnitude_. Specifically, _SmoothDiff_ leverages automatic differentiation to decompose the expected values of Jacobians across a network architecture, directly targeting only the non-linearities responsible for shattered gradients and making it easy to implement. We demonstrate SmoothDiff's excellent speed and performance in a number of experiments and benchmarks. Thus, SmoothDiff greatly enhances the usability (quality and speed) of SmoothGrad -- a popular workhorse of XAI.

Smoothed Differentiation Efficiently Mitigates Shattered Gradients in Explanations
Adrian Hill? Neal McKee Johannes MaeB
Stefan Blücher! Klaus-Robert Müller! 256
BIFOLD Berlin Institute for the Foundations of Learning and Data, Berlin, Germany *Machine Learning Group, TU Berlin, Berlin, Germany
*ernstein Center for Computational Neuroscience, Berlin, Germany "TU Berlin, Berlin, Germany
'Department of Artificial Intelligence, Korea University, Scoul, Korca Max Planck Institut für Informatik, Saarbrücken, Germany
hi hill@tu-berlin.d ballbccn--brrin..cm masss@tu-berlin.. b sluecher@tu-berlin..
klaus robertobr.... heaeeert.ueeleegtu--brrrrl.
Abstract
Explaining complex machine learning models is fundamental challenge when developing safe and trustworthy deep learning applications. To date, broad selec- tion of explainable AI (XAI) algorithms exist. One popular choice is SmoothGrad, which has been conceived to alleviate the well-known shattered gradient problem by smoothing gradients through convolution. SmoothGrad proposes solve this high-dimension convolution integral by sampling typically approximating the convolution with limited precision. Higher numbers of samples would amount higher precision in approximating the convolution but also to higher computing demand, therefore in practice only few samples are used in SmoothGrad. In this work we propose well founded novel method SmoothDiff resolve this tradcoff yielding a speedup of over two orders of magnitude. Specifically, SmoothDiff leverages automatic differentiation to decompose the expected values of Jacobians across a network architecture, directly targeting only the non-linearities respon- sible for shattered gradients and making easy implement. We demonstrate SmoothDiff's excellent speed and performance in a number of experiments and benchmarks. Thus, SmoothDiff greatly enhances the usability (quality and speed) SmoothGrad a popular workhorse of XAI.
Introduction
XAT has in the past years become a popular resource to further user's understanding of various machine learning models such as neural networks, kernel methods, LSTMs and transformers, see e.g. [1]. Among others, XAI allows to debug models, scrutinize data sets and find Clever-Hare effects in supervised and unsupervised models [2, 3J. Post-hoc explanation methods, see [1], can e.g. be based on gradients [4, 5, 6]. first- and higher-order Taylor expansions around meaningful root points [7, 8, 10], and Shapley values [11, 12].
In this work will place our focus the inner workings of a classifier by inspecting the gradient of its prediction with respect to its input features. Usually, the focus on the gradient ff.()) of the maximally activated output logit, however any other logit can be inspected as well, resulting in an explanation for that respective class (see Appendix A for notation). We limit ourselves to the fixed- model regime, in which the model cannot be retrained for the purpose of interpretability.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

(a)
0.0025
Gradient
0.0000
SmoothDiff
0.0025
SmoothGrad
-0.0050
B-Smoothing
Convolution
0.0075
-1.0
0.5
0.0 Input
0.5
1.0
(b)
"red fax
'sviing
"barbershop'
race car
"red wine"
"balloon"
racket"
"dung beetle"
Input
Bater Shop
Gracient
Smooth
(n-10)
SmoothGrad Smoothöral (converged)
(n-10)
Figure 1: (a) Visualization of the shattered gradient problem on a 1D deep ReLU network. Gaussian convolution removes high-frequency components of the shattered gradient and is well approximated by both SmoothDiff and SmoothGrad. (b) On high-dimensional ImageNet models, SmoothGrad suffers from low sample efficiency, whereas at just 10 samples, SmoothDiff's level noise reduction matches fully converged SmoothGrad. See Appendix for implementation details.
1.1 Preliminaries
Shattered gradient problem. With growing depth of neural networks, the vector field of gradients over the input space I gets increasingly noisy, as shown in [13]. For small changes in inputs X, resulting gradient vectors fa( vary strongly, resembling white noise. This phenomenon has been termed the shattered gradient problem (SGP), namely the SGP renders them uninformative due to bad signal to noise ratio, as shown in Figure 1: For 1-dimensional toy example we observe that the SGP results in large step-wise fluctuations of the gradient; also for gradient explanations in vision models (here VGG-19 [14]), the SGP leads to highly noisy heatmaps.
Gaussian convolutions. Since the SGP closely resembles white noise in deep networks, it motivates the use of a Gaussian filter in the input domain to suppress high-frequency signals: if images are perceptually robust to small amounts of noise, explanations should also be stable with respect to minor input variations. This implies that explanations should not vary rapidly and have little high-frequency content in the input domain. Mathematically, the desired Gaussian filter operation corresponds to convolution with a Gaussian kernel over the input domain 2.. This in turn is equal the expected value
2

---

/ Vfc(t)g(z - 2)de - Benx, [V]c
(Vic * (x)
(1)
where N, is the Normal distribution analogous to the kernel in terms of covariances [15]. The variance of this distribution represents trade-off between the strength of the smoothing and using information from regions of the input space that are longer relevant the actual input. It should be chosen large enough to aggregate gradient information over similar inputs, but small enough to not change the classification or subjective qualitative nature of the input humans.
SmoothGrad. While the convolution in (1) is generally intractable or least unfeasible, it can be approximated via sampling [6]. Using samples &. drawn from N₂ we obtain the definition of SmoothGrad (SG) as
SG, (2) = (.,
(2)
Since this corresponds to an averaging of input gradients over noisy inputs, the computational cost of SG grows linearly in the amount of samples, requiring n forward and n backward passes through the model, with a suggested sample size of = 50 [6]. While SG converges to the convolution (1) in the infinite sample limit, it is well-known that computationally tractable amounts of samples may not be sufficient in practice, say, for ImageNet samples, as is visually apparent in the heatmaps of Figure (b) and demonstrated in Section 3.1. Due to this, SmoothGrad is commonly seen as a perturbation- based method instead of one based on Gaussian convolution.
Gradient explanations rewritten as factorized Jacobians. For any function ʃ R -> Rm the gradient of the e-th output corresponds to the e-th row of the Jacobian Jz) For a deep neural network f f fN-1 o... o f² composed of N functions f", this gradient can therefore be derived via the chain rule
(f((=)"= [J,(Z)] = (z())) = J,x(("))
(3)
J..))
where denote the input activation of the n-th function a", such that the input a² and a==1 = (a")). We would like to stress that such gradients can efficiently be computed using reverse mode automatic differentiation (AD), sequentially accumulating the JectornJacobian products (VJPs) on the right-hand side of (3) from the left to the right [16, 17, 18]. Since this computation requires knowledge of all activations a a' a forward pass through the model must first be computed, saving all activations in memory for the subsequent backward pass. By writing the forward pass in AD frameworks such as PyTorch [19] and JAX [20], gradients explanations (3) can be computed up to numerical accuracy without requiring any additional code.
1.2 Contributions
In this work, we provide the following key contributions:
1. We motivate Gaussian convolutions in embedding space as a solution to the shattered gradient problem [13], emphasize that SmoothGrad numerically approximates said convolution in the infinite sample limit, and demonstrate that commonly used SmoothGrad sample sizes are typi- cally insufficient.
2. We introduce SmoothDiff based on AD, which is both faster and more sample efficient than SmoothGrad in approximating Gaussian convolutions. We provide experimental benchmarks and clear guidelines on how to implement SmoothDiff for arbitrary differentiable model architectures. We demonstrate that SmfothDiff directly and efficiently addresses the shattered gradient problem.
1.3 Related work
NoiseGrad is variant SG that computes an average of gradient explanations over perturbed model weights instead of perturbed inputs [21]. It is therefore not connected to input space convolutions.
3

---

Perturbing both inputs and model weights results the FusionGrad method. In [22], low-pass filter applied to gradient explanations in image space. Since the smoothing not applied in input space, this approach does not directly address the SGP and runs into the risk of removing meaningful high- frequency spatial information from heatmaps. In [23], the high curvature of neural network decision boundaries and how it can be exploited to manipulate explanations is explored. The proposed 3-Smoothing method smoothes the decision boundary by replacing ReLUs with Softplus activations on the backward pass. As we show in Appendix G, this is good approximation of Gaussian convolution in the case of a single-layer neural network. However, as shown in Figure l(a), the approximation error grows with network depth, after the first layer, subsequent activations may not be Gaussian anymore.
2 SmoothDiff
We now propose SioothDiff, a computationally efficient approximation of the Gaussian convolu- tion. Inserting the factorized neural network gradient (3) into the convolution (1), we obtain
fV g)(x) = =e" Ea-N, [J (a J (a (+))
(4)
in which all intermediate activations a' deterministically depend on the random variable a. By neglecting cross coos-coovriianess between Jacobians, this expected value of product of Jacobians can be approximated as a product of expected values of Jacobians, resulting in the definition of SmoothDiff:
SD(x) =e=- EaN (J,x(a*)
E (+))
(5)
JNM
Note that this definition is based on the gradient decomposition (3), however, here we propose to replace Jacobians J.. by the expected values of Jacobians Jy..
2.1 Jacobian factorization
The chain rule can be used to factorize neural network Jacobians to arbitrary levels of granularity: from layer-wise Jacobians down to derivatives of elementary functions like addition and multipli- cation. For SmoothDiff, this raises the question of optimal Jacobian factorization (5), as this choice affects which cross-covariance terms are neglected, trading an approximation bias for increased sample efficiency and computational performance. While in theory many options arise, in practice the choice of factorization for given model architecture is typically straightforward.
We provide the following intuition: Since neural networks f : R + Rm are universal function approximators, their Jacobians-whiek correspond to linearizations of said universal function be arbitrary m X n matrices. Instcad of estimating such "unconstrained" network Jacobians directly (2), sample efficiency can be improved by decomposing the network down to functions with constrained Jacobian structure. This includes:
1. Linear and affine functions. Since their Jacobians constant, j J is known.
Functions with sparse Jacobians (i.c. containing mostly zeros). Examples include vectorized scalar functions, such as element-wise activation functions, whose Jacobians are diagonal matrices: Since the i-th output only depends on the i-th input, Jt,- = J, oz,
2f = 0 for + j.
This property appears common activation functions. For ReLU, softplus and tanh, it holds that 0 < 0f 1.
Functions with bounded partial derivatives 0f, Jæ,
4. Functions with a closed-form analytical solution to the expected Jacobian J. Point 1 is special case of this.
Applied to common network architectures, these principles discourage us from naively applying the factorization on layer-wise basis. Instead, element-wise activation functions should be separated from otherwise affine functions. The Jacobian of a dense layer f(x) = a(g(x)) = a(W W a + b) should therefore be decomposed as
= Jg = J. W.
(6)
4

---

In other words: instead of estimating the dense matrix J, containing n entries, where n is the input and m the output dimensionality, we should leverage our knowledge of the constant Jacobian J J g W and only estimate the m bounded entries in the sparse diagonal matrix J,, resulting in higher sample efficiency. We give an implementation example of this in the following.
2.2 Example implementation
SmoothDiff can be implemented as a modiffied-backprep method,' replacing the sequential compu- tation of VJPs (3) by what we term pector-expected-Jacobiee products (VEJPs) in (5), sequentially evaluating v" y- on a single backward pass. For linear and affine functions with expected Jacobian j= J, it follows that v" J = v" J. Applying SmoothDiff to such functions requires no additional code, common AD frameworks such Py Torch and JAX already implement corresponding VJPs. SifothDiff only requires implementing VEJPs for functions in the model which are non-linear with respect their input, such that J + J. the model contains functions for which J has analytical solution, sampling estimates can be obtained by computing n forward passes through the model. As motivating example, we now discuss how to implement VEJP for ReLUs This should provide a blueprint on how to implement SmoothDiff on variety of current and future model architectures.
As discussed in the previous section, the Jacobian Ja of any element-wis activation function is diagonal matrix. For ReLUs specifically, the diagonal entries correspond to
1 if a; 20
Jo, (a) (2)0
0 elsc
where 1,20 is the indicator function returning one for inputs greater equal zero. From this, we can derive the expected Jacobian J, as
o, E B [a)] = lim 2-000
1 1 (a:),
(8)
where the right-hand side corresponds to the expected ratio of positive activations. A sampling approximation this term can obtained by computing a finite amount of forward passes through the model with inputs & drawn from Ng. while keeping track of the count of non-zero activations Pt. For a given vector this results in the VEJP
0 J.] = U;
= 02
Pj
(9)
such that the SioothDiff explanation (5) can be computed in single matrix-free backward pass. An analogous approach can be taken to estimate J of max pooling layers, whose entries correspond to binary indicator functions of maximally activated inputs, requiring a sampling estimate of the ratio of activations with maximum value. Pscudocode implementation for both ReLUs and max pooling layers are given in Appendix I. While our exposition emphasized sequential models for the sake of simplicity, more complex, branching models such ResNets [24] are automatically handled by AD systems. In fact, implementing the two aforementioned VEJPs is all it takes to run SmoothDiff on common vision models such VGG and ResNets.
exxexaaaii
2.3 Fallback VEJP
SmoothDiff can be applied to arbitrary differentiable models. For any function (c.g. an arbitrary layer), fallback VEJP be implemented that samples forward passes, computes the full Jacobian for each sample, and approximates the expected value by keeping track the running mean. The naive computation of a Jacobian requires either as many VJPs (reverse mode) or as many Jacobian-vector products (JVPs) as it has inputs (forward mode). If the Jacobian exhibits sparsity, automatic sparse differentiation [17, 25] can drastically reduce the number of required VJPs/JVPs (see [26, 27] for applications in deep learning).
the function has outputs
'The term modified- pandified-backprop method misnomer based on implementation details. Backpropagation is a special case of reverse mode AD, while Jacobian transformations (such as our expected Jacobians) can in many cases also be implemented for forward mode AD systems. Refer to [17] and [15] for an overview of AD techniques.
5

---

For functions with diagonal Jacobians (c.g. activation functions) this fallback VEJP is efficient, only requiring single VJP per sample. For large and dense Jacobians however, requires lot of compute and memory, negating SmoothDiff's performance benefits over SG.
2.4 Computational performance and other properties
As we will demonstrate in Section 3.1, estimating factorized Jacobians according to Section 2.1 is significantly more sample efficient than estimating the Jacobian of the entire neural network. In practice, SmoothDiff therefore requires an order of magnitude fewer samples than SG to achieve visually comparable (see Section 3.3 and Appendix K) amount of smoothing or noise reduction.
Additionally, for single class explanations of sample size n, SmoothDiff only requires n forward and a single backward pass, whereas SG requires n forward and n backward passes. Denoting the unit time complexity of a single forward pass f(x) by т, the corresponding backward pass has been shown to be in the same order of time complexity O(7) [28], and we (conservatively) estimate their costs be equal. For single-class explanations with samples, SmoothDiff' yields stark improvement requiring n + 1 (forward or backward) passes in comparison to SG's 2n.
The largest performance gains occur when multiple explanations need to be computed for given input, e.g. multi-class explanations and explanations with respect to concepts or filters that restrict the propagation through specific neurons. This is required for spectral analysis of explanations [29], concept relevance propagation [30], and higher-order explanation techniques [9, 10, 31J. So far, such methods have predominantly used the highly efficient LRP method [7, 8]. Assuming explanations of & classes (or panncets/fiiltersy LRP requires only a single forward pass and & backward passes. In contrast, SG only estimates class dlassddepnndent smoothed gradients (2) and needs to compute forward and n backward passes for each of the k classes, resulting in a total of 2nt passes.
Since SmoothDiff estimates the full smoothed Jacobian (5) using matrix-freee operators (VEJPs), can be applied at a similar time complexity to LRP. Once VEJPs are estimated in n forward passes, they don't i need to be recomputed and only one backward pass per class (or concept ffilerr is needed, explanation of all k = 1000 ImageNet classes
resulting in total of n n k passes. Assuming using n = 50 samples, SmoothDiff yields a computational speedup of two orders of magnitude" over SG's approximation to Gaussian filtering and approaches three orders of magnitude including the increased sample efficiency demonstrated in Section 3.
In addition to convergence and performance improvements, SmoothDiff directly addresses the SGP, as we demonstrate in Appendix H. We further discuss in Appendix J how SmoothDiff's expected network Jacobian increases the rank compared to the regular network Jacobian J,, a desirable property for explanation methods that has been found to allow for more expressiveness and higher variation of explanations across classes [29].
2.5 Limitations
While SmoothDiff can be applied to arbitrary models, performance benefits are obtained by deriving performant VEJP (see Section 2.3). Such VEJPs have yet to be derived for attention layers, currently negating SmoothDiff's performance Smmoothiifsseerrrmaance benefits over SG on ViT [32]. This limitation could be addressed through partial application of SmoothDiff, falling back to VJPs when VEJPs are available.
Experiments
We evaluate SmoothDiff and SG on a pre-trained VGG-19 model [14] and the ImageNet dataset [33]. All experiments were run on a NVIDIA A100 80GB GPU and AMD EPYC 9124 CPU. We use the Julia-XAI ecosystem [34, 35], Flux.jl deep learning framework [36, 37] and implement VEJPs using ChainRules,j/ [38] and Zygote,jl [39].
We provide the complete source code to reproduce our experiments at https: t/tittu/ com/ including SmoothDiff reference implementations in Julia [40]
http://mmottiiff-eeeeiiiiie and PyTorch
"The suggested sample size for SG [6].
e SmoothDifff requires + A 1050 forward and backward passes, whereas SG requires 2nk 10a.

---

Smooth
SmocthGrad
Mean SSIM
0.9
x0.2
Walltime (s)
0.2
0.8
0.1
0.6
0.5
0.4
0.0
10
20
30
50
60
10
20
30
40
50
60
Number of samples
Number of samples
Figure 2: SmoothDiff improves SG in terms of both sample efficiency and computational performance. Left: Improvements sample efficiency, measured by the similarity to converged SG explanations in terms of SSIM [41]. At the recommended sample size of 50, SG achieves SSIM 0f..8. SifothDiff requires five times fewer samples to achieve similar SSIM, highlighted by the green arrow. Right: Improvements in computational efficiency, measured by walltime per sample. Both Sifoth and SG scale linearly, however SmoothDiff is approximately twice as fast across sample sizes.
3.1 Convergence comparison
Since the Gaussian convolution (1) has no analytical solution, we use converged+ SG explanations over batch of 128 randomly drawn ImageNet [33] images as our reference. Convergence towards this reference is quantified using the structural similarity index measure (SSIM) [41]. standard deviation of 0.5 is used for the sampling distribution N₂ in both SmoothDiff and SG.
The resulting mean SSIM are shown on the left of Figure Since SG is used as a reference, by construction approaches SSIM of 1 with increasing sample counts. We observe that through the factorization in (5), SmoothDiff trades a small inherent bias form of neglected cross-covariance terms for large increase in sample efficiency. Using the recommended sample size of n 50, SG reaches a SSIM of 0.8, whereas with SmoothDiff, the same SSIM is obtained in just 10 samples, resulting in inereased sample efficiency of factor 5. For discussion of the neglected cross-covari- ance terms, refer to Appendix E.
3.2 Benchmarks
We benchmark the performance of SmoothDiff and SG function of sample size by computing explanations for batches of 128 ImageNet images. As shown on the right side of Figure 2, SmoothDiff approximately twice as fast SG across all sample sizes, further compounding the increased sample efficiency demonstrated in Section 3.1. These computational performance improvements match theoretical considerations in Section 2.4, where single-class explanations using SmoothDiff are shown to require forward and backward passes, whereas SG requires 2n.
3.3 Heatmaps
For qualitative comparison of SmoothDiff and SG convergence behavior we visualize heatmaps over increasing sample sizes in Figure 3. Even at small sample sizes, SmoothDiff leads to well- contoured heatmaps with little noise, while SG only progressively reduces noise effects function of n; it becomes apparent from Figure 3 that even at the recommended parametrization of SG, i.e. n 50, there is further room for improvement. Further heatmaps including examples of multi-class explanations can be found Appendix K.
3.4 Quantitative results
In addition to the qualitative results in Section 3.3, we use pixel-fliping to quantify SmoothDiff's performance. In the absence of ground truth, pixel-flipping measures the faithfulness of an XAI method by perturbing features in the order of their attribution strength. We use the symmetric
*More specifically, SmoothGrad explanations using
10 samples.

---

n=4
n=8
n=16
n=32
n=
n=
"ace Car
SmoothDif
input
SmeothGrad
Gradient
n=32
n=64
'barsershop'
SmocthDiff
mout
Barber Shop
Muss
SHIT FALAME
SmoothSrad
Gradient
Figure 3: Qualitative comparison of SmoothDiff and SG heatmap convergence over increasing sample sizes. Even at very small sample sizes, SmoothDif? leads to well-contoured heatmaps with little noise.
relevance gain (SRG) measure [42], which largely removes the influence of imputing methods on pixel-flipping. This is achieved by computing the area under two pixel-fliping curves: one deleting features in order of most influential first (MIF), the other in order of least influential first (LIF). The SRG defined as the difference between the LIF and MIF measures. Figure shows the results using mean imputing and standard deviation of 0.5 for SmoothDiff and SG. SmoothDiff reaches higher peak SRG and does so at lower sample size. remains superior or equal to SmoothGrad up approximately the recommended sample size of SmoothGrad.
We additionally compare optimal SmoothDiff and SmoothGrad settings from Figure 4 to regular gradient explanations and B-Smoothing (see Section 1.3 and Appendix G), which also attempts smooth gradients. The pixel-flipping results are summarized Table 1, demonstrating that SmoothDiff outperforms other methods in the SRG, as well as the LIF and MIF measures [42].
0.30
SRG
0.28
SmoothDiff
0.26
SmoothGrad
10
20
30
50
Number of samples
Figure 4: Comparison of SmoothDiff' and SG in terms of pixel-flipping performance, measured by SRG [42]. While the performance of SG increases with sample size up to n 40, SmoothDiff's performance peaks just 4 samples.
'In terms of the area under the tixel-flipping curve, deleting LIF is equivalent to inserting MIF The SRG measure therefore combines insertion and delction tests.

---

Table 1: Pixel-flipping results (t: higher is better, 4: lower is better).
Method
Parameters
LIF
MIF +
SRG f
SmoothDiff'
0.5, nt
0.382
0.066
0.316
SmoothGrad
0.5,
40
0.373
0.069
0.304
g-Smoothing
1.0
0.316
0.086
0.230
S-Smoothing = 2.0
0.311
0.093
0.218
g-Smoothing Ø 0.5
0.293
0.098
0.195
Gradient
0.276
0.111
0.166
For additional quantitative results in terms of localization, complexity and robustness, refer to Appendix F, which compares SmoothDiff with gradients, SG, SmootGrad-Squarer [43], Integrated Gradients [44), LRP composites [45, 46], GradCAM [47], and several random baselines.
4 Discussion
SmoothGrad is a popular gradient-based choice for explaining ML models based on convolving gradients. While originally conceived for compensating shattered gradient effects, it is challenged by the need to accurately approximate the convolution, which would, in principle, require expensive sampling. In practice, SmoothGrad users compromise by trading accuracy with speed. In our work we contributed by proposing SmoothDiff which making use of the well-known mathematical structure of the Jacobian in neural networks together with automatic differentiation functionalities. Specifically, we show that the expected Jacobian can be decomposed efficiently into independent parts, with focus on the relevant network non-linearities only. Automatic differentiation helps to readily implement this idea using vector-Jacobian products without further work. This results in higher accuracy of the smoothed gradient convolution estimation at fewer samples: resulting in speedups of up to two orders of magnitude for multi-class explanations over the original Smooth- Grad. This means that SmoothDiff helps to enhance the usability (speed and quality) of SmoothGrad greatly as seen across all simulations and benchmarks explored. As smoothing comes with a reduction of curvature or second order derivatives, we conjecture that smoothed explanations as the ones studied here may also provide higher resilience in the spirit of [23].
We can see three potentially interesting directions to further improve SmoothDiff: (i) Instead of approximating Gaussian convolutions by random samples, more efficient means of quadrature should be investigated. (ii) Moving beyond Gaussian filtering over the entire input domain, the geometry of the data manifold can taken into account for convolutional smoothing, as shown in [48]. Insights from SmoothDiff Diff could also be applied to other methods integrating over sensitivities, e.g. Integrated Gradients [44]. (iii) Instead of sampling activations on multiple forward passes, probability distributions could be propagated in single forward pass, possibly further accelerating the method.
Future work will furthermore explore smoothed differentiation as in SmoothDiff beyond XAI, for example in applications of AI for the sciences where smooth priors in terms of differential equations exist, say, in the laws of physics.
Contributions
AH devised the project and main conceptual ideas. He wrote the majority of the manuscript, including experiments, benchmarks, figures, tables and code. NM contributed to the initial design of the method during his lab rotation under the guidance of AH. He evaluated the 1D example in Figure I(a) and Appendix B, compared color schemes in Appendix D, wrote the initial draft of Appendix H,, created the ReLU pseudocode listing Appendix I, and plotted Figure 2 from All's experimental results. JM wrote the Tororc implementation of SmoothDiff, evaluated the GridPG experiments Appendix F.2 and contributed to Scction 2.4. SB advised the pixel-flipping experi-

---

ments in Section 3,4 and helped frame the paper. KRM supervised the project and helped frame, structure and write the paper. All authors discussed the results and commented on the manuscript.
Acknowledgements
We would like to thank the anonymous reviewers for their feedback, especially reviewer "Meht". We also thank Christopher Anders, Shinichi Nakajima, Anna Hedström, and Stefan Gugler for their valuable feedback and insightful discussions. Furthermore, we gratefully acknowledge funding from the German Federal Ministry of Education and Research under the grant BIFOLD25B. KRM was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (ITP) grant funded by the Korea government (MSIT) (No. Intelligence Graduate School Program, Korea University) and grant funded by the Korea govermment (MSIT) (No. RS-2024-00457882, Al Research Hub Project).
Artificial
References
[1] W. Samek, G. Montavon, S. Lapuschkin, C. Anders, and K.-R. Müller, "Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications," " Proceedings of the IEEE, vol. 109, no. 3, pp. 247 278, Mar. 2021, doi: 10. R0.1099//0C..222.33000
[2] S. LLapuschkin, S. Wäldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller, "Unmask- ing Clever Hans predictors and assessing what machines really learn," " Nature communications, vol. 10, p. 1096, 2019.
[3] J. Kauffmman, J. Dippel, L. Ruff, W. Samek, K.-R. Müller, and G. Montavon, "Explainable Al reveals Clever Hans effects in unsupervised learning models," " Nature Machine Intelligence, vol. 7, pp. 412-422, 2025.
[4] D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K.-R. Müller, "How to Explain Individual Classification Decisions," " Journal of Machine Learning Research, vol. 11, no. 61, pp. 1803-1833 2010, http://mmm..or/ppaprss.... 1/bachrens ht..
[5] K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep Inside Convolutional Networks: Visual- ising Image Classification Models and Saliency Maps." " http://arxiv.orr/abs/ 12..6033
[6] D. Smilkov, Thorat, B. Kim, Viégas, and M. Wattenberg, "SmoothGrad: removing noise by adding noise." http://arxiv.org/abb// 700.0000
[7] S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek, "On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation," " PloS one, vol. 10, no. 7, p. e130140, 2015.
[8] G. Montavon, S. Lapuschkin, A. Binder, W. Samek, and K.-R. Müller, "Explaining nonlinear classification decisions with deep Taylor decomposition." Pattern Recognition, vol. 65, pp. 212-222, May 2017, doi: pt..01///..tcoo.2016..... 11..0...............
[9] O. Eberle, J. Büttner, F. Kräutli, K.-R. Müller, M. Valleriani, and G. Montavon, "Building and interpreting de similarity models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. pp. 1149 1161,1111111111 2022.
[10] T. Schnake al., "Higher-order explanations of graph neural networks via relevant walks," IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 11, pp. 7581 7596, 2022.
[11] F. Strumbelj and I. Kononenko, "An Efficient Explanation of Individual Classifications using Game Theory," Journal of Machine Learning Research, vol. 11, no. 1, pp. 1-18, 2010, http:// jmlr.org/papers/v httppppprss/llsstummbbleeee (attrrummeejlO.hhmmm
[12] S. M. Lundberg and S.-I. Lee, "A Unified Approach Interpreting Model Predic- tions," in Advances in Neural Information Processing Systems, Curran Associates,
10

---

Inc., 2017. http://wwwwoccaacaccc/cccccc///a 67767-Abstract.hmml
[13] D. Balduzzi, M. Frean, L. Leary, J. Lewis, K. W.-D. Ma, and McWilliams, "The Shattered Gradients Problem: If resnets are the answer, then what is the question?? in Proceedings of the 34th International Conference on Machine Learning, PMLR, Jul. 2017, pp. 342-350. https:// proceedings.llr.resss/vv hww.r/ngg..m....pss////aaauiiiii 70/balduzzi//
7b.html.
[14] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition." " http://arxivv.rg/abbb// http://aab////...1//1 1556
[15] M. Blondel and V. Roulet, "The Elements of Differentiable Programming." http://arxiv.org/ abs/2403. 14666
[16] Y. A. LeCun, L. Bottou, G. B. OrI, and K.-R. Müller, "Efficient BackProp," " Neural Networks: Tricks of the Trade: Second Edition. Springer, Berlin, Heidelberg, pp. 9-48, 2012. doi: 10. 10.1007/97833-622-5289-9 3.
[17] A. Griewank and A. Walther, Evaluating Derivatives. in Other Titles in Applied Mathematics. Society for Industrial, Applied Mathematics, 2008. doi: 10. 077719997788771111 17761.
[18] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind, "Automatic differentiation in machine learning: survey," Journal of machine learning research, vol. 18, no. 153, pp. 1-
43, 2018.
[19] A. Paszke et al., "PyTorch: An Imperative Style, High-Performance Deep Learning Library," " in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2019. https:/prrocedingss.eem 0la97001722740-Abstraacct
files//paper//11990 0ieefggree/0000//ooooooaaa2teee 19hashobbbbc222888777722
[20] J. Bradbury al., "JAX: composable transformations of Python+NumPy programs." " http:// github.com/goooleejax
[21] K. Bykov, A. Hedström, S. Nakajima, and M. M.-C. Höhne, "NoiseGrad - Enhancing Explanations by Introducing Stochasticity to Model Weights," " Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 6, pp. 6132 6140, Jun. 2022, doi: 10.1609/ nei.v3366.20560
[22] S. Muzellec, Fel, V. Boutin, L. Andéol, R. Vanrullen, and T. Serre, "Saliency strikes back: How filtering out high frequencies improves white-box explanations," in Proceedings of the 41st International Conference on Machine Learning, PMLR, Jul. 2024, pp. 37041-37075. https:/ppoceedings.llrprssss wttp://snn..mmm..rs////ss/// 2tt.peessss22255mmmme2e2222a.2..
[23] A.-K. Dombrowski, M. Alber, C. Anders, M. Ackermann, K.-R. Müller, and P. Kessel, "Expla- nations can manipulated and geometry is to blame," Advances in Neural Information Processing Systems, Curran Associates, Inc., 2019. https://papers.nip wwwppee.nnis.ccppppppp. httpsccpppppfffllspppprrrre 220009944555e44ha44aaaaaaaaaa http:///BBScccoooo////aaooaocasa 201 9//hsshhbb836601ldaaa
[24] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," " in 2016 IEFE Conference Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 770-778 doi: 10. N0.1109/CVPR.2016.0
[25] A. H. Gebremedhin, F. Manne, and A. Pothen, "What Color Is Your Jacobian? Graph Coloring for Computing Derivatives," SIAM Review, vol. 47, no. 4, pp. 629-705, Jan. 2005, doi: 10/ cmwds4.
[26] A. Hill and G. Dalle, "Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation," " Transactions on Machine Learning Research, Jun. 2025, https://
[27] A. Hill, G. Dalle, and A. Montoison, "An Illustrated Guide to Automatic Sparse Differenti- ation," " in The Fourth Blogpost Track ICLR 2025, 2025. https:/oppmeeviiw.nettomm'dd ykZibuSbJj.
11

---

[28] W. Baur and V. Strassen, "The complexity of partial derivatives," Theoretical Computer Science, vol. 22, no. 3, pp. 317-330, Feb. 1983, doi: 10.10160330433975533))10000
[29] J. MaeB, G. Montavon, S. Nakajima, K.-R. Müller, and T. Schnake, "Uncovering the Structure of Explanation Quality with Spectral Analysis." http://arxivooraab2222.....5555
[30] R. Achtibat er al., "From attribution maps to human-understandable explanations through Concept Relevance Propagation," " Nature Machine Intelligence, vol. 5, no. 9, pp. 1006 1019, Sep. 2023, doi: 10. 10388442255-023-00711--
[31] L. Linhardt, K.-R. Müller, and G. Montavon, "Preemptively pruning Clever-Hans strategies in deep neural networks," Information Fusion, vol. 103, p. 102094, Mar. 2024, doi: 10.1016/ j.inffus.22231022099
[32] A. Dosovitskiy et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," in International Conference on Learning Representations, 2021. https://ppenreview. douforrum'id-YicbFii
[33] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," " in 2009 IEEE Conference on Computer Vision and Pattern
Recognition, Jun. 2009, pp. 248 255. doi: 10. 0..109/CVPR.2009.55000
[34] A. Hill, "hulaa-AA/Viiiolleaaaaaa v2.0.1." https://eenodd.ogggooood/1/11222
[35] A. Hill, "ExplainableAI.ji"" https://zenodo.or//eeccotto org/rrccords155176555
[36] M. Innes et al., "Fashionable Modelling with Flux," " CoRR, 2018, https://anxiv.orrab////1111 01457.
[37] M. Innes, "Flux: Elegant machine learning with Julia," Journal Open Source Software, vol.
3, no. 25, p. 602, May 2018, doi: 10.21 1060000
[38] F. White et al., "JuliaDiffChainRules] v1.72.3." http:///coodoooorrrrrrrrr//////
[39] M. Innes, "Don't Unroll Adjoint: Diffcrentiating SSA-Form Programs," " CoRR, 2018, http:// arxiv.org httpg//aab//1100995553 1810.07951.
[40] J. Bezanson, A. Edelman, S. Karpinski, and V. B. Shah, "Julia: A fresh approach to numerical computing." SlAMreview vol. 59, no. pp. 65 66-98,22296688888 2017, https:://oi..rrr
[41] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004, doi: 10. (09/TIP.2003.8198)
[42] S. Bluecher, Vielhaben, and N. Strodthoff, "Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks," Transactions on Machine Learning Research, Mar. 2024, http://pppnneeveeeeee/ffoomoodoo XdiUU
[43] S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim, "A Benchmark for Interpretability Meth- ods in Deep Neural Networks," " in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2019. https::/papers..ipp wwtppscccpppppeee files/paper/20 9atp9//hhhhhaae htshhh//c448550000000 1a099daa5c55554100 O-AAbsrract.html
[44] M. Sundararajan, A. Taly, and Q Yan, "Axiomatic attribution for deep networks," in Interna- tional conference on machine learning, PMLR, 2017, pp. 3319-3322 3328.
[45] G. Montavon, A. Binder, S. Lapuschkin, W. Samek, and K.-R. Müller, "Layer-wise relevance propagation: overview," " Explainable Al: interpreting, explaining and visualizing deep learning, pp. 193-209, 2019.
[46] M. Kohlbrenner, A. Bauer, S. Nakajima, A. Binder, W. Samek, and S. Lapuschkin, "Towards Best Practice in Explaining Neural Network Decisions with LRP," in 2020 Inter- national Joint Conference on Neural Networks (IJNN), Jul. 2020, pp. 1-7. doi: 10.1109/ UCCNN86605.0020 9206975.
12

---

[47] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, "Grad-cam: Visual explanations from deep networks via gradient-based localization," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 618 626.
[48] L. Zhou, C. Ma, Z. Wang, and X. Shi, "Rethinking the Principle of Gradient Smooth Methods in Model Explanation." http://arki..orraass/22000.77
[49] L. Arras, A. Osman, and W. Samek, "CLEVR-XAI: benchmark dataset for the ground truth evaluation of neural network explanations," Information Fusion, vol. 81, pp. 14 14-40, May 2022, doi: 10. w/t.///..ffff..20000 20w.....2........00
[50] F. Crameri, G. E. Shephard, and J.J. Heron, "The misuse of colour in science communication," " Nature Communications, vol. 11, no. 1, p. 5444, Oct. 2020, doi: 0001:1////544477000-1-11919
[51] F. Crameri, "Scientific colour maps." " https:://cmoddooor//rooo///////
[52] P. Kovesi, "Good Colour Maps: How to Design Them,"
[es], Sep. 2015,
http://aaxxiv.orgabbs/ 1at.bb///509000000..
[53] R. Bujack, T. L. Turton, F. Samsel, C. Ware, D. H. Rogers, and J. Ahrens, "The Good, the Bad, and the Ugly: A Theoretical Framework for the Assessment of Continuous Colormaps," " IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 1, pp. 923 903-933, Jan. 2018, doi: 10. 10.1100 100.///C.2000..774..5
[54] M. R. Luo, G. Cui, and B. Rigg, "The development of the CIE 2000 colour-difference formula: CIEDE2000," Color Research & Application, vol. 26, no. 5, pp. 340-350, 2001, doi: 10. 1002/ c04.
[55] A. Hedström et al., "Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond," " Journal of Machine Learning Research, vol. 24, no. 34, pp. 1-11, 2023, http://jmlr.org// 21tpppp//////2201144/422
[56] S. Gao, Z.-Y. Li, M.-H. Yang. M.-M. Cheng, J. Han, and P. Torr, "Large-scale Unsupervised Semantic Segmentation," " TPAMI, 2022.
[57] N. Kokhlikyan et al., "Captum: A unified and generic model interpretability library for pytorch," " arXiv preprint urXiv:2009. 07896, 2020.
[58] C. Anders, D. Neumann, W. Samek, K.-R. Müller, and S. Lapuschkin, "Software for dataset- wide XAI: from local explanations to global insights with Zennit, CoRelAy, and ViRelAy," arXiv preprini arXiv: 106. 13200, 2021.
[59] D. Alvarez-Melis and T. S. Jaakkola, "On the Robustness of Interpretability Methods." http:// www.oorg/abs/18000 https/810000000004
[60] D. Alvarez Melis and T. Jaakkola, "Towards Robust Interpretability with Self-Explaining Neural Networks," in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2018. https:/prroceedigg.nmue www.iinsgn...uiss...ommmu cctppppeec/ppperrr.c. files//aper/2211 Mitss/aapr220118hhh////000000020 Binsh/c99006999228800000 62339994ffc76-AAsstaaa.tttt
[61] C.-K. Ych, C.-Y. Hsich, A. Suggala, D. Inouye, and P. K. Ravikumar, "On the (in) fidelity and sensitivity of explanations," Advances in neural information processing systems, vol. 32, 2019.
[62] M. Böhle, M. Fritz, and B. Schiele, "Convolutional Dynamic Alignment Networks for Inter- pretable Classifications," " in 2021 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2021, pp. 10024-10033. doi: 10.1109/CVPR44443.2222 .00990.
[63] M. Böhle, N. Singh, M. Fritz, and B. Schiele, "B-cos alignment for inherently interpretable CNNs and vision transformers," " IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 46, no. 6, pp. 4504 1504-458,, 2024.
[64] U. Bhatt, A. Weller, and M. F. Moura, "Evaluating and Aggregating Feature-based Model Explanations," " in Proceedings of the Tweht International Joint Conference on Artifi-
13

---

cial Intelligence, Yokohama, Japan: International Joint Conferences on Artificial Intelligence Organization, Jul. 2020, pp. 3016-3022. doi: 10224963/jjcaa22222441.
[65] P. Chalasami, Chen, A. R. Chowdhury, X. Wu, and Jha, "Concise Explanations of Neural Networks using Adversarial Training," in Proceedings of the 37th International Conference on Machine Learning, PMLR, Nov. 2020, pp. 1383- 1391. https://pooeeddngg..ll.pssssss/s chalasani20a.html.
[66] G. Montúfar, R. Pascanu, K. Cho, and Y. Bengio, "On the Number of Linear Regions of Deep Neural Networks," in Advances Neural Information Processing Systems, Curran Associates, Inc., 2014. https:/pproeedding.eeurjjj paper fttpe//ffeeeee//201111 hatpaapeee2na/aaaaaaaaaaaaaaaaa 4/hash/ff6622444404444d a 49667766173322244bbtrrr.......
[67] M. Ancona, E. Ceolini, C. Öztireli, and M. Gross, "Towards better understanding of gradient- based attribution methods for Deep Neural Networks," " Feb. 2018. https://openreview.net forum?id--S211R9JAV.
[68] A.-K. Dombrowski, J. E. Gerken, and Kessel, "Diffeomorphic Explanations with Normal- izing Flows," in ICML Workshop Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021. httpps//pceccciie..cfffoo//ttt/p
14

---

A Notation
We denote scalar quantities by lowercase letters æ, vector quantities by bold lowercase letters a and matrices by bold uppercase letters X. The coefficients in vector are written as T4 the rows of a matrix X as ,,, and the coefficients as |X] = Xij. using row and column indices respectively. All vectors æ are column-vectors and all x² are row-vectors. For a vector-to-scalar function f R= - R, we use (x(x) E I for the gradient vector. For vector-to-vector function f: F:R" + R", we denote as (x) E R=xm the Jacobian matrix of first-order partial derivatives 2f. The i-th basis vector is written as e,. We reserve the index c for the selection of the output 82,
neurons, such that for neural network ʃ and an input æ, the predicted logit for the e-th class corresponds to (x))
B Experiment details- Figure 1
(a) 1-dimensional toy example: To demonstrate the SGP on gradient-based XAI methods in Figure (a), we define a deep neural network similar to [13]. We use a sequence of 16 fully connected ReLU layers with 64 hidden neurons each, mapping scalar inputs to scalar outputs. Both weights and biases are drawn from a normal distribution with mean zero and variance N. Both SmoothDiff' and SG explanations are computed using o 0.05 and n 10 samples. Since the explanations are stochastic (inputs are randomly sampled), we visualize the means and 5th- to 95th-percentile bands over 1024 ovaluations per input. d-Smoothing was computed using 8 = v L ~ 117, motivated by scaling up the analytical derivation for a single layer [23]. The Gaussian convolution was computed numerically from the gradient.
(b) Vision model:
Explanations are computed on a pretrained VGG-19 model [14] and randomly selected images from the ImageNet dataset [33]. They are visualized using the heatmapping tech- niques described in Appendix D. Both SmoothDiff and SG are computed using 0.5. Since the analytical Gaussian convolution is intractable and an accurate numerical computation is infeasible, was approximated by computing SG with 10 samples (see Section 3.1).
C Relevance pooling
Most XAI methods, including SmoothDiff, have in common that the dimensionality of their numer- ical explanations matches that of the input features. The inputs to ImageNet models are (normalized) RGB images: diccesiimnniinal tensors that include width, height and color channel dimensions. Explanations of these inputs therefore also contain three scparate values per pixel, one for each color channel (red, green, blue). For the purpose of heatmapping (see Appendix D) well as several evaluation metrics (see Appendix F), these three values need to be reduced to a single scalar value by applying relevance pooling function Rpoo R + E to each pixel [49]. Popular options include summation, max pooling and the L² nn-oomm( (hereafter referred to norm-pooling).
As motivated in Section 1, explanations can be based on different mathematical objects, e.g. gradi- ents, Taylor expansions and Shapley values. The choice of relevance pooling depends on the XAI method: For LRP, summation is the most common choice, it maintains its property of relevance conservation [7]. Sum-pooled explanations can contain both positive and negative values and are therefore commonly visualized using diverging color maps, see e.g. LRP's characteristie red-white- blue heatmaps.
For gradient- based methods however, summation is not an option. We want to illustrate this with thought experiment: When using gradient-based method to explain why an image of red fire truck is correctly classified by an idealized robust model, we might expect the gradient to be positive on the red color channel (a more intense red positively affecting the classification) and negative on the blue and green color channels (also intensifying the red color). In this examplc, negative sensitivities on the blue and green color channels do not imply that the pixel of negative importance to the classifier. Sum-pooling should therefore be avoided, as it can in the worst case result in a negative value even though the pixel was of high sensitivity. Instead, we suggest the use of the norm-pooling. measuring the pixel-wise gradient magnitude. Since norm-pooled explanations are non-negative,
15

---

they should be visualized with sequential color maps (see Appendix D). As we will demonstrate in Appendix F, the choice of relevance pooling has significant impact on the evaluation XAI methods, crucial aspect that remains underexplored in the literature.
D Heatmapping
We visualize numerical explanations as heatmaps using the VisionHeatmaps.ii package [34] from the Julia-XAl ecosystem [35]. To map gradient-base based explanations onto color scheme, reduce the color channel dimension by applying norm-pooling (see Appendix C). Finally, to avoid desaturation due to numerical outliers, the color scheme is mapped to range between 0 and the 99.9th percentile of the pooled explanation, effectively normalizing values within the explanation.
Batlow
Jet
100
Perceptual Velocity
600
Luminance (CIELab)
80
(CIEDE2000)
400
60
200
20
0.0
0.5
1.0
0.0
0.5
1.0
Relative position in color map
Relative position in color map
Figure 5: Comparison of perceptual properties of the jet and batlow color maps.
We use the perceptually uniform batlow color map [50, 51J. For sequential color maps, a linear increase in luminance is desirable, as it is the primary perceptual dimension perceived as intensity, as well as the one best resolved high frequencies [52]. Additionally, constant change in overall perceptual difference between adjacent colors ensures uniformity [53]. Figure 5 compares bat lotlow with the popular jet color map. measuring luminance L' in the CIELab color space and overall perceptual velocity using the CIEDE2000 distance metric [54]. While discourage the use of the jet color map due to its perceptual non- non-uniformity, especially combination with highly processed transparent overlays, popularity in the field.
provide such SifothDiff heatmaps in Figure 6 due to their ongoing
"red fox"
"swing"
"barbershop"
raae
"red wine"
"balloon"
"racket"
"dung beetle"
Shop
Batlow
(overtay)
Figure 6: Comparison of heatmapping techniques on SifothDiff explanations (n = 10, o 0.5).
16

---

E
Characterizing neglected cross-covariances
As discussed in Section 3.1, SmoothDiff trades a small inherent bias in form neglected cross- covariance terms in (5) for a large increase in sample efficiency. To quantify the bias, we modify the convergence experiment from Section 3.1, treating SmoothDiff and SG explanations (i.c. smoothed gradients) vectors. We then measure their similarity to the converged SG reference explanation by computing cosine similarities and magnitudes.
The resulting convergence plots are shown Figure 7: In terms of cosine similarity, Smooth shows superior convergence behavior up SG's recommended sample size of 50. This indicates that the direction of SmoothDiff explanations aligns well with the reference. In terms of magnitude however, SmoothDiff has a clear bias: While SG converges towards the magnitude of the reference (i.e. a relative magnitude of Tone), SifothDiff converges towards a much smaller relative magnitude of 0.441. We suspect that this difference in magnitude is caused by the neglected cross-covariances.
As described in Appendix D, explanations are relevance pooled and normalized during heatmapping. Heatmaps therefore only depend on the direction of the explanation vector, not the magnitude, limiting the effect of neglected cross-covariances. This observation is in line with the SSIM conver- gence plot in Figure 2.
SmoothDiff
SmoothGrad
Cosine Similarity
1.0
0.8
0.6
0.4
10
20
30
40
50
Relative Magnitude
3
2
0
Number of samples
Figure 7: Convergence behavior of SmoothDiff and SG towards the reference explanation in terms of cosine similarity (upper) and relative magnitude of the explanation (lower). Convergence with the reference is obtained a cosine similarity and relative magnitude of one.
F Further quantitative results
In addition to the pixel-flipping experiments which evaluate faithfulness in Section 3.4, we also evaluate metrics in the domains of robustness, localization and complexity using the Quantus toolkit [55]. Since some localization metrics require ground truth segmentation masks that localize the object of the target class, a dataset of 256 input images, labels and segmentation masks was created by sub-sampling correctly classified images from the ImageNet-S-50 dataset [56].
In addition to the Gradient, SifothDiff and SG methods in Section 3, also evaluate SmoothGrad- Squared [43], Integrated Gradients [44], GradCAM [47], two IRP composites [45, 46), and three random baselines. Reference implementations from the Captum software package [57] are used for Integrated Gradients and GradCAM, while Zennit [58] is used for LRP, namely the EpsilonPL and Epsi LonA phanetttaBBetl composites. Metrics are evaluated on pre-trained VGG-19 and ResNet-18 models using Quantus' default parameters.
As motivated in Appendix C, relevance pooling significantly impacts both heatmapping and evalu- ation metrics. Since the goal of iA to further humans' understanding of machine learning models,
17

---

we argue that evaluation metrics should apply the same relevance pooling functions that are used to compute heatmapping visualizations. We therefore evaluate gradient-based methods (Gradient, SmoothGrad, SmoothDiff, Integrated Gradients) with norm-pooling and methods based on Taylor expansions (LRP composites, Input x Gradient) with sum-pooling.
In [43], a variant of SG called SmoothGrad- -Suuudddd (SG-SQ) shown to perform well using sum- pooling. Instead of averaging gradients, SG-SQ averages squared gradients. While not exactly analogous," 7 we introduce SmoothDiff-Squaee (SD-SQ), which we define as the square of SmoothDiff explanation
(SD)
(10)
We want to emphasize that sum-pooled SD-SQ explanation is the square of norm-pooled SmoothDiff explanation, blurring the lines between XAI method and relevance pooling. To further quantify the influence of relevance pooling on evaluation metries, we introduce three random baselines that are equally uninformative:
Random (L²): random explanation drawn from a normal distribution, norm-pooled
Random (): random explanation drawn from a normal distribution, sum-pooled
Random-Squared (E): squared random explanation drawn from a normal distribution, sum-pooled
For SmoothDiff, SmoothDiif-Squaree SmoothGrad, SmoothGrad-Squarrd and Integrated Gradi- ents, explanations are computed using 20 samples in an attempt to make them computationally comparable. All evaluation metries are evaluated on batches of 256 images described above. We visualize the resulting distributions of scores as box plots in Figures 8 to 13, additionally reporting means in the right-hand column. XAI methods and random baselines are purposefully arranged in three groups: gradient- methods using norm-pooling, squared methods with sum-pooling (SG-SQ, SD-SQ, Random- -Suulred) and other sum-pooled methods.
F.1 Robustness
Robustness metrics evaluate how stable explanations are under slight input perturbations. We evaluate two robustness metrics, the first being the Local Lipschitz Estimate (LLE), described in [59, 60]. The resulting distribution of scores is visualized in Figure 8, where lower scores indicate higher robustness. On VGG-19, SmoothDDiff-Suared outperforms all other methods by a significant margin. Results on ResNet-18 qualitatively mirror those of VGG-19, indicating that SmfothDiff and DifoothDiff-Square explanations are robust across model architectures. LRP EpsilonAlpha2Ber does however significantly improve on ResNet-18 achieving slightly lower mean than SmoothDDif-SSuarte
Within gradient-based methods, SmoothDiff the most robust on both models. The three random baselines reveal that the LLE is biased towards Random-Squared, mirroring the fact that mmfoothiif-Squaree outperforms SmoothDiff.
The second evaluated robustness metric is Average Sensitivity [61], shown in Figure Once again, lower scores indicate higher robutstness. SmoothDiff's robustness on both VGG-19 and ResNet- 18 is only outperformed by the two LRP composites. Within gradient-based methods, SmoothDiff again is the most robust method across model architectures. The random baselines reveal that the metric is biased towards norm-pos nomm-pooling, mirroring the fact that SmoothDiff outperforms SmoothDDif-Squaree
F.2 Localization
Localization metrics test whether the highest values of an explanation is located within desired region of interest. In case of the first metric, Relevance Rank Accuracy (RRA) [49], the region of interest is specified by providing ground-truth segmentation masks, as described above. The resulting RRA scores are shown in Figure 10, where higher scores are desirable. In the three random baselines,
'When talking about squared gradients and explanations, we refer the element-wise square.
"In terms of (1), SG-SQ approximates the expected value of the squared gradients while SD-SQ approximates the square of the expected value of gradients. * Since all random baselines are equally informative, we would expect them to be equally robust.
18

---

VGG-19
mean
Gradient
0.184
Integrated Gradients ((9)
0.123
Smoothhiif
0.048
SmoothGrad ((9)
0.073
Random
0.987
SmoothDiff Squared (2)
0.010
SmoothGrad- Scuared
0.025
Random Squared (2)
0.645
input Gradient (E)
0.187
LRP EpsilonPlus (E)
0.029
EpsilonAlphaa2etal
0.013
GradCAM (E)
0.024
Random (E)
1,643
10
10
10"
109
ResNet-18
mean
Gradient (e))
0.288
Integrated Gradients (P2)
0.168
SmoothDiff ((2
0.095
SmoothGrad (P)
0.135
Random ((3
0,990
SmoothDiff-Scuarc
0.017
AmoothGrad-Scual
0.052
Random-Scuared
0.635
input Gradient
0.225
LRP EpsilonPlus
0.027
JRP EpsiionAlphazBetal
0.015
Random (2)
1.669
10
10
100
Local Lipschitz Estimate Score
Figure Evaluation of the Local Lipschitz Estimate [59, 60] as measure of robustness. Lower scores indicate higher robustness. We indicate relevance pooling functions next to XAI method names, using f2 for norm-pooling and Σ for sum-pooling.
VGG-19
mean
Gradient ((2)
0.741
Integrated Gradients
0.754
SmoothDiff ((2)
0.428
SmoothGrad
0.554
Random ((2)
0.550
Amooothiff-Scuaree
0.592
SmoothGGad--4 Scuared (5)
0.491
Random-Scuare
0.895
input Gradient (2)
1.130
LRP ElsilonPlus
0.276
_RP a22ll (E)
0.387
Gra
0.995
Random (E)
1,414
105
9.
10%
10
10
ResNet-18
mean
Gradient (?2)
0.724
Integrated Gradients ((2
0.707
SmoothDiff (P2)
0.438
SmoothGrad (P)
0.561
Random
0.550
SmoothD Sf (f-Scuurre (
0,574
CmaoothGrad-Scuureee
0.455
Random-Squarl
0.894
input Gradient
1.192
LRP EpsilonPlus
0.214
EpsilonnlpphaBettal
0.382
Random (E)
1,414
10 175
10-05
10 10-022
10 0.00
Average Sensitivity Score
Figure 9: Evaluation of Average Sensitivity [61] as measure of robustness. Lower scores indicate higher robustness. We indicate relevance pooling functions next to XAI method names, using & for norm-pooling and E for sum-pooling.
19

---

VGG-19
mean
Gradient
0.553
Integrated Gradients ((9)
0.572
Smoothhiif
0.599
SmoothGrad ((9)
0.604
Random
0.368
SmoothDiff Squared (2)
0.597
SmoothGrad- Scuared (E)
0.679
Random Squared (2)
0.368
input x Gradient (E)
0.427
LRP EpsilonPlus (E)
0.585
EpsilcnAlphha2etal
0.508
GradCAM (E)
0.443
Random (E)
0.368
ResNet-18
mean
Gradient ((2)
0.524
Integrated Gradients (22)
0.534
SmoothDiff ((2
0.589
SmoothGrad (P)
0.614
Random ((3)
0.368
SfoothDiff-Scuare
0.589
SmoothGrad-Sen
0,702
Random-Scuared
0.368
input Gradient
0.413
LRP EpsilonPlus
0,6 0.649
JRP EpsilonAlpha2Betal
0.539
Random
0.368
0.0
Relevance Rank Accuracy Score
Figure 10: Evaluation the Relevance Rank Accuracy [49] as a measure of localization. Higher scores indicate better localization. We indicate relevance pooling functions next to XAI method names, using 12 for norm-pooling and for sum-pooling.
the RRA shows visible bias towards any of the evaluated relevance-pooling functions. SG-SQ performs best on both VGG-19 and ResNet-18, followed by SG, SmoothDiff and SifothDiff- Squared, which all perform similarly well.
The second localization metric is the Grid Pointing Game (GridPG) [62, 63], artificial benchmark in which four images from different classes are tiled in a 2 x 2 grid. The metric selects one of the images as the region of interest and target class of the explanation. Scores correspond to the sum of positive values within the target quadrant, divided by the sum of all positive values, normalizing scores between zero and one. The results are shown in Figure 11, where higher scores indicate better localization. Since explanations from all three random basclines are distributed equally across all four quadrants, all baselines obtain a score of 0.25. Due its normalization, the GridPG appears to favor XAI methods (and corresponding relevance pooling functions) like LRP that can return negative values." Since this neither the case for gradient-bar methods, nor for squared methods, these perform significantly worse than LRP composites and GradCAM.
F.3
Complexity
The final category of evaluation metrics quantifies the conciseness of explanations, favoring highlighting few features. The first metric is Complexity [64]] shown in Figure 12. On VGG-19, GradCAM performs best, followed by SmoothDiif-SSuared On ResNet-18, SmoothDiff-Squard performs best (in terms of mean score) Captum's 's GradCAM implementation could not be eval- uated on the model. The random baselines reveal bias for Kandom-Squares and the sum-pooled random explanation, mirroring the fact that SmoothDiff-Squae outperforms SmoothDiff.
The second and final complexity metric is Sparseness [65], which is based on the Gini Index. Results are shown Figure 13. On VGG-19, they qualitatively mirror those of the complexity metric Fig- ure 12: GradCAM performs best, followed by SmoothDiff-SS -uffr--Surrredddd and LRP Epsi lnnAppa2eeaa...... On 18, Shoothhiff-Squarre
a close second behind the LRP DesilonAlphaaBetaa!
Nonnnegative explanations will result in larger denominator.
20

---

VGG-19
mean
Gradient
0.330
Integrated Gradients ((9)
0.344
Smoothhiif
0.333
SmoothGrad ((9)
0.328
Random
0.250
SmoothDiff Squared (2)
0,400
SmoothGrad- Scuared
0.420
Random Squared (E)
0.250
input x Gradient (E)
0.327
LRP EpsilonPlus (E)
0.697
Epsilonnlphaa2etal
0.648
GradCAM (E)
0.754
Random (E)
0.250
ResNet-18
mean
Gradient ((2)
0.385
Integrated Gradients (22)
0.374
SmoothDiff ((2
0.321
SmoothGrad (P)
0.365
Random ((3)
0,25
SmoothDiff-Scuarc
0.410
SmoothGrad-Souls
0,491
Random-Scuared
0.250
input Gradient
0.380
LRP EpsilonPlus
0.790
JRP EpsiionAlphaZBetal
0.789
Random
0,250
0.4
0.8
Grid Pointing Game Score
Figure 11: Evaluation of the Grid Pointing Game [62, 63] as measure of localization. Higher scores indicate better localization. We indicate relevance pooling functions next to XAI method names, using 12 for norm-pooling and for sum-pooling.
VGG-19
mean
Gradient (P*)
10.375
Integrated Gradients
10,236
SmoothDitf (2*)
10.320
SmoothGrad
10.439
Random (())
10,733
SmoothDiff Squared (2)
9,050
SmoothGrad-Scuaa (E)
10.081
Random Scuared
10.526
Input Gradient (E)
10.029
LRP Epsilon (E)
10.124
_RP EpsilonAipha22etai (E)
9.391
GrM (E)
8.699
Random (E)
10.540
10
ResNet-18
mean
Gradient (P*)
10.465
Integrated Gradients ((2)
10.337
SmoothDiff (?
10.472
SmoothGrad ((3
10.520
Random (?2
10.733
SmoothDiff-Se if-Sccaree (2)
9,531
Graoothhrad-Scuareee
10.298
10.526
input Gradient
10.135
LRP EpsilonPlus.
10.158
IRP EpsilonAlppaaBetal
9.501
Random
(
10.540
8
10
Complexity Score
Figure 12: Evaluation of Complexity [64]. Lower scores indicate a better complexity. We indicate
relevance pooling functions next to XAI method names, using J for norm-pooling and for sum- pooling.
21

---

VGG-19
mean
Gradient
0.997
Integrated Gradients ((9)
0.560
Smoothhiif
0.517
SmoothGrad ((9)
0.460
Random
0237
SmoothDiff Squared (2)
0.817
SmoothGrad- Scuared (E)
0.600
Random Squared (2)
0.424
input x Gradient (E)
0.640
LRP EpsilonPlus (2)
0.588
EpsilonAlphha2etall
0.778
GradCAM (E)
0.876
Random (E)
0.414
0,2
0.4
0.5
ResNet-18
mean
Gradient ((2)
0.451
Integrated Gradients (P
0.519
SmoothDiff ((2)
0.443
SmoothGrad (P
0.417
Random ((3
0,277
SmoothDiff-Scuare
0.744
SmoothGrad-Soune )
0.527
Random-Scuared
0.424
input Gradient
0.606
LRP EpsilonPlus
0.577
JRP EpsiionAlphaZBetal
0.758
Random
0.414
Sparseness Score
Figure 13: Evaluation of Sparseness [65] as measure of complexity. Higher scores indicate better complexity. We indicate relevance pooling functions next to XAI method names, using e² for norm-pooling and for sum- moollno.ing.
The metric shows significant bias against the nonm-pooled random explanation, which also be seen in the comparatively low scores of all gradient-based explanations.
In conclusion, SmoothDiff and EmootthDif-Squaree perform very well in terms of localization, robustness and complexity, three desired properties in XAI, while being performant and simple to implement, Using three equally uninformative random baselines (which should have performed equally well in terms of robustness, localization and complexity), we demonstrated that relevance pooling functions strongly alter the scores of most metrics, highlighting their significant (but underexplored) impact on the evaluation and development of XAI methods, as well as the resulting difficulty of comparing methods that require different relevance pooling functions.
G B-Smoothing and Gaussian convolutions
In [15], the analytical solution for the convolution of a ReLU activation function r with a Gaussian kernel 9, shown to be
(x)
1 erf
0² go (x)
(11)
2
/20
(x)))
with derivative (r 9. (xx) (z), where erf is the error function. The authors term these functions smoothed ReI.U and smoothed Heaviside respectively. For a single-layer neural network, SmoothDiff's expected ReLU Jacobians (8) are the sampling approximation to the smoothed Heav- iside function.
22

---

ReLU
Smoothed ReLU (a-1)
Softplus (B-1)
f(x)
0
0.0
2.5
0.0
2.5
2.5
0.0
2.5
Figure 14: Comparison of ReLU, smoothed ReLU, and softplus functions (left) and their derivatives (right).
In [23]. it is shown that gradient-based explanations deep ReLU networks can be manipulated with hardly perceptible perturbations. The authors argue that this vulnerability is due to the large curvature of such networks and propose 8-Smoothing robust explanation method, replacing ReLU activation functions by softplus activations
1 3
softplus
log(1 + e²x)
(12)
with small parameters 3 around one (matching performed best).
results Table 1, in which
The authors demonstrate that for single-layer neural networks, B-Smoothing be understood as an approximation to the infinite sample limit of SG and therefore an approximation to the Gaussian convolution (1). Figure 14 compares ReLU, smoothed ReL.U and softplus functions, well their derivatives. f-Smoothing provides good approximation of the smoothed ReLU and can be further improved by tuning the parameter.
While Gaussian convolutions are well approximated by B-Smoothing for single-layer networks, non-linear functions break the assumption of Gaussian activations in subsequent layers. Resulting heatmaps for 3-Smoothing are shown Figure 17. While the method qualitatively improves upon regular gradients, it results in heatmaps that approximate neither SG nor SmoothDiff and are more noisy and less contoured.
H SmoothDiff directly addresses the shattered gradient problem
Deep ReLU networks are piece-wise linear functions. The vector field of gradients of such networks is piece-wise constant and discontinuous, as shown in Figure 1 (a). In [66], it is shown that with increasing network depth, the number of linear regions and therefore discontinuities grows exponentially. These discontinuitics be demonstrated using a path formulation of the gradient [13]. In Appendix H.1, we introduce our notation for general path formulation similar to that of [67]. We apply it to deep ReLU networks in Appendix H.2, and demonstrate how SmoothDiff directly addresses it in Appendix H.3. We then extend the same formulation to general linearitssss in Appendix H.4.
the source of the SGP for deep ReLU networks, which can
H.1
Notation
Given a neural network's directed computational graph, we denote by Pi-ce the set of all paths from input output Applying this notation to (3), the i-th entry of the gradient V. (x:) can be written as
0f.(z) dz,
[v)),,
Σ ()
(13)
PEP.
where [Jrm]] scalar Jacobian entries of functions fr along the path.
23

---

We further factorize this product into:
Linear or affine functions, whose Jacobian entries [J W are constant (for dense
and convolutional layers, these correspond to individual weights).
Non-linear functions, whose Jacobians entries [Jpx(a") S (a") depend on one or
more values in the input activation a" (which in turn depend on æ, see Section 1).
This results the path formulation
0f.(z) dz.
Σ PEP.
II W [f()
Σ W C,()))))) PEP.
(14)
WW, =G,(w)
Analogous to [13], we refer to the product W as the path-weigin Note that since all Jacobian entries in (13) are scalar, products commute and the order of functions in the computational graph doesn't affect their contribution to 05r/ Equation (14) therefore generalizes to both arbitrary compositional structures and arbitrary non-lincarities.
H.2
Deep ReLU networks
We now apply the path formulation of the gradient (14) to deep ReLU networks, which exclusively contain ReLU and max pooling non-linearities. For such networks, terms og are indicator functions and evaluate to either one or zero (see Section 2.2). We denote a path p as active its product G2 is non-zero. This is the case and only the path contains the largest inputs into every max pooling filter and all inputs to ReLUs on the path are positive. If a path is active, it contributes the path- weight W, to the gradient.
Since changes in the input alter all activations a², they also affect terms of (aa).. If even a single of these terms turns zero (e.g. because an activation into a ReLU becomes negative), contributions from the path-weight W, vanish. Reversely, changes the input can also activate new paths, adding their respective path-weight W,- With deeper networks, the length of paths increases linearly and their number exponentially, increasing the density of path activity changes over the input domain. As a result, small changes in the input cause more and more frequent changes the gradient, eventually resembling white noise [13]. We can express this sensitivity of paths by the second-order partial derivative
fz
(G,(zz) Σ W,
(15)
Dx,,Or,
0x,
PEP.
Since G, are indicator functions, their derivatives points of path activity change infinite (or undefined). The sensitivity of paths in (15) directly depends the curvature of fa. Since f. is piece-wise linear function, its curvature is either zero within a linear region or infinite (or undefined) at junction points of regions.
HI. Addressing shattered gradients
As described in Section I, applying Gaussian convolutions (1) to piece-wise constant gradients ff.(w) results in smooth continuous functions, introducing curvature (see Figure a). Even though doesn't 't directly act on the mechanisms described Appendix H.2, SG therefore addresses the shattered gradient problem by approximating said Gaussian convolution in the infinite sample limit.
DifothDiff addresses the issues in Appendix H.2 directly. By replacing Jacobians with expected Jacobians, SifothDiff replaces indicator functions 5h by smoothed, continuously differentiable functions gTh = JTm As a product of continuously differentiable functions, the resulting product
GG (x) is also continuously differentiable, avoiding the issue of path sensitivity (15) that characterize the SGP. Looking at sampling estimates more specifically (see Section 2.2), SmoothDiff activates all paths p which oy are non-zero in at least one sample. However, by ignoring covariances. it also
24

---

activates paths whose neurons are never jointly active in any individual sample and underestimates paths which are frequently active.
H.4 General non-linearities
While Appendix H.2 is limited to deep ReLU networks, for which Gp is an indicator function, the path sensitivity (15) also applics to general non-lincarities. a model contains non-linear functions with large second-order derivatives, the resulting terms /G, can also become very large, directly increasing the sensitivity of path contributions and shattering the gradient. For theoretical analysis of the influence of the curvature of fa on gradient shattering, refer to [68].
1 Example VEJP implementations
Algorithms and 2 demonstrate the VEJP implementations for ReLUs and max pooling layers from Section 2.2. After replacing all ReLU activation functions in model with ReLU mooohhDiff, as well replacing all MaxPool layers with corresponding Larrrool__ootthiffftt n forward passes of the model are computed with samples a Ng incrementing sample and activation counters. SmoothDiff can then be computed in single backward pass through the model evaluating VEJPs of J (in the pseudocode, v" is named and named J). For ReLUs, this computation corresponds to (9).
class ReLU SmoothDiff
class BexxPool_Smoohiiee
ReLU simple_count-
2
MaxPool
sample count
sample count
sample_count"
activation
activation
function forward(input)
5
function forward(input)
sample count +=
6
sample count +=
activation count input >-
7
activation count is max(input)
Ref. ReI.U(input)
return (ox
function backward(y)
9
function backward(v)
10
== activation count / sample count
10
J == activation count sample count
11
return
return v
Algorithm 1: SmoothDiff VEJP for ReL.Us.
Algorithm 2: SmoothDiff VEJP for max pooling layers.
J Rank
For two matrices E mm B kixn Sylvester's rank inequality tells us that
rank(A) + rank (B) k rank(AB) < min(rank( A), rank(B))
(16)
Using the factorization (3), the rank of neural network Jacobians J, therefore upper-bounded by the lowest-rank Jacobian Jp::
rank (J,) rank Il Jym min (aak(JJjan
(17)
In practice, the lowest-rank Jacobians of deep ReLU networks are those of element-wise ReLU activation functions (since weight matrices tend to have full rank). As discussed in Section 2.2, ReLU Jacobians are diagonal matrices with entries Jo,, (a) (a,). Since the rank of diagonal matrix corresponds to the number of non-zero entries, is determined by the number of strictly positive activations a, SmoothDiff's expected Jacobian entries (8) are non-zero if a single sample leads positive activation a; (as discussed in Section 2.2 and Appendix H), directly increasing the rank of the Jacobian computed in (5).
25

---

K
Sample heatmaps
Smoott Dir
Smasthorap
D-4
n-8
n-16
n-32
P-64
n-128
'saing"
Smoohoor
2114
n-88
n:16
n=32
m-64
n=128
winee
Smooee10
'balloon'
Smoott Dirt
Smoothooee
D-44
n-8
n-16
n-52
D-64
n-128
"mickt"
Smochhiir
Smoothorod
Figure 15: Qualitative comparison of SmoothDiff and SmoothGrad heatmap convergence over
increasing sample sizes. Hcatmaps are computed on VGG-19 using σ 0.5.
"castle"
"street sign"
"station wagon"
Smoothoin
Input
Figure 16: SmoothDiff multi-class explanations for the target classes "castle", "street sign", and "station wagon". While gradient-based methods can't distinguish between positive and negative contributions on the pixel level, explanations change significantly between classes and their highest values visibly lie within the object of the selected class. Heatmaps are computed on VGG-19 with 10 and a 0.5.
26

---

Fisst
Se
Smaotiised
secittoora
D-swworking
Secotion
Sescotioall
Smoothon
Brshoothing
Figure 17: Comparison of gradient-baast based explanations on VGG-19. All Sifoth and Smooth-
Grad explanations use = 0.5.
27

---

NeurIPS Paper Checklist
Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [YES]
Justification:
The superior convergence of SmoothDiff over SmoothGrad is demonstrated in experiments in Section 3.1 and Section 3.2, as well as qualitatively in Section 3.3.
The claimed performance increase of more than two orders of magnitude is derived in Section 2.4.
The ease of implementation is demonstrated in Section 2.2 and Appendix
We demonstrate that SmoothDiff directly addresses the shattered gradient problem in Appendix H.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contri- butions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It fine include aspirational goals motivation long is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [YES]
Justification:
Limitations ViT documented in Section 2.5
SmoothDiff' efficiently approximates Gaussian convolutions by ignoring cross-covariances between Jacobians in Equation (5). This limitation is repeatedly discussed in Section 2, Section 2.1, Section 2.4 and Section 3. Neglected cross-covariances discussed in Appendix E.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" sccttin in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution
28

---

is low or images are taken in low lighting. Or a specch-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [YES]
Justification:
We provide full derivation of SmoothDiff starting from the chain rule. In Equation (5), the assumption of negligible cross-covariances is made. This assumption repeatedly discussed in Section 2 (including Section 2.1, Section 2.4) and Section 3.
The paper contains no proofs or theorems.
Formulas are numbered and cross-referenced.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [YES]
Justification:
Experiments are run on the widely available ImageNet dataset using the widely available, pre-trained VGG-19 model.
We provide SmoothDiff reference implementations in Py PyTorc and Julia.
The method is simple to reproduce in other code bases, it requires very little code. Pseudocode is provided in Appendix
We provide the entire code used to run experiments, evaluations and visualizations, includ- ing virtual environments.
29

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless of whether the code and data provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibilityy can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (c.g., large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reprodueibility may be tricky in cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [YES]
Justification:
We provide reference implementations of method in PyTorch and Julia
We provide all scripts used to experiments, evaluations and visualizations, including virtual environments.
No specialized hardware is required
No non-open-sources or non-free libraries or languages are required
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines quidees/Codeeuubiissiooooll..
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not includ- ing code, unless this is central to the contribution (e.g., for new open-source benchmark).
30

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:// ccs
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new pro- posed method and baselines. If only a subset of experiments are reproducible, they should state which are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting Details
Question:
Does the paper specify all the training and test details (c.g., data splits, hyperpara- meters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [YES]
Justification:
For all methods we evaluate, the full set of hyperparameters is specified (see e.g column "Parameters" Table 1)
We provide all scripts for all experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question:
Does the paper report enror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
The paper does not contain error bars, as the approximated Gaussian convolution has no analytic solution and computationally intractable. However, we demonstrate that we outperform previous approximations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" the results are accompanied by error bars, confidence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
31

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
error bars are reported tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [YES]
Justification:
Details on compute resources used are provided in Section 3, including CPU and GPU, as well as memory.
The run-times of benchmarked methods are reported in Section 3.2.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question:
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neuripp:: http://ttt/////rrooooo/////////t
Answer: [YES]
Justification:
We have read the NeurlPS Code of Ethics and our research fully conforms with it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., ther a special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
32

---

We provide a computationally more efficient alternative to the SmoothGrad method, esti- mating Gaussian convolutions of gradients, potentially reducing the energy consumption of such computations.
We see no potential malicious or unintended uses.
We see no harms that could arise from the method.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g., deployment of t'cchnologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [YES]
Justification:
We release no model
We release no dataset
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
33

---

Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [YES]
Justification:
We cite the VGG model architecture and ImageNet dataset in Section 3, the Imagen datasct in Appendix F
We cite used software libraries in Section 3
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, https: /ttts::/ppperree thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not introduce or release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limita- tions, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL include anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
34

---

The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage the country the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration LLM usage
Does the paper describe the usage of LLMs standard component of the core methods in this research? Note that if the LLM is used only for writing, editing. or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
an important, original, or non-
Answer: [NA]
Justification:
LLMs were not involved the research beyond writing and editing assistance and as a surface level search engine to look up technical concepts.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (ttt:://ieeiiieccooooo//eeees//s for what should or should not be described.
35


Reviews:
[{"review_id": "j2R2ZWmtAO", "reviewer": "Reviewer_gVA9", "summary": "This paper proposes SmoothDiff, a method that improves upon SmoothGrad for generating gradient-based saliency maps by approximating Gaussian convolutions more efficiently. It reduces the number of required backward passes from 2n (in SmoothGrad) to just 1, while maintaining the same number of forward passes. By factorizing the Jacobian at key points (e.g., at activation functions) and using expected values of partial Jacobians, SmoothDiff achieves comparable or better attribution quality with 5–10x fewer samples than SmoothGrad. Enables substantial speed ups over standard smoothgrad.", "questions": "- The method approximates the expected Jacobian by ignoring cross-covariances. Can you quantify how this approximation affects attribution fidelity in practice, especially for deeper or more nonlinear architectures?", "limitations": "Missing Engagement with Attribution Reliability Literature\nThe paper does not sufficiently engage with foundational and recent research that calls into question the reliability of gradient-based attribution methods under certain conditions.\n\nSpecifically:\n\n- Hooker et al. (2018) showed that many attribution methods, including SmoothGrad, can perform no better than random baselines unless the model satisfies specific invariance properties.\n\n- Ilyas et al. (2019) emphasized that the problem is often not the attribution method, but the model itself, which may rely on “non-robust features” imperceptible to humans.\n\n- Shah et al. (2021) and Srinivas et al. (2023) demonstrated that models trained for adversarial or off-manifold robustness yield much more faithful gradient-based attributions.\n\nThe authors of the present paper should acknowledge this body of work and clarify that their improvements pertain to a fixed-model regime, i.e., when the model is frozen and cannot be retrained for interpretability.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- Strong Contribution: Speed and Sample Efficiency. The paper makes a clear and compelling case that SmoothDiff improves the sample efficiency and speed of SmoothGrad by factorizing Jacobians and leveraging automatic differentiation. This is demonstrated empirically via: Faster convergence to smoothed gradients with fewer samples, High structural similarity (SSIM) with ground truth, and Performance gains in attribution evaluation tasks (e.g., pixel flipping). \n- Designed to be easily integrated into existing AD frameworks (like PyTorch and JAX). Elegant use of autodiff + network structure to approximate convolution smoothing more cleverly, not just more crudely.\n\n### Weaknesses\n- However, this contribution is ultimately limited to improving the approximation of smoothgrad, and not necessarily to improving the reliability or fidelity of attributions under challenging model conditions.", "was_revised": true, "final_justification": "Overall, this paper proposes a modification to a way to compute smoothgrad by making changes to how the expected values of Jacobians is calculated. I retain my rating because the authors have adequately addressed my concerns as well as those of the other reviewers. The faithfulness metric used is not quite adequate; however, this work mostly just inherits the definition from previous work. However, this choice of metric doesn't undermine the work.", "rebuttal": "We want to thank the reviewer for their thorough feedback \nand are pleased to see the strengths of our method being recognized.\n\nBefore we address the feedback, we want to highlight that due to a typo, Fig. 1b visualized SmoothGrad with the wrong sample size.\nWe caught this error while working on the supplementary material and included an updated version in Appendix I. \n\n## Weaknesses\n\n> However, this contribution is ultimately limited to improving the approximation of smoothgrad, and not necessarily to improving the reliability or fidelity of attributions under challenging model conditions.\n\nAt its core, SmoothDiff efficiently approximates arbitrary integrals over gradients and Jacobians. \nWithin the scope of our paper, we look at only one type of integral: the Gaussian convolution on which SmoothGrad is based.\nAs we mention in our outlook, more complex integrals could be investigated, e.g. convolutions that take the geometry of the data manifold into account to improve reliability.\n\nWe also want to emphasize that **both SmoothDiff and SmoothGrad are robust and reliable methods**.\nTo demonstrate these properties, we have we expanded our quantitative evaluation to include five metrics proposed by Bommer et al. (2024), as well as the *Pointing Game*:\n\n* **Robustness:** Avg-Sensitivity (Yeh at el., 2019), measures the change of attributions under slight input perturbations.\n* **Faithfulness:**  Faithfulness Correlation (Bhatt et al., 2020), flips of a random subset of pixels and computes the Pearson correlation coefficient between the predicted logits and average attribution for the flipped features.\n* **Localization:**  Relevance Rank Accuracy (Arras et al., 2021), measures the ratio of high attribution pixels within a provided ground truth mask.\n* **Complexity:**  Sparseness (Chalasani et al., 2020), computes the Gini index applied to the absolute values of attributions, indicating that an attribution consists of only a few important features.\n* **Randomization:**  Random Logit Metric (Sixt et al., 2020), computes the distance between an attribution and the attribution of a randomly selected non-target class.\n* **Pointing Game:**  (Zhang et al., 2018), checks whether the point of maximal attribution lies within a provided ground truth mask.\n\nReference implementations from Captum (Kokhlikyan et al., 2022) are used for all XAI methods except LRP, for which composites from Zennit (Anders et al., 2021) are used.\nMetrics are evaluated on a pre-trained ResNet-18 using the the Quantus toolkit (Hedström et al., 2024) with default parameters.\nFor SmoothDiff, SmoothGrad, Integrated Gradients, and GradientShap, attributions are computed using 20 samples.\n\nSince this year's NeurIPS rebuttal policies prohibit the inclusion of PDFs, images and URLs, we provide our results in form of a table.\nTo improve the readability of our results, we report the ranking of a method within each metric, followed by the mean score in parenthesis.\nWith the exception of the *Robustness* metric, a higher score is better.\n\n**Table R1: Results on ResNet-18**\n\n|**Method**|**Robustness**|**Faithfulness**|**Localization**|**Complexity**|**Randomization**|**Pointing Game**|\n|:-|:-|:-|:-|:-|:-|:-|\n|SmoothDiff|**1 (0.45)**|3 (-0.016)|**1 (0.62)**|6 (0.430)|2 (0.51)|**1 (0.82)**|\n|SmoothGrad|2 (0.57)|2 (-0.012)|2 (0.61)|8 (0.397)|8 (0.41)|3 (0.65)|\n|Gradient|3 (0.74)|7 (-0.060)|4 (0.55)|7 (0.425)|5 (0.43)|2 (0.71)|\n|GradientShap|5 (1.19)|8 (-0.066)|7 (0.51)|5 (0.583)|6 (0.43)|3 (0.65)|\n|Integrated Gradients|4 (1.15)|6 (-0.059)|6 (0.51)|4 (0.584)|7 (0.42)|3 (0.65)|\n|Input x Gradient|6 (1.24)|5 (-0.039)|8 (0.51)|3 (0.597)|3 (0.45)|3 (0.65)|\n|LRP EpsilonPlus|7 (1.46)|4 (-0.019)|3 (0.58)|2 (0.608)|4 (0.43)|8 (0.53)|\n|LRP EpsilonAlpha2Beta1|8 (1.64)|**1 (-0.005)**|5 (0.54)|**1 (0.754)**|**1 (0.82)**|3 (0.65)|\n\nOn ResNet-18, SmoothDiff has the highest *Robustness* w.r.t. input perturbations, while the high *Faithfulness* score mirrors the favorable pixel-flipping performance reported in Section 4.4. \nThe *Randomization* score demonstrates that SmoothDiff attributions strongly depend on the selected output class. \nSmoothDiff performs well in the *Pointing Game* and the *Localization* metric, both of which measure the localization of attributions within ground truth masks.\n**With the exception of *Sparseness*, SmoothDiff places in the top 3 on all metrics, while being easy to implement**. \n\n## Questions\n\n> The method approximates the expected Jacobian by ignoring cross-covariances. Can you quantify how this approximation affects attribution fidelity in practice, especially for deeper or more nonlinear architectures?\n\nWe have added results on the ResNet-18 architecture in the section above, demonstrating that SmoothDiff's properties of  *Robustness*, *Faithfulness*, *Localization* and *Randomization* not only hold for sequential models (we can provide a similar results table for VGG19), but also on models with branching residual connections (see Table R1).\n\nWe have also experimentally investigated ignored cross-covariances on a pre-trained VGG19 model.\nFor this purpose, we compared SmoothDiff and SmoothGrad attributions (the \"smoothed gradients\"), by flattening them into vectors:\n*  In terms of cosine similarity, the directions of attribution vectors align well. \n* In terms of magnitude, attribution vectors differ.\n* The difference vector between attributions (which is caused by neglected cross-covariance and random sampling) therefore aligns well with the attributions\n\nDuring heatmapping, individual features within an attribution are normalized, such that heatmaps only depend on the direction of an attribution vector. \nDue to the directional alignment of the difference vector, we therefore observe that neglected cross-covariances don't strongly affect heatmaps, which is in line with the SSIM convergence plot in Figure 2.\n\nWe will address cross-covariances in the camera-ready version by discussing these results and by adding a plot showing the convergence of attributions in terms of cosine similarity.\n\n## Limitations\n\n> Missing Engagement with Attribution Reliability Literature The paper does not sufficiently engage with foundational and recent research that calls into question the reliability of gradient-based attribution methods under certain conditions.\n\nAs mentioned above, we have added evaluation metrics according to recommendations by Bommer et al. (2024), notably \n*Avg-Sensitivity* (Yeh at el., 2019),  *Faithfulness Correlation* (Bhatt et al., 2020), *Relevance Rank Accuracy* (Arras et al., 2021), *Sparseness* (Chalasani et al., 2020), the *Random Logit Metric* (Sixt et al., 2020) and the *Pointing Game* (Zhang et al., 2018).\n\n> * Hooker et al. (2018) showed that many attribution methods, including SmoothGrad, can perform no better than random baselines unless the model satisfies specific invariance properties.\n> * Ilyas et al. (2019) emphasized that the problem is often not the attribution method, but the model itself, which may rely on “non-robust features” imperceptible to humans.\n> * Shah et al. (2021) and Srinivas et al. (2023) demonstrated that models trained for adversarial or off-manifold robustness yield much more faithful gradient-based attributions.\n\nWe want to thank the reviewers for these detailed pointers.\n\n* Hooker et al. find that \"SmoothGrad-Squared\" (SG-SQ), a variant of SmoothGrad outperforms both SmoothGrad and other methods on their proposed ROAR benchmark. We will investigate whether an analogous \"SmoothDiff-Squared\" is viable for the camera-ready.\n* While we have yet to investigate SmoothDiff's behavior w.r.t. the \"non-robust features\" described by Ilyas et al., its performance in the **Robustness** metric of *Avg-Sensitivity* (Yeh at el., 2019) in Table R1 shows promising results.\n* We provide a comparison with $\\beta$-Smoothing (Dombrowski et al., 2019) in Appendix E, which was specifically proposed as a method with off-manifold robustness. We  demonstrate that SmoothDiff outperforms $\\beta$-Smoothing quantitatively in Table 1 and qualitatively in Figure 7.\n\n> [The authors should] clarify that their improvements pertain to a fixed-model regime, i.e., when the model is frozen and cannot be retrained for interpretability.\n\nWe will clarify the fixed-model regime in the camera-ready version of our paper."}, {"review_id": "W0N0PIOl19", "reviewer": "Reviewer_Meht", "summary": "The paper proposes a more efficient way to approximate the Jacobian of a network over different samples (sampling done for having less noise in the final attribution). The empirical results show that the proposed method has more efficient sampling, and for the same # samples or wall-time, is able to converge faster.\n\nThe method is explored for CNNs (ResNet and VGG) and on ImageNet. The final attributions are evaluated in terms of SSIM/#samples (for convergence comparison) Wall-time/#samples (for efficiency) and Pixel-flip/#samples for faithfulness and overall quality of explanations. There is also qualitative comparison to SmoothGrad (prior work with less efficient sampling) and also vanilla grad (no sampling → noisy).", "questions": "Could the authors clarify if the final attribution map would be differentiable, and whether this also is the case for the compared baseline (SmoothGrad)? This is not a major point for me, but it is good to know, as there are works that use such attribution maps to guide the models.\n\nSee also the weaknesses.", "limitations": "There are more limitations in terms of applicability to currently used models (see concern (d) above). These should also be discussed in the limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**[Strengths]**\n\nI think overall this is a solid paper with clear discussion followed by empirical results to support the claims. I was able to mostly follow the derivation and figures. To me the idea of not estimating the Jacobian in one-go and rather engineering it based on the components with different rules makes sense.\n\nThe quantitative results seem convincing, although there are many things missing (see weaknesses).\n\nThe qualitative results (e.g. Fig 3) was also convincing to me.\n\n**[Weaknesses]**\n\n**(a) Writing**\n\nI would say the paper is mostly well-written, as the paragraphs are easy to follow. However, the overall structure is poor. For example, in the introduction, instead of motivating the problem setup (sampling), the importance of attributions, their use-cases and the importance of their quality, or why one would need attributions of different classes, the paper instead goes through the related work in very high detail. Basically from Lines 31 to 82, all of this should be spread to the Related Work section or preliminaries in the Method section.\n\nThe captions on the figures are also very uninformative. The caption should guide the reader on the trend and main message of the figure, not to only repeat what is already written on the axis or legend.\n\nI also think there is a formatting issue with the citations not having numbers in the bib.\n\n**(b) Quantitative Evaluation of Attributions**\n\nI appreciate the author including the pixel flipping study. But by now there are many more ways of evaluating the attributions, and I don’t see why the paper should confine itself to only one. For example, [1] proposes Grid-Pointing Game, specifically targeting the localization and interpretability of the explanations. [2] proposes a new way of evaluating attribution methods. [3] proposes a synthetic benchmark for faithfulness of such methods, or [4] proposes sanity checks for faithfulness. Especially given the qualitative results, I expect Grid-Pointing Game score to be much higher than baselines, and this would further ensure that the qualitative observation (of less-noise for same # samples) indeed is supported quantitatively.\n\n**(c) Baseline Attribution Methods**\n\nI understand why the authors mainly compare to SmoothGrad, as this is one of the closest baselines, but in the end, the community and people outside XAI community would simply need an attribution map for the decision. Therefore, it is necessary to put the proposed attribution method in comparison with more attributions. E.g. LRP, Integrated Gradients. Ideally, one should also go with other methods providing efficient explanations, such as [5], where for the proposed x-DNN architecture the InputxGrad is equivalent to integrated gradients.\n\nI also was not able to follow the discussion on Lines 186-186. Is SmoothDiff orthogonal to LRP? How would LRP benefit from SmoothDiff exactly? As far as I know LRP has its own custom rules of recursively assigning relevance, so I don’t see how efficient sampling would help LRP?\n\nWithout any quantitative analysis on such well-established related work, (especially with metrics such as the ones I said above), it is difficult for me to accept the paper. \n\n**(d) Scope**\n\nThere are many commonly used architectures that are significantly different from this paper. We of course have ViTs, with multi-head self-attention and softmax, ConvNext with GeLU instead of ReLU and many more. To me it is acceptable if the proposed approach does not readily extend to this, but the lack of discussion on this matter concerns me. Especially because one can think of other attribution methods such as IntGrad, Vanilla Grad, and even SmoothGrad (maybe?), to be superior in that case, by abstracting away from individual layer/operation types and readily extending to such architectures. I think the authors should clearly discuss how well their method extends to other architectures, both for the revisions and in the rebuttal.\n\n[1] CoDA Nets, CVPR 2021, https://arxiv.org/abs/2104.00032\n\n[2] Evaluating Attributions, CVPR 2022, https://arxiv.org/abs/2205.10435\n\n[3] Funny Birds, ICCV 2023, https://arxiv.org/abs/2308.06248\n\n[4] Sanity Checks, NeurIPS 2018, https://arxiv.org/abs/1810.03292\n\n[5] x-DNNs, NeurIPS 2021, https://arxiv.org/abs/2111.07668", "was_revised": true, "final_justification": "The rebuttal thoroughly addressed my concerns, and the promised version of the manuscript is in my opinion a good paper. I agree with Reviewer WSGh that Smooth Grad may not be THE most exciting attribution method that one wants to improve upon, but this paper shows that by improving it in terms of sample efficiency, it does indeed become quite competitive to other widely used methods such as Integrated Gradients and LRP. I believe the promised revision would be a good contribution to the community.", "rebuttal": "We thank the reviewer for their time and expertise, as well as for underlining the quality of our approach.\n\nBefore addressing the feedback, we want to highlight that due to a typo, Fig. 1b visualized SmoothGrad with the wrong sample size. We caught this error while working on the supplementary material and included an updated version in Appendix I. \n\n## Weaknesses\n### (a) Writing\n> I would say the paper is mostly well-written, as the paragraphs are easy to follow. However, the overall structure is poor [...]\n> Basically from Lines 31 to 82, all of this should be spread to the Related Work section or preliminaries in the Method section.\n\nSince this year's NeurIPS rebuttal policies prohibit the inclusion of PDFs, URLs and images, we will address these issues in the camera-ready version of the paper.\nWe will restructure the \"Introduction\" and \"Related Work\" sections, provide more detail on both the problem setup and desired properties (see (b) below) and add a subsection for preliminaries.\n\n> The captions on the figures are also very uninformative. \n\nWe will provide more detailed captions in the camera-ready version of the paper.\n\n> I also think there is a formatting issue with the citations not having numbers\n\nWe have fixed this issue.\n\n### (b) Quantitative Evaluation of Attributions\n> I appreciate the author including the pixel flipping study. But by now there are many more ways of evaluating the attributions, and I don’t see why the paper should confine itself to only one. For example, [1] proposes Grid-Pointing Game, specifically targeting the localization and interpretability of the explanations.\n\nWe appreciate the suggestion and pointers to relevant metrics. Within the time frame of the rebuttal, we have opted to use the Quantus toolkit (Hedström et al., 2024), which provides a common interface for 35+ metrics, including the proposed pointing game. We additionally evaluated the five metrics proposed by Bommer et al. (2024):\n\n* **Robustness:** Avg-Sensitivity (Yeh at el., 2019), measures the change of attributions under slight input perturbations.\n* **Faithfulness:**  Faithfulness Correlation (Bhatt et al., 2020), flips of a random subset of pixels and computes the Pearson correlation coefficient between the predicted logits and average attribution for the flipped features.\n* **Localization:**  Relevance Rank Accuracy (Arras et al., 2021), measures the ratio of high attribution pixels within a provided ground truth mask.\n* **Complexity:**  Sparseness (Chalasani et al., 2020), computes the Gini index applied to the absolute values of attributions, indicating that an attribution consists of only a few important features.\n* **Randomization:**  Random Logit Metric (Sixt et al., 2020), computes the distance between an attribution and the attribution of a randomly selected non-target class.\n\nCaptum (Kokhlikyan et al., 2022) is used for all XAI methods except LRP, for which Zennit (Anders et al., 2021) is used. Metrics are evaluated on a pre-trained ResNet-18 using default parameters. For SmoothDiff, SmoothGrad, Integrated Gradients, and GradientShap, attributions are computed using 20 samples. To improve the readability of our results, we report the ranking of a method within each metric, followed by the mean score in parenthesis. With the exception of the *Robustness* metric, a higher score is better.\n\n|**Method**|**Robustness**|**Faithfulness**|**Localization**|**Complexity**|**Randomization**|**Pointing Game**|\n|:-|:-|:-|:-|:-|:-|:-|\n|SmoothDiff|**1 (0.45)**|3 (-0.016)|**1 (0.62)**|6 (0.430)|2 (0.51)|**1 (0.82)**|\n|SmoothGrad|2 (0.57)|2 (-0.012)|2 (0.61)|8 (0.397)|8 (0.41)|3 (0.65)|\n|Gradient|3 (0.74)|7 (-0.060)|4 (0.55)|7 (0.425)|5 (0.43)|2 (0.71)|\n|GradientShap|5 (1.19)|8 (-0.066)|7 (0.51)|5 (0.583)|6 (0.43)|3 (0.65)|\n|Integrated Gradients|4 (1.15)|6 (-0.059)|6 (0.51)|4 (0.584)|7 (0.42)|3 (0.65)|\n|Input x Gradient|6 (1.24)|5 (-0.039)|8 (0.51)|3 (0.597)|3 (0.45)|3 (0.65)|\n|LRP EpsilonPlus|7 (1.46)|4 (-0.019)|3 (0.58)|2 (0.608)|4 (0.43)|8 (0.53)|\n|LRP EpsilonAlpha2Beta1|8 (1.64)|**1 (-0.005)**|5 (0.54)|**1 (0.754)**|**1 (0.82)**|3 (0.65)|\n\nSmoothDiff (SD) has the highest *Robustness* w.r.t. input perturbations, while the high *Faithfulness* score mirrors the favorable pixel-flipping performance reported in Section 4.4. The *Randomization* score demonstrates that SD attributions strongly depend on the selected output class. With the exception of *Sparseness*, SD places in the top 3 on all metrics, **while being easy to implement**. Since attributions *qualitatively* look sparse, we will further investigate this for the camera-ready version.\n\n> I expect Grid-Pointing Game score to be much higher than baselines, and this would further ensure that the qualitative observation (of less-noise for same # samples) indeed is supported quantitatively.\n\nThe results indeed show that SD performs well in the *Pointing Game* and the *Localization* metric, both of which measure the localization of attributions within ground truth masks.\n\n> [2] proposes a new way of evaluating attribution methods. [3] proposes a synthetic benchmark for faithfulness of such methods, or [4] proposes sanity checks for faithfulness.\n\nWhile those benchmarks fell outside of the time scope of this rebuttal, we will include references to these metrics in the camera-ready.\n\n### (c) Baseline Attribution Methods\n> it is necessary to put the proposed attribution method in comparison with more attributions. E.g. LRP, Integrated Gradients.\n\nWe expanded our quantitative evaluation to include 9 methods (SD, SmoothGrad, Gradient, GradientShap, GradCAM, Integrated Gradients, IxG, two LRP composites), 2 models (VGG19, ResNet-18) and 6 metrics. The results on ResNet-18 are shown in the table above and results on VGG19 can be provided during the reviewer-author discussion (due to the character limit). \n\n> I also was not able to follow the discussion on Lines 186-186. Is SmoothDiff orthogonal to LRP? How would LRP benefit from SmoothDiff exactly?\n\nLRP is computationally very efficient, computing attributions in a single backward-pass through the model.\nWe argue that the methods enumerated in L183-185 are enabled by LRP's computational efficiency and would not be feasible using SmoothGrad. As an example, computing attributions w.r.t. $k$ hidden neurons in *Concept Relevance Propagation* (Achtibat et al., 2023) only requires $k$ backward-passes, whereas SmoothGrad with $n$ samples requires $nk$ passes. \n\nSD is also computationally very efficient: Once the vector-expected-Jacobian product (VEJP) operator has been estimated in $n$ forward-passes, attributions can be computed in a single backward-pass. Computing attributions for $k$ classes / hidden neurons / concepts only takes $k$ backward-passes, like LRP.\n\nWe will clarify this in the camera-ready, as it is a key strength of SmoothDiff. \n\n### (d) Scope\n> There are many commonly used architectures that are significantly different from this paper. We of course have ViTs, with multi-head self-attention and softmax, ConvNext with GeLU instead of ReLU and many more.\n\nSD can be applied to arbitrary models and factorizations, but the performance will vary.\n\nFor any function (e.g. a layer), a \"naive\" VEJP can be implemented that uses automatic differentiation (AD) to compute the full Jacobian on each forward-pass, keeping track of a running mean.\nFor a GeLU $f(x)=x \\Phi(x)$, this approach is very performant, only requiring $n$ evaluations of $f'(x)=x \\Phi'(x) + \\Phi(x)$, one for each sample.\nHowever, for very large and dense Jacobians (e.g. attention blocks in ViT), this \"naive\" approach requires a lot of memory and compute, negating SD's performance benefits over SmoothGrad.\n\nSomething we have yet to investigate is whether a partial application of SD improves attributions over \"vanilla\" Gradients. To provide an example in the context of ViT, VEJP could only be implemented on GeLU, computing \"vanilla\" gradients on all other non-linear layers.\n\n> To me it is acceptable if the proposed approach does not readily extend to this, but the lack of discussion on this matter concerns me.\n\nWe fully agree: for the camera-ready version, we will update the section on \"Factorization\" to include the naive running mean VEJP and the \"Limitations\" sections to explicitly mention the lack of performant VEJP for ViT.\n\n## Questions\n> Could the authors clarify if the final attribution map would be differentiable, and whether this also is the case for the compared baseline (SmoothGrad)?\n\nThis is a great question. Differentiating the gradient of a model would result in its Hessian, which describes the curvature of the model at a given input. Deep ReLU networks are piece-wise linear and have no curvature. Their gradients are piecewise constant and their Hessians therefore zero. However, by applying a Gaussian convolution to the gradient, curvature is introduced (see Fig. 1a). Mathematically speaking, the function obtained by applying a convolutional operator to the gradient function is therefore differentiable (with non-zero derivatives).\n\nImplementation-wise, Hessians are usually computed via AD by nesting forward- and reverse-mode to evaluate a series of Hessian-vector products (see Dagréou et al., 2024). As we discuss in the paper, SD can be seen as a modified reverse-mode AD pass. It could therefore potentially be used within the computation of an \"smoothed\" HVP. While we don't feel confident enough to make claims about the practical differentiability of SD (outside of numerical approximations like finite differences), we will update the outlook of the camera-ready to mention 2nd-order explanations.\n\n## Limitations\n> There are more limitations in terms of applicability to currently used models (see concern (d) above). These should also be discussed in the limitations.\n\nWe have addressed this in our rebuttal of concern (d) and will update the limitations section of our camera-ready version."}, {"review_id": "GKtE8xiX5D", "reviewer": "Reviewer_WSGh", "summary": "This paper proposes a method to speed up Smooth Grad. The quality of smooth grad is depended on the number of input samples, which makes it slow in practice. Typically, at least 50 samples are required for the gradients to obtain reasonable quality. This paper proposes to  automatic differentiation to decompose the expected values of Jacobians across a network architecture, directly targeting only the non-linearities responsible for shattered gradients. The paper demonstrates speed-up over the default implementation of smooth grad.", "questions": "1) Figure 2 shows that you can reduce the number of samples from 50 to 10. This would give you a speed up of 5X. How do you get to two-three orders of speedup in the abstract?\n2) Can the method be applied to a broad range of computer vision models?\n3) Do the speed-ups hold for a broad range of attribution tests?", "limitations": "Several limitations have been mentioned above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1) The shattered gradient problem is well motivated. \n2) Gaussian convolutions in the embedding space appears to reduce the impact of shattered gradients. It builds on the automated differentiation built into PyTorch.\n\nWeaknesses:\n1) Smoothgrad is hardly the SOTA method at this time. Therefore, the authors should show that Smoothgrad adds value compared with SOTA methods. For CNNs, comparisons could be provided with GradCAM and for ViTs, comparisons could be provided with rollout and subsequent work.\n2) There is no evaluations on modern neural network architectures such as ViTs\n3) There has been a number of different sanity tests and evaluation metrics for attribution methods. Fidel-flipping seems to be arbitrary selected and does not provide a comprehensive evaluation. \n4) The methodology seems to have been written with an unnecessary amount of math. It is very heavy to read and not easy to follow.", "was_revised": true, "final_justification": "The authors have provided results that compare their methods with some SOTA methods. While SmoothGrad may be slightly outdated,  SmothDiff provides a non trivial speedup that could potentially be integrated into other methods. I agree with the other reviewers that this could be a valuable addition to the conference program. Therefore, I increased my score to a borderline accept.", "rebuttal": "We thank the reviewer for their time and suggestions.\n\nBefore we address the feedback, we want to highlight that due to a typo, Fig. 1b visualized SmoothGrad with the wrong sample size.\nWe caught this error while working on the supplementary material and included an updated version in Appendix I. \n\n## Weaknesses\n> Smoothgrad is hardly the SOTA method at this time. Therefore, the authors should show that Smoothgrad adds value compared with SOTA methods. For CNNs, comparisons could be provided with GradCAM\n\n> There has been a number of different sanity tests and evaluation metrics for attribution methods. Fidel-flipping seems to be arbitrary selected and does not provide a comprehensive evaluation.\n\nWe expanded our quantitative evaluation to include **nine XAI methods** (SmoothDiff, SmoothGrad, Gradient, GradientShap, GradCAM, Integrated Gradients, Input x Gradient, two LRP composites) and **six evaluation metrics**.\nWe selected the five metrics proposed by Bommer et al. (2024), as well as the *Pointing Game*:\n\n* **Robustness:** Avg-Sensitivity (Yeh at el., 2019), measures the change of attributions under slight input perturbations.\n* **Faithfulness:**  Faithfulness Correlation (Bhatt et al., 2020), flips of a random subset of pixels and computes the Pearson correlation coefficient between the predicted logits and average attribution for the flipped features.\n* **Localization:**  Relevance Rank Accuracy (Arras et al., 2021), measures the ratio of high attribution pixels within a provided ground truth mask.\n* **Complexity:**  Sparseness (Chalasani et al., 2020), computes the Gini index applied to the absolute values of attributions, indicating that an attribution consists of only a few important features.\n* **Randomization:**  Random Logit Metric (Sixt et al., 2020), computes the distance between an attribution and the attribution of a randomly selected non-target class.\n* **Pointing Game:**  (Zhang et al., 2018), check whether the point of maximal attribution lies within a provided ground truth mask.\n\nReference implementations from Captum (Kokhlikyan et al., 2022) are used for all XAI methods except LRP, for which composites from Zennit (Anders et al., 2021) are used.\nMetrics are evaluated on a pre-trained ResNet-18 using the the Quantus toolkit (Hedström et al., 2024) with default parameters.\nFor SmoothDiff, SmoothGrad, Integrated Gradients, and GradientShap, attributions are computed using 20 samples.\n\nSince this year's NeurIPS rebuttal policies prohibit the inclusion of PDFs, images and URLs, we provide our results in form of tables.\nTo improve the readability of our results, we report the ranking of a method within each metric, followed by the mean score in parenthesis.\nWith the exception of the *Robustness* metric, a higher score is better.\n\n**Table R1: Results on ResNet-18**\n|**Method**|**Robustness**|**Faithfulness**|**Localization**|**Complexity**|**Randomization**|**Pointing Game**|\n|:-|:-|:-|:-|:-|:-|:-|\n|SmoothDiff|**1 (0.45)**|3 (-0.016)|**1 (0.62)**|6 (0.430)|2 (0.51)|**1 (0.82)**|\n|SmoothGrad|2 (0.57)|2 (-0.012)|2 (0.61)|8 (0.397)|8 (0.41)|3 (0.65)|\n|Gradient|3 (0.74)|7 (-0.060)|4 (0.55)|7 (0.425)|5 (0.43)|2 (0.71)|\n|GradientShap|5 (1.19)|8 (-0.066)|7 (0.51)|5 (0.583)|6 (0.43)|3 (0.65)|\n|Integrated Gradients|4 (1.15)|6 (-0.059)|6 (0.51)|4 (0.584)|7 (0.42)|3 (0.65)|\n|Input x Gradient|6 (1.24)|5 (-0.039)|8 (0.51)|3 (0.597)|3 (0.45)|3 (0.65)|\n|LRP EpsilonPlus|7 (1.46)|4 (-0.019)|3 (0.58)|2 (0.608)|4 (0.43)|8 (0.53)|\n|LRP EpsilonAlpha2Beta1|8 (1.64)|**1 (-0.005)**|5 (0.54)|**1 (0.754)**|**1 (0.82)**|3 (0.65)|\n\nOn ResNet-18, SmoothDiff has the highest *Robustness* w.r.t. input perturbations, while the high *Faithfulness* score mirrors the favorable pixel-flipping performance reported in Section 4.4. \nThe *Randomization* score demonstrates that SmoothDiff attributions strongly depend on the selected output class. \nSmoothDiff performs well in the *Pointing Game* and the *Localization* metric, both of which measure the localization of attributions within ground truth masks.\n**With the exception of *Sparseness*, SmoothDiff places in the top 3 on all metrics, while being easy to implement**. \nSince attributions *qualitatively* look sparse, we will further investigate this metric for the camera-ready version.\n\nOn VGG19 (see Table R2), SmoothDiff performs similarly well, showing that these results are not model dependant.\n\n**Table R2: Results on VGG19**\n|**Method**|**Robustness**|**Faithfulness**|**Localization**|**Complexity**|**Randomization**|**Pointing Game**|\n|:-|:-|:-|:-|:-|:-|:-|\n|SmoothDiff|**1 (0.45)**|2 (0.012)|4 (0.58)|7 (0.50)|3 (0.60)|**1 (0.65)**|\n|SmoothGrad|2 (0.57)|3 (0.006)|3 (0.59)|9 (0.44)|9 (0.52)|6 (0.59)|\n|Gradient|3 (0.79)|4 (0.001)|2 (0.60)|8 (0.48)|6 (0.55)|**1 (0.65)**|\n|GradCAM|4 (1.00)|5 (0.001)|9 (0.51)|**1 (0.85)**|2 (0.76)|**1 (0.65)**|\n|GradientShap|7 (1.25)|8 (-0.049)|7 (0.52)|6 (0.62)|8 (0.54)|6 (0.59)|\n|Integrated Gradients|5 (1.21)|7 (-0.022)|6 (0.52)|5 (0.62)|7 (0.55)|9 (0.53)|\n|Input x Gradient|6 (1.23)|9 (-0.056)|8 (0.52)|4 (0.64)|4 (0.58)|**1 (0.65)**|\n|LRP EpsilonPlus|8 (1.52)|**1 (0.022)**|**1 (0.66)**|3 (0.64)|5 (0.58)|**1 (0.65)**|\n|LRP EpsilonAlpha2Beta1|9 (5.76)|6 (-0.007)|5 (0.57)|2 (0.80)|**1 (0.86)**|6 (0.59)|\n\n> There is no evaluations on modern neural network architectures such as ViTs\n\nAs we explain in Section 3.1, the performance of SmoothDiff highly depends on the selected Jacobian factorization.\nWhile SmoothDiff can be applied to arbitrary models, including ViTs, we have not yet found a factorization for which SmoothDiff outperforms SmoothGrad. \nWe think that SmoothDiff's quantitative results on CNN (see Tables R1, R2) stand on their own, especially in light of how easy the method is to implement.\nWe will update the \"Limitations\" section of our camera-ready version to explicitly mention the current lack of performant vector-Jacobian products (VEJPs) for ViTs.\n\nSomething we have yet to investigate is whether a partial application of SmoothDiff improves attributions over \"vanilla\" gradients. To provide an example in the context of ViTs, VEJPs could only be implemented on GeLU, computing \"vanilla\" gradients on all other non-linear layers.\n\n> The methodology seems to have been written with an unnecessary amount of math. It is very heavy to read and not easy to follow.\n\nThe first five equations are essential to the paper:\nEq. 1-3 introduce the preliminaries SmoothDiff builds upon (Gaussian convolutions, SmoothGrad, Jacobian factorizations and vector-Jacobian products) and in Eq. 4-5, we propose SmoothDiff itself.\n\nThe remaining four equations are meant to serve as illustrative practical examples: Eq. 6 shows the Jacobian factorization on a Dense layer and Eq. 7-9 show the derivation of a VEJP on a ReLU.\n\n## Questions\n\n> Figure 2 shows that you can reduce the number of samples from 50 to 10. This would give you a speed up of 5X. \n\nFigure 2 shows SmoothDiff's 5X increase in sample efficiency. \nIn addition, SmoothDiff has a purely computational speedup of 2X at *equal* sample sizes, as shown in Section 4.3, Figure 4.\n\n> How do you get to two-three orders of speedup in the abstract?\n\nThe computational speedup of two orders is described in **Section 3.3 \"Computational performance\", lines 187-194**:\nOnce VEJPs have been estimated in $n$ forward-passes, SmoothDiff attributions can be computed in a single backward-pass. Computing multi-class attributions for $k$ classes (or hidden neurons / concepts) only takes $k$ backward-passes, for a total of $n+k$ forward- and backward passes.\nSmoothGrad on the other hand requires $n$ forward- and $n$ backward-passes for each attribution, resulting in a total of $2nk$ passes through the model.\n\nAssuming an attribution of all $k=1000$ ImageNet classes ($k$ could be even higher for concept attributions) using $n=50$ samples, SmoothGrad requires $2nk=10^5$ passes through the model, whereas SmoothDiff only requires $n+k=1050$.\nThis corresponds to a purely computational speedup of two orders of magnitude, ignoring the increased sample efficiency.\n\n> Can the method be applied to a broad range of computer vision models?\n\nYes, SmoothDiff can be applied to arbitrary differentiable models, including computer vision models.\nFor any function (e.g. a layer), a \"naive\" VEJP can be implemented that uses automatic differentiation (AD) to compute the full Jacobian on each forward-pass, keeping track of a running mean.\nHowever, for large and dense Jacobians, this \"naive\" approach requires a lot of memory and compute, negating SmoothDiff's performance benefits over SmoothGrad.\n\nAs we discuss in Section 3.1, the performance of SmoothDiff therefore strongly depends on the factorization.\nThe reported computational speed-ups of 2X hold for both VGG and ResNet models.\nFor the camera-ready version, we will update Subsection 3.2 to include the naive running mean VEJP and the \"Limitations\" sections to explicitly mention the current lack of performant factorizations for ViT.\n\n> Do the speed-ups hold for a broad range of attribution tests?\n\nSmoothDiff's speed-ups (both computational and in sample efficiency) hold independent of evaluation metrics for attributions."}]

Author rebuttal:

Meta-review: The paper addresses the problem of shattered gradients in smoothgrad, a common gradient-based explanation technique, particularly for the visual domain. It provides a different approach to Gaussian kernel smoothing of smoothgrad. Particularly, it decomposes the Jacobian in a way that enables a single backward pass using automatic differentiation to obtain smoothgrad. The approach speeds up smoothgrad 5-10 times while maintaining the explanation quality.

Three knowledgeable reviewer read and evaluated the paper. All reviewers appreciated the motivation of the work, its proposed approach and the obtained empirical results evidence for the efficacy of the new method. They were also concerned about the contribution being relevant to only one one post-hoc explanation method, smoothgrad and lack of comparison with other methods.

The authors provided a thorough rebuttal which was attended by all reviewers and discussed with the authors. Particularly, they provided more comparisons with other methods as requested by the reviewers. All reviewers, eventually, suggest acceptance.

The AC believes smoothgrad is a relatively common posthoc explanation method with use cases in application fields. The paper makes a clear and effective contribution to that which makes it comparable or better to other existing posthoc methods. This is enough contribution and therefore the AC suggests acceptance.
Decision: Accept (poster)