Title: Gradient strikes back: How filtering out high frequencies improves explanations
Venue: NeurIPS 2023
Accepted: no
Keywords: Fourier

Explainability

Attribution methods

Deep Learning

Computer Vision
Abstract: Recent years have witnessed an explosion in the development of novel prediction-based attribution methods, which have slowly been supplanting older gradient-based methods to explain the decisions of deep neural networks. However, it is still not clear why prediction-based methods outperform gradient-based ones.
Here, we start with an empirical observation: these two approaches yield attribution maps with very different power spectra, with gradient-based methods revealing more high-frequency content than prediction-based methods.
This observation raises multiple questions: What is the source of this high-frequency information, and does it truly reflect decisions made by the system? Lastly, why would the absence of high-frequency information in prediction-based methods yield better explainability scores along multiple metrics? We analyze the gradient of three representative visual classification models and observe that it contains noisy information emanating from high-frequencies. Furthermore, our analysis reveals that the operations used in Convolutional Neural Networks (CNNs) for downsampling appear to be a significant source of this high-frequency content -- suggesting aliasing as a possible underlying basis. We then apply an optimal low-pass filter for attribution maps and demonstrate that it improves gradient-based attribution methods. We show that (i) removing high-frequency noise yields significant improvements in the explainability scores obtained with gradient-based methods across multiple models -- leading to (ii) a novel ranking of state-of-the-art methods with gradient-based methods at the top. We believe that our results will spur renewed interest in simpler and computationally more efficient gradient-based methods for explainability.

Abstract

Recent years have witnessed an explosion in the development of novel predictionbased attribution methods, which have slowly been supplanting older gradientbased methods to explain the decisions of deep neural networks. However, it is still not clear why prediction-based methods outperform gradient-based ones. Here, we start with an empirical observation: these two approaches yield attribution maps with very different power spectra, with gradient-based methods revealing more high-frequency content than prediction-based methods. This observation raises multiple questions: What is the source of this high-frequency information, and does it truly reflect decisions made by the system? Lastly, why would the absence of high-frequency information in prediction-based methods yield better explainability scores along multiple metrics? We analyze the gradient of three representative visual classification models and observe that it contains noisy information emanating from high-frequencies. Furthermore, our analysis reveals that the operations used in Convolutional Neural Networks (CNNs) for downsampling appear to be a significant source of this high-frequency content - suggesting aliasing as a possible underlying basis. We then apply an optimal low-pass filter for attribution maps and demonstrate that it improves gradient-based attribution methods. We show that (i) removing high-frequency noise yields significant improvements in the explainability scores obtained with gradient-based methods across multiple models - leading to (ii) a novel ranking of state-of-the-art methods with gradient-based methods at the top. We believe that our results will spur renewed interest in simpler and computationally more efficient gradient-based methods for explainability.

1 Introduction 23

Explaining and interpreting the decision of AI architectures is an important area of research towards 24

enabling the development of more interpretable models. Explainability methods (XAI) aim to provide 25

insights into the strategies used by models to arrive at their decision. This is expected to lead to the 26

development of better models that are more accurate, robust, and better aligned with humans. 27

One of the first attribution methods proposed, 'Saliency' [1], consists of back-propagating a model's 28

decision back to an input image to highlight areas that most affected the final decision. The method 29

remains relatively simple and computationally efficient, but it is also known to be noisy and to lead to 30

attribution maps that are often hard to interpret. Multiple methods have been proposed since to try to 31

improve on these limitations. These methods fall broadly into two main classes. (i) Gradient-based 32

methods extend Saliency [1] by smoothing the resulting attribution maps [2-8]. However, these 33

so-called white-box methods require access to all the model's components, which is not always 34

Originalexplanation

Original

FORGrad

FORGradexplanation

spectrum

spectrum

Saliency

FORGrad:

Faithfulness: 0.46

Low-passfilteringwith

Faithfulness: 0.81

optimal cutoff o*

Grad. Input

Faithfulness:0.65

possible. Conversely, prediction-based methods, also called black-box methods [3, 9-11], alter the 35

input of the model to produce an explanation based on the resulting change in the output. Those 36

37

methods are computationally inefficient and are known to sometimes fail to capture all the diagnostic

38

information, but they currently lead to the best fidelity scores across all explainability methods.

Overall, there are dozens of attribution methods available but relatively little is understood about 39

what makes certain methods more accurate than others. 40

Here, we start with the observation made across multiple studies [12, 4, 13] that the attribution maps 41 derived with Saliency are very noisy. Generally, these maps highlight sparse pixel activations around 42 a region of interest, and they are often hard to interpret. Because Saliency is simply the gradient of 43 the score function with respect to the input, we suggest that the noise originates from the gradient 44 itself: in other words, because the gradient is noisy, the explanation provided by Saliency is also 45 noisy. To try to better understand the origin of this noise, we compare the Fourier power spectra of 46 gradient-based methods (including Saliency) against prediction-based methods and observe that they 47 differ quite markedly. We discern significant differences between the two classes of approaches, with 48 gradient-based methods returning higher frequency content and prediction-based methods returning 49 lower frequency content. In the remainder of this paper, we will show that: 50

51

The gradient is indeed noisy, and this noise is especially present in the high-frequencies.

52

53

54

55

56

57

58

59

We then look for the origin of these high frequencies in vision models. Our findings show that downsampling operations (via MaxPooling or strides) are the main sources of high frequencies, and training the model does not alleviate the issue.

We then propose to repair Saliency - as well as other gradient-based methods - by introducing FORGrad (FOurier Reparation of the Gradient). This method consists in estimating the optimal amount of high frequencies to remove per model to make gradient-based methods surpass the prediction-based family of attribution methods.

2 Related Work

Attribution methods for black-box models Various methods have been developed to compute 60 importance scores for individual pixels or groups of pixels. For black-box (prediction-based) attribu61 tion methods, the analytical form and potential internal states of the model are unknown. The first 62 method, Occlusion [3], masks individual image regions, one at a time, using an occluding mask set to 63 a baseline value. The corresponding prediction scores are assigned to all pixels within the occluded 64 region, providing an easily interpretable explanation. However, occlusion fails to account for the 65 joint (higher-order) interactions between multiple image regions. For instance, occluding two image 66

regions individually may only have a minimal impact on the model's prediction, such as removing a 67

single eye or mouth component from a face. However, occluding these two regions together may lead 68

to a substantial change in the model's prediction if these regions interact non-linearly, as expected 69 in a deep neural network. Sobol [10], along with related methods such as LIME [11] and RISE [9], 70 address this problem by randomly perturbing multiple regions of the input image simultaneously. 71 Interestingly, recent studies, including RISE [9] and Sobol [10], have demonstrated that black-box 72 attribution methods can rival and even surpass the commonly used white-box methods without relying 73 on internal states. 74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

Attribution methods for white-box models The gradient-based methods, that we propose to improve here, were first introduced in [14] and improved in [2-4]. They consist in explaining the decisions of a model by back-propagating the gradient from the output to the input, indicating which pixels affect the decision score the most. However, this family of methods is limited because they focus on the influence of individual pixels in an infinitesimal neighborhood in the input image. For instance, it has been shown that gradients often vanish when the prediction score to be explained is near the maximum value [6]. Integrated Gradient [6] and SmoothGrad [5] partially address this issue by accumulating gradients. Another family of attribution methods relies on the neural network's activation, like CAM [7], which computes an attribution score based on a weighted sum of feature channel activities - right before the classification layer. GradCAM [8] extends CAM via the use of gradients, re-weighting each feature channel to take into account their importance for the predicted class. Nevertheless, the choice of the layer has a huge impact on the quality of the explanation. Our contribution proposes to overcome some of the mentioned issues by removing the noise present in the gradients in the form of high frequencies.

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

Fourier analysis of vision models Very little work has been proposed to analyze vision models and methods from a Fourier perspective. The closest, [15], used Fourier analysis to investigate the impact of DNNs optimization parameters and methods without a specific focus on vision. Additional work has focused on the analysis and development of adversarial attacks in the Fourier domain, [16, 17], while others [18-20] have proposed to defend against adversarial attacks by transforming the input image in the Fourier domain. Jo and Bengio [21] examined whether CNNs rely on high-level features by using Fourier-filtered images. None of the mentioned studies make a link between explainability and attribution methods with Fourier analysis.

3 Decomposing the gradient: An analysis of frequency content in attribution methods

Notations We consider a general supervised learning setting, where a classifier f : X → Y maps images from an input space X ⊆ R W × H to an output space Y ⊆ R . Let ( x 1 , . . . , x N ) be a set of images which contains N samples drawn from a probability distribution ∀ i ∈ { 1 · · · n } , x i ∼ D . Moreover, we respectively denote F and F -1 the Discrete Fourier Transform (DFT) on R W × H and its inverse. Therefore: ∀ x ∈ X , F ( x ) ∈ C W × H and ( F -1 ◦ F )( x ) = x . Additionally, when we visualize the Fourier spectrum, we always shift the low-frequency components to the center of the spectrum. We recall that an attribution method is a function φ : X → R W × H that maps an input of interest to its corresponding importance scores φ ( x ) . Finally, we denote by φ σ ( x ) the attribution method where high frequencies have been filtered using a cutoff value of σ .

Sobol

HSIC

RISE

Grad-CAM

Grad-CAM++

Occlusion

·Low

energy

Saliency

Grad.Input

SmoothGrad

Int.Gradients

SquareGrad

VarGrad

High

energy

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

3.1 Different signatures for different categories of methods

In this work, we analyze the Fourier signature of several attribution methods. To do so, we compute the feature map φ ( x ) for most existing attribution methods on representative models of the literature (ResNet50 in Figure 2). From these importance maps, we extract the corresponding amplitude of the Fourier spectrum, | ( F ◦ φ )( x ) | . In Figure 2, we show the average power spectra, over 1,000 images, for an array of methods. Upon visual inspection, it is obvious that certain methods tend to emphasize higher frequencies in their explanations, while others concentrate on lower frequencies. Interestingly, these differences can be traced to the class of methods: Black-box methods, which do not rely on gradients, exhibit frequency footprints dominated by very low frequencies, whereas white-box methods exhibit footprints that extend into higher frequencies. To quantify our observations, we employ two metrics to measure the complexity of the attribution maps. The first metric employs a Laplacian-based operator [22, 23] that evaluates the presence of high frequencies in images by analyzing their second derivative. The second metric involves measuring the file size of the image after undergoing lossless compression [24, 25], which we refer to as 'High-frequency content' throughout this study (as it can be seen as a loose approximation of Kolmogorov complexity).

Both metrics validate our visual observations, as depicted in Figure 3 (see Laplace quantity

in appendix). It is evident that black-box meth- ods (shown in dark in the figure) exhibit fewer

high frequencies compared to white-box meth- ods. This observation provides valuable insight

into where these methods extract information from the model to compute their explanations.

3.2 High-frequencies are just noise in the gradient

Naturally, gradient-based methods will be subject to the characteristics of the gradient itself. Consequently, when the gradient is subject to noise, the resulting explanation provided by such methods becomes similarly noisy. In light of this observation, we propose to demonstrate that the gradients obtained from three state-ofthe-art models (ResNet50 [26], ViT [27], and ConvNeXT [28]) do indeed contain noise, predominantly present in high-frequency components. To achieve this, we suggest an approach

Prediction-based methods1Gradient-basedmethods

10000

8000

1

Highfrequencies

6000

4000

2000

0

Sobol

GradCAMPP

HSIC

GradCAM

Occlusion

SquareGrad

VarGrad

Gradientlnput

Saliency

SmoothGrad

IntegratedGradients

Rise

ResNet50

ViT

ConvNeXT

101

error

101

101

=10

100

estimatione

=50

10-1

10-1

10-1

=120

2

10-3

10-2

10-3

=224

10-3

10-5

10-3

10-1

101

10-5

10-3

10-1

101

10-5

10-3

10-1

101

Radius ofε

143 that involves selectively removing high-frequency gradient information by employing various fre144 quency cutoffs σ . By computing residuals between f ( x + ε ) and its first order filtered by σ 145 decomposition around x that we denote by f ( x ) + ε ∇ σ x f ( x ) , we generate corresponding curves for 146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

different scales of ε , which are presented in Figure 4. As anticipated, our observations reveal that the curves exhibiting reduced high-frequency content (from σ < 224 to σ = 10) closely align with the one of the non-filtered gradient ( σ = 224). In other words, the gradient remains approximately as informative, even after removing high-frequency information. This implies that the high-frequency content primarily contains noisy information within the gradient.

3.3 Investigating the mechanisms introducing noise

Next, we investigated the underlying operations responsible for the introduction of such content by computing the power of high-frequency content in the gradients at the level of all the layers. Notably, we observed a consistent trend in CNNs where high-frequency content tends to increase and jump at each block, indicative of downsampling operation through strided convolutions or pooling. This observation aligns with the findings of [29, 30], suggesting that downsampling operations via MaxPooling or strided convolution can introduce noise. To verify this hypothesis, we substituted these specific operations in two representative CNNs, namely ResNet50 [26] (which incorporates strided convolutions and MaxPooling) and VGG16 [31] (details in appendix) with AveragePooling. This replacement ensured the preservation of information continuity in the gradient. The resulting plots for ResNet50 (VGG) are presented in Figures 5 - bottom curve (see appendix for VGG), displaying the power of high-frequency content using Kolmogorov image compression and Laplace-operator (see appendix) at each step. The depicted red shades represent the amount of high-frequency content in both models. We observe that prior to the initial dimension change, the quantity of high-frequency content remain comparable, suggesting that operations within a block of the same dimension does not significantly increase the power of high-frequency content. However, with the introduction of a downsampling layer, the curves for each model diverge, indicating a bigger contribution to the introduction of high frequencies by striding or MaxPooling compared to AveragePooling. Our findings corroborate the observations of [29], as the gradients (even averaged) following MaxPooling or strides exhibit checkerboard patterns, providing a plausible explanation for our quantitative observation of increased high-frequency content. We employ the same pipeline to calculate the high-frequency content for both a random model and a trained model, using the identical set of models including ViT [27]. The resulting curves are depicted in Figures 5 - top curve, for ResNet50 (see appendix for VGG16 and for ViT), showcasing that there is no discrepancy in high-frequency content between the trained and random CNNs. Given our previous section's demonstration that high-frequencies carry negligible information for the model, one would expect that training could potentially eliminate this content, leaving only relevant information to be processed. However, as our observations indicate the absence of such behavior despite the models accomplishing the task, we propose that the models were unable to adapt the gradient's content, thereby suggesting it to be an inherent by-product of downsampling operations. In the case of the ViT, however, training appears to introduce some high frequencies from the initial operation, potentially arising from transformers' pre-processing functions, such as image flattening via patches. These multiple findings suggest that high-frequency content emerges as a by-product of particular operations, predominantly observed in CNNs, which the models are unable to modulate during training. We therefore propose to consider most of the high-frequency content as noise. Consequently, when generating explanations for the models' decisions, it is justifiable to disregard high frequencies as they offer limited or negligible information.

3.4 FORGrad: a simple strategy to remove noise

An adapted σ ⋆ per model With FORGrad , we propose to remove high-frequency content, considered as noise, in order to obtain an optimal explanation related to the optimal frequency band from the gradient. We therefore propose to apply a low-pass filter on the Fourier spectrum of the gradient, employing multiple frequency cutoffs spaced evenly apart. For each filtered explanation, we compute the score from two different metrics. The first one Deletion - denoted D ( φ ( x ) [9] - is a measure of the decrease in the likelihood of a particular class as the important pixels (identified by the saliency map) are systematically removed from the image. If the likelihood of the class experiences a rapid decrease, resulting in a small area under the probability curve, this is a strong indication of a good explanation. Complementary, Insertion, I ( φ ( x )) [9] measures the significance of the pixels based on their capacity to create an image, and is calculated by measuring the increase in the probability of the class of interest as pixels are added in accordance with the generated importance map. Overall, we propose a heuristic to optimize our σ ⋆ , representing the ideal bandwidth maximizing the difference σ ⋆ = arg max σ E x D ( φ σ ( x )) -I ( φ σ ( x )) , combining the score of both metrics, on a subset of the validation set of ImageNet (1,000 images). Using both deletion and

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

Size:7

Size:14

fc,1000

avg ppov2

and2048channels

3convolutions,512

and 2048channels

3convolutions,512

and2048channels

3convolutions,512

and1024channels

3convolutions 256

and1024channels

3convolutions256

and 1024channels

3convolutions256

and512channels

3 convolutions,128

and512channels

3 convolutions,128

and512channels

3convolutions,128

256cbannels

3convolutions,64 and

256cbannels

3convolutions,64 and

256cbannels

3convolutions,64 and

7x7conv,64/2

....

104

ResNet-Trained

ResNet-Untrained

High frequencies

103

■

■

104

ResNet-AveragePooling

ResNet-MaxPooling+Strided convolutions

WLMM

103

....Downsampling

insertion metrics can provide a more comprehensive evaluation of the quality of the attribution map or saliency map generated for a given model. The deletion metric is useful for identifying important regions of an image that contribute to a model's decision, while the insertion metric is valuable for assessing the quality of the generated saliency map in terms of its ability to reconstruct the original image. By combining both metrics, we aim to assess the quality of the explanations generated by considering the impact of pixel removal and addition on the likelihood and significance of the target class, respectively. In the latter sections, we will consider the faithfulness metric to be the combination [Deletion-Insertion] Additionally, we also evaluate FORGrad on a third metric, MuFidelity, F ( φ ( x )) , [32]. The fidelity correlation metric serves to verify the correlation between the attribution score and a random subset of pixels. To achieve this, a set of pixels is randomly chosen and set to a baseline state, after which a prediction score is obtained. The fidelity correlation metric evaluates the correlation between the decrease in the score and the significance of the explanation for each random subset created.

Theoretical foundations In this section, we build on the empirically demonstrated assumption that the gradient is noisy and prove, through a Fourier perspective, that FORGrad effectively recovers the true gradient. Moreover, assuming that the noise is originally Gaussian, we characterize the distribution of the noise in Fourier space. Finally, we propose a convergence bound for SmoothGrad, valid on finite samples, showing that it also recovers the true gradient at the cost of multiple samplings.

We denote ∥·∥ F as the Frobenius norm and ∥·∥ 2 as the spectral norm. Note that ∥·∥ 2 ≤ ∥·∥ F in order to interpret our results. Finally, we define K σ ∈ { 0 , 1 } W × H as the binary Fourier mask, parameterized by σ , that we used to filter high frequency, where each element K σ ( i,j ) is determined as K σ ( i,j ) = 1 | i -W 2 |≤ σ 1 | j -H 2 |≤ σ, with 1 the indicator function abd ¯ K σ = 1 -K σ . As we have discussed above, the gradient of deep models is noisy, and in the following work, we consider that we only have access to ∇ x ˆ f ( x ) , a noisy estimator of ∇ x f ( x ) such that ∇ x ˆ f ( x ) = ∇ x f ( x ) + ε with ε ∈ R W × H . We do not assume any randomness for the noise so far. The following proposition develops the squared residual of the filtered noisy gradient as compared to the true one. Under the condition of finding the optimal filter, the gap between the two is naturally norm of the remaining noise post filtering.

Proposition 3.1. Let f : X → Y a predictor, and denote ∇ ̂ f = ∇ f + ε as the noisy gradient of f , with ε ∈ R W × H . For σ ∗ = inf { σ : ∥F ( ∇ f ) ⊙ ¯ K σ ∥ 2 F = 0 } , we have

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

where ⊙ is the Hadamard product, K σ ∗ a binary mask for low-pass filtering of frequency σ , and ¯ K σ ∗ is the opposite mask.

Remark 3.2 . This result holds as long as we find σ ∗ . There always exists a σ ∗ as the set always contains σ = max( H,W ) which does not alter the Fourier spectrum of an image of size W × H . However, finding σ ∗ poses a challenge, leading us to leverage XAI metrics as a heuristic.

With the information that the remaining gap between the filtered estimator and the true gradient is the remaining noise, of which the norm is upper bounded by the one of the original noise, we aim at measuring the reduction of the noise. In that way, we demonstrate the always-positive effect of FORGrad on gradient methods. In particular, under the assumption of Gaussian noise, we derive the distribution of the ratio ∥ ε ∥ 2 F / ∥F -1 ( F ( ε ) ⊙ K σ ∗ ) ∥ 2 F .

Proposition 3.3. Let the noise ε ∈ R W × H follow a normal distribution ε ∼ N (0 , ς ) ⊗ N . Then the norm of the Fourier spectra of the noise ∥F ( ε ) ∥ 2 F ∼ Γ( k = 2 WH,θ = ς 2 WH ) and filtered noise ∥F ( ε ) ⊙ K σ ∥ 2 F ∼ Γ( k = 8 σ 2 , θ = 4 ς 2 σ 2 ) follow Gamma distributions.

Therefore, the ratio of the two distributions R = ∥F ( ε ) ∥ 2 F / ∥F ( ε ) ⊙ K σ ∥ 2 F follows a Beta prime distribution R ∼ β ′ ( 2 N, 8 σ 2 , 1 , WH 4 σ 2 ) .

This result allows us to directly compute the distribution of the ratio of the norm of the original noise on the norm of the filtered noise (up to a scaling factor, by Parseval's Theorem). Naturally, this distribution depends on the parameter σ of the filtering. From this, we can deduce probabilistic results, such as, for σ = 10 and ς = 1 , the ratio of the norms is larger than 70 with probability almost one.

Finally, in the following proposition, we obtain a non-asymptotic result on the concentration of the SmoothGrad procedure to its expected value based on the Matrix Bernstein inequality [33].

Proposition 3.4. We recall that SmoothGrad is defined as SG = 1 n ∑ n i =1 ∇ x f ( x + δ i ) with ∀ i =1 ,...,n δ i ∈ N (0 , ς ) . Here we compute SG on the noisy estimate of the gradient ∇ x ˆ f ( x + δ i ) ∈ R W × H , which is then a random matrix ̂ SG . Assuming our predictor f ∈ L-Lip ( X ) is L -Lipschitz. We denote ∥·∥ 2 as the spectral norm, and define the variance as V ( SG ) = max ( ∥ E (( SG -E SG ) · ( SG -E SG ) T ) ∥ 2 , ∥ E (( SG -E SG ) T · ( SG -E SG )) ∥ 2 ) . We then have, for t > 0 ,

Our results suggest that in order to effectively eliminate noise using the SmoothGrad method, several iterations are required as opposed to ours. For instance, to be at least t = L 10 away from its expected value, with probability at most 0 . 01 , we need n ≈ 700 iterations, for ς = 1 . Furthermore, the noisy SmoothGrad gradually approaches the expected outcome of the non-noisy SmoothGrad. Additionally, SmoothGrad alleviate the noise but at the cost of employing Monte-Carlo sampling.

4 Gradient-based methods perform better and are more efficient

The new explanations are free from noise Figure 6 presents qualitative examples of corrected gradients obtained using the Gradient Input method [34] combined with FORGrad . As we analyze the different images, we observe that gradually removing high frequencies from the gradients has a notable impact on the resulting explanation. The initially noisy patterns transform into larger patches until the saliency map effectively highlights the key features that represent the object for categorization. However, it is crucial to consider the optimal value of σ ⋆ , as exceeding this threshold leads to the map spreading too widely and the explanation becoming less informative. This observation is further supported by the curve on the right, which demonstrates the evolution of the faithfulness score as σ changes. Prior to finding the optimal σ , the faithfulness score fluctuates around the initial value before gradually increasing to reach its optimal level. As expected, when all the information is removed (represented by the last point on the x-axis), the fidelity score drops to zero.

A new ranking of attribution methods We apply FORGrad on all the gradient-based attribution methods and report the scores in Table 1 for three models: ResNet50 [26], ViT [27] and ConvNeXT [28]. GradCAM methods can't be tested on ViT because they are based on convolution so are limited to CNNs. We can observe two notable findings. Firstly, it is rare to encounter cases where

=224

α=80

=20

0* = 10

=5

0.09

Faithfulness

0.08

0.07

0.06

0.05

224

168

112

56

Original

FORGrad

sigma

explanation

explanation

the scores after applying FORGrad are lower than the scores obtained before. In such instances, the 283 decrease in scores is typically observed in only one metric, either Deletion or Insertion. However, 284 since the other metric is optimized, the overall Faithfulness, as measured by [Deletion-Insertion], 285 remains at least as good as before. Secondly, even without explicitly optimizing the Fidelity metric, 286 we observe an improvement in this score across all methods and the three models analyzed. Fur287 thermore, after applying FORGrad , we observe that the scores of several gradient-based methods 288 surpass or at least match those of prediction-based methods. Notably, these gradient-based methods 289 offer the additional advantage of being significantly more computationally efficient, as evident from 290 the complexity column. In order to determine the best method for each model, we propose to 291 aggregate the scores from the three metrics to obtain a single global score for each method and model. 292 This resulting score, is denoted as I ( φ ( x )) + F ( φ ( x )) -D ( φ ( x )) , corresponding to the sum of 293 1-Deletion, Insertion and Fidelity score. Interestingly, in Table 2, we observe that the rankings change 294 when we incorporate FORGrad into the analysis. This shift leads to the inclusion of at least two 295 gradient-based methods among the top-5 for all three models. In the case of ResNet50, all five of the 296 top-performing methods are gradient-based, whereas only one of them occupied a position in the 297 previous ranking. Although some prediction-based methods, such as Sobol and HSIC , consistently 298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

demonstrate good performance, we demonstrate that gradient-based methods such as SmoothGrad and VarGrad now perform nearly as well, with the added advantage of computational efficiency.

5 Limitations

In our study, we have proposed to find an optimal σ value representing an ideal cutoff to improve explanations of gradient-based methods. However, we acknowledge that this optimal value is highly dependent on the dataset, perhaps more so than on the model itself. Furthermore, while we have chosen a single value that maximizes the scores across 1,000 images, it may be beneficial to use different values for individual images, but would increase the computational costs. We also optimize our value of σ only on 2 metrics, deletion and insertion. Even though it turns out to also increase the fidelity score, we could potentially obtain even better results by optimizing on this metric as well. It's however, once again, a very resource-consuming method that we chose to avoid. Furthermore, in our ranking computation, we combine metrics that do not precisely capture the same information. While Deletion and Insertion can be aggregated, particularly since we optimize the difference between them, it should be noted that Deletion, Insertion, and Fidelity are not directly comparable even if they range between 0 and 1. We have proposed one approach to integrate these metrics and derive a ranking based on the three scores. However, an alternative could involve producing separate rankings for each individual score. If we had followed this approach, the FORGrad methods would have emerged as the top-5 for both ResNet50 and ViT, according to MuFidelity.

6 Conclusion

This work started with an empirical observation: prediction-based and gradient-based methods exhibit distinct power spectra in their attribution maps - with gradient-based methods exhibiting higher power in the high frequencies compared to prediction-based methods. This led us to wonder whether the frequency content of model gradients is merely noisy information. We demonstrate that removing this content does not impair our ability to approximate the gradient and conclude that high frequencies predominantly carry non-essential information. We further conducted an in-depth analysis of gradient frequency content in CNNs across processing layers and found that downsampling operations, such as max pooling and striding, contribute to the introduction of high frequencies. This points to model aliasing as a likely cause of this high-frequency content. Interestingly, even with training, CNNs are unable to prevent this phenomenon. These results hence raise the question: Could high-frequencies be filtered out to improve the explanations derived from attribution methods? We design an optimal filter, σ ⋆ , and show that the filtering of attribution maps leads to significant improvements in the quality of the explanations. These improvements were most pronounced for gradient-based methods, which ended up approaching and sometimes even surpassing the much more compute-intensive prediction-based methods. Overall, our work leads to a surprising result that the almost forgotten gradient-based methods turn out to contain all the information needed to provide a faithful explanation of a model's decision and that they can be as interpretable as the newest methods. In future work, it would be worth exploring the influence of this noise on the model's performance and evaluating whether replacing certain operations that introduce noise could affect both the accuracy and robustness of the models. Furthermore, considering that many adversarial attacks are gradient-based and often exploit additive noise patterns, it is worth investigating whether these attacks target the noisy high-frequency content in the gradients and whether they might be prevented by using operations not introducing high-frequencies.

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

References

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop, Proceedings of the International Conference on Learning Representations (ICLR) , 2013.

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop Proceedings of the International Conference on Learning Representations (ICLR) , 2014.

Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Proceedings of the IEEE European Conference on Computer Vision (ECCV) , 2014.

Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In Workshop Proceedings of the International Conference on Learning Representations (ICLR) , 2014.

Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. In Workshop on Visualization for Deep Learning, Proceedings of the International Conference on Machine Learning (ICML) , 2017.

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the International Conference on Machine Learning (ICML) , 2017.

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2921-2929, 2016.

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2017.

Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. In Proceedings of the British Machine Vision Conference (BMVC) , 2018.

Thomas Fel, Rémi Cadène, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas Serre. Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis. Advances in Neural Information Processing Systems , 34:26005-26014, 2021.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?": Explaining the predictions of any classifier. In Knowledge Discovery and Data Mining (KDD) , 2016.

Beomsu Kim, Junghoon Seo, Seunghyeon Jeon, Jamyoung Koo, Jeongyeol Choe, and Taegyun Jeon. Why are saliency maps noisy? cause of and solution to noisy saliency maps. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) , pages 4149-4157. IEEE, 2019.

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision , pages 618-626, 2017.

David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller. How to explain individual classification decisions. The Journal of Machine Learning Research , 11: 1803-1831, 2010.

Zhiqin John Xu. Understanding training and generalization in deep learning by fourier analysis. arXiv preprint arXiv:1808.04295 , 2018.

Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective on model robustness in computer vision. Advances in Neural Information Processing Systems , 32, 2019.

Yusuke Tsuzuku and Issei Sato. On the structural sensitivity of deep convolutional networks to the directions of fourier basis functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 51-60, 2019.

Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv:1608.00853 , 2016.

Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117 , 2017.

Sibo Song, Yueru Chen, Ngai-Man Cheung, and C-C Jay Kuo. Defense against adversarial attacks with saak transform. arXiv preprint arXiv:1808.01785 , 2018.

Jason Jo and Yoshua Bengio. Measuring the tendency of cnns to learn surface statistical regularities. arXiv 393 preprint arXiv:1711.11561 , 2017. 394

Ramesh C. Jain, Rangachar Kasturi, and Brian G. Schunck. Machine vision. 1995. 395

Said Pertuz, Domenec Puig, and Miguel Angel Garcia. Analysis of focus measure operators for shape396 from-focus. Pattern Recognition , 46(5):1415-1432, 2013. 397

398

399

Hector Zenil, Jean-Paul Delahaye, and Cédric Gaucherel. Image characterization and classification by physical complexity. Complexity , 17(3):26-42, 2012.

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

Andrei N Kolmogorov. On tables of random numbers. Sankhy¯ a: The Indian Journal of Statistics, Series A , pages 369-376, 1963.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 11976-11986, June 2022.

Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill , 2017. doi: 10.23915/distill.00007. https://distill.pub/2017/feature-visualization.

Xueyan Zou, Fanyi Xiao, Zhiding Yu, Yuheng Li, and Yong Jae Lee. Delving deeper into anti-aliasing in convnets. International Journal of Computer Vision , 131(1):67-81, 2023.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Umang Bhatt, Adrian Weller, and José MF Moura. Evaluating and aggregating feature-based model explanations. arXiv preprint arXiv:2005.00631 , 2020.

Joel A Tropp et al. An introduction to matrix concentration inequalities. Foundations and Trends® in Machine Learning , 8(1-2):1-230, 2015.

Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradientbased attribution methods for deep neural networks. In Proceedings of the International Conference on Learning Representations (ICLR) , 2018.

Junghoon Seo, Jeongyeol Choe, Jamyoung Koo, Seunghyeon Jeon, Beomsu Kim, and Taegyun Jeon. Noise-adding methods of saliency map as series of higher order partial derivative. In Workshop on Human Interpretability in Machine Learning, Proceedings of the International Conference on Machine Learning (ICML) , 2018.

Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2018.

Paul Novello, Thomas Fel, and David Vigouroux. Making sense of dependence: Efficient black-box explanations using dependence measure. In Advances in Neural Information Processing Systems (NeurIPS) , 2022.

Thomas Fel, Remi Cadene, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas Serre. Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis. In Advances in Neural Information Processing Systems (NeurIPS) , 2021.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the issue of noisy gradient-based explanations caused by high-frequency content. The authors empirically identify that several down-sampling operations, such as MaxPooling or stride convolution, could be the primary source of these high frequencies. To address this, they present FORGrad, which removes high-frequency content in gradients by leveraging low-pass filter on the Fourier spectrum. The results on three faithfulness metrics further show the effectiveness of FORGrad in improving existing gradient-based explanation methods.\n\nSTRENGTHS:\n1. The paper is well-written, and the motivation is sound. \n2. The analysis regarding the high-frequency noises in gradients is interesting and could benefit the further designs of explanation methods.\n3. The proposed FORGrad achieves promising results on three faithfulness metrics, potentially improving existing gradient-based methods.\n\nWEAKNESSES:\n1. Throughout the paper, the term “High frequencies” is extensively employed to describe the phenomenon in gradients. However, how to clearly define “high” in actual deployment is not well explained. This could become especially problematic considering the significant variations in model architectures and datasets, thus degrading the significance of this work.\n2. The argument, “High frequencies are just noise in the gradient,” is too strong. The analysis in Section 3.2 can only show that the high-frequency information has less effect on model decisions. This does not necessarily mean that such high-frequency information is always noise. Moreover, the perturbation-based analysis also has many limitations, e.g., the OOD problem [1]. It’d be better if the authors could provide another set of experiments to validate their findings.\n3. It is nice to see that FORGrad could co-work with many gradient-based methods. However, in many cases, the improvement introduced by FORGrad is minor/marginal, e.g., Int.Grad* on ConvNeXT. It thus remains unclear whether FORGrad is genuinely effective or merely a result of randomness.\n4. The source of high-frequency gradients is not systematically investigated. The analysis in Section 3.3 only covers two mechanisms, i.e., downsampling and training. This narrow focus fails to provide substantial evidence for the claim that these mechanisms are the main reasons, as other relevant factors are not adequately discussed.\n\n[1] Peter Hase, Harry Xie, Mohit Bansal. The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations. NeurIPS 2021.\n\nQUESTIONS:\n1. Grad-CAM and Grad-CAM++ should be categorized as gradient-based methods?\n2. Does the parameter $\\sigma$ have a consistent value across all explanation methods, model architectures, and layers? If not, how can we determine the optimal value?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper analyzes the two types of attribution methods: gradient-based/ white-box methods and prediction-based/ black-box methods. The authors observe that the faithfulness of black-box methods surpasses the white-box methods despite white-box methods accessing the internals of the classifier being explained. The authors suspect the presence of noise in the gradients to be a contributing factor to its limited faithfulness. They further perform Fourier spectral analysis to examine the distribution of attribution signal frequencies. This analysis sheds light on the existence of high-frequency noise signals in the gradients, a by-product of pooling and stride operations, which are propagated as attribution signals. The obtained Fourier spectrum is filtered using a low-pass filter with an optimal cut-off threshold learned based on optimizing the Insertion and Deletion metrics, which are proxies for the faithfulness of the explanations. This yields a faithful explanation from the white-box methods surpassing the faithfulness that black-box methods achieved earlier.\n\nSTRENGTHS:\n* The methodology is effectively presented, incorporating suitable figures and accompanying text that elucidate the significance of these visuals. The paper follows a systematic approach in introducing concepts, starting with the motivation, followed by the hypothesis, supporting evidence, and ultimately presenting a solution to the problem. This coherent structure enhances the elegance of the methodical exposition.\n\n* The analysis of the gradient signals is sound, and the link between noise in the gradient signals and explainability is adequately established. \n\n* The solution’s strengths are its simplicity - performing a FFT and identifying a suitable frequency threshold; and modularity - can be integrated with many whitebox explainability approaches. It is a simple post-processing strategy to extract high fidelity explanations.\n\n* The experiments are somewhat comprehensive - with comparison against some of the prevalent explainability approaches.\n\nWEAKNESSES:\n- *Possible misrepresentation of the related literature:* Reference [3] is described as both a black-box method and a white-box method in different sections. This inconsistency is also reiterated in Line 76. Similarly, in Table 1, Grad CAM and Grad CAM++ are categorized as prediction-based/black-box methods despite their reliance on gradients. These methods employ gradient backpropagation (typically) up to the last convolution layer to generate the CAM, which contradicts the authors’ characterization as purely black-box approaches.\n\n- *Missing related literature and experiments:* The analysis of attribution-based explainability approaches overlooks significant contributions that introduce litmus tests for these methods, such as the work by Adebayo et al. NeurIPS 2018 and Sixt et al. ICML 2020. Sixt et al. explores the convergence of activations in saliency maps of gradient-based techniques when subjected to litmus tests proposed by Adebayo et al. It would be intriguing to investigate potential parallels between the convergence analysis presented by Sixt et al. and the spectral analysis conducted in the current paper. The application of Adebayo et al.'s litmus tests are necessary, and a comparative evaluation of the results before and after applying the proposed frequency cut-off indicating an increase in the faithfulness of the explanations could further strengthen the contribution. This is important considering the sweeping claim of the paper that gradient based attribution is reliable.\n\n- *Missing experiments:* While the authors’ have conducted quantitative evaluation to highlight the effectiveness of the solution, the absence of qualitative evaluation is stark. It is evident from the Figures that the attribution map generated looks ‘better’ after the frequency cutoff. However, whether this attribution map measures up against other approaches (both white box and black box approaches) qualitatively needs to be checked. Human subject experiments to this effect are necessary.\n \n- *Disconnected theoretical analysis:* The theoretical analysis seems disconnected from the empirical decisions in the current draft. The need for the theoretical analysis has to be well-motivated through a thorough rewrite. \n\n**Minor suggestions**\n- References [1] and [2] are just the same, with just a difference in the year. The author may need to check their bib source to remove the irrelevant bib entry and modify the paper just to retain the relevant entry.\n- Line 84 activations have been mistyped as activities. \n- Line 96 appears like a colloquial lingo. Please rewrite to make the statement formal to the research community. \n- Please check for possible typos in Line 244.\n\nQUESTIONS:\nplease see the weaknesses section\n\nLIMITATIONS:\nAn analysis of the limitations has been honestly presented", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper explores the differences between prediction-based and gradient-based attribution methods for explaining the decisions of deep neural networks. The authors observe that these two approaches produce attribution maps with distinct power spectra, with gradient-based methods exhibiting more high-frequency content. This discrepancy raises questions about the origin and relevance of high-frequency information and why its absence in prediction-based methods leads to better explainability scores. By analyzing the gradient of visual classification models, the authors identify downsampling operations in Convolutional Neural Networks (CNNs) as a significant source of high-frequency content, potentially due to aliasing. To address this issue, they propose applying an optimal low-pass filter to improve gradient-based attribution methods, leading to enhanced explainability scores.\n\nSTRENGTHS:\n1. `Simple yet Effective Solution`: One of the key strengths of the paper is its proposal of a straightforward yet highly effective solution. By applying a frequency cut low-pass filter to the attribution maps, the authors successfully remove the high-frequency noise and improve the interpretability of the results. This simple intervention offers a practical \"plug-and-play\" approach that can provide meaningful and understandable attributions for human interpretation. The demonstrated enhancements in explainability scores serve as compelling evidence of the efficacy of this approach, showcasing its practical value in real-world applications.\n\nWEAKNESSES:\n1. `Flaw in Observation`: The paper's strong observation regarding the higher frequency content in gradient-based methods compared to prediction-based methods may be misleading. This difference is primarily attributed to the **distinct instances that each method focuses on, rather than being inherent properties of the attribution categories themselves**. Prediction-based methods typically concentrate on `image patches, super-pixels, or high-level features of specific receptive fields`, whereas gradient-based methods calculate derivatives with respect to each pixel individually. If gradients were calculated with respect to patches, super-pixels, or features, the resulting attributions would also exhibit dominance in low-frequency components.\n\n2. `Naiveness in Theoretical Justification`: The paper's theoretical justification appears simplistic and lacks depth. It adopts a traditional signal processing approach, assuming the presence of noise with a specific form (such as uniform or Gaussian) and demonstrates the efficacy of a linear low-pass band cut filter. However, several critical questions remain unanswered. Firstly, the paper fails to address **why gradients with respect to the input are inherently noisy and exhibit high variance**. This noise can be attributed to the non-linearity and no-smoothness[A] of the network itself, and the downsampling operation might be just one contributing factor among others. Secondly, the derivation of the proposed solution seems to be a mere reproduction of standard textbook content on signal denoising, without providing any specific insights or considerations tailored to the problem at hand.\n\n3. `Methodological Simplicity`: The application of a low-pass filter to gradients may be considered too simplistic for publication in a prestigious conference like NeurIPS. Similar techniques, such as those presented in reference [5] and other variance reduction methods, have already been established in the field. The paper does not introduce any novel or sophisticated methodologies, which might limit its overall contribution and impact.\n\n[A] Wang Z, Wang H, Ramkumar S, et al. Smoothed geometry for robust attribution[J]. Advances in neural information processing systems, 2020, 33: 13623-13634.\n\nQUESTIONS:\nIf you can assume a Gaussian for the gradient noise and has closed-form solution if noise scale ($\\sigma$) is known, why still need to do hyper-parameter tuning in Figure 6? A more elegant solution should be estimate the scale and do it analytically.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates why gradient-based attributions contain more high-frequencies compared to prediction-based\nattribution. The authors argue that these high-frequency components are not relevant for the model and can\ntherefore seen as noise. Different saliency methods are analyzed on three different image models (ConvNeXT, ResNET, ViT). They further investigate the source of the noise and find that the downsampling-block is the the main cause for this.  Finally, the authors find that removing the noise from saliency maps yields an improved performance on\nvarious explanation metrics.\n\nSTRENGTHS:\n- In general, I think the paper is novel. The noise in attribution maps and its effect on interpretability metrics\nare not completly understood yet.\n- The paper analyzes the causes of the noise in detail\n- FORGrad makes it easy to interpolate between detailed and smooth attribution maps.\n- The focus of the paper is clear and generally well executed.\n\nWEAKNESSES:\n- **Missing relevant work**: The work does not discuss the findings of [(Balduzzi et al. 2017)](http://proceedings.mlr.press/v70/balduzzi17b/balduzzi17b.pdf). This work investigates the effect of Residual connections on the noise of neural networks. Although this work has a focus more on saliency maps, the findings\nof (Balduzzi et al. 2017) should be considered in the discussion.\n\n\n- **How relevant are high-frequencies?** : \"As anticipated, our observations reveal that the curves exhibiting reduced high-frequency content (from σ < 224 to σ = 10) closely align with the one of the non-filtered gradient (σ = 224).\" (line 148). However, the distance between the curves σ < 224 and σ = 10 are quite substantial in Figure 4.\nFor ConvNeXT, the distance is almost one magnitude (the log-scale makes it look closer together).\nCould you please report the maximum error between the curves in the supplementary material?\n\n- **The conclusion is a bit off**:\n\n  > (333) Overall, our work leads to a surprising result – that the almost forgotten gradient-based methods turn out to contain all the information needed to provide a faithful explanation of a model’s decision and that they can be as interpretable as the newest methods.\n\n  I disagree with multiple points in the conclusion:\n  - Gradient-based attribution almost forgotten? The integrated gradients paper received over 1000 citations last year (500 citations this year already). (source: semantic scholar)\n  - I would be cautious about the faithfulness claim. It could also be that the metrics are biased and prefer a smoother attribution map.\n\n- **Why not change the model?**: I would consider the raw gradient to be just the most faithful local representation of the model. If you prefer a smoother version, would it not make more sense to make changes to the underlying network architecture? This proposed methods feels like treating the symptoms instead of resolving the core issues.\n\n\nTo summarize, I am cautious about the main-take away that apply smoothing to gradient-based attribution methods makes them faithful. It would be great to also analyze if the faithfulness metrics are biased toward smooth attribuiton maps. Furthermore, I want to see the maximum error is between the different sigma-curves. It might be more significant\nthan suggested by the authors. A large error would questions the assumption that a smoothed gradient is more faithful.\n\nQUESTIONS:\n- How would you describe the relationship of your work to Balduzzi et al. 2017?\n\nLIMITATIONS:\nThe limitation section reads very technical and misses a very important point: the Deletion, Insertion, and Fidelity metrics are only proxy metrics for interpretability. Performing well on them does not mean that the attribution map is actually interpretable. Only a user study could answer this question.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}