Denoising Diffusion Path: Attribution Noise Reduction with An Auxiliary Diffusion Model

Authors: yiming  lei
zilong  li
junping  zhang
hongming  shan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 14:06:27

Abstract: The explainability of deep neural networks (DNNs) is critical for trust and reliability in AI systems. Path-based attribution methods, such as integrated gradients (IG), aim to explain predictions by accumulating gradients along a path from a baseline to the target image. However, noise accumulated during this process can significantly distort the explanation. While existing methods primarily concentrate on finding alternative paths to circumvent noise, they overlook a critical issue: intermediate-step images frequently diverge from the distribution of training data, further intensifying the impact of noise. This work presents a novel Denoising Diffusion Path (DDPath) to tackle this challenge by harnessing the power of diffusionmodels for denoising. By exploiting the inherent ability of diffusion models to progressively remove noise from an image, DDPath constructs a piece-wise linear path. Each segment of this path ensures that samples drawn from a Gaussian distribution are centered around the target image. This approach facilitates a gradual reduction of noise along the path. We further demonstrate that DDPath adheres to essential axiomatic properties for attribution methods and can be seamlessly integrated with existing methods such as IG. Extensive experimental results demonstrate that DDPath can significantly reduce noise in the attributions—resulting in clearer explanations—and achieves better quantitative results than traditional path-based methods.

Main paper: 
 introduction: 
 Deep neural networks (DNNs) have achieved remarkable success in various tasks, but their opaque decision-making processes remain a significant challenge and are critical to those high-staking scenarios like medical diagnosis [1] and autonomous driving [2] . Explainable Artificial Intelligence (XAI) aims to bridge this gap by providing insights into how DNNs make their predictions, where the commonly used interpretation methods include class activation mapping (CAM)-based [3, 4, 5, 6, 7] and path-based methods [8, 9, 10, 11, 12] .Theoretically, the path-based methods comply with the rigorous axiomatic properties, such as the implementation invariance and symmetry-preserving [8] , contributing significantly to the interpretation field, and we also focus on this kind of technique. Path-based attribution methods, such as integrated gradients (IG) [8] that is based on game-theoretic idea [13] , offer a valuable tool for XAI by accumulating gradients along a path from a baseline image to the target image being explained. However, [8] and SmoothGrad [9] struggle to maintain a continuously increasing Softmax probability along the integration path. This behavior can be counterintuitive and contradict human cognition, where the confidence in a prediction should generally rise as evidence accumulates. In contrast, the proposed DDPath achieves a more natural behavior by ensuring a continuously increasing Softmax probability along the path, even if the path itself exhibits fluctuations.these methods suffer from a crucial limitation: noise accumulation along the path. This noise can significantly distort the explanation, making it difficult to identify the features truly contributing to the DNN's decision.Existing approaches primarily focus on finding alternative paths to bypass noise regions. SmoothGrad progressively added noise to the image and achieved the effect of noise reduction, and the authors have verified that adding noise can help reduce noise during inference [9] . Blur IG successively blurred the input image with the Gaussian kernels that varied along the path [10] . Blur IG does not require a pre-defined "baseline" image which is critical to the original IG [8] . Guided IG is a general concept, i.e., a superset of path methods, which avoids the unrelated regions with high gradients by minimizing the attributions at every feature (or pixel) across this superset [11] . While the above alternatives can be beneficial, they neglect an essential issue: during the path construction, the intermediate-step images were modified manually by operations like noising or blurring, i.e., independent of the input image, resulting in them deviating significantly from the data distribution the DNN was trained on. This distribution shift further amplifies noise and hinders interpretability.In this paper, we intend to reduce explanation noise for path methods from a new perspective: the explanation noise stems from the distribution shift of intermediate-step images when calculating their gradients along the path, because the shifted images offer biased predictions for a pre-trained classification model (Fig. 1(c )), then influence the attributions. Hence, it is necessary to make the distributions of intermediate-step images closer to that of the original input image, so that the gradients back-propagated from accurate predictions are more relative to classes. As shown in Fig. 1 (a), the traditional paths P 1 , P 2 , and P 3 are independent of the input distribution even though they have the same starting point and the endpoint. Therefore, we aim to develop a path that simultaneously approaches the real data and implies progressive noise reduction.Inspired by the recently advanced diffusion models, which progressively add noise in the forward process and denoising during the reverse sampling process [14, 15, 16] , it is natural to correlate the reverse denoising process with the attribution path. On the other hand, the reverse process can recover the images that comply with the original data distribution despite the noisy intermediates. To this end, we propose a Denoising Diffusion Path (DDPath) for the attribution of deep neural networks. First, we define a novel denoising diffusion path that aligns the attribution path with the reverse sampling process by scaling the sampling steps. This enables the attribution path to incorporate the ability of generative modeling of diffusion models and the resultant intermediate-step images possess the approximated distributions with that of a classification model. Similar to existing path methods, the DDPath is also approximated by Riemann approximation [8, 12] . Furthermore, we demonstrate that the DDPath satisfies the corresponding axioms. Second, the DDPath can be easily combined with previous path methods and we developed the DDPath-IG, DDPath-BlurIG, and DDPath-GIG.In practice, we apply the pre-trained classifier-guided diffusion model to construct the DDPath [16] . Note that we do not attempt to investigate many advanced diffusion models in this paper, we pay more attention to exploring the reverse diffusion process to work with DNN attribution, which has not been discussed in previous attribution studies.Contributions. We summarize the main contributions of this paper as follows. (i) We propose a novel Denoising Diffusion Path (DDPath) for DNNs attribution. (ii) DDPath is theoretically compatible with current path-based methods, and we develop DDPath-IG, DDPath-BlurIG, and DDPath-GIG counterparts enhancing the baseline methods. (iii) DDPath can be easily implemented by applying a pre-trained classifier-guided diffusion model. (iv) Experimental results demonstrate the effectiveness of DDPath on both qualitative saliency maps and quantitative evaluations of insertion and deletion scores and accuracy information curves (AIC) [17, 12] .Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The contributions are clearly stated, and we have highlighted the scope of DNN attribution/interpretation/explanation. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 Gradient-based attribution. Integrated gradients (IG) [8] is designed to address the shortcomings of traditional saliency maps. By integrating gradients along a straight-line path from a baseline image to the target image, IG adheres to the sensitivity axiom and implementation invariance axiom. This property guarantees that the generated explanations are interpretable and consistent. While IG has been a significant advancement in XAI, subsequent research has focused on further improving its performance and addressing its limitations. Boundary-based integrated gradient [18] enhances precision with a boundary search mechanism and better baseline selection, while adversarial gradient integration (AGI) seeks higher accuracy through non-linear ascending trajectories [19] . However, AGI relies heavily on the quality of adversarial samples. Efforts like guided integrated gradients (GIG) address noise in the IG path, but GIG suffers from computational cost and limitations to image data [11] . Similarly, Fast-IG [20] and expected gradient (EG) [21] face limitations in efficiency or dependence on input features. These shortcomings in existing gradient-based methods motivate our work on DDPath. DDPath aims to provide cleaner and more interpretable attributions by tackling noise accumulation along the integration path. DDPath is designed to realize the progressive emergence of the image signal and gradual noise reduction along the path.Classifier-guided diffusion models. Recent advanced diffusion models like score-based diffusion models [22] and denoising diffusion probabilistic models (DDPMs) [14] have greatly facilitated the progress of generative modeling tasks. Of particular relevance to our work is the concept of classifier-guided diffusion models introduced by Ho et al. [23] . This framework guides the diffusion process using a pre-trained classifier, essentially learning to "reverse" the noise addition and correctly reach an input the classifier recognizes. This establishes a crucial link between diffusion models and classification tasks, paving the way for their application in interpretability, which is precisely the focus of DDPath. By leveraging this concept, DDPath benefits from the model's ability to progressively remove noise. This denoising capability directly tackles the challenge of noise accumulation in the attribution path, leading to cleaner and more interpretable explanations. Notably, explicitly constructing an attribution path with diffusion models has not been discussed in current attribution studies.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The limitations have been discussed and included in the conclusion section. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminary: 
 Before introducing our DDPath, we recall the path-based attribution framework and the corresponding axioms to be satisfied.Integrated gradients (IG) [8] is a pioneer work for deep visual model attribution with complete axiomatic properties, resulting in enormous axiomatic-based attribution methods. Assume that the F is a deep neural network to be explained, IG accumulates gradients along a linear path γ IG (α) = x ′ + α(x -x ′ ) via path integration:EQUATIONParameter frozen Pre-trained diffusion model Classifier guidance Current approaches focus more on finding a better path, i.e., ∂γ(α) ∂α , ignoring the intermediate points∂f (γ(α))∂γi(α) that is referred to as the inherent distribution shifts. On the other hand, an appropriate baseline is essential to traditional path-based attribution methods. The black baseline (or black image) suits IG better than a noise baseline, and this also causes difficulty in attributing black or dark regions of interest. Andrei et al. addressed this obstacle by applying both black and white baselines [17] . Hence, these manually designed baselines have their own biases. In this paper, the proposed DDPath can simply work with a noise baseline with which existing methods cannot work well.Sensitivity. The sensitivity axiom dictates that if a single feature changes between a baseline and a target image while causing different predictions, the attribution method must assign a non-zero attribution score to the differing feature. Otherwise, the attribution method might be insensitive to crucial changes. Implementation invariance. An attribution method follows the implementation invariance principle when, for the same pair of input data and predicted output, regardless of the specific neural network architecture or implementation details, the attribution scores remain consistent.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]Answer: [No] Justification: The data employed in this study are the publicly available, and the underlying source code will be released later. Guidelines: Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
denoising diffusion path: 
 For the target image x and a baseline x ′ , traditional attribution mainly considers x i -x ′ i , i.e., the differences between the i-th feature of the image and baseline, measuring how the classification model can behave with the gradual appearance of the i-th feature. In our diffusion path, such differences turned to the gradual appearance of images while the disappearance of noises, there are no direct relationships between intermediate steps and the original noisy baseline in that the sampling of the reverse diffusion process generates these intermediate images. It is practically implemented with Riemann approximation that will be discussed in Sec. 4.3. Definition 1. (Denoising Diffusion Path or DDPath) This path is a piece-wise linear function [24, 25] built upon the reversely sampled sequence with a maximum step number T : x T , . . . , x t , x t-1 , . . . , x 0 , which is defined asEQUATIONwhere x T is the noisy signal baseline and x 0 is the finally sampled image, each pieceN α (µ α (x), Σ α (x)) is a set of samples complied with a Gaussian distribution with mean and variance concerning the target image x. The diffusion sample step t is aligned with the path coefficient α by setting α = t T ∈ [0, 1]. Theoretically, DDPath is also a type of definition in terms of a set of all possible paths, which is similar to the adaptive path in GuidedIG [11] and Shapley values [26] . That is to say, for each loop from the baseline to the target image, the path is composed of sampled images from every piece N α (µ α (x), Σ α (x)). For the rest of this paper, the γ(α) denotes the proposed DDPath if without a specific statement.Based on Definition 1, we discuss how to attribute along the DDPath. Specifically, we showcase that DDPath-IG, DDPath-BlurIG, and DDPath-GuidedIG enhance the corresponding baseline methods. First, for the DDPath-IG, we can directly replace the linear path in Eq. ( 1) with the DDPath: Definition 2. (DDPath-IG) Given a diffusion model E θ pre-trained using classifier guidance, f ϕ is the corresponding pre-trained classifier, for the i-th feature in the input x of class y, its attribution is the integrated gradients along the DDPath:EQUATIONthe N α (μ θ (x), Σ θ (x)) denotes the distribution parameterized by the diffusion model E θ , andμθ (x) = ρ • µ θ (x) + κ • Σ∇ xα log p ϕ (y|x α ).The ρ and κ are scaling factors controlling the mean and the gradient term variation.A critical problem of sampling with the diffusion model is that the generated images are diverse when sampling from the noise signals. In DDPath-IG, we solve this obstacle by simply enforcing the sampling centered at the target image, i.e., the mean and variance are calculated by the original image x. We use the same sampling strategy for the following DDPath-BlurIG and DDPath-GIG.Definition 3. (DDPath-BlurIG) Given the Gaussian kernels along the path parameter α, L(x, y, α)= ∞ m=-∞ ∞ n=-∞ 1 πα e -x 2 +y 2 α •γ(α)(x-m, y-n), then the attribution of the i-th feature is obtained by:EQUATIONwhere the t is the number of steps in the Riemann approximation, and α t = t • αt T [10] .DDPath-BlurIG applies the DDPath and blurs the sampled images along the denoising path. That is to say, it scales the spaces of all sampled pieces in Definition 3 while preserving the data distributions within pieces to approach the real data distribution in Fig. 1 .Definition 4. (DDPath-GIG) Given an IG path γ IG (α) [8] and a DDPath γ(α), the objective of DDPath-GIG is defined as:EQUATION)where λ is the coefficient that balances the two terms, N is the number of features (or pixels), and Γ contains all possible paths of DDPath.In Definition 4, the traditional path is replaced with our DDPath, and in practice, the Γ is implemented by repeating random sampling loops. Therefore, the first term of Eq. ( 5) aims to find a better denoising path γ(α) that avoids those regions causing noisy explanations, and the second term ensures the diffusion path does not deviate severely off the shortest path, decreasing the likelihood of crossing areas that are too out-of-distribution.In this section, taking DDPath-IG as an example, we show that it satisfies the axiomatic properties in [8] . First, the DDPath-IG satisfies the sensitivity that the image differs from the noisy baseline, and then the partial derivatives of differing features are non-zero. Second, the DDPath is agnostic to the architecture of DNNs so that the DDPath-IG also satisfies the completeness that [27, 28, 8] . Third, the calculation of partial derivatives of DDPath-IG follows the chain rule so that the attributions are invariant to network implementations, therefore the DDPath-IG satisfies the implementation invariance. Fourth, recall that the IG is the unique path method that is symmetry-preserving (Theorem 1 in [8] ), the DDPath-IG also maintains this property.N i A i = F (x) -F (x ′ )Algorithm 1 Algorithm of DDPath-IG. Require: Target image x and its label y, the initial noisy baseline x ′ randomly sampled from a Gaussian; target model F (•), diffusion trained classifier h ϕ , the diffusion model E θ , total number of step T . Return: Attribution for the target image x:A = 1 T T -1 t=0 g t . 1: for t = 0 to T -1 do 2:if t == 0 then 3:x t = x ′ ▷ Noise baseline 4:else 5:x t = x ′ t ▷ Sampled image in t -1 step 6:end if7: ρ = 1 -t T , κ = t T ▷ Scaling coefficients 8: μθ (x) = ρ • µ θ (x) + κ • Σ∇ xt log p ϕ (y|x t ).▷ Update sampling mean 9:x ′ t ∼ N t (μ θ (x), Σ θ (x)).▷ Sampling 10:g t = ∂(F (xt)) ∂xt .▷ Calculate gradients 11: end forTo ensure clarity, we reiterate that this paper concentrates on establishing a correlation between the attribution path and the reverse diffusion process. Specifically, we implement the DDPath-IG with a pre-trained diffusion model trained with classifier guidance [16] . The algorithm of DDPath-IG is shown in Algorithm 1, and the algorithms of DDPath-BlurIG and DDPath-GIG are provided in the Appendix. The implementation of the integral is also approximated by the Riemman approximation, taking DDPath-IG as an example:EQUATIONFor the classifier-guided diffusion sampling in [16] , the sampling mean μθ (x) = µ + sΣ∇ xt log p ϕ (y|x t ) scales the classifier gradient term with s that corresponds to the sampling steps t. However, in this work, the proposed denoising path should guarantee consistency between the baseline and the target image. Therefore, in Definition 2, the sampling mean is centered at the image x while the classifier gradients are calculated via step-wise intermediate images x t , and we use a simple scaling scheme with ρ and κ:EQUATION)This scheme is reasonable in that at the very beginning of sampling, we do not expect the mean to shift severely ensuring less data distribution shift. For the variance term, the gradients ∇ xt are noisy because of the inaccurate predictions at initial steps. With the increase of t, more accurate gradients can be obtained by more correct predictions. Hence, the sampling means μθ (x) become mainly dominated by this gradient of step-wise inputs, demonstrating that the DDPath progressively enables the increasing prediction scores by preserving data distribution along the path. 
experimental setup: 
 Datasets. Following previous studies [10, 11, 12] , we evaluate the effectiveness of DDPath on the validation set of ImageNet-1k [30] that contains 50, 000 images of 1, 000 classes. Furthermore, we conducted a pointing game experiment on MS COCO validation set [31] .Models and baselines. For the classification model, we use ResNet-50 [32] and VGG-16/-19 [29] as backbone. The baseline attribution methods are Guided-BP [33] , IG [8] , Smooth IG [9] , Blur IG [10] , and Guided IG [11] . All the experiments are implemented by PyTorch [34] and conducted on an NVIDIA A100 GPU. The number of sampling steps for DDPath methods is 250. Following previous DNN interpretation works [35, 6, 7, 36] , we compare the Insertion and Deletion scores and provide the corresponding curves and the saliency maps in Fig. 3 . The insertion process starts with a blurred image, and then iteratively injects original image information (3.6% of total pixels) into the blurred version, guided by the saliency map values. Regions with higher saliency scores are prioritized for insertion, gradually revealing the informative parts of the original image and leading to its full reconstruction. Conversely, deletion identifies relevant pixels (3.6%) in the blurred image based on the saliency map and replaces them with their corresponding values from the original image. This process essentially "unmasks" the informative regions by strategically replacing noise with the original content. In Fig. 3 , we can see that the DDPath-IG captures more comprehensive information and more details of the object, e.g., the birds' wings and lobster feet. For the insertion and deletion curves, the DDPath achieves better quantitative insertion AUC values, indicating the image information progressively injected is more important. Although DDPath does not obtain the best deletion AUCs, this is trivial to significant improvement on Insertion and better saliency maps. More saliency maps are provided in Figs. 6 and 7 in Appendix.In this section, we investigate the path length (or sampling steps for DDPath). Fig. 4 shows the saliency maps of different methods at increased steps. With the path length increase, baseline methods are difficult to obtain interpretation performance gains further while achieving attribution noise reduction. Although they achieve considerable saliency maps at early steps, they still exhibit a weaker noise reduction effect. IG exhibits little variances on saliency maps which is caused by linear path. GIG tends to find an adaptive path resulting in less structural consistency along the whole path. The DDPath-IG provides us with a consistent emergence of the salient regions while preserving consistent structures of the object. From the curves at the right part in Fig. 4 , we can see that DDPath performs worse at the early stages, and it obtains more improvements by increasing the path length, however, the baselines gain less.To evaluate the effectiveness of DDPath in pinpointing the most salient pixels, we conducted a "pointing game" on the MS COCO 2017 validation set. This approach, similar to those used in Score-CAM [6] and Group-CAM [7] , assesses localization accuracy. We calculated the metric Hits Hits+Misses to quantify how well the identified salient pixels coincided with the annotated bounding boxes in the data, where the "Hits" counts the number of the most salient pixels that fall in the bounding box, and "Misses" otherwise. Higher scores indicate that DDPath excels at highlighting the most relevant image regions for the model's prediction. In Table 2 , the DDPath counterparts consistently outperform the baseline methods, and we argue that this is severely caused by the noisy salient pixels shifted away from the target objects, i.e., out of the bounding box. For example, in Fig. 3 , the GIG of the first case, GIG and BlurIG in the second case, and SmoothGrad in the third case. Therefore, the DDPath not only reduces those noises but also avoids the incorrect salient pixels. Scaling scheme. First, we discuss the scaling scheme defined in Eq. ( 7). The above experiments involving DDPath used ρ = 1 -t T , κ = t T by default, which maintains a stably decreased weight for sampling mean. Here, we reverse such scaling as ρ = t T , κ = 1 -t T to enable a smaller mean and larger variance at the initial steps. In Table 3 , we compare the two scaling schemes in terms of Insertion (Ins.), Deletion (Del.), and accuracy information curves (AIC) [17, 11] . We can see that the Reverse setting performs worse than the Default, demonstrating that sampling with a smaller mean and a larger variance at early steps is inferior in preserving information from the input image. Compared with the baseline methods, the DDPath with Reverse scaling performs better in AIC and Insertion, showing the considerable effect of avoiding distribution shift with DDPath. However, DDPath with Reverse scaling achieves higher deletion values, and we claim that this is due to the larger initial variance caused edge detection effect over the whole image, which compromises the target object. In Fig. 5 , the Reverse results highlight more noises and more edges, neglecting the inner regions of the objects.Reverse Reverse Default DefaultAttribution with noise. Here, we compare the manually adding noise with our DDPath. In Table 4 , we implement Noise counterparts for IG, BlurIG, and GIG. SmoothGrad is also a method of constructing a noisy path. Compared with Table 1 , IG-Noise, BlurIG-Noise, and GIG-Noise obtain slight improvements against the vanilla versions in terms of AIC and Insertion, but they are still inferior to DDPath versions. This verifies that the DDPath enables better noise reduction and accurate predictions for the points on the path.  
conclusion: 
 This paper introduces the Denoising Diffusion Path (DDPath), a novel approach for mitigating noise accumulation in path-based attribution methods. DDPath leverages the power of diffusion models to construct a path where noise is progressively removed, leading to significantly cleaner and more interpretable attributions. We demonstrate that DDPath adheres to essential axiomatic properties and integrates seamlessly with existing methods like Integrated Gradients, requiring only a pre-trained classifier-guided diffusion model. Extensive evaluations showcase the superiority of DDPath compared to traditional path-based methods, achieving explanations with less noise and better alignment with the DNN's decision-making process.Broader impact and limitation. This paper brought new insights into the attribution of DNNs with simple implementations, i.e., classifier-guided diffusion, and it will trigger more related research in this direction by applying more advanced diffusion models. Moreover, it also provides a possible way of investigating the knowledge of large language models via diffusion models to realize the true human-understandable vision-language consistent explanations. A key limitation of DDPath is that it requires a longer path (more sampling steps) than current methods. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: in the Conclusion section, we have highlighted that the proposed method can inspire more research on DNN attribution using diffusion models.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [Yes]Justification: the pre-trained diffusion model used in this paper has been publicly released on GitHub.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: the code, data, and models were cited in our paper.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: A.1 Algorithms of DDPath-BlurIG and DDPath-GIG Algorithm 2 Algorithm of DDPath-BlurIG. Require: Target image x and its label y, the initial noisy baseline x ′ randomly sampled from a Gaussian; target model F (•), diffusion trained classifier h ϕ , the diffusion model E θ , total number of step T , the t-step Gaussian blur kernels L(t). Return: Attribution for the target image x,A = 1 T T -1 t=0 g t . 1: for t = 0 to T -1 do 2:if t == 0 then 3:x t = x ′ ▷ Noise baseline 4:else 5:x t = x ′ t ▷ Sampled image in t -1 step 6:end if7: ρ = 1 -t T , κ = t T ▷ Scaling coefficients 8: μθ (x) = ρ • µ θ (x) + κ • Σ∇ xt log p ϕ (y|x t ).▷ Update sampling mean 9: end ifx ′ t ∼ N t (μ θ (x), Σ θ (x)); x ′ t = L(x ′ t ,8: ρ = 1 -t T , κ = t T ▷ Scaling coefficients 9: μθ (x) = ρ • µ θ (x) + κ • Σ∇ xt log p ϕ (y|x t ).▷ Update sampling mean 10:x← x ′ t ∼ N t (μ θ (x), Σ θ (x)). ▷ Sampling 11:repeat until δ ≤ 112: y i ← ∞, ∀I|x i = X I i ; d target ← d total (1 -t T ); d current ← ||x -X I ||d S ← i∈S |x i -X I i |; δ ← dcurrent-dtarget d S ; temp ← x 20:if δ > 1 then 21:x i ← X I i , ∀ ∈ S 22:else 23:x i ← (1 -δ)x i + δX I i , ∀i ∈ S 24:end if 25: y i = 0, ∀i = ∞; attr i = attr i + (x i -temp i )y i ,To investigate the diffusion model size, we apply the released diffusion models by [16] . Note that these diffusion models of different sizes correspond to different image resolutions, including 64 × 64, 128 × 128, 256 × 256, and 512 × 512. The visualization results can be found in Fig. 8 . We can see that larger models generate larger resolutions of saliency maps, and they illustrate more fine-grained details.We applied two approaches to generate adversarial samples, one is the fast gradient sign attack (FGSM) described by Goodfellow et al. [37] , and the other is adding simple Gaussian noise. We compared the results of IG and DDPath-IG in terms of Insertion and Deletion values, these results and the saliency maps are shown in Fig. 9 . Interestingly, the IG generated saliency maps with degraded quality, while the DDPath-IG are more robust to adversarial samples (FGSM and Gaussian). We evaluated our DDPath by setting ρ = 1 -( t T ) a and κ = ( t T ) a with both a = 0.5 and a = 2, and we note that the linear scaling used in this paper is equal to that of a = 1. As shown in Table 5 , we can see that the DDPath-IG surpasses the baseline IG among different a values, indicating the effectiveness of our DDPath. When a = 2, the weight of the mean term decreases slowly at the early step, ensuring better preservation of the main object in the images. Besides, the weight of the class-related variance term increases fast at higher steps, enabling better preservation of discriminative information and object details, and this is consistent with the mechanism of task weights in [38] . In contrast, when a = 0.5, the variance weight increases fast at early steps while the noises are still severe. Hence, the class-related information can be affected by the noises while influencing the classification results and attribution qualities as shown in Fig. 10 . The setting of a = 1 is a trade-off in our experiments.  


Decision: This paper has received consistent feedback from all three reviewers. The reviewers engaged in thorough discussion and rebuttal, with most of their concerns being addressed, ultimately reaching a consensus. The paper proposes a novel Denoising Diffusion Path (DDPath) for DNN attribution, which can be easily implemented using a pre-trained classifier-guided diffusion model. Comprehensive experiments  and ablation studies validate the effectiveness of DDPath. Therefore,  the AC  has decided to accept this paper.

Reviews: review: paper_summary: paper_summary: This paper proposes Denoising Diffusion Path (DDPath), a method to reduce noise in path-based attribution for deep neural networks. DDPath uses diffusion models to create a path with decreasing noise, resulting in clearer attributions. It can be integrated with existing methods like Integrated Gradients and maintains axiomatic properties. Experiments on ImageNet and MS COCO show DDPath outperforms traditional methods in producing clear explanations and improving metrics like Insertion and Deletion scores. The paper includes analysis, comparisons, and discussion of limitations.
strength_weakness: strength_weakness: - Computational cost: The method requires significantly more sampling steps (250) than traditional attribution methods, which could limit its practical application in time-sensitive scenarios or for large-scale analyses.
- Marginal quantitative improvements: While the paper shows improvements over baseline methods, the differences in quantitative metrics (e.g., Insertion and Deletion scores in Table 1) are relatively small. This raises questions about the practical significance of the improvements, especially given the increased computational cost.
- Unconvincing visual results: The saliency maps presented in Figure 3 do not demonstrate a dramatic improvement over previous methods. While there are some differences, they are subtle and it's not immediately clear that DDPath provides substantially clearer or more interpretable attributions than existing techniques.
- Limited to classification models: The paper only demonstrates the method's effectiveness on image classification tasks. There's no exploration or discussion of how DDPath might apply to or perform on non-classification models, such as regression, detection, or generative tasks.
Reliance on pre-trained diffusion models: The approach depends on having a suitable pre-trained diffusion model, which may not always be available for all domains or tasks.
questions: questions: - Does image variation resulted from diffusion model demonstrate more correlation with image semantics than previous image variation methods?
- Does the improvement in attribution quality provide significant improvement in attribution-based applications?
- Does the attribution quality vary as pertained diffusion model size/quality varies?
- If an image is generated from adversarial attack on diffusion model, will the attribution quality be impacted?
limitations: limitations: The author discusses limitation of the method that it requires longer paths.
paper_summary: paper_summary: This paper proposes a Denoising Diffusion Path (DDPath) to address the challenge in path-based attribution methods, where intermediate steps often deviate from the training data distribution. By leveraging the denoising power of classifier-guided diffusion models, DDPath constructs a piece-wise linear path to ensure gradual denoising, resulting in cleaner and more interpretable attributions. It also demonstrates that DDPath satisfies the axiomatic properties and can be integrated seamlessly with existing methods like Integrated Gradient.
strength_weakness: strength_weakness: * I am not sure about the details of the Algorithm 1. What is the input timestep for the pre-trained diffusion model and the classifier? Do noisy baselines $\textbf{x}_{t}$ align with the noise scheduling in diffusion models?

* I am confused about the use of the diffusion model. Why the diffusion model is used to estimate the noise from the target image $\textbf{x}$? Is the target image a noisy image?
questions: questions: * Please clarify the details of the diffusion model mentioned in the weaknesses.

* Although DDPath requires multiple feed-forward of the diffusion model and the classifier, there is no analysis of the time-complexity comparison with other methods.

* What about using denoising task weights in DTR [1] instead of a linear scaling scheme? For example, defining each scaling factor as $\rho = 1 - {(\frac{t}{T})}^{\alpha}$ and  $\kappa = {(\frac{t}{T})}^{\alpha}$, and experiments with $\alpha = 0.5, 2$ provide a tighter connection to the diffusion model.

[1] Park et al., Denoising Task Routing for Diffusion Models, ICLR 2024.
limitations: limitations: The authors have discussed them.
paper_summary: paper_summary: This paper proposes an approach to interpreting deep neural networks, specifically they focus on a path-based approach where they utilize the denoising ability of the diffusion model to construct a path to be integrated to compute the attribution of the target image.
strength_weakness: strength_weakness: 1. The presentation in Section 3 is somewhat abbreviated, for example, there is no definition of the lower x^', which can be confusing to those unfamiliar with this area.

2. The proposed method requires the introduction of classifier-guided diffusion, which requires a pre-trained diffusion and noise-aware classifier, and it is not clear whether the method is applicable when not belonging to a predefined category or when the category is difficult to define.
questions: questions: see weaknesss
limitations: limitations: The authors briefly discuss their limitations

Comments: title: Response to Reviewer rjLS
Response to further concerns by Reviewer qJDk
Response to Concern on embedding models (such as CLIP)
Response to comment by Reviewer 2oA3
comment: Thank you for your response. I have raised my score.
Dear reviewer rjLS,

    We sincerely appreciate your time and valuable feedback. We would be grateful to know if our response is satisfactory or if there are any additional points we should address. Looking forward to hear from you.
Dear reviewer qJDk, we would like to express our sincere gratitude once again for your response and insightful questions. We are eager to know if our further explanations and experimental results can well addressed your concerns.
Thanks for your valuable feedback. 

First, the proposed DDPath is model-agnostic and **CAN** be extended to embedding models such as CLIP. Specifically, one can fine-tune the CLIP image encoder coupled with a classifier (such as a Linear layer). Then, the DDPath can work on this trained model (CLIP image encoder + classifier) to obtain saliency maps using DDPath-IG. Theoretically, DDPath is adaptive to both ViT and CNN backbones pre-trained by CLIP. However, to the best of our knowledge, path-based attribution studies have not been focused on ViT models, and one of the important reasons is that using path-based attribution methods can result in grid-like artifacts caused by the image-patch nature of ViT. Also, this may provide us with an interesting further research direction in XAI. 

Second, we conducted additional experiments by simply fine-tuning a linear classifier with CLIP’s image encoder (ResNet-50) on CIFAR100 and Flowers102 datasets. We report the top-1 accuracy and interpretability metrics. Note that the primary goal of this experiment is not to achieve optimal classification accuracy but to demonstrate the feasibility and effectiveness of the DDPath method. We can see that DDPath-IG outperforms IG on both Insertion and Deletion values, demonstrating the effectiveness of applying the DDPath to downstream tasks using pre-trained embedding models.

Table R1 Classification accuracy (%) and interpretability metrics (IG / DDPath-IG) obtained by fine-tuning CLIP-ResNet-50

|  | CIFAR100 | Flowers102 |
| --- | --- | --- |
| Accuracy | 72.55 | 88.25 |
| Insertion$\uparrow$ | 25.6 / **29.2** | 23.3 / **26.7** |
| Deletion$\downarrow$ | 14.9 / **12.8** | 17.5 / **16.2** |

Through the above discussions and experiments, we hope we have solved your additional concern.
Thanks for your response. These addressed many of my concerns. I have one additional questions regrading to your response to W 4: Limited to classification models. Can this method be extended to study embedding models (such as CLIP) and attribute image features for downstream tasks?
We sincerely appreciate your valuable insight and positive feedback. We will include the additional experiments and discussions in the next version.
Thanks for the response. It well addressed my concerns and I appreciate the detailed answers have enhanced my understanding of the paper. I will increase my rating to weak accept.