Original: R E - CALIBRATING F EATURE A TTRIBUTIONS FOR M ODEL I NTERPRETATION
Revised: R E - CALIBRATING F EATURE A TTRIBUTIONS FOR M ODEL I NTERPRETATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The ability to interpret machine learning models is critical for high-stakes applications.
Revised: The ability to interpret machine learning models is critical for high-stakes applications.

Original: Due to its desirable theoretical properties, path integration is a widely used scheme for feature attribution to interpret model predictions.
Revised: Due to its desirable theoretical properties, path integration is a widely used scheme for feature attribution to interpret model predictions.

Original: However, the methods implementing this scheme currently rely on absolute attribution scores to eventually provide sensible interpretations.
Revised: However, the methods implementing this scheme currently rely on absolute attribution scores to eventually provide sensible interpretations.

Original: This not only contradicts the premise that the features with larger attribution scores are more relevant to the model prediction, but also conflicts with the theoretical settings for which the desirable properties of the attributions are proven.
Revised: This not only contradicts the premise that the features with larger attribution scores are more relevant to the model prediction, but also conflicts with the theoretical settings for which the desirable properties of the attributions are proven.

Original: We address this by devising a method to first compute an appropriate reference for the path integration scheme.
Revised: We address this by devising a method to first compute an appropriate reference for the path integration scheme.

Original: This reference further helps in identifying valid interpolation points on a desired integration path.
Revised: This reference further helps in identifying valid interpolation points on a desired integration path.

Original: The reference is computed in a gradient ascending direction on the model’s loss surface, while the interpolations are performed by analyzing the model gradients and variations between the reference and the input.
Revised: The reference is computed in a gradient ascending direction on the model’s loss surface, while the interpolations are performed by analyzing the model gradients and variations between the reference and the input.

Original: The eventual integration is effectively performed along a non-linear path.
Revised: The eventual integration is effectively performed along a non-linear path.

Original: Our scheme can be incorporated into the existing integral-based attribution methods.
Revised: Our scheme can be incorporated into the existing integral-based attribution methods.

Original: We also devise an effective sampling and integration procedure that enables employing our scheme with multireference path integration efficiently.
Revised: We also devise an effective sampling and integration procedure that enables employing our scheme with multireference path integration efficiently.

Original: We achieve a marked performance boost for a range of integral-based attribution methods on both local and global evaluation metrics by enhancing them with our scheme.
Revised: We achieve a marked performance boost for a range of integral-based attribution methods on both local and global evaluation metrics by enhancing them with our scheme.

Original: Our extensive results also show improved sensitivity, sanity preservation and model robustness with the proposed re-calibration of the attribution techniques with our method.
Revised: Our extensive results also show improved sensitivity, sanity preservation and model robustness with the proposed re-calibration of the attribution techniques with our method.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: How to interpret deep learning predictions is a major concern for the real-world applications, especially in the high-stakes domains.
Revised: How to interpret deep learning predictions is a major concern for the real-world applications, especially in the high-stakes domains.

Original: Feature attribution methods explain a model’s predictions by assigning importance scores (attributions) to the input features (Simonyan et al., 2014; Springenberg et al., 2015; Shrikumar et al., 2017).
Revised: Feature attribution methods explain a model’s prediction by assigning importance scores (attributions) to the input features (Simonyan et al., 2014; Springenberg et al., 2015; Shrikumar et al., 2017).

Original: They assert that features with larger attribution scores are more relevant to the model prediction than those with smaller scores.
Revised: They assert that features with larger attribution scores are more relevant to the model prediction than those with smaller scores.

Original: Among a variety of attribution methods, integral-based techniques (Sundararajan et al., 2017; Sturmfels et al., 2020) are particularly attractive because they satisfy certain desirable axiomatic properties, which others do not.
Revised: Among a variety of attribution methods, integral-based techniques (Sundararajan et al., 2017; Sturmfels et al., 2020) are particularly attractive because they satisfy certain desirable axiomatic properties, which others do not.

Original: This also makes them suitable for model regularization (Chen et al., 2019; Erion et al., 2021).
Revised: This also makes them suitable for model regularization (Chen et al., 2019; Erion et al., 2021).

Original: Inspired by the cooperative game theory, integral-based attribution methods introduce a reference to represent the absence of the input signal.
Revised: Inspired by the cooperative game theory, integral-based attribution methods introduce a reference to represent the absence of the input signal.

Original: This allows them to calculate the attribution for the presence of an input feature (Sundararajan et al., 2017; Erion et al., 2021; Pan et al., 2021).
Revised: This allows them to calculate the attribution for the presence of an input feature (Sundararajan et al., 2017; Erion et al., 2021; Pan et al., 2021).

Original: Since the attribution is computed with respect to a reference, finding the correct reference that represents the absence of a feature in a true sense, is critical for the reliability of these methods.
Revised: Since the attribution is computed with respect to a reference, finding the correct reference that represents the absence of a feature in a true sense, is critical for the reliability of these methods.

Original: In their Integrated
Revised: In their Integrated

Original: Gradient (IG) method, Sundararajan et al.
Revised: Gradient (IG) method, Sundararajan et al.

Original: (2017) chose a black image (zero input) as the reference.
Revised: (2017) chose a black image (zero input) as the reference.

Original: However, this result is always assigning zero attribution to black input pixels.
Revised: However, this result is always assigning zero attribution to black input pixels.

Original: Sturmfels et al.
Revised: Sturmfels et al.

Original: (2020) confirmed that a fixed reference renders an attribution method blind to the features that the reference employs. To resolve that, multiple references are now commonly used by the attribution methods. The choices of multiple references include, uniform noise references (IG-Uniform) (Sturmfels et al., 2020), Gaussian noise references (IG-SG and IG-SQ) (Smilkov et al., 2017; Hooker et al., 2019), training distribution (EG) (Erion et al., 2021) and adversarial examples (AGI) (Pan et al., 2021), etc. However, the attribution maps computed by these methods face frequent variations between positive and negative attribution scores, see Figure 1 top row.
Revised: Input

Original: To handle that, these methods must use

Original: [Figure]

Original: IG-Uniform
Revised: IG-Uniform

Original: IG-SG
Revised: IG-SG

Original: IG-SQ
Revised: IG-SQ

Original: EG
Revised: EG

Original: AGI
Revised: AGI

Original: V a n ill a
Revised: V a n ill a

Original: A b s o l u t e V a l u e
Revised: A b s o l u t e V a l u e

Original: R e - C a li b r a t e d
Revised: R e - C a li b r a t e d

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 5.
Revised: AUC: 5.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 3.
Revised: AUC: 3.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 4.
Revised: AUC: 4.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 6.
Revised: AUC: 6.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 10.
Revised: AUC: 10.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 5.
Revised: AUC: 5.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 4.
Revised: AUC: 4.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 4.
Revised: AUC: 4.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 7.
Revised: AUC: 7.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 11.
Revised: AUC: 11.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 14.
Revised: AUC: 14.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 13.
Revised: AUC: 13.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 33.
Revised: AUC: 33.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 13.
Revised: AUC: 13.

Original: Pixel Pert.
Revised: Pixel Pert.

Original: AUC: 24.
Revised: AUC: 24.

Original: [Figure]
Revised: [Figure]

Original: + - absolute scores to generate plausible explanations, see Figure 1 middle row.
Revised: + - references include uniform references ( et references ( Smilkov et Hooker et training distribution ( examples ( (Pan et

Revised: , 2021), etc.

Revised: However, the attribution maps computed by these methods face frequent variations between positive and negative attribution scores, see Figure 1 top row.

Revised: To handle that, these methods must use absolute scores to generate plausible explanations, see Figure 1 middle row.

Original: Numerically, computing the absolute values re-orders the estimated attributions for the pixels, which violates the primary assertion of the attribution methods.
Revised: Numerically, computing the absolute values re-orders the estimated attributions for the pixels, which violates the primary assertion of the attribution methods.

Original: Moreover, relying only on the magnitude of the attributions is also in contradiction with the axiomatic properties, which are proven for the actual numerical scores.
Revised: Moreover, relying only on the magnitude of the attributions is also in contradiction with the axiomatic properties, which are proven for the actual numerical scores.

Original: This inconsistency compromises the desirability of these methods.
Revised: This inconsistency compromises the desirability of these methods.

Original: To address these problems, we develop a method to compute the desired reference for an input along the model’s gradient ascending direction.
Revised: To address these problems, we develop a method to compute the desired reference for an input along the model’s gradient ascending direction.

Original: Moreover, we allow the gradient integration along a nonlinear path from the reference to the input.
Revised: Moreover, we allow the gradient integration along a nonlinear path from the reference to the input.

Original: This is made possible by systematically identifying valid interpolation points on the path.
Revised: This is made possible by systematically identifying valid interpolation points on the path.

Original: It eventually enables us to directly use the actual, instead of the absolute attributions for model interpretation.
Revised: It eventually enables us to directly use the actual, instead of the absolute attributions for model interpretation.

Original: We further devise a technique to efficiently compute the integral with valid references which can be estimated using the predefined references employed by the existing methods.
Revised: We further devise a technique to efficiently compute the integral with valid references which can be estimated using the predefined references employed by the existing methods.

Original: This enables us to leverage our technique to re-calibrate the attributions of the existing methods without additional computational overhead.
Revised: This enables us to leverage our technique to re-calibrate the attributions of the existing methods without additional computational overhead.

Original: Figure 1 bottom row shows the attribution maps calculated by the popular integral-based attribution methods calibrated with our technique.
Revised: Figure 1 bottom row shows the attribution maps calculated by the popular integral-based attribution methods calibrated with our technique.

Original: These maps are computed with the actual attributions scores, not the absolute values.
Revised: These maps are computed with the actual attributions scores, not the absolute values.

Original: Hence, they conform to the primary assertion of the attribution methods and to the settings used in establishing their theoretical properties.
Revised: Hence, they conform to the primary assertion of the attribution methods and to the settings used in establishing their theoretical properties.

Original: Moreover, they achieve better quantitative scores.
Revised: Moreover, they achieve better quantitative scores.

Original: In our experiments, quantitative evaluation is performed with pixel perturbation (Samek et al., 2016) and DiffROAR (Shah et al., 2021) on ImageNet-2012 validation set (Russakovsky et al., 2015),
Revised: In our experiments, quantitative evaluation is performed with pixel perturbation (Samek et al., 2016) and DiffROAR (Shah et al., 2021) on ImageNet-2012 validation set (Russakovsky et al., 2015),

Original: CIFAR-100 and CIFAR-10 (Krizhevsky et al., 2009).
Revised: CIFAR-100 and CIFAR-10 (Krizhevsky et al., 2009).

Original: We show a marked performance improvement for a range of integral-based attribution methods by re-calibrating them with our technique.
Revised: We show a marked performance improvement for a range of integral-based attribution methods by re-calibrating them with our technique.

Original: We also provide a detailed sensitivity analysis of the improved methods with Sensitivity-n (Ancona et al., 2018), and passing the sanity checks (Adebayo et al., 2018).
Revised: We also provide a detailed sensitivity analysis of the improved methods with Sensitivity-n (Ancona et al., 2018), and passing the sanity checks (Adebayo et al., 2018).

Original: Moreover, we also show consistent improvements in attribution prior based regularization (Erion et al., 2021) with our technique.
Revised: Moreover, we also show consistent improvements in attribution prior based regularization (Erion et al., 2021) with our technique.

Original: A considerable performance gain for a variety of techniques and models across a range of evaluation metrics ascertains the effectiveness of our re-calibration method.
Revised: A considerable performance gain for a variety of techniques and models across a range of evaluation metrics ascertains the effectiveness of our re-calibration method.

Original: In summary, we make the following major contributions.
Revised: In summary, we make the following major contributions.

Original: We propose a method to compute a reliable reference for integral-based attribution methods.
Revised: We propose a method to compute a reliable reference for integral-based attribution methods.

Original: We devise a technique to efficiently integrate over a non-linear path that enables meaningful interpretations using the actual, as opposed to absolute attribution scores.
Revised: We devise a technique to efficiently integrate over a non-linear path that enables meaningful interpretations using the actual, as opposed to absolute attribution scores.

Original: We enhance a range of existing methods with our findings to demonstrate a considerable performance boost across the board over multiple evaluation 
Revised: We enhance a range of existing methods with our findings to demonstrate a considerable performance boost across the board over multiple evaluation metrics, and for model regularization.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Feature attribution methods explain a model’s prediction by assigning attribution scores to the input features.
Revised: Feature attribution methods explain a model’s prediction by assigning attribution scores to the input features.

Original: These techniques can be roughly categorized into perturbation based, and back-propagation based methods.
Revised: These techniques can be roughly categorized into perturbation based, and back-propagation based methods.

Original: The former, e.g., (Zeiler & Fergus, 2014; Fong & Vedaldi, 2017), calculates the attribution scores by repeatedly perturbing the input features and analyzing the resulting effects on the model output.
Revised: The former, e.g., (Zeiler & Fergus, 2014; Fong & Vedaldi, 2017), calculates the attribution scores by repeatedly perturbing the input features and analyzing the resulting effects on the model output.

Original: Back-propagation methods, e.g., (Simonyan et al., 2014; Sundararajan et al., 2017) estimate the attribution scores using a backward pass of model gradients, and enjoy multiple advantages over the former (Ross et al., 2017; Chen et al., 2019).
Revised: Back-propagation methods, e.g., (Simonyan et al., 2014; Sundararajan et al., 2017) estimate the attribution scores using a backward pass of model gradients, and enjoy multiple advantages over the former (Ross et al., 2017; Chen et al., 2019).

Original: Since our contribution relates to the back-propagation methods, we focus on the related work in this sub-category.
Revised: Since our contribution relates to the back-propagation methods, we focus on the related work in this sub-category.

Original: Gradient-Based Methods: Simonyan et al.
Revised: Gradient-Based Methods: Simonyan et al.

Original: (2014) first employed input gradients for class-specific saliency mapping for model interpretation.
Revised: (2014) first employed input gradients for class-specific saliency mapping for model interpretation.

Original: Deconvolutional Network (Zeiler & Fergus, 2014) and
Revised: Deconvolutional Network (Zeiler & Fergus, 2014) and

Original: Guided Backpropagation (Springenberg et al., 2015) also propagate the activation information in the backward direction to highlight important input features.
Revised: Guided Backpropagation (Springenberg et al., 2015) also propagate the activation information in the backward direction to highlight important input features.

Original: Layer-wise Relevance Propagation (Bach et al., 2015) defines the relevance of neurons to recursively assign the attributions to the input.
Revised: Layer-wise Relevance Propagation (Bach et al., 2015) defines the relevance of neurons to recursively assign the attributions to the input.

Original: Similarly, DeepLIFT (Shrikumar et al., 2017) assigns attribution score to the input by comparing the relative contributions of features using a reference input.
Revised: Similarly, DeepLIFT (Shrikumar et al., 2017) assigns attribution scores to the input by comparing the relative contributions of features using a reference input.

Original: Integral-Based Methods: Integrated Gradients (IG) (Sundararajan et al., 2017) computes an integral for the input attribution along a linear path from the input to a reference representing the absence of the input feature.
Revised: Integral-Based Methods: Integrated Gradients (IG) (Sundararajan et al., 2017) computes an integral for the input attribution along a linear path from the input to a reference representing the absence of the input feature.

Original: However, the zero reference chosen by the IG makes it blind to black pixels.
Revised: However, the zero reference chosen by the IG makes it blind to black pixels.

Original: To overcome that, Expected Gradients (Erion et al., 2021) samples multiple references from the model’s training distribution.
Revised: To overcome that, Expected Gradients (Erion et al., 2021) samples multiple references from the model’s training distribution.

Original: IG-SG (Smilkov et al., 2017) and IG-SQ (Hooker et al., 2019) perturb input images with Gaussian noise as the references.
Revised: IG-SG (Smilkov et al., 2017) and IG-SQ (Hooker et al., 2019) perturb input images with Gaussian noise as the references.

Original: In addition, Adversarial Gradient Integral (Pan et al., 2021) adversarially perturbs the input to compute the desired reference.
Revised: In addition, Adversarial Gradient Integral (Pan et al., 2021) adversarially perturbs the input to compute the desired reference.

Original: Sturmfels et al.
Revised: Sturmfels et al.

Original: compared different references for the path attribution methods, demonstrating their weaknesses.
Revised: compared different references for the path attribution methods, demonstrating their weaknesses.

Original: Evaluation Metrics: A fair benchmarking is critical to establish the reliability of attribution methods.
Revised: Evaluation Metrics: A fair benchmarking is critical to establish the reliability of attribution methods.

Original: Pixel perturbation (Samek et al., 2016; Petsiuk et al., 2018; Srinivas & Fleuret, 2019) is one of the popular quantitative evaluation techniques, which removes the most or least salient pixels (as deemed by the attribution method) to note their effects on model output changes.
Revised: Pixel perturbation (Samek et al., 2016; Petsiuk et al., 2018; Srinivas & Fleuret, 2019) is one of the popular quantitative evaluation techniques, which removes the most or least salient pixels (as deemed by the attribution method) to note their effects on model output changes.

Original: It is operated on individual inputs, hence considered a local metric.
Revised: It is operated on individual inputs, hence considered a local metric.

Original: ROAR (RemOve And Retrain) (Hooker et al., 2019) and DiffROAR (Shah et al., 2021) are popular global metrics that utilize retraining over data distribution after removing the attributed pixels from the data.
Revised: ROAR (RemOve And Retrain) (Hooker et al., 2019) and DiffROAR (Shah et al., 2021) are popular global metrics that utilize retraining over data distribution after removing the attributed pixels from the data.

Original: Sensitivity-n (Ancona et al., 2018) is proposed to test the sensitivity of different feature groups while accounting for the attribution scores.
Revised: Sensitivity-n (Ancona et al., 2018) is proposed to test the sensitivity of different feature groups while accounting for the attribution scores.

Original: Adebayo et al.
Revised: Adebayo et al.

Original: (2018) also proposed sanity checks to verify reliability of the attribution maps computed by different methods.
Revised: (2018) also proposed sanity checks to verify reliability of the attribution maps computed by different methods.

Original: All these metrics evaluate different aspects of the attribution techniques.
Revised: All these metrics evaluate different aspects of the attribution techniques.

Original: We employ all of them for an extensive benchmarking.
Revised: We employ all of them for an extensive benchmarking.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: For classification, a machine learning model maps an input x =
Revised: For classification, a machine learning model maps an input x =

Original: [ x 1 , . .
Revised: [ x 1 , . .

Original: , x n ] ∈ R n to an output score
Revised: , x n ] ∈ R n to an output score

Original: S c ( x ) for class c .
Revised: S c ( x ) for class c .

Original: For explaining the model prediction, an attribution method M aims at attributing this score to the input features (pixels) by an attribution map M c ( x ) =
Revised: For explaining the model prediction, an attribution method M aims at attributing this score to the input features (pixels) by an attribution map M c ( x ) =

Original: [ M c 1 ( x ) , . . .
Revised: [ M c 1 ( x ) , . . .

Original: , M cn ( x )] .
Revised: , M cn ( x )] .

Original: Input Gradients:
Revised: Input Gradients:

Original: To explain model predictions, Input Gradients (Simonyan et al., 2014) computes model gradients with respect to the input x as the importance scores (attributions).
Revised: To explain model predictions, Input Gradients (Simonyan et al., 2014) computes model gradients with respect to the input x as the importance scores (attributions).

Original: Following the notations from above, for class c , the attribution of the i -th feature x
Revised: Following the notations from above, for class c , the attribution of the i -th feature x

Original: i calculated by this method is
Revised: i calculated by this method is

Original: [Equation]
Revised: [Equation]

Original: Simonyan et al.
Revised: Simonyan et al.

Original: (2014) arrived at this strategy by applying the first-order Taylor expansion to the non-linear model: S c ( x ) ≈ w T ∗ x + b .
Revised: (2014) arrived at this strategy by applying the first-order Taylor expansion to the non-linear model: S c ( x ) ≈ w T ∗ x + b .

Original: This allows the attribution of each input feature to be represented by the input gradient w i = ∂S c ( x ) /∂x i .
Revised: This allows the attribution of each input feature to be represented by the input gradient w i = ∂S c ( x ) /∂x i .

Original: However, due to the high non-linearity of deep models, the input gradients cannot provide reliable attributions.
Revised: However, due to the high non-linearity of deep models, the input gradients cannot provide reliable attributions.

Original: Ancona et al.
Revised: Ancona et al.

Original: (2018) also regarded the input gradients as a local attribution method that cannot provide a complete explanation for a specific prediction.
Revised: (2018) also regarded the input gradients as a local attribution method that cannot provide a complete explanation for a specific prediction.

Original: Input gradients can mainly indicate how to obtain the desired output by changing the features around the original input .
Revised: Input gradients can mainly indicate how to obtain the desired output by changing the features around the original input .

Original: This notion also has similarities to adversarial examples and other feature visualization methods (Goodfellow et al., 2015;
Revised: This notion also has similarities to adversarial examples and other feature visualization methods (Goodfellow et al., 2015;

Original: Carter et al., 2019).
Revised: Carter et al., 2019).

Original: Integrated Gradients:
Revised: Integrated Gradients:

Original: Different from the input gradients with local interpretability, integral-based attribution methods, e.g., Integrated Gradients (IG) (Sundararajan et al., 2017), calculate an integral over a path to estimate the attributions.
Revised: Different from the input gradients with local interpretability, integral-based attribution methods, e.g., Integrated Gradients (IG) (Sundararajan et al., 2017), calculate an integral over a path to estimate the attributions.

Original: Inspired by the cooperative game theory, IG chooses a reference to represent the complete absence of the input signals, which enables it to integrate gradients from the reference to estimate the attribution for the presence of the input features.
Revised: Inspired by the cooperative game theory, IG chooses a reference to represent the complete absence of the input signals, which enables it to integrate gradients from the reference to estimate the attribution for the presence of the input features.

Original: The IG calculates the attribution for the i -th input feature x i with the reference x ′ =
Revised: The IG calculates the attribution for the i -th input feature x i with the reference x ′ =

Original: [ x ′ 1 , . . .
Revised: [ x ′ 1 , . . .

Original: , x ′ n ] ∈ R n as follows
Revised: , x ′ n ] ∈ R n as follows

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where α represents the step along a linear path from the reference to the input.
Revised: where α represents the step along a linear path from the reference to the input.

Original: Since the integral of
Revised: Since the integral of

Original: Equation (2) satisfies the fundamental theorem of calculus, IG can satisfy the desirable completeness
Revised: Equation (2) satisfies the fundamental theorem of calculus, IG can satisfy the desirable completeness

Original: axiom, i.e., the output changes can be completely attributed to the input features: (cid:80) ni
Revised: axiom, i.e., the output changes can be completely attributed to the input features: (cid:80) ni

Original: =1 M ci ( x, x ′ ) =
Revised: =1 M ci ( x, x ′ ) =

Original: S c ( x ) − S c ( x ′ ) .
Revised: S c ( x ) − S c ( x ′ ) .

Original: In contrast to the input gradients, integral-based attribution methods are regarded as global attribution methods (Ancona et al., 2018), which can estimate the marginal effect from the reference to the input for model prediction.
Revised: In contrast to the input gradients, integral-based attribution methods are regarded as global attribution methods (Ancona et al., 2018), which can estimate the marginal effect from the reference to the input for model prediction.

Original: Existing Inconsistency: The primary assertion of the attribution methods is that the input features with larger attribution scores are more relevant to the model predictions.
Revised: Existing Inconsistency: The primary assertion of the attribution methods is that the input features with larger attribution scores are more relevant to the model predictions.

Original: However, the attributions computed by the back-propagation techniques suffer from frequent variations between positive and negative scores, even among the neighboring pixels of a (relatively) uniform image region.
Revised: However, the attributions computed by the back-propagation techniques suffer from frequent variations between positive and negative scores, even among the neighboring pixels of a (relatively) uniform image region.

Original: Thus, for plausible interpretations, these methods eventually use the absolute values of the attributions instead of the actual numerical scores (Sundararajan et al., 2017; Srinivas & Fleuret, 2019).
Revised: Thus, for plausible interpretations, these methods eventually use the absolute values of the attributions instead of the actual numerical scores (Sundararajan et al., 2017; Srinivas & Fleuret, 2019).

Original: However, considering the absolute values not only re-orders the computed pixel importances, it also contradicts the theoretical settings assumed to prove the desirable axiomatic properties of these techniques.
Revised: However, considering the absolute values not only re-orders the computed pixel importances, it also contradicts the theoretical settings assumed to prove the desirable axiomatic properties of these techniques.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In this section, we first assume a reliable attribution calculation with an automatically chosen reference.
Revised: In this section, we first assume a reliable attribution calculation with an automatically chosen reference.

Original: Then we propose a technique to estimate the proposed attribution, which enables us to reveal a reason for the inconsistency problem.
Revised: Then we propose a technique to estimate the proposed attribution, which enables us to reveal a reason for the inconsistency problem.

Original: Finally, the proposed technique is extended to popular attribution methods for calibrating their attributions within limited computational resources.
Revised: Finally, the proposed technique is extended to popular attribution methods for calibrating their attributions within limited computational resources.

Original: 4. P URSUING R ELIABLE A TTRIBUTION
Revised: 4. P URSUING R ELIABLE A TTRIBUTION

Original: The reference image in an integral-based method has the central importance.
Revised: The reference image in an integral-based method has the central importance.

Original: As noted earlier, the zero image of IG (Sundararajan et al., 2017) renders it blind to the black pixels in the input.
Revised: As noted earlier, the zero image of IG (Sundararajan et al., 2017) renders it blind to the black pixels in the input.

Original: The primary motivation behind using multiple references created with uniform noise (Sturmfels et al., 2020), Gaussian noise (Smilkov et al., 2017) or training samples (Erion et al., 2021), etc., is to avoid this blindness issue.
Revised: The primary motivation behind using multiple references created with uniform noise (Sturmfels et al., 2020), Gaussian noise (Smilkov et al., 2017) or training samples (Erion et al., 2021), etc., is to avoid this blindness issue.

Original: These methods create multiple references in the hope that some of those can correctly capture the abstract notion of feature absence, thereby resulting in a reliable attribution.
Revised: These methods create multiple references in the hope that some of those can correctly capture the abstract notion of feature absence, thereby resulting in a reliable attribution.

Original: Instead of hoping to accidentally stumble on the desired reference, we propose to systematically identify it.
Revised: Instead of hoping to accidentally stumble on the desired reference, we propose to systematically identify it.

Original: To that end, we can modify the input image with the model gradients to construct the reference.
Revised: To that end, we can modify the input image with the model gradients to construct the reference.

Original: Optimizing the input in the gradient ascending direction leads to local features that decrease the target prediction scores, providing a reasonable analogy of feature ‘absence’ in computational sense.
Revised: Optimizing the input in the gradient ascending direction leads to local features that decrease the target prediction scores, providing a reasonable analogy of feature ‘absence’ in computational sense.

Original: Such a reference is computed with respect to the model itself, instead of relying on an external operator, which can also ensure the model-fidelity of the eventually computed attribution scores.
Revised: Such a reference is computed with respect to the model itself, instead of relying on an external operator, which can also ensure the model-fidelity of the eventually computed attribution scores.

Original: We also emphasize that considering a uniform reference for all the input features is not ideal, as feature absence is a relative notion which is not handled well by uniformity assumption.
Revised: We also emphasize that considering a uniform reference for all the input features is not ideal, as feature absence is a relative notion which is not handled well by uniformity assumption.

Original: Hence, a set of references for an input is more desirable, which we also employ.
Revised: Hence, a set of references for an input is more desirable, which we also employ.

Original: Given a feature x i of the input x , we hypothesize a computable desired reference x ′ =
Revised: Given a feature x i of the input x , we hypothesize a computable desired reference x ′ =

Original: D i within a reference set D .
Revised: D i within a reference set D .

Original: Then, the attribution M ci of x i can be calculated by integrating the gradients along the integration path from the input x to the reference x ′ as
Revised: Then, the attribution M c i of x i can be calculated by integrating the gradients along the integration path from the input x to the reference x ′ as

Original: [Equation]
Revised: [Equation]

Original: where γ i ( · ) indicates a (possibly non-linear) integration path function defined by x and x ′ , with the step γ i ( α ) over a domain Ω ∈ R n , and ˜ x is the interpolation point that resides on this path.
Revised: where γ i ( · ) indicates a (possibly non-linear) integration path function defined by x and x ′ , with the step γ i ( α ) over a domain Ω ∈ R n , and ˜ x is the interpolation point that resides on this path.

Original: We note that our idea of computable reference along the gradient ascending direction has some similarity with the Adversarial Gradient Integration (Pan et al., 2021) which integrates gradients along the path from the input to class-specific adversarial examples.
Revised: We note that our idea of computable reference along the gradient ascending direction has some similarity with the Adversarial Gradient Integration (Pan et al., 2021) which integrates gradients along the path from the input to class-specific adversarial examples.

Original: However, in contrast to AGI, we allow different features to have different paths.
Revised: However, in contrast to AGI, we allow different features to have different paths to different references.

Original: This effectively enables integration along a non-linear path defined by the reference set.
Revised: This effectively enables integration along a non-linear path defined by the reference set.

Original: This notion is widely different from AGI, or any other existing method.

Original: 4. I NTEGRAL E STIMATION WITH D ESIRED I NTERPOLATIONS
Revised: 4. I NTEGRAL E STIMATION WITH D ESIRED I NTERPOLATIONS

Original: So far, we have only set our compass in the direction of pursuing reliable attributions.
Revised: So far, we have only set our compass in the direction of pursuing reliable attributions.

Original: Different from the linear path employed by the IG-like methods (Sundararajan et al., 2017; Erion et al., 2021), this pursuit requires integration of gradients along a non-linear path γ
Revised: Different from the linear path employed by the IG-like methods (Sundararajan et al., 2017; Erion et al., 2021), this pursuit requires integration of gradients along a non-linear path γ

Original: i , as per Equation (3).
Revised: i , as per Equation (3).

Original: To compute the required integral, the key is to identify the interpolation points that reside on the integration path from the desired reference x ′ to the input x (or vice versa).
Revised: To compute the required integral, the key is to identify the interpolation points that reside on the integration path from the desired reference x ′ to the input x (or vice versa).

Original: Assuming an interpolation point ˜ x to be in the same input space R n , it is possible to identify it systematically.
Revised: Assuming an interpolation point ˜ x to be in the same input space R n , it is possible to identify it systematically.

Original: Recall that, the gradient can induce a change in the input feature towards the desired reference, as discussed in the previous section.
Revised: Recall that, the gradient can induce a change in the input feature towards the desired reference, as discussed in the previous section.

Original: Given an input feature x i of input x , the signed input gradient within the set {∇ + x i , ∇ 0 x i , ∇ − x i } indicates the direction of the variation δ i from x i to its local desired reference x ′ =
Revised: Given an input feature x i of input x , the signed input gradient within the set {∇ + x i , ∇ 0 x i , ∇ − x i } indicates the direction of the variation δ i from x i to its local desired reference x ′ =

Original: D i along the gradient ascending direction.
Revised: D i along the gradient ascending direction.

Original: Thus, the gradient with respect to an interpolation point ˜ x residing on the path between the input and the reference, has the same sign as the variation δ i ∈ {
Revised: Thus, the gradient with respect to an interpolation point ˜ x residing on the path between the input and the reference, has the same sign as the variation δ i ∈ {

Original: δ + i , δ 

Revised: δ + i , δ

Original: i , δ − i } over the i -th feature of ˜ x .
Revised: i , δ − i } over the i -th feature of ˜ x .

Original: If the interpolation point does not reside on the desired path, the gradient ∇ ˜ x
Revised: If the interpolation point does not reside on the desired path, the gradient ∇ ˜ x

Original: i always has the opposite sign compared to δ i .
Revised: i always has the opposite sign compared to δ i .

Original: To exemplify, let an input feature x i have a positive gradient ∇ + x i , and this indicates x i should take a negative step towards its local desired reference ( x i − δ
Revised: To exemplify, let an input feature x i have a positive gradient ∇ + x i , and this indicates x i should take a negative step towards its local desired reference ( x i − δ

Original: + i = x ′ i ).
Revised: + i = x ′ i ).

Original: This implies that the variation δ i is positive, i.e., δ + i .
Revised: This implies that the variation δ i is positive, i.e., δ + i .

Original: As such, we can identify the interpolation point ˜ x that resides on our desired integration path by the product between the gradient ∇ ˜ x i and the variation δ i .
Revised: As such, we can identify the interpolation point ˜ x that resides on our desired integration path by the product between the gradient ∇ ˜ x i and the variation δ i .

Original: In light of the above discussion, we can integrate gradients over identified interpolations.
Revised: In light of the above discussion, we can integrate gradients over identified interpolations.

Original: If we can obtain the variation between x i and x ′ i , the required integral can be estimated.
Revised: If we can obtain the variation between x i and x ′ i , the required integral can be estimated.

Original: Defining the interpolations as a set ˜ X with m interpolations [ ˜ X 1 , . . 
Revised: Defining the interpolations as a set ˜ X = { ˜ x (1) , . . .

Original:  ˜ X m ] residing on the integration path, we can obtain the variation from x i to x ′ i by following Lemma 1.
Revised: , ˜ x ( m ) } with m interpolations residing on the integration path, we can obtain the variation from x i to x ′ i by following Lemma 1.

Original: Weintentionally use ˜ X j  instead of ˜ x here to stress that the interpolation points are in the image space, and ˜ X i,j is the i -th feature of ˜ X j  .
Revised: We use ˜ x ( j ) instead of ˜ x here to distinguish multiple interpolated image points, and ˜ x ( j ) i is the i -th feature of ˜ x ( j ) .

Original: Lemma 1.
Revised: Lemma 1.

Original: Given two  points x i and x ′ i , and n uniformly distributed interpolation points{ ˜ X
Revised: Given two feature points x i and x ′ i , and m uniformly distributed interpolation points ˜ x

Original: i, 1 , . . . ,
Revised: (1) i , . . . ,

Original: ˜ X i,m } between x i and x ′ i . The variation δ i
Revised: ˜ x ( m ) i between x i and x ′ i , the variation δ i

Original: =
Revised: =

Original: x i − x ′ i is proportional to the average distance from x i to all interpolation points
Revised: x i − x ′ i is proportional to the average distance from x i to all interpolation points

Original: ˜ X i,j , and the proportionality coefficient is 2.
Revised: ˜ x i , and the proportionality coefficient is 2.

Original: We provide proof of Lemma 1 in Apendix A.1.
Revised: We provide proof of Lemma 1 in Appendix A.1.

Original: Following Lemma 1, the attribution M i defined in
Revised: Following Lemma 1, the attribution M i defined in

Original: Equation (3) can be estimated over m interpolations as follows
Revised: Equation (3) can be estimated over m interpolations as follows

Original: [Equation]
Revised: [Equation]

Original: where λ is the proportionality coefficient, and δ i,j is the variation between x i and ˜ X i,j .
Revised: where λ is the proportionality coefficient, and δ ( j ) i is the variation between x i and ˜ x ( j ) i .

Original: The above insights are simple yet powerful.
Revised: The above insights are simple yet powerful.

Original: They can be leveraged to re-calibrate attribution scores computed by the existing path attribution methods.
Revised: They can be leveraged to re-calibrate attribution scores computed by the existing path attribution methods.

Original: Current integral-based attribution methods with multiple references, e.g., IG-SG, EG and AGI, employ different strategies to sample the references and integrate the gradients of interpolation points from the input to the references.
Revised: Current integral-based attribution methods with multiple references, e.g., IG-SG, EG and AGI, employ different strategies to sample the references and integrate the gradients of interpolation points from the input to the references.

Original: These points can be viewed as adopting different sampling strategies to approach the input space via Monte Carlo estimation (Erion et al., 2021).
Revised: These points can be viewed as adopting different sampling strategies to approach the input space via Monte Carlo estimation (Erion et al., 2021).

Original: This view allows us to utilize our technique with these attribution methods to modulate their interpolations for the integral estimation.
Revised: This view allows us to utilize our technique with these attribution methods to modulate their interpolations for the integral estimation.

Original: In the text to follow, we distinguish the interpolation points resulting from incorporating our insights into an attribution technique as valid interpolations.
Revised: In the text to follow, we distinguish the interpolation points resulting from incorporating our insights into an attribution technique as valid interpolations.

Original: In essence, current methods end up integrating attributions using interpolation points which are not valid, which causes them to compute sub-optimal attributions.
Revised: In essence, current methods end up integrating attributions using interpolation points which are not valid, which causes them to compute sub-optimal attributions.

Revised: [Figure]

Revised: Algorithm 1: input : Input x , reference set D , number of steps n .

Original: In Figure 2, the notion of integration using valid interpolation point is illustrated.
Revised: output: Attribution M i 1 initialize : M ← ∅ 2 for each x ′ in D do // Iterate references. for k ← 1 to n do // Integral path. [Equation]

Revised: ∇ x

Revised: [Equation]

Revised: [Equation]

Revised: x i − ˜ x i ) ≥ 0 then //

Revised: Integrate gradients with valid interpolations.

Revised: [Equation]

Revised: In Figure 2, the notion of integration using valid interpolation points is illustrated.

Original: The central idea of identifying the valid interpolations for any reference, followed by the integration with Equationis generic.
Revised: The central idea of identifying the valid interpolations for any reference, followed by the integration with Equationis generic.

Original: In the figure, for the input x , x ′ is the desired reference in a gradient ascending direction.
Revised: In the figure, for the input x , x ′ is the desired reference in a gradient ascending direction.

Original: The first step produces interpolation points to different references.
Revised: The first step produces interpolation points to different references.

Original: Then, the variations δ i,j and the corresponding gradients ∇ ˜ X
Revised: Then, the variations δ ( j ) i and the corresponding gradients ∇ ˜ x

Original: i,j are calculated in the second step.
Revised: ( j ) i are calculated in the second step.

Original: The red and blue colors indicate the positive and negative values in δ i,j and ∇ ˜ X
Revised: The red and blue colors indicate the positive and negative values in δ ( j ) i and ∇ ˜ x

Original: i,j whose sign change after crossing the input feature x i and the desired reference x ′ i separately.
Revised: ( j ) i whose sign change after crossing the input feature x i and the desired reference x ′ i separately.

Original: The variation and the gradient always have the same signs for the valid interpolation points along the integration path in the neighborhood of the input.
Revised: The variation and the gradient always have the same signs for the valid interpolation points along the integration path in the neighborhood of the input.

Original: This 
Revised: This observation is used to identify the valid interpolations.

Original: [Figure]
Revised: Finally, the attribution M i is computed over the identified valid interpolations following Equation 4.

Original: √ √ √
Revised: We also present the pseudo-code to re-calibrate the current integral-based methods with valid interpolations in Algorithm 1. Given a feature x i of the input x , and a reference set D identified by the method’s own strategy, we iterate over the references (Line 2). For each reference, the method computes the gradients of interpolated images ˜ x from the input to the reference (Lines 3-5).

Original: [Figure] × [Figure] observation is used to identify the  valid interpolations.
Revised: However, different from the existing practice, we let the integral to be estimated on valid interpolations that satisfy the condition that ∇ x i · ( x i − ˜ x i ) is positive (Line 6-8).

Original: Finally,  the attribution M i is computed over the identified valid interpolations following Equation 4.
Revised: Finally, we compute the integral with the averaging operation (Lines 9).

Original: 4. E FFICIENT A TTRIBUTION R E - CALIBRATION WITH V ALID R EFERENCES
Revised: 4. E FFICIENT A TTRIBUTION R E - CALIBRATION WITH V ALID R EFERENCES

Original: Sampling sufficient references and interpolation points allow us to estimate an accurate integral.
Revised: Sampling sufficient references and interpolation points allow us to estimate an accurate integral.

Original: However, limited computational resources must be accounted for by the attribution methods.
Revised: However, limited computational resources must be accounted for by the attribution methods.

Original: To estimate an accurate integral with limited resources, we employ importance sampling (Kloek &
Revised: To estimate an accurate integral with limited resources, we employ importance sampling (Kloek &

Original: Van Dijk, 1978).
Revised: Van Dijk, 1978).

Original: Let f ( · ) be a function of n inputs X
Revised: Let f ( · ) be a function of n inputs X

Original: = ( X 1 , . . .
Revised: = { x (1) , . . .

Original: , X n )  over the domain Ω ∈ R n .
Revised: , x ( n ) } over the domain Ω ∈ R n .

Original: Then, the integral is defined as v = (cid:82) Ω f ( X ) dX .
Revised: Then, the integral is defined as v = (cid:82) Ω f ( X ) dX .

Original: Importance sampling approaches this integral as
Revised: Importance sampling approaches this integral as

Original: [Equation]
Revised: [Equation]

Original: (5) where g ( X ) > 0 for X ∈ Ω is a probability function called the importance function.
Revised: where g ( X ) > 0 for X ∈ Ω is a probability function called the importance function.

Original: In order to estimate the integral, we generate n samples X i  from g ( X ) and estimate the integral by the following sample-mean formula.
Revised: In order to estimate the integral, we generate n samples x ( i ) from g ( X ) and estimate the integral by the following sample-mean formula.

Original: [Equation]
Revised: [Equation]

Original: where the estimator ˆ v can converge to the integral v .
Revised: where the estimator ˆ v can converge to the integral v .

Original: Instead of randomly sampling inputs in the input space, importance sampling enables us to sample input with the importance function for efficiently estimating the integral.
Revised: Instead of randomly sampling inputs in the input space, importance sampling enables us to sample input with the importance function for efficiently estimating the integral.

Original: Let f ( X ) /g
Revised: Let f ( X ) /g

Original: ( X ) be a function β ( X ) , we have the following Lemma.
Revised: ( X ) be a function β ( X ) , we have the following Lemma.

Original: Lemma 2.
Revised: Lemma 2.

Original: If the ratio between the function f ( X ) and β ( X ) has small fluctuations, the estimator ˆ v can be approached by the mean formula ˆ v = 1 n (cid:80) ni =1 β ( X i )  , where X i  , . . .
Revised: If the ratio between the function f ( X ) and β ( X ) has small fluctuations, the estimator ˆ v can be approached by the mean formula ˆ v = 1 n (cid:80) n i =1 β ( x ( i ) ) , where x ( i ) , . . .

Original: , X n  are sampled uniformly from g ( x ) .
Revised: , x ( n ) are sampled uniformly from g ( x ) .

Original: Let β ( X ) be defined on a linear integration path from the input to the reference.
Revised: Let β ( X ) be defined on a linear integration path from the input to the reference.

Original: If the underlying distribution of g ( X ) fluctuates slightly, we can estimate the integral along a linear path with β ( X ) .
Revised: If the underlying distribution of g ( X ) fluctuates slightly, we can estimate the integral along a linear path with β ( X ) .

Original: Since only slight fluctuations cannot be guaranteed along the full path in our case, we approach f ( X ) with β ( X ) · g ( X ) in different segments.
Revised: Since only slight fluctuations cannot be guaranteed along the full path in our case, we approach f ( X ) with β ( X ) · g ( X ) in different segments.

Original: Since the integral estimated from an input to its desired reference is also expected to have a positive attribution, we relax the condition of the integral estimated on the valid interpolations with ∇ · δ ≤ 0 to the valid reference that has a positive integral estimation for approaching a segment in f ( X ) .
Revised: Since the integral estimated from an input to its desired reference is also expected to have a positive attribution, we relax the condition of the integral estimated on the valid interpolations with ∇ · δ ≤ 0 to the valid reference that has a positive integral estimation for approaching a segment in f ( X ) .

Original: Then, we average positive integrations in k segments for estimating the final integral as
Revised: Then, we average positive integrations in k segments for estimating the final integral as

Original: [Equation]
Revised: [Equation]

Original: The integral estimation is also guided by Lemma 1 where λ
Revised: The integral estimation is also guided by Lemma 1.

Original: = 2 is the proportionality coefficient.

Original: Using the same number of references, the proposed technique enablescurrent attribution methods to calculate more precise attributions without incurring needless computational overhead.
Revised: Using the same number of references, the proposed technique enables attribution methods to calculate more precise attributions without incurring needless computational overhead.

Original: input : Input x , reference set D , number of steps n .

Original: output: Attribution M ∗ i 1 initialize : M i = 0 , ¯ g =2 for each x ′ in D do //

Original: Iterate references.

Original: for k ← 1 to n do // Integral path.

Original: ˜ x ←

Original: [Equation]

Original: kn

Original: [Equation]

Original: x − x ′ ) //

Original: Average gradients.

Original: [Equation]

Original: g ←

Original: [Equation]

Original: /n if ¯ g i · ( x i − x ′ i ) ≥ 0 then //

Original: Integrate effective reference. [Equation] [Equation] We present the pseudo-code of our technique to recalibrate the current integral-based methods in Algorithm 1.
Revised: In Appendix A.2, we provide the pseudo-code of our technique.

Original: Given a feature x i of the input x , and a reference set D identified by the method’s own strategy, we iterate over the references (Line 2).

Original: For each reference, the method integrates the gradients from the input to the reference (Lines 3-6).

Original: However, different from the existing practice, we let the integral to be estimated on valid references that satisfy the condition that ¯ g · ( x i − x ′ i ) is positive (Line 7).

Original: Finally, we approximate the integral with the averaging operation (Lines 8-10) in light of the theory presented above.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: To evaluate the proposed technique, we conduct experiments to test both performance and sensitivity.
Revised: To evaluate the proposed technique, we conduct experiments to test both performance and sensitivity.

Original: We take existing popular multi-reference attribution techniques as baseline methods; including IGSG (Smilkov et al., 2017), IG-SQ(Hooker et al., 2019), IG-Uniform (Sturmfels et al., 2020), EG (Pan et al., 2021) and AGI(Pan et al., 2021).IG-SG and IG-SQ define multiple references by adding
Revised: We take existing popular multi-reference attribution techniques as baseline methods; including IGSG (Smilkov et al., 2017), IG-SQ(Hooker et al., 2019), IG-Uniform (Sturmfels et al., 2020), EG (Pan et al., 2021) and AGI(Pan et al., 2021).

Original: Gaussian noise to the input whereas IG-SQ squares the gradients in the integral calculation.

Original: Similarly, IG-Uniform randomly samples references from uniform noise.

Original: EG samples training images as the reference to assume the underlying training distribution as the reference.

Original: AGI computes class-specific adversarial examples as references.

Original: All these methods enhanced with the proposed re-calibration technique are benchmarked in our experiments.
Revised: All these methods enhanced with the proposed re-calibration technique are benchmarked in our experiments.

Original: Additionally, other feature attribution methods, including InputGrad (Simonyan et al., 2014), Integrated Gradients (IG) (Sundararajan et al., 2017), SmoothGrad (SG) (Smilkov et al., 2017) and FullGrad (Srinivas & Fleuret, 2019) are also compared in our evaluation.
Revised: Additionally, other feature attribution methods, including InputGrad (Simonyan et al., 2014), Integrated Gradients (IG) (Sundararajan et al., 2017), SmoothGrad (SG) (Smilkov et al., 2017) and FullGrad (Srinivas & Fleuret, 2019) are also compared in our evaluation.

Original: We also provide more results on employing valid references and interpolations in Appendix A.3.1.
Revised: More detailed hyper-parameter settings and extended experimental results are provided in Appendices A.3 and A.4.

Original: 5. P IXEL P ERTURBATION P ERFORMANCE
Revised: 5. P IXEL P ERTURBATION P ERFORMANCE

Original: Pixel perturbation is a widely used quantitative evaluation metric for the attribution methods (Samek et al., 2016; Ancona et al., 2018; Sturmfels et al., 2020).
Revised: Pixel perturbation is a widely used quantitative evaluation metric for the attribution methods (Samek et al., 2016; Ancona et al., 2018; Srinivas & Fleuret, 2019; Yang et al., 2023).

Original: Since an attribution map is expected to identify the correct order of relative importance of the input features, pixel perturbation evaluation iteratively removes the most or the least salient pixels (Pixel Insertion and Deletion) and measures the output changes to quantify the performance of attribution methods.
Revised: Since an attribution map is expected to identify the correct order of relative importance of the input features, pixel perturbation evaluation iteratively removes the most or the least salient pixels (Pixel Insertion and Deletion) and measures the output changes to quantify the performance of attribution methods.

Original: Instead of separately testing pixel insertion or deletion, we combine the two metrics to provide a more comprehensive evaluation in a unified manner.
Revised: In this part, we employ DiffID metric

Original: Specifically, we test the fractional output change difference between the k % most salient pixels( k pixel insertion) and  1 − k % least salient pixels ( 1 − k pixel deletion).
Revised: (Yang et al., 2023) to measure the performance difference on images with inserted k % most salient pixels and deleted 1 − k % least salient pixels.

Original: Figure 3 reports the fractional output change difference with pixel deletion and insertion on ImageNet 2012 Validation set (Russakovsky et al., 2015) evaluated on ResNet-34 (He et al., 2016b) and 
Revised: Figure 3 reports the fractional output change difference with pixel deletion and insertion on ImageNet 2012 Validation set (Russakovsky et al., 2015) evaluated on ResNet-34 (He et al., 2016b) and [Figure] [Figure]

Original: VGG-16 (Simonyan & Zisserman, 2015).
Revised: VGG-16 (Simonyan & Zisserman, 2015).

Original: Higher values in these curves indicate better performance.
Revised: Higher values in these curves indicate better performance.

Original: The Absolute Value curves are obtained by allowing the methods to replace the negative attributions with their absolute values.
Revised: The Absolute Value curves are obtained by allowing the methods to replace the negative attributions with their absolute values.

Original: For the Vanilla values, we do not allow this operation.
Revised: For the Vanilla values, we do not allow this operation.

Original: The results are reported for five popular baseline multi-reference integral-based attribution methods.
Revised: The results are reported for five popular baseline multi-reference integral-based attribution methods.

Original: We can observe that the proposed re-calibration technique consistently improves all the baseline methods by[Figure] r a c t i o n a l o u t pu t c h a n g e d i ff e r e n c e
Revised: We can observe that the proposed re-calibration technique consistently improves all the baseline methods by a large margin.

Original: IG-Uniform

Original: IG-SG

Original: IG-SQ

Original: EG

Original: AGI

Original: [Figure]

Original: c t i o n a l o u t pu t c

Original: h a n g e d i ff e r e n c e

Original: [Figure]

Original: a large margin.

Original: It is also clear that the original methods heavily rely on taking the absolute value to provide explanations.
Revised: It is also clear that the original methods heavily rely on taking the absolute value to provide explanations.

Original: The proposed technique strongly improves IG-Uniform, IG-SG and AGI.
Revised: The proposed technique strongly improves IG-Uniform, IG-SG and AGI.

Original: The improvements for EG and IG-SQ are also reasonably large.
Revised: The improvements for EG and IG-SQ are also reasonably large.

Original: These results ascertain that our technique can enable an across-the-board performance gain for the existing methods.
Revised: These results ascertain that our technique can enable an across-the-board performance gain for the existing methods.

Original: In Figure 4a and 4b, we provide further comparisons of our enhanced baselines with the popular original exiting methods, including InputGrad, IG, SG and FullGrad.
Revised: In Figure 4a and 4b, we provide further comparisons on DiffID baseline with the popular original exiting methods, including InputGrad, IG, SG and FullGrad.

Original: A noticeable performance gap is clearly visible between these methods and the enhanced baselines, favoring the enhanced techniques.
Revised: A noticeable performance gap is clearly visible between these methods and the enhanced baselines, favoring the enhanced techniques.

Original: The results show that our method provides a notable positive off-set to the state-of-theart in integral-based attribution.
Revised: The results show that our method provides a notable positive off-set to the state-of-the-art in integralbased attribution.

Original: 5. R EMOVE AND R ETRAIN P ERFORMANCE
Revised: 5. R EMOVE AND R ETRAIN P ERFORMANCE

Original: Although pixel perturbation can provide an estimate of the interpretability of the attribution methods, Hooker et al.
Revised: Although pixel perturbation can provide an estimate of the interpretability of the attribution methods, Hooker et al.

Original: (2019) argued that pixel perturbations also cause a distribution shift for the inputs, and thus it is unclear whether the output change comes from removing informative pixels or the distribution shift.
Revised: (2019) argued that pixel perturbations also cause a distribution shift for the inputs, and thus it is unclear whether the output change comes from removing informative pixels or the distribution shift.

Original: To alleviate the effect of input distribution shift, RemOve and Retrain (ROAR) (Hooker et al., 2019) is a commonly used metric that retrains the model with the perturbed images produced by the pixel perturbation and measures the output change.
Revised: To alleviate the effect of input distribution shift, RemOve and Retrain (ROAR) (Hooker et al., 2019) is a commonly used metric that retrains the model with the perturbed images produced by the pixel perturbation and measures the output change.

Original: To provide a more comprehensive evaluations, DiffROAR (Shah et al., 2021) extends the ROAR metric to test the difference between the model retrained with the k % most and least salient pixels.
Revised: To provide a more comprehensive evaluation, DiffROAR (Shah et al., 2021) extends the ROAR metric to test the difference between the model retrained with the k % most and least salient pixels.

Original: To thoroughly establish the effectiveness of our technique, we also conduct experiments using DiffROAR.
Revised: To thoroughly establish the effectiveness of our technique, we also conduct experiments using DiffROAR.

Original: We again enhance the popular attribution methods IG-Uniform, IG-SG, IG-SQ, EG and AGI for this evaluation.
Revised: We again enhance the popular attribution methods IG-Uniform, IG-SG, IG-SQ, EG and AGI for this evaluation.

Original: We retrain the PreActResNet-18 (He et al., 2016a) on both CIFAR-100 and CIFAR-10 datasets (Krizhevsky et al., 2009). Figure 5 shows the accuracy change difference on the model retrained with the most and least important k % pixels separately. Since retraining models with different perturbed inputs causes these models to fit different input distributions, we can observe that the
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: A cc u r a c y c h a n g e d i ff e r e n c e 
Revised: r s o n C o rr e l a t i o n C o e ff

Original: IG-Uniform
Revised: i

Original: IG-SG IG-SQ
Revised: c

Original: EG AGI
Revised: i e n t

Original: [Figure] r s o n C o rr e l a t i o n
Revised: We retrain the PreActResNet-18 (He et al., 2016a) on both CIFAR-100 and CIFAR-10 datasets (Krizhevsky et al., 2009).

Original: C o e ff i
Revised: Figure 5 shows the accuracy change difference on the model retrained with the most and least important k % pixels separately.

Original: c i e n t curves in Figure 5 have different undulations as compared to the pixel perturbation experiments.
Revised: Since retraining models with different perturbed inputs causes these models to fit different input distributions, we can observe that the curves in Figure 5 have different undulations as compared to the pixel perturbation experiments.

Original: Nevertheless, the results establish that our method again generally improves the performance for all the methods.
Revised: Nevertheless, the results establish that our method again generally improves the performance for all the methods.

Original: We also summarize our quantitative evaluation using DiffROAR in Figure 4c and 4d for CIFAR-10 and CIFAR-100 datasets.
Revised: We also summarize our quantitative evaluation using DiffROAR in Figure 4c and 4d for CIFAR-10 and CIFAR-100 datasets.

Original: The figure compares the results with other existing methods which are used in their original form.
Revised: The figure compares the results with other existing methods which are used in their original form.

Original: These results also consistently favor our technique of enhancing the methods.
Revised: These results also consistently favor our technique of enhancing the methods.

Original: 5. S ENSITIVITY - N P ERFORMANCE & F URTHER RESULTS
Revised: 5. S ENSITIVITY - N P ERFORMANCE & F URTHER RESULTS

Original: Ancona et al.
Revised: Ancona et al.

Original: (2018) proposed Sensitivity-n metric to verify that the model output variations are sensitive to the features as prescribed by the attribution method.
Revised: (2018) proposed Sensitivity-n metric to verify that the model output variations are sensitive to the features as prescribed by the attribution method.

Original: Given any subset of features x S =
Revised: Given any subset of features x S =

Original: [ x 1 , . . .
Revised: [ x 1 , . . .

Original: , x m ] ⊆ x , Sensitivity-n requires the sum of attributions M i to hold (cid:80) mi =1 M i =
Revised: , x m ] ⊆ x , Sensitivity-n requires the sum of attributions M i to hold (cid:80) mi =1 M i =

Original: S c ( x ) − S c ( x [ x S =0] ) .
Revised: S c ( x ) − S c ( x [ x S =0] ) .

Original: Although no method can be expected to fully satisfy Sensitivity-n due to practical reasons, the definition still enables a pragmatic benchmarking scheme.
Revised: Although no method can be expected to fully satisfy Sensitivity-n due to practical reasons, the definition still enables a pragmatic benchmarking scheme.

Original: By varying the feature fraction in the subsets in the range [0.1, 0.9] of the total features, we test the Pearson Correlation
Revised: By varying the feature fraction in the subsets in the range [0.1, 0.9] of the total features, we test the Pearson Correlation

Original: Coefficient (PCC) computed between the sum of the attributions and the target output variation, shown in Figure 6.
Revised: Coefficient (PCC) computed between the sum of the attributions and the target output variation, shown in Figure 6.

Original: For each feature subset, we sample 100 different subsets of the input features with a uniform probability distribution.
Revised: For each feature subset, we sample 100 different subsets of the input features with a uniform probability distribution.

Original: The PCC is averaged over 1000 testing images of ImageNet 2012 Validation set.
Revised: The PCC is averaged over 1000 testing images of ImageNet 2012 Validation set.

Original: Compared with the vanilla attribution map, we can observe that the proposed re-calibration generally shows better sensitivity-n results for all the baseline methods.
Revised: Compared with the vanilla attribution map, we can observe that the proposed re-calibration generally shows better sensitivity-n results for all the baseline methods.

Original: In Appendix
Revised: In Appendix

Original: A.3.3, we provide more results for the Sensitivity-n metric.
Revised: A.4.3, we provide more results for the Sensitivity-n metric.

Original: There, we also demonstrate that our re-calibration does not cause sanity check failures (Adebayo et al., 2018).

Original: Additionally, we also conclusively establish that the proposed re-calibration is also able to enhance model interpretability through attribution prior (Erion et al., 2021).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We identified an inconsistency between the theoretical treatment of path integral-based attribution and its practical implementation in various methods.
Revised: We identified an inconsistency between the theoretical treatment of path integral-based attribution and its practical implementation in various methods.

Original: We address this by devising a scheme that re-calibrates attribution computation by identifying a desired reference and selecting only valid interpolation points for integration.
Revised: We address this by devising a scheme that re-calibrates attribution computation by identifying a desired reference and selecting only valid interpolation points for integration.

Original: This scheme is applicable to all integral-based attribution methods.
Revised: This scheme is applicable to all integral-based attribution methods.

Original: Enhancing a range of multi-reference methods with it, we showed considerable performance gain on multiple evaluation metrics.
Revised: Enhancing a range of multi-reference methods with it, we showed considerable performance gain on multiple evaluation metrics.

Original: We also demonstrated sanity preservation and improved model robustness by using the proposed scheme as attribution prior regularization.
Revised: We also demonstrated sanity preservation and improved model robustness by using the proposed scheme as attribution prior regularization.

Original: E THIC S TATEMENT
Revised: E THIC S TATEMENT

Original: Explaining the model prediction is highly relevant to the safety and privacy of users.
Revised: Explaining the model prediction is highly relevant to the safety and privacy of users.

Original: In this paper, we improve the back-propagation based feature attribution methods.
Revised: In this paper, we improve the back-propagation based feature attribution methods.

Original: The improved attribution methods need to fully access the model and inputs to produce explanations.
Revised: The improved attribution methods need to fully access the model and inputs to produce explanations.

Original: Therefore, it is critical to carefully protect the safety of the training data and trained models for practical applications.
Revised: Therefore, it is critical to carefully protect the safety of the training data and trained models for practical applications.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: The detailed experimental setup including the hyper-parameter choice of compared attribution methods on different datasets with the specific experimental framework and platform is mentioned for reproducing our experiments in Appendix A.2.
Revised: The detailed experimental setup including the hyper-parameter choice of compared attribution methods on different datasets with the specific experimental framework and platform is mentioned for reproducing our experiments in Appendix A.3.

Original: In addition, the implementation code can be found in the supplementary material
Revised: In addition, the implementation code can be found in the supplementary material

Revised: A CKNOWLEDGMENT

Revised: This research was supported by ARC Discovery Grant 190102443.

Revised: Professor Ajmal Mian is the recipient of an Australian Research Council Future Fellowship Award (project number FT210100268) funded by the Australian Government.

Revised: Dr. Naveed Akhtar is the recipient of Office of National

Revised: Intelligence, National Intelligence Postdoctoral Grant (project number NIPG-2021-001) funded by the Australian Government.

Original: A A PPENDIX C ONTENTS A Appendix
Revised: A A PPENDIX C ONTENTS A Appendix

Original: A.1 Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.1 Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Revised: A.2 Pseudo-code to Re-Calibration Attribution with Valid References . . . . . . . . . .

Original: A.2 Experimental Setup . ..
Revised: A.3 Experimental Setup . .

Original: A.2.1 Hyperparameter Choice . . .. . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.3.1 Hyperparameter Choice . . .

Original: A.2.2 Experimental Software and Platform . . ..
Revised: A.3.2 Experimental Software and Platform . . .

Original: A.3 Extended Experiments . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.4 Extended Experiments . .

Original: A.3.1 Extended Experiments on Valid Interpolations and Valid References . . . .
Revised: A.4.1 Extended Experiments on Valid Interpolations and Valid References . . . .

Original: A.3.2 Extended Experiments on Attribution Prior . . . . . . . . . . . . . . . . .
Revised: A.4.2 Extended Experiments on Attribution Prior . . . . . . . . . . . . . . . . .

Original: A.3.3 Extended Experiments on Sensitivity-n . . . . . . . . . . . . . . . . . . .
Revised: A.4.3 Extended Experiments on Sensitivity-n . . . . . . . . . . . . . . . . . . .

Original: A.3.4 Extended Experiments on Sanity Checks . . . . . . . . . . . . . . . . . .
Revised: A.4.4 Extended Experiments on Sanity Checks . . . . . . . . . . . . . . . . . .

Original: A.4 Visual Inspection . .  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: A.5 Visual Inspection . . A.6 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: A. P ROOF
Revised: A. P ROOF

Original: In this section, we provide the proof of Lemma 1 and Lemma 2.
Revised: In this section, we provide the proof of Lemma 1 and Lemma 2.

Original: Proof of Lemma 1.
Revised: Proof of Lemma 1.

Original: Given two points x i and x ′ i , n interpolations ˜ X  i  uniformly distributed between x i and x ′ i .
Revised: Given two points x i and x ′ i , n interpolations ˜ X = { 1 n ( x i − x ′ i ) , . . . , nn ( x i − x ′ i ) } uniformly distributed between x i and x ′ i .

Original: The average distance ¯ d from x i to all interpolation points˜ X i is calculated as follows.
Revised: The average distance ¯ d from x i to all interpolation points is calculated as follows.

Original: [Equation]
Revised: [Equation]

Original: Then, the limit of 1+ n 2 n ( x i − x ′ i ) , as the number n of interpolations approaches positive infinity, is 12 ( x i − x ′ i ) , donated as follows.
Revised: Then, the limit of 1+ n 2 n ( x i − x ′ i ) , as the number n of interpolations approaches positive infinity, is 12 ( x i − x ′ i ) , donated as follows.

Original: [Equation]
Revised: [Equation]

Original: Thus, the distance δ = x
Revised: Thus, the distance δ = x

Original: i − x ′ i between x i and x ′ i is proportional to the the average distance ¯
Revised: i − x ′ i between x i and x ′ i is proportional to the the average distance ¯

Original: d
Revised: d

Original: with coefficient 2 donated as δ = 2 ¯ d .
Revised: with coefficient 2 donated as δ = 2 ¯ d .

Original: Proof of Lemma 2.
Revised: Proof of Lemma 2.

Original: If the ratio f ( x ) /β
Revised: If the ratio f ( x ) /β

Original: ( x ) fluctuates slightly, the importance function g ( x ) from which we sample the inputs can be approximatively viewed as a uniform probability density function.
Revised: ( x ) fluctuates slightly, the importance function g ( x ) from which we sample the inputs can be approximatively viewed as a uniform probability density function.

Original: Therefore, the importance scores β ( X 1 ) , . . . ,
Revised: Therefore, the importance scores β ( X 1 ) , . . . ,

Original: β ( X n ) can be calculated with the uniformly sampled inputs X 1 , . . .
Revised: β ( X n ) can be calculated with the uniformly sampled inputs X 1 , . . .

Original: , X n as follows.
Revised: , X n as follows.

Original: [Equation]
Revised: [Equation]

Original: Thus, the β ( X i ) can be used to estimate the integral ˆ v .
Revised: Thus, the β ( X i ) can be used to estimate the integral ˆ v .

Original: A.  E XPERIMENTAL S ETUP
Revised: A. P SEUDO - CODE TO R E -C ALIBRATION A TTRIBUTION WITH V ALID R EFERENCES Algorithm

Revised: [Table] , number of steps n . output

Revised: [Equation]

Revised: 1 initialize : M ← ∅ , ¯ g ←2 for each

Revised: x

Revised: in

Revised: D do //

Revised: Iterate references.

Revised: for k ← 1 to n do // path.

Revised: [Equation]

Revised: // Compute the average gradient.

Revised: [Equation]

Revised: x

Revised: [Equation]

Revised: x ′ i ) ≥ 0 then

Revised: [Equation]

Revised: Integrate effective reference

Revised: [Equation]

Revised: In this section, we present the pseudo-code of our technique to re-calibrate the integral-based methods with valid references in Algorithm 2.

Revised: Given a feature x i of the input x , and a reference set D identified by the method’s own strategy, we iterate over the references (Line 2).

Revised: For each reference, the method computes an average ¯ g of gradients with respect to the interpolated image ˜ x

Revised: from the input x to the reference x ′ (Lines 3-6).

Revised: Different from the existing methods, we let the integral to be estimated on the valid references that satisfy the condition that ¯ g · ( x i − x ′ i ) is positive (Lines 7-9).

Revised: Finally, we approximate the integral with the averaging operation (Line 10).

Revised: As such, We can recalibrate the integral-based methods by identifying valid references with valid references without the additional computational overhead.

Revised: A. E XPERIMENTAL S ETUP

Original: In this section, we provide details of the experimental setup, including the hyperparameter choice and the experimental software and platform.
Revised: In this section, we provide details of the experimental setup, including the hyperparameter choice and the experimental software and platform.

Original: A.2. H YPERPARAMETER C HOICE
Revised: A.3. H YPERPARAMETER C HOICE

Original: Hyperparameter Choice of Attribution Methods.
Revised: Hyperparameter Choice of Attribution Methods.

Original: In our experiments, we employ five popular integral-based attribution methods as our baseline methods including IG-Uniform Sturmfels et al. (2020), IG-SG Smilkov et al. (2017), IG-SQ Hooker et al. (2019), EG Erion et al.
Revised: In our experiments, we employ five popular integral-based attribution methods as our baseline methods including IG-Uniform (Sturmfels et al., 2020), IG-SG (Smilkov et al., 2017), IG-SQ (Hooker et al., 2019), EG (Erion et al., 2021) and

Original: (2021) and
Revised: AGI (Pan et al., 2021).

Original:  AGI Pan et al.
Revised: Specifically, IG-Uniform defines multiple by sampling references from uniform noise whereas IG-SQ squares the gradients in the integral calculation. Similarly, IG-SG adds Gaussian noise to the input as references. EG samples training images as the reference to assume the underlying training distribution as the reference. AGI computes class-specific adversarial examples as references.

Original: For the experiments on the ImageNet-2012 dataset Russakovsky et al. , we select 10 references and 5 interpolations (k=5) for IG-Uniform, IG-SG and IG-SQ.
Revised: For the experiments on the ImageNet-2012 dataset (Russakovsky et al., 2015), we select 10 references and 5 interpolations (k=5) for IG-Uniform, IG-SG and IG-SQ.

Original: Besides, we chose 50 references and one random interpolation (k=1) for EG, and set 10 interpolations (k=10) and 5 class-specific adversarial references for AGI as the recommended settings in these papers.
Revised: Besides, we chose 50 references and one random interpolation (k=1) for EG, and set 10 interpolations (k=10) and 5 class-specific adversarial references for AGI as the recommended settings in these papers.

Original: For a fair comparison, we ensure all these methods use the same number of 50 back propagations.
Revised: For a fair comparison, we ensure all these methods use the same number of 50 back propagations.

Original: Considering the small input size of CIFAR-10 and CIFAR-100 datasets Krizhevsky et al. (2009), we employed 30 back propagations for all the baseline attribution methods.
Revised: Considering the small input size of CIFAR-10 and CIFAR-100 datasets (Krizhevsky et al., 2009), we employed 30 back propagations for all the baseline attribution methods.

Original: We employed the same hyperparameter settings for all the experiments in this paper (e.g., with pixel perturbation,DiffROAR,
Revised: We employed the same hyperparameter settings for all the experiments in this paper (e.g., with pixel perturbation,

Original:  Sensitivity-n, etc.).
Revised: DiffROAR, Sensitivity-n, etc.).

Original: In addition to the baseline methods, other compared methods that rely on multiple gradient computations including Integrated Gradients (IG) and SmoothGrad (SG). They are also implemented with the same hyperparameters.
Revised: In addition to the baseline methods, other compared methods that rely on multiple gradient computations including Integrated Gradients (IG) and SmoothGrad (SG) are also implemented with the same hyperparameters.

Original: In IG, we produce 50 interpolations (k=50) for the final integral estimation.
Revised: In IG, we produce 50 interpolations (k=50) for the final integral estimation.

Original: In SG, we average 50 gradients on inputs with Gaussian noise.
Revised: In SG, we average 50 gradients on inputs with Gaussian noise.

Original: Hyperparameter Choice of Explained Models.
Revised: Hyperparameter Choice of Explained Models.

Original: Three deep models including VGG-16, ResNet34 and PreActResNet-18 are chosen to be explained by different attribution methods in our experiments.
Revised: Three deep models including VGG-16, ResNet34 and PreActResNet-18 are chosen to be explained by different attribution methods in our experiments.

Original: For the pixel perturbation and Sensitivity-n benchmarks, VGG-16 and ResNet-34 networks are trained on the ImageNet-2012 training dataset.
Revised: For the pixel perturbation and Sensitivity-n benchmarks, VGG-16 and ResNet-34 networks are trained on the ImageNet-2012 training dataset.

Original: For the DiffROAR benchmark, we first train a PreActResNet-18 on CIFAR-10 and CIFAR-100 training datasets separately.
Revised: For the DiffROAR benchmark, we first train a PreActResNet-18 on CIFAR-10 and CIFAR-100 training datasets separately.

Original: Then, the
Revised: Then, the

Original: PreActResNet-18 is fine-tuned for a total of 10 epochs with the initial learning rate 10 − 2 decayed by 10 on the 5-th and 7-th epochs.
Revised: PreActResNet-18 is fine-tuned for a total of 10 epochs with the initial learning rate 10 − 2 decayed by 10 on the 5-th and 7-th epochs.

Original: A.2. E XPERIMENTAL S OFTWARE AND P LATFORM
Revised: A.3. E XPERIMENTAL S OFTWARE AND P LATFORM

Original: All the experiments were conducted on a Linux machine with an NVIDIA GTX 3090Ti GPU with 24GB memory and a 16-core 3.9GHz Intel Core i9-12900K CPU and 125GB main memory.
Revised: All the experiments were conducted on a Linux machine with an NVIDIA GTX 3090Ti GPU with 24GB memory and a 16-core 3.9GHz Intel Core i9-12900K CPU and 125GB main memory.

Original: All attribution methods are tested and trained on PyTorch deep learning framework (v1.12.1) with Python language.
Revised: All attribution methods are tested and trained on PyTorch deep learning framework (v1.12.1) with Python language.

Original: A. E XTENDED E XPERIMENTS
Revised: A. E XTENDED E XPERIMENTS

Original: Below extended experiments are conducted to test the efficacy of the proposed technique.
Revised: Below extended experiments are conducted to test the efficacy of the proposed technique.

Original: A.3. E XTENDED E XPERIMENTS ON V ALID I NTERPOLATIONS AND V ALID R EFERENCES
Revised: A.4. E XTENDED E XPERIMENTS ON V ALID I NTERPOLATIONS AND V ALID R EFERENCES

Original: In this paper, we propose two techniques to identify valid interpolations and references for estimating an integral along a non-linear integration path.
Revised: In this paper, we propose two techniques to identify valid interpolations and references for estimating an integral along a non-linear integration path.

Original: Here, we provide more detailed experiments for further understanding the valid interpolations and references.
Revised: Here, we provide more detailed experiments for further understanding the valid interpolations and references.

Original: We randomly sample 1,000 testing images from the ImageNet 2012 Validation set to show the effectiveness of both valid interpolations and references.
Revised: We randomly sample 1,000 testing images from the ImageNet 2012 Validation set to show the effectiveness of both valid interpolations and references.

Original: Here, we take IG-SG as the baseline attribution method.
Revised: Here, we take IG-Uniform as the baseline attribution method.

Original: On both ResNet-34 and VGG-16, we re-calibrate attribution maps produced by IG-SG with valid interpolations and references.
Revised: On both ResNetand VGG-16, we re-calibrate attribution maps produced by IG-Uniform with valid interpolations and references.

Original: In Figure 7 and 8, we show the AUC of the fractional output change difference under different numbers of references with a fixed number of interpolations (k=5).
Revised: In Figure 7 and 8, we show the AUC of the fractional output change difference under different numbers of references with a fixed number of interpolations (k=5).

Original: Both valid references and interpolations can re-calibrate attributions.
Revised: Both valid references and interpolations can re-calibrate attributions.

Original: In comparison with the experimental results of the vanilla attribution map produced by IG-SG in Figure 3, Figure 7 and Figure 8 show that both valid interpolations and references can improve the interpretability of IGSG by a large margin.
Revised: In comparison with the experimental results of the vanilla attribution map produced by IG-Uniform in Figure 3, Figureand Figure 8 show that both valid interpolations and references can improve the interpretability of IG-Uniform by a large margin.

Original: The significant improvement demonstrates the effectiveness of the assumed desired reference and the proposed re-calibration technique.
Revised: The significant improvement demonstrates the effectiveness of the assumed desired reference and the proposed re-calibration technique.

Original: Valid References lead to efficient integral estimation.
Revised: Valid References lead to efficient integral estimation.

Original: Different from valid interpolations, valid references employ importance sampling to efficiently estimate the integration.
Revised: Different from valid interpolations, valid references employ importance sampling to efficiently estimate the integration.

Original: Figure 7a shows that integrating with valid references enables the attribution method to achieve high performance with  a small number of references.
Revised: Figure 7a shows that integrating with valid references enables the attribution method to achieve high performance with [Figure] [Figure] a small number of references.

Original: However, estimating an accurate integral with valid interpolations requires a large number of references.
Revised: However, estimating an accurate integral with valid interpolations requires a large number of references.

Original: More references lead to high performance.
Revised: More references lead to high performance.

Original: Figures 7b&7c and Figure 8b&8c shows the relationship between the AUC of fractional output change difference and the number of references.
Revised: Figures 7b&7c and Figure 8b&8c shows the relationship between the AUC of fractional output change difference and the number of references.

Original: We can observe that more references and interpolations can lead to high performance by re-calibrating with both valid interpolations and references, which demonstrates that more references can cover a more accurate integration path.
Revised: We can observe that more references and interpolations can lead to high performance by re-calibrating with both valid interpolations and references, which demonstrates that more references can cover a more accurate integration path.

Original: Drawbacks and superiority of valid references.
Revised: Drawbacks and superiority of valid references.

Original: In Figure8, we can observe that IG-SG recalibrated with valid references cannot outperform the results calibrated with valid interpolations on VGG-16, which is inconsistent with the results on ResNet-34.
Revised: In Figure we can observe that IG-Uniform re-calibrated with valid references cannot outperform the results calibrated with valid interpolations on VGG-16, which is inconsistent with the results on ResNet-34.

Original: The reason is that the valid reference relies on a relaxed condition to approach an accurate integral.
Revised: The reason is that the valid reference relies on a relaxed condition to approach an accurate integral.

Original: However, VGG-16 has a more complicated decision pathway, which is hard to estimate an accurate integral along a linear integration path.
Revised: However, VGG-16 has a more complicated decision pathway, which is hard to estimate an accurate integral along a linear integration path.

Original: As compared to VGG-16, ResNet-34 with a more flattened decision pathway enables attribution methods to obtain accurate results with valid references, which also aligns with the existing works that show the residual block can flatten the loss landscape (Li et al., 2018).
Revised: As compared to VGG-16, ResNet-34 with a more flattened decision pathway enables attribution methods to obtain accurate results with valid references, which also aligns with the existing works that show the residual block can flatten the loss landscape (Li et al., 2018).

Original: Although a complicated model impedes the performance of valid references, valid references can still improve the reliability of baseline methods.
Revised: Although a complicated model impedes the performance of valid references, valid references can still improve the reliability of baseline methods.

Original: Moreover, identifying valid references require less memory consumption than identifying valid interpolations.
Revised: Moreover, identifying valid references require less memory consumption than identifying valid interpolations.

Original: Thus, the efficiency of identifying valid references enables the attribution method incorporated into the model training process.
Revised: Thus, the efficiency of identifying valid references enables the attribution method incorporated into the model training process.

Original: In addition, training a robust model can flatten the loss landscape of the model, which also favors the proposed valid references for robust model training.
Revised: In addition, training a robust model can flatten the loss landscape of the model, which also favors the proposed valid references for robust model training.

Original: In Appendix A.3.2, we perform experiment of regularizing the behavior of the model with feature attribution.
Revised: In Appendix A.4.2, we perform experiments of regularizing the behavior of the model with feature attribution.

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: A.3. E XTENDED E XPERIMENTS ON A TTRIBUTION P RIOR
Revised: A.4. E XTENDED E XPERIMENTS ON A TTRIBUTION P RIOR

Original: In addition to explaining a model prediction, attribution methods are also expected to regularize the behavior for constructing a robust model.
Revised: In addition to explaining a model prediction, attribution methods are also expected to regularize the behavior for constructing a robust model.

Original: Therefore, many regularization terms (Ross et al., 2017;
Revised: Therefore, many regularization terms (Ross et al., 2017;

Original: Erion et al., 2021) are proposed to regularize the behavior by the attributions for obtaining a robust model.
Revised: Erion et al., 2021) are proposed to regularize the behavior by the attributions for obtaining a robust model.

Original: Erion et al.
Revised: Erion et al.

Original: (2021) assert a prior for the image classification task that adjacent features or pixels should have similar attributions for the model prediction.
Revised: (2021) assert a prior for the image classification task that adjacent features or pixels should have similar attributions for the model prediction.

Original: To regularize the model behavior with this prior knowledge, they employed a Laplace zero-mean prior on the difference between attributions of neighborhood pixels to encourage the adjacent pixels to have similar attributions.
Revised: To regularize the model behavior with this prior knowledge, they employed a Laplace zero-mean prior on the difference between attributions of neighborhood pixels to encourage the adjacent pixels to have similar attributions.

Original: Since the regularization term encourages the input pixels to make the prediction jointly the trained model is robust to the input with Gaussian noise.
Revised: Since the regularization term encourages the input pixels to make the prediction jointly, the trained model is robust to the input with Gaussian noise.

Original: In this part we also combine the prior with the re-calibrated attributions as the regularization term for training a robust model.
Revised: In this part, we also combine the prior with the re-calibrated attributions as the regularization term for training a robust model.

Original: We train two robust models with the regularization term on MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2009) datasets.
Revised: We train two robust models with the regularization term on MNIST (LeCun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2009) datasets.

Original: For MNIST, we train a multi-layer perceptron (MLP) with two convolution layers and one fully connected layer.
Revised: For MNIST, we train a multi-layer perceptron (MLP) with two convolution layers and one fully connected layer.

Original: On CIFAR-10 dataset, we take a PreActResNet-18 (He et al., 2016a) as the baseline network.
Revised: On CIFAR-10 dataset, we take a PreActResNet-18 (He et al., 2016a) as the baseline network.

Original: On both datasets, we employ one interpolation (k=1) and 5 references randomly sampled from the training dataset in the training process.
Revised: On both datasets, we employ one interpolation (k=1) and 5 references randomly sampled from the training dataset in the training process.

Original: For each dataset, we compare the original model, the model trained with EG regularization and the model trained with re-calibrated EG regularization to test their robustness on the test set with Gaussian noise.
Revised: For each dataset, we compare the original model, the model trained with EG regularization and the model trained with re-calibrated EG regularization to test their robustness on the test set with Gaussian noise.

Original: Figure 9 shows the accuracy change with the increase of the standard deviation of Gaussian noise on the three models.
Revised: Figure 9 shows the accuracy change with the increase of the standard deviation of Gaussian noise on the three models.

Original: Although two robust models (EG and EG Ours) can improve the robustness of the baseline model on Gaussian noised input, we can observe that our method can outperform the regularization term with Expected
Revised: Although two robust models (EG and EG Ours) can improve the robustness of the baseline model on Gaussian noised input, we can observe that our method can outperform the regularization term with Expected

Original: Gradients (EG) on both two datasets, which demonstrates that the proposed re-calibration technique can effectively enhance the reliability of EG.[Figure] [Figure]
Revised: Gradients (EG) on both two datasets, which demonstrates that the proposed re-calibration technique can effectively enhance the reliability of EG.

Original: A.3. E XTENDED E XPERIMENTS ON S ENSITIVITY - N
Revised: A.4. E XTENDED E XPERIMENTS ON S ENSITIVITY - N

Original: In this part, we provide more experimental results on Sensitivity-n metric.
Revised: In this part, we provide more experimental results on Sensitivity-n metric.

Original: Figure 10 shows the results of ResNet-34 on Sensitivity-n, which shows that the proposed technique can also improve the sensitivity of the model on ResNet-34.
Revised: Figure 10 shows the results of ResNet-34 on Sensitivity-n, which shows that the proposed technique can also improve the sensitivity of the model on ResNet-34.

Original: In addition, Figure 11 shows the results of PreActResNet18 on CIFAR-10 and CIFAR-100 datasets on Sensitivity-n metric.
Revised: In addition, Figure 11 shows the results of PreActResNet18 on CIFAR-10 and CIFAR-100 datasets on Sensitivity-n metric.

Original: These experimental results also ascertain that our re-calibration technique enables the vanilla methods to improve their sensitivity under smaller sampling set sizes.
Revised: These experimental results also ascertain that our re-calibration technique enables the vanilla methods to improve their sensitivity under smaller sampling set sizes.

Original: Since Sensitivity-n benchmark randomly removes pixels from the input to test their sensitivity, the re-calibrated attribution maps with well-aligned explanations for the foreground object are more robust to the random removal than the vanilla attribution maps. 
Revised: Since Sensitivity-n benchmark randomly removes pixels from the input to test their sensitivity, the re-calibrated attribution maps with well-aligned explanations for the foreground object are more robust to the random removal than the vanilla attribution maps. [Figure] r s o n C o rr e l a t i o n C

Revised: o e ff

Revised: i

Original: A.3. E XTENDED E XPERIMENTS ON S ANITY C HECKS
Revised: c i e n t [Figure] a r s o n C o

Revised: rr e l a t i o n

Revised: C o e ff i

Revised: c i e n t

Revised: [Figure]

Revised: A.4. E XTENDED E XPERIMENTS ON S ANITY C HECKS

Original: In addition to the quantitative evaluation benchmarks, Adebayo et al.
Revised: In addition to the quantitative evaluation benchmarks, Adebayo et al.

Original: (2018) proposed two sanity checks that attribution methods should pass including: (i) the method should provide different attributions for the model trained with randomly permuted labels, and (ii) the attribution map should show a substantial change to the model parameter randomization.
Revised: (2018) proposed two sanity checks that attribution methods should pass including: (i) the method should provide different attributions for the model trained with randomly permuted labels, and (ii) the attribution map should show a substantial change to the model parameter randomization.

Original: In this part, we test the baseline methods and their re-calibrated variants for the two sanity checks.
Revised: In this part, we test the baseline methods and their re-calibrated variants for the two sanity checks.

Original: Data randomization test.
Revised: Data randomization test.

Original: Since deep models can easily fit random labels Zhang et al.
Revised: Since deep models can easily fit random labels Zhang et al.

Original: (2017), data randomization test permutes the training labels to train a model and test the sanity of attribution methods.
Revised: (2017), data randomization test permutes the training labels to train a model and test the sanity of attribution methods.

Original: In Figure 12, we compare the attribution methods for the models trained with true and random labels.
Revised: In Figure 12, we compare the attribution methods for the models trained with true and random labels.

Original: The first four columns show the comparison between the attribution maps cali- [Figure] P e a r s o n C o
Revised: The first four columns show the comparison between the attribution maps calibrated with our technique and taken absolute values.

Original: rr

Original: e l a t i o n

Original: C o e ff i

Original: c i e n t

Original: [Figure]

Original: brated with our technique and taken absolute values.

Original: The second four columns show the comparison between re-calibrated attribution maps and the vanilla attribution maps.
Revised: The second four columns show the comparison between re-calibrated attribution maps and the vanilla attribution maps.

Original: Compared with the baseline methods, our method leads to a larger change in the calculated attribution maps.
Revised: Compared with the baseline methods, our method leads to a larger change in the calculated attribution maps.

Original: In addition, we quantitatively compare the similarity between the attribution maps trained with true and random labels.
Revised: In addition, we quantitatively compare the similarity between the attribution maps trained with true and random labels.

Original: Figure 13 shows the comparison of Spearman rank correlation between the attribution maps produced by the baseline methods and re-calibrated methods.
Revised: Figure 13 shows the comparison of Spearman rank correlation between the attribution maps produced by the baseline methods and re-calibrated methods.

Original: Compared with both absolute attributions and vanilla attributions, the experimental results show that the proposed re-calibration technique improves the sanity of the baseline attribution methods for the data randomization test.
Revised: Compared with both absolute attributions and vanilla attributions, the experimental results show that the proposed re-calibration technique improves the sanity of the baseline attribution methods for the data randomization test.

Original: Model parameter randomization test
Revised: Model parameter randomization test

Original: In addition to the sanity of the data randomization, the attribution methods are also expected to pass the model parameter randomization test.
Revised: In addition to the sanity of the data randomization, the attribution methods are also expected to pass the model parameter randomization test.

Original: In our experiments, we successively randomize the weights of an Inception V3 model (Szegedy et al., 2016) from top to bottom layers.
Revised: In our experiments, we successively randomize the weights of an Inception V3 model (Szegedy et al., 2016) from top to bottom layers.

Original: Figures 14 - 17 show the examples of the attributions maps as the parameter cascading from the top to bottom.
Revised: Figures 14 - 17 show the examples of the attributions maps as the parameter cascading from the top to bottom.

Original: We mark attribution maps produced by the baseline methodand our re-calibration method with blue boxes for easy comparison. We can observe that the calibrated attribution maps are more sensitive with the increased randomization layer numbers, which shows the proposed technique enables the baseline methods to improve the sensitivity for the model parameter randomization test. A. V ISUAL I NSPECTION In this section, we provide visualization examples of calculated attribution maps for visual inspection.
Revised: We mark attribution maps produced by the baseline method

Original: Figures 18 - 20 show the comparison of vanilla attribution maps, attribution maps computed with absolute values and the attribution maps re-calibrated with our technique on ResNet-34.

Original: In addition, Figures 21 - 23 show the comparison of attribution maps on VGG-16.

Original: We can observe that the attributions re-calibrated with our technique enable the maps to better align with the foreground objects.

Original: In addition, vanilla attribution maps suffer from a large variation between positive and negative scores on both ResNet-34 and VGG-16.

Original: [Figure]
Revised: [Figure]

Original: RandomLabels
Revised: RandomLabels

Original: [Figure]
Revised: [Figure]

Original: TrueLabels
Revised: TrueLabels

Original: RandomLabels
Revised: RandomLabels

Original: True Labels
Revised: True Labels

Original: Random Labels
Revised: Random Labels

Original: True Labels
Revised: True Labels

Original: Random Labels
Revised: Random Labels

Original:  Ours and Absolute Values 
Revised: Values Ours and Vanilla Values and our re-calibration method with blue boxes for easy comparison. We can observe that the calibrated attribution maps are more sensitive with the increased randomization layer numbers, which shows the proposed technique enables the baseline methods to improve the sensitivity for the model parameter randomization test. A. V ISUAL I NSPECTION

Original: Ours and Vanilla Values
Revised: In this section, we provide visualization examples of calculated attribution maps for visual inspection. Figures 18 - 20 show the comparison of vanilla attribution maps, attribution maps computed with absolute values and the attribution maps re-calibrated with our technique on ResNet-34. In addition, Figures 21 - 23 show the comparison of attribution maps on VGG-16.

Original:  [Figure] 
Revised: We can observe that the attributions re-calibrated with our technique enable the maps to better align with the foreground objects. In addition, vanilla attribution maps suffer from a large variation between positive and negative scores on both ResNet-34 and VGG-16. [Figure] A. L IMITATIONS

Revised: In this paper, to improve the inconsistency of integral-based attribution methods, we assume different references for different input features to re-calibrate attributions.

Revised: Although our method has been shown to enhance the reliability of popular attribution methods, the modification also carries the risk of violating the completeness axiom.

Revised: Therefore, exploring an explainable method that guarantees the consistency between practice and their theoretical guarantee and their practical implementation is still an important task.

Revised: In addition, we devise our method based on theoretical insights.

Revised: However, precisely quantifying the impact of these insights is not possible due to the intrinsic ambiguities in the quantitative evaluation metrics used in this research direction.

Revised: This is a limitation of this domain in general.

Revised: Nevertheless, we verify that our method consistently improves the evaluation scores.

Revised: In practice, it may be important to verify that the used evaluation metric represents the real-world scenario appropriately.

Original: O r i g i n a l
Revised: O r i g i n a l

Original: F C
Revised: F C

Original: M i x e d _7 c
Revised: M i x e d _7 c

Original: M i x e d _7a
Revised: M i x e d _7a

Original: M i x e d _6 e
Revised: M i x e d _6 e

Original: M i x e d _6a
Revised: M i x e d _6a

Original: M i x e d _5 d M i x e d _5 b
Revised: M i x e d _5 d M i x e d _5 b

Original: [Figure]
Revised: [Figure]

Revised: SG

Original: C o n v2 d _ 4a_3x
Revised: C o n v2 d _ 4a_3x

Original: C o n v2 d _ 3 b _1x C o n v2 d _ 2 b _3x
Revised: C o n v2 d _ 3 b _1x C o n v2 d _ 2 b _3x

Original: C o n v2 d _ 2a_3x C o n v2 d _ 1a_3x
Revised: C o n v2 d _ 2a_3x C o n v2 d _ 1a_3x

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Revised: i

Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: O r i g i n a l
Revised: O r i g i n a l

Original: F C
Revised: F C

Original: M i x e d _7 c
Revised: M i x e d _7 c

Original: M i x e d _7a
Revised: M i x e d _7a

Original: M i x e d _6 e
Revised: M i x e d _6 e

Original: M i x e d _6a
Revised: M i x e d _6a

Original: M i x e d _5 d M i x e d _5 b
Revised: M i x e d _5 d M i x e d _5 b

Original: [Figure]
Revised: [Figure]

Revised: SG

Original: C o n v2 d _ 4a_3x
Revised: C o n v2 d _ 4a_3x

Original: C o n v2 d _ 3 b _1x C o n v2 d _ 2 b _3x
Revised: C o n v2 d _ 3 b _1x C o n v2 d _ 2 b _3x

Original: C o n v2 d _ 2a_3x C o n v2 d _ 1a_3x
Revised: C o n v2 d _ 2a_3x C o n v2 d _ 1a_3x

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: i

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]