Start Smart: Leveraging Gradients For Enhancing Mask-based XAI Methods
Authors: Buelent Uendes
Shujian Yu
Mark Hoogendoorn
Venue: ICLR 2025 Poster (2025)
Keywords: XAI
mask-based explanations
rate-distortion explanation
information bottleneck
Primary area: interpretability and explainable AI

Abstract: Mask-based explanation methods offer a powerful framework for interpreting deep learning model predictions across diverse data modalities, such as images and time series, in which the central idea is to identify an instance-dependent mask that minimizes the performance drop from the resulting masked input. Different objectives for learning such masks have been proposed, all of which, in our view, can be unified under an information-theoretic framework that balances performance degradation of the masked input with the complexity of the resulting masked representation. Typically, these methods initialize the masks either uniformly or as all-ones.
In this paper, we argue that an effective mask initialization strategy is as important as the development of novel learning objectives, particularly in light of the significant computational costs associated with existing mask-based explanation methods. To this end, we introduce a new gradient-based initialization technique called StartGrad, which is the first initialization method specifically designed for mask-based post-hoc explainability methods. Compared to commonly used strategies, StartGrad is provably superior at initialization in striking the aforementioned trade-off. Despite its simplicity, our experiments demonstrate that StartGrad enhances the optimization process of various state-of-the-art mask-explanation methods by reaching target metrics faster and, in some cases, boosting their overall performance.

Published as a conference paper at ICLR 2025
START SMART: LEVERAGING GRADIENTS FOR ENHANCING MASK-BASED XAI METHODS
Buelent Uendes, Shujian Yu & Mark Hoogendoorn Vrije Universiteit Amsterdam The Netherlands {b. uendes (ondees)vvu.... @vu.n1
ABSTRACT
Mask-based explanation methods offer a powerful framework for interpreting deep learning model predictions across diverse data modalities, such as images and time series, in which the central idea is to identify an instance-dependent mask that minimizes the performance drop from the resulting masked input. Dif- ferent objectives for learning such masks have been proposed, all of which, in our view, can be unified under an
-tthoricc framework that balances performance degradation of the masked input with the complexity of the result- ing masked representation. Typically, these methods initialize the masks either uniformly as all-ones. In this paper, we argue that effective mask initial- ization strategy is as important as the development of novel learning objectives, particularly in light of the significant computational costs associated with existing mask-based explanation methods. To this end, we introduce new gradient- gradient-based initialization technique called StartGrad, which the first initialization method specifically designed for mask-based post-hoc explainability methods. Compared to commonly used strategies, StartGrad is provably superior initialization striking the aforementioned trade-off. Despite its simplicity, our experiments demonstrate that StartGrad enhances the optimization process of various state-of- the-art mask-explannn explaaxtlnnnnoon methods by reaching target metrics faster and, in some cases, boosting their overall performance.
INTRODUCTION
As machine learning models become deeply integrated into critical areas such as healthcare or medicine, the need for transparency and interpretability grows ever more urgent (Vellido) (020). Explainable AI (XAI) research has responded by developing various XAI frameworks, with saliency methods-as also referred to feature attribution methods at the forefront. These methods seek to highlight the most relevant inputs that drive a model's prediction, offering insights into how complex models, often regarded as black boxes, make decisions.
Mask-based explanation methods, a particularly powerful subset of feature attribution techniques, aim learn sparse masks that highlight the key inputs driving a prediction. These methods are highly flexible: the objective function can be designed to capture specific notions of relevance, while constraints like sparsity or smoothness can be added meet criteria for 'good' explanation. This is often formalized through an information- -ieeorrtttt framework, such as the information bottleneck (IB) principle (Tishby al. 1999) and the rate- distorriitiioon function (Thomas & Joy (006). The care- fully designed objective function in mask-based methods allows them to adapt effectively different data modalities, resulting superior performance compared to approaches such as Saliency maps (Simonyan et al.) (2014), Integrated Gradients (Sundararajan et al. 2017). SmoothGrad (Smilkov) 2016) and SHAP (Lundberg & Lee 2017). These approaches have been also shown produce explanations that fragile and
ct al. 2017). or surrogate-based methods like LIME [Ribeiro et al.
noisy, or even ones that can be manipulated (Adebayo et al. 2018: Ghorbani et al. 2019 Slack
et al. 2020).
Despite their effectiveness, state-of-the bassd explanation methods come with heavy com- putational costs due their prolonged optimization process with hundreds of epochs of iterations,

---

Published as a conference paper at ICLR 2025
especially when compared to maadient-based saliency techniques like SmoothGrad (Smilkov al. 2017) or Integrated Gradients (Sundararajan et al. 2017]. For instance, the most advanced mask- based methods in the vision domain such as the recently proposed WaveletX (Kolek et al. 2023) and ShearletX (Kolek et al. 2023), require orders of magnitude more execution time to generate explanations than their gradient-based counterparts.
This trade-off between performance and speed presents a major limitation for mask-based methods, especially in time-sensitive applications where rapid decisions are critical. As result, users are often forced to choose between the superior faithfulness of mask-based explanations and the faster, but potentially less reliable, gradient- based alternatives. This tension between explanation accuracy and real-time applicability remains a key challenge in making mask- based methods more widely applicable in high-stakes environments such as healthcare.
To address the challenge of balancing performance and computational efficiency in mask-based methods, we draw on two key insights: First, while the way how initialize masks in existing mask-based XAI methods is usually neglected, it plays crucial role in optimization in terms of both running time and the final achievable maximum or minimum value. Second, although gradient-based saliency methods are not explicitly designed to meet desiderata of high-quality explanations, they do provide valuable signals about the model's 's decision-making process with minimal computational overhead. By combining these two insights, initialization technique specifically designed for post-hoe explanation methods. StartGrad leverages gradient signals to provide provably superior initialization masks in terms of minimal distortion and sparsity two essential criteria for mask-based explanation methods-compare to commonly used strategies. By doing so, StartGrad harnesses the strengths of gradient-based approaches to enhance existing mask-based explainability techniques.
propose StartGrad, novel gradient-basel mask
We summarize our contributions below:
We introduce StantGra,, a novel gradient-based mask initialization algorithm grounded in the rate distortion explanation (RDE) framework (Macdonald et al.) (201)), representing the first initialization technique explicitly designed to enhance the performance of mask-based explanation methods.
We prove that StartGrad is superior at initialization compared to other initialization strate- gies in reducing distortion and improving sparsity-two essential criteria for effective mask-based explanations.
Extensive experiments vision and time-series tasks demonstrate that StartGrad enables state-of-the-art methods like ShearletX (Kolek et al. (002)) and ExtremalMask (Enguehard, 2023) to reach target metrics faster while also improving overall performance in some cases.
To the best of our knowledge, this work presents the first comprehensive theoretical and empirical analysis of mask initialization techniques across both vision and time-series do- mains, providing critical insights for improving mask-based explanation methods.
2 RELATED WORK
Feature attribution methods can be divided into white-box, gray-box, and black-box approaches, depending on the amount of information required generate an explanation (Muzellec al. 2024). Mask-based explanation methods are considered black-box attribution techniques, as they do not require access to the model's internal architecture and rely only on model predictions. Recently, these methods gained attraction in the community and have been subsequently applied to different
domains such as vision (Fong & Vedaldi 2017 Petsiuk et al.] 2018 Kolek et al. 2022 2023)
time series (Crabbé & Van Der Schaar 2021 Enguchard 2023 Liu et al.) 2024 Zichuan Liu, 2024). key advantage that formalizes desiderata for "good" explanations, such as parsimony and fidelity to the model and often formalized via the IB principle (Tishby et al.) 1999) or the RDE framework (Macdonald et al. 2019) which formalize the trade-off between explanation complexity and predictive accuracy.
these techniques is their explicit optimization of an objective function
Despite their strong performance, to iterative optimization, limiting their practicality in real-world applications. However, much of
approaches are often computationally expensive due
2

---

Published as a conference paper at ICLR 2025
the previous work has mainly investigated designing novel objective functions suited for either the data modality at hand (Ying et al., 2019, Crabbé Van Der Schaar 2021). proposing new architec- tural design choices such learning the perturbation Junction(Engucheee
2023 Liu et al. 2024). or even learning a trainable masking model that can produce masks after training (Dabkowski & Gal 2017). To the best of our knowledge, no previous work has investigated the role of mask ini- tialization scheme itself, which is surprising given the role initialization schemes can have on the overall performance of the optimization result (Glorot & Bengio 2010, He al. 2015). In fact, we noticed that across data modalities, different initialization techniques are used. For instance, mask-based XAI methods in the time-series domain initialize their masks uniformly at start (Crabbé & Van Der Schaar 2021 Enguehard [2023] Liu et al. 2024 whereas vision explanation models
use an all-ones initialization scheme (Kolek et al. 2022 2023).
BACKGROUND
Different objectives have been developed to learn an "optimal" mask m. Based on our observa- tion, these objectives can be unified under the same umbrella, formalizing the trade-off between the per-
complexity of the masked input and its predictive performance from an information-therreet spective. At their core, all methods revolve around these two fundamental concepts, with additional constraints, such as smoothness, often incorporated to further refine the explanations.
Given a pre-trained predictive model & Rd + [0, 1] for a total of c classes and an input instance Rd, the goal of the mask-based post-hoc explanation to a black-box output (x(x) is identify a sparse mask m E {0, 1)dd over x, such that the resulting perturbed input x leads to the minimum performance drop in the model's prediction (Fong & Vedaldi 2017 Macdonald et al.) (019).. Usu-
ally, X == m @ x + (1 - m) @ u, where u E RR is random noise following predefined probability distribution V, such Gaussian distribution.
3.1 THE AATE-DISTORTIC EXPLANATION (RDE) FRAMEWORK
Formally, the objective of the RDE framework can be defined as the following constrained optimiza-
tion problem (Macdonald al. 2019 Kolek et al. 2022):
min mE (
Ear [D (x), 2 (x)) s.t. |mllo S.
(I)
where D : [0, 1Je x 10, + R4 quantifies the performance difference between the classifier's prediction for the original input and the perturbed input X, term which is also referred to as the "distortion". s E {1...., d} controls the sparsity of the mask m. The mean-squared error loss function is often used as the distortion measure D
In practice, the RDE optimization framework in equation is relaxed using both a continuous mask m [0. 1]d and || ||1 norm:
min EV-V D ($) (x)) + A/|m|||, mE[0,1]
which can be optimized using gradient-based methods. Moreover, instead of applying the RDE framework the original input domain, one can first transform x using an invertible function F
Bd + R², such as Wavelet transform (Kolek et al., (202)) and Shearlet transform (Kolck et al. 2023 and enforce sparsity in the frequency domain. In this scenario, the resulting objective becomes:
min En-v [D ( @ F(x) I]{
m) 3 u)), [x(x) + A||m]|ı-
3.2 THE INFORMATION BOTTLENECK (IB) EXPLANATION FRAMEWORK
An alternative, yet increasingly popular, objective for learning the mask is based on the IB prin- ciple (Tisbyy et al. 1999 Gilad Bachrach et al. 2003): :
min - -T x) (I(x; mc(0,1)
Strictly speaking, these methods start with constant value of 0.5 which the expected value of an uniform distribution 1). PI fact, Kolek et al. (022)) report that the choice of the distortion function has limited effect on the perfor- mance of their proposed mask-basee vision explanation methods.
3

---

Published as a conference paper at ICLR 2025
where -): represents the mutual information between two random variables, 9 +x(() is the black-box output, / > O is a Lagrange multiplier. Maximizing I [(:: x) ensures that the perturbed input X retains sufficient information to approximate explain the black-box output ý (Bang al.) 2021). while minimizing [x; x) encourages compression such that x does not capture any redun- dant information in that is irrelevant explain In other words, the compression terms essentially encourages the conditional independence between X and x, conditioning on (Fischer 2020).
Same to the RDE framework, the mask can be applied in latent space (Schulz et al. 2020 Demir et al. 2021). In this case, the compression terms becomes I (z, z @ m + +1 m) u), in which
(x) refers the [-th layer output of an instance X.
Recently, the general idea of IB has been extended to the design of built-in interpretable deep learn- ing architectures for image data (Choi et al., 2024) and graphs (Yu al. (02T). where ý replaced by the ground-truth label However, our paper remains focused on post-hoc explanations.
From our perspective, there is a clear resemblance between equation 4 and the RDE objective in equation 2 as illustrated in Proposition and Remark
Proposition 1 Maximizing [ú;; x) equivalent to minimizing the expected Kullback Leibler (KL) divergence E (Dxx(" (xxx); (x(x))), under the choice of KL divergence as the distortion measure the RDE framework. Furthermore, the expected KL. divergence provides an upper bound for the $e(x))li..
( llss, i..,, E (DKI. (D.()) +c(x))) 2 Tlog2 F
(1)+c())
Remark In the IB implementation. the compression term I T(x: X) can simply be replaced by an entropy term H(x (Strouse & Schwab 2017: Kirsch et al. 2020). In fact, the simplest way to compress H (5) by encouraging the number of explanatory variables be small, i.e., encouraging
m to be sparse (Bang et al. 2027 Tao et al. 2020). This regularization can be expressed as [|m
or approximated with [m]]1
Proof. All proofs can be found in Appendix A
4 METHOD
In this section, we present StartGrad, our novel gradient-based mask initialization algorithm. Unlike traditional approaches, StartGrad leverages gradient information to provide tailored initialization of masks based on the specific characteristic of the sample and model. This gradient-informel initialization not only provide more effective starting point initialization in light of the previously discussed RDE and IB framework, but can also significantly boost the performance of subsequent mask-based explanation algorithms will show in section
4.1 APPROXIMATING THE DISTORTION LOCALLY
Given a differentiable classifier e.. we can use a first-order Taylor expansion to approximate the prediction for the distorted input ž:
(5) where Vx represents the gradient of the model's prediction with respect to the original input feature x and Ax = x denotes the distortion induced due to the mask m. To ensure that this approximation accurate, we require that & locally linear in a small neighborhood around x. Formally, this neighborhood is defined as the open ball B, (a) = {@ E Rd | æ1 < e). where e <> 0 ensures that higher-order terms in the Taylor expansion of &. are negligible.
(x) R (x + (x). Ax
Using || .Ip norm choice for the distortion function D, can leverage equation 5 to approx- imate the distortion:
D (2), (x) (x)) ~ | (x) Ax|1p
Σ
V 2x Azil") S
IIf the mapping from x to & is deterministic, we always have this equivalence; however, this may not hold true in practice, such when unmasked regions of image are substituted with random Gaussian noise.
4

---

Published as a conference paper at ICLR 2025
From the approximation in equation requires controlling the term V &e
it follows that minimizing the distortion D (#)( (x))
Axlp p- In particular, we can leverage the gradient infor-
mation to the minimize the distortion Features with small gradient magnitudes vV 2x(x) have less impact the distortion term and can be therefore masked without significantly increasing D. This insight leads to the following heuristic:
M. = {1 if Ve,(e(() is "large". if Va, (x(x) is "small".
Since n., (x(x) indicates the sensitivity of the model's prediction changes in the corresponding feature, masking features with "small"" gradients ensures that any induced changes minimally impact the model prediction. This approach is crucial maintaining the predictive performance while modifying or explaining the input, effectively balancing sparsity and distortion.
4.2 GRADIENT-BASED MASK INITIALIZATION
key challenge in our approach is setting appropriate thresholds distinguish between "large" and "small" gradients when defining the mask in equation 7 As discussed in section 3.1 we avoid binary threshold by leveraging continuous masks. Specifically, we define the absolute gradient of the model's prediction with respect to the class of interest as:
(x = @b
(8)
Ox
For multi-channel inputs such as RGB images, we take the maximum value across channels. We then apply transformation function R + [0, 11 which normalizes the gradient signal Sx(x)
into the range [0, 17s. The primary challenge lies in selecting a suitable transformation function that preserves the relative importance of each feature.
Empirically, we observe that the absolute gradient values follow a highly skewed distribution, similar to Laplace distributior (see Appendix D). Given this skewness, we employ the Quantile Transfor- mation Function (QTF), which maps the gradient values a uniform distribution 2/ 1). The QTF has two key ad advantages: (1) it handles the skewness in the data effectively, and (2) it preserves the relative feature importance based on the absolute gradient values. Furthermore, using the QTF facil- itates direct comparison with the standard uniform mask initialization scheme, thereby simplifying gives pseudocode for our gradient-base based
the evaluation of our proposed framework. Algorithm [ initialization method which call StartGrad
Algorithm 1: Gradient- Mask Initialization (StarGGrad) Input
Pre-trained classifier + input samples {a:) with æ; E Rd, quantile transformation T Rd + [0, 1]d
Hyperparameters:
Output distribution (0, 1), number of quantiles (bins) for T Masks M [0, 1| X
Output
Initialize mask list M : and quantile transformer T; 2 for i + to N do
arg max o (x.)) // Class prediction
S, - V (xx))) 1. // Gradient magnitudes
m; < T(S.) // Quantile transform
Append m; to M;
end for return M
4 "An alternative strategy trivially set Ax Oa. i.e. 1.,. thereby keeping all features and eliminat- ing distortion entirely. However, this solution fails to satisfy the sparsity objective of the RDE framework and thus suhoptimal for the overall task.
This behavior has been previously observed in other domains well, such as time series Ismail et al.
2020 and graph-based models (Xie al.) 2022)
For simplicity, we assume for the pseudocode in algorithm I that we operate in the same input space X. In
Appendix B more general version provided where we we invertible and differentiable function F.
visualization of StartGrad is also provided in the Appendix, it was moved there due space constraints.
5

---

Published as a conference paper at ICLR 2025
4.3 BALANCING SPARSITY AND DISTORTION UNDER THE RDE FRAMEWORK
As outlined in section 3.1 the primary objective of mask-based explanation algorithms is typically identify sparse mask that minimally distorts the model's predictions While our previous dis- cussion in 4.1 has centered on minimizing distortion at initialization, we now turn our attention to sparsity. comparing the standard initialization strategies of all-ones and uniform to our newly in- troduced algorithm StartGrad. Specifically. we provide formal proof' that StartGrad is provably superior at initialization in balancing the trade-off between distortion and sparsity, as formalized in the RDE framework, when compared to other mask initialization strategies. To the best of our knowledge, this represents the first formal comparison different mask initialization methods.
Proposition 2 Let Dx(x) with &e Rd + 10, 1le be classifier that differentiable and locally linear in the neighborhood of an input x E R.. Formally, this neighborhood is defined as the open ball (x) = {ā R - -- x|| < e}. where e ensures that higher- terms in the Taylor expansion of D. are negligible. Additionally, let Mynif represent the mniformly-initialinn
mask and ggrd the mask initialized based on the gradient of (x(x) with respect to x, transformed using quantile function so that Mgrad E [0, 1ja. Then, given norm || lpp with p z as a choice for the distortion function D, the following inequality holds at initialization:
E Emgnt v V [D (&o( (grad); (x(x)) Al[lggaa [|] Emagauy [D (Xmii) 2x(x)) + Al[manngll i]
with Xunif = Munif ④ X + (1 Monif) ② u and &grad = Migrad c x + (1 Mgrad) O u, where E Rd is random perturbation drawn a predefined distribution and À is hyperparameter encouraging sparsity in the masks.
Let (x)x) with be : [0. 1] be classifier and let Mones denote the mask initialized with all ones, i.e. Mones la where 1a is a vector of ones with dimension and mgrad denote the mask initialized based the gradient of (x(x) with respect to x, transformed using a quantile function so that mgod E []. Given a norm 1 with 2 as a choice for the distortion function D, the following inequality holds at initialization:
Proposition 3
E (D (Reg)))) (x(x)) + Al|mgradlh]] S E (D (Xones))) Fc(x)) + Mln [11]
for
2 21 + (2-22111 1) SA
with Rones = Mones + (1 Mones) ) C u and &gnad Mgrad O (1 Mgnd) O u, where u E Rd is random perturbation drawnfrom a predefined distribution and À is hyperparameter encouraging sparsity in the masks.
Proof. All proofs can be found in Appendix
Remark 2 In case we want to explain the class prediction of classifier (x(x) for a specific cate- gory, i.e. Proposition reduces
2
Remark Proposition holds in the same manner for the comparison between manif and Mones-
Proposition 2 and 3 together provide principled approach for selecting initialization strategy within the RDE framework. Our proposed gradient-basee initialization algorithm yields an expected loss in terms of distortion and sparsity that less than or equal that of uniform initialization. Additionally, StartGrad outperforms all-ones initialization in high-dimensional settings (where d is large) which is a typical setting for deep learning models, and consequently, mask-based explanation approaches.
In addition sparsity and distortion, some mask- based algorithms also regularize the smoothness of the mask (Fong & Vedaldi 2017). Furthermore, rather than minimizing distortion, certain algorithms aim to find the smallest mask that induces the maximum distortion (Enguehard 2023).

---

Published as a conference paper at ICLR 2025
5 EXPERIMENTS
In this section, we evaluate our gradient-based initialization scheme using state-of-the-ar mask- based explanation methods across two challenging data modalities: vision and time series Vision is domain where mask-based explanation methods are well-established, while time-series data, though less studied in the XAI community, is critical in fields like healthcare and finance, gaining more attention in recent years (Crabbé & Van Der Schaar 2021 Enguehard [2023 Liu et al. 2024
Queen et al. 2023; Zichuan Liu 2024). These domains present distinct challenges for gradient estimation Vision involves spatial complexity, while time-series data introduces temporal depen- dencies (Ismail et al. 2020). This cross-domain evaluation demonstrates how StartGrad adapts to varied data characteristies and model architectures, underscoring the robustness and potential of our proposed initialization algorithm.
5.1
VISION
We evaluate StartGrad using two state-offthe-art mask-based explanation methods: WaveletX, ShearletX (Kolek et al. 2023) and a well-established third method, f.e. PixelMask (Fong & Vedaldi 2017). These vision methods were chosen as they apply masking in different input spaces and vary in their objective formulations, creating a diverse and challenging testbed for our proposed initial- ization method.
For quantitative evaluation, we use the conciseness-precisensss (CP) Pixel and LI scores introduced by Kolek et al. (2023), calculated on 500 random samples from the ImageNet validation dataset metrics are designed to reward masks that preserve classification accuracy while minimizing the amount of information used, making them well-suited for mask-based methods that prioritize sparsity and compressed explanations. Unlike faithfulness metries (e.g., insertion and deletion), which rely on an inherent ranking of feature importance, CP scores are specifically designed for methods like ours that optimize binary masks without order- ing feature relevance (Kolek et al. 2023). Additionally, using CP scores ensures consistency and comparability with the baseline methods Kolek al. (2023) 9
(Deng et al. 2009). These informatiion-thoreeee
As in Kolek et al. (2023), we use a pretrained ResNet18 model (He et al. 2016) and VGGI6 (Si- monyan & Zisserman, 2014) for classification. We also evaluate StartGrad vision transformers Dosovitskiy et al. (021) and swin transformers (Liu et al. 2022). Due to space constraints, only results for the ResNet18 model are shown in the main body of the paper, with results for other archi- tectures provided in Appendix F. F.4 and F33 The qualitative findings discussed here are vastly consistent across all tested models. For all mask-based vision models, we use the default parameter settings as reported in the respective papers to facilitate comparisons. Further details on the expla- nation models and hyperparameters are in Appendix C.I Additional results and ablation studies are provided in section D.3 and
F
In Appendix F.18
F.20 we also provide visual results.
Results. Table [ shows the median performance difference between StartGrad and both all-ones and uniform initalization strategies. Across all mask-based explanation methods, StartGrad signif- icantly improves performance compared the uniform initialization which shares the underlying initialization distribution. This underscores the advantage of using gradient signals to guide the mask initialization process.
For PixeIMask (Fong& & Vedaldi 2017) and WaveletX (Kolek al. 2023), StartGrad particularly enhances performance at early iteration steps, whereas at the final iteration step (300) no statistical differences can be observed for these two mask- explanation methods. However, initializing ShearletX (Kolek et al.) 2023) with our proposed StartGrad algorithm leads to a substantial and statistically significant performance improvement all iteration steps.
In addition to measuring performance at a given iteration step, we examine how StartGrad reduces runtime (as measured in iteration steps) when aiming for specific target metric. However, defining a universally "good" target score is inherently challenging due to the absence of a clear threshold. To tackle this, we define the per image target score as the highest attainable score across all three initializations and measure subsequently the iterations required to this specific normalized score.
*Our code is publicly available at httpps:// /tlp::///itiie /gggttt..bbucc//gg con/ Bue entUendes/ SarrtUendes/Starree StartGrad
For detailed justification of using the CP scores in place of faithfulness, see Appendix C.2.2
7

---

Published as a conference paper at ICLR 2025
Table 1: Median pairwise performance difference between StartGrad and alternative initialization methods across iteration steps for 500 ImageNet (Deng et al. 2009) samples using ResNet18 (He al., 20T6) classifier. Baseline refers the initialization scheme originally used for each mask explanation method. Statistical significance is marked as *** levels, respectively, using a one-sided Wilcoxon signed-rank test with Bonferroni correction. For PixelMask (Fong & Vedaldi 2017), CP-Pixel and CP-LI scores are identical as it operates only in the pixel space.
and for 1%, 5%, and 10%
A CP-Pixel
A CP-LI t
Iteration steps
Iteration steps
Method
Reference initialization
50
100
300
50
100
300
All-ones (baseline)
2.09""
0.72
0.33
2.09"
0.72
0.33
PixeIMask
Uniform
1.11
1.26
1.12
1.11 T*.
1.26
1.12
All-ones (baseline) Uniform
2.55
0.73"
0.11
0.71
0.37
0.14
WaveletX
2.35
3.39
5.20
0.60
tx
0.88
1.04
All-ones (baseline)
52.91
135.28
236.37**
2.89* ame
20.67*th
51.82
ShearletX
Uniform
58.65
130.71
208.74
3.09
33.42
65.57
As shown in Figure 1. StartGrad outperforms the other two initializations strategies for PixeIMask (Fong & Vedaldi 2017) and WaveletX (Kolek et al. (023) for medium target scores and for Shear- letX (Kolek et al. 2023 beginning from a score of 0.3.
300
300
300
StartGrrid lourst
lours)
250
250
All-ones
All-anss
2.00
Uniform
Unifcem
Uniform
150
50-
0.3 04
0.8
0. 0.8 0.7
0.1 0.2
0.5
lormallzed
CP Pixel
(a) PixelMask
(b) WaveletX
(c) ShearletX
Figure 1: Normalized CP-Pixel score vs. iteration steps for the PixeIMask (ong & Vedaldi 2017
(a), WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-Pixel score. This target score is defined the highest CP-Pixel value achieved across all three initialization methods for each model, then normalized by the overall maximum score ob- served. The shaded regions represent the standard error across 500 randomly selected ImageNet (Deng et al. 2009) validation samples, evaluated using a pretrained ResNet18 (He et al. 2016) clas- sifier.
Given the strong performance of the StartGrad initialization method for the ShearletX (Kolek al. 2023) model, we seek to evaluate how quickly mask-based algorithm initialized with StartGrad can reach the final performance levels of the all-ones and uniform initialization strategies. As can be seen from table 2 we achieve large speedups up to 6.73, further illustrating the potential of StartGrra and designing mask-based initialization strategies in general. To further understand the dynamics of the optimization process and the effectiveness of StartGrad, we analyze its role in guiding mask-based explanation methods. As shown in Appendix F.6 F.7 F8 initializing the mask using gradient signals, implemented in StartGrad, provides crucial early advantage by guiding the optimization process toward regions of minimal distortion, consistent with theoretical predictions. This early advantage is critical, uniform initialization fails to catch up in terms of distortion reduction throughout the optimization process which underlines the importance of an effective initialization..
8

---

Published as a conference paper at ICLR 2025
Table Average iteration steps for the ShearletX (Kolek al. 2023) method initialized with Start- Grad to match the performance of the all-ones or uniform initial initializations. Reported are the iteration steps to match the interquartile mean (IQM) and median of the CP-Pixel and CP-L1 scores, along with speedup calculated as the ratio of reference initialization to StartGrad iteration steps. All exper- iments use 500 randomly selected ImageNet (Deng et al. 2009) samples with a pretrained R (He et al., 2016) classifier, and metrics are computed over 300 iterations. Average and standard er- rors are estimated via bootstrapping (250 resamples). For each metric, f denotes higher is better and J lower is better. Values that are two standard errors away from the bascline (300 iterations, 1 for speedup) are highlighted in bold.
Target metric (at 300 iterations)
CP-Pixel
CP-L1
Reference
Iterations +
Speedup f
Iterations 4
Speedup f
All-ones (bascline)
IQM Median
44.57 + 2.36
6.73 + 0.35
71.09 + 6.47
4.24 = 0.38 3,45 0.38
49.60 + 3.39
6.06 + 0.41
87.70 = 9.98
IQM Median
45.50 3.18
6.60 + 0.45
72.92
8.68
4.16 0.47
Uniform
50.58 + 4.26
5.95 + 0.49
94.44 + 15.10
3.24 0.47
5.2
TIME SERIES
Following Tonekaboni et al. (2020). Crabbé & Van Der Schaar 2021 and Enguehard 2023),
test StartGrad on two commonly used synthetic benchmark datasets, i.c. state and switch-feature
data both of which
hidden Markov model (HMM) generate the data. In line with Crabbé &
Van Der Schaar (2021): 1: Enguehard (2023) we train one-layer GRU (Cho et af.) (014)) for each of
the experiments. As a baseline mask-based explanation method we use the recently introduced Ex- tremalMask (Enguehard 2023), which showed state-of-the-ar performance on the aforementioned datasets. As a perturbation model, we use bidirectional one-layer GRU (Cho al. 2014) model. We evaluate StartGrad using the following four metrics: area under recall (AUR), area under preci- sion (AUP). information (I) and mask entropy (E), where the last two were introduced by Crabbé & Van Der Schaar (202T). For both synthetic datasets, we use the same experimental setup in pre- vious studies (Crabbé Van Der Schaar 202T] Enguehard] 2023] Liu et al., 2024). i.e. we generate 1,000 samples and train the classifier on 800 training examples, and evaluate the performance on the remaining 200 samples while reporting results across five folds. We use the default hyperparameter settings well as the same random seed used by Enguehard 2023) to facilitate fair comparisons.
A complete description of the datasets, models and hyperparameters used in this study can be found in the Appendix C.3 Due to space constraints, only report the results the dwitch-featie dataset for the preservation game version of the ExtremalMask (Enguchard 2023) model. All re-
maining results are provided in section E
Results. As illustrated in Table 3 initializing ExtremalMask (Enguehard 2023 with StartGrad yields strong performance improvements especially in the early iteration steps. At iteration 50, StartGrad outperforms the baseline (uniform) in all metrics, with notably performance improve- ments. Compared the all-ones initialization, StartGrad provides performance improvements in 3 out 4 metrics, with the exception of AUP which is slightly lower. However, the all-ones mask initialization has low corresponding AUR score, indicating that ExtremalMask (Enguehard) (003) initialized with all- all-ones method misses lot of important features. This advantage is maintained through iteration 100. By iteration 500, all methods converge to similar final performance, demon- strating that StartGrad provides a strong initial boost without compromising the overall outcome for ExtremalMask (Enguchard) (003). The findings are consistent across datasets and objective formulations as shown in Appendix Db
LIMITATIONS AND FUTURE RESEARCH
Despite the promising results of the StartGrad initialization algorithm, few limitations require atten- tion:

---

Published as a conference paper at ICLR 2025
Table 3: Average performance for the ExtremalMask (Enguchard 2023 explanation method (preservation game objective) across iteration steps for the switch-feature dataset when initialized with StartGrad, all-ones and uniformly. The reported numbers are the mean and standard deviation across five folds. For each metric, f indicates that higher is better, and + that lower is better. Base- line refers to the initialization scheme originally used for respective mask explanation method. Outcomes that are one standard deviation away from the second-best ones are highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.849 +: 0.049
0.942 + 0.039
0.982 + 0.008
0.986 + 0.004
AUP f
All-ones
0.952 + 0.003
0.967 +: 0.004
0.986 + 0.003
0.986 + 0.003
StartGrad (ours)
0.922 + 0.004
0.984 + 0.003
0.987 + 0.003
0.987 + 0.003
Uniform (baseline)
0.746 + 0.029
0.737 = 0.023
0.747 = 0.020
0.748 = 0.020
AUR t
All-ones StartGrad (ours)
0.696 + 0.015
0.769 + 0.017
0.751 + 0.020
0.750 + 0.020
0.805 + 0.014
0.758 + 0.020
0.751 0.020
0.751 + 0.021
Uniform (baseline)
1.069 + 0.085
1.876 0.229
2.513 + 0.079
2.533 = 0.078
I [109]
All-ones
1.905 + 0.061
2.446
2.541 0.079
2.539 - 0,072
StartGrad (ours)
2.498 + 0.105
2.533 0.083
+
2.540 - 0.082
Uniform (baseline)
2.325 + 0.151
1.659 + 0.197
1.281 0.071
1.267 + 0.072
[10*]4 4
All-ones
1.984 + 0.066
1.277 0.070
1.276 + 0.072
StartGrad (ours)
1.688 + 0.103
1.370 + 0.074
1.276 + 0.069
1.278 + 0.069
First, StartGrad's effectiveness relies on accurate gradient estimation, making it potentially sensitive to noise. In Appendix F.3 we assess the impact of noisy gradients in the vision domain by adding Gaussian noise to the gradient signal. Appendix F.13 examines uninformative gradients by shuffling values and explores an adversarial setting where gradicnt- mask associations are inverted. Finally, Appendix F.10 evaluates sensitivity to gradient estimation methods using SmoothGrad (Smilkov) et al. 2017). Our findings show StartGrad remains robust to noise and alternative estimation tech- niques, matching uniform initialization in uninformative scenarios. However, its failure recover from adversarially initialized masks highlights the importance of proper initialization.
Second, the choice of the transformation function T that maps the gradients into continuous mask is crucial for StartGrad as illustrated in Appendix [..TT] Future work could investigate alternative transformations functions other than the QTF. Another promising direction for future work filter- ing high-frequency artifacts in gradient signals before applying the QTF transform, as this has been shown to improve the quality of the resulting gradient signal (Muzellec et al. 2024).
Third, compared to other initialization techniques. StartGrad involves some computational overhead. Yet, as illustrated in Appendix D.2] the additional computational costs are negligible, especially in light of the presented performance gains and speedups.
7 CONCLUSION
In this work, we present the first comprehensive theoretical and empirical analysis of mask ini- tialization techniques and introduce StartGrad, a novel gradient- madienttbbsed mask initialization method. Specifically designed for post-hoc mask-based explanation methods, we demonstrate that StartGrad provably superior in terms of distortion and sparsity at initialization compared to other techniques. Despite its simplicity, our experiments demonstrate that StartGrad enhances the optimization process of various Mate-of-the-ar mask-based explanation methods, particularly by improving performance early
Given its effectiveness, StartGrad can serve as a strong baseline initialization method moving for- ward. We also hope that this work sparks further interest in exploring strategies for effective initial- ization techniques in mask- explanation methods in an attempt to improve their practicality in real-world applications.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work is funded by Stress in Action. The research project 'Stress Action': www.stress-in- action.nl is financially supported by the Dutch Research Council and the Dutch Ministry of Educa- tion, Culture and Science (NWO gravitation grant number 024.005.010).
REFERENCES
Julius Adebayo, Justin Gilmer, Michael Muelly, lan Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS' 18, pp. 3525-9536 Red Hook, NY, USA, 2018. Curran Associates Inc.
Seojin Bang. Pengtao Xie, Heewook Lee, Wei Wu, and Eric Xing. Explaining black-box by using deep variational information bottleneck approach. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 11396-11404, 2021.
Wouter Bant, Ádám Divák, Jasper Eppink, and Floris Six Dijkstra. On the reproducibility "learning perturbations explain time series predictions". Transactions Machine Learn- ing Research, 2024. ISSN 2835-8856. URL
net/ www.nneenn.otii.ie forum?id
hPZgtpfgIX Reproducibility Certification.
Clément L Canonne. A short note on an incquality between kl and tv. arXiv preprint 0xXi:22002.07198, 2022.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. On the properties of neural machine translation: Encoder-decoder approaches, 2014. URL https://arxiv.
orq hacp/ab//aae 1409. 1259
Changkyu Choi, Shujian Yu, Michael Kampffmeyer, Amt-Borr Salberg, Nils Olav Handegard, and Robert Jenssen. Dib-x: Formulating explainability principles for a self-explainable model through information theoretic learning. In IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 7170-7174, 2024.
Jonathan Crabbé and Mihaela Van Der Schaar. Explaining time series predictions with dynamic masks. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Con- ference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 2166-2177. PMLR, 18-24 Jul 2021. / www...prres/vlli htt.ee//ingg.mmrpprsssss///1// crabbe21a.html
URL https: http://pooceednng..lle
Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp- Red Hook, NY, USA, 2017. Curran Associates Inc ISBN 9781510860964.
Ugur Demir, Ismail Irmakci, Elif Keles, Ahmet Topcu, Ziyue Xu, Concetto Spampinato, Sachin Jambawalikar, Evrim Turkbey, Baris Turkbey, and Ulas Bagci. Information bottleneck attribution for visual explanations of diagnosis and prognosis. In Machine Learning in Medical Imaging: 12th International Workshop, MI.MI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12, pp. 396-405, Springer, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-255, 2009. doi: 10. 20W.VV.R..009.22000
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. Utthttss: http:::::///ppeeeeieeeeee net/ forum? id=YicbFdNTTy

---

Published as a conference paper at ICLR 2025
Joseph Enguehard. Learning perturbations explain time series predictions. In Proceedings of the 40th International Conference on Machine Learning, pp. 9329-9344 PMLR, 2023. URL htttps://pat httpss:/prroeeeinggs ml. www. /epprsssa/22/////// /222//222/////////22/// 2t20///nnuaaarr2aaaa.. html ISSN: 2640-
3498.
Ian Fischer. The conditional entropy bottleneck. Entropy. 22(9):999, 2020.
Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful per- turbation. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3449-3451 2017. doi: 10. 10/11C/.2.2............
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation neural networks is fragile. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01))3681-233) 1-3688, Jul. 2019. doi: 10. 10.1609/aaai, v33101. 133101.3301666 36801 URL ntttps://ojs.aattt org...org//iiee/eeeeeeeee php/ /php/AAAT/ AAAT/
article/vi ew 4252
Ran Gilad-Bachrach Amir Navot, and Naftali Tishby. An information theoretic tradeoff between complexity and accuracy. Bernhard Schölkopf and Manfred K. Warmuth (eds.), Learning The- ory and Kernel Machines, volume 2777, pp. 595-609. Springer Berlin Heidelberg. 2003. ISBN
978-3-540-40720-1 978-3-540-45167-9,
doi: 10. 111-0077778-3-54-4511666
URL http:
//link. springer.......... com/ 10. 1007/978
540 45167 9 43
Series Title: Lecture
Notes in Computer Science.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington (eds.), Proceedings of the Thirleenth Interna- fional Conference on Artificial Intelligence and Statistics, volume Proceedings of Machine Learming Research, pp. 249-256 Chia Laguna Resort, Sardinia, Italy, 13-15 May 2010. PMLR. URL httpp:://proceeeinngats mww.ediinnss........mmmmmmmmmmmm mwr. prwss www.:ppees...aapeee http://aww..........ese/ogoooooo
html
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp- 770-778, 2016.
doi: 10. 00.1109/CVPR.20169900
Aya Abdelsalam Ismail, Mohamed Gunady, Hector Corrada Bravo, and Soheil Feizi. Bench- marking deep learning interpretability in time series predictions. In Advances in Neu- ral Information Processing Systems, volume 33, pp. 6441-6452. Curran Associates, Inc., http://ppeeeeeii....eeerereeeeee httcc//aerrffflss/eeeeeeeeeeeeee files/paper/2020/ 2020. URL https::/pprceeedinn
haah:7188$cccc033303000022222222 hbt.:/00...2ddd-bbtrrt........ Abstract.tttttt
Diederik P. Kingma and Jimmy Ba. Adam:
method for stochastic optimization.
CoRR, abs/1412.69800, 2014. URL httpss:/aap: semanticscho ott.i/oolaa..r//oopuusst000
6628106
Andreas Kirsch, Clare Lyle, and Yarin Gal. Unpacking information bottlenecks: Unifying information-theoretiic objectives in deep learning. arXiv preprint UrXiv:2003.12532 2020.
Stefan Kolek, Duc Anh Nguyen, Ron Levie, Joan Bruna, and Gitta Kutyniok. Cartoon explanations of image classifiers. In European Conference of Computer Vision (ECCV), 2022.
Stefan Kolek, Robert Windesheim, Hector Andrade Loarca, Gitta Kutyniok, and Ron Levie. Ex- plaining image classifiers with multiscale directional image representation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
Gitta Kutyniok, Wang- Q Lim, and Rafael Reisenhofer. Shearlab 3d: Faithful digital shearlet trans- forms based on compactly supported shearlets. ACM Trans. Math. Softw, 42(1), Jamuary 2016. ISSN 0098-3500 doi: 10. 145/2740960. URL httpp::///oi.. wtt................... 11455/274490
12

---

Published as a conference paper at ICLR 2025
Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, Furu Wei, and Baining Guo. Swin transformer v2: Scaling up capacity and reso- lution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 12009-12019, June 2022.
Zichuan Liu, Yingying Zhang, Tianchun Wang, Zefan Wang, Dongsheng Luo, Mengnan Du, Min Wu, Yi Wang, Chunlin Chen, Lunting Fan, and Qingsong Wen. Explaining time series via con- trastive and locally sparse perturbations. In Proceedings of the 12th International Conference on Learning Representations, pp. 1-21, 2024.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL htttp:://proceeddgss...oo neurips http://www..aaeer.eeeeo./.pppppp wttp:/cerppppleppese.eeseepppepe paper/2017/f11 e/ 80200800862222 91 978 (a220888811113222747444444228888
Jan Macdonald, Stephan Wäldchen, Sascha Hauch, and Gitta Kutyniok. A rate-distortion framework for explaining neural network decisions, 2019.
Sabine Muzellec, Thomas FEL,, Victor Boutin, Léo Andéol, Rufin VanRullen, and Thomas Serre. Saliency strikes back: How filtering out high frequencies improves white-box expla-
nations. In Forty-first International Conference on Machine Learning, 2024. URL https:
//openzeview.co wwwee//ooormiimeeeeeeeeee 0000GGW
Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: Randomized input sampling for explanation of black-box models. In Proceedings of the British Machine Vision Conference (BMVC), 2018.
Mark S Pinsker. Information and information stability of random variables and processes. Holden- Day, 1964.
Zoe Piran, Ravid Shwartz-Ziv, and Naftali Tishby. The dual information bottleneck. arXiv preprint arXiv:2006.0m 04641, 2020.
Owen Queen, Thomas Hartvigsen, Teddy Koker, He Huan, Theodoros Tsiligkaridis, and Marinka Zitnik. Encoding time-series explanations through self-supervised model behavior consistency. In Proceedings of Neural Information Processing Systems, NeurIPS, 2023.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 1135-1144, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322. doi: 10.1145/2939672. 2939778. URL httpttttt///.oo..ooo
Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In International Conference on Learning Representations, 2020.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 118-626, 2017. doi: 10. 10.1109//CC.22077...
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CORR, abs/1400. 1556, 2014. httts:://aaaa wtt.ppp....ttttcccscc..ooaaaaooe httesccooloorrg//o/o
CorpusID: ID: ::: 14124313
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Confer- ence on Learning Representations, 2014.
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM
Conference AI, Ethics, and Sociery, AIES 20, pp. 180-186, New York, NY, USA, 2020. Association for Computing Machinery, ISBN 978 4581450371000. doi: 10.1145/3375627.337555
U wwtp/tttt:/// http:///oooooog..1111 wwt.//....../33555217111 33758330
13

---

Published as a conference paper at ICLR 2025
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. In Workshop on Visualization for Deep Learning. ICML, 2017.
DJ Strouse and David J Schwab. The deterministic information bottleneck. Neural computation, 29 (6):1611-1300, 2017.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning. volume 70 of Proceedings of Machine Learning Research, pp. 23199-3322 PMLR, 06- 11 Aug 2017.
URL https::/pprceeeding mlr www.lrpppe/s//ee v0/ /ttp://////o/dddaaaaaaaaaaaaaa
html
Ruo Yu Tao, Vincent François-Lavet, and Joelle Pineau. Novelty search in representational space for sample efficient exploration. Advances in Neural Information Processing Systems, 33:811 14 8126, 2020.
MTCAJ Thomas and A Thomas Joy. Elements of information theory. Wiley-Interscience, 2006.
Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference on Communication, Control and Computing, pp- 368-377, 1999.
Sana Tonekaboni, Shalmali Joshi, Kieran Campbell, David K Duvenaud, and Anna Goldenberg. What went wrong and when? instance-wise feature importance for time-series black-box mod- els. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 799-809. Curran Associates, Inc., 2020. URL https:: https::/pproeeddngs...e.... wwwperipssccppeepe. cc/paper.pppp.....eeeeee Saiaaper__ile/pppppaaaa files//aperr/202// pdf
Alfredo Vellido. The importance of interpretability and visualization in machine learning for ap- plications in medicine and health care. Neural Comput. Appl., 1869993 December 2020. ISSN 0941-0643. doi: 10. 00.10077000021-019-00011------11 1007/s00521- 01000199--00 04051-w
URL htttpp::/doi.ozgg///
Yaochen Xie, Sumeet Katariya, Xianfeng Tang, Edward Huang, Nikhil Rao, Karthik Sub- bian, and Shuiwang Ji. Task-agnostic graph explanations. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural In- formation Processing Systems, volume 35, pp. 12027-12039. Curran Associates, Inc., 2022. URL httpps://prceedd_ng www.rneeeii..... wttpssscccccaaaaeee lorrrr_fllee fites//ppe/sse//////////// File/ BB2224477Tarr3300000002222222220
pdff
Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovee. GNNExplainer; generating explanations for graph neural networks. Curran Associates Inc., Red Hook, NY, USA, 2019.
Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition. In International Conference on Learning Representattonss, 2021.
Jimeng Shi Xu Zheng Zhuomin Chen Lei Song Wenqian Dong Jayantha Obeysekera Farhad Shi- rani Dongsheng Luo Zichuan Liu, Tianchun Wang. Timex++: Learning time-series explanations with information bottleneck. In Proceedings of the 41st International Conference Machine Learning, 2024.
14

---

Published as a conference paper at ICLR 2025
A PROOFS
A.1
RESEMBLANCE BETWEEN IB OBJECTIVE AND RDE OBJECTIVE
The IB optimization trade-off can considered as generalized rate-distortion problem Tishby et al.) 1999 Piran et al. 2020) with the distortion function between an instance x and (compressed) representation taken as the KL-divergence between their predictions of the black-box output j/:
(x; x) DKI. (x..p(()));;11
Σ
x(0 log
p(U
(9)
p(0 x)
The expected distortion E[d (x; x)] simply reduces to (xx: 0) - (x;; 0). This is because:
Ep(x.5) [di (x: x)]
p x) p(]))) - E! > p(0 log p(O st]
pxx x)x(p((( p()xx) ΣΣΣ p(x,
x
Σ
p p logr(ū)x) Σ Σ
Σ p(x, logr(ū/>)
X
Ellog p(ū x)] Elogg
-(()) + (ū) -
-(0:x)-I(( -
(10)
where the third line of equation uses the Markov chain property 1 + x + x, ie., p(o|x x) = p())x). Line five uses the definition of conditional entropy, i.e. (ū]xx) -Ellog ((0 [z)] and line
six the definition of mutual information I(ý: x) = H()) H() respectively.
Since [(0: x) is constant that is independent of the optimization, minimizing the expected IB distortion EE (p()))p(()))))pppp [[] equivalent to maximizing T(0; x).
Note that, in neural networks implementation, p( x) is approximated by the output of last softmax layer, i.e., (x). Similarly, x) is approximated by (x)..
On the other hand, due to the Pinsker's inequality (Pinsker 1964 that for any two distributions p and
Canonne 2022) which implies
min(DRu (p; (p: (xl((:))) 2
1p AlF
(11)
2 log
from which we obtain DKL (KL (x)( (x)) 2 2 log
()())
p (x)
To summarize, maximizing / (0: x) is equivalent to minimizing the expected Kollback-Leiblee (KL) divergence E (Dxr (x) (x))), and it provides upper-bound other loss functions such as
the -loss E (x) 4 (x) li).
To motivate the link between the compression term in the principle and the sparsity term m, in the RDE objective, we begin by rewriting the mutual information 7(x; x) as:
1(x; (()) + (x)) 11 [x, x)
(12)
Since (x) is constant when optimizing for a sparse mask m E [0, 1]d within the IB principle, it can be dropped, yielding:
[xx; H(x) H(x x)
(13)
Using the conditional entropy definition, get:
15

---

Published as a conference paper at ICLR 2025
H(x, = x(xx))+++ + (x)
(14)
Dropping (x) for the same reason as above, we obtain:
[(x; x) H()) H(
(15)
Since x C x, the conditional entropy (x is small, leading to the approximation:
[(x: x) ~ H())
(16)
This demonstrates that the mutual information (x; x) can be effectively approximated by the en- tropy of the masked input, (x) (Strouse & Schwab) 2017 Kirsch et al. 2020). A practical way minimize (x) is to reduce the number of explanatory variables, which corresponds to encouraging sparsity in m (Bang et al.) 2021 Tao al. 2020). This sparsity constraint is commonly represented
by ||ml|o and approximated using | for optimization purposes. Thus, the sparsity term in the
RDE objective conceptually aligns with the compression term in the IB principle
A.2 EXPECTED RDE LOSS AT INITIALIZATION UNIFORM VERSUS GRADIENT BASED INITIALIZATION
Let x E Rd denote the original input with dimensionality d. Further let Manif E [0. 1] be the uni- formly initialized mask where each m; is independently sampled from (0, 1). Similarly, denote Mgrdd the gradient-based initialized mask as described in section 4 where we transformed the gra-
dient values using quantile transformation function so that Mggrad E [0, 1] We aim to prove the
following inequality:
Emgnst. V [D (Pa(a) (x(x)) + A/ 4/IMgrad [1] < E a (D (Th (Xuntt),, Fx(x)) + All A[mmmar[la]
(17)
Here, 8grad Mgral ① x (1 Mgraa) O u and Kuiit Munit O x (1 Munit) ② u are the distorted versions of the input E Rd with random perturbation E Rd drawn from predefined probability distribution V such as Gaussian. Here, represents a hyperparameter, encouraging sparsity Mgrad and Munif-
Let further denote (x) a classifier's prediction for the input x E Rd where T IRd [0, 1
with 1 representing the number of classes. Assume that the classifier's prediction (x is differentiable and locally linear in the neighborhood x. Formally, this neighborhood defined as the open ball (x) {w E Rd || x!| < c}. where 0 ensures that higher-order terms in the Taylor expansion 4. are negligible.
Then, the first-order Taylor expansion around x:
(x) Rd (x(x) + Vx (x- (x-x)
(18)
where VxQ. (x) represents the gradient of (x) with respect to x and Ax x-x x. The distortion
D(D.(x), &. (x)) can therefore be approximated, for norm || Ilp. as:
D ( (x)) ( {x)x)) (x)x.( AxIIp
(19)
16

---

Published as a conference paper at ICLR 2025
Using this, we get the following expression:
Emgns - [[V Ax|1p + A||m Al|mgradlli] S E E V1V Ax|1p + >| [ (20)
Since Meniform: Mirrad ~ ((( the terms Emg.. [Al[maadlle and Em.. [A] mmmrllaa simplify to 2d as the expected value for each m, is 5.2 Both sparsity terms can be omitted from the comparison since they are equal. Therefore, we only need to show that:
E Emgnat u (x)x). (Rgrad x)1pp] S
E Emair V [1
(Runir x)]]
(21)
holds. By using the definitions for Xyrad and Nunit, get the following after rearranging terms:
X grad X (Mgrad 1) O (x x-u), u) Xunif
(22)
(Munif 1) O (x-u). u).
(23)
Hence, we can rearrange 21
Emgan.wvV V O Mggrad grad
1 (x - (x)lll < E Em
Vx @ (Munil 1) o (x - -u u)]lpp
(24)
The key observation is that the gradient-based mask Mgrad is constructed to minimize the distortion by assigning values of m, close to I when Vx, &x(x) is large, thus reducing the effect perturbations in important directions. On the other hand, the uniform mask Manif is assigned independently of the gradient, making it less likely reduce distortion effectively. Consequently, the gradient-based method is expected to achieve lower distortion compared to the uniform initialization al initialization by design, thereby validating the claim.
This completes the proof.
A.3 EXPECTED RDE LOSS AT INITIALIZATION ALL-ONES VERSUS GRADIENT-BASED INITIALIZATION
Let Mones denote the mask initialized with all ones, i.c. Mones = 1. where 1a is a vector of ones with dimension d. Let further denote (x a classifier's prediction for the input x E Rd where T. IRd [0, 1] with 2 1 representing the number of classes. Given the function D [0, 1] x [0, 1] >> R. that measures the distortion between the classifier's prediction for the original input and the "distorted' input &, the all-ones initialization results in (x)(x), (x()))
we have x = x. Furthermore, let Mgrad ~ (0, 1)d denote the mask initialized using the gradient-based We aim to show that
scheme as described in section
E E V [D (X), (x)) + X||mgnall11] < X| Mones
(25)
holds, where À represents a hyperparameter balancing the trade-off between distortion and sparsity. Since Mones is initialized with all ones, we have Momes = d. For mgrad ~ 24(0, 1)d the expected value of the l -norm is given by Em [|mmgadl] 2. Therefore, we can simplify the inequality as follows:
Em u-g (D (+ xx(x)) 2 < Xd
X
(26)
X
E u-g [D oh (X). (x))]
2 d
(27)
we use a norm || with 2 as a choice for the distortion function D, we can upper bound the expected distortion the left-hand side of equation in the following way:
17

---

Published as a conference paper at ICLR 2025
u- (D()(()), (x(x)) S (1+ (2 1)Lc22) (2-1)((22) 2 d
(28)
as & Rd + [0, 1fe for classification setting we consider, where Ei-1 (e,(()) for any input x E IR.. Therefore, the gradient-based mask initialization has a lower expected loss in terms of distortion and sparsity initialization, if
2 2 + (2 1) S
(29)
d
holds. In high-dimensional settings, which are typical for deep learning models and, consequently, for mask-based explanation methods, this condition is generally satisfied.
This completes the proof.
18

---

Published as a conference paper at ICLR 2025
B PSEUDOCODE
B.1 GENERAL VERSION OF THE STARTGRAD ALGORITHM
While we assumed in the pseudocode for Algorithm that we operate in the same input space we outline in Algorithm 2 more general version of the StartGrad algorithm, where we first use an invertible mapping F.
Algorithm 2: General version:
Mask Initialization (Star(Grad)
Input
Pre-trained classifier 4c. imput samples { with æ; E Rd, quantile transformation T R [0, 1]d invertible and differentiable function F Id R*
Hyperparameters: Output distribution ((00, 1), number of quantiles (bins) for T Output
Masks M [0, Nj
Initialize mask list M : and quantile transformer T; for f to N do
arg max(# (x.)) // Class prediction
S: V F(a)) (o(x)))
Gradient magnitudes
m, < (S,)) // Quantile transform
Append m, to M;
end for return M
Figure visually illustrates the pseudocode.
Get model prediction
Pratrained Blackbox model & (x)
$ Dxy 0.97
Input to be explained d
Obtain gradient magnitudes
4 < 0.97
VV (x
Gradient values
Use Quantile Transformation Function (QTF)
1500
Output distribution:
Gradient distribution
24(0,
StartGrad mask coefficients
Figure 2: Visual illustration of the StartGrad algorithm.
19

---

Published as a conference paper at ICLR 2025
C EXPERIMENTAL SETTINGS AND DETAILS
C.I VISION
Here we describe the explanation methods used for the vision part of the experiments in more detail. We use the same notation as already introduced section
For all vision experiments, we optimize the mask coefficients for 300 steps, using Adam optimizer Kingma & Ba 2014) with learning rate of 10-1
C.1.1 PIXELMASK
The objective for the PixelMask model first introduced by Fong & Vedaldi (2017) is:
min E v D (3) 2x())) + X |m mc [011|
(30)
We set A = 1, and estimate the expectation via Monte Carlo sampling across 16 samples, where we draw the perturbation from uniform distribution. In particular, use uniform noise from [p - 0, + r), where and a are the empirical mean and standard deviation the pixel values of the image as is been done in (Kolek et al. 2022 2023).
For the PixelMask (Fong & Vedaldi 2017) model, we use 65,536 bins for the QTF function in Algorithm T which equals the width and the height (256 by 256) of the images used for the pretrained vision models.
C.1.2 SHEARLETX & WAVELETX
For the ShearletX Kolek et al. 2023) and WavelerX model (Kolek et al., 2023), we utilized the original implementations and follow their experimental setting closely.
With the same notation as used section 3 Kolek et al. (2023) extend equation 30 in the following manner:
min E [D (D,(()) (x)x + A + >e1l (m c F(x))ll m)
(31)
@u) denotes now the distorted input image after applying the mask in the transformed space. The WaveletX model uses for the F the discrete wavelet transform (DWT), whereas the ShearletX makes use of the digital shearlet transform (Kutyniok et al.) 2016 Kolek et 2023). For the WaveletX we use the daubechies-3 as a mother wavelet with five scales.
where x := F= (mx
The expectation in 31 is approximated using Monte Carlo sampling using 16 samples where the perturbation for scale a and shearing s sampled uniformly from [Me.s da.ss Ha.s + aa.s). Here Cass and Ma,s are the empirical standard deviation and mean of the image's 's shearlet coefficients at scale and shearing s. line with Kolek et al. (2023), we set for ShearletX XS = 1 and X For WaveletX we set λ 1 and d 10. We use the mean-squared error as a choice for the distortion function D.
For the ShearletX and WaveletX Kolek al. 2023). use 10,000 as the number of bins for the former, and the following bins for each of the scales, 144, 16900, 4489, 1296, 400, 144 for the latter.
C.2 VISION EVALUATION METRICS
C.2.1 CP-LI AND CP-PIXEL. SCORE
As outlined in section 5 we use the conciseness conciseness-preiiseeess (CP) Pixel and LI score first introduced by Kolek et al. (2023) quantitative performance metrics which are defined as follows:
20

---

Published as a conference paper at ICLR 2025
Retained class probability
CP
(32)
Retained image information
where the numerator is defined by:
(x(x)
Retained class probability (x(x))
(33)
while the denominator in case of the CP-Pixel is defined as:
|m ② x|] |x||
Retained image information Pixel
(34)
whereas for the CP-L1 we have:
|m 3 &cll
Retained image information LI
(35)
||c|]]
where E R represents the coefficients of the latent space such as the shearlet or wavelet space. Note that for the PixelMask model, the CP-L1 and CP-Pixel metrics identical the masking only applied in the pixel space.
Given the aforementioned formulations of the CP scores, one can notice that these scores strive to balance both, preciseness and sparsity of the explanation. Higher scores indicate superior masks Kolek et al. 2023).
C.2.2 JUSTIFICATION OF CP-L1 AND CP-PIXEL SCORE AS OPPOSED TO DELETION AND INSERTION SCORE
Attribution methods like Integrated Gradients (Sundararajan et al. 2017 and SmoothGrad (Smilkov al., (207)) are typically evaluated using insertion and deletion scores. Unlike the mask-based explanation methods in this study, they provide a clear feature importance ranking, usually in pixel space. In contrast, mask-based methods do not inherently rank coefficient relevance but instead optimize binary, sparse mask using relaxed 1 norm to approximate the ideal to norm. This binary nature complicates insertion and deletion scores, which require an ordered ranking. This issue was previously highlighted by Kolek et al. (2023), who introduced CP-scores; we refer their work for further discussion.
To highlight the issues with using the insertion and deletion metrics evaluate mask-based explana- tion methods, we computed the faithfulness scores which is the difference between the former two for PixelMask (Fong & Vedaldi 2017). WaveletX, and ShearletX (Kolek et al. 2023) across mul- tiple iterations, as shown in the following figure. As we can see from the figure the faithfulness scores decreases with increasing iteration steps across all three methods and all initializations and reach their maximum scores at very early iterations steps which is in sharp contrast to the results obtained using the CP-scores and also counterintuitive as this implies that the learned mask gets less faithful over time. This illustrates the problem of faithfulness score which presumes an inherent ordering of the mask values, even though the mask-based explanation methods optimize relaxed binary problem with no inherent ordering. For comparison reasons, we included two non-masked based attribution methods Integrated Gradients (Sundararajan et al. (017) and GradCAM (Selvaraju al. 2017)
For completeness, Table 4 includes a comparison between the two aforementioned gradient-based methods and the mask-based explanation methods used in this study using the CP-Pixel and CP-LI evaluation metric. The results clearly demonstrate that mask- based explanation methods consistently outperform non-mask- mask-based methods across all iteration steps. However, it is important to note that mask-based methods, with their learned binary mask, are explicitly designed to balance distortion and sparsity, giving them advantage in this trade-off.
21

---

Published as a conference paper at ICLR 2025
02
0.0
25
150
195
250
360
(a) Faithfulness across iterations: PixelMask
0.4
0.3
022
0.1
Aeaici Auciin
0.0
deders
(b) Faithfulness across itcrations: WaveletX
0.6
0.3
Palnhhiee
02
Oud
Number Lanaioor
Faithfulness iterations: ShearletX
Figure 3: Faithfulness scores across iteration for the PixelMask Fong & Vedaldi 2017) (a),
WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dotted line). For comparison reasons, we added two non-masked based explanation methods, i.e. Gradient- weighted Class Activation Mapping (Grad-CAM) (Selvaraju et al. 2017) and Integrated Gradients (Sundararajan al., 2017). As these methods are not iteratively optimized, the faithfulness values for these methods remain constant. The curves represent the interquartile mean (IQM) across 100 randomly selected ImageNet validation samples (Deng et al., 2009), evaluated with pretrained ResNet18 classifier (He al. 2016). indicates that higher better. We can that the faithful- ness score decreases across all methods and initializations over the course of optimization which is counterintuitive and in contrast the CP-scores, thereby illustrating the problem associated with using faithfulness scores as an evaluation metric for mask-based explanation methods that do not inherently order their mask-valuee according to importance.
22

---

Published as a conference paper at ICLR 2025
Table 4: Interquartile mean score (IQM) for the PixelMask (Fong Vedaldi 2017). WaveletX
(Kolek et al. 2023), ShearletX (Kolek et al. 2023), Integrated Gradients (IG) (Sundararajan et al. 2017). Greghent-weighted Class Activation Mapping (Grad-CAM) (Selvaraju al. 2017) of the CP-Pixel and CP-LI score across iteration steps. All mask-based methods are initialized with Start- Grad. To map the gradient-based attributions (IG, GradCAM) into the desired range of [0, IJ. either min-max normalization (Min-Max) quantile transformation function (QTF) were used, All experiments use 500 randomly selected ImageNet (Deng et al., 2009) samples evaluated on pre- trained ResNet18 (He et al. 2016) classifier. For each metric, indicates that higher is better, and + that lower is better. Values that are two standard errors away from the second-best best values are highlighted in bold. As IG (Sundararajan al., (017)) and GradCAM (Selvaraju al.) 2017) both methods that are not optimized iteratively, their scores remain constant across the shown iter- ation steps. The CP-Pixel and CP-LI scores differ only for the WaveletX (Kolek al. 2023 and
ShearletX (Kolek et al. 2023), as only these methods operate also in a latent space.
Iteration steps
Metric
Initialization
50
100
300
PixeIMask
37.395 + 1.119
51.217 + 1.478
65.766 + 1.767
WaveletX
48.973 + 0.604
64.551 + 0.809
77.534 + 0.976
ShearletX
216.908 + 6.997
359.803 + 9.675
481.314 11.829
CP-Pixel f
IG (Min-Max)
3.237 = 0.015
3.237 0.015
3.237 0.015
IG (QTF)
2.037 0.001
2.037 0.001
2.037
GradCAM (Min-Max)
2.841 = 0.019
2.841 0.019
2.841 + 0.019
GradCAM (QTF)
1.987 0.004
+
1.987 0.004
PixeIMask
37.395 = 1.119
51.217 = 1.477
65.766 1.767
WaveletX
11.208 + 0.128
14.505 = 0.173
17.484 + 0.204
ShearletX
123.791 + 4.760
199.193 + 6.384
270.438 + 7.931
CP-L1
IG (Min-Max)
3.237 = 0.015
#.011
3.237 = 0.015
(QTF)
2.037 = 0.001
2.037 0.001
2.037 + 0.001
GradCAM (Min-Max)
2.841 0.019
2.841 0.019
2.841 + 0.019
GradCAM (QTF)
1.987 = 0..04
1.987 0.004
1.987 + 0.004
C.3
TIME SERIES
C.3.1 STATE DATASET
The dataset for the synthetic state dataset as introduced by Tonekaboni et al. (020)) generates data according 2-state hidden Markov model (HMM). Let t E It 2' represent the discrete time steps and s, E {0, 1} denote the specific states. For this dataset, the input x E R is generated using multivariate Normal distribution I, N(s, Es,). where the mean Nss and the variance- covariance matrix Eg, are dependent on the state at 8;. In particular, we set pmn = [0.1, 1.6, 0.5) and in = 0-0.1 0.4, 1.5). The variance-covariance matrix is given by:
0.8
0.01
0.017
0.01
0.8
0.0
0.01
0.0
0.8
The first feature dimension is irrelevant for the label yt, whereas the label is distributed as:
y Bernoulli(p))
(1 + expl 12x22,22) -1
s; = 0 if s,
P: (11 + expl -2x3311 )
(36)
23

---

Published as a conference paper at ICLR 2025
The transition probabilities are given by:
0.1
0.9
T=
0.01
0.9
with the starting state at = O being determined with equal probability, i.e. π = 12 2 For the experiments, we generate 1,000 time series, 800 for training, and 200 for testing.
1
C.3.2 FEATTRE DATASET
Following the setup introduced by Tonekaboni al. (2020),
the switch-feature dataset uses Gaus- sian process mixture in place of the multivariate normal distribution with means j
00.
0.5
0.2
/
0.0
1.0
0.0
0.2
0.2
0.8.
The Gaussian process time is governed by RBF kernel with y 0.2, with marginal feature set to 0.1 for all states. In a similar vein to the state dataset, the label distributed as:
~ Bernoulli(p.)
(1 + expl -3r1,e1) if 8t =
=
(1 + expl -3222 -1 if 8: 1
(37)
+ expl 3r -1 if 8, 2
The transition probabilities are given by:
00.5
0.02
0.3
T
0.02
0.95
0.03
00.03
0.02
0.95
with the starting state at = 0 being determined with equal probability, i.e. π = [ 3 3 For the experiments, generate 1,000 time series, 800 for training, and 200 for testing.
C.4 EXTREMAI.MAK
ExtremalMask (Enguehard 2023) is mask-based explanation method specifically designed for multivariate time-series data and defined by:
min mé 4
((Fa(X), ox()) + Am|Imm]| + Ax]]
(38)
where & = m C x (1- m) 2 (x(x and x E RdeT where is the input dimensionality, and T represents the time dimension. respectively.
Equation 38 denotes the preservation game which aims to find the sparsest mask that preserves the most class probability. In the deletion game formulation of ExtremalMask (Enguehard (023). the objective function is defined as:
min mc (0 .
D (x) + X1 m/] + [x))]
(39)
In line with the experiments conducted in Enguehard (203). ). we use one-layer, bidirectional GRU (hoo et al. 2014) model with hidden dimensionality of 200 choice for the fo- Furthermore,
24

---

Published as a conference paper at ICLR 2025
the Adam Kingma & Ba 2014) optimizer with learning rate 10-1, We run the ExtremalMask
[Enguehard 2023) model for 500 iterations. We use the mean-squared error as a choice for the distortion function.
As opposed to the original formulation of the distortion objective 39 we use a slightly modified version of the objective which has shown significantly boost the performance (Bant et al. 2024):
min me(o.1juz2 ((.,(8).0.(0)) + A1ll1 ml1a A2))fo(x)
x|
(40)
For all the experiments, we fit single layer one-directional GRU (Cho al. 2014) as a baseline classification model with hidden dimension 200 and train it for 50 epochs, batch size 128, learning rate of e with the Adam (Kingma Ba 2014) optimizer. We minimize the cross-entropy loss for training the GRU model.
For both time-series datasets, we use 600 (which equal the signal length times the feature dimension) as the number of bins for the Quantile Transformation Function (QTF) in our StartGrad algorithm
C.5 TIME-SERIES EVALUATION METRICS
Given that the ground truth salient features are know in our synthetic dataset, we the standard performance metric area under recall (AUR) and area under precision (AUP). We further evaluate StartGrad using two additional metrics, information and mask entropy, with
(Ar = Σ log(1 - m4,s)
(41)
LSEA
for A C [1: 7] x [1: dx] defining the information, and
(A) Σ m4, log(me,)) + +1 (1- (1--mam log(1 mm,
(42)
the entropy respectively (Crabbé & Van Der Schaar 2021 A higher score indicates a better score for information, whereas lower score indicates a better performance with respect entropy.
25

---

Published as a conference paper at ICLR 2025
D VISION EXPERIMENTS
D.1 DISTRIBUTION OF ABSOLUTE GRADIENT VALUES
123
200
53
6.0
6.4
3.6
6.R
1.0
Figure 4: Absolute gradient values for the input pixels for 100 randomly selected validation Ima- geNet (Deng et al.) 2009) samples using pretrained ResNet18 (He et (2066) model. Here, the gradient was taken with respect to the class prediction.
30392
0002
6002
4022
2003
0.0
5.4
3.5
0.6
10
Figure 5: Absolute gradient values in the shearlet domain for the input pixels for 100 randomly selected validation ImageNet (Deng al.) (009) samples using pretrained ResNet18 (He et al., 2016) model. Here, the gradient was taken with respect to the class prediction.
D.2 RUNTIME ANALYSIS
On a Apple Macbook Pro with a MI chip, we ran the StartGrad algorithm for all three vision models, i.e. PixeIMask (Fong& & Vedaldi 2017), WaveletX (Kolek et al.) 2023) and ShearletX (Kolek et al. 2023) across 100 randomly selected ImageNet validation samples (Deng et al. 2009) and obtain the tollowing average runtime (in seconds):
PixelMask: 3.62s
WaveletX: 0.72s
ShearletX: 1.31s
26

---

Published as a conference paper at ICLR 2025
D.3
ADDITIONAL FIGURES FOR VISION EXPERIMENTS
70
70
60
60
40
40-
30
30-
10:
Al-anes Uniform
10-
Aloons Uniform
0
90
100
150
200
250
305
0
50
100
150
200
250
350
Iterations
Iterations
(a) PixelMask: CP-Pixel f
(b) PixelMask: CP-L1 1
70-
50
30
10-
(baselinc)
(sazaline)
Startinad touns)
StantGred (ours)
fniform
Uniiorm
50
200
150
Iterati
Iterations
(c) WaveletX: CP-Pixel f
(d) WaveletX: CP-LI f
300
250
400
130.
10M
200
100-
Al-orms (base me)
50
Alkones (cavelin))
StartGrod lours)
Stantrrd (ours)
unform
Unifomm
30
100
130
200
250
300
0
30
100
150
200
250
500
Iteration ona
Iterations
(c) ShearletX: CP-Pixel
(f) ShearletX: CP-LI 1
Figure 6: Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the PixelMask (Fong& & Vedaldi (017) (first row), WaveletX (Kolek et al. 2023) (second row) and ShearletX (Kolek et al. (023) (third row). Baseline refers to the originally used initialization scheme for the respective mask explanation method. The solid line represents the average of the interquartile mean (1QM) performance across 500 randomly selected validation ImageNet (Deng al. 2009) samples, while the shaded area denotes the standard errors respectively. 1 indicates that higher is better, and J. that lower is better. We use pretrained ResNet18 (He et al. 2016) classifier.
27

---

Published as a conference paper at ICLR 2025
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0.0
0.1
0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP Score
(a) PixelMask: CP-L1
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0.0
0.1
0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP L1 Score
(b) WaveletX: CP-L1
300
StartGrad (ours)
250
All-anes
Iterations
200
Uniform
150
100
50
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0
Normalized CP L1 Score
(c) ShearletX: CP-L1
Figure 7: Normalized CP-LI score vs. iteration steps for the PixeIMask (Fogg & Vedaldi 2017
(a), WaveletX (Kolek al. 2023 (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The
represent the average number of iteration steps needed reach normalized target CP-L1 score. This target score is defined as the highest CP-L1 value achieved across all three initialization methods for each model, then normalized by the overall maximum score observed. The shaded regions represent the standard eтror across 500 randomly selected ImageNet validation sam- ples (Deng al. x009)xs, with a pretrained ResNet18 (He al., (016) classifier. StartGrad enables all three models to achieve target scores with fewer iterations compared to standard initialization schemes, with the effect being most notable for ShearletX (Kolek al. (003)..
28

---

Published as a conference paper at ICLR 2025
E TIME-SERIES EXPERIMENTS
In this section, the main results from the time-series experiments are presented which were omitted from the main text for the sake of brevity. A complete description of the datasets, models, hyperpa- rameters and metrics used can be found in the Appendix C.3
E.1 FIGURES
E.1.1 STATE DATASET
68
0.6
0.6-
6.4
AUR
5.2
5.0
0.0
50
120
350
200
250
102
15o
450
450
500
1000
150
203
250
300
250
<0a
450
Number iterations
Number iterations
Information 1101)
20-
Aomen Lo
10.
150
300 StartGrad (ours)
inttialization
inttaaization
(a) State: Preservation game
0.8
0.0
0.6
5.0
0.0
150 5b
são
zão
250 sho
sin
450 são
160
160
200
zin
s0n zia sin eso
Number iterations
Number iteratiors
stion 110f)
20-
1o7]
Infomm Infommstion
:0.
120 250 260
200
300
initializaion
iniiaalztiion
StartGrrdd (aurs)
(b) State: Deletion game
Figure 8: Comparison of the performance the StartGrad (ours), all-ones and uniform (baseline) initialization on the state synthetic dataset. A one-layer GRU (Cho et al. (014)) is employed as a clas- sifier, whereas the Extremal (Enguehard) 2023) is used as a mask-based explanation method. The first panel (a) shows the results for the preservation game formulation of the ExtremalMask (Enguehard] 2023) model, where the goal is to preserve class probability as possible, while return- ing sparse mask. The second panel (b) shows the results for the deletion game formulation of the ExtremalMask (Enguehard) 2023) model, where the objective is to find the least possible changes the original input that distorts the class probability maximally. The solid line represents the av- erage across five runs, with the shaded area highlighting the standard deviation. The preservation and deletion games objective versions are shown for both datasets. For each metric, 1 indicates that higher is better, and that lower is better.
29

---

Published as a conference paper at ICLR 2025
E.1.2
FEATURE DATASET
3.0
5.6
AUR
0.4
5.2
50
130
150
200
250 300
500
100
150
200
250
300
250
400 450
Number of Ittrations
Number iteratiors
25
Infommtton (20f)
Emeroryy (10")
20- is
1.5
20-
50 120 250 200
230
102
150
420
460
300
100
150 200
250
200
250
400 450
Number of Iterations
Number iterations
-.-- uniferm inibalization (baseline)
allones indivlization
StartGrad (ours)
(a) Switch: Preservation game
3.0
0.5
0.4
0.2
5o
130
150
200
250
200
420
459
500
50
100
150
200
250
300 250
400 450 500
Number of Iterations
Number iterations
30.
(109 (20f)
20-
Entroay (100)
Informaten
15-
320 52
250
200
250
420
300
100
400 450 150 200
250
300
250
Number Iterations
Number of iterations
-.--: uniferm iniialication (baseline)
all-ones intialiaation
StartGrad (ours)
(b) Switch: Deletion game
Figure 9: Comparison of the performance of the StartGrad (ours), all-ones and uniform (baseline) initialization on the switch-feature synthetic dataset. A one-layer GRU (Cho et al. 2014) is em- ployed a classifier, whereas the ExtremalMask (Enguehard) 2023) is used as mask-based expla- nation method. The first panel (a) shows the results for the preservation game formulation of the ExtremalMask (Enguehard 2023 model, where the goal is preserve class probability as possi- ble, while returning a sparse mask. The second panel (b) shows the results for the deletion game formulation of the ExtremalMask (Enguehard 2023) model, where the objective possible changes to the original input that distorts the class probability maximally. The solid line represents the average across five runs, with the shaded area highlighting the standard deviation. The preservation and deletion games objective versions are shown for both datasets. For each metric, indicates that higher better, and that lower is better.
find the least
30

---

Published as a conference paper at ICLR 2025
E.2 AVERAGE PERFORMANCE RESULTS
E.2. STATE DATASET
Table 5: Average performance for the ExtremalMask (Enguehard 2023 explanation method (preservation game objective) across iteration steps for the state dataset when initialized with Start- Grad, all-ones and uniformly. The reported numbers are the mean and standard deviation across five folds. For each metric, indicates that higher is better, and that lower better. Baseline refers to the initialization scheme originally used for the respective mask explanation method. Outcomes that are one standard deviation away from the second-best one are highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.719 + 0.016
0.800 0.013
0.855 = 0.008
0.856
0.008
AUP f
All-ones
0.879 + 0.006
0.847 0.008
0.860 = 0.007
0.860
0.007
StartGrad (ours)
0.840 + 0.007
0.862 + 0.007
0.861 = 0.008
0.861 + 0.007
Uniform (baseline)
0.694 + 0.011
0.728 + 0.009
0.741 = 0.010
0.742 + 0.010
AUR
All-ones
0.691 0.013
0.755 0.010
0.745 = 0.010
0.745 0.010
StartGrad (ours)
0.794 + 0.007
0.749 0.010
0.746 = 0.010
0.746 + 0.010
Uniform (baseline)
1.034 + 0.052
2.923 + 0.122
3.485 0.059
3.505 + 0.061
I [10*]
All-ones StartGrad (ours)
2.599 + 0.125
3.398 + 0.074
3,532 = 0.064
3,532 0.066
3.538 = 0.074
3.538 - 0.060
3.545 0.059
3.544 = 0.059
Uniform (baseline)
1.914 + 0.137
0.921 + 0.060
0.735 0.027
0.729 + 0.027
E [104] 4
All-ones
1.473 + 0.075
1.047 + 0.037
0.729 = 0.027
0.726 + 0.026
StartGrad (ours)
1.232 0.055
0.753 0.033
0.720 = 0.027
0.720 0.027
Table Average performance for the ExtremalMask (Enguehard 2023 explanation method (dele-
tion game objective) across iteration steps for the state dataset when initialized with StartGrad, all-ones and uniformly. The reported numbers are the mean and standard deviation across five folds. For each metric, indicates that higher is better, and J that lower is better. Baseline refers to the initialization scheme originally used for the respective mask explanation method. Outcomes that are one standard deviation away from the second-best one are highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.630 + 0.029
0.712 + 0.024
0.821 0.028
0.870 + 0.026
AUP T
All-ones
0.824 - 0.017
0.895 + 0.017
0.909 + 0.022
0.916 0.022
StartGrad (ours)
0.843 0.018
0.895 1 0.024
0.917 1 0.025
0.925 1 0.025
Uniform (baseline)
0.673 0.012
0.717 1 0.011
0.754
0.752 + 0.009
AUR
All-ones
0.481 + 0.020
0.047
0.752 + 0.010
StartGrad (ours)
0.815 + 0.004
0.763 - 0.007
0.009
0.754 + 0.009
Uniform (baseline)
1.072 + 0.049
3.948 0.064
4.326
4.377 + 0.039
[10*]
All-ones
0.739 = 0.052
0.891 0.030
4.340
4.395 + 0.042
StartGrad (ours)
4.415 + 0.037
4.416 + 0.042
4,416 0.041
4.413 + 0.040
Uniform (baseline)
1.536 + 0.053
0.479 + 0.019
0.348 0.048
0.283 + 0.037
410*]
All-ones
1.632 0.119
1.875 0.135
0.287
0.036
0.264 + 0.341
StartGrad (ours)
0.932 + 0.016
0.350 + 0.015
0.261 + 0.031
0.260 + 0.032
31

---

Published as a conference paper at ICLR 2025
E.2.2 SWITCH- FEATURE DATASET
Average performance for the ExtremalMask (Enguehard 2023) explanation method (dele- tion game objective) across iteration steps for the switch-feature dataset when initialized with Start- Grad, all-ones and uniformly. The reported numbers are the mean and standard deviation across five folds. For each metric, indicates that higher is better, and that lower is better. Baseline refers the initialization scheme originally used for the respective mask explanation method. Outcomes that are one standard deviation away from the second- best one are highlighted in bold.
Table 7:
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.825 + 0.025
0.875 + 0.041
0.913 + 0.071
0.932 + 0,078
AUP f
All-ones
0.762 + 0.016
0.705
+ 0.047
0.764 + 0.013
0.755 + 0.015
StartGrad (ours)
0.869 + 0.007
0.908
0.019
0.936 + 0.048
0.948 + 0.048
Uniform (baseline)
0.629 + 0.017
0.622 + 0.017
0.631 + 0.027
0.629 + 0.029
AUR 1
All-ones
0.695 + 0.038
0.545 +: 0.026
0.563 + 0.012
0.547 + 0.013
StartGrad (ours)
0.693 0.011
0.634 = 0.013
0.629 0.024
0.628 = 0.029
Uniform (baseline)
0.781 + 0.052
1.532
2.232 0.217
2.248 = 0.241
[109] f
All-ones
0.923 + 0.128
0.559
+ 0.053
1.410 + 0.037
1.548 + 0.058
StartGrad (ours)
1.734 0.103
2.046 0.072
2.211 0.155
2.220 + 0.168
Uniform (baseline)
2.639 + 0.163
1.842 0.077
1.410 + 0.082
1.378 + 0.074
E [10*] 4
All-ones
2.411 + 0.210
3.006 0,022
1.985 + 0.053
1.717 - 0.051
StartGrad (ours)
2.399 + 0.044
1.680 + 0.065
1.398 + 0.073
1.366 - 0.060
32

---

Published as a conference paper at ICLR 2025
F ADDITIONAL ABLATION STUDIES
F.1 VISION EXPERIMENTS WITH VGG16 AS BASELINE MODEL
F.1.1 FIGURES FOR THE VGG16 EXPERIMENT
120
120
100
100
10H 1P
10
20
Aenes
20.
dicere
StarSredd icur)
StatGrrd lounl
Lniifom
In firm
50
103
153
200
250
336
2
53
300
250
200
250
300
(a) PixeIMask: CP-Pixel f
(b) PixeIMask: CP-L1
100
25-
25-
10M
Nivene theas tn!
Iasadhal
200
150
202
252
362
5
59
:00
:50
200
250
iterations
(c) WaveletX: CP-Pixel
(d) WaveletX: CP-L1 f
1350
$10
10H C7 Paed
402
200
302
200
tbea Ins
135
Startorud lounl
Suctrras louna
365
150
250
Iterations
(e) ShearletX: CP-Pixel f
(l) ShearletX: CP-L1
Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the PixeIMask (Fong & Vedaldi 2017) (first row), WaveletX (Kolek et al., 2023) (sec- ond row) and ShearletX (Kolek (003)) (third row) explanation models. Baseline refers to the originally used initialization scheme for the respective mask explanation method. The solid line represents the average of the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng et al. 2009) samples, while the shaded area denotes the standard errors respectively. For each metric, 1 iidicattss that higher better, and that lower is better. We employ a pretrained VGG16 (Simonyan & Zisserman 2014) model as a classifier.
Figure 10:
33

---

Published as a conference paper at ICLR 2025
300
300
StartGrad (ours)
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
Uniform
150
150
100
100
0-
0.0
0.1
0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.0
0.1
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Pixel Score
Normalized Pixel Score
(a) PixeIMask: CP-Pixel
(b) WaveletX: CP-Pixel
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0
Normalized CP -Pxel Score
(c) ShearletX: CP-Pixel
Figure 11: Normalized CP-Pixel score vs. iteration steps for the PixelMask (Fong & Vedaldi 2017
(a), WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-L1 score. This target score is defined as the highest CP-L1 value achieved across all three initialization methods for each model, then normalized by the overall maximum score observed. The shaded regions represent the standard error across 500 randomly selected ImageNet validation samples (Deng et al., 2009). with pretrained VGG16 (Simonyan & Zissermann 2014) classifier. StartGrad allows all three models reach target scores in fewer iterations than the uniform initial-
ization. Compared to all-ones initialization, it accelerates PixelMask (Fong & Vedaldi (017) and
ShearletX (Kolek al. 2023) while performing slightly worse for WaveletX (Kolek et 2023 for
medium to high scores.
34

---

Published as a conference paper at ICLR 2025
300
StartGrad (ours)
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
200
Uniform
150
150
100
100
50
0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.0 0.1
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP Score
Normalized CP L11 Score
(a) PixeIMask: CP-L1
(b) WaveletX: CP-L1
300
StartGrad (ours)
250
All-ones
Iterations
Uniform
150
100
0:
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP Score
(c) ShearletX: CP-L.
Figure 12: Normalized CP-LI score iteration steps for the PixelMask Fong & Vedaldi 2017
(a), WaveletX (Kolek et al. 2023 (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-L1 score. This target score is defined as the highest CP-L1 value achieved across all three initialization methods for each model, then normalized by the overall maximum score observed. The shaded regions represent the standard error across 500 randomly selected ImageNet validation samples (Deng et al.) 2009). with pretrained VGG16 (Simonyan & Zisserman, 2014) classifier. StartGrad allows all three models to reach target scores in fewer iterations than the uniform initial- ization. Compared to all-ones initialization, it accelerates PixelMask (Fong & Vedaldi (017) and
ShearletX (Kolek et al. (2023) while performing slightly worse for WaveletX (Kolek et al. 2023) for
medium to high scores.
35

---

Published as a conference paper at ICLR 2025
F.2 COMPARISON PERFORMANCE DIFFERENCES STARTGRAD VS. BASELINES
Table 8: Median pairwise performance difference between StartGrad and baseline initialization methods across different iteration steps for 500 randomly selected validation ImageNet (Deng al. 2009) samples using a VGGI6 (Simonyan & Zisserman, (014) classifier. Baseline refers to the originally used initialization scheme for the respective mask explanation method. Statistical signifi- cance is denoted by #* and at the 1%, 5%, and 10% levels, respectively, based on an one-sided Wilcoxon signed-rank test with Bonferroni correction at the method and metric level. Since Pixel- Mask (Fong & Vedaldi (017) does only apply a mask to the pixel space, the CP-Pixel and CP-L1 scores are identical.
A CP-Pixel 1
A CP-L1 f
Iteration steps
Iteration steps
Method
Baseline initialization
50
100
300
50
100
300
All-ones (baseline)
2.29
2.04"* ***
1.65 *M
2.29"
2.04
1.65
PixelMask
Uniform
2.60
4.06
3.33
2.60 "
4.06"*
3.33
All-ones (baseline)
3.61 s*x
0.88
-0.11
1.11 3.*
0.52
0.10
WaveletX
Uniform
2.56 xxx
2.04 19
2.82 tx
0.85 *
0.67
0.85
All-ones (baseline)
180.31 6**
454.65
767. 667.72"
55.22
192.11
351.26"*
ShearletX
Uniform
173.49
462.95
770.24
52.93 *.*
201.90
370.77
Table 9: Average iteration steps required for the ShearletX (Kolek et al. 2023) method initialized with StartGrad match the corresponding performance of the same method initialized with all-ones or uniform. The table reports the time (in iteration steps) needed to match the interquartile mean (IQM) and median performance of the CP-Pixel and CP-L1 score obtained. Additionally, speedup is calculated as the ratio between the baseline iteration steps and the time taken under StartGrad initialization. All experiments use 500 random ImageNet (Deng al.. 2009) samples evaluated on pretrained VGG16 (Simonyan & Zisserman, 2014) classifier. The target metrics are obtained for running the ShearletX method across 300 iteration steps. To obtain the average and standard error, we use bootstrapping across 250 resampled sets. For each metric, f indicates that higher is better, and J. that lower better. Values that are standard errors away from the baseline values (300 for iteration, for speedup) are highlighted in bold.
Target metric
CP-Pixel
CP-L1
Reference
(at 300 iterations)
Iterations |
Speedup
Iterations |
Speedup
All-ones
IQM
33.10 + 1.55
9.05 + 0.42
41.43 + 2.30
7.24 + 0.40
(baseline)
Median
36.75 + 2.57
8.18 + 0.58
48.82 + 4.08
6.17 + 0.52
IQM
32.58 + 1.50
9.20 + 0.42
40.79 + 2.23
7.35 + 0.40
Uniform
Median
36.23 2.49
8.29 0.58
48.20
4.02
6.25 0.53
36

---

Published as a conference paper at ICLR 2025
F.3
VISION EXPERIMENTS WITH SWIN TRANSFORMER AS BASELINE MODEL
F.3.1
FIGURES FOR THE SWIN TRANSFORMER EXPERIMENT
200
250
150
150
10 PPae
30
(hareline
itane Ivo
0
102
162
250
930
03
100
T5n
200
253
Itaratiora
Iterationa
(a) PixeIMask: CP-Pixcl f
(b) PixeIMask: CP-L1
200
10M -Pieel
1:50
dore lir)
(tasslin Raanias (our))
Souttrrdd (oum) Unfom
150 Itorations
200
Ituratiors
(c) WaveletX: CP-Pixel 1
(d) WaveletX: CP-L1 1
500-
205-
500-
400
200
158-
200-
100.
allenes Stattiral Iounl
Al aiss
SuetDrus brunl
Unierm
300
360
200
250
Ituratoon
Itorations
(c) ShearletX: CP-Pixel f
(f) ShearletX: CP-L1
Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the PixeIMask (Fong& & Vedaldi 2017) (first row), WaveletX (Kolek et al., 2023) (sec- ond row) and ShearletX (Kolkk et 2023) (third row) explanation models. Bascline refers to the originally used initialization scheme for the respective mask explanation method. The solid line represents the average of the interquartile mean (IQM) performance across 250 randomly selected validation ImageNet (Deng et al. 2009) samples, while the shaded area denotes the standard errors respectively. For each metric, 1 indicates that higher is better, and J that lower is better. We employ a pretrained Swin Transformer (Liu et al. 2022) model classifier.
Figure 13:
37

---

Published as a conference paper at ICLR 2025
300
300
StartGrad (ours)
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
Uniform
150
150
100
100
0-
0.0
0.1
0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Pixel Score
Normalized Pixel Score
(a) PixeIMask: CP-Pixel
(b) WaveletX: CP-Pixel
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Pixlixel
(c) ShearletX: CP-Pixel
Figure 14: Normalized CP-Pixel score vs. iteration steps for the PixelMask (Fong & Vedaldi 2017
(a), WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-Pixel score. This target score is defined the highest CP-Pixel value achieved across all three initialization methods for each model, then normalized by the overall maximum score ob- served. The shaded regions represent the standard validation samples (Deng et al. 2009), with a pretrained Swin Transformer (Liu et al., (022) clas- sifier. StartGrad enables all three models achieve target scores with fewer iterations compared to uniform initialization scheme. Compared to the all-ones initialization, StartGrad enables to achieve target scores with fewer iterations for the WaveletX (Kolek et al.] 2023) and ShearletX (Kolek al. 2023) model, and performs on par for the PixelMask (Fong & Vedaldi 2017) model.
across 250 randomly selected ImageNet
38

---

Published as a conference paper at ICLR 2025
300
300
StartGrad (ours)
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
Uniform
150
150
100
100
0-
0.0
0.1
0.2
0.3
0.4 0.5 0.6 0.7 0.8 0.9
0.0
0.1
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP L1 Score
Normalized CP Score
(a) PixeIMask: CP-LI
(b) WaveletX: CP-L.
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0
0.0
0.1 0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized CP-L1 Score
ShearletX: CP-LI
Figure 15: Normalized CP-L1 score vs. iteration steps for the PixelMask Fong & Vedaldi 2017)
(a), WaveletX (Kolek et al. 2023) (b), ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-L1 score. This target score is defined as the highest CP-L1 value achieved across all three initialization methods for each model, then normalized by the overall maximum score observed. The shaded regions represent the standard error across 250 randomly selected ImageNet validation samples (Deng et al. 2009), with a pretrained Swin Transformer (Liu et al.] 2022) classifier. Start- Grad enables all three models to achieve target scores with fewer ilerations compared uniform initialization scheme. Compared to the all-ones initialization, StartGrad enables to achieve target scores with fewer iterations for the WaveletX (Kolek al. (2023) and ShearletX (Kolek al. 2023) model, and performs on par for the PixelMask (Fong & Vedaldi 2017) model.
39

---

Published as a conference paper at ICLR 2025
F.4
VISION EXPERIMENTS WITH VISION TRANSFORMER AS BASELINE MODEL
F.4.1
FIGURES FOR THE VISION TRANSFORMER EXPERIMENT
175
175
230
152.
200
100
10M
75
35.
(hareltes
thane lro
01
102
162
250
330
03
T5n Rterations
200
253
Itarationa
(a) PixeIMask: CP-Pixel f
(b) PixeIMask: CP-L1
45
140
120
10M
40-
10
dore lir)
(tastlin)
Souttrrd (uum)
Ranniss (our))
Ituratior
Itarations
(c) WaveletX: CP-Pixel
(d) WaveletX: CP-L1 1
500
120-
302-
200
200
105-
Alleno Stattiia Iounl
Al siss
SuatDrus bruol
Unitum
300
160
200
Ituratoor
Itarations
(c) ShearletX: CP-Pixel f
(f) ShearletX: CP-L1
Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the PixeIMask (Fong & Vedaldi 2017) (first row), WaveletX (Kolek et al., 2023) (sec- ond row) and ShearletX (Kllkk et 2003) (third row) explanation models. Bascline refers to the originally used initialization scheme for the respective mask explanation method. The solid line represents the average of the interquartile mean (IQM) performance across 100 randomly selected validation ImageNet (Deng et al. 2009) samples, while the shaded area denotes the standard errors respectively. For each metric, 1 indicates that higher is better, and J that lower is better. We employ a pretrained Vision Transformer model (Dosovitskiy et al. 2021) model as a classifier.
Figure 16:
40

---

Published as a conference paper at ICLR 2025
300
300
StartGrad
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
Uniform
150
150
100
100
0-
0.0
0.1
0.2
0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0
Normalized Pixel Score
Normalized CP Pixel Score
(a) PixeIMask: CP-Pixel
(b) WaveletX: CP-Pixel
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0
0.0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Normalized Pixel
(c) ShearletX: CP-Pixel
Figure 17: Normalized CP-Pixel score vs. iteration steps for the PixelMask (Fong & Vedaldi 2017
(a), WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dot- ted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-Pixel score. This target score is defined the highest CP-Pixel value achieved across all three initialization methods for each model, then normalized by the overall maximum score ob- served. The shaded regions represent the standard validation samples (Deng et al., 2009). with a Vision Transformer model (Dosovitskiy et al.) 2021) classifier. StartGrad enables three models achieve target scores with fewer iterations com- pared to uniform initialization scheme. Compared to the all-ones initialization, StartGrad enables to achieve target scores with fewer iterations for the PixelMask (Fong Vedaldi 2017) and ShearletX
across 100 randomly selected ImageNet
(Kolek et al. 2023) model, and performs on par for the WaveletX (Kolek et al. 2023 model.
41

---

Published as a conference paper at ICLR 2025
300
300
StartGrad (ours)
StartGrad (ours)
250
All-ones
250
All-ones
Iterations
Uniform
Iterations
Uniform
150
150
100
100
0-
0.0
0.1
0.2
0.3
0.4 0.5 0.6 0.7 0.8 0.9
0.0
0.1
0.2
0.3
0.4 0.5 0.6 0.7 0.8 0.9
Normalized L1 Score
Normalized CP L1 Score
(a) PixeIMask: CP-LI
(b) WaveletX: CP-L.
300
StartGrad (ours)
250
All-ones
Iterations
200
Uniform
150
100
50
0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.0
Normalized LP Score
ShearletX: CP-L1
Figure 18: Normalized CP-L1 score vs. iteration steps for the PixeIMask (Fong & Vedaldi 2017) (a),
WaveletX (Kolek et al. 2023) (b), and ShearletX (Kolek et al. 2023) (c) models, comparing three initialization methods: StartGrad (ours) (solid line), All-ones (dashed line), and Uniform (dotted line). The curves represent the average number of iteration steps needed to reach a normalized target CP-L1 score. This target score is defined as the highest CP-L1 value achieved across all three initialization methods for each model, then normalized by the overall maximum score observed. The shaded regions represent the standard error across 100 random ImageNet validation samples (Deng et al, 2009), with Vision Transformer model (Dosovitskiy et al. (202T) classifier. StartGrad allows all three models reach target scores in fewer iterations than the uniform initialization. Compared to all-ones initialization, it accelerates PixeIMask (Fong & Vedaldi (017)) and ShearletX (Kolek
al. 2023) while performing on par with WaveletX (Kolek al. (003).
42

---

Published as a conference paper at ICLR 2025
F.5
EFFECT OF NOISY GRADIENTS ON STARTGRAD
70
72-
50
63-
50
50-
1QM
10M
Stantrr4
Sanddd
20-
20-
0.01
0.00.0
10
- 0.05
StarGGed
10-
SwanCrrd of 6.05
StanCred
Swattoed
0-
50
100
150
200
259
300
30
100
1.0
200
250
300
Iterations
Iterations
(a) PixeIMask: CP-Pixel f
(b) PixeIMask: CP-L11 1
80
70-
60-
50
30
6--0000
gf 0.00
StanGGred
Stantreed
0.01
0.00
10-
StantGrid
2.5
Spartbred
StartGndd
SpartGrad
0.0
1.0
250
Iterations
Merations
(c) WaveletX: CP-Pixel
(d) WaveletX: CP-L1 f
250-
203-
IQM CP Ritel
150
200
Stantradd
Saatdod
0.00
0.00
Suntdodd
100
50
0.05
0.05
StarGGrdd
StanCradd
of - 0.0
of 0.20
50
100
150
200
25/0
30
100
1:50
200
250
300
Ibs rations
Iberations
(c) ShearletX: CP-Pixel f
(f) ShearletX: CP-L1
Figure 19: Investigation of StartGrad's robustness to noisy gradients by progressively multiply- ing the gradient with Gaussian noise of increasing standard deviation for the PixeIMask (Fong & 2017) (first row), WaveletX (Kolek et al., (023) (second row) and ShearletX (Kolek et al.) 2023) (third row) explanation models. The solid line represents the average of the interquartile mean (1QM) performance across 500 randomly selected validation ImageNet (Deng et al., 2009) samples, while the shaded area denotes the standard errors respectively. t indicates that higher is better, and L that lower is better. We use a pretrained ResNet18 88 (He et al. 2016) classifier.
43

---

Published as a conference paper at ICLR 2025
F.6 IMPACT OF GRADIENT- BRADDENNTBBSED INITIALIZATION ON DISTORTION AND SPARSITY DYNAMICS FOR PIXELMASK
0.8
Retained probability
0.6
0.5
0.4
0.3
StartGrad (ours)
0.2
Uniform
50
100
150
200
250
300
Iterations
(a) PixelMask: Retained probability
Retained information Pixel
StartGrad (ours)
Uniform
0.3
0.2
0.1
50
100
150
200
250
300
Iterations
(b) PixelMask: Retained information Pixel
Figure 20: Overview of optimization dynamics of retained probability (a), measured as the ratio between the masked input prediction and the original prediction, and retained information (in pixel space) (b), for the StartGrad (ours) and uniform initialization method, using the PixeIMask (Fong) Vedaldi 2017) model. We employ a pretrained ResNet18 (He et al. 2016) model as a classi- fier. The solid lines represent the average performance across 500 randomly selected ImageNet validation samples (Deng et al. 2009). For each metric, 1 indicates that higher better, and 4 that lower is better. Results indicate that the gradient-informatte mask guides the mask-based explanation method effectively to achieve minimal distortion (as in- dicated by high retained probability) early in the optimization process, consistent with theoretical predictions. However, this comes at the cost of reduced sparsity, as evidenced by higher retained information (b). In contrast, uniform initialization exhibits better sparsity (lower retained informa- tion) but struggles to retain critical information as indicated by comparable low retained probability, highlighting trade-off between higher retained probability and increased sparsity. The optimization dynamics further reveal that the early iteration steps especially crucial for retaining probability, as the mask-based explanation methods quickly converge terms of retained probability for both initialization schemes. This behavior underscores the importance of leveraging gradient signals for efficient initialization, achieving a favorable starting point for optimization terms of minimal dis- tortion (high retained probability).
that StartGrad uses to initialize the
44

---

Published as a conference paper at ICLR 2025
F.7 IMPACT OF MRADIENT-BASED INITIALIZATION ON DISTORTION AND SPARSITY DYNAMICS FOR WAVELETX
1.2
Retained probability
0.6
0.4
0.2
StartGrad (ours)
Uniform
50
100
150
200
250
300
Iterations
(a) WaveletX: Retained probability f
StartGrad (ours)
Retained information L1
Uniform
0.4
0.3
0.2
0.1
50
100
150 Iterations
200
250
300
(b) WaveletX: Retained information
Figure 21: Overview of optimization dynamics of retained probability (a), measured as the ratio between the masked input prediction and the original prediction, and retained information (T,) (b), for the StartGrad (ours) and uniform initialization method, using the WaveletX model (Kolek al.) 2023). We employ a pretrained ResNet18 (He et al. 2016) model as a classifier. The solid lines represent the average performance across 500 randomly ImageNet validation (Deng al. 2009)
samples. For each metric, f indicates that higher is better, and + that lower is better. Results indicate that the mnddient-information that StartGrad uses initialize the mask guides the mask-based expla- nation method effectively to achieve minimal distortion (as indicated by high retained probability) early in the optimization process, consistent with theoretical predictions. However, this comes the cost of reduced sparsity, as evidenced by higher retained information (b). In contrast, uniform initialization exhibits better sparsity (lower retained information) but struggles retain critical infor- mation indicated by comparable low retained probability, highlighting trade-off between higher retained probability and increased sparsity. The optimization dynamics further reveal that the early iteration steps especially crucial for retaining probability, the mask-based explanation meth- ods quickly converge in terms retained probability for both initialization schemes. This behavior underscores the importance of leveraging gradient signals for efficient initialization, achieving fa- vorable starting point for optimization in terms of minimal distortion (high retained probability).
45

---

Published as a conference paper at ICLR 2025
F.8 IMPACT OF GRADIENT-BASE INITIALIZATION ON DISTORTION AND SPARSITY DYNAMICS FOR SHEARLETX
1.3
1.2
Retained probability
1.0
0.9
0.8
0.7
StartGrad (ours)
Uniform
0.5
50
100
150 Iterations
200
250
300
(a) ShearletX: Retained probability f
StartGrad (ours)
Retained information L1
Uniform
0.3
0.2
0.1
50
100
150 Iterations
200
250
300
(b) ShearletX: Retained information
Figure 22: Overview of optimization dynamics of retained probability (a), measured as the ratio between the masked input prediction and the original prediction, and retained information (T,) (b), for the StartGrad (ours) and uniform initialization method, using the ShearletX model (Kolek al.) 2023). We employ a pretrained ResNet18 (He et al. 2016) model as a classifier. The solid lines represent the average performance across 500 randomly selected ImageNet (Deng al. 2009) vali- dation samples. For each metric, 1 indicates that higher is better, and J. that lower is better. Results indicate that the iradient-information that StartGrad uses to initialize the mask guides the mask- based explanation method effectively to achieve minimal distortion (as indicated by high retained probability) early in the optimization process, consistent with theoretical predictions. However, this comes at the cost of reduced sparsity, as evidenced by higher retained information (b). In contrast, uniform initialization exhibits better sparsity (lower retained information) but struggles retain critical information as indicated by comparable low retained probability, highlighting a trade-off be- tween higher retained probability and increased sparsity. The optimization dynamics further reveal that the early iteration steps are especially crucial for retaining probability, as the mask- ex- planation methods quickly converge in terms retained probability for both initialization schemes. This behavior underscores the importance of leveraging gradient signals for efficient initialization, achieving a favorable starting point for optimization in terms of minimal distortion (high retained probability).
46

---

Published as a conference paper at ICLR 2025
F.9
EFFECT OF UNINFORMATIVE AND ADVERSARIAL GRADIENTS ON STARTGRAD
StartGrad (adversarial gradients)
StartGrad (baseline)
60
StartGrad (unirformaivve gradients)
Uniform
I0M CP Pixel
40
20
0.
200
Iterations
(a) WaveletX: CP-Pixel
16
StartGrad (adversarial gradients)
StartGrad (baseline)
StartGrad (uninformative gradients)
Uniform
10.
1QM CP
50
100
15.
200
250
300
Iterations
(b) WaveletX: CP-L1 1
Figure 23: Investigation of the robustness of StartGrad initialization under scenarios where gradients are either uninformative adversarial (misleading) for the WaveletX model (Kolek et al. (023). In the uninformative setting, gradient values were randomly shuffled to destroy their structural signal, effectively rendering them meaningless for guiding mask initialization. In the adversarial setting, the correspondence between gradient values and mask initialization values was reversed, such that features with higher gradient values assigned lower mask values and vice versa. vhrsa.The solid line represents the average the interquartile mean (IQM) performance across 250 randomly selected validation ImageNet (Deng al. 2009) samples, while the shaded area denotes the standard emros respectively. For each metric, 1 indicates that higher is better, and I. that lower is better. We employ a pretrained ResNet18 (He al. (2016) model as classifier, Results reveal that under uninformative gradient conditions, StartGrad's performance is comparable to the baseline uniform initialization, indicating that StartGrad relies on meaningful gradient signals offer an advantage. the ad- versarial setting, performance degradation highlights the importance of smart initialization, as the algorithm struggles to recover from poorly initialized mask during optimization. These findings emphasize the critical role of robust and informed initialization in achieving optimal performance.
47

---

Published as a conference paper at ICLR 2025
F.10
EFFECT OF USING DIFFERENT GRADIENT METHODS ON STARTGRAD
60-
50
50
10M CP Piael
40
40
30-
StantGrad (haendine)
StantGred thase Ine)
StertGied Smoothrad
StartGred + SmoothGrad
50
100
150
200
750
300
50
103
15a
203
250
300
(a) PixelMask: CP-Pixel f
(b) PixelMask: CP-LIt
37.5
70-
25.0
50
52.
30.0
10
10
Startirrd Smoothin
SmuuthGral
150
200
150 Baratiins
250
Rarattons
(c) WaveletX: CP-Pixel f
(d) WaveletX: CP-L1 1
530
250
430
200
10 Poat
320
250
10M
100
50
Startirdd (bascira) Startoiad Smoothrrod
Startorud tbase ine) Startorod it SmasthGrol
50
100
150
200
250
300
30
100
150
200
450
300
(c) ShearletX: CP-Pixel
1
(f) ShearletX: CP-L11
Figure 24: Effect of using a more advanced gradient-based saliency method (here SmoothGrad (Smilkov et al. (017)) in connection with our proposed StartGrad algorithm for the PixeIMask (Fong & Vedaldi 2017) (first row), WaveletX (Kolek et al.) (023) (second row) and ShearletX (Kolek et al.) 2023) (third row) explanation models. The solid line represents the average of the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng et al., 2009) samples, while the shaded area denotes the standard errors respectively. For each metric, t indicates that higher is better, and J. that lower is better. We employ a pretrained ResNet18 8 (He al. 2016) model as a classifier.
48

---

Published as a conference paper at ICLR 2025
F.11
EFFECT OF USING A DIFFERENT TRANSFORMATION FUNCTION INSTEAD OF QTF
50
50
10M CP
-0
30
10
10
scaling!
25a
Iorations
Rerrtionn
(a) PixeIMask: CP-Pixel Pixel f
(b) PixelMask: CP-LIt
34-
10N CP Piae
10
8
-0
20
10
StartGrad Joaee Ine)
StarGred (casel ne)
StartGrae 3t1 m-ma scaling!
StarCrad (min-ma scatin)
50
100
150
200
250
300
50
100
150
203
250
355
(c) WaveletX: CP-Pixel f
(d) WaveletX: CP-LI
500
420
200
320
350
10H CP
10
200
100
100
50
StanGrid (besel StanErnd
scalingl
120
25a
Itaattons
Barations
(c) ShearletX: CP-Pixel f
(f) ShearletX: CP-L11 1
Figure 25: Effect of using a different transformation function in place of the proposed quantile transformation function (QTF) for the PixelMask (Fong & Vedaldi 2017) (first row), WaveletX (Kolek et al.) 2023) (second row) and ShearletX (Kolek et (003) (third row). We took the square root before the min-max scaling to account for the skewness typically observed in the dataset used (see section D). The solid line represents the average of the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng et al.] 2009) samples, while the shaded area denotes the standard errors respectively. 1 indicates that higher is better, and + that lower is better. We employ pretrained ResNet18 (He et al. 2016) model classifier.
49

---

Published as a conference paper at ICLR 2025
F.12
EFFECT OF VARYING A1 FOR PIXELMASK
210
230
230
200
10M CP Piue
10H
50
50-
250
Staritions
Barattions
(a) StartGrrad (ours): CP-Pixel
(b) StartGrad (ours): CP-L1
200
200
10M CP Plael
150
150
10M CO
90
50-
J. 0.
31 10
50
100
1550
200
250
300
30
100
150
200
259
310
(c) Uniform: CP-Pixel f
(d) Uniform: CP-LI1 1
230
10W CP Pivell
10M CP
100
100
01
10
150
250
Rerotoons
Rerations
(e) All-ones (baseline): CP-Pixel f
(1) All-ones (baseline): CP-L1 1
Figure 26: Effect of varying the A₁ hyperparameter for the PixelMask (Fong & Vedaldi 2017 model initialized with StartGrad (first row), uniform (second row) and all-ones (third row) on the interquar- tile mean (IQM) performance across 250 randomly selected validation ImageNet (Deng al. 2009) samples for the Ci-Pixel score (first column) and the CP-L1 score (second column). For each met- ric. indicates that higher is better, and that lower better. employ a pretrained ResNet18 (He et al.) (0016) model as a classifier. Note that the CP-Pixel score is identical to the CP-LI score as PixelMask (Fong& & Vedaldi 2017) framework does not apply the mask to a latent representation. Increasing \ improves the CP-LI and CP-Pixel metric across all initialization methods.
50

---

Published as a conference paper at ICLR 2025
F.13 PIXELMASK PERFORMANCE d
10
200
150
10M CP
100
All-ones (baseline)
StartGrad (ouns)
Uniform
50
100
150
200
250
300
Iterations
(a) PixeIMask: CP-Pixel f
250
200
150
10M
100
50
All-ones (baseline)
StartGrad (ours)
Uniffrm
Iterations
(b) PixelMask: CP-L1
Figure 27: Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the PixelMask (Fong & Vedaldi 2017) model for X 10. The solid line repre- sents the average of the interquartile mean (IQM) CP-Pixel (a) and CP-L1 (b) performance across 250 randomly selected validation ImageNet (Deng et al. 2009) samples, while the shaded area de- notes the standard emrors respectively. For each metric, t indicates that higher is better, and J. that lower is better. The CP-Pixel score is identical to CP-LI score as the mask-based model does not apply the mask latent representation. We employ pretrained ResNet18 model (He et al., 2016) model as a classifier. We see that compared to all-ones and uniform, StartGrad leads to performance boost.
51

---

Published as a conference paper at ICLR 2025
F.14 EFFECT OF VARYING A1 FOR WAVELETX
50
20
101
0
0.1.b
10
250
Itaritions
Baraations
(a) StartGrrad (ours): CP-Pixel
(b) StartGrad (ours): CP-L1
12
50.
50
10
0.1,
9.
3,
3,
Ap- 10,
Aum 10
50
100
1.50
2000
250
300
100
150
200
259
310
(c) Uniform: CP-Pixel f
(d) Uniform: CP-L1 1
70
20
40
30
20
D.1, -0
150
150
250
Rerotoons
Rerations
(e) All-ones (baseline): CP-Pixel f
(1) All-ones (baseline): CP-L1 1
Figure 28: Effect of varying the X₁ hyperparameter for the WaveletX (Kolek al. 2023) model ini- tialized with StartGrad (first row), uniform (second row) and all-ones (third row) on the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng al. 2009) samples for the CP-Pixel score (first column) and the CP-L1 score (second column). For each met- ric. f indicates that higher better, and | that lower is better. We employ a pretrained ResNet18 (He) et al. 2016) model as a classifier. Increasing A improves the CP-L1 metric across all initialization methods, but decreases the CP-Pixel metric. The hyperparameter choice of A = and X₂ 10 therefore represents the middle ground with respect to CP performance.
52

---

Published as a conference paper at ICLR 2025
F.15
EFFECT OF VARYING X2 FOR WAVELETX
16-
70
50
12
10
-11
250
Staritions
Barittions
(a) StartGrrad (ours): CP-Pixel
(h) StartGrad (ours): CP-L1
40
10e CP Plaet
20
1.
3,
2,
22 2.-2--1-1
50
100
155
200
250
300
100
150
200
239
310
(c) Uniform: CP-Pixel f
(d) Uniform: CP-L1 1
10
10w
10
0.1
150
250
Rerobonn
Rerations
(e) All-ones: CP-Pixel 1
(1) All-ones: CP-L1
Figure 29: Effect of varying the X2 hyperparameter for the WaveletX (Kolek al. 2023) model ini- tialized with StartGrad (first row), uniform (second row) and all-ones (third row) on the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng al. 2009) samples for the CP-Pixel score (first column) and the CP-LI score (second column). For each met- ric, indicates that higher is better, and that lower better. employ a pretrained ResNet18 (He et al.) 2016) model classifier. Increasing the X2 improves the scores across all initialization methods. The hyperparameter choice of \ and X2 leads the best overall performance across all initialization methods.
53

---

Published as a conference paper at ICLR 2025
F.16
EFFECT OF VARYING A1 FOR SHEARLETX
450
10M CP Pha
320
10%
220
230
120
1==12
2:-1.
250
Itaritions
Barattions
(a) StartGrrad (ours): CP-Pixel
(b) StartGrad (ours): CP-L11
250
230
100 CP Phael
10M L1
250
100
1.0
50
01.1
20.10.10
20.30
50
100
155
200
250
300
100
150
200
239
310
(c) Uniform: CP-Pixel f
(d) Uniform: CP-L11 1
250
100
10M
250
20-
0.
10.000.00
20.
150
250
Reroboons
Berations
(e) All-ones: CP-Pixel 1
(1) All-ones: CP-L1
Figure 30: Effect of varying the hyperparameter for the ShearletX model initialized with Start- Grad (first row), uniform (second row) and all-ones (third row) on the interquartile mean (IQM) performance for the CP-Pixel score (first column) and the CP-LI score (second column). For each metric, 1 indicates that higher is better, and J. that lower is better. We employ pretrained ResNet18 (He et al. 2016) model as classifier. Due to time constraints, 500 samples were used for the À and X2 2, whereas for the other hyperparameter we used sample size of 50 explaining the larger variation. With the exception of the CP-Pixel score for the StartGrad initialization, increasing the d increases both the CP-LI and CP-Pixel scores across all initialization methods for both metrics.
54

---

Published as a conference paper at ICLR 2025
F.17
EFFECT OF VARYING X2 FOR SHEARLETX
1200
320
250
620
150
400
200
250
Staritions
Barittions
(a) StartGrrad (ours): CP-Pixel
(h) StartGrad (ours): CP-L11
250
300
250
200
Phael
150
L0N
50
1.20
An 1.-1
50
100
1.50
2000
250
300
100
150
200
250
300
(c) Uniform: CP-Pixel f
(d) Uniform: CP-L1 1
310
250
200
100 CP Plve
230
120
50-
Ap 0.2 02
Aun 1.
150 Reroboons
150
250
Rerratioss
(e) All-ones: CP-Pixel 1
(1) All-ones: CP-L1
Figure 31: Effect of varying the X2 hyperparameter for the ShearletX (Kolek al. 2023) model ini- tialized with StartGrad (first row), uniform (second row) and all-ones (third row) on the interquartile mean (IQM) performance across 500 randomly selected validation ImageNet (Deng al. 2009) samples for the CP-Pixel score (first column) and the CP-LI score (second column). For each met- ric. f indicates that higher better, and | that lower is better. We employ a pretrained ResNet18 (He) et al. 2016) model as classifier. Increasing the Xo increases both the CP-L1 and CP-Pixel scores across all initialization methods for both metrics.
55

---

Published as a conference paper at ICLR 2025
F.18
VISUALIZATIONS PIXELMASK
Model prodiction: "bmiinoo
PxelMask (StariGrad)
MaskMack (Uniforn)
PoetMask (All-ones)
spoonbill
(StartGrad)
(All ones)
Model prediction: bulbul
PixelKask (StartGred)
MxeeMMsk [Uniform)
PixelMask (All-onest
applied to randomly selected validation ImageNet (Deng 2009) samples using pretrained ResNet18 (He al. 2016) classifier. The optimization was run for 25 iteration steps to emphasize early visual differences between initialization methods.
Figure 32: Illustrative visual examples of PixeIMask Fong & Vedaldi 2017
56

---

Published as a conference paper at ICLR 2025
F.19
VISUALIZATIONS WAVELETX
Model prodiction: 1aminnoo
WaveletX (StartGrad)
WaveletX (Uniform)
WaveletX (All-anes)
prodiction: spoonbill
WaveletX (StartGrad)
WaveletX (Uniform)
WaveletX (All-anes)
Model prediction: bulbul
Waveletx (StartGrra)
WaveletX (Uniform)
WaveletX (All-anes)
Figure 33: Illustrative visual examples of WaveletX (Kolkk et al. (023) applied to randomly selected validation ImageNet (Deng et al., 2009) samples using a pretrained ResNet18 (He et al., 2016) classifier. The optimization was run for 25 iteration steps to emphasize early visual differences between initialization methods.
57

---

Published as a conference paper at ICLR 2025
F.20
VISUALIZATIONS SHEARLETX
Model prediction: 1aminnoo
ShearietK (StanGrrad)
Sheartetx (Unifomn)
Sheariet (All-ones)
spoonbill
(Star:Grad)
(Uniform)
(AlI ones)
Model prediction: bulbul
ShearietX (StartGrad)
ShearletX [Uniform!
Sheerletx (All-ones)
Figure 34: Illustrative visual examples of ShearletX Kolek et al. 2023) applied to randomly selected validation ImageNet (Deng et al. 2009) samples using pretrained ResNet18 (He al. 2016) classifier. The optimization was run for 25 iteration steps to emphasize early visual differences between initialization methods. The results indicate that initializing with StartGrad preserves edges better, which are key for ShearletX's 's performance, compared to alternative initialization methods. This advantage edge preservation likely explains StartGrad's superior performance by guiding the optimization to focus critical features, such edges, for improved mask-bases explanations.
58

---

Published as a conference paper at ICLR 2025
F.21
RANDOM SEED STABILITY
F.21.1 VISION
350-
80
1.5
(hareltee
2.-
ttane
60-
102
162
250
330
03
T5n
200
253
Itaratiora
Iterations
(a) WaveletX seed 3: CP-Pixel
(b) WaveletX seed 3: CP-LI f
12.5
10k 1PCCP-11
35.0
3.5
10-
dore lirl
(tsstlin
Souttrnd
Rannias (sur))
0.4
Iterstion
Itarations
(c) WaveletX seed 5: CP-Pixel f
(d) WaveletX seed 5: CP-LI f
37.5
30
35.5
40
10H
30-
26-
1.4-
alleno
Stattrna
SuatDrus bruol Ueiiarn
Unieum
5.4
100
100
200
Itaratoor
Iterations
(c) WaveletX seed 7: CP-Pixel 1
(f) WaveletX seed 7: CP-L1 1
Comparison of StartGrad initialization (ours) with standard baseline initialization schemes for the WaveletX (Kolek et al. (023) explanation models across different seeds. Base- line refers to the originally used initialization scheme for the respective mask explanation method. The solid line represents the average of the interquartile mean (IQM) performance across 500 ran- domly selected validation ImageNet (Deng et al., 2009) samples, while the shaded area denotes the standard errors respectively. For each metric, 1 indicates that higher is better, and 4. that lower is better. We employ a pretrained ResNet18 (He et al.) (016) model as a classifier. The initialization methods show stable behavior across random seeds: StartGrad consistently outperforms uniform initialization and performs better (seeds 3, 5) or on par (for later iteration steps seed 7) with all-ones initialization.
Figure 35:
59

---

Published as a conference paper at ICLR 2025
F.21.2 TIME SERIES
Table 10: Average performance of the ExtremalMask method (Enguehard 2023) (preservation game objective) across iteration steps on the state dataset using StartGrrd, all-ones, and uniform initial- izations. Reported values are the mean and standard deviation of nine runs of the experiments using different random seeds. Each experiment averages results over five folds as stated in the main paper. Metrics denoted with (higher is better) or (lower is better). Baseline refers the origi- nal initialization scheme for the method. Results more than one standard deviation better than the second- best highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.723 + 0.024
0.805 + 0.025
0.855 + 0.030
0.857 + 0.030
AUP 1
All-ones
0.875 + 0.023
0.845 + 0.028
0.862 - 0.030
0.862 + 0.030
StartGrad (ours)
0.834 + 0.031
0.863 = 0.031
0.863 + 0.031
0.863 + 0.031
Uniform (baseline)
0.692 + 0.009
0.727 +: 0.008
0.738 + 0.008
0.739 + 0.007
AUR
All-ones
0.688 0.010
0.755 = 0.008
0
0.744 = 0.008
StartGrad (ours)
0.795 + 0.006
0.751 + 0.008
0.746 = 0.008
Uniform (baseline)
1.035 + 0.037
2.863 + 0.189
3.356 + 0.120
3.376 + 0.117
I [10"]
All-ones
2.474 0.098
3.269 + 0.112
3.409 0.117
3.418 + 0.117
StartGrad (ours)
3.353 + 0.163
3.405 + 0.134
3.421 + 0.121
3,423 + 0.121
Uniform (baseline)
1.866 + 0.088
0.941 + 0,084
0.777 + 0.063
0.770 + 0.063
E [104]
All-ones
1.561 + 0.058
1.096 + 0.059
0.773 + 0.060
0.769 - 0.060
StartGrad (ours)
1.286 - 0.070
0.820 + 0.067
0.767 + 0.060
0.766 + 0.060
Table 11: Average performance of the ExtremalMask method (Enguchard 2023) (deletion game objective) across iteration steps on the state dataset using StartGrad, all-ones, and uniform initial- izations. Reported values are the mean and standard deviation of nine runs of the experiments using different random seeds. Fach experiment averages results over five folds as stated in the main paper. Metrics are denoted with (higher is better) or (lower is better). Baseline refers the origi- nal initialization scheme for the method. Results more than standard deviation better than the second-ber are highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.648 + 0.051
0.721 + 0.050
0.817 + 0.062
0.853 + 0.064
AUP t
All-ones
0.818 0.084
0.880 + 0.083
0.884 0.072
0.886 + 0.070
StartGrad (ours)
0.815 + 0.062
0.853 + 0.074
0.871 0.072
0.875 + 0.071
Uniform (baseline)
0.674 + 0.010
0.712 + 0.009
0.743 + 0.012
0.743 + 0.013
AUR
All-ones
0.459 + 0.023
0.601 + 0.015
0.739 0.011
0.741 + 0.012
StartGrad (ours)
0.804 + 0.009
0.751 + 0.012
+
0.739 + 0.012
Uniform (baseline)
1.112 + 0.072
3.888 + 0.067
4.217 0.083
4.256 +.09
[109] t
All-ones
0.664 + 0.056
0.932 - 0.047
4.223 0.078
4.268 + 0.084
StartGrad (ours)
4.245 + 0.070
4.270 + 0.083
4.267 + 0.084
4.266 + 0.084
Uniform (baseline)
1.523 + 0.101
0.493 + 0.019
0.386 0.016
0.338 + 0.020
E [10*]
All-ones
1.753 + 0.047
1.838 + 0.071
0.343 0.026
0.317 + 0.026
StartGrad (ours)
1.004 0.033
0.414 + 0.015
0.311 0.021
0.304 + 0.021
60

---

(preservation game Published as a conference paper at ICLR 2025
Table 12: Average performance of the ExtremalMask method (Enguehard 2023 objective) across iteration steps on the switch-feature dataset using StartGrad, all-ones, and uniform initializations. Reported values are the mean and standard deviation of nine runs of the experiments using different random seeds. Each experiment averages results over five folds stated in the main paper. Metries are denoted with t (higher better) or (lower is better). Baseline refers to original initialization scheme for the method. Results more than one standard deviation better than the second-bee best are highlighted bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.878 + 0.007
0.964 +: 0.003
0.984 + 0.004
0.985 + 0.004
AUP
All-ones
0.947 + 0.002
0.961 + 0.006
0.984 + 0.004
0.986 + 0.003
StartGrad (ours)
0.920 + 0.008
0.982 + 0.003
0.986 + 0.003
0.986 + 0.003
Uniform (baseline)
0.724 + 0.012
0.725 + 0.013
0.733 + 0.018
0.732 + 0.017
AUR
All-ones
0.689 0.010
0.761
0.014
0.739 0.018
0.738 + 0.018
StartGrad (ours)
0.799 + 0.016
0.747 + 0.019
0.739 = 0.019
0.739 + 0.018
Uniform (baseline)
1.032 + 0.037
2.039 0.149
2.661 0.194
2.668 + 0.198
I [100% f
All-ones
1.978 + 0.059
2.597 = 0.162
2.678 + 0.198
2.674 0.200
StartGrad (ours)
2.617 = 0.183
2.668 = 0.193
0.200 2.675
2.674 + 0.198
Uniform (baseline)
2.266 + 0.098
1.487 + 0.080
1.126 0.109
1.162 + 0.114
E [10*] 4
All-ones StartGrad (ours)
1.933 + 0.034
1.475 + 0.075
1.137 0.110
1.137 + 0.109
1.604 0.104
1.232 = 0.099
1.138 + 0.107
1.136 + 0.107
Table 13: Average performance of the ExtremalMask method (Enguehard] 2023) (deletion game objective) across iteration steps on the switch-feature dataset using StartGrad, all-ones, and uniform initializations. Reported values are the mean and standard deviation of nine runs of the experiments using different random seeds. Each experiment averages results over five folds stated in the main paper. Metrics are denoted with t (higher better) or + (lower is better). Baseline refers to the original initialization scheme for the method. Results more than one standard deviation better than the second-best are highlighted in bold.
Iteration steps
Metric
Initialization
50
100
300
500
Uniform (baseline)
0.835 + 0.039
0.894 + 0.050
0.930 + 0.060
0.935 + 0.058
AUP f
All-ones
0.764 0.075
0.751
= 0.074
0.804 0.086
0.805 = 0.093
StartGrad (ours)
0.880 + 0.024
0.926 + 0.031
0.949 0.027
0.954 + 0.026
Uniform (baseline)
0.602 + 0.029
0.598 0.029
0.607 = 0.032
0.599 + 0.032
AUR
All-ones
0.638 = 0.054
0.527 + 0.025
0.554 0.039
0.547 = 0.043
StartGrad (ours)
0.670 + 0.020
0.607 + 0.019
0.598 0.012
0.595 + 0.011
Uniform (baseline)
0.763 + 0.053
1.620 + 0.189
2.139 + 0.211
2.107 + 0.188
I [10*] t
All-ones
0.830 + 0.099
0.553 + 0.028
1.354 + 0.180
1.570 + 0.277
StartGrad (ours)
1.740 0.094
1.987 + 0.096
2.100 0.055
2.098 + 0.044
Uniform (baseline)
2.500 0.084
1.685 0.097
1.388 0.036
1.365 + 0.031
E [10*]
All-ones
2.339 + 0.073
2.860 + 0.101
1.889 + 0.079
1.649 + 0.099
StartGrad (ours)
2.404 + 0.028
1.613 + 0.051
1.354 0.036
1.336 + 0.036
61


Reviews:
[{"review_id": "ZOT10unvpG", "reviewer": "Reviewer_Ev4u", "summary": "This paper introduces StartGrad, a novel gradient-based initialization technique specifically designed for mask-based explainability methods in deep learning. While recent research has focused on developing new objective functions for mask-based explanations, the authors identify that initialization strategies have been overlooked despite their importance for optimization performance. The key contributions are (1) StartGrad: A new initialization algorithm that leverages gradient information to provide better starting points for mask optimization, transforming gradient values into initial masks using quantile transformation. (2) Theoretical Framework: A formal analysis showing that StartGrad is provably superior at initialization compared to standard strategies in balancing the fundamental tradeoff between distortion and sparsity. The authors unify existing mask-based methods under an information-theoretic framework and show how StartGrad can enhance their performance while maintaining simplicity of implementation.", "questions": "1. The effectiveness of StartGrad is heavily dependent on the accuracy of gradient signals. Could you clarify if there are specific scenarios (e.g., highly non-linear models or adversarial settings) where gradient inaccuracies significantly degrade performance? (e.g.  understanding if and how StartGrad could be improved or adapted in scenarios where gradients are less reliable could address concerns about robustness)", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper identifies and addresses an overlooked aspect of mask-based XAI methods - initialization strategy. While much work has focused on objective functions, this is the first paper to systematically study initialization.\n\n2. The information-theoretic unification of existing mask-based methods provides a novel theoretical framework for analyzing these approaches.\n\n3. The empirical evaluation is comprehensive (1) Covers different domains (vision, time-series), (2) applicable to multiple state-of-the-art methods (PixelMask, WaveletX, ShearletX, ExtremalMask), (3) The ablation studies thoroughly examine robustness to noisy gradients and alternative implementation choices.\n\n4. The paper is well-structured and clearly written, with a logical flow from motivation to theory to experiments.", "weaknesses": "1. Reliance on Simplified Assumptions in Theoretical Analysis: The theoretical foundations of StartGrad depend on assumptions such as local linearity and neighborhood smoothness of the classifier's prediction function. While these assumptions are necessary for proving the benefits of StartGrad, they may not hold in highly non-linear or real-world scenarios, especially in models with complex architectures.\n\n2. Domain Generalizability: The experiments are mainly centered on vision and time-series data, leaving an open question about how well StartGrad performs on other data modalities such as graphs or text. \n\n3. Narrow Experimental Scope in Terms of Model Variants: The study mainly evaluates StartGrad on specific XAI models like PixelMask, WaveletX, and ShearletX, using ResNet18 and VGG16 classifiers. The conclusions would be more compelling if tested across a broader set of models, such as transformers or graph neural networks\n\n4. No analysis of explanation stability across different random seeds.\n\n5.  Missing comparison with recent developments in efficient xai methods", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eAvH5eORuB", "reviewer": "Reviewer_yTDi", "summary": "This paper introduced a new gradient-based mask initialisation technique for mask-based explanation methods called StartGrad. This initialisation method trades-off mask performance against complexity of the resulting masked representation. The authors have completed a number of experiments that demonstrate StartGrad's ability to improve the performance of mask-based methods based on speeding up their optimisation process.", "questions": "This initialisation method seems to be sensitive to $\\lambda$. Does the choice of $\\lambda$ greatly affect the performance and speed of StartGrad? And how do the choice of $\\lambda_1$ and $\\lambda_2$ mentioned in the appendix affect those performances?\n    The authors are advised to add sensitivity analysis on these hyperparameters. In addition, for $\\lambda_1$ and $\\lambda_2$, it is better for the experiment to show the impact of them separately. For example, analysis a trade-off or a similar effect on performance between them.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The theory and proofs in this paper are more than adequate, demonstrating the feasibility of balancing between complexity and information bottlenecks through information theory.\n\n2. The paper has sufficient context, motivation. The authors have a solid understanding of the trade-off in mask-based XAI.", "weaknesses": "1.  The **experiments** in this paper are insufficient. the authors have only done experiments with $\\lambda$ = 1,2 and 1,10. Since this hyperparameter directly affects the balance of trade-off, it is better to provide sufficient ablation study on lambda. \n   It would be useful for the authors to mention the impact of different orders of magnitude of $\\lambda$ on network performance. For example, a graph with $log_\\lambda$ on the x-axis and performance on the y-axis could be completed to evaluate the impact of lambda.\n\n2. The contribution of this paper does not seem obvious enough. In some metrics, the method proposed by the authors decreases in comparison to baseline in the early iterations, while it is essentially flat in the later period (as shown in Table 3). It is quite doubtful that StartGrad is able to accelerate with improving performance as the authors claimed.\n    For example,  in table 3,  StartGrad has less AUP than all ones strategy at 50 iteration steps and less AUR at 100 iteration steps. The 50 steps AUR has even better performence than 100 steps.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2a3Ndq1YHB", "reviewer": "Reviewer_1UMB", "summary": "The paper introduces a post-hoc explainability technique within the domain of XAI, focusing on a mask-based approach to identify which parts of an input are most crucial for generating an explanation. Existing approaches aim to optimize the mask such that the masked input achieves similar predictive performance as the original. However, the authors address an overlooked aspect: the initialization of these masks. Their method operates on the premise that standard initialization techniques are suboptimal. They propose and mathematically demonstrate that initializing the mask based on the most salient gradients is more effective. By using a quantile transformation function to identify the top gradients, they initialize the mask in a more targeted manner. This technique, named StartGrad, aims to improve the performance of mask-based methods from the start.", "questions": "See most of the questions on the weakness.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The writing is clear and easy to read.\nThe paper provides mathematical proofs to support most of its claims.\nThe topic is engaging and relevant.", "weaknesses": "**1. Typographical Error**\n\nIt appears that lines 217–218 contradict Equation 7. This discrepancy could lead to confusion and should be addressed for clarity.\n\n**2. Overclaims**\n\nThe paper seems to overstate certain findings. For instance, lines 175–176 present an equation without constraints on the mask, and much of the analysis substitutes entropy with a norm p, which limits generalizability. Propositions 2 and 3 focus solely on initialization rather than the optimization process as a whole. This raises a question: can we be confident that StartGrad improves optimization across all optimizers, or might this only hold in specific cases?\n\n**3. Experimental Section**\n\nResults and Scope: Table 1 does not appear to provide results for StartGrad, which limits the insight we can gain into the method’s effectiveness. Additionally, the experimental section is underdeveloped. The current setup only uses ResNet-18 on a subset of 500 random ImageNet images, which restricts the findings’ generalizability in the vision domain. Including additional datasets, such as Pascal-Part [2] and Monumai[1], where ground truth is available, would strengthen the empirical validation. It would also be interesting to test the techniques on more DNN such as Swin Transformer and ViT.\n*Convergence Speed:* It would be valuable to evaluate whether the proposed initialization method enhances the convergence speed of XAI algorithms.\n*Comparative Analysis:* Comparing StartGrad to other post-hoc explainability [3,4,5,6,7,8] techniques would provide more context and relevance to the findings.\n\n**4. Metrics Used**\n\nThe choice of metrics is unclear. A more detailed explanation of the metrics used in the evaluation would be beneficial. Additionally, considering established XAI metrics from frameworks like Quantus or Xplique could improve comparability and transparency.\n\n[1] Lamas, Alberto, et al. \"MonuMAI: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification.\" Neurocomputing 420 (2021): 266-280.\n\n[2] Chen, Xianjie, et al. \"Detect what you can: Detecting and representing objects using holistic models and body parts.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.\n\n[3] Selvaraju, R. R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., & Batra, D. (2016). Grad-CAM: Why did you say that?. arXiv preprint arXiv:1611.07450.\n\nJamil, Md Shafayat, et al. \"Advanced gradcam++: improved visual explanations of CNN decisions in diabetic retinopathy.\" Computer Vision and Image Analysis for Industry 4.0. Chapman and Hall/CRC, 2023. 64-75.\n\n[4] Wang, Haofan, et al. \"Score-CAM: Score-weighted visual explanations for convolutional neural networks.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 2020.\n\n[5] Muhammad, Mohammed Bany, and Mohammed Yeasin. \"Eigen-cam: Class activation map using principal components.\" 2020 international joint conference on neural networks (IJCNN). IEEE, 2020.\n\n[6] Srinivas, Suraj, and François Fleuret. \"Full-gradient representation for neural network visualization.\" Advances in neural information processing systems 32 (2019).\n\n[7] Sattarzadeh, Sam, et al. \"Integrated grad-cam: Sensitivity-aware visual explanation of deep convolutional networks via integrated gradient-based scoring.\" ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021.\n\n[8] Kapishnikov, Andrei, et al. \"Guided integrated gradients: An adaptive path method for removing noise.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jH6KzuX9uf", "reviewer": "Reviewer_A8Pa", "summary": "This paper introduce StartGrad, a efficient novel-gradient based mask initialize method designed for mask-based post-hoc attribution method, such as ShearletX or WaveletX. StartGrad utilizes gradients of the input features to initialize the masks more effectively, resulting fast convergence. The authors also theoretically proved that StartGrad is superior to commonly used initialized strategies like uniform or all-ones initialization. \n\nThe authors conduct extensive experiments across both vision and time-series domains. The results showed that StartGrad consistantly accelerates the optimization process.", "questions": "Can the RDE framework's attribution maps handle multi-label objects in a single image, such as an image containing both a cat and a dog? (This question is not related to the paper's rating; it's just out of the reviewer's curiosity.)\n\nTo quantify attribution methods on vision dataset, the fidelity metrics Most Relevant First (MoRF) deletion and Least Relevant First (LeRF) deletion are widely used. If these fidelity metrics were employed, comparison with other state-of-the-art attribution methods, such as the Layer-wise Relevance Propagation (LRP) family and the Class Activation Map (CAM) family, would have been possible. Why did the authors choose the conciseness-preciseness (CP) Pixel and L1 scores rather than MoRF and LeRF?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is well written; especially when the authors introduce rate-distortion explanation(RDE) framework. The methodology, theoretical background is well organized and clearly explained.\n\nGood novelty. The authors introduced a new perspective by focusing on mask initialization that has been overlooked. This approach seems to be radical rather than incremental.\n\nThe authors provide solid theoretical proofs under the RDE framework, showing that StartGrad offers a better trade-off between distortion and sparsity compared to traditional approaches.\n\nThe authors conducted extensive experiments across different domains(vision and time-series) with state-of-art RDE explanation methods, and demonstrated the effectiveness and practicality of StartGrad.\n\nExtensive ablation studies. The authors provide extensive ablation studies, analyzing effect of various components such as quantile transformation.", "weaknesses": "Lack of Visualization: It would be beneficial to include a figure explaining the StartGrad method. Additionally, providing qualitative results for StartGrad would enhance the paper—for example, illustrating the differences in attribution maps after 50 iterations with different initializations of ExtremalMask.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)