Minimizing False-Positive Attributions in Explanations of Non-Linear Models
Authors: Anders Gjølbye
Stefan Haufe
Lars Kai Hansen
Venue: NeurIPS 2025 poster ()
Keywords: Explainable AI
Interpretability
Suppressor Variables
Non-Linear Problems
Machine Learning
EEG
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: PatternLocal is a novel XAI method that refines local linearization approaches to reduce false-positive feature attributions in non-linear explanations.

Abstract: Suppressor variables can influence model predictions without being dependent on the target outcome, and they pose a significant challenge for Explainable AI (XAI) methods. These variables may cause false-positive feature attributions, undermining the utility of explanations. Although effective remedies exist for linear models, their extension to non-linear models and instance-based explanations has remained limited. We introduce PatternLocal, a novel XAI technique that addresses this gap. PatternLocal begins with a locally linear surrogate, e.g., LIME, KernelSHAP, or gradient-based methods, and transforms the resulting discriminative model weights into a generative representation, thereby suppressing the influence of suppressor variables while preserving local fidelity. In extensive hyperparameter optimization on the XAI-TRIS benchmark, PatternLocal consistently outperformed other XAI methods and reduced false-positive attributions when explaining non-linear tasks, thereby enabling more reliable and actionable insights. We further evaluate PatternLocal on an EEG motor imagery dataset, demonstrating physiologically plausible explanations.

Minimizing False-Positive Attributions in Explanations of Non-Linear Models
Anders Gjølbye' Stefan 2 Lars Kai Hansen'
Technical University of Denmark Technische Universität Berlin
3 sPhysikalisch- Technische Bundesanstalt, Berlin agjma@dtu.dk
'Charité Universirätsmediinn Berlin
1kai@dtu.
Abstract
Suppressor variables can influence model predictions without being dependent on the target outcome, and they pose significant challenge for Explainable AI (XAI) methods. These variables may cause false-positive feature attributions, undermin- ing the utility of explanations. Although effective remedies exist for linear models, their extension to non non-linear models and limited. We introduce PotternLccal, a novel XAI technique that addresses this gap. PotternLocal begins with locally linear surrogate, e.g., LIME, KernelSHAP. gradient- based methods, and transforms the resulting discriminative model weights into generative representation, thereby suppressing the influence of suppressor variables while preserving local fidelity. In extensive hyperparameter optimization the XAI-TRIS benchmark, PatternLocal consistently outperformed other XAI methods and reduced false-positive attributions when explaining non-linear tasks, thereby enabling more reliable and actionable insights. We further evaluate Pattern- Local on an EEG motor imagery dataset, demonstrating physiologically plausible explanations.
explanations has remained
Introduction
In recent years, the growing demand for transparent, accountable, and ethical Al systems across various industries and research communities has driven substantial advancements in explainable artificial intelligence (XAI). The need to understand machine learning models' decision-making processes is well-established, particularly in high-stakes domains such as healthcare, finance, and criminal justice. While researchers have made promising progress with so-called self-explainable models such as prototypical networks, these approaches often fail achieve the same downstream performance as conventional models and typically require complete retraining (22). Consequently, researchers focus significant attention on post-hoc XAI methods that apply to existing models without retraining. Post-hoc XAI methods encompass a diverse array of approaches, including architecture- specific techniques such as Layer-wise Relevance Propagation (LRP) [. concept- maseett-bsedd based explanations
like TCAV [14]. and model-agnostic frameworks such as SHAP 17 and LIME (21] These methods promise to support model validation, data quality checks, and actionable intervention suggestions
233. To provide evidence, researchers evaluate XAI methods against criteria such as faithfulness
to the model, robustness to perturbations, and human interpretability (32). However, none of these criteria guarantee that the features highlighted important are discriminative for the target variable, an assumption behind many uses of XAI [13.
35 have shown that widely used XAI methods often assign importance to suppressor variables 33 features that affect model predictions without having a direct statistical dependency with the target. Suppressors may capture correlations in noise or other auxiliary signals that models exploit improve predictions without offering causal or correlational insights. For instance, a model might predict epilepsy from seizure activity in specific brain region, but to
Several studies 34
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Instances
Ground Truth
PatternLocd
(b)
(d)
Uit imcar explainer w to decicin boundery to explon insterer
(2) Cussimact neightbur- haod prourd with locality komee II
(3) Latimate local covariacee stracture
(4) Transform linear explainer w into generative patiem
Figure 1: Conceptual step-by-step overview of how the PatternLocal method generates local expla- nations. classifier is trained on the dataset (a). with both the dataset and the model's decision boundary visualized in 2D projection. Many XAI methods produce importance maps (c) through local linearization (c.g., LIME, KerneISHAP). PatterLLcall enhances this process by transforming the local discriminative surrogate into generative explanation (d), thereby reducing the influence of irrelevant or misleading features. The resulting importance maps (c-d) can be directly compared with the ground-truth attributions (b).
optimally do so, also relies on noise probe from an unaffected region; common XAI methods may then highlight both regions, mistakenly assigning significance to the irrelevant area.
To support interpretations that rely statistical relevance, feature should depend the target variable 34 13]. The Pattern method 12] formalizes this idea by separating prediction from explanation: discriminative model makes predictions, and corresponding generative model provides explanations. Extensions such as PatternNet and PatternAtrriuttin generalize this idea to deep neural networks [15] but have shown weaknesses on non-linear suppressor benchmarks like XAI-TRIS [8]. Unlike human-centered evaluations, benchmarks like XAI-TRIS provide controlled environments with objective ground-truth explanations evaluate the robustness of XAI methods against suppressors and other uninformative variables. Also, model-agnostic agnostic methods such as SHAP, LIME, and LRP largely ignore training data structure, making them prone suppressor bias
34
35 8
In this work, we introduce PatternL.ocal, data-driven, model-agnostic XAI method that extends the Pattern approach to non-linear cases. PatternLoca converts locally discriminative explanations, produced by methods such as LIME or KemelSHAP, into generative representations that more accurately reflect statistical relevance; see Figure I This transformation helps reduce the impact of suppressor and other non-informative variables while maintaining local fidelity to the model's decision behavior. PatternLocal assumes access to representative training data and a meaningful input simplification (e.g., superpixels or feature masks) shared across samples. We evaluate PotternLocal on the XAI-TRIS benchmark, artificial lesion MRI benchmark, and an EEG Motor imagery dataset, and compare it with a range of established XAI methods. The results show that PatternLocal consistently provides more reliable and interpretable explanations without changing the underlying model.
2 Preliminary
2.1 Notation
Let X R and y C E denote the input and output spaces, respectively. Consider model f:X V. For an instance x, E X' we wish to explain the prediction f(x.). ). Because highly complex models such deep networks locally by a simpler, intrinsically interpretable model g. We therefore restrict attention to local explanations for the specific instance x, rather than to the behavior of f X' as whole.
notoriously hard to interpret, they approximated
Many XAI methods first replace x, with a dinensoameeiionnl or more semantically meaningful representation x", hxx (x.) TRDD", In image analysis, for example, x", may encode superpixels; in other settings, it may correspond to low-rank projection simply a subset of raw features.

---

Such simplification often makes for easier interpretation and, when D' D, also reduces the computational cost. Sce/Appendix for a summary of the notation.
Remark. In general, models that predict y from are termed backward or discriminative whereas models that reconstruct x (or its sources) from latent factors are called forward generative, We adopt this convention to avoid terminological clashes.
2.2 Suppressor Variables
Various studies [e.g.. [2] demonstrated that linear models may need to assign significant non-zero weight to correlated noise variables (suppressors). Given x E TR., assume each instance is generated from linear model of the form
Am E; m
my
m
E ER.
where K is the number of latent factors, A la ak] E Dx contains the activation patterns ag, and denotes additive noise. A linear backward model attempts to recover the latent vector m via
m W x,
W R
Consider D = 2 and K = with m + d and 222 = d, where the scalar m is the signal of interest and d a suppressor. The signal is exactly recovered by m 21 22 w x with w [1, -1]". Interpreting directly as feature importance, however, suggests and 22 contribute equally, even though the second channel carries only the suppressor, which removed entirely mn by subtraction. This issue with suppressor variables extends to common explainable Al methods used to interpret machine learning models 33
Remark. While suppressor variables can improve prediction accuracy, they do not reflect genuine statistical dependence on the target. Such variables should be excluded from explanations, as they have no direct indirect (causal, anti-causal, confounded) link to the target. This differs from phenomena like the Clever Hans effect or shortcut learning, where models rely on spurious yet predictive features.
2.3 Forward vs. backward models
Instead of using the weights of the linear backward model, Haufe et al. [2] propose use the corresponding forward model. They show that the weight matrix W has unique forward model counterpart
A W
where Σx Cov [x] and ΣM Cov mi which recovers the activation pattern in Eq. (1) estimated factors uncorrelated, EM diagonal and Eq. (3) reduces A σ Ex W
If the
Cov x, m.. The resulting global feature importance map for the entire data set D C X'
s Pattern (D) = Ex W,
(4)
which mitigates suppressor variables for the linear model. For example, in unregularized linear discriminant analysis, where W Σx (M4 --) with / denoting the class means, this approach completely removes the influence suppressor variables in the activation pattern.
2.4 Local explainability methods
LIME. LIME works by constructing local surrogate model of f around x!: the surrogate's 's coefficients form the feature- immoortacce explanation for x (2T]. Its original formulation is argmingeG: [(f.g g, IIx ) + 2(g), where G is a class of interpretable functions, L is loss evaluated on samples in the simplified space, II; local kernel, and Ω penalizes model complexity. In the common usage where L is squared loss and g is linear, can be written as
LIME (x.) = LIME argmin Ex
- 2 v + A AR(v),
(5)
where a regularization weighted by X, 2 is the simplified input space, and h=1 - maps simplified sample back the original space. This notation is similar to the one by Tan et al. [31] For

---

image explanations, the default choices are xxx; (z') = exp(-11 ( 1 2211//2)2 R(v) = 1 and hxg (z') = z = x, B(z') + - (1 B(z'),, with Uni( (0, 1) 1) denoting element-win wise product, r being reference value, and where B {0, 1] + {0, 1) copies each bit zj to every pixel in superpixel. As mentioned, LIME can assign non-zero importance to suppressor variables
35
KernelSHAP Gradient methods. As shown by Lundberg and Lee 17]. Tan
31
KemeISHAP can be written in the form of Eq. (5) by setting R(v) = 0 and II (z'
(D' - 1 [D' choose |='Ilo) ||='Ilo (D' - || [lo]] Tan et al. [1] further demonstrate that Eq. (5)
unifies SmoothGrad (28 and the plain Gradient method (37.
3
LotternLocal
We now unify the Pattern insight that forward model parameters reveal statistically relevant features with the ability to probe highly non-linear models locally with methods like LIME. We call this PutternLocal. PatternLocal converts any linear surrogate explanation w produced around an instance x, into a data-driven pattern that (I) suppresses false positives caused by suppressor variables, (2) remains faithful to the model because builds on the very surrogate that approximates the decision boundary, (3) preserves local fidelity by weighting the surrounding data with a local kernel, and (4) remains practical in high- sigh-dimensional settings by operating in a simplified, low-dimensional space with regularized estimation.
(1) vaaiias mitigation. Patternl Local mitigates suppressor effects by regressing the neighborhood of x, onto the local surrogate output j w x under Gaussian noise model. This reconstruction of x from I effectively dewg-weights suppressor contributions.
(2) Model faithfulness. PatternLocal inherits w from Eq. (5) which approximates the model f locally around x,. By regressing the simplified input x on the surrogate response ý rather than the true label PatternLocal ensures that explanations describe the model's ''s behavior, not the underlying task.
(3) Local fidelity. In non-linear models, the important features can vary across different regions of the input space. PatternLocal enforces locality by weighting all terms in its regression objective using the local kernel llx This ensures that the explanation focuses on the neighborhood around x, and does not reflect global structure.
(4) Feasibility and dimensionality, PatternLocal assumes sufficient local samples near x, and oper- ates in the simplified space defined by hx which aligns with Eq. (5) and reduces the dimensionality. To maintain stability in sparse or high-dimensional settings, the regression includes a regularization term Q
3.1 Formal objective
Let hx, X + IR D' be the simplified input representation used in Eq. (5) and w be its solution. Also let ÿ = w hx, (x) denote the local surrogate prediction. PatternLocal estimates pattern by regressing the local prediction onto the simplified input of the original data within the local neighborhood of x,
(6) When Q(1) = |/]| the objective can be solved with a kernel-weighted Lasso regression. For Q(u)= I 1 the problem reduces to kernel-weighted ridge regression with closed-form solution
s
PanemLocal (x.) = a argmin Ex~Px [xx: (hx, (x)) [xx, (x) - u uōl?] + (u))..
given by
a/2- = ( (EIL, [8]] + AT) "En,, [hx, (x) = COVE, [hx. (x), 0]
Varn
A
where expectations, covariances, and variances are taken with respect to the normalized kernel Ixx: (hx, (x)) over X. See Appendix B.1 for derivations. [Eq. (7) therefore expresses at, as the weighl- covariance between the simplified features and the surrogate response, scaled by its regularized variance. Also see Appendix B.3 for more sample complexity.

---

LIME
KerneSSIAA
Gradient
PatternLocal
1.0
Fealuree Importance
0.5
0.0
0..
191 0.817
M9 0.516
W9 0.774
0.027
2
-2
-
-2 0
Figure 2: crature-importaiee comparison on the XOR toy problem. We draw 2500 i.idd samples from the generative process of BB. (8) so the label depends solely on the interaction between 21 and T2, while 23 is suppressor that carries no marginal predictive signal. Each plot shows (æ1 (21,)2) pairs, colored by the local normalized feature importance that four XAI methods assign to the suppressor variable I's when applied to the smooth classifier. Every plot reports the empirical mean magnitude of this attribution across all test points. Whereas LIME, KemeelSHAP, and Gradient all attribute substantial importance to 23, PatternLocal correctly drives the attribution of 2g to (almost) zero.
3.2 Toy example
We present a small-scale toy example to better illustrate the intuition behind the Patternl Local method. Let ma, m2 (0, 1) and d ~ (10, be independent latent variables. We observe
x
[m
+ d m2
X
32
y sign(mmm2),
(8)
d
which induces a classical XOR-like dependency between (I1, (2) and the label y. The variable ms
acts as suppressor. The optimal classifier for this non-linear problem is given by ÿ = (x))(((x)).
f(x) = (m1 - xx2 22)) A smooth approximation to this classifier is f=(x) tanh((1 (1/r) fxx).. < < Fixing an input x,, the first-order (linear) approximation of the decision function at this point is fx( ~ (x.) + w (x (x) with
22. + 23
1 sech?
(H
(x,) C.
31. 33
C,
T1.
T22 203.
noting that wi W2 Wy 0. Therefore, we generally have wi * O, implying that local linear XAI methods (e.g., gradients, LIME, KernelSHAP) will incorrectly assign nonzero importance to the suppressor variable Zz.
Let Ixx, (x) = *[[|x x,l be any isotropic local kernel. For X = and identity mapping
hx. (x) = x, the PatternLocal explanation vector is defined as a = CoVII, [x, 0// Varn., [V] and
7 w x.
CoviL,, [x, Vl
w x. Consider the third component 03 = COVTIE. [23, /// VarnI [i], we compute
a Varm. [3]
Covi,, [23, x1] 0g.
COVII, [23, 22] -0 CoviI, [23, 23] =0?
Thus,
COVIL, [23, = wi COVII, [xg, x1] w2 COVIL. [I3, x22 + wa COVIL. [I3,
1 W 3 03 W2 w3) 0,
and hence ag O. Similar calculation can done for ai d2 which yields x [w), W2, 0 T. See Appendix B.2 for derivations. We conclude that Patternl.ocal correctly assigns zero feature importance to suppressor variable æ3, contrast to standard local linear methods. This is also confirmed with simulation
in Figure
5

---

y y=
y=
y=
y=1
MASK
INST.
MASK
INST.
MASK
INST.
MASK
INST.
LIN
WHITE
CORR
XOR CORR
WHITE
RIGID CORR
RIGID
WHITE
(a) Scenario CORR with 0.2, where noise is corre- lated with the label, introducing a suppressor variable.
(b) Scenario WHITE with 0.2, where noise independent (white) thus no suppressor variable.
Figure 3: Examples of 64 x 64 instances from the XAI-TRIS Benchmark across six different scenario types. Each row represents a distinct structural pattern (LIN, XOR, RIGID), and each column pair shows a binary label class (y 0 and y 1) along with the corresponding instance and attribution mask. On the right (3a), correlated noise introduces a spurious suppressor effect, while on the left (3b), white noise does not.
4
Experiments
The experimental workflow begins with the construction of synthetic image datasets whose pixel-level ground truth attribution maps are known by design, giving us complete control over class-conditional distributions We train each candidate classifier on a standard trainvvalidation split and assess its performance on an unseen test set. For every test image, the selected XAI method produces normalized importance map in the range [-1, 1]. These maps are then quantitatively compared with the ground-trut ttruh attributions using the evaluation metrics introduced below. All XAI hyperparameters are tuned on the validation split before the final assessment ensure fair comparison.
4.1 Datasets
We focus the XAI-TRIS benchmark dataset [8] and the artificial lesion MRI dataset (19] Both datasets specifically enable the assessment of XAI methods in the context of suppressor variables.
XAI-TRIS Benchmark. We follow the methodology outlined in Clark et al. 8 and provide complete details in [Appendix C.I The XAI-TRIS benchmark dataset consists 64 x 64 images (with N 40 40,000 samples) generated by combining class-dependent foreground tetromino shape with background noise. The relative weighting between signal and noise set by a. We consider two noise scenarios:
WHITE: uncorrelated Gaussian noise,
CORR: spatially smoothed Gaussian noise (leading to suppressor variables).
We evaluate three scenarios:
LIN: A -'sshpped tetromino near the top-left indicates = 0; an 'L' "L'-shaped tetromino near the bottom-right indicates y =
XOR: Both tetrominoes appear with differing signs depending on
RIGID: Each tetromino is randomly translated and rotated by 90" increments.
The tetromino shape determines the set of important (ground-truth) pixels. For training and evaluation,
each dataset is split into Dirain Deal- and Drest a 90/5/5 ratio. Figure shows examples of all scenarios. Spatially correlated noise in the CORR setting introduces suppressor pixels, background locations correlated with the foreground through the smoothing operator G. Although these pixels carry direct class information, the model can exploit them to denoise the true signal 12
"Code available htthg:///gthbbb.ommgeeeyyyyytttt

---

Artificial lesion MRI dataset. Following Oliveira et al. [9]] the Artificial Lesion MRI dataset contains grayscale medical downscaled images, with D 128 128 16384. It includes N = 7 500 samples, evenly split into Dimin. Daal. and Dess Each sample simulates a brain MRI slice with artificial lesions of predefined shapes and intensities, loosely resembling real-world white matter hyperintensities (WMH). Unlike XAI-TRIS, this dataset introduces challenges with more complex, organic lesion structures and anatomical background variations, making it a valuable benchmark for neuroimaging explainability. See See Appendix C.2 for more details.
EEG Motor Imagery dataset. We use the PhysioNet EEG Motor Movemen//lmagery dataset containing recordings from 109 subjects performing motor tasks. EEG was acquired from 64 scalp electrodes at 160 Hz following the 10-10 system. We focus on runs RO3, RO7, and R11, where subjects repeatedly opened and closed their left or right fist. Labels T1 and T2 mark left- and right-fist movements, defining a binary classification problem (y- = 1 for left, y = for right). We use 19 standard 10-20 electrodes, resample to 100 Hz, apply 0.5 Hz high-pass filter, and re-reference the common average. 3s window (Is before to 2s after onset) extracted for each event, yielding 19 channels and 300 time points per trial. After removing invalid segments, the final dataset comprises N = 4,927 balanced trials (2,456 right, 2,471 left), split into D D and Diest in a 90/5/5 ratio.
24
4.2 Models
We use two types of classifiers for the synthetic datasets: a multi-layer perceptron (MLP) and a convo- lutional neural network (CNN). For the EEG Motor Imagery dataset, we use the ShallowFBCSPNee from Schirrmeister al. 225 Each classifier f X + y is trained on the training dataset Dirain- We train the models with early stopping based on validation performance on V and then evaluate them on the test set Dust We consider a classifier to have generalized to the classification problem if its test accuracy exceeds 90%. See Apeendxxi D. for details model architecture, training, and performance.
4.3 XAI methods
We evaluate nine methods on the models, including the PatternLocal approach and baselines. The XAI methods applied are: LIME (21). Integrated Gradients (30], Saliency (27). DeepLift [26). GradientShap [1O], and GuidedBackProp [29]. Additionally. we incorporate two edge detection filters, Sobel and Laplace, which prior work shows outperform more advanced XAI techniques in suppressor variable scenarios []. Note that KernelSHAP is special case of LIME. Since we perform extensive hyperparameter optimization, we may also select KernelSHAP during this process.
For the XAI-TRIS dataset, we benchmark LIME and PatternLocal with three input-simplification schemes for hx, (-): the identity mapping, a superpixel representation, and a low-rank approximation. We evaluate the same three variants the artificial-lesion MRI dataset. Superpixels generated on a uniform grid for XAI-TRIS, whereas we use the slic algorithm for the MRI dataset.
4.4 Metrics
We evaluate explanation quality using two complementary metrics: the Earth Mover's Distance (EMD) and the Importance Mass Error (IME). See Appendix F.T for a discussion of additional metrics such as faithfulness.
Earth Mover's Distance (EMD). The Earth Mover's Distance (EMD) measures the optimal cost to transform one distribution into another. For continuous-vallnn importance map and ground truth Ft, both normalized to have the same mass, compute the EMD using the Euclidean distance as the ground metric. The optimal transport cost, OT([s) F+), ), is calculated following the algorithm proposed by Bonneel et al. ( and implemented in the Python Optimal Transport library [11.. We define a normalized EMD performance score as
EMD OT([s), F/) 18max
(10)
where 6mar is the maximum Euclidean distance between any two pixels.
7

---

Importance Mass Error (IME). As argued in Clark et al. g and Arras et al. (2), it is plausible that the given model only uses subset of the important pixels for its prediction. Thus. false positives should be preferentially penalized while false negatives should be largely ignored. To this end, we provide an additional metric, Importance Mass Error (IME),
card(
card(s)
IME
(11)
4.5 Hyperparameter optimization
For both the synthetic XAI-TRIS Benchmark and the semi-synthetic Artificial Lesion MRI dataset, we conduct extensive hyperparameter optimization across all datasets, scenarios, SNR levels, and corresponding models to ensure a fair evaluation of all XAI methods. We employ Bayesian op- timization using the Tree-structured Parzen Estimator (TPE) algorithm [4], implemented via the hyperopt package [5]. which efficiently manages mixed continuous and categorical search spaces. The optimization objective minimizes the Earth Mover's Distance (EMD) between the absolute normalized explanation and the ground truth on the validation set. Additional implementation details are provided in Appendix D.2
LINEAR
XOR
RIGID
0.30
0.39
0.31
WETTE
0.27
0.13
0.21
0.14
0.1
0.2
0.3
0.4
0.5
0.6
0.7
088
0.9
0.01
022
0.3
0.4
0.5
0.6
077
088
0.9
0.06
0.5
0.6
0.7
0.5
0.
CORR
0.27
0.11
0.21
0.25
0.07
0.13
0.00
0.05
(SNR)
(SNR)
or (SNK)
LotteroLocal
Sobel
IntigrateeKirsdients
DeepLift GradicutShap
GuidedBackprop
LIME
Laplace
Saliency
(a) Earth mover's distance (EMD)
LINEAR
XOR
RIGID
0.50
1.00
0.94
WHITE
0.78
0.37
0.61
0.25
01
02
0.3
04
0.5
06
07
08
0.9
0.04
02
0,4
0.5
0.6
0.7
08
0.9
0.65
0.5
0.6
0.48
0.94
0.94
CORR
0.42
0.56
0.72
0.38
0.49
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.1
02
0.5
0.4
0.5
0.6
0.7
0.8
0.9
0.1
0.2
013
0.44
0.5
0.5
0.7
(SNR)
(SNR)
(SNR)
Poca
IriegeatedGrudients Saliency
DeepLift
GuidedBackprop
LIME
Laplace
GrudienSShap
(b) Importance mass error (IME)
Figure 4: Quantitative evaluation of feature importance maps for the MLP model using 64 x 64 XAI-TRIS benchmark images generated by various methods, as function of the SNR. The MLP model achieves at least 90% accuracy, and the method's 's hyperparameters were optimized for EMD. The function hx, (.) is the identity function, and R(-) = Q(-) = || l|g. The top panel (4a) shows results based on EMD, while the bottom panel 4b) presents performance in terms of IME. PatternLocal demonstrates significant improvements over most methods. Shaded regions indicate
standard deviation.

---

5 Results
XAI-TRIS Benchmark. We evaluate the performance of the XAI methods on the XAI-TRIS benchmark both qualitatively and quantitatively. We conduct qualitative evaluation through visual inspection, which is standard in XAI research [e.g.. 3 17 8 and illustrate it in Appendix E. The quantitative evaluation uses ground truth along with the EMD and IME metries, as shown in Figure 4. The results presented here are for the MLP model, hy. (-) as the identity function and R(.)) Q-) ||. 122 Results for the CNN model. hx (.) as superpixels or low-rank approximation, and for R(-) Q(-) = ll I|,, are provided in Appendix E.1
In Figure 4 we present the aggregated results of the methods across the test set for the EMD and IME metrics. Overall, the PatternLocal method substantially outperforms all other methods in the XOR and RIGID scenarios. However, we observe mixed results in the RIGID CORR scenario: PatternLocal performs substantially better than other XAI methods but comparable to or worse than the filter methods Sobel and Laplace. These filter methods also performed well in Clark et al. (8 due to XAI-TRIS images having rigid bodies with sharp lines, making them especially easy for filter-based methods to detect. This advantage does not apply to images with more complex backgrounds, such as those in the artificial lesion MRI dataset.
Additional ablation studies on hyperparameters' impact and the suppressor variable's role are in Appendix G
Artificial lesion MRI dataset. We evaluate the XAI methods using the same procedure on the artificial lesion MRI dataset. The results reported here correspond to the CNN model, with hx, (-) defined over superpixels and R(-) Q(-) = || 112- Figure 5 illluttrates the performance of Putlern- Local compared to LIME and the filter- based methods Sobel and Laplace. We provide additional qualitative and quantitative benchmark results in AAppendix E.2
EEG Motor Imagery dataset. Since the EEG motor imagery dataset does not have ground-truth. we assess the explanations' physiological plausibility. This didactic example demonstrates how explanations can be meaningfully analyzed when they align with the underlying neurophysiology.
We used Pattern with the bandwidth set to the median pairwise distance, hx. (-) as the identity function and regularization. Applying Lotternlocal to the motor-imagery EEG data produced explanations with clear temporal and spatial structure. The explanations concentrate on task onset and retain distinct oscillatory patterns in the alpha and gamma bands. When apply spatio- spectral decomposition (SSD) [18 to single-trial explanations, the resulting components remain
Instance
Ground Truth
PatternLocaa
LIME
Sobel
Laplace
1.0
0.5
Fectuure Importance
0.0
0.5
Figure 5: Examples from the artificial lesion MRI dataset, showing two MRI slices (rows). The first column displays the original MRI slices with artificially generated lesions; the second column (Ground Truth) shows the true lesion locations. The subsequent columns show feature importance maps from PattemLocal. LIME, Sobel, and Laplace. We use CNN model. hx, (+), with superpixels. and set R(-) = Q(-) = || 1| for both PatternLocal and LIME. Note that the filter-based methods fail in these examples due to the lack of clear edges. Overlaid on the ground truth are superpixels with feature importance above 0.9 from PatternLocal (green) and LIME (red). PatternL explanations appear better aligned with the lesion locations. The overlap is not perfect, as the superpixel resolution limits it.

---

Temporal
SSD (8-12
SSD (30-50H2)
Source (8- (8-12Hz) Source (30-50Hz)
(a)
(b)
(d)
(e)
PatternL Local Local
mm
0.92
0.84
explanation signals of the PatternLocal explanation shows marked increase around cue onset. (b-c) First SSD patterns derived from the explanation in the alpha (8-12 Hz) and gamma (30-50 Hz) bands have good dipole fit (alpha r 0.92, gamma r = 0.84). (d-e) eL.ORETA source localization of the SSD-reconstruuceed explanation highlights peri-rolandic cortex with stronger power in the left hemisphere, consistent with contralateral activation during right-hand imagery. SSD is applied directly to single-trial explanations before source reconstruction.
Figure 6: (a)
the expected motor regions. For the right-hand example shown in Figure the source reconstruction reveals increased left-hemisphere activity, reflecting contralateral motor engagement. Across the set, dipole fits of SSD patterns indicate higher physiological plausibility for PatternLocal (0.756 = 0.090), with the raw instances showing similar results (0.738 0.013) and LIME performing worse (0.604 + 0.013). These findings demonstrate that PatternLocal yields explanations consistent with the physical forward model, enabling valid fime-freyuency and source analyses, unlike methods such as LIME, which do not permit meaningful decomposition of their outputs in this way.
physiologically interpretable and localize with eLORETA 20
6 Conclusion
This paper presents PatternLocal, data-driven, model-agnostic XAI method that improves local explanations by reducing false-positive attributions caused by suppressor variables. By transform- ing standard local surrogate outputs into more reliable feature attributions, PatternLocal enhances interpretability without retraining the underlying model. Across three datasets, including XAI-TRIS, artificial lesion MRI, and EEG motor imagery, PatternLocal consistently outperforms existing meth- ods, providing more precise and trustworthy explanations, particularly in challenging scenarios with misleading noise.
Limitations
The primary limitation of PatternLocal is its reliance on access to representative training data in the neighborhood of each explanation. While this is feasible, it may be challenging in privacy-sensitive or data-scarce environments. Additionally, because PatternLocal fits local generative model, assumes a degree of consistency or alignment across samples in the input space. This assumption holds well in structured domains like medical imaging, where standardized views are common. However, may break down in less constrained settings, such natural images or user-ge user-generated content, where spatial alignment across instances cannot guaranteed. PotternLocal reduces false-positive feature attribution; however, like other XAI methods, it can still misattribute importance, so saliency maps should be seen as suggestive rather than definitive.
10

---

Acknowledgments
We thank Scbastian Weichwald, William Jchn-Schioler, Teresa Dorszewski, and Lina Skerath for their contributions during the early stages of this work. We are also grateful to the anonymous reviewers for their valuable feedback. This work was supported by the Danish Data Science Academy, (X0694999) and VILLUM FONDEN (40516), as
funded by the Novo Nordisk Foundation well as the Pioneer Centre for AI (DNRF P1) and the Novo Nordisk Foundation ("Cognitive spaces- Next generation explainability", NNF220C0076907).
References
[1] Ameen Ali. Thomas Schnake, Oliver Eberle, Grégoire Montavon, Klaus Robuut Müller, and Lior Wolf. XAI for transformers: Better explanations through conservative propagation International Conference on Machine Learning, pages 431-451. PMER, 2022.
[2] Leila Arras, Ahmed Osman, and Wojeiech Samek. CLEVR-XAI: A benchmark datasct for the ground truth evaluation of neural network explanations) Inf. Fusion, 81:14 40, 2022.
[3] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On Pixel-Wiss Explanations for Non-Linear Classifier Decisions by Layer- Wise Relevance Propagation PLOS ONE, 10(7):1-46, 07 2015.
[4] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for Hyper- Parameter Optimization| In Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 201 I., pages 2546-2554, 2011.
[5] James Bergstra, Daniel Yamins, and David Cox. Making Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlania, GA, USA, 16-21 June 2013, volume 28 of JMLR Workshop and Conference Proceedings, pages 115-123. JMLR.org. 2013,
[6] Stefan Blücher, Johanna Vielhaben, and Nils Strodthoff |PredDiff: Explanations and interactions from conditional expectations Artif. Intell., 312:103774, 2022.
[7]
Nicolas Bonneel, Michiel van de Panne, Sylvain Paris, and Wolfgang Heidrich. Displacement interpolation
using Lagrangian mass transport ACM Trans. Gruph., 3566):158, 2011.
[8]
Benedict Clark, Rick Wilming, and Stefan Haufe. XAI-TRIS: non- lineli image benehmarks to quantify
Talse positive post-hoe attribution feature importance Mach. Learn., (199)6871-6910 2024.
[9]
Anthony J. Conger. AA Revised Definition for Suppressor Variables: a Guide To Their Identification and Interpretation Educational and Psychological Measurement, 34(1 (4,6,,5546, 1974.
[10] Gabriel G. Erion. Joseph D. Janizek, Pascal Sturmfels. Scott M. Lundberg, and Su-In Lee. Improving
performance of deep learning models with axiomatic attribution priors and expected gradients Nat. Mach.
Intell., 3(7)x620-631, 2021.
[11] Rémi Flamary. Nicolas Courty, Alexandre Gramfort, Mokhtar Z. Alaya, Aurélie Boisbunon, Stanislas Chambon, Laetitia Chapel, Adrien Corenflos, Kilian Fatras. Nemo Fournier, Léo Gautheron, Nathalie T. H. Gayraud, Hicham Janati, Alain Rakotomamonjy. levgen Redko, Antoine Rolet, Antony Schutz, Vivien Seguy, Danica J. Sutherland, Romain Tavenard, Alexander Tong, and Titouan Vayer. POT: Python Optimal Transport Mach. Learn. Res., 22:78: 1-::8, 2021.
[12] Stefan Haufe, Frank C. Meinecke, Kai Görgen, Sven Dähne, John-Dylan Haynes, Benjamin Blankertz, and Felix Bießmann. On the interpretation of weight vectors of linear models in multivariate neuroimaging Neurolmage, 87:96 1700 2014.
[13] Stefan Haufe, Rick Wilming, Benedict Clark, Rustam Zhumagambetov, Danny Panknin, and Ahoène Boubekki. Explainable Al needs formal notions of explanation correctness) CoRR, abs/2409. 145900, 2024.
[14] Been Kim, Martin Wattenberg. Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. terpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) International conference machine learning, pages 2668 2668-2677 PMLR, 2018.
In-
In
[15] Pieter-Jan Kindermans, Kristof T. Schütt, Maximilian Alber, Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne. Learning how to explain neural networks: PatterNeet and PattermAtrributinn In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.nee 2018
[16] Olivier Ledoit and Michael Wolf. [ well-condiitoneed estimator for large-dimensional covariance matrices Journal of Multivariate Analysis, 88(2):3654111, 2004.
[17] Scott M. Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions In Advances in Neural Information Processing Systems 30: Annual Conference on Newral Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4774, 2017.
11

---

[18] Vadim V. Nikulin, Guido Nolte, and Gabriel Curio. A novel method for reliable and fast extraction of neuronal EEG/MEG oscillations on the basis of spatio-spectral decomposition Neurolmage, 55(4): 1528-1535, 2011. doi: neu.oiiinur
[19] Marta Oliveira, Rick Wilming, Benedict Clark, Céline Budding, Fabian Fitel, Kerstin Ritter, and Stefan Haufe. |Benchmarking the influence of pre-training on explanation performance in MR image classification Frontiers Artif. Intell., 7, 2024.
[20]
Roberto D. Pascual-Ne Marquqi Discrete, 3D distributed, linear imaging methods of electric neuronal activity
Part exact, zero error localization IEEE SPMB, 2007.
[21] Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin, (Whyy Should I Trust You?"; Explaining the Predictions of Any Classifier In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135-1144 ACM, 2016.
[22]
Cynthia Rudin. SSoo explaining black box machine learning models for high stakes decisions and use interpretable models instead| Nat. Mach. Intell., 165):206 215, 2019.
[23] Wojciech Samek and Klaus-Robert Müller. Towards Explainable Artificial Intelligence In Explainable Al: Interpreting, Explaining and Visualizing Deep Learning, volume 11700 of Lecture Notes in Computer Science, pages 5-22. Springer, 2019.
[24] G. Schalk, J. McFarland, T. Hinterberger, N. Birbaumer, and J. R. Wolpaw, BC12000: general- purposs brain-computer interface (BCI) system IEEE Transactions on Biomedical Engineering, 51(6):0034-1043
2004.
[25] Robin Tibor Schimmeister. Jost Tobias Springenberg. Lukas Dominique Josef Fiederer, Martin Glasstetter, Katharina Eggensperger, Michael Tangermann, Frank Hutter, Wolfram Burgard, and Tonio Ball. Deep learning with convolutional neural networks for EEG decoding and visualization Human Brain Mapping.
38(11):5391-544 2017.
[26] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning Important Features Through Propa gating Activation Differences) In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3145 3153, PMLR, 2017.
[27] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep Inside Convolutional Networks: Visu- alising Image Classification Models and Saliency Maps] In 2nd International Conference on Learning Representations, ICLR 2014, Banff. AB, Canada, April 14-16, 2014. Workshop Track Proceedings, 2014.
[28]
Daniel Smilkov, Nikhil Thorat, Been Kim. Fernanda Viégas, and Martin Wattenberg. Smoothhradd removing noise by adding noise, CoRR, abs/1706.03825, 2017.
[29] Jost Tobias Springenberg, Alexcy Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. Striving for Simplicity: The All Convolutional Nea In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings, 2015.
[30] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. [Axiomatic Attribution for Deep Networks Proceed- ings of the 34th International Conference on Machine Learning. ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 33 3319-3328. PMLR, 2017.
[31] Zeren Tan, Yang Tian, and Jian Li. GLIME: General, Stable and Local LIME Explanation In Advances in Neural Information Processing Systems, volume 36, pages 0222003627777 Curran Associates, Inc., 2023.
[32] Erico Tjoa and Cuntai Guan. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI IEEE Trans. Neural Networks Learn. Syst. 32(11))4793-4413 2021.
[33] Sebastian Weichwald, Timm Meyer, Ozan Özdenizci, Bernhard Schölkopf, Tonio Ball, and Moritz Grosse- Wentrup. Causal interpretation rules for encoding and decoding models in neuroimaging Neurolmage, 11::8-55,, 2015.
[34] Rick Wilming, Céline Budding, Klaus-Robert Müller, and Stefan Haufe. Scrutinizing XAI using linear ground-truth data with suppressor variables Mach. Learn., 1 11155)1903-1193, 2022.
[35] Rick Wilming, Leo Kieslich, Benedict Clark, and Stefan Haufe. Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables In International Conference Machine Learning, ICMI, 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 37091-37107. PMLR, 2023.
[36] Jianxin Yin, Zhi Geng, Runze Li, and Hansheng Wang. Nonparametric Covariance Model Sinica, 20(1):469 1000-47,,2000 2010.
Statistica
[37] Matthew D. Zeiler and Rob Fergus. Visualizing and Understanding Convolutional Networks In Computer Vision ECCV 20/4, pages 818-833, Cham, 2014. Springer International Publishing. ISBN 978-3-319 10590-1.
12

---

A Notation
Table 1: Summary of notation.
Symbol
Description
N' C C
Input space of dimension D.
y PCR
Output space (scalar).
XEX
A data point and its label.
X,
Instance whose prediction f(x.) is explained.
D, D'
D: input dimensionality; D' < D: simplified dimensionality.
Simplifeed/semanti representation of x.
h: - DRX
: X
Forward and inverse mapping between spaces.
T:x->
Original model being explained.
9
Interpretable surrogate model fit locally around x..
())
Local kernel weighting samples by proximity in simplified space. Regularization in LIME and PattemLocal. Regularization strength.
R(-), Q0-)
A
a
Signal-to-noise mixing coefficient synthetic data. Simplified artificial sample and its space.
x',
S.IEE s Patl
Peature-importaan maps.
Coefficient vector the linear surrogate.
a
PatternLoca explanation vector via forward model regression.
Regression coefficients that minimize Eq, 5 or Eq. 6
V
W
Backward weight matrix by Pattern method. Covariance of x in used by the Pattern method.
w
Ex
21, 22
Features in the XOR example (Ix is a suppressor variable). Ground-truth relevant pixels for instance x..
F (x.)
13

---

B Mathematical details
B.1 Derivation of equation
77:
-eernel-eiightdd ridge solution
We start with the PatternLocal definition from Eq. (6)
PattemLocall
(x+)
arg min WER Dr
Ex Txx (hx, (x)) [xx, (x) u u + ||||||2
where the surrogate response is y = w hx, (x) and the kernel l1x; : IRD - Rzo is normalized. For any function
define
En [[)) = Ex [Tx; ( (Ix. (x)) [xx)],
which coincides with the usual expectation because of the kernel's normalization. Using this notation, we set
(uu Bug [xx (x) -u uūll + A /|
We can expand the squares like
hxx (x) - u uūl: hx. (x) hx. (x) ÿÿ hx, (x) u + z²² ui
and taking the kernel- expectation gives
J(u) = Eug lxx. (x) hx, (x)] 2 u En 17 hx. (x)] + u uuuu EII [T2] + Au u.
Then setting Vu)( yields
-2 ET ( hx. (x)] + 2 (EET [V A) u 0,
hence
(En, [ + 2) u = Eng hxx (x) Ø)
Because En,, [17] + X > 0, the inverse exists and the ridge solution is
ag (En.,: [r + >)
Eu,, (bx, (x) Ø]]
Assuming that hx, (x) and is centered, the first-order moments vanish and we obtain the compact form
Covi hx,, (x), 77
Varn
[ +
B.2 Derivation for toy example
Optimal classifier
The optimal classifier for the non-linear problem in Eq. (8 is given by
f(x) = (x1 - -x3)(22 + 23),
as we can write
f(x) (x) xx2 + xz) ((mi d) d)((m2 d) + d) m1m2,
and we have y = sign(m m2). However, the gradient of sign(f(xx) not well-defined when
fxx 0 (0r (x) + 23)
D3
T2
T3-
These conditions define union of two hyperplanes in R across which the sign function is discontin- uous, and hence not differentiable. address this, we instead use a smooth approximation to sign function
fx = tanh (+
0 0<T<1,
which is infinitely differentiable for x E 3 As - 0, f+(x) converges pointwise sign (f(x)), while maintaining smooth and well-defined gradient everywhere.
14

---

Gradient.
To derive the local linear approximation of the smooth decision function, we apply the chain rule
Vx-(x) = sech? (-)(x)
fx(x).
Expanding f(x) 2122 + 2123 2223
zz.. compute its gradient
22 23
vf(x)=
333
J11 22 2r3.
Evaluating at a point x, = (2i1 22n, T3+), we define
sech? (f)(x))
C,
and obtain the local weight vector
x2+ + T3+
W Vfx(x.) =
I1x T3
I1 I22 20
Thus, the first-order approximation becomes
f+(x)= R (x+) + w (x x.).
Finally, note that the identity W1 W2 Wys holds.
PotternLocal. Let IIx, (z) - x+| be an isotropic kernel and write En,, H for expectations
taken with respect to the normalized kernel weights. The non-zero variances Var(m)) = Var(m2) = 1 and Var(d) og yield the full feature-covariance matrix
+
o²
o
oz
Mn.. COVIL, [x,
d
2
+
oz
-ož
o
With identity mapping lxx, (x) = x and A = 0, the PatternLocal explanation is
CoviI, [x,0] Ø1
wwws x.
Varn,. [
Using the gradient weights w (wi, W22 wa) of Eq. (9) in the main text, obeying w1 102 + w3 0.
(1 + of) W1 0z + 0z
CoviI, [x,0] Enl. =
-o3wn + (1 + 03) - o2 wz
0z 0z + 0 w/s
The third component simplifies immediately,
Covn (ra, Ø] o w2 + wa) 0, 03 0. For the remaining two components, the same identity once obtain
Covn [21, wi,
Covn [ 5] w2-
Consequently
W1
a x
W2
which removes the suppressor variable z3 even though Wa 0.
15

---

B.3 Sample complexity of weighe- moment estimation
important to note that an explicit matrix covariance estimate (and its multiplication) is not always required, especially when the surrogate already contains an inverse-cowe covarricce form. The key computation PatternLccal is the wernel-weighted moment
CoViL
hxx (x), Ø]
0122
ÿ == -=wl lx. (x):
Varm +
which for binary classification requires only (1) the scalar local variance of and (2) the vector cross- covariance with the simplified features hx, (x). Both moments are estimated with a Nadaraya-Watson kernel smoother. For any function gxx) we write
Σ Miee Ko [|x, (x,) 011) g(x;)
Elg
E K Koll (x,) OII)
where Kg is isotropic kernel with bandwidth in the D'-dimensional interpretable space, and Nloe is the number of local samples.
Classical Nadaraya-Wait Watson analysis [36 gives for each entry of the conditional covariance
bias = O(a²),
a D' 1) var
Balancing these two terms yields the optimal bandwidth o² x Tloc -1/(4+D')) mean-squared error
and the corresponding
MSE O Mio 4/(++D)))
Equivalently, to achieve an e-accurate estimate the required local moments, understood as RMSE < one needs
Tae 1X(E (4+D)/22)
which scales only with the simplified input dimension D' D.
When D' is not small compared to Tipe. we apply Sedoit-Woll shrinkage to the local covariance. Ledoit and Wolf [16] analyze linear shrinkage estimator that remains invertible even when D' > floes well-conditioned in probability, and is asymptotically optimal when D' D'/ruse stays bounded. Importantly, PotternLocal does not require matrix inverse, since Eq. (7) divides only by the scalar variance of ý.
provide sample-complexity bounds that show exponentially faster convergence and substantially fewer samples than LIME under sub- Gaussian local sampling and regularization. PotternLccal can adopt the same sampling strategy, which further reduces the number data points per explanation without altering its closed form update.
Finally, recent locality-aware sampling results for GLIME 31
16

---

C Dataset generation
C.1 XAI-TRIS Benchmark
Following the methodology outlined in Clark et al. [8. the XAI-TRIS benchmark dataset consists of images of size 64 x 64, represented as D = ((x(m), y ) | The feature dimensionality is D 64 4 096, and the dataset contains N = 40 000 samples. These samples independent and identically distributed (i.i.d.) realizations of random variables X and Y, governed by the joint probability density function (x,
Each instance x")) comprises a foreground signal a a E RD, which is class and defines the ground truth, combined with background noise (7() PR The additive generation process is defined as
x (n) = (P(( ) (H (Hoa" ())) + a) ()
(12)
where determines the relative contribution of signal and noise. The signal ( is based on tetromino shapes that depend on the binary class label y'") ~ Bernoulli (t) To smooth the signal, a Gaussian
spatial filter H R D + R_ with standard deviation asmooth = 1.5 is applied, using maximum support threshold of 5%.
The background noise (*)) is sampled as m7(m) (N ID). Two noise scenarios considered:
1. WHITE: The operator G is the identity function, resulting in uncorrelated noise.
2. CORR: The operator G : RRD + D is Gaussian spatial filter with Smmooth = 10,
introducing spatially correlated noise.
We analyze three distinct binary classification scenarios:
1. LIN:
The operator R(*) is the identity operator. The signal (n)) is defined as a -ha-pe tetromino at located near the top-left y = 0, and as an 'L' "L'-shaped tetromino at located near the bottom-right if y 1.
2. XOR: The operator ((") is the identity operator, and each instance contains both a and a For = 0, the signals are a XOR++ a + a and a XOR signals are XOR+ a al and XOR- -a at.
-a - al. For = 1, the
3. RIGID: The operator R(1) applies rigid-body transformation. In this case, the tetromino shapes a and a are randomly translated and rotated in increments of 90%.
Lastly, the transformed signal and noise components are horizontally concatenated into matrices and normalized by the Frobenius norm and weighted sum is calculated with the scalar E [0. 1] which determines the signal-to- ratio (SNR).
This results in six scenarios across 64 x 64 image sizes. The ground truth set, representing the important pixels based on the positions of the tetromino shapes,
F+(((")) {d (L...., D} (R(*) (H a (") a + 0}.
(13)
For the LIN scenario, the presence of a is as informative as the absence of a and vice versa. Therefore, the set of important pixels for this setting
(x(()) = {d (L....,D.- : (H a 0) V (H o a #
(14)
The ground truth F+ (x) defines important pixels based on the data generation process. However, model may rely only subset of these for its prediction, which is equally valid. As in Clark et al.
g
we employ metrics to de-emphasize the impact of false-negative omissions.
For model training and later analysis, each dataset is split into three subsets Dirain D and Drest with 90/5/5 ratio.
XAI-TRIS Benchmark for Ablation Study Additionally, we generate another XAI-TRIS bench- mark dataset following the same procedure but for D 82 64 and N 10000. This is only used for the ablation study in Appendix G
17

---

C.2 Artificial Lesion MRI Benchmark
The methodology of constructing the Artificial Lesion MRI Benchmark dataset is described in Oliveira et al. [99] and we provide brief overview here. See Figure for examples of instances.
Source images. They selected 007 TI-weighted axial volumes of healthy adults (22-37 y) from the Human Connectome Project (HCP). After the standard PSL/FREESURFER "minimal" preprocessing and defacing pipelines, each slice with background occupancy < 55% was retained. Raw matrices (260 x 311) were zero- pro--padeed vertically, centre-cropped horizontally to 270 x 270, clipped the intensity range [0, 0. 0. and bicubic somn-sampled to 128 x 128 so as to match the resolution used in the main paper.
Lesion synthesis. (i) Prototype masks. White Gaussian noise (256 256) was smoothed with 2 kernel, Otsu-thresholded, and morphologically processed (erosion-openile ---niinnenerooono) to yield irregular blobs. (ii Shape selection. From every connected component they computed the compactness
4 p²
CE (0, 1].
where A is area and its perimeter. Components with 2 0.8 were labelled regular, those with S 0.4 irregular. (ii)) Boundary refinement. Candidate shapes were zero-padded by two pixels and softened with 0.75 Gaussian blur, yielding final binary masks L {0, 1) 256
Image composition. For every background slice B: (i) Draw k ~ [ 4, 5} lesions Li*k of the same shape class. (ii) Uniformly translate each L, inside the brain bounding box, rejecting placements with overlap (IOU> 0). (iii) Aggregate the lesion mask (xx, y) = max, L (z, y) and mix intensities
x (xx, v) [1 L(z, v) + (xx
fixing the signal-to-noke weighting 0.5 for all experiments. Pixels with L(x, y) constitute the exact attribution ground truth supplied to evaluation metrics.
MASK
INST.
MASK
INST.
MASK
INST.
1.0
IRREGULA AR REGULAR
0
0.5
0.0
0.5
1.0
Figure 7: Example of the Artificial Lesion MRI Benchmark dataset. Each pair panels shows the lesion mask (MASK) and the corresponding synthetic Tl Tl-weighted slice after intensity blending (INST.). Three independent examples are displayed for each of the two lesion-shape classes: compact regular lesions (top row, 0) and low-compactness irregular lesions (bottom row, y 1). All slices are 128 x 128 pixels.
18

---

D Experimental details
D.1 Model architectures and training
All classifiers are implemented PyTorch 2.6 and PyTorch-Lightning 2.5. Training, early stopping, and checkpointing are handled entirely by the Lightning trainer; the relevant source code the GitHuu repository for full transparency. Unless stated otherwise, all models are trained with the same optimization hyperparameters listed in Table 2
Multi-layer perceptron (MLP). The network a 4-layer perceptron that receives a flattened input vector. Hidden dimensions are (128, 64, 62, 16} with ReLU activations, batch-normalizate after every linear layer, and 25% drop-out regularization. The output layer is fully connected with [ logits.
Convolutional neural network (CNN). The canonical CNN processes single-channel square images of side 64 x 64 or 128 x 128 pixels, selected by the input_size argument (4096, 16384 vector elements, respectively). stacks four convolution blocks
Conv(1->4, + (4~>8, - Conv(8 (16 k=4) - Conv(16-332, k=4),
each followed by ReLU and 2x2 max-pool, after which the feature map flattened and passed to a single fully connected output layer. When the input is 8x8, we employ compact architecture with two 3x3 convolutions (channels 1--8+16),, each followed by batch-norm, ReLU, max-pool, and 25% drop-out. The resulting 4x4 feature map feeds single fully connected classifier.
Optimisation and early stopping. Parameters updated with Adam (7o x 10-4). A ReduceLROnPlatesn scheduler monitors validation accuracy and scales the learning rate by 0.1 after 100 epochs. Training terminates via early stopping under the same patience condition, and the model state with the highest validation accuracy is restored before final testing. Mini-batch contain 128 samples, and the hard cap on epochs is 500, although the best model usually after around 60-120 epochs.
Table 2: Hyperparameters used for model training.
Hyperparameter
Value
Initial learning rate
1 x 4
Batch size
128
LR-scheduler factor
0.1
Patience (LR + early stop)
100 epochs 500
Maximum training epochs
Optimiser
Adam
Loss function
Cross-entropy
All random seeds are fixed through pl. exactly reproducible. For every model used, the test accuracy exceeds 90 %.
to ensure that the reported numbers
D.2 Hyperparameter optimization
Optimization procedure. We perform Bayesian optimization with the Tree-of- Parzen Estimators (TPE) algorithm [4] as implemented in the hyperopt package [5]. Each run consists of 200 trials; during every trial, the candidate configuration is evaluated the validation set, and the mean Earth Mover's Distance (EMD) between the normalized explanation and the ground-truth mask is minimized.
Locally linear methods. LIME and PatternI.co Local are tuned independently. Patternl Local optimizes the three parameters listed in Table 3 and the LIME parameters, yielding four variables in total.
19

---

Table Hyperparameter search spaces explored by the Bayesian optimizer. A dash (-) indicates that no hyperparameters were tuned. For the 128 x 128 images, the upper bound of the bandwidth for LIME and Patternl increased.
Method
Hyperparameters (range)
LIME
Bandwidth E [0.5. 30.0]
Bandwidth E [0.5, 30.0). Regularization À [10-5, 102)
PatternLocal
Kernel fgaussian, epanechnikov]
Laplace
Sobel
Saliency
GuidedBackprop
DeepLift
Integrated Gradients
Tasteps
[10, 200). Method E (riemann_ttrapeee trapezoid, gausslegendre)
GradientShap
Nsaaplls [5, 50]. noise E [0.00, 0.30]
Baseline methods. We run with their default implementations for filter-based methods (Laplace, Sohel) and several XAI methods (Saliency, DeepLift, GuidedBackprop), Integrated Gradients and GradientShap have modest search spaces that cover the step count, integration scheme, and, for GradientShap, the noise characteristics.
D.3 Computational details
Hardware. All experiments were executed on a local nigh-perrformncce computing (HPC) cluster
equipped with Intel Xeon E5-2650 v4 CPUs (12 cores, 24 threads, 2.20 GHz) and 256 GB RAM per node. No dedicated GPUs were required. Jobs were managed with SLURM 22. 05 and ran under AlmaLinux 9.5.
Software.
The codebase is primarily written in Python 3.13.0 O. Key libraries are:
NumPy 2.1.3,
PyTorch 2.6 for model definition,
PyTorch-Lightning 2.5 for model training and evaluation,
scikit-learn 1.6. for classical baselines and metrics, hyperopt 0.2.7 for Bayesian optimization,
0.9.5 for Barth-Mover-Distanee evaluation,
Hydra 1.3.2 for experiment handling.
The requirements. txt file includes exact versions and is included in the project repository.
Runtime for classifiers. Training single classifier XAI-TRIS the Mrificial-Lesionn MRI data required ~10-15 min on one CPU core. By far the dominant cost stemmed from hyperparameter optimization of the XAI methods. For each combination of scenario (LIN, XDR, RIGID) and signal- to-noise ratio (a E (0.1...., 0.90)) we ran 200 TPE iterations, evaluating on the full validation split (500-2000 instances, depending the dataset). After 1500-150 iterations, confirmed the conver- gence plateaus. Each optimization run was parallelized across 12 CPU cores via multi multiprocessing resulting wall-clock time of 12-24 hours.
20

---

Runtime for XAI methods. We report run- time measurements for the XAI methods in Table 4 PatternLocaal adds a single weighted regression per explanation on top of the cost of the local surrogate. Since fitting the surro- gate (e.g., with LIME) is the most expensive step, the additional regression keeps the run- time the same order of magnitude. More- over, PatternLocal only regresses samples in the local neighborhood, and in practice, we further reduce cost by discarding samples with very low weight. Like LIME, we use superpix- els to reduce computational cost and include low-rank approximation. See more details in Appendix E.
Table 4: Runtime of XAI methods.
XAI method
Avg. runtime + std
PatternLocal
12.9 = 1.5
LIME
7.4 0.3
PatternLoccal + Superpixel
3.6 1.6+0.3 0.3
LIME + Superpixel
1.4 1.4+0.1
PatteraLocal LowRank
7.7 0.4
LIME LowRank
6.9
Although runtimes difficult to compare across different hardware and implementations, control for this by keeping all settings fixed and averaging over 1000 instances of the XAI-TRIS dataset on an AMD EPYC 9124 16c/32t 3.0 GHz machine (also for LIME inference). Given that PatternLccll is only slightly slower than other model-agnostic methods such as LIME or KemnlSSHA,, while consistently achieving better explanations, believe the overhead is justified.
21

---

E Extended results
E.1 XAI-TRIS Benchmark
Quantitative evaluation. In Figure 8 we investigates how different choices of hx, (.) and Q(-) =
| affect explanation quality. We compare Patternlooca and LIME with low-rank representation and superpixel representations.
LINEAR
XOR
RIGID
0.19
0.29
WHITE
0.26
0.07
0.13
0.25
0.01
0.04
05
06
077
08
02
0.39
0.36
0.20
CORR
0.26
0.11
0.20
0.14
0.07
0.1
0.2
0.5
044
0.5
0.6
0.7
08
0.9
0.1
0.2
0.5
0.4
0.5
0.6
0.7
08
09
0.1
0.2
0.3
04
0.5
0.55
0.7
0.5
09
(SNR)
(SNR)
(SNR)
LattemLocaal (LawRank LASSO)
(Superpixel)
Patteral Local (Lowwank)
Patteml.ccal (Superiixel)
(Lowwank)
(a) Earth mover's 's distance (EMD)
LINEAR
XOR
RIGID
WH1TT
0.37
0.68
0.73
0.31
0.36
0.50
0.04
0.37
044
0.45
0.83
0.94
CORR
0.29
0.1
02
0.3
0.4
0.5
0.6
07
08
0.9
0.11
01
02
0.3
0.4
0.5
0.6
0.7
88
09
0.34
0.1
0.2
0.3
0.
0.5
0.5
(SNR)
Patteralocca
PateelLocal (LowRank + LASSO)
LIME
LIME (Superpixel)
PatteraLocal (LowRank)
PatierLoccal (Supapixel)
LIME (LowRank)
(b) Importance mass error (IME)
Figure 8: Quantitative evaluation of feature importance maps for the MLP model using 64 x 64 XAI-TRIS benchmark images generated by variants of the PatternI Local and LIME method, as a function of the SNR. The MLP model achieves at least 90% accuracy, and method hyperparameters were optimized for EMD. The top panel (8a) shows results based on EMD, while the bottom panel (8b) presents performance in terms of IME. Shaded regions indicate standard deviation as the standard eтror is too small to be visible.
Remark. Not all curves decrease with a, which can be explained by two effects. For edge-detector baselines such as Sobel and Laplace, increasing a sharpens tetromino boundaries and causes the filters to respond outside the true region, increasing the EMD/MME For local surrogate methods such as LIME and PatternLocal higher improves the signal-to-noke noise ratio and often leads to better local fits. However, the spike around a Rs 0.9 on WHITE-LINEAR is likely numerical: as shown in Figure 5] the dataset becomes rank-deficient at high a, making the regression unstable.
Table 5: Empirical matrix rank as a function of contrast a. The drop to rank explains the spike for LINEAR WHITE, while RIGID WHITE remains full-rank longer, resulting smoother curves.
(a) LINEAR WHITE
(b) RIGID WHITE
a
0.5
0.6
0.7
0.8
0.9
Q
0.6
0.7
0.9
rank
4096
4096
4096
3554
rank
4096
4096
4096
4096
2881
22

---

Grouad Touth
Internted Gradeens
Guited Backgrop
Instance
PutteruLLcoo
LIME
Sobel
Laplsce
Sallency
DeepLift
Gradicushupp
L12 (022)
L
0.5
301 00
WITTT
Fesuue Impartune
208 (0
RaCia (0.5)
0.
2ICID (0.3)
Figure 9: Qualitative comparison on XAI-TRIS for the MLP model, hx. (-) being the identity function
and Q(-) = || 111- Each row shows one of the six benchmark configurations: LIN WHITE (a 0.2). LIN CORR (a = 0.2), XOR WHITE (a = 0.2), XOR CORR (a: = 0.2), RIGID WHITE (a 0.5). and RIGID CORR (a 0.3). Columns display the input Instance, the Ground Truth attribution mask, our PatternLocal, and nine baselines (LIME, Sobel, Laplace, IntegrateGGradient Saliency, DeepLift, GradientShap, GuidedBackprop) PatternLocal aligns closest with the ground truth across all scenarios, whereas filter-based methods only succeed in the RIGID CORR case.
we show qualitative examples of explanations the XAI- TRIS Benchmark dataset obtained with the MLP model. For PatternI Local and LIME we set hx (-) to the identity and use the squared l f2-penalty, R(-) = Q(.)
Qualitative evaluation. In Figure 9
I| It is evident that PatternLocal produces better explanations, most markedly in the WHITE scenarios and consistently better than LIME in every setting. As expected, the filter-based methods (Sobel, Laplace) perform competitively only in the RIGID CORR case.
Figure 10 shows examples of explanation for different choices of hx, (.) and Q(-) = || 1l1. We compare PatternLocal and LIME with low-rank representation and superpixel representations. We use the naming PatternLocal (hx. (-) = id, Q(-) = 11-112), PatternLocal LowRank (hx, (.) = lowrank, Q(-) = ||| (12), and PatternL LowRank LASSO (hx. (-) superpixel and Q(-) = || 111).
F,2 Artificial Lesion MRT Benchmark
The Artificial Lesion MRI Benchmark dataset 19 was created to provide semi-realistic, fully controlled setting for evaluating XAI methods on MRI-based lesion detection. Each image is down-sampled axial slice of size 128 x 128 containing one more synthetically inserted lesions. The input dimensionality is highly imbalanced (N 2 500 < D = 16384). This pronounced D> N regime makes local surrogate approaches such LIME, KernelSHAP, and our PatternLocal particularly sensitive to the choice of the imput-iimllffiattion mapping hx, (+) and to regularization. In all experiments, surrogates, and set both regularizers to the squared & norm, R(-) = Q(.) ||.113. The classifier is CNN model achieving 90% accuracy on the test
therefore segment each slice into superpixels with slic before fitting the
Quantitative evaluation. Table rjrppors the mean Earth Mover's Distance (EMD) and Importance Mass Error (IME) over the entire test set. PatternLccal and LIME underperform relative to the other XAI methods. We suspect that this is primarily because the latter suppresses many low-level attributions that inflate the error of the PatternLocal and LIME methods. Crucially, this strategy considers every superpixel, even those that neither method deems highly relevant. It is not uncommon to perform feature selection or only consider the most important superpixels. more realistic assessment is obtained by considering only superpixels whose importance exceeds a high threshold.
23

---

PutternLocal LASSO
PatternLocaa Buwkaak
PatternLocul Superpixel
Instance
Truth
PatternLocal
LIME
Low
Superpixel
(c
KET
L
L
12
+
+
L
L
L
L
120
T
Feature Importance
L
L
L
L
L
L
H
+
0000 Xnn
L
L
L
L
RIOID (0.5)
-0.5
RICID (0.3)
Figure 10: Qualitative comparison on XAI-TRIS for the MLP model for various choices of hx. (-) and Q(-) || |/.. Each row shows one of the six benchmark configurations: LIN WHITE (a 0.2), LIN CORR (a = 0.2),, XOR WHITE (a 0.2), XOR CORR (a 0.2), RIGID WHITE (a = 0.5). and RIGID CORR (a 0.3). Columns display the input Instance, the Ground Truth attribution mask, and four variants of PatternLocal and three variants of LIME. PatternLocal is robust across representations, degrading only the most challenging RIGID CORR setting when low-rank approximation used.
LIME benefits markedly from the low-rank approximation but deteriorates for superpixels.
Restricting the comparison to above 0.9 Table 7) changes the ranking: PatternLocal now attains the lowest EMD and markedly reduced IME, whereas LIME and the edge filters deteriorate. The gradient-based methods improve their IME scores but exhibit extreme variance, indicating that their few
attributions positioned correctly only about half the time. Additionally, average, PatternLoca selects 2.5 1.2 superpixels per slice vs. 4.5 + 2.1 for LIME. The contrast between the two evaluation protocols highlights well-known drawback of superpixel explanations: much of the attribution mass is distributed over moderately important segments that are irrelevant for human interpretation yet heavily penalize set-wise metrics such EMD and IME. By focusing on high- high-confidence regions, PatternLocal exposes its intended advantage in reducing false-positive attributions.
Table Comparison on the Artificial Lesion MRI benchmark (all features).
Method
EMD
IME
PatternI (superpixel)
0.166 + 0.034
0.969 + 0.016
LIME (superpixel)
0.171 0.025
0.971 + 0.009
Laplace
0.131 + 0.022
0.956 0.011
Sobel
0.137 0.022
0.950
Integrated Gradients
0.101 + 0.021
0.026
Saliency
0.124 0.022
0.933
DeepLift
0.103 + 0.021
0.826
GradientShap
0.102 0.021
0.898
Guided Backprop
0.124 + 0.021
+.0118

---

Table 7: Comparison on the Artificial Lesion MRI benchmark (top features).
Method
EMD
IME
PatternLocal (superpixel)
0.102 + 0.065
0.827 + 0.057
LIME (superpixel)
0.196 0.062
0.976 0.077
Laplace
0.185 + 0.056
Sobel
0.276 0.051
1.000++00003
Integrated Gradients
0.166 = 0.038
0.538
Saliency
0.171 0.048
0.804
DeepLift
0.171 = 0.047
0.569
Grap
0.158 0.051
Guided Backprop
0.167 0.049
0.803 = 0.336
Qualitative evaluation. Explanations are shown in Figure Across all slices, PatternLoca highlights fewer superpixels, and the ones it does highlight align more closely with the ground-truth lesions than those produced by LIME. Both methods are nonetheless limited by the superpixel resolution: lesions that cross superpixel boundaries cannot be perfectly recovered. Filter-based methods (Sobel, Laplace) fail completely because the lesions are diffuse and lack strong boundaries; XAI methods (Integrated Gradients, Saliency, DeepLift, GradientShap, GuidedBackprop)) tend to produce noisy attribution maps with only occasional alignment to the lesions.
Ground
PatternLocal Superpisel
LIME
Integrased Gradienu
Guiide
Instance
Truth
Superpisel
Subel
Laplace
Saliency
Dzepliitt
Grodienthhup
Backsrop
1.0
Imaootanee
0,0
-0.3
Figure 11: Examples from the artificial lesion MRI dataset, showing MRI slices (rows). The first column displays the original MRI slices with artificially generated lesions; the second column (Ground Truth) shows the true lesion locations. The subsequent columns show feature importance maps from PotternLocal, LIME, Sobel, Laplace, IntegratedGradieets, Saliency, DeepLift, GradientShap, and GuidedBackprop.. We use a CNN model, hx, (.). with superpixels, and set R(.) Q(-) = || 112 for both Patternl and LIME. Overlaid on the ground truth are superpixels with feature importance above 0.9 from PatternLocaal (green) and LIME (red). Patternl Local explanations appear better aligned with the lesion locations.
25

---

Temporal
SSD (8-12Hz)
SSD (30-50H2)
Source (8- 12-12
Source (30-50Hz)
(b)
(d)
(e)
Instance
0.84
-= 0.68
LIME
0s
Is
0.57
r 0.49
Pattern ocal
0
0.90
Figure 12: Physiological validation of BEG explanations for representative right-hand motor imagery trial. (a) Channel-averaged explanation signals for a single instance, LIME, and PatternLocal. PatternLocal shows sharp, cue-locked increase around stimulus onset, whereas LIME produces broader and less distinct response. (b-c) Apatio-Spectrey Decomposition (SSD) topographies derived from the explanation signals in the alpha (8-12 Hz) and gamma (30-50 Hz) ranges, compared with SSD patterns from the raw EEG. The r values denote the goodness of fit of a single dipole to each pattern; PatternLocal yields the most consistent and physiologically plausible fits across both frequency ranges. (d-e) Source estimates obtained with eLORETA from SSD-filtered explanation signals show activation over peri-rolandic cortex with stronger responses contralateral to the imagined hand. Overall, PatternLocal provides spatial patterns and source activity that better reflect known motor-imagery physiology than LIME under matched conditions.
E.3
EEG Motor Imagery dataset
Figure 12 illustrates a trial from the EEG motor imagery experiment used for physiological valida- tion. It shows how explanation signals capture task-relevant temporal, spectral, and spatial patterns compared to the raw EEG. The figure summarizes results across methods, highlighting that Pattern- Local produces more distinct cue-locked responses and source activations consistent with known motor-imagery physiology.
26

---

Additional Discussion
F.1 Evaluation metries
Many studies in explainable Al evaluate attribution methods using faithfulness-wwt based metrics such as insertion, deletion, accuracy drop. These metrics measure how the model's output changes when parts of the input are removed or revealed. They do not use ground-trute information about which features are relevant, but instead reward explanations that match the model's predictive behavior, suppressor-aware benchmarks, this is problematic. Removing these features still alters the model's output if model relies on suppressor variables due to statistical dependencies rather than the true signal. As result, faithfulness metrics can incorrectly reward attributions that highlight suppressors. This behavior follows from how such metrics are defined and is not simply an implementation artifact.
Wilming et al. 33 provide theoretical analysis of suppressor variables and show that many popular XAI methods assign non-zero importance suppressor features when the data contain correlated noise. Blücher et al. [6] further show that the PredDiff method, which is based on conditional expectations, is closely connected to Shapley values. Since Shapley values distribute credit according to statistical dependence rather than causal relevance, they can also assign importance to suppressors or other correlated but task irreeerteeevnn features. Therefore, evaluation metrics implicitly following this logic cannot distinguish between true causal relevance and statistical influence. They may judge an explanation as "faithful" even when it highlights misleading parts of the input.
Haufe et al. 3 argue that faithfulness to the model is insufficient for correct explanation and that mislead scientific interpretation model validation. They call for formal criteria of correctness that go beyond reproducing model behavior. It is concerning that many of the most commonly used XAI methods and evaluation metries share the same weakness: both can appear consistent with the model while failing to reflect the underlying ground truth.
relying solely on
We also include the mean squared error (MSE) in ablation studies for completeness. MSE does not capture spatial structure and penalizes false positives and negatives equally, yet produces the same ranking of methods our findings. Overall, metrics that do not use ground-truth relevance, such as faithfulness- scores, reward faithful but misleading explanations. In suppressor-aware settings, this risk is particularly severe. Ground-truth-bass metrics are therefore more appropriate for evaluating whether an explanation truly identifies the relevant features.
primary metrics. This consistency supports the robustness of
27

---

G
Ablation Studies
G.1
Mixing white and correlated noise
Instead of varying the signal-to-noise ratio (SNR), we can fix the SNR and introduce new parameter control the balance between white and correlated noise. Specifically, the purely white-noise scenario (WHITE), while 3 = 1 corresponds to the purely Correlated-nois scenario (CORR). Under this setup, the data-genee data-generation process can be written as
0 corresponds to
(")) a(H!")) (H o (m)) + (1 (3(G m + (1 (1- 3mm
(15)
where a controls the SNR, 3 governs the relative contributions of white versus correlated noise, G is the same Gaussian spatial filter used in the CORR scenario, and (x) (n) represent the background noise sources.
772
Figure shows the XOR scenario under this new scheme, demonstrating that as 3 increases (i.e., as correlated noise becomes more dominant), the performance gap between LIME and PatternLocal grows accordingly.
Earth Mover's 's Distance (EMD)
Importance Mass Erro (IME)
Mean Squared (MSE)
1000000000
0.66
0.9
0.1
0.08
0.04
0.0
0.4
0.6
1.0
0.15
0.2
0.4
3.0
-0.01
0.2
08
Fraction of Correlated Noise (B)
Fraction of Correlated Noise (B)
Fraction of Correlated Noise (B)
Pattern ocal
Sobel Laplace
IneegatedGradients
Decplifft
GuidedBackprop
LIME
Saliency
Gradienthapp
Figure 13: Comparison of methods in the new XOR scenario at fixed SNR of = 0.2, with / varying from 0 (WHITE) to 1 (CORR). Results are shown for the three metries: Earth Mover's Distance (EMD),
Importance Mass Error (IME), and Mean Squared Error (MSE). As / increases, the performance of all methods decreases, except for PotternLocal, which remains stable.
G.2 Model architecture and hyperparameter objective
Figures 14 17 summarise the ablation study carried out in this appendix. For every explanation method under investigation we systematically vary two experimental factors:
Model architecture. We compare a multi-layer perceptron (MLP) with a convolutional neural network (CNN).
Hyper-parameter objective. Method hyperparameters tuned either for minimum Earth Mover's Distance (EMD) or for minimum mean squared emror (MSE) on the validation set.
This yields the four result groups
MI.P EMD
Figure 14)
MLP MSE
Figure 15
CNN + EMD
Figure 16
CNN + MSE
Figure 17
Each figure contains three panels that report, a function of the signal-to-noise ratio (SNR), the Earth Mover's Distance (top), the Importance Mass Error (middle), and, included here for completeness, the Mean Squared Error (bottom). Shaded bands correspond to one standard deviation across the test images.
28

---

LINEAR
XOR
RIGID
WHITE
0.15
0.09
0.15
0.04
0.07
0.15
0.37
0.34
CORR
0.24
0.10
0.1
022
0.3
0.4
0.5
0.6
0.7
08
0.9
0.04
0.1
02
0.3
0.4
0.5
0.5
0.7
03
0.9
0.05
0.1
0.2
0.3
0.4
0.5
0.5
0.7
0.3
Parterl ocal
Sobel
IntegratedGradients
Deeplift
GuidedBackprap
LIME
I Laplace
Saliency
GmddeetShap
(a) Earth mover's distance (EMD)
LINEAR
XOR
RIGID
0.98
0.95
WH1TC
0.41
0.67
0.77
0.33
0.41
0.90
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
022
0.3
0.4
0.5
0.
08
0.5
0.6
0.7
0.8
0.9
0.91
0.97
CDRR
0.35
0.40
0.59
0.26
0.35
0.40
(SNR)
(SNR)
(SNR)
PocaersLocal
-
Sobel Laplace
IntegratedGrabients
Deeplift GrudienSShap
GuidedBackprup
LIME
Saliency
(b) Importance mass error (IME)
LINEAR
XOR
RIGID
0.11
0.24
0.30
WH1TE
0.16
0.11
0.02
0.1
0.2
0.5
0.4
0.5
86
07
0.5
0.9
0.00
0.2
0.3
0.4
0.5
06
0.7
03
09
0.00
0.5
0.0
0.7
0.5
0.9
CORR
0.09
0.14
0.22
0.06
0.07
0.11
0.1
02
(SNR)
(SNR)
(SNR)
Parternl ocal
Sohel
IntegratedKindients
Deepl
GuidedRackprop
LIME
Laplace
Saliency
GradientShap
(c) Mean squared error (MSF)
Figure 14: Quantitative evaluation of -eature-importance maps for the MLP model on 8 XAI- TRIS benchmark images generated by several methods as function of the SNR. The classifier attains least 90% accuracy and all method hyperparameters were tuned for EMD. We take hx. (-) id and I((-) Q(-) = 11-112- Panels (a)-() report EMD, IME, and MSE, respectively; shaded bands denote one standard deviation.
29

---

LINEAR
XOR
RIGID
0.10
0.18
0.31
WHITE
0.23
0.11
01
0.2
0.55
0.4
0.5
0.6
0.7
0.5
0.9
0.04
0.2
0.5
0.6
0.5
06
0.7
0.8
01
0.07
0.5
0.6
0.55
0.9
CORR
0.14
0.12
0.30
0.12
0.08
0.15
0.05
(SNR)
(SNIR)
(SNRR)
Pastern
Sobel Laplace
IntegratedKiddeets
Deeplift
GuidedBackprop
LIME
Suliency
GradienSSapp
(a) Earth mover's 's distance (EMD)
LINEAR
XOR
RIGID
0.49
0.94
0.96
0.35
0.41
0.59
0.26
0.5
0.6
0.7
0.8
0.14
03
0.41
0.45
0.91
0.97
CORR
0.41
0.65
0.78
0.1
0.2
0.5
0.4
0.5
0.6
0.7
08
0.
0.1
02
0.5
0.4
0.5
0.6
0.7
08
0.9
0.1
0.2
01.
01.
0..
0.5
0.7
0.5
09
(SNI)
Patterol Lccal
Sobel Laplace
InfegratedKrradiens Saliency
Deeplift
GuidedBackprop
GrudiensShap
(b) Importance mass error (IME)
LINEAR
XOR
RIGID
WHTTE
0.08
0.30
0.13
0.05
0.00
0.07
0.00
0.13
0.21
0.33
CORR
0.20
0.2
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.00
0.1
02
0.3
0.4
0.5
0.5
0.7
0.5
0.9
0.00
0.1
0.2
0.3
0.4
0.5
0.5
0.7
Partern ocal
Sohel
IntegratedGmmdients
Deepliftt
GuidedBackprap
IJME
Laplace
Saliency
Gmddeenthhap
Mean squared (MSE)
Figure 15: Quantitative evaluation of TRIS benchmark images, with method hyperparameters tuned for MSE. All remaining settings
maps for the MLP model on 8 x 8 XAI-
match Fig. [4
30

---

LINEAR
XOR
RIGID
0.10
0.18
0.31
WHITE
0.23
0.11
01
0.2
0.55
0.4
0.5
0.6
0.7
0.5
0.9
0.04
0.2
0.5
0.6
0.5
06
0.7
0.8
01
0.07
0.5
0.6
0.55
0.9
CORR
0.14
0.12
0.30
0.12
0.08
0.15
0.05
(SNR)
(SNIR)
(SNRR)
Pastern
Sobel Laplace
IntegratedKiddeets
Deeplift
GuidedBackprop
LIME
Suliency
GradienSSapp
(a) Earth mover's 's distance (EMD)
LINEAR
XOR
RIGID
0.49
0.94
0.96
0.35
0.41
0.59
0.26
0.5
0.6
0.7
0.8
0.14
03
0.41
0.45
0.91
0.97
CORR
0.41
0.65
0.78
0.1
0.2
0.5
0.4
0.5
0.6
0.7
08
0.
0.1
02
0.5
0.4
0.5
0.6
0.7
08
0.9
0.1
0.2
01.
01.
0..
0.5
0.7
0.5
09
(SNI)
Patterol Lccl
Sobel Laplace
InfegratedKrradiens Saliency
Deeplift
GuidedBackprop
GrudiensShap
(b) Importance mass error (IME)
LINEAR
XOR
RIGID
WHTTE
0.08
0.30
0.13
0.05
0.00
0.07
0.00
0.13
0.21
0.33
CORR
0.2
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.00
0.1
02
0.3
0.4
0.5
0.5
0.7
0.5
0.9
0.00
0.1
0.2
0.3
0.4
0.5
0.5
0.7
Partern ocal
Sohel
IntegratedGrmdients
Deeplift
GuidedBackprap
IJME
Laplace
Saliency
Gmddeenthhpp
Mean squared (MSE)
Figure 16: Quantitative evaluation of feature-importance maps for the CNN model on 8 x 8 XAI- TRIS benchmark images with hyperparameters tuned for EMD. The network achieves at least 90% accuracy; other settings as in Fig. 14
31

---

LINEAR
XOR
RIGID
0.16
0.18
0.31
WEITE
0.14
0.34
0.23
0.00
0.15
0.07
0.1
0.2
0.3
04
0.
0.6
0.7
0.5
0.9
0.2
0.3
0.4
0.5
06
0.7
08
0.5
0.5
0.6
0.7
0.8
0.9
0.34
CORR
0.12
0.08
0.15
0.10
0.04
0.05
(SNR)
(SNR)
(SNR)
ooca
*
Sobel
IntegratedGradients
Deepliift
GuidedBackprop
LIME
I Laplace
Saliency
GrudientShap
(a) Earth mover's 's distance (EMD)
LINEAR
XOR
RIGID
0.49
0.94
0.90
WHITE
0.67
0.77
0.26
0.14
0.41
0.1
02
0.3
0.4
0.5
0.6
0.7
0.8
0.9
022
0.5
0.4
0.5
06
0.7
03
0.5
0.5
0.6
0.5
0.9
0.91
0.97
CORR
0.41
0.65
0.73
0.33
0.40
0.90
0.3
04
0.5
0.6
0.7
0.8
0.9
0.1
02
0.5
(SNR)
(SNR)
(SNK)
Patternl
Sobel Laplace
IntegratedKindiens
Deeplift
GuidedBackprop
LIME
Saliency
GradicntShhpp
(b) Importance mass error (IME)
LINEAR
XOR
RIGID
0.24
0.20
WHITE
0.05
0.00
0.07
0.02
0.00
0.00
0.5
0.13
0.21
0.38
CORR
0.09
0.14
0.20
0.11
0.1
0.2
0.5
0.4
0.5
0.6
077
08
09
0.1
0.2
0.5
0.4
0.5
0.6
07
08
09
0.1
0.2
033
01.
035
0.5
0.7
0.8
09
(SNR)
(SNR)
(SNR)
IntegnatedGradients
Deeplift
GuidedBackprap
Laplace
Saliency
GradientShap
(c) Mean squared ervor (MSF)
Figure 17: Quantitative evaluation of feature-importance maps for the CNN model with hyperparam- eters tuned for MSE. Experimental conditions mirror those of Fig. 14
32

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly present the core contributions of the paper, including the development of PatternLocal, its theoretical foundation, and its empirical advantages. The key limitation, reliance on training data, is also mentioned early in the introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper includes a dedicated "Limitations" section that discusses key as- sumptions and constraints of the method, particularly its reliance access representative training data in the local neighborhood and challenges in unstructured domains.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
33

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Theoretical results are presented in [Section 3.2 through detailed toy example, with all assumptions explicitly stated. Complete derivations and proofs are included either in the main text or the supplementary material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed description of the methodology and experimental setup in the main text, with additional specifics the appendices. In the camera-reade neaer version, the GitHub repository includes all necessary datasets, code, scripts, and results to fully reproduce the experimental findings.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
34

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The code, data, checkpoints, and results are provided via GitHub link in the camera-ready version. The code is self-contained, executable, and provided with guidelines on how run it.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//iips.cc/pple wttpccpppppppppeeeeeeeeeeeeeeeee
heallic/ggiiesse guwdes/
sonooooooc for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
The paper experiment section specifies training/validationntes details includ- ing data splits, hyperparameters, and optimization procedures with further details in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
35

---

The paper includes standard deviation to reflect distribution variance in Sec tion 5] As stated in Figure 4fs caption, error bars are omitted because they are too small to be visualized due to the large number of samples used, implying high statistical confidence in the results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative eтror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The supplementary material provides details the compute resources and estimated execution times. Due to the high computational demands, the raw experimental results are also shared via GitHub for transparency and reproducibility.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics hattps://neurd/ httpeeerrppppppppeeeeeeeeeeeeeee
Answer: [Yes]
The paper uses only synthetic and publicly available benchmark datasets, including de-identified human data, and does not involve any new human-subject experi- ments, private data, or deprecated resources. The research aims to reduce misleading feature attributions and to enhance transparency and accountability in AI model interpretation.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
36

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: PatternLoca seeks more trustworthy explanations by mitigating suppressor variables, which can benefit safety-critical auditing; nonetheless, like any XAI tool it can mislead, so over-reliance on saliency maps is risky. This briefly mentioned in limitations.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release any models datasets with high risk for misuse; it relies solely on synthetic and publicly available benchmark datasets designed for controlled evaluation.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
37

---

12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external datasets and methods used, such as XAI-TRIS and artificial lesion MRL, are properly cited in the paper with references their original publications. These are publicly available benchmarks and tools with known licenses, and no terms of use have been violated.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, papprsaiptettteee thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce release any datasets, models, code assets; proposes novel method (PatterLocal) evaluated on existing benchmarks like XAI-TRIS and the artificial lesion MRI dataset.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any crowdsourcing or research with human subjects; all experiments are conducted using synthetic and benchmark datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
38

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve human subjects or participant studies; all analyses are based synthetic and publicly available benchmark datasets.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The paper does not large language models (LLMs) part of the methodology. Any usage was limited to writing, editing, or formatting support.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
39


Reviews:
[{"review_id": "aPOxmH5HsU", "reviewer": "Reviewer_zrTT", "summary": "The authors investigate the problem of XAI methods mis-attributing *suppressor variables*; that is, variables that affect model prediction but do not have statistical dependency with the target. An existing method, *Pattern*, can remove the effect of suppressor variables for feature attribution, however this method is global and therefore may not work well when applied to complex, nonlinear models. To solve this problem, the authors propose, *PatternLocal*, which extends  Pattern to be able to work on local neighborhoods in conjunction with locally discriminative surrogate models such as KernelSHAP or LIME. The proposed method is evaluated on the XAI-TRIS synthetic dataset and an MRI benchmark dataset.", "questions": "* The prediction model is effectively a surrogate of the data generating process. The attribution method is then a surrogate of the prediction model (which is also a surrogate). PatternLocal adds another layer on top of that. Have you evaluated how accurate PatternLocal is with respect to the model prediction or ground truth target? For example, many works on attribution methods use faithfulness or fidelity metrics to evaluate their performance [1].\n\n[1] https://arxiv.org/pdf/2211.05667", "limitations": "Limitations are addressed on Page 9.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n* The proposed method is a natural extension of Pattern, and is fairly intuitive.\n* The experimental results on synthetic datasets show that PatternLocal outperforms competing methods in suppressing attributions for suppressor variables.\n\n**Weaknesses:**\n* Writing Quality and Notational Issues. I found the manuscript unnecessarily difficult to understand. In particular, the notation and writing could be more rigorous (see **Minor Issues** and **Questions** below). It would be helpful if there was a written algorithm that summarized PatternLocal. Figure 1 is very confusing -- it is not clear what the different colors, dotted lines, marker shapes, heatmaps, etc. represent, since they are not defined.\n* Motivation. The purpose of most attribution methods (e.g. LIME, KernelSHAP) is to identify the most relevant features for the model prediction. If the model uses a suppressor variable in its prediction, then assigning that variable a nonzero attribution seems like expected behavior. It is not clear to me why a user of LIME or KernelSHAP would want to suppress attributions suppressor variables rather than training a better model. It would be helpful if the authors could add more discussion in the manuscript regarding motivation and use-cases.\n* Missing related works section. It would be helpful to have more discussion around related works.\n* Limited theoretical contribution. The paper could be improved if there was some theoretical results, e.g. sample complexity (especially since one of the stated limitations is having enough training data in the neighborhood of each explanation).\n\n**Miscellaneous Minor Issues / Questions**\n* Line 64: \"Throughout we therefore restrict…\" \n* Line 125 says \"regressing the local surrogate output… onto the neighborhood of x\", but line 128 says \"by regressing the simplified input x on the surrogate model's response…\". These two lines seem to contradict each other.\n* The variable $s$ seems to be overloaded -- it's sometimes used to represent a latent vector (equation 1), and sometimes it's a function (equation 6) which is also equal to a? In Eq. 10, $s$ is also an importance map.\n* In equation 5,  $\\mathcal{Z}$ is treated as a distribution over which you take an expectation, but it's defined as a \"simplified input space\". There is a similar issue in equation 6 for $\\mathcal{X}$.\n* Equation 6 has the term: $(h_{x}(x) - u \\tilde y)^2$. Is u a D'-dimensional vector or a scalar? Should there be a $l_1$ or $l_2$ norm here?\n* $\\mathcal{F}^+$ in equation 10 is not defined. Is this supposed to be the same as $F^+$?\n* In Section 4.4, what does $|s|$ represent? Equation 10 seems to imply that it's an absolute value or normalization, but Equation 11 implies that $|s|$ is an integer.", "was_revised": true, "final_justification": "My initial score was 3. The authors' responses addressed most of my concerns around motivation and proposed changes to fix notation and add sample complexity results, therefore I increased my score to 4. It is difficult to recommend a score of 5, since one of the original concerns was writing quality and we cannot see the revised manuscript.", "rebuttal": "# Reviewer zrTT\n\nThank you for the detailed and constructive review. We greatly appreciate that you found the problem of suppressor variables important, the extension of the Pattern principle to a local setting novel and intuitive, and the empirical gains on suppressor benchmarks compelling. **Note, some subscripts were omitted to avoid OpenReview formatting issues.**\n\n### Questions and Comments\n\n---\n\n> \"Writing Quality and Notational Issues. I found the manuscript unnecessarily difficult to understand … It would be helpful if there was a written algorithm that summarized PatternLocal.\"\n> \n\nWe’ve updated Section 3: PatternLocal and added a high-level algorithmic description at the start (see response to Reviewer 7PSJ). This gives the reader a quick overview before the discussion of advantages like suppressor-variable mitigation, model faithfulness, local fidelity, and feasibility. We’ve also included a notation table in the appendix (see response to Reviewer 7PSJ), referenced in Section 2.1. These changes help clarify the manuscript and address the earlier notation issues.\n\n---\n\n> \"Figure 1 is very confusing ... \"\n> \n\nWe appreciate this feedback. In the revision the caption for Figure 1 now explicitly describes each element: colors denote class labels in the 2D projection, solid versus dotted lines indicate the model's decision boundary and the local linear surrogate, markers indicate sampled neighbors and the test point $\\mathbf{x}_\\star$, and the heatmaps show the resulting importance maps from the discriminative surrogate and the generative PatternLocal transformation. To avoid clutter we reduced the number of graphical elements and added a legend.\n\n---\n\n> \"Motivation. The purpose of most attribution methods (e.g. LIME, KernelSHAP) is to identify the most relevant features for the model prediction. If the model uses a suppressor variable in its prediction, then assigning that variable a nonzero attribution seems like expected behavior. It is not clear to me why a user of LIME or KernelSHAP would want to suppress attributions suppressor variables rather than training a better model. It would be helpful if the authors could add more discussion in the manuscript regarding motivation and use-cases.\"\n> \n\nWe agree that standard XAI methods should show which features a model uses, but our goal is to spotlights those that are also statistically linked to the outcome. The difference lies between features the model exploits and features that truly matter for the target. Suppressor variables may raise accuracy by canceling noise yet hold no real link to the target; explanations should hide or flag them so users are not misled.\n\nTraining a stronger model may not remove suppressors, especially in complex domains where the optimal model may always need to latch onto context information to remove variance from target-related variables. PatternLocal tackles this by turning the surrogate’s local model into a generative explanation aligned with the data distribution, thus down-weighting features unconnected to the target.\n\nWe now underscore this point in the Introduction and Section 2.2 and outline domains, such as science and medicine, where suppressor insensitive explanations gives more actionable insights, whereas highlighting a suppressor could misleed. See the following added remark:\n\nRemark: It can be desirable for an optimal model to use suppressor variables in its prediction, however for the later explanations, we wish to filter-out these suppressor variables as they have no direct influence on the target variable in the data generating process nor any other statistical association (be it causal, anti-causal or through confounding). It is also not to be confused with the Clever Hans Effect or Shortcut learning where a model uses undesirable features.\n\n---\n\n> \"Missing related works section. It would be helpful to have more discussion around related works.\"\n> \n\nThe related work was previously part of the Introduction, but we've now added a dedicated section. In addition to the original Pattern method, PatternNet, PatternAttribution, and local explainers like LIME and KernelSHAP, we now cover concept-based methods (e.g. TCAV), generative approaches like autoencoder-based explanations, and other suppressor-mitigation techniques. We also relate our work to causal attribution, connecting it to faithfulness metrics, infidelity, insertion/deletion scores (Chen et al., 2024).\n\nNew source added: What makes a Good?: A harmonized view of properties of explanations by Chen et al., 2024.\n\n---\n\n> \"Limited theoretical contribution. The paper could be improved if there was some theoretical results, e.g. sample complexity ...\"\n> \n\nWhile PatternLocal builds on the insight of transforming discriminative weights into generative patterns, extending this idea to local surrogates requires a non-trivial adaptation. The paper derives the kernel-weighted regression objective (Eq. 6) and its closed-form solution for $\\ell_2$ regularization (Eq. 7). We further have added a discussion on sample complexity which can be seen under the response to Reviewer zNrP. These theoretical additions clarify how PatternLocal generalizes Pattern to non-linear settings and highlight its novelty.\n\n---\n\n> \"Line 125 says 'regressing the local surrogate output… onto the neighborhood of x', but line 128 says 'by regressing the simplified input x on the surrogate model's response' ...\"\n> \n\nWe appreciate the reviewers taking time to highlight this issue. Line 125 has been changed and should of course say *\"PatternLocal extends this idea to the local setting by regressing the neighborhood of $x_\\star$ onto the local surrogate output $\\hat{y} = \\mathbf{w}^\\top \\mathbf{x}$, thus removing suppressor effects specific to the instance.\"*\n\n---\n\n> \"In equation 5, $\\mathcal{Z}$ is treated as a distribution over which you take an expectation, but it's defined as a \"simplified input space\". There is a similar issue in equation 6 for $\\mathcal{X}$… Equation 6 has the term: $(h_{\\mathbf{x}_\\star}(\\mathbf{x}) - \\mathbf{u} \\hat{y})$ . Is u a $D'$-dimensional vector or a scalar? Should there be a or norm here?\"\n> \n\nThank you for pointing out the notational ambiguity. In the original text, $\\mathcal{Z}$ and $\\mathcal{X}$ were used both as sets and random variables. To clarify, we now introduce explicit probability measures: $P_{\\mathcal Z}$ is the sampling distribution over the simplified input space $\\mathcal Z$ (uniform Bernoulli in basic LIME), and $P_{\\mathcal X}$ is the data distribution on $\\mathcal X \\subseteq \\mathbb{R}^{D}$, approximated by the training set.\n\nYou're also right that it should be a norm. This was fixed right after submission and already appears correctly in Appendix A. \n\nWe'll also add a clarifying note in Section 2.1. These updates match the notation in GLIME (Tan et al., 2023) and address the issue raised.\n\n---\n\n> The variable $\\mathbf{s}$ seems to be overloaded -- it's sometimes used to represent a latent vector (equation 1), and sometimes it's a function (equation 6) which is also equal to a? In Eq. 10, is also an importance map.\n> \n\nWe understand some of the confusion. In Eq. 1, $\\mathbf{s}$ is a latent vector, which is to be consistent with the notation of (Haufe et. al. 2014). To avoid confusion, we have changed the symbol of the latent vector in the entire Section 2.2 (Suppressor variables), Section 2.3 (Forward vs. backward models) and Section 3.2 (Toy example) to use $\\mathbf{m}$ instead. \nIn the rest of the manuscript, $\\mathbf{s}$, is consistently used to refer to importance map / saliency maps for different XAI methods. We use superscript, to indicate which one we are referring to e.g. $\\mathbf{s}^\\text{Pattern}$, $\\mathbf{s}^\\text{LIME}$, $\\mathbf{s}^\\text{PatternLocal}$. We have changed the notation to $\\mathbf{s}(\\mathbf{x}\\star)$ to remind of the dependency on $\\mathbf{x}_\\star$.\n\n---\n\n> $\\mathcal{F}^+$ in equation 10 is not defined. Is this supposed to be the same as ? … In Section 4.4, what does $|\\mathbf{s}|$ represent? Equation 10 seems to imply that it's an absolute value or normalization, but Equation 11 implies that is an integer.\n> \n\nWe apologize for the oversight. We have fixed the issue. See response to Reviewer Xj2D for details.\n\n> The prediction model is effectively a surrogate of the data generating process. The attribution method is then a surrogate of the prediction model (which is also a surrogate). PatternLocal adds another layer on top of that. Have you evaluated how accurate PatternLocal is with respect to the model prediction or ground truth target? For example, many works on attribution methods use faithfulness or fidelity metrics to evaluate their performance.\n> \n\nWe partially agree with the first statement. We would rather say that PatternLocal updates the same surrogate model to de-emphasize suppressor variables. We believe this is a necessary approach and as mentioned in the response to Reviewer zNrP, this does not influence running time much.\n\nIn regards to the metric, we have added metrics on local fidelity of the surrogate model as the weighted error as also requested by Reviewer zNrP. We do not believe that faithfulness is a suitable metric for ground truth evaluation and as it can lead to misleading results. See answer to Reviewer Xj2D regarding a discussion on faithfulness metrics.\n\n## Final note\n\nWe thank Reviewer zrTT once more for their constructive feedback. As detailed above, we have answered every question, clarified all ambiguities and updated the manuscript accordingly. We therefore kindly ask you to reconsider your scores in light of these improvements. Should any further points arise, we are happy to provide additional clarifications at any time."}, {"review_id": "8sAAewAfDF", "reviewer": "Reviewer_zNrp", "summary": "The paper studies the phenomenon of  “suppressor variables”, which are features that influence model predictions but are not statistically dependent on the output variable, potentially leading to false-positive feature attributions.\n\nThe authors propose PatternLocal, a novel  and model agnostic XAI technique that generalizes the Pattern approach into the local, non-linear setting. PatternLocal operates in three steps: it (1) builds a locally linear surrogate model around an instance of interest (via LIME, SHAP, or gradients), (2) transforms the surrogate’s discriminative weights into a generative explanation using local data statistics, and (3) outputs instance-level attributions that suppress the effect of suppressor variables while preserving local model fidelity.\n\nThe authors conduct empirical validation on the XAI-TRIS benchmark and an artificial lesion MRI dataset, which are specialized benchmarks that offer suppressor-variable scenarios. After hyperparemter tuning, PatternLocal tends to outperforms established XAI methods on the Earth Mover’s Distance (EMD) and Importance Mass Error (IME) metric.", "questions": "Why focus on Earth Mover’s Distance (EMD) and Importance Mass Error (IME) and not simply using more standard metrics like MSE or PSNR for  visual explanation comparison? How sensitive are your conclusions to metric selection?\n\nThe paper uses exhaustive hyperparameter tuning for all methods. Is this necessary for PatternLocal to perform well, or do optimal hyperparameters at least generalize in some way? How does PatternLocal perform in more typical, default or lightly-tuned scenarios compared to common default settings of existing explanations? \n\nWhat kind of explanations does PatternLocal produce when be applied to datasets like ImageNet or tabular data, where finding sufficient, well-aligned local neighbors is difficult? \n\n Could access to generative models or learned latent spaces provide a substitute for local neighborhood samples where data is limited or not aligned? Would such extensions preserve PatternLocal’s suppressor-mitigation properties?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n-\tStudying the behavior of explanation methods in the presence of “suppressor variables” is relevant. \n\n-\tThe paper proposes a theoretical sound way to extend Pattern approach to non-linear models by building on top of existing (model agnostic) explanation approaches.\n\n-\tThe paper is well written and easy to follow.\n\nWeaknesses:\n\n- The theoretical contributions are limited as the method is a direct extension of the existing Pattern method. \n\n-\tThe method appears to be computationally way more expensive compared to the baselines and the experimental results only partly justify the effort despite extensive hyperparameter search. \n\n- All experiments are primarily on synthetic, or semi-synthetic image data. The generalization to complex real-world datasets remains unclear.", "was_revised": true, "final_justification": "My initial score was 3 and I have increased it to 4 after reading the authors response. However, I still have some doubt regarding the overall utiliy of the method as requires standardized views and the existance of sufficiently similar samples in the local neighborhood.  Providing more evidence on how the method can be applied more broadly would substantially strengthen the paper.", "rebuttal": "# Reviewer zNrP\n\nThank you for recognizing the relevance of suppressor-variable mitigation, the soundness of our derivation, and the careful experimental design. We address each of your concerns below and describe the changes made to the paper.  **Note, some subscripts were omitted to avoid OpenReview formatting issues.**\n\n## Questions and Comments\n\n---\n\n> \"The theoretical contributions are limited as the method is a direct extension of the existing Pattern method.\"\n> \n\nWhile PatternLocal builds on the insight of transforming discriminative weights into generative patterns, extending this idea to local surrogates requires a non-trivial adaptation. The revised paper now explicitly derives the kernel-weighted regression objective (Eq. 6) and its closed-form solution for $\\ell_2$ regularization (Eq. 7). We further provide a discussion on sample complexity (as requested by Reviewer zrTT) and the conditions under which the local regression yields unbiased estimates. These theoretical additions clarify how PatternLocal generalizes Pattern to non-linear settings and highlight its novelty.\n\nIt is important to note that an explicit matrix covariance estimate (and its multiplication) is not always required, especially when the surrogate already contains an inverse-covariance form. The core step in PatternLocal is the kernel-weighted moment\n\n$a_{\\ell2} = \\frac{\\text{Cov}\\bigl[h(\\mathbf{x}), \\hat{y}\\bigr]}{\\text{Var}[\\hat{y}] + \\lambda}$ where $\\hat{y} = \\mathbf{w}^\\top h(\\mathbf{x})$\n\nwhich for binary classification only needs (i) the scalar local variance of $\\hat y$ and (ii) the vector cross-covariance with the simplified features $h_{\\mathbf{x}_\\star}(\\mathbf{x})$. We estimate those moments with a Nadaraya–Watson (NW) kernel smoother. For any function $g$\n\n$\\hat{\\mathbb{E}}[g] = \\frac{\\sum_{i=1}^{n^{\\text{loc}}} K(\\|h(\\mathbf{x}(i))-\\mathbf{0}\\|) g(\\mathbf{x}(i))}{\\sum_{i=1}^{n^{\\text{loc}}} K (\\|h(\\mathbf{x}(i))-\\mathbf{0}\\|)}$,\n\nwhere $K_\\sigma$ is an isotropic kernel of bandwidth $\\sigma$ in the $D'$-dimensional interpretable space and $n_{\\text{loc}}$ the number of samples in the local neighborhood.  Nadaraya–Watson analysis (Yin et al, 2010) gives that the MSE is minimized at $\\sigma^{\\star}\\propto n_{\\text{loc}}^{-1/(4+D')}$, yielding $\\mathrm{MSE}=O(n_{\\text{loc}}^{-4/(4+D')})$. Equivalently, to achieve an $\\varepsilon$-accurate estimate one needs $n_{\\text{loc}}=\\Omega(\\varepsilon^{-(4+D')/2})$, which scales only with the *interpretable* dimension $D' \\ll D$. Additionally when $D'$ is not small relative to $n_{\\text{loc}}$, we apply Ledoit–Wolf shrinkage to the local covariance. (Ledoit & Wolf, 2004) prove this remains consistent even if $D'/n_{\\text{loc}}>0$, and crucially PatternLocal never needs an explicit matrix inverse, because Eq. 7 divides by the *scalar* variance of $\\hat y$. Finally, recent locality-aware sampling theory shows that GLIME (Tan et al, 2023) needs only $n_{\\text{loc}} = O(D'\\log D')$ samples for a stable linear fit. PatternLocal can inherit the same sampling strategy, further reducing the data required per explanation without altering its closed-form update.\n\n---\n\n> \"All experiments are primarily on synthetic, or semi-synthetic image data. The generalization to complex real-world datasets remains unclear … What kind of explanations does PatternLocal produce when be applied to datasets like ImageNet or tabular data, where finding sufficient, well-aligned local neighbors is difficult?\"\n> \n\nWe acknowledge that our evaluation datasets (XAI-TRIS and the lesion MRI) are controlled or semi-synthetic. The choice was deliberate: these benchmarks provide *ground-truth feature importance* to quantitatively evaluate suppressor mitigation. As mentioned in **Limitations,** for real-life structured domains like medical imaging, where standardized views are common, we expect the PatternLocal method to perform well. In the setting of ImageNet, we expect the PatternLocal to have a harder time as there might be too few samples in the local neighborhood.\n\nWe have added example with EEG signal on Event-Related Potential (ERP) tasks. More specifically EEG Motor Movement Dataset, where participants either move their left or right hand at some stimulus. Here we know that no pre-stimulus signal is correlated with the target variable, however we observe common XAI methods highlight pre-stimulus activity while PatternLocal does not. This is a real-life dataset where PatternLocal outperforms other methods. \n\n---\n\n> \"Why focus on Earth Mover's Distance (EMD) and Importance Mass Error (IME) and not simply using more standard metrics like MSE or PSNR for visual explanation comparison? How sensitive are your conclusions to metric selection?\"\n> \n\nWe chose EMD and IME because they both de-emphasize the impact of false-negative omissions of features in the ground truth on performance, while emphasizing the impact of false-positive attributions of importance to pixels not contained in the ground truth (Clark et al, 2024). We have added a motivation for these metrics in Section 4.4. In addition, we already do include the MSE metric in Appendix E: Ablation Studies, which still demonstrates the advantages of using PatternLocal. Thus we conclude that it is not sensitive to metric choice. As requested by mentioned by Reviewer Xj2D, we also include discussion on insertion/deletion. We have also added metrics on local fidelity of the surrogate model as the weighted error as requested by Reviewer zNrP. See answer to Reviewer Xj2D for a discussion of faithfulness metrics, which we believe is not a good metric for ground truth evaluation.\n\n---\n\n> \"The method appears to be computationally way more expensive compared to the baselines and the experimental results only partly justify the effort despite extensive hyperparameter search … The paper uses exhaustive hyperparameter tuning for all methods. Is this necessary for PatternLocal to perform well, or do optimal hyperparameters at least generalize in some way? How does PatternLocal perform in more typical, default or lightly-tuned scenarios compared to common default settings of existing explanations?\"\n> \n\nWe now provide runtime measurements (see table below). PatternLocal adds one extra weighted regression per explanation on top of the cost of the local surrogate. The computationally heavy part is fitting the initial surrogate model with methods like LIME and the extra regression keeps the runtime in the same order of magnitude. As PatternLocal only regresses in the local neighborhood, it is in practice also drastically improved by only considering samples with a weight above a certain threshold. Besides this, to mitigate this issue, like LIME, we already use superpixels to reduce the computational cost and, and have also included the LowRank approximation as seen in Appendix D: Extended results. Given that the runtime is only slightly higher than comparable model-agnostic methods like LIME, KernelSHAP, etc. but consistently out performs them, we believe that it is more than justified. Even though it is difficult to compare runtimes due to different hardware, implementations and hyperparameters. Here we have kept everything the same and averaged the runtime over 1000 instances of the XAI-TRIS dataset on an AMD EPYC 9124 16c/32t 3.0 GHz (also for model inference by LIME).\n\n| XAI method | Average runtime ± std (s) |\n| --- | --- |\n| PatternLocal | 12.9 ± 1.5 |\n| LIME | 7.4 ± 0.3 |\n| PatternLocal + Superpixel | 3.6 ± 0.3 |\n| LIME + Superpixel | 1.4 ± 0.1 |\n| PatternLocal + LowRank | 7.7 ± 0.4 |\n| LIME + LowRank | 6.9 ± 0.2 |\n\nWe performed an exhaustive search for every method solely to avoid putting the baselines at a disadvantage; PatternLocal inherits the sensitivity to hyperparameters that LIME is known to have. If you look at the hyperparameter space Appendix C.2, PatternLocal only requires bandwidth, kernel and regularization parameter. For the XAI-TRIS dataset the kernel choice was nearly always the Gaussian kernel and regularization parameter was near to none, thus it was really only the bandwidth that made the difference. Choosing the bandwidth as the median pair-wise distance in the neighborhood also yields favorable results for PatternLocal. We therefore conclude that PatternLocal is not overly sensitive to hyperparameters; nevertheless, we include guidance in the appendix on choosing them.\n\n---\n\n> \"Could access to generative models or learned latent spaces provide a substitute for local neighborhood samples where data is limited or not aligned? Would such extensions preserve PatternLocal's suppressor-mitigation properties?\"\n> \n\nThis is an insightful suggestion, and we have added a paragraph discussing it. PatternLocal relies on a representative local neighborhood to estimate the forward pattern. When such neighborhoods are scarce, one could sample neighbors from a generative model conditioned on $\\mathbf{x}_\\star$, e.g., a VAE or a diffusion model, to augment the data. We note, however, that generative models may introduce bias if the latent space does not faithfully capture the local data distribution. Suppressor mitigation depends on accurate (explicit or implicit) estimates of the covariance between simplified features and the surrogate output; sampling from an unaligned latent space could reintroduce spurious correlations. We therefore consider integration with generative models as promising future work ,but beyond the scope of this paper.\n\n---\n\n## Final note\n\nWe thank Reviewer zNrP once more for their constructive feedback. As detailed above, we have answered every question, clarified all ambiguities and updated the manuscript accordingly. We therefore kindly ask you to reconsider your scores in light of these improvements. Should any further points arise, we are happy to provide additional clarification at any time."}, {"review_id": "HuokhhQUoC", "reviewer": "Reviewer_Xj2D", "summary": "This study tackles the problem that existing XAI methods tend to assign high importance to suppressor variables—features that are not directly related to the prediction target but cancel out noise and thereby boost the model’s overall accuracy.\nPrevious work has proposed a method to suppress the influence of suppressor variables in linear models.\nThe proposed method in this study extends those ideas so they can be applied to non-linear predictive models.\nConcretely, around the test instance to be explained, the method trains a local linear surrogate model of the non-linear predictor using LIME.\nIt then estimates parameters that map the surrogate model’s outputs back to simplified inputs of that test instance, and treats those estimated parameters as the feature importances.\nExperiments on datasets constructed to contain suppressor variables demonstrate that the proposed approach produces correct explanations, outperforming existing feature attribution methods.", "questions": "If you have rebuttals to the points listed under Weaknesses, please present them.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n- Extending the Pattern proposed by Haufe et al. to work with non-linear models is both novel and practically valuable, and the derivation of the proposed method appears sound.\n- The toy example convincingly shows that—even for a non-linear model—the proposed approach can drive the importance of a suppressor variable to zero, effectively illustrating the method’s validity.\n- On the XAI-TRIS benchmark, the proposed method performs better than many alternative explanation techniques on images corrupted by noise and suppressor variables, demonstrating its usefulness.\n\n# Weaknesses\n- It is unclear why simply adding an L1 regularizer in LIME would not suffice to push suppressor-variable importance toward zero. In the experiments, both LIME and the proposed method seem to use L2 regularization, yet the behavior under L1 regularization is not examined in depth.\n- The evaluation metrics employed are not standard. For image classification, metrics such as Insertion, Deletion, or Infidelity are commonly used to measure explanatory faithfulness, whereas the paper adopts a non-standard metric called EMD; no compelling reason is given for this choice.\n-The validity of that metric is questionable. In (10), defining EMD, |\\mathbf{S}| appears to denote the absolute values of the estimated importance map, but taking absolute values discards whether a feature contributes positively or negatively. In IME (11), the numerator and denominator are summed independently, so it is unclear how false positives and false negatives are captured. Moreover, the meaning of superscripts such as |\\mathcal{F}^+| and |\\mathbf{s}| is ambiguous—do they denote set cardinality, absolute value, or vector norm?", "was_revised": true, "final_justification": "The authors' explanations for the points I raised were convincing, and my concerns have been resolved. Therefore, I raised my score.", "rebuttal": "# Reviewer Xj2D\n\nThank you for recognizing the novelty and practical value of extending the Pattern method to non-linear settings and for noting the clarity of our toy example. We share your interest in L1-regularized baselines and standard faithfulness metrics. We therefore begin by clarifying notation, elaborating on metric choice, and adding new experiments. **Note, some subscripts were omitted to avoid OpenReview formatting issues.**\n\n## Questions and Comments\n\n> \"It is unclear why simply adding an L1 regularizer in LIME would not suffice to push suppressor-variable importance toward zero. In the experiments, both LIME and the proposed method seem to use L2 regularization, yet the behavior under L1 regularization is not examined in depth.\"\n> \n\nAn $\\ell_1$ penalty encourages sparsity but does not distinguish between suppressors and informative features. In the toy XOR example and on XAI-TRIS, LIME with $\\ell_1$ regularization reduces the magnitude of all coefficients but continues to assign non-zero importance to suppressors. PatternLocal, in contrast, transforms the discriminative surrogate into a generative pattern that down-weights features uncorrelated with the surrogate output. In the revised paper, we include in Appendix D: Extended results experiments comparing LIME with $\\ell_1$ regularization to PatternLocal. Across all tasks PatternLocal drives suppressor attributions closer to zero and have lower EMD and IME scores. This indicate that PatternLocal adds value beyond sparsity.\n\n---\n\n> \"The evaluation metrics employed are not standard. For image classification, metrics such as Insertion, Deletion, or Infidelity are commonly used to measure explanatory faithfulness, whereas the paper adopts a non-standard metric called EMD; no compelling reason is given for this choice. The validity of that metric is questionable.\"\n> \n\nWe chose EMD and IME because they are tailored to suppressor-aware benchmarks: EMD measures the spatial cost of moving attribution mass to ground-truth locations, and IME penalizes false positives more than false negatives. These metrics were introduced by (Clark et al. 2024) for the XAI-TRIS benchmark datasets. In Appendix E: Ablation Studies, we have already included the MSE measure, which aligns with the conclusion drawn from the EMD and IME metric. Nonetheless, we agree and justify this choice more in Section 4.4. We have added metrics on local fidelity of the surrogate model as the weighted error as requested by Reviewer zNrP\n\nRegarding faithfulness, metrics like insertion and deletion, on the other hand, are not based on an explicit ground-truth. However, they implicitly incentivize attribution to suppressors, which we criticize. In fact, faithfulness metrics are strongly linked and, under some conditions, equivalent to Shapley values for attribution, as shown by (Wilming et al, 2023, Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables) and (Blücher et al, 2022, PredDiff: Explanations and interactions from conditional expectations).\n\nIn short, we do not believe that faithfulness metrics should be considered the gold standard to evaluate attributions as faithful attributions can lead to incorrect interpretations and mistakes in downstream tasks such as scientific discovery or model validation as also noted in (Haufe et al 2024, Explainable AI needs formal notions of explanation correctness). Therefore, we see it as a problematic development in the field that both the most prominent XAI methods and the most common metrics to evaluate them share the same limitations.\n\n---\n\n> \"In (10), defining EMD, $|s|$ appears to denote the absolute values of the estimated importance map, but taking absolute values discards whether a feature contributes positively or negatively. In IME (11), the numerator and denominator are summed independently, so it is unclear how false positives and false negatives are captured\"\n> \n\nThe absolute‐value notation in Eq. 10 and the independent sums in Eq. 11 are intentional. For the XAI-TRIS benchmark we only care whether a feature is highlighted at all, not whether its contribution is numerically positive or negative. In addition, in many cases (XAI-TRIS LINEAR and XAI-TRIS XOR), the absence of one feature is as informative as the presence of another feature. By not taking the absolute value, we would penalize models for this.So in short, measuring unsigned mass therefore lets Earth Mover’s Distance (EMD) and Importance Mass Error (IME) focus on the presence or absence of attribution while avoiding unnecessary penalties.\n\n\n> \"Moreover, the meaning of superscripts such as $|\\mathcal{F}^+|$ and $|\\mathbf{s}|$ is ambiguous—do they denote set cardinality, absolute value, or vector norm?\"\n>\n\nThank you for highlighting this ambiguity. It was also noted by Reviewer zrTT. We use $|\\mathbf{s}|$ to refer the absolute value in Eq. 10 and within the summation in Eq. 11. We instead use the notation $\\text{card}(\\cdot)$ to refer to the cardinality e.g. $\\text{card}(\\mathcal{F}^+)$ and $\\text{card}(\\mathbf{s})$.\n\nFurthermore, we have defined $\\mathcal{F}^+$ in Appendix B, Eq. 13 and 14. but have now included the definition in Section 4.4. It is defined as the set of ground-truth salient pixels for the model's prediction, following (Clark et al, 2024).\n\n---\n\n## Final note\n\nWe thank Reviewer Xj2D once more for their constructive feedback. As detailed above, we have answered every question, clarified all ambiguities and updated the manuscript accordingly. We therefore kindly ask you to reconsider your scores in light of these improvements. Should any further points arise, we are happy to provide additional clarifications at any time."}, {"review_id": "igO8UcjsMG", "reviewer": "Reviewer_7PSJ", "summary": "The manuscript introduces a new XAI method for vision tasks called PatternLocal. Similar to previous methods such as LIME, the proposed PatternLocal method constructs a local linear surrogate, and then constructs a generative model using the resulting weights. It is claimed that the process mitigates the effects of suppressor variables in generating explanations (false-positive attribution).", "questions": "> Please explain the expected behavior of each plot as a function of the x-axis variable (alpha).\n> Please see the weaknesses section for additional comments on the manuscript and organization, and provide clarifications in your rebuttal if needed.", "limitations": "NA", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n>- The topic is timely and the issue of false-positive attribution (particularly due to suppressor variables) is well known and solutions such as those provided in this work are of interest. \n>- The examples, such as the Toy Example of Section 3.2 and the example in Section 2.2 of the suppressor variable are helpful and well-explained. \n\nWeaknesses: The manuscript is generally not well-written. The notation is not consistent and the explanations are incomplete and vague. More detailed comments are provided below:\n>- On Page 3, Line 67, $h_{x_*}$ is introduced as a mapping that produces a low-dimensional semantically meaningful representation. The definition is vague and needs additional explanation. Furthermore, the output is denoted as $x'_*$, however, in Equation 2, the input to the inverse $h^{-1}$ is shown by z (which seems more suitable). The notational inconsistency is repeated in many other instances and reduces the readability of the manuscript.\n>- It is not clear why on Page 3, Line 94, w is used instead of W, whereas in all previous instances, W was used to represent the transform matrix.\n>- In Page 3, Equation 2, $\\hat{s}$ is recovered with respect to which loss metric? (e.g., is it to minimize the mean squared error?)\n>- The manuscript is not well-organized. For instance, the advantages of PatternLocal are discussed in Page 4 in detail before the method is introduced even at a high level. It seems more appropriate to provide a high level description of the main ideas in PatterLocal first, and then describe its advantages compared to existing methods. \n>- In Figure 4, it is not clear what the x-axis represents. It is stated in the caption that it represents the SNR, however, the value ranges from 0 to 1. It seems from the supplementary material that it represents alpha (as a surrogate for SNR). If so, should we expect that the plots be decreasing in alpha? (as the noise decreases) If so, why are some of the plots increasing in alpha?", "was_revised": true, "final_justification": "As mentioned in my response to the author's rebuttal, the rebuttal has addressed some of my concerns and technical questions. \n\nThe authors have also mentioned that they will address the notational issues in the original manuscript prior to final publication, and their described plan of action is satisfactory to address many of the issues. However, it would have been more helpful if these issues were addressed prior to the original submission as the reviewers will not have the opportunity to review the modified manuscript afterwards.\n\nGiven the above, I will increase my score to Borderline Accept.", "rebuttal": "# Reviewer 7PSJ\n\nThank you for emphasizing the timeliness of addressing false-positive attributions and for appreciating our explanatory examples. As you and others noted, notation inconsistencies, figure clarity, and manuscript organization need improvement. **Note, some subscripts were omitted to avoid OpenReview formatting issues.**\n\n## Questions and Comments\n\n> \"On Page 3, Line 67, $h$ is introduced as a mapping that produces a low-dimensional semantically meaningful representation. The definition is vague and needs additional explanation. Furthermore, the output is denoted as $x_\\star'$ , however, in Equation 2, the input to the inverse $h^{-1}$ is shown by z (which seems more suitable). The notational inconsistency is repeated in many other instances and reduces the readability of the manuscript.\"\n> \n\nWe thank Reviewer 7PSJ for pointing out the notational issues. Below, we clarify the mapping $h$, its role in Eq. 5 and Eq. 6, and the changes we've made to fix the inconsistencies.\n\nThe map $h: \\mathcal X \\to \\mathbb R^{D'}$ (changed to $h: \\mathbb R^{D} \\to \\mathbb R^{D'}$) defines an instance-specific simplification: for a given input $\\mathbf{x}_\\star$, it maps any other input $\\mathbf{x}$ (real) or $\\mathbf{z}$ (synthetic) to a lower-dimensional, interpretable space via $x' = h(\\mathbf{x})$. The form of $h$ depends on the domain (e.g., superpixels or low-rank projection), and we keep the subscript to show it's tied to the reference instance.\n\nTo simplify notation, we now use a prime $'$ to mark variables in the reduced space e.g., $\\mathbf{x}'$, $\\mathbf{z}'$, $D'$. Real data from $\\mathcal D$ are denoted $\\mathbf{x}$, and synthetic samples are $\\mathbf{z}$, giving:\n\n$\\mathbf{x}' = h(\\mathbf{x}), \\quad \\mathbf{z}' = h(\\mathbf{z})$\n\nThis holds for Eq. 5 (LIME/KernelSHAP on synthetics) and Eq. 6 (PatternLocal on real data). We also define the inverse consistently:\n\n$h^{-1}: \\mathbb{R}^{D'} \\to \\mathbb{R}^{D}, \\quad h: \\mathbb{R}^{D} \\to \\mathbb{R}^{D'}$\n\nFor LIME, $h^{-1}(\\mathbf{z}')$ fills zeros with a reference value $r$ and upsamples the mask back to $\\mathbb R^{D}$, exactly as in Ribeiro et al. (2016). In the low-rank case, it's the linear decoder. PatternLocal does not use synethetic samples but uses $h$ to map real samples to simplified sapce before optimization.\n\nWe've added a one-page notation table in the appendix (linked from Section 2.1) to aid readability. We hope these updates address the reviewer’s concerns.\n\n| Symbol | Description |\n| --- | --- |\n| $\\mathcal{X} \\subset \\mathbb{R}^D$ | Input space of dimension $D$. |\n| $\\mathcal{Y} \\subset \\mathbb{R}$ | Output space (scalar). |\n| $\\mathbf{x} \\in \\mathcal{X},\\; y \\in \\mathcal{Y}$ | A data point and its label. |\n| $\\mathbf{x}^{\\star}$ | The specific instance whose prediction $f(\\mathbf{x}^{\\star})$ we wish to explain. |\n| $D,\\; D'$ | $D$: input dimensionality; $D' \\ll D$: simplified input dimensionality. |\n| $\\mathbf{x}' = \\mathbf{h}(\\mathbf{x})$ | Simplified/semantic representation of data point $\\mathbf{x}$. |\n| $\\mathbf{h} : \\mathbb{R}^{D} \\to \\mathbb{R}^{D'}$, $\\mathbf{h}^{-1} : \\mathbb{R}^{D'} \\to \\mathbb{R}^{D}$ | Forward and inverse mapping between original and simplified spaces. |\n| $f$ | The original model being explained, $f : \\mathcal{X} \\to \\mathcal{Y}$. |\n| $g$ | Interpretable surrogate model fit locally around $x^{\\star}$. |\n| $\\mathcal{D} \\subset \\mathcal{X}$ | Training data used by PatternLocal for local statistics. |\n| $\\Pi_{x'_{\\star}}(\\cdot)$ | Locality kernel down-weighting samples distant from $\\mathbf{x}'_{\\star}$. |\n| $R(\\cdot)$, $Q(\\cdot)$ | Regularization in LIME and PatternLocal. |\n| $\\lambda$ | Regularization strength. |\n| $\\alpha$ | Signal-to-noise mixing coefficient for synthetic data. |\n| $\\mathbf{z}'$, $\\mathcal{Z}$ | Simplified artificial samples and their space. |\n| $\\mathbf{s}$, $\\mathbf{s}^{\\text{LIME}}$, $\\mathbf{s}^{\\text{PatternLocal}}$ | Feature-importance scores (saliency maps); either generic or for specific method. |\n| $\\mathbf{w}$ | Coefficient vector from linear surrogate models (Eq. 5). |\n| $\\mathbf{a}$ | PatternLocal's explanation vector via forward-model regression (Eq. 6). |\n| $\\tilde{y} = \\mathbf{w}^{\\top} \\mathbf{h}_{\\mathbf{x}^{\\star}}(\\mathbf{x})$ | Surrogate prediction in PatternLocal. |\n| $\\mathbf{u}$, $\\mathbf{v}$ | Regression coefficients that minimize Eq. 5 or Eq. 6 loss (equals $\\mathbf{a}$ or $\\mathbf{w}$ at optimum). |\n| $\\mathbf{W}$ | Backward weight matrix: $\\hat{\\mathbf{m}} = \\mathbf{W}^{\\top} \\mathbf{x}$. |\n| $\\Sigma_X$ | Covariance matrix of input $\\mathbf{x}$, used in $A = \\Sigma_X W \\Sigma_M^{-1}$. |\n| $x_1, x_2, x_3$ | Features in the XOR example; $x_3$ is a suppressor variable. |\n| $\\mathcal{F}^+(x_\\star)$ | Ground-truth relevant pixels for data point. |\n\n---\n\n> \"It is not clear why on Page 3, Line 94, w is used instead of W, whereas in all previous instances, W was used to represent the transform matrix.\"\n> \n\nThank you. It has been changed to $\\mathbf{W}$.\n\n---\n\n> \"\"In Page 3, Equation 2, $\\hat{s}$ is recovered with respect to which loss metric? (e.g., is it to minimize the mean squared error?)\n> \n\nEq. 2 states a linear projection; the meaning of the recovered $\\hat{\\mathbf{s}}$ (renamed to $\\hat{\\mathbf{m}}$) and the implicit \"loss metric\" come entirely from the way $\\mathbf{W}$ was learned. It could be recovered with minimizing squared error like in regression or by maximizing class separation like LDA. A deeper explanation is provided in (Haufe et. al. 2014).\n\n---\n\n> \"The manuscript is not well-organized. For instance, the advantages of PatternLocal are discussed in Page 4 in detail before the method is introduced even at a high level...\"\n> \n\nWe apologize for the extra time reviewing this may have caused. We have included a high-level algorithmic explanation of the method (as requested by Reviewer zrTT) beforehand. The order in Section 3 is thus now. High-level description → Advantages → Formal Objective → Toy Example. \n\n| Step | Description |\n| --- | --- |\n| **1. Fit local discriminative surrogate** | Pick a linear explainer (e.g., LIME, KernelSHAP, first-order gradient) and solve its optimization to obtain the weight vector $\\mathbf{w}$ that best approximates the model's decision boundary around the point $\\mathbf{x}_\\star$ (Eq. 5). |\n| **2. Construct data neighborhood** | From the training set, collect samples near $\\mathbf{x}_\\star$ and weight them using the same locality kernel $\\Pi$. |\n| **3. Transform surrogate into generative pattern** | Regress the surrogate scores $\\hat{y}=\\mathbf{w}^\\top h (\\mathbf{x})$ back onto the simplified inputs $h (\\mathbf{x})$  using the weighted neighborhood. The resulting coefficient vector $\\mathbf{a}$ (Eq. 6) defines a generative pattern that minimizes suppressor variables. |\n| **4. Produce saliency / importance map** | Return $\\mathbf{a}$ as the PatternLocal explanation for $\\mathbf{x}_\\star$. If a simplified representation (e.g. super pixels) was used, expand $\\mathbf{a}$ back to the input resolution. |\n\nOther changes include a dedicated Related Works section, notational fixes, further clarification on notation and more than can be seen in the response to the other reviewers.\n\n---\n\n> \"In Figure 4, it is not clear what the x-axis represents. It is stated in the caption that it represents the SNR, however, the value ranges from 0 to 1. It seems from the supplementary material that it represents alpha (as a surrogate for SNR). If so, should we expect that the plots be decreasing in alpha? (as the noise decreases) If so, why are some of the plots increasing in alpha? … Please explain the expected behavior of each plot as a function of the x-axis variable (alpha).\"\n> \n\nThanks for pointing out the missing axis label. We've added $\\alpha$ to the x-axis in Figures 4, 7, 12, 13, and 15. Signal-to-noise ratio (SNR) is controlled by $\\alpha \\in [0, 1]$: higher $\\alpha$ means stronger signal. Not all curves decrease with $\\alpha$ due to two effects:\n\n1. **Edge-detector baselines (Sobel, Laplace):**  \n   As $\\alpha$ increases, tetromino edges sharpen, and filters respond more outside true boundaries. This leads to more false positives, raising EMD/IME.\n\n2. **LIME, PatternLocal, etc.:**  \n   Their scores generally improve with $\\alpha$, since higher SNR aids local fit. The spike at $\\alpha \\approx 0.9$ (WHITE-LINEAR) is likely a numerical issue—at high $\\alpha$, the dataset becomes rank-deficient, making LIME's regression (and thus PatternLocal) unstable.\n\nEmpirical rank of selected datasets:\n\n| $\\alpha$ | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| **XAI-TRIS WHITE LINEAR** | 4096 | 4096 | 4096 | 4096 | 4096 | 4096 | 4096 | 3554 | **2** |\n\n| $\\alpha$ | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 |\n| --- | --- | --- | --- | --- | --- |\n| **XAI-TRIS WHITE RIGID** | 4096 | 4096 | 4096 | 4096 | 2881 |\n\nThe drop to rank 2 explains the WHITE-LINEAR spike. RIGID stays full-rank longer, so its curve stays smooth. We’ve added this to the manuscript and now warn that high $\\alpha$ may cause rank-deficiency.\n\n---\n\n## Final note\n\nWe thank Reviewer 7PSJ once more for their constructive feedback. As detailed above, we have answered every question, clarified all ambiguities and updated the manuscript accordingly. We therefore kindly ask you to reconsider your scores in light of these improvements. Should any further points arise, we are happy to provide additional clarifications at any time."}]

Author rebuttal:

Meta-review: The paper addresses false positive attributions caused by suppressor variables in explanations of nonlinear models. It introduces PatternLocal, which fits a local linear surrogate, then converts its discriminative weights into a generative pattern using neighborhood statistics, reducing suppressor influence while preserving local fidelity. 

Reviewers highlight a clear problem framing, a principled extension of Pattern to local nonlinear settings, and compelling empirical gains across suppressor aware metrics. The method is model agnostic, inexpensive beyond the surrogate fit, and reports local fidelity and runtime.

Reviewers note that the initial manuscript had notational inconsistencies and limited theory. Evaluation leans on synthetic or semi synthetic images, with performance depending on well aligned local neighborhoods and stable surrogates. Sensitivity to surrogate hyperparameters remains. Broader demonstrations on diverse real world modalities and stronger generalization guarantees could further strengthen the work.

The paper tackles a consequential XAI failure mode, offers a simple and sound transformation that integrates with standard local explainers, and shows consistent improvements with reasonable cost. The added EEG study and theory address key doubts. After discussion, reviewers increased scores and signaled consensus for acceptance. An EEG example and sample complexity analysis were added. Reviewers acknowledged improvements and raised scores.
Decision: Accept (poster)