Unlearning-based Neural Interpretations
Authors: Ching Lam Choi
Alexandre Duplessis
Serge Belongie
Venue: ICLR 2025 Oral (2025)
Keywords: Explainability
Attribution
Debiasing
Bias
Primary area: interpretability and explainable AI
TL;DR: UNI computes a debiased, adaptive baseline for gradient-based interpretations by perturbing the input towards an unlearning direction of steepest ascent.

Abstract: Gradient-based interpretations often require an anchor point of comparison to avoid saturation in computing feature importance. We show that current baselines defined using static functions—constant mapping, averaging or blurring—inject harmful colour, texture or frequency assumptions that deviate from model behaviour. This leads to accumulation of irregular gradients, resulting in attribution maps that are biased, fragile and manipulable. Departing from the static approach, we propose $\texttt{UNI}$ to compute an (un)learnable, debiased and adaptive baseline by perturbing the input towards an $\textit{unlearning direction}$ of steepest ascent. Our method discovers reliable baselines and succeeds in erasing salient features, which in turn locally smooths the high-curvature decision boundaries. Our analyses point to unlearning as a promising avenue for generating faithful, efficient and robust interpretations.

Published as a conference paper at ICLR 2025
UNLEARNING-BASER NEURAL INTERPRETATIONS
Ching Lam Choi'
Alexandre Duplessis
CSAIL,, Department of EECS
Department of Computer Science University of Oxford
Massachusetts Institute of Technology chingl amemit edu
a alexandre wwwpessisssee.cccccc ox, ac. uk
Serge Belongie
Pioneer Centre for AI
University of Copenhagen
www.llllleeeeeeee .u.dk
ABSTRACT
Gradient- interpretations often require an anchor point of comparison to avoid saturation in computing feature importance. We show that current baselines defined using static cioctions-contarn mapping, averaging or blurring- colour, texture or frequency assumptions that deviate from model behaviour. This leads to accumulation of irregular gradients, resulting in attribution maps that are biased, fragile and manipulable. Departing from the static approach, we propose UNI to compute an (un)lcarnable, debiased and adaptive baseline by perturbing the input towards an unlearning direction of steepest ascent. Our method discovers reliable baselines and succeeds in erasing salient features, which in turn locally smooths the high-curvature decision boundaries. Our analyses point to unlearning promising avenue for generating faithful, efficient and robust interpretations.
harmful
INTRODUCTION
The utility of large models is hampered by their lack of explainability and robustness guarantees.
Yet breakthroughs in language modelling (Meta, 2024 Anthropic [2024] Jiang et al. [023]] Google
2024| [Achiam et al. 2023) and generative computer vision (Rombach et al. 2022] Liu al. 2023
Deepmind 2024] Brooks et al. 2024) yield promising high-stakes applications, spanning domains of
healthcare, scientific discovery, law and finance. As such, being able to interpret these models has become primary concern for researchers, policymakers and the general populace, with international calls for explainability, accountability and fairness in AI decision- making (European Commission 2021 White House OSTP 2022 [Bengio al. (023). To this end, recent works focus the 2 main directions of making models inherently explainable (Böhle et al. [022] Brendel & Bethge 2018 Koh
al. 2020
Boolle et al. 2021; Chen Chen
al. [0099] Ross 2017) and post-hoc interpretable (Bau
et al. 2017; Kim et al. 2018; Zhou et al., 2018 Ghorbani et al. (0195).. Unfortunately. the former is marred by the status quo of proprietary models and prohibitive training costs. This motivates seeking robust attributions which reliably explain model predictions, to facilitate better risk assessment and
trade-off calibration (Böhle et al. 2022 Broshi-Veler Kim 2017).
Post-hoc methods explain black-box model's output by attributing its decision back to predictive features of the input. They achieve this via leveraging components the model itself (e.g. gradients and activations), or through approximation with simpler, interpretable simulator. A desirable post-hoc explanation should exhibit high faithfulness- - to rationale-consistent (Yeh et al. 2019
Atanasova et al. 2020) with respect to model's decision function; low sensitivity yield reliably similar saliency predictions for input features the same local neighbourhood (Alvarez Mclis & Jaakkola 2018: Ghorbani et al. 2019b); low complexity the explanation should be functionally simpler and more understandable than the original black-box model (Bhatt et al. 2021).
Gradient- based saliency methods widely used for feature attribution, due their simplicity, efficiency and post-hoc accessibility. This can be further decomposed into families: perturbative,
Research work completed during internship at the Pioneer Centre for Al and University of Copenhagen.

---

Published as a conference paper at ICLR 2025
Fc):;0)
Fc(:;0)
Figure I: Left: Confidence of original model image x and baseline x'. Right: Confidence of unlearned model at image x. After unlearning in the model space 0 + 0, we optimise the baseline to match the unlearned input confidence, such that (x 0) = F= 0).
backpropagative and path-based, which we detail in Section o Gradient- based attribution is intuitive since the first-order derivative reveals which features significantly influence the model's classification decision. However, naively using local gradients yields unfaithful attributions due to saturation, where the non-linear output function flattens in vicinity of the input and zero gradients are computed (Sundararajan et al. 2017: 2016). To improve gradient-sensitivity, later methods introduce baseline input for reference, and backpropagate the difference in activation scores on a path between the reference and image-of-innerew (Shrikumar et al., 2016 Sundararajan et al., 2017). The baseline chosen to be devoid of predictive features and far away from the saturated local neighbourhood. However, such methods accumulate gradient noise when interpolating from the baseline to the input, leading to high local sensitivity (Ancona et al., 2018). Consequently, attribution maps become disconnected, sparse and irregular, where the saliency scores fluctuate wildly between neighbouring pixels of the same object and are visually noisy (Adebayo et al.) 2018). This noise accumulation has two root causes a poorly chosen baseline and high-curvature output manifold along the path features. Previous works (Sturmfels et al.) 2020 Xu et al. 2020) have sought better baselines by empirically comparing between using black image, gaussian noised image, a gaussian blurred image, a uniformly noised image, an inverted colour image, as well as averaging attributions over several baseline choices. However, the correct baseline to represent a lack of salient features depends heavily on the specific classification task, on the trained model and on the input image. Indeed, the optimal baseline varies for each lask-model-imag combination (Akhtar & Jalwana 2023): ;
the baseline problem remains largely unsolved. Turning to the second problem of high-curvature
output manifold, because trained neural networks exhibit approximately piece-wise linear decision boundaries (Goodfellow et al. 2014), inputs near function transitions are vulnerable to perturbative attacks. By simply adding norm- form-bounded imperceptible adversarial noise to the input image, attackers can dramatically alter the attribution map without changing the model's class prediction (Ghorbani ct al. 2019a| Dombrowski et al., 2019). Methods of mitigation include explicit smoothing via averaging over multiple noised gradient attributions (Smilkov al., 2017] adaptively optimising the integration path of attribution (Kapishnikov et al., 30 imposing an attribution prior during training and optimising it at each step (Erion et al.) (202I) However, all of these proposals starkly increase the complexity of attribution, requiring computationally costly forward and backward propagation steps.
To tackle the problematic triad of 1. post-hoc attribution biases, poor baseline defiiition, 3. high- curvalure outpul manifold, we propose UNI to discover debiased baselines by locally unlearning inputs, i.e. perturbing them in the unlearning direction of steepest ascent, as visualised in Figure [] Towards better baselines, our unlearned reference is by definition explicitly optimised to lower output class confidence and can empirically erase or occlude salient features. We also say that the unlearned baseline is specific and featureless w.r.t. each task wosk-model-inpeee combination. Unlike the practice of using a black image maseline-which creates a post-hoc colour bias that darker pixels are less likely to be salient, UNI does not impose additional, pixel-wise colour, scale or geometric assumptions that are not already present in the model itself. Finally, we address the high-curvature decision boundaries problem by realising that this is a product of the training process targettd unlearning smooths the decision boundary of the model within the vicinity of the input. For a more detailed overview on the principle of machine unlearning, we refer the reader Section of the supplement. We empirically
2

---

Published as a conference paper at ICLR 2025
verify this local smoothing effect by measuring the normal curvature of the model function before and after unlearning, we also demonstrate that unlearning makes attributions resistant to perturbative attacks. Our contributions can be summarised follows:
1. Post-hoc attribution can impose new biases. We approach the baseline challenge from the fresh lens of post-hoc biases. We show that static baselines (e.g. black, blurred, random noise) inject additional colour, texture and frequency assumptions that are not present in the original model's decision rule, which leads explanation infidelity and inconsistency.
2. A well-chosen baseline is specific and featureless. We establish theoretically grounded principles for sound bascline definitions, by formalising the idea of an "absence of signal" through an unlearning direction of steepest ascent in model loss. By unlearning predictive features in the model space and matching this reference model's ss activations with a perturbation in the input space, introduce a new definition of "feature absence" and novel attribution algorithm.
3. Unlearning reduces the curvature of decision boundaries and increases robustness. Targeted unlearning simulates function statistics of unseen data, and smooths the curvature of the output manifold around the sample. This is characterised by low geodesic path curvature and bounded principal curvature of the output surface. This points to reduced variability of the gradient vector under small-none norm input perturbations, leading to better attribution robustness and faithfulness.
2 PRELIMINARIES
We consider feature attribution for trained deep neural networks within image classification. Informally, we seek to assign scores to each pixel of an image for quantifying the pixel's s influence (sign and magnitude) on the predicted output class confidence. It is noteworthy that attributions can be signed: a negative value indicates that removing the pixel increases the target class probability.
2.1 NOTATION
The imput (feature) space is denoted as X' Rdx, where dx is the number of pixels in image. The output (label) space is y C Rdy; y is the set of all probability distributions on the set of classes. The model space is denoted as F y² A trained model F x r (xr) Fay (x)) returns the probability score Fe (x) of each class c. Attribution methods are thus functions A [....,dx] x F ((,...,dy d X X' + I. where A(i, F, the importance score of pixel of image x for the prediction made by Fe. For convenience, we use the shorthand A.(x) to refer to the attributed saliency score of a pixel for specific class prediction e {1, dy }. We express
a linear path feature as 7(2', .x, a. Rdx x Rxx x [0, 1] + Bdxx where = (1 - a)2²' + ax and employ shorthands 70) = x', (1) x.
GRADIENT-BASED ATTRIBUTIONS IN A NUTSHELL
3.1 LIMITATIONS
Taking the local gradients of a model's 's output confidence map Fo(x)) for target class is a tried and tested method for generating explanations. Commonly termed Simple Gradients (Erhan et al. 2009 Baehrens et al. 2010 Simonyan et al. 2013). (x) = V z; Fz( can be efficiently computed for most model architectures. However, it encounters output saturation when activation functions like ReLU and Sigmoid are used, leading to zero gradients (hence null attribution) even for important features (Sundararajan et al., 2017 2016). DeepLIFT (Shrikumar et al. (0166) reduces saturation by introducing a "reference state". A feature's 's saliency score is decomposed into positive and negative contributions by backpropagating and comparing each neuron's 's activations to that of the baseline. Integrated Gradients (IG) (Sundararajan et al. 2017) similarly utilises a reference, black image and computes the integral of gradients interpolated on straight line between the image and the baseline.
S a==
A!((2) = (x, 16xxx() x())
Po + a(x - x²)) da
Practically, the integral is approximated by a Riemann sum. Of existing methods, IG promises desirable, game-theoreti, properties of "sensitivity", "implementation invariance". "completeness"
3

---

Published as a conference paper at ICLR 2025
Algorithm UNI: unlearning direction,
Swan (0.999)
UNI (ours)
baseline matching and path-attribution
1: Given model F(., 0): inputs (æ. y)
2: Choose unlearning step-size ny
PGD steps T, budget E, step-size μc; Riemann approximation steps B
3: Initialise perturbation 8
4: Unlearning direction.
i + ") V. (.
5: for t = 0,... ... T do
Black
Blurred
Random
C D (1; 0) F 0):0))
Attribution Baseline
g++1 8¹ r VaC
8 8
||8++11|
6: end for
7: Baseline definition x' = x x 6J
8: Attributions computation: APTi (
Figure 2: We visualise post-hoc biases imposed by static daselines-blac baseline (colour), blurred (texture), random (frequency). UNI learns to mask out predictive features used by the model, generating reliable attributions.
(x.
k B (x ):0)
Va,
Fe
B
and "linearity". We consequently focus analysing and developing the IG framework, though the proposal to unlearn baselines can be applied to most mainstream gradient- based saliency methods. Despite the advantages of IG, its soundness depends on good baseline definition-an an input which represents the "absence" of predictive features; also on having stable path features-a straight-line of increasing output confidence along the path integral from baseline to target image. In the conventional setting where a black image used, [Akhtar Jalwana (2023) prove that IG assumptions are violated due to ambiguous path features, where extrema of model confidences lie along the integration path instead of at the endpoints of the baseline (supposed minimum) and input image (supposed maximum). Sturmfels et al. (2020) enumerate problems with other baselines obtained via gaussian blurring, maximum distance projection, uniform noise. Despite the diversity of baseline alternatives, no candidate is optimal for each and every attributions setting. For instance, models trained with image augmentations (e.g. colour jittering, rescaling, gaussian blur) yield equivalent or even higher confidences for blurred and lightly-noised baselines- -we need baselines that are well-optimised for each -ooee-ooo----eeeeeeeeeeee input combination. Without principled baselines, problems of non-conformant intermediate paths and counter-intuitive attribution scores will doubtlessly persist.
3.2 POST-HIC BIASES ARE IMPOSED
Since the baseline represents absence of or reduction in salient features, static baseline functions (e.g. black, blurred, noised) implicitly assume that similar features (e.g. dark, smooth, high-frequency) irrelevant for model prediction. To illustrate this intuition, we can consider IG with black baseline, wherein it becomes more difficult to attribute dark but salient pixels. Due to the colour bias that "near-black features are unimportant", the term (z,-2;) is small and requires a disproportionaatly large gradient V (-).) (+) to yield negligiie attribution scores. Indeed, this is what we observe in Figures 3 where darker features belonging to the ofject-of-interee cannot be reliably identified. We further empirically verify that each static baseline imposes its own post-hoc bias by experimenting on ImageNet-C (Hendrycks & Dietterich 2019). Corresponding to the 3 popular baseline choices for IG (all-black, gaussian blurred, gaussian noised), we focus on the families of digital (brightening and saturation), blur (gaussian and defocus blur) and noise (gaussian and shot noise) common-comee cormmonnoorrpttons... Figures 4 12 demonstrate that IG with blurred baseline fails to attribute blurred inputs due to saturation and overly smoothed image textures; Figures [5] [3 visualise how a noised IG baseline encounters high-frequency noise and outputs irregular, high-variance attribution scores, even for
4

---

Published as a conference paper at ICLR 2025
adjacent pixels belonging to the same object. We crucially emphasise that such colour, texture and frequency biases are not present naturally the pre-trained model but rather injected implicitly by a suboptimal choice of static baseline. The observation that poor baseline choices create attribution bias has so far been overlooked. As such, we depart entirely from the line of work on alternative static baseline towards adaptively (un)learning baselines with gradient-based optimisation. UNT eliminates all external assumptions except for the model's own predictive bias.
Bodtes
Beghtestng
Seanitee
Becter Kap
Table 1: Path monotonicity scores with Spearman correlation coefficient (higher better). Integrating from a "featureless' baseline to the sample should give a path of monotonically increasing prediction confidence.
UNI
IG
BlurGG
GIG
ResNet-18
.97
.69 4.400
.57 4.676
.45 = 446
Eff-v2-s
.95
#.una
.28 #.615
.34 4.613
.38
ConvNeXI-T
.99
#.121
.76 4.379
.77 4.6
.46 485
VGG-16-bn
.94
252
4.474
.60
4.544
.46 479
ViT-B-16 16 SwinT
.89 2.396
.71 1.399
.27 1.648
.44
Figure 3: When the brightness or saturation is altered, IG with black baseline fails identify dark features, such the boat's hull (R3) or the top of the boot (RI).
.97 180
.88 2.326
.88 484
.45 447
Hig Medde Coemten Barr
Deken Bar Busiie
May
Vay Borler Genvics
Skd rodo
Bodbe
Key
Figure 4: Under gaussian or defocus blur, IG with a blurred baseline suffers from saturation; has overly smooth texture; does not yield meaningful features.
Figure 5: Gaussian and shot noise create visual artifacts prominent in baseline IG. Frequency hias leads to disparate scores for adjacent pixels.
4 UNI: UNLFARNING-BASED NEURAL INTERPRETATIONS
4.1 BASELINE DESIDERATA
desirable baseline should preserve the -heeretce properties of path- aath-attribution (Section 3.1 and refrain from imposing post-hoc attribution biases (Section 3.2). For every given task-model image triad, a well-choser baseline should be 1. image-specific be connected via a path feature of low curvature to the original image; 2. reflect only the model's predictive biases saaase--saliett image features should be excluded from the baseline; 3. less cusk-informative than the original image- interpolating from the baseline towards the input image should yield path of increasing predictive confidence. We now introduce the UNI pipeline: first, unlearn predictive information in the model space; then, activation-matele matchiig between unlearned and trained models mine featureless baseline in the image space; finally, interpolate along the low-curvature, conformant and consistent path from baseline image to compute reliable explanations in the attributions space. Figure visuals and Table 7 results attest to UNI's ability to compute specific, unlearned baselines for attribution.
4.2 DESIRABLE PATH FEATURES
Proximity The meaningfulness the attributions highly depends on the meaningfulness of the path. We aim for smooth transition between absence and presence of features; and this intuitively cannot achieved if the baseline and input are too far apart. [Srinivas & Fleuret (2019) formalises this intuition through the concept of weak dependence, and proves that this property only be
5

---

Published as a conference paper at ICLR 2025
compatible with completeness in the where the baseline and the input lie in the same connected component (in the case of piecewise-lings
models). An obvious implementation of this proximity condition in the general case is to bound the distance || z'Il to certain value This is strictly enforced in Algorithm by normalising the perturbation at each step I.
Low Curvature. The curvature of the model prediction along the integrated path has been identified Dombrowski et al. (2019) as one of the key factors influencing both the sensitivity and faithfulness of the computed attributions. We substantiate the intuition that smooth and regular path is preferred by analysing the Riemannian sum calculation. Assuming that the function
E [0, 1] + VF. (ar' + afx x²)) is derivable with continuous derivative (i.e. C²) on the segment ar), elementary calculations and the application of the error in the Riemann approximation of the attribution,
inequality give the following
[ a==
g (a) da- (x,
B
M M| 2'12
(x, - 21{)
(2)
B
B
2
exists by continuity of g' on [0, 1]. Thus, lower curvature along the path implies a lower value of the constant M, which in turn implies lower error in the integration calculation. A smaller value B of Riemann steps is needed to achieve the same precision. More generally, a low curvature (i.e. eigenvalues of the hessian) on and in neighbourhood of the baseline and path reduces the variability of the calculated gradients under normm perturbations, increasing the sensitivity and consistency of the method. Empirically, we observe a much lower curvature of paths computed by UNI, as per Table I and Appendix Figures 20 Figure also confirms the increased robustness Riemann sum error induced.
where M maXae(0,1] Zo maXa
dg
8² (FF.(c'ta(z=2))
da
21
25
Monotonic. Intuitively, the path defined by interpolating from the "featureless" baseline x'' to the input image x should be monotonically increasing in output class confidence. the image level, for all j, k such that j S k, since Ilr(j) - =|| 2 (l)(k) - zlI. therefore the predictive confidence should be non-decreasing and order-prest pedderrpes:::::::: Fe(r(()) I (x()).)). Constraining ? to be monotonically increasing suffices to satisfy a weak version of the criteria for valid muthfaature (Akhtar & Jalwana 2023; sgn (V.FF.()))) (V ((=))()))) = 1 is naturally met.
4.3
EFFECTS OF UNLEARNING AND MATCHING
We explain the success of UNI with the illustrative example of a three gaussians mixture model. Figure Saauur/@@omputttte unlearning and activation matching for model learned on three data points with gradient descent. F' chosen to be the output of the three gaussian components (G1, G2, G3). Note that the perturbation is not E- e-nomaalieed for clearer visualisation. We highlight two observations:
The UNI path is monotonous, of low-curvature and proximal. Conversely, the path to the random baseline is long, non non-monotonous, and goes through several zones of high second derivative.
Optimizing KL divergence (G1, G2, G3) produces better baseline. Figure 6b visualises the unlearning objective (i.e. the target probability after unlearning), which gives four points of intersection with the base model (a, b, c and d). By constraining proximity of the baseline with the e parameter, we restrain the optima found by gradient descent (on the global probability) to the closest two points and 6. UNI is then able yield the more optimal of the two, by optimising on each gaussian output. In fact, the idea of activation matching is to satisfy the crucial weak dependence property for conformal path attribution (Akhtarr & Jalwana, 2023). Since modern ReLU networks have decision boundaries representable as piecewise linear functions (Xiong et al.) 2020), activation matching supervises the baseline to use the same (piecewise linear) weights. In our case, we want to find baseline for which G and G2 do not play role, which is not the case for a. This is why A optimises on F and not on Fe-
Finally, normalisation serves to regularise baseline GD learning and account for pathological cases where the locally shortest path would lead to further intersections than the closest one.
5 EXPERIMENTS
We experiment on ImageNet-Ie IK (Deng et al., 2009). ImageNet-C (Hendrycks & Dietterich) 2019) and compare against various path-based and madient-based attribution methods. This includes IG

---

Published as a conference paper at ICLR 2025
Data Poins
USI Boviee
(a) Unlearned Model
(b) Base Model
Figure 6: UNI baseline Gaussian mixture model of three gaussians G1, G2, G3, each of fixed variance, parametrised by their mean and a scaling factor. (b) shows the model trained on the three datapoints (1,1), (2, 5) and (4, 3), while (a) shows the model after gradient ascent step the datapoint (2, 5). The path between UNI Baseline and the image is highlighted by arrows in (b).
(Sundararajan et al. 2017), BlurIG (Xu et al. 2020), GIG (Kapishnikov et al., 2021), AGI (Pan et al.
2021). GBP (Springenberg et al. 20144) and DeepLIFT (Shrikumar et al., 2016). We consider a diverse sct of pre- tre-trained computer vision backbone models (Paskke et al. 2019), including ResNet-18 (He al., 2016), EfffcientNet-v222. -2-mmlll (Tan & Le 202T), ConvNe Tiny T' (Liu al. 2022), VGG-16-bn (Simonyan & Zisserman (0155), ViT-B 16 Dosovviskiy et al.] 2020) and Swin- Twa.sTrrnnrrr---ee
(Liu al. 2021). Unless otherwise specified, we the following hyperparameters: unlearning step size 1; PGD with T 10 steps, a budget of E 0.25, step size μ 0.1; Riemann approximation with = 15 steps. We further extend UNI to the NLP domain, to interpret generative language models using activation patching (Heimersheim & Nanda 2024). UNI complements activation matching by computing a stable baseline without trading off attribution scalability, as observed in Appendix Table Bnd Figure Our results verify UNT 's high faithfulness, stability and robustness.
Table MuFidelity scores measure the correlation between subset of pixels' impact on the output (i.e. change in predictive confidence) and assigned saliency scores. Since attribution methods can yield strong positive or negative correlations, we report the absolute scores.
UNI
IG
BlurIG
GIG
AGI
GBP
DecpLIFT
R8
.12 1.124
.06
1.068
.07
.07 2.080
.10 4.110
.09
.08
EfficientNetv2s
.06 1.046
.05
1.043
.05 2.044
.05 2.044
.06 1.045
.05 1.043
.05 1.043
ConvNeXt-Tinn
.16 1.115
.11 1.086
.15 1.121
.18 1.149
.17 1.131
.09 1.072
.11 1.084
VGG-16-bn
.18 141
.08 4.066
.09 4.07a
.13 1.108
.14 1.104
.13 1018
.10 4.082
ViT-B 16
.15
.10 4.074
.10 -.07
.11 4.079
.14 1.104
.09
.10 #.072
Swin-T-Tiny
.13 100
.09 2.071
.12 1012
.12 2.14
.13 1.12
.09 1.060
.10 #:078
5.1 FAITHFULNESS
We report MuFidelity scores (Bhatt al. 2021). i.e. the faithfulness of an attribution function A. to a model F, at a sample v, for subset of features of size [S], given by (f(FF, We record the (absolute) correlation
corr se())) (Eies Ffi,F, F,,,)), Fx(x) F. Fe(
coefficient between a randomly sampled subset of pixels and their attribution scores. In line with open source exemplars (Fel et al. 2022a),, set |S] to be 25% of the total pixel count (slightly higher than the referenced 20%) as is required to adjust for ImageNet's complexity and for obtaining less noisy measurements across all baseline methods. As from Table UNI outperforms other methods across all settings but one, indicating high faithfulness. We supplement these numbers with visual comparisons in Appendix Figures 14
15
16 17
T8
against IG (black and noised baselines),
BlurlG, GIG, AGI, GBP, DeepLift. Furthermore, we report deletion and insertion scores (Petsiuk ct al. 2018 causally-motivatee evaluation metric for interpretability methods--hhich measures the decrease (deletion) or increase (insertion) of model's output confidence as salient pixels are removed (from the original image) or inserted (into a featureless baseline). steep drop in model confidence under pixel deletion results in a desirable and small area under the curve (AUC) score; a sharp rise under pixel insertion results in a desirably large AUC. Salient pixels are removed in descending order of importance, identified by the tested interpretability method. We evaluate with
7

---

Published as a conference paper at ICLR 2025
step size of 10% and average over 10,000 random image samples, where each step, the next- 10% most salient pixels are removed or inserted for inference. UNI reliably identifies pixels which are crucial for sample classification, achieving marked improvements especially in insertion AUC scores.
Table 3: Deletion AUC 4 measures how confidence drops as pixels are removed (lower better).
UNI
IG
BlurIG
GIG
AGI
GBP
DeepLIFT
ResNet-18 18
.06 2.128
.10 2.174
.27 2.252
.11 5.150
.13 +.147
.08 2.0
.13 1.165
EfficientNetv25
.19 2.212
.26 1.217
.50 1.158
.19 2:216
.18 1.207
.23 161
.27 4.215
ConvNeXt-Ting
.11 1.139
.16 4.164
.46 4.172
.21 4.100
.17 +.123
.16 2.09
.21 1.102
VGG-16-bn
.08 1.143
.12 1.181
.18 1.241
.10 1.163
.14 4.178
.14 1.104
.12 1.166
ViT-B_1e
.14 1.1
.22 4:207
.60 4.166
4.190
.13 1.102
.23 #:141
4.189
Swin-T-Tiny
.13 1818
.22 2:217
.47 1.174
.22 2.207
.21 4.172
.21 #.123
.23 4.207
Table 4: Insertion AUC measures how confidence rises pixels are inserted (higher better).
UNI
IG
BlurIG
GIG
AGI
GBP
DeepLIFT
ResNet-
1.188
4.045
131
.36 2.048
.56 2.00s
.11
6.0
.18 #.042
EfficientNet22s
.64 2.227
.38
4.127
.51 223
.37 2.138
.38 2.204
.23 #.19
.37 1.137
ConvNeXI-Tiny
.63 1.281
.21
1.114
.40 2.25
.56 1.122
.52 1.086
.22 1.160
.17 1.162
VGG- 16-bn
.56 +.385
4.061
2.274
.38 2.071
+.078
.26 2.057
.17 2.056
ViT-B
.71 1.237
.32 1.107
.59 2.29
.28 1.125
.43 1.089
.35 1.172
.28 1.123
Swin-T-Tiny
.68 2.245
.28
.63
.26
.25 4.136
.31
.26 1.122
5.2
ROBUSTNESS
Next, we evaluate UNI's robustness to fragility adversarial attacks on model interpretations. Following Ghorbani et al. (019a), we design norm-bounder attacks to maximise the disagreement in attributions whilst constraining that the prediction label remains unchanged. We consider standard 4oo attack designed with FGSM (Goodfellow et al. 2014), with perturbation budget E 8/255.
89 =arg [|y Syy dx
max
d (A(i, A(i, F, -5))
(3)
subject to arg max For (x) = arg max For +8) =c
We report robustness results using distance measures- Spearman correlation coefficient in Table 5 and top-k pixel intersection score in Table 6 pre and post attack. While other methods like DeepL.FT (DL), BlurIG, Integrated Gradients (IG) are misled to output irrelevant feature saliencies. UNI robustly maintains attribution consistency and achieves the lowest attack attribution disagreement scores (before and after FGSM attacks) for both metrics.
5.3 STABILITY
We compare UNI and other methods' sensitivity to Riemann approximation noise, which manifests in visual artefacts and misattribution of salient features. As seen from Figures [ UNI reliably finds unlearned, "featureless" baselines for consistent attribution, regardless of the number of approximation steps E [1, 15, 30). This is due the low geodesic curvature of o UNI which approximately minimises the local distance between points used in Riemann approximation.
Table 5: Robustness: Spearman's correlation coefficient. Higher scores indicate better path consistency pre/post FGSM attacks.
Table Robustness: Top- 1000 pixel intersection. Higher percentages indicate better attribution reliability pre/post FGSM attacks.
UNI
IG
BlurIG
SG
DeepL
UNI
IG
BlurIG
SG
DeepL
R8
.271
.088
.084
.014
.139
ResNet-18
37.3
20.0
25.3
18.2
21.8
Eff-v2-s
.302
.009
.076
.008
Eff-v2-s
39.4
17.4
23.3
18.6
18.0
ConvNeXi-T
.292
.010
.127
.011
.012
Conv-NeXi-T
34.8
15.0
26.2
16.7
15.1
VGG- 16- bn
.290
.143
.098
.014
.108
VGG-
35.7
25.5
25.3
18.8
25.2
ViT-B-16
.319
.018
.066
.023
.023
VIT-B-16
40.7
17.1
21.7
19.6
17.2
SwinT
.271
.088
.084
.014
.139
SwinT
37.3
25.3
18.2
24.8
8

---

Published as a conference paper at ICLR 2025
Image
Baseline
1 step
15 steps
30 steps
R
UNI
IG (black)
Path Stability
1G (blur)
UNi vs. IS Stability
IG (noise)
Figure 7: UNI path features monotonically increase in output confidence when interpolating from baseline to input. This eliminates instability and inconsistency problems caused by extrema and turning points along the Riemann approximation path, which is present in other methods.
RELATED WORK
Machine unlearning. We draw inspiration from the high-level principle of unlearning, which concerns the targeted "forgetting" of a data-point for trained model, by localising relevant information stored in network weights and introducing updates or perturbations (Bourtoule et al. 2021). Formally, machine unlearning can be divided into exact and approximate unlearning (Nguyen et al. 2022). Exact unlearning seeks indistinguishability guarantees for output and weight distributions, between model not trained sample and one that has unlearned said sample (Ginart et al., 2019 Thudi et al. 2022, Brophy & Lowd (02T]. However, provable exact unlearning is only achieved under full re-training, which can computationally infeasible. Hence, approximate unlearning was proposed stemming from r-differential privacy (Dwork) (011)) and certified removal mechanisms (Guo et al. 2020 Golatkar et al.) (0220). The former guarantees unlearning for = i.e. the sample has null influence on the decision function; the latter unlearns with first/second order gradient updates, achieving max max-divergenne bounds for single unlearning samples. Unlearning naturally lends itself to path-based attribution, to localise then delete information in the weight space, for the purposes of defining an "unlearned" activation. This "unlearned" activation can be used match the corresponding, "featureless" input, where salient features have been deleted during the unlearning process. While the connection to interpretability is new, a few recent works intriguingly connect machine unlcarning the task of debiasing classification models during training and evaluation (Chen et al. 2024; Kim et al. 2019 Bevan & Atapour- Abarghouei] (202)).
Perturbative methods. et al.] (005)) including LIME (Ribeiro et al., 2016), SHAP, KernelSHAP and GradientSHAP
Perturbative methods perturb inputs to change and explain outputs (Sculley
(Lundberg et al.). RKHS-SHAP (Chau et al.] 2002), ConceptSHAP (Yeh et al. 2020). InterSHAP
(Janzing et al.] 2020), and DiCE (Kommiya Mothilal et al.) 2021). l) LIME variants optimise
simulator of minimal functional complexity able to match the black-box model's local behaviour
for a given input-label pair. SHAP (Lundberg al.) consolidates LIME, DeepLIFT (Shrikumar et al. 2016). Layerwise Relevance Propagation (LRP) (Montavon et al. 2019) under the general, theoeteeo framework of additive feature attribution methods. For this framework, they outline the desired properties of local accuracy, missingness, consistency; they propose SHAP values as a feature importance measure which satisfies these properties under mild assumptions to generate model-agnostic explanations. However, such methods fail to give global insight of the model's s decision function and are highly unstable due to the reliance on local perturbations (Fel et al. 2022b). Bordt et al. (2022) show that this leads to variability, inconsistency and unreliability in generated explanations, where different methods give incongruent explanations which cannot be acted on. Recent works have made considerable progress, including RISE (Petsiuk et al. 2018). which strives

---

Published as a conference paper at ICLR 2025
to causally explain model predictions by approximating the necessary saliency of pixels through random masking; Sobol (Fel et al. 2021), which adapts Sobol indices for perturbation masks towards variance- suriance-based sensitivity analysis; and FORGrad (Muzellec et al. 2023), which filters out high- frequency gradient noise induced by white-box methods (and network pooling or striding operations)
and which can be complementarily applied to further UNI'S 's explanation faithfulness and efficiency.
Backpropagative methods. Beginning with simple gradients (Erhan et al. 2009 Simonyan et al. 2013), this family of methods also, IRP (Montavon et al.] 2019). Deepi. IFT (Shrikumar et al.
2016), DeConvNet (Zeiler& & Fergus 2014). Guided Backpropagation (Springenberg et al., (0144) and
GradCAM (Selvaraju et al. ( leverages gradients of the output w.r.t. the input proportionally project predictions back to the imput space, for some given neuron activity of interest. Gradients of neural networks are, however, highly noisy and locally sensitive they can only crudely localise salient feature regions. While this issue is partially remedied by SmoothGrad (Smilkov et al.) 2017). we still observe that gradient-based saliency methods have higher sample complexity for generalisation than normal supervised training (Choi & Farnia, 2024) and often yield inconsistent attributions for unseen images at test time.
Path-based attribution. This family of post-hoc attributions is attractive due to its grounding in cooperative game-theory (Friedman (004). It comprises Integrated Gradients (Sundararajan et al. 2017), Adversarial Gradient Integration (Pan et al. (202I), Expected Gradients (EG) (Erion et al.
2021 Guided Integrated Gradients (GIG) (Kapishnikov et al. 2021) and BlurIG (Xu et al. (0220). Path attribution typically relies on a baseline a "vanilla" image devoid of features; a path an often linear path from the featureless baseline to the target image-an along which the path integral is computed for every pixel. Granular control over the attribution process comes with difficulties of defining an unambiguously featureless baseline (for each (model, image) pair) (Sturmfels al. 2020 and then defining a reliable path of increasing label confidence without intermediate inflection points (Akhtar& & Jalwanal 2023). To measure the discriminativenesss of features identified by attribution methods and the extent to which model predictions depend on them, experimental benchmarks and metries such as ROAR (Hooker et 2019). DifiRAOR (Shah et al., (20T)) deletion/insertion score (Petsiuk et al. 2018), the Hilbert-Schmidt independence criterion (HSIC) (Novello et al. 2022) and the Pointing Game (Zhang et al. 2018a) have been proposed.
CONCLUSION
In this work, we formally discuss the limitations of current path-attribution frameworks, outline a new principle for optimising baseline and path features, as well as introduce the UNI algorithm for unlearning-base based neural interpretations. We empirically show that present reliance on static baselines imposes undesirable post-hoc biases which are alien to the model's 's decision function. We account for and mitigate various infidelity, inconsistency and instability issues path- anth-atrribttinn by defining principled baselines and conformant path features. UNI leverages insights from unlearning to eliminate task-salient features and mimic baseline activations the "absence of signal". It discovers low-curvature, stable paths with monotonically increasing output confidence, which preserves the completeness axiom necessary for path attribution. We visually, numerically and formally establish the utility of UNI as a means compute robust, meaningful and debiased image attributions.
The contributions of UNI extend beyond the presented method and analyses, towards investigating machine un unlearning as a tool for white-box interpretabilitty Unlearning at different granularities allows us to audit the various levels of a model's learned feature hierarchy. In this work, we illustrate how first-order, sample-wise unlearning can identify salient input features important for a single prediction. A promising future direction involves interpreting higher-level, semantically complex concepts required to learn task or fit data distribution. by instead unlearning a set of concept- clustered exemplars. It is also of interest to delve into how interpretability methods impose additional assumptions onto trained models, prompting questions such as how to best design and align the correct interpretability method for a given model; how to use attribution methods to compare and contrast the inductive biases of different network architectures, of models trained with robust versus non-robust objectives, of models trained using different equivariant data augmentation strategies. Further technical extensions to UNI include going beyond first-order approximate unlearning towards certified, second-order machine unlearning; as well as granular investigations of how the baseline definition, model's robustness and model's 's inductive biases exert influence on path attribution results.
10

---

Published as a conference paper at ICLR 2025
8 ACKNOWLEDGEMENTS
This work was supported in part by the Pioneer Centre for AI, DNRF grant number Pl.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprini arXiv:2303.0877 2023.
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), NeurlPS, volume 31. Curran Associates, Inc.,
2018. URL https::/ppooceeeee/oettt F1 215799/aee2441ll2222///aaaaeeee PPP .pdf]
nwur www.eurpps.ccppppeeeeee cc.paper.pppp...pcoomcp wttpppeefflsspppper//aaee/ es/paper/2018.
Naveed Akhtar and Mohammad A. A. K. Jalwana. Towards credible visual model interpretation with path attribution. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 439-457. PMLR, 23-29 Jul 2023. URL htwps: httppss//ppocee..nggggs wtt.://s.m.pppr..ss.meemee
http//aaaaaraaaaa.aacaa w22p///ataa22..atttt.... mlr wrw.prrsssss.eeee /v202//2222222
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. NeurIPS, 31, 2018
Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards hetter understanding of basde attribution methods for deep neural networks. In ICLR, 2018. URL https
Anthropic. Introducing the generation of claude. Mar 2024. URL
https: /ww
Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. A diagnostic study of explainability techniques for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3256-3274, 2020.
David Bachrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe. Katja Hansen, and Klaus-Robert Müller. How to explain individual classification decisions. Journal of Machine Learning Research, (013):1803-1811, 2010. URL http://jmlr..re www.o/ggggp////a/....ee
baehrensl0a.1.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6541 66549, 2017.
Yoshua Bengio, Geoffrey Hinton, Andrew Yao, Dawn Song, Pieter Abbeel, Yuval Noah Harari, Gillian Hadfield, et al. Managing ai risks in an era
Ya-Qin Zhang, Lan Xue, Shai of rapid progress. arXiv preprint arXiv:2310. 17688, 2023.
Peter Bevan and Amir Atapour-Abarghouei Skin deep unlearning: Artefact and instrument debiasing in the context of melanoma classification. In International Conference on Machine Learning, pp. 1874-1892. PMLR, 2022.
Umang Bhatt, Adrian Weller, and José MF Moura. Evaluating and aggregating feature-based model explanations. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp-
2021.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O' B'Brien Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling, 2023. URL httpts:/t/aaatte org/ abs /2044 01373
11

---

Published as a conference paper at ICLR 2025
Moritz Bohle, Mario Fritz, and Bemt Schiele. Convolutional dynamic alignment networks for interpretable classifications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition (CVPR), pp. 10029-10038, June 2021.
Moritz Böhle, Mario Fritz, and Bernt Schiele. B-cos networks: Alignment is all we need for interpretability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10329-10338, 2022.
Sebastian Bordt, Michèle Finck, Eric Raidl, and Ulrike von Luxburg. Post-hoc explanations fail to achieve their purpose in adversarial contexts. In Proceedings of the 2022 ACM Conference Fairness, Accountability, and Transparency. FAccT '2, pp. 891-905, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450393522. doi: 10.1145/3531 11..533111666333331
URL https:
3533153
Lucas Bourtoule, Varun Chandrasekaran, Christopher Prooquette-Choo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141-159. IEEE Computer Society, 2021.
Wieland Brendel and Matthias Bethge. Approximating cnns with bag-of-local-fenn surprisingly well on imagenet. In International Conference on Leurning Representations, 2018.
models works
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng. Ricky Wang, and Aditya Ramesh. Video generation models world simulators. 2024. URL pttntttss://ppeeee i comm research/
video generat oon-iooo-noone models assa-ss--- as wor ld simulators
Jonathan Brophy and Daniel Lowd. Machine unlearning for random forests. International Conference on Machine Learming, pp. 1092-1104. PMLR, 2021.
Siu Lun Chau, Robert Hu, Javier Gonzalez, and Dino Sejdinovic. RKHS-SHA Shapley values for kernel methods. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL htttp:://ppenreeeeeeeee htte//oormmiid=gne222
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: Deep learning for interpretable image recognition. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: http::::/pprcceediii ngs neur www.ips.cc//eeec cc/paper_ppppppppppppe _i esspaper/2019) File/ach Tale/adETi 2tt.fff2222111120011118822222200
Ruizhe Chen, Jianfei Yang, Huimin Xiong. Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng, Joey Tianyi Zhou, Jian Wu, and Zuozhu Liu. Fast model debias with machine unlearning. Advances in Neural Information Processing Systems, 36, 2024.
Ching Lam Choi and Farzan Farnia. On the generalization of gradient- based neural network interpretations, 2024.
Arthur Commy, Augustine N. Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga- Alonso. Towards automated circuit discovery for mechanistic interpretabilityy. 2023. URL https
7/
14997
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. autoencoders find highly interpretable features language models, 2023. URL https
Sparse
//arxiva abs/abbs//200 08600
Google Deepmind. Veo: Our most capable generative video model. May 2024. URL https 7/ deepmii nd. wteeepmidd..ooggll///.eeee
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pp. 248-25. 2009. doi:
12

---

Published as a conference paper at ICLR 2025
Ann-Kathrin Dombrowski. Maximillian Alber, Christopher Anders, Marcel Ackermann, Klaus- Robert Müller, and Pan Kessel. Explanations can be manipulated and geometry is to blame. Advances in neural information processing systems, 32, 2019.
Finale Doshi Velez and Been Kim. Towards rigorous science of interpretable machine learning. arXiv preprint Stti::/702.08000 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Reprexentations, 2020.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The llama herd of models. arXiv preprinl 2xXi::247..21782 2024.
Cynthia Dwork. Differential Privacy, pp. 348. Springer US, Boston, MA, 2011. ISBN 978-1- 4419-5906-5, doi: 10.1007/978- 54444499006-555555 UtL/httpp:///a...or//00.. 978 1 100-4449-596---1
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. Visualizing higher-layer features of a deep network. 2009.
Gabriel Erion, Joseph D Janizek, Pascal Sturmfels, Scott M Lundberg, and Su-In Lee. Improving performance deep learning models with axiomatic attribution priors and expected gradients. Nature machine intelligence, 3(7):620-631, 2021.
European Commission.
Laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts. CNECT, Apr
2021. URL proposal regulation laying down hesi--harmooo harmonised ceseedduulless rules- calles-artifiiial intel gence
europs htt.://cccoo........ooaaaaaaaaaa ew/en/library/
Thomas Fel, Rémi Cadène, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas
Serre. Look at the variance! efficient black-box explanations with sobol-based sensitivity analysis.
Advances in neural information processing systems, 34:26005-26014 2021.
Thomas Fel, Lucas Hervier, David Vigouroux, Antonin Poche, Justin Plakoo, Remi Cadene, Mathieu Chalvidal, Julien Colin, Thibaut Boissin, Louis Bethune, Agustin Picard, Claire Nicodeme, Laurent Gardes, Gregory Flandin, and Thomas Serre. Xplique: deep learning explainability toolbox. Workshop Explainable Artificial Intelligence for Computer Vision (CVPR), 2022a.
Thomas Fel, David Vigouroux, Rémi Cadène, and Thomas Serre. How good is your explanation? algorithmic stability measures to assess the quality of explanations for deep neural networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 720-730, 2022b.
Jiahai Feng and Jacob Steinhardt. How do language models bind entities in context?, 2024. URL attp//1a./31.111..
https: //axiiv/
Eric J. Friedman. Paths and consistency in additive cost sharing. Int. J. Game Theory, 32(4):001-518 aug 2004. ISSN 0020-7276.
doi: 10.
URL
/007/500822040011
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. In AAAI, volume 33, pp. 3681-3688, 2019a.
Amirata Ghorbani, James Wexler. James Y Zou, and Been Kim. Towards automatic concept-based explanations. H. Wallach, Larochelle, A. Beygelzimer. F. d'Alché Buc, Fox, and R. Gamett (eds.), NeurlPS, volume 32, 2019h. URL http:httss::pprcceeeeeeec
nwwrips..cccmmm cc/ppee pdf
Fi es httee//paper/2119//// /ttp://p./AA.//////////aaa....32
Antonio Ginart, Melody Guan, Gregory Valiant, and James Zou. Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems, 32, 2019.
13

---

Published as a conference paper at ICLR 2025
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 9304-93122 2020.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 2014.
Google.
Introducing paligemma,
gemma 2, and an upgraded responsible ai May 2024 URL oopps////eeooooorr..oooooooooooo wow.leooo..oooooo.oooooooooooo com/ com/en/
toolkit.
Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning, pp. 3832-3842, 2020.
Michael Hanna, Ollie Liu, and Alexandre Variengien. How does gpt-2 compute greater-than?: Interpreting mathematical abilities in pre-trained language model, 2023. URL https://
arxiv. wwxivv.rrg/aae abwp:///b//33.....o
Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms, 2024. URL
2403. 17806
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Stefan Heimersheim and Neel Nanda. How to use and interpret activation patching, 2024. URL https://arxil: otgg abs/ 2404.4404111011 15255
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. In International Conference on Learning Representations, 2019.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretability methods deep neural networks. NeurlPS, 32, 2019.
Dominik Janzing, Lenon Minorics, and Patrick Bloebaum. Feature relevance quantification in explainable ai: A causal problem. In Proceedings of the Twenty Third International Conference Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 2907-2916. PMLR, 26-28 Aug 2020. URL /proceed ngs .m....m.. mm
C8/
janzing html
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.
Mistral 7b. arXiv preprint arXiv:2310.06825 2023.
Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: An adaptive path method for removing noise. In CVPR, pp. 5050-5058, 2021.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pp. 2668-260860 2677. PMLR, 2018.
Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9012-9020 2019.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Hal Daumé and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348. PMLR, 13-18 Jul 2020. URL https: http::::://ppcceed..o. m wttpppssss/l11kkhhhapa html
14

---

Published as a conference paper at ICLR 2025
Ramaravind Kommiya Mothilal, Divyat Mahajan, Chenhao Tan, and Amit Sharma. Towards unifying feature attribution and counterfactual explanations: Different means to the same end. In Proceedings of the 2021 AAAI/ACM Conference Al, Ethics. and Society, AIES '21, pp. 652-663. Association for Computing Machinery, 2021. ISBN 9781450384735 doi: 10.1 145/34661702.344233
1....11455
3462597
Haotian Liu, Chunyuan Li. Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NewrIPS, 2023. htaatpp://ppnneeieew.nett/eeeeee
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1196-119866 2022.
Scott M. Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees. Nature machine intelligence, 2. doi: 1031038/ s42256-019-0138--9 URL https::/ppsatt .sff 1ttss.ooo///bl11/1111779999
Samuel Marks, Can Rager, Eric J. Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models, 2024. URL https:
abs 19647
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023. URL httthhhts///aree
httgg/bb/22223 05262
Meta. Introducing meta Ilama 3: The most capable openly available 11m date. Apr 2024. URL https::/aa..mete wtwp:tta.coooaaaooooooam wwaea//log//// meta- 1lama-33 3/
Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojciech Samek, and Klaus-Robert Müller. Liser Relevance PoiiceePPPooco Propagation:: An Overview, pp. 193-209. Springer International URL
Publishing, Cham, 2019. ISBN 978-3-030-22954-6 doi: 10.
https: org 100100079910
61
Sabine Muzellec, Léo Andéol, Thomas Fel, Rufin VanRullen, and Thomas Serre. Gradient strikes back: How filtering out high frequencies improves explanations. arXiv preprint arXiv:2307.09591, 2023.
Neel Nanda. Attribution patching: Activation patching at industrial scale, 2023.
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoe Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:220.002299 2022.
Paul Novello, Thomas Fel, and David Vigouroux. Making sense of dependence: Efficient black-box explanations using dependence measure. Advances in Neural Information Processing Systems, 35: 4344-4357 2022.
Deng Pan, Xin Li, and Dongxiao Zhu. Explaining deep neural network models with adversarial gradient integration. IJCAI, pp. 2876-2883. International Joint Conferences on Artificial Intelligence Organization, 8 2021. doi: 10. 00.24963/jcca..2221/366 URL https::/doi...rr. 10.24962 00.24963/15ca1.22221
Main Track.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. Curran Associates, Inc., 2019. URL http: httttt:://aaapree www.eeesssseriiiisc.cc http-pooooooa-aaaaappaaaaeeeeeee higeeehhhhhhhhc wtwp:p/eeoooooocee-----e-aaaaaaa Library.
2015-pytorcss an wmperative-styyiaeaea
orroo deep-
pdd
15

---

Published as a conference paper at ICLR 2025
Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: randomized input sampling for explanation of black-box models. In BMVC, pp. 151. BMVA Press, 2018. URL http::/bb/cc2011. org/
pontennts/papers/IGGG
pdf
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '1, pp. 1135-1144, New York, NY, USA, 2016.
Association for Computing Machinery, ISBN 978 1450342322. doi: 10.1145/999972222397777
URL httptttsttt.oooooooeeooo 1ttp://0.111552999399133 httpppppw//aaa..c.......///////3
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi Velez. Right for the right reasons: Training differentiable models by constraining their explanations. Proceedings of the Twenty- Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 2662-26770 26700 2017. doi: 1119663/jcai22017/317 URL fatttss::/ /wiii 2ttp//00.2293/11jaa.a. 2017/371
D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, and Dan Dennison. Hidden technical debt in machine learning systems. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https: htittp:://prooeeddinng pww.eedingegggeooeeeoe....s.. neurips Paper. httee//00tt./tlleefffffffaaaata2
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In CVPR, pp. 618-626, 2017.
Harshay Shah, Prateek Jain, and Praneeth Netrapalli. Do input gradients highlight discriminative features? Advances in Neural Information Processing Systems, 34:2046-222 2059, 2021.
Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just black box: Learning important features through propagating activation differences. arXiv preprint 1rXiv:1605.0/713, 2016.
K Simonyan and A Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations (ICLR 2015), 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint urXiv: 1312.6044 2013.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint
2017.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6800 2014.
Suraj Srinivas and Francois Fleuret. Full-gradient representation for neural network visualization. 2019.
Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis, 2023. URL https? //arsáv.croo/as/2200.... 15054
Pascal Sturmfels, Scott Lundherg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 2020. doi: //239912/iistill.00002....
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Gradients of counterfactuals, 2016.
16

---

Published as a conference paper at ICLR 2025
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
Aaquib Syed, Can Rager, and Arthur Conmy. Attribution patching outperforms automated circuit discovery. arXiv preprint arXiv:2310. 2023.
Mingxing Tan and Quoe Le. Efficientnetv2: Smaller models and faster training. In International conference on machine learning, pp. 10096-10106, PMLR, 2021.
Anvith Thudi, Gabriel Deza, Varun Chandrasckaran, and Nicolas Papernot Unrolling sgd: Understanding factors influencing machine unlearning, 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pp. 303-319. IEEE, 2022.
Jesse Vig. Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Simas Sakenis, Jason Huang. Yaron Singer, and Stuart Shieber. Causal mediation analysis for interpreting neural nlp: The case of gender bias, 2020. URL https://aaxii:t oww. abs/2004 12265
Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: circuit for indirect object identification in gpt-2 small, 2022. URL
https://arxiv wtt. .
White House OSTP. Blueprint for an bill of rights: Making automated systems work for the american people. Oct 2022. URL http:: www www.whhhssssee www.:heeoooueeoooooooooooo.. wpwp://-//oteett/////// up meads/2022/100 http://lleprinttmm for- AT----an------1---- -ightl-----tggtss....oo pdd
H. Xiong, L. Huang, M. Yu, L. Liu, F. Zhu, and L. Shao. On the number of linear regions of convolutional neural networks, 2020. Uttpstts://axxx wttp://ww.rr//b////00/.. 00978
Shawn Xu, Subhashini Venugopalan, and Mukund Sundararajan. Attribution in scale and space. In CVPR, June 2020.
Chih-Kuan Yeh, Cheng- Yu Hsieh, Arun Suggala, David Inouye, and Pradeep K Ravikumar. On the (in) fidelity and sensitivity of explanations. NeurlPS, 32, 2019.
Shih-Kuan Ych, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar, On completeness-awart concept-based explanations deep neural networks. Advances in neural information processing systems, 33:20554-20565, 2020.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 12, 2014, Proceedings, Part I 13, pp. 818-833. Springer, 2014.
Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. IJCV, 126(10): 10:-11084111 201 8a.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018b.
Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
17

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 VERIFYING ATTRIBUTION SPECIFICITY
To verify that UNI computes explanations that are specific to each mask-modelinnpu triplet, we compare its saliency attributions across models for the same image input. Visually, we observe in Figure 8 that attributions differ significantly and even reflect the inductive biases of respective models (e.g. grid-like artefacts are present in ViT attributions whereas smoother attributions are computed for convolutional architectures). We further present numerical results in Table LPIPS Zhang et al. 201 scores reflect the dissimilartty/distance hetween the original image and the unlearned baseline; the percentage change in confidence scores reflect how the unlearned baseline effectively reduces predictive confidence (relative to the original input).
Table 7: UNI computes different baselines for network architectures with different inductive biases on the same input, as seen from the drop in model confidence (A% % Confidence) and image- baseline similarity scores (L.IISSag. aPlPatex).
Image
RIS
BIPZs
CvnXi
VGG16
VIT
SwinT
% Confidence LPIPS
LPIP alex
ResNet-18
-82.3%
.021 = 1.025
.003 000
Eff-v2-s
-76.9%
.025 1 1.024
.004 2.011
ConvNeXt-T
-95.1%
.018 2.016 .002 003
VGG-16-bn
-71.6%
.017 1.020
.001 #.002
ViT-B-16 -69.7%
.014 4.015
.004 0.007
SwinT
-84.6%
.014 2:015
.002 002
Figure 8: UNI computes different attributions
explain the predictions of cach model.
A.2 PRELIMINARY RESULTS ON NLP
Table 8: Faithifulness: L2 -i-Dattee from activation patching to attribution patching results on the residual stream (averaged over 100 samples).
UNI
Random
Pb-hi
3.12
6.64
GPT2-medium
15.26
35.00
Llama-3.2-IB 5.25
10.17
We extend the testing of our method to the case of Natural Language Processing (NLP). We choose to test the application of UNI in the general framework of generative models (which includes classification models), and attribution of not only inputs but more generally activations. Activation patching (Heimersheim & Nanda 2024h is one of the most widely used technique in Mechanistic Interpretability. and more generally to study the properties of LLMs' internals (Vig et al. 2020 Meng et al. 2023 Wang al.) 2022 Feng & Steinhardt 2024 [unninnham et al. (2023) Stolfo et al. 2023
Hanna et al. (023). This attribution method consists in analyzing a model's 's output variation after replacing its internal activations, following the equation:
A F(xle = e(x')) - F(x)
4
where e denotes one activation in the model. z' a chosen baseline, and F a function of the model's output (usually the logit value of the maximum probability token of the model run 2:).
18

---

Published as a conference paper at ICLR 2025
Pytra Randon Activsoo
Pynta
Layer
Liger
Pytre Randern Atribution
Pychis 1h UUI Attibution
Pardon
12%
Figure 9: Visual comparison of attribution results for Activation vs. Attribution patching, with UNI versus Random baselines, on Pythia-Ib-vO. 1b-v0. Each cell shows the logit variation obtained by patching at a specific token and layer the residual stream of our baseline with the original activation.
Unfortunately activation patching is computationally costly, especially for purposes such as circuit discovery (Conmy et al. (2023). One of the main alternatives that solves the scalability problem
attribution patching (Nanda
2023 Syed et al. 2023). which computes a first order Taylor
approximation of Equation|4
A ATTR (xT) = (e(x) V
(5)
for which the attributions for all of the activations can be computed at the same time (no patching of one single activation is performed). Despite its scalability, attribution patching suffers from lack of faithfulness for causal interventions, mainly due to saturation and lack of linearity of the studied dependencies.
The analogy with integrated gradients seems quite striking, and indeed two recent works (to our knowledge) have tried to investigate the use of IG for more faithful attribution patching. While Marks et al. (2024) applies very computationally complex version of such method to small models, Hanna et al. (2024) proves the potential of IG-bdsed attribution patching, while showing it still gets outperformed by activation patching.
We here provide a new UNI-based attribution method algorithm outputting faithful attributions while maintaining the scalability advantage of attribution patching. Mainly, we apply Algorithm T to compute a baseline x' that is then used to compute Equation 1
UNT-
(z) (e(x) e(x'))" TV.F (UNI(())
where we take x to be the embedding of the input, to allow for continuous operations on it. Considering the known high faithfulness of activation patching (Hanna al. 2024), we approximate faithufiness
19

---

Published as a conference paper at ICLR 2025
of attributions computed from baseline, the Ly-distance between these attributions and the activation patching ones. The dataset used is a subset of 100 counterfactual prompts taken from Meng al. (2023), and three different models tested: Pythia- 1b-vi (Bıderman et al. 2023), GPT2-medium (Radford et al. 2019) and Llama-3.2-18 (Dubey et al. 2024). The results can be seen in T and visuals in Figure 9 Note that fine-tuning of UNT hyperparameters has been done, so that we expect even better results when adapting for each models. Unsurprisingly. decoding the baselines by shortest distance the rows of the embedding matrix yields the same input, and direct decoding of the perturbation & doesn't I provide any interesting information.
A.3 ADDITIONAL VISUALISATIONS
We supplement the main text with visualisations of the UNI baseline, attributions and path features (properties, stability and robustness). We additionally include figures elucidating the colour, texture and frequency biases post-hoc imposed by path attribution methods. From Figure [0] we observe the stability of UNI path features: attributions
be reliably and efficiently computed with Riemann approximation. In Figures 11 12 and 13 we present visualisations on ImageNet-C, highlighting how static choices of baselines may bias the path-attribution procedure, leading to null noisy explanations. UNI does not impose additional post-hoc assumptions that are alien to the model's decision function. Furthermore, we present qualitative comparisons of attribution results of pre- trained models on the ImageNet-Ik 1K test set, in Figures 14
18 and 19 UNI I attributions are visibly better localised and more semantically meaningtul. Finally, we visualise the consistent, geodesic paths of monotonically increasing output confidence, discovered by UNI. As seen from
1. 16
Figures 20 21 22 23 24 and 25 while other path attribution methods might encounter extrema and turning points along the interpolation path from baseline to input, UNI's 's path features are monotonic and preserve the crucial completeness property on which the path attribution framework depends.
Baseline
step
steps
30
Baseline
step
sleps
30 steps
IG
Image
Image
(ours)
(ours)
Baseline
stop
15 steps
30 steps
Baseline
step
15 stops
30 steps
Image
Image
UNI (ours)
UNI (ours)
Figure 10: Comparison of attribution maps computed by Integrated Gradients and UNI, for a pre- trained ResNet-18 on the ImageNet-IK test set. UNI occludes and unlearns predictive input features; reliably localises predictive image regions; can be efficiently computed with only Riemann step.
20

---

Published as a conference paper at ICLR 2025
Mup
Baseline
Brightening
Suturation
Baseline
Map
UNI
UNI
BbokiG
BlckiG
UNI
UNT
BlookiG
Blocka
UNI
UNI
BlookGG
BiackiG
Figure 11: Colour bias: When image's brightness saturation is altered, IG with a black baseline fails to identify dark features, such as the wings of the butterfly (R1) or black jacket (R3).
Map
Baseline
Gaussian Blur
Defocus Blur
Baseline
Map
UNI
UN
Bluri
UNI
UNI
Cola
BluiG
UNE
UNI
Bluit
BlurKG
Figure 12: Texture bias: Using blurred baseline for IG leads to smoothness assumption in image texture, which leads to missingness attribution when the input also gaussian or defocus blurred.
Map
Baseline
Gaussian noise
Shot noise
Baseline
Map
UNI
UNI
Nooelt
Nossett
LNI
UNI
Nowelli
Noteat
UNI
UNI
NooelG
Figure 13: Frequency bias: A gaussian noised baseline for IG renders it vulnerable to high frgh-frryeency corruptions. Adding gaussian or shot noise to the image yields unmeaningful, noisy attributions.
21

---

Published as a conference paper at ICLR 2025
@SSW
Heeptan
GokkedBachhpp
Dkppt.11
USI tuars
AGE
1G
Madic
ENI lours)
AGI
86
Bhrfic
Guldedii
Ni (vsta)
Deept.t
UN
AGE
Strtic
ENE
Bhrti::
Deggha
Culledii
Duppii
Gelkddic
U/d (uav))
(ser))
(sste)
Deppl.it
Gabkolic
UNI ionul
AGI
16
Blvito
AGI
5G
Btvi6
Gilletii
1668
Dept.f
Cakkdli:
IG/sso
Figure 14: Comparing attributions (ResNet-18): UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
22

---

Published as a conference paper at ICLR 2025
Deepl.21
UNI jouns)
AGI
Deepl.2t
Gade
Goleddii
(aale)
UNB laery)
AGI
Merii
USI juur))
AGI
8G
Btatic
Galletii
noter
DecyULut
Gakk
UNI luns)
MGI
Marto
UNI (uins
4G1
8G
Hario
Peepi it
Gasbdiic
nele)
Doeghath
Gaskedii
10 note
Static
UNI toars!
AGI
DG
Marii
Deguiit
GaloediG
DicpLLtt
fembe)
Figure 15: Comparing attributions (Efficien/Net UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
23

---

Published as a conference paper at ICLR 2025
Deepln
(anbe)
Despl.tt
GottedBadddrp
UNI (aar)
AGE
1G
Mado
UNE Lourt)
AGI
3G
Bhrfii
Heeglah
Galdedii
(usle)
Deepl.t
Getbelic
Be (esur
UNI
AGE
Marfi
ENE
AGI
Bletti
Dowslah
GuidelDG
Gebbelfi
UTM (swr)
Mavi0
(sur))
Chadeett)
Deryp
Geobelti
UNI momul
AGE
RG
Mavit
USI (omm)
AGI
50
Ravi0
Deezlih
Gleteiii
Despl.f
Gaskettt
Figure 16: Comparing attributions (Com:NeXi-Tiny)) UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
24

---

Published as a conference paper at ICLR 2025
Derp1at
poard
(Ni
AGE
DigpLat
GobkodBludPrpp
fiele)
Deglin
Coldedllehhhp
(wsur)
UM (var)
AG1
Blerid
AGI
16
Bhrlc
Ivep1A8
Galkett:
IG Fmiber
UNI isard
AGI
30
ENT
AGE
16
Bhr
Deeplah
Galledtc
IG nsder
Deeplan
10 (wawc)
Elartt
ENtrcaoi
AGI
1D
Dhriii
Deeplit
GelkefBuddPrpp
Galdidii
Dieplin
Guldiddi
Figure 17: Comparing attributions (VGG-16-bil UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
25

---

Published as a conference paper at ICLR 2025
Deeplat
row
Guand
Elard
(NT
AGE
Bhrtc
Doepl.2
GobbofBuukPrup
(wile)
Deepl.i
Goddedllehhhpp
Chiladti
AG1
3G
AGI
16
Bhrti
IveptA8
Galk-eti:
IG Fmibe
Gullidii
IG Innoo)
UNI (surd
AGI
Barld
ENi
AGE
16
Bha
Dveplah
GeshelbukPrr
IG noter
Dieplai
10 Iwsic)
EMtrcari
AGI
16
Dhrii
Degplit
GelkeBBuddPrp
Galdedic
DispLin
Guldeddi
Figure 18: Comparing attributions (ViT-B_JO 16): UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
26

---

Published as a conference paper at ICLR 2025
UNI (oars)
AGI
DeppL.
(embel
GabAedic
UNI lier!
AGI
Muric
UNI puan)
AGI
9G
Btrii
Galedlic
mobel
DeeyL.4
Gabledti
TO (mibi
UNI leand
AGI
Martd
UNI Juans
4G1
8G
Barh0
Deepl.2h
GaslediG
To roiee)
Deey1
Gasledto
IG Sadee
Murii
UNI
AGI
BG
BtarkG
Drgshtt
GelkeBuudPPa
Galaddic
recbe)
DiepLut
Grlbeedt
Figure 19: Comparing attributions Seein-Tranffrmmr UNI attributions demonstrate higher saliency, fidelity and faithfulness relative to conventional baselines on the ImageNet test set.
27

---

Published as a conference paper at ICLR 2025
Image
FRRRS
Path
Image
Diec Cola
COLA
Diee Col
Path
Image
SUBRRTY
UKT Maccy
Path
Image
Path
Figure 20: Comparing paths (ResNer-18): UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
28

---

Published as a conference paper at ICLR 2025
Image
Fobbiityy
Stamily
Path
Image
Path
Image
IN 15 StAnl
1h/o Staciitt
LMM 20 SaAaeby
Path
Image
Path
Figure 21: Comparing paths (EfficientNett-2-smmlll/ UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
29

---

Published as a conference paper at ICLR 2025
Image
Sabiity
Path
Image
Path
Image
LIST NEN0EE
Macciyy
UNE Ma.
Path
Image
Path
Figure 22: Comparing paths (ComNNeXt-1n -Tiy)):; UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
30

---

Published as a conference paper at ICLR 2025
Image
Sabaity
Path
Image
Path
Image
URL. BiMaeily
Path
Image
Path
Sanerh
Figure 23:
Comparing paths (VGG-16-bi) UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
31

---

Published as a conference paper at ICLR 2025
Image
IRI B MaMeeil
Path
Image
Path
Image
U. Swbiity
IN VM. 15 StANd
UNTOE Seacily
UM 15 SAAEE
Path
Image
Path
Figure 24: Comparing paths (VIT-B_166 UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
32

---

Published as a conference paper at ICLR 2025
Image
Subbety
45 Suriiyy
TO Scubeity
UNIVS Star
UBIV 15 Sovely
Path
Image
Syailly
Path
POLARITY
Image
UNSCENTER
PROE
USI
Path
Image
UM 55 15 SY
UNLOS 500 Iy
URIVS M000
1MUSS 5004
Path
Figure 25: Comparing paths (Swin- Sovin-Transformee---nyyy UNI discovers geodesic paths of monotonically increasing output confidence, preserving the completeness property required for robust attributions.
33


Reviews:
[{"review_id": "hGjqmTgx7C", "reviewer": "Reviewer_nTJj", "summary": "This paper suggests a method for creating references for IG and other backpropagation-based explanation models. The reference is based on an untrained model with perturbations applied to the original image with respect to the model. The intuition, insights, and experimental results are presented as evidence.", "questions": "1. The images shown in the figure may suggest that the white color appears relatively weaker compared to the black color. Is there any bias towards the white color? If not, why is the gray part of the swimsuit not highlighted? In Figure 2, the white swan also shows a similar tendency. Do you have any qualitative measures for this?\n\n2. Why do you use KL divergence? Have you tried different methods as well?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Authors well-summarize the previous search, and clearly explain the required properties needed for the reference. The intution is clearly written, and well-evidenced by experimental results.", "weaknesses": "1. Even though they effectively present their intuition, there is no theoretical justification for their method. In particular, the pseudo-code is not explained enough. Additionally, it is unclear why the authors chose to implement it in the way they described in the code. The method itself needs further elaboration.\n\n2. The suggested method is limited to explainable methods that require a reference, which restricts the broader applicability of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cTKTJ7thLk", "reviewer": "Reviewer_G3s3", "summary": "The authors identify that current baseline approaches in path attribution methods (like Integrated Gradients) inject harmful assumptions about color, texture, and frequency that can lead to biased and fragile attributions. UNI proposes computing baselines by perturbing inputs in an \"unlearning\" direction, to erase salient features while maintaining model-specific properties. The method demonstrates improved faithfulness, robustness, and stability compared to existing approaches across multiple model architectures and datasets.", "questions": "No particular questions", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper offers a well-structured analysis of an important problem in neural network interpretability. The authors identify a specific, previously overlooked issue: how static baseline approaches in attribution methods can introduce unintended biases in three distinct categories - color (shown through experiments with brightness/saturation changes), texture (demonstrated via gaussian/defocus blur tests), and frequency (validated through gaussian/shot noise experiments). \n\nThis observation is methodically supported through experiments across a comprehensive set of modern architectures, including both CNN-based models and transformer-based models. Their UNI method shows measurable improvements in three key areas: faithfulness (with MuFidelity scores showing consistent improvements across architectures), robustness (demonstrated through FGSM adversarial attacks, achieving higher Spearman correlation coefficients), and stability (shown through detailed Riemann approximation experiments that work even with just one step). \n\nThe theoretical foundation includes careful mathematical analysis where they provide explicit bounds on Riemann approximation error and prove how their approach maintains monotonicity and completeness properties. The extensive experimental results in the appendix, spanning all (20+!) detailed figures, provide clear visual evidence of how their unlearning-based approach improves upon traditional static baselines across different types of images and corruptions.\n\nWell done!", "weaknesses": "Largely this is a well written paper, here are a few potential avenues for improvement :\n- Would it be possible to make stronger theoretical guarantees about the optimality of the unlearned baseline?\n- Image classification baselines are pretty standard, I wonder how these baselines will look like in other domains such as NLP and audio.\n- There could be some more discussion around hyperparameters, an often overlooked aspect in explainability literature\n- It is mentioned that other techniques are inefficient, a detailed analysis of computational overhead might solidify contributions more strongly in that aspect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VB3W1ldbUv", "reviewer": "Reviewer_Wrfy", "summary": "The authors introduces a novel approach to enhance gradient-based post-hoc attributions.\nThe authors argue that static baselines (e.g., black images, blurred noise) impose unintended biases on attribution maps, leading to fragility and unfaithful interpretations. The proposed UNI framework generates a dynamic baseline by perturbing the input in an unlearning direction of steepest ascent, theoretically optimizing for a featureless reference that aligns with the model's predictive function. \nThis adaptive baseline improves the path attribution by reducing curvature and enhancing robustness, demonstrated empirically through comparisons on benchmark datasets like IN and various baseline models.", "questions": "- Could you adapt the method to a low dimensional subspace path (eg delta is parametrized by 7x7 mask that we interpolate and then optimize?)\n- How would the proposed method adapt if applied to models beyond image classification, such as language models or multimodal networks?\n- Could the authors consider a variant of UNI that leverages multi-scale features for tasks involving more intricate hierarchical dependencies?\n\n\n**Overall, I think the paper is well-executed with thorough analysis and comprehensive results. While the contribution may be of limited impact within the broader field of explainable AI (see Major issues), I tend to give a marginal accept.**", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Innovative Baseline Approach**: The idea of using unlearning to create adaptive baselines is novel and addresses a known limitation in static baseline approaches.\n2. **Robustness and Faithfulness**: The empirical results, including MuFidelity scores and robustness to adversarial perturbations, highlight the potential of the method to produce more reliable attributions.\n3. **Comprehensive Experiments**: The paper includes evaluations on multiple models and datasets, adding credibility to the proposed approach.", "weaknesses": "However, I spotted some problems, some major (**M**) and minor (**m**):\n\n**M1.** Literature Gaps: the related work section omits significant prior research on black-box interpretability (RISE, Sobol...) and miss some really important faithfulness metrics. Also a paper from last year that could be relevant (Saliency strike back) could be discussed.\n\n**M2.** Faithfulness Metrics: while the paper discusses MuFidelity as a metric, it does not consider complementary metrics such as deletion and insertion, which could offer a broader validation of faithfulness and robustness. Including these would provide a more comprehensive evaluation of the method's performance. Also MuFidelity contain many Hyperparameter that are known to favorise certain method, could you state how did you approximate MuFidelity (number of sample)  and also the value of $|S|$ ?\n\n**M3.** Theoretical Justification: the paper lacks a theoretical discussion on how UNI would perform under linear cases or simple functions. Providing proofs or theoretical insights for these cases would enhance the paper's depth and foundational strength.\n\n**M4.** Broader Impact on Understanding Neural Networks: The practical implications of this method for gaining new insights into neural networks remain unclear. How much new knowledge / what have learn with this method that we haven't with other attribution methods ?\n\nNow for the minor problem:\n\n**m1.** Limited Baseline Comparisons: the paper mainly contrasts UNI with IG and a few other attribution methods. Expanding the baseline comparison to include more recent methods (including black-box one such as RISE or HSIC that are know to have good results on faitfhulness metrics) and varied methods would strengthen the experimental section.\n\n**m2.** Minor Typos: I havent found a lot of typos, just line 468: \"simultatenously\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QHAG5FMHGM", "reviewer": "Reviewer_rhWZ", "summary": "This paper studies the specific problem of baseline selection in feature attribution. The authors argue that the optimal baseline choice depends on the task-model-input combination and propose an unlearning-based approach to improve baseline selection. By combining the baseline selection method with a well-known path method — IG, the resulting attribution method UNI demonstrates improved performance in various aspects according to the experimental results.", "questions": "1. Could the authors elaborate on why activation matching ensures the removal of salient features? Furthermore, given an activation vector $F(x;\\hat{\\theta})$, there may be multiple matches in the decision space within the budget $\\varepsilon$. Is there any guarantee of the convergence for this matching, or is it based solely on empirical observations?\n2. The goal of the unlearning is to derive $F(x;\\hat{\\theta})$ for matching. What advantage does it offer over directly adopting a non-informative activation, say $(\\frac{1}{d_Y},\\ldots,\\frac{1}{d_Y})$? Could the latter be even more powerful as it removes features that are informative in general?\n3. Could the authors clarify the meaning of $F_c(x_i)$ at lines 297-298 mean? It is unclear how the model processes a single feature $x_i$ instead of an instance $x$.\n4. How are the two conditions in lines 297-298 satisfied? \n    - Assuming the authors intended $F_c(x)$ rather than $F_c(x_i)$, the monotonicity of the class confidence does not necessarily imply a consistent sign of the derivatives w.r.t. each feature $x_i$. This indicates that the sign of $\\nabla_{\\tilde{x}_i}F_c(\\tilde{x})$ may vary along the path, which contradicts the claim of the first condition.\n    - Solving the gradient saturation problem is one motivation for IG. However, if a feature $x_i$ has a saturated gradient at $x$, i.e. $\\nabla_{x_i}F_c(x)\\approx0$, the second condition becomes problematic as it requires the derivate of $x_i$ to remain 0 over the integration path, leaving gradient saturation unsolved.\n5. Could the authors clarify the relationship between completeness and path monotonicity? Completeness, as given by the gradient theorem, should be path-independent.\n6. Many attributions provided by UNI closely align with the details of the input images. To demonstrate that the explanations are not simply grayscale reproductions of the inputs, could the authors provide example attributions of different models on the same inputs to show whether they distinguish from each other (under the assumption that different models behave differently)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is clearly motivated and easy to follow.\n2. The authors collect a set of desired properties for the integration path, which could benefit following work on baseline selection.\n3. The experiments are carefully designed, thoroughly evaluating the proposed method from different perspectives that support the authors’ claims.", "weaknesses": "1. The description of the proposed method is somewhat oversimplified, raising concerns about the overall depth of the work. It would benefit from further elaboration and discussions on each step in Algorithm 1. \n2. The reasoning behind why activation-matching ensures the removal of all salient features is unclear.\n3. Further clarification on some technical details about monotonicity would be beneficial. Please refer to questions 3, 4, and 5 for specific points.\n4. There are some minor errors and formatting issues, for example: \n    - Different formats of $\\varepsilon$ in Algorithm 1.\n    - Citation wrapped by parenthesis as part of a sentence on line 261; repeated parentheses on line 269.\n    - Line 349 refers to \"Figure 3\", which appears to be a mislabeling of \"Table 3\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)