Rethinking Visual Counterfactual Explanations Through Region Constraint
Authors: Bartlomiej Sobieski
Jakub Grzywaczewski
Bartłomiej Sadlej
Matthew Tivnan
Przemyslaw Biecek
Venue: ICLR 2025 Poster (2025)
Keywords: visual
counterfactual
explanations
diffusion
generative
explainable
Primary area: interpretability and explainable AI
TL;DR: We propose a framework for visual counterfactual explanations with region constraint and develop an algorithm based on Image-to-Image Schrödinger Bridge that sets new state-of-the-art.

Abstract: Visual counterfactual explanations (VCEs) have recently gained immense popularity as a tool for clarifying the decision-making process of image classifiers. This trend is largely motivated by what these explanations promise to deliver -- indicate semantically meaningful factors that change the classifier's decision. However, we argue that current state-of-the-art approaches lack a crucial component -- the region constraint -- whose absence prevents from drawing explicit conclusions, and may even lead to faulty reasoning due to phenomenons like confirmation bias. To address the issue of previous methods, which modify images in a very entangled and widely dispersed manner, we propose region-constrained VCEs (RVCEs), which assume that only a predefined image region can be modified to influence the model's prediction. To effectively sample from this subclass of VCEs, we propose Region-Constrained Counterfactual Schrödinger Bridge (RCSB), an adaptation of a tractable subclass of Schrödinger Bridges to the problem of conditional inpainting, where the conditioning signal originates from the classifier of interest. In addition to setting a new state-of-the-art by a large margin, we extend RCSB to allow for exact counterfactual reasoning, where the predefined region contains only the factor of interest, and incorporating the user to actively interact with the RVCE by predefining the regions manually.

Published as a conference paper at ICLR 2025
RETHINKING VISUAL COUNTERFACTUAL
EXPLANATIONS THROUGH REGION CONSTRAINT
Bartlomiej Sobieski
Jakub Grzywaczewski
University of Warsaw s.sobieski @uw.
Warsaw University of Technology jakub. wtt..gggwaaaess...ssssesoeo
Bartlomiej Sadlej
Matthew Tivnan
University of Warsaw b. s.saleeeestudee uw. edu.pl
Harvard Medical School mti mtivnan@ngh. harvard. edu
Przemyslaw Biecek
University of Warsaw, Warsaw University of Technology przemyslaw, www.bbieeeeeeee......
ABSTRACT
Visual counterfactual explanations (VCEs) have recently gained immense popu- larity as tool for clarifying the trend is largely motivated by what these explanations promise to deliver - indicate semantically meaningful factors that change the classifier's 's decision. However, we argue that current state-of-ther approaches lack crucial component the re- gion constraint whose absence prevents from drawing explicit conclusions, and may even lead faulty reasoning due to phenomenons like confirmation bias. To address the issue of previous methods, which modify images in a very entangled and widely dispersed manner, we propose which assume that only a predefined image region can be modified to influence the model's prediction. To effectively sample from this subclass of VCEs, we pro- pose Region-Constrainee Constrained Counterfactual Schrödinger Bridges (RCSB), an adap- tation of tractable subclass of Schrödinger Bridges to the problem of conditional inpainting, where the conditioning signal originates from the classifier of interest. In addition to setting a new state-of-the-arr by a large margin, we extend RCSB allow for exact counterfactual reasoning, where the predefined region contains only the factor of interest, and incorporating the user to actively interact with the RVCE by predefining the regions manually.
process of image classifiers. This
constraindd VCEs (RVCEs),
INTRODUCTION
Visual counterfactual explanations (VCEs) aim explaining the decision melssion-making process of an image classifier by modifying the input image in semantically meaningful and minimal way so that decision changes. Over time, they have become an independent research direction with the latest methods presenting impressive and visually appealing results. Nevertheless, in this work show that they possess a fundamental flaw at a conceptual level the lack region constraint and its proper utilization.
Consider the image x" in Fig. I which the classifier f correctly predicts to be a jay. In essence, VCEs focus semantically editing x* that the prediction of changes to some target class bulbul in this case hence providing an answer to a specific what-if ques- tion, through which the model's reasoning is explained. Consider x", denoted as XyCE, obtained with a recent state-of-the-art (SOTA) method. While Xyce successful changing the prediction of f and can be considered both realistic and mantically close to x", answering why f now predicts it as a bulbul is close to impossible.
example VCE for
"Corresoonding author

---

Published as a conference paper at ICLR 2025
The algorithm simultaneously modifies the bird's head and feathers, changes the texture of the branch and even modifies the copy- right caption. The entanglement and disper- sion of introduced changes hence leaves the question unanswered. We argue that to cir- cumvent these fundamental difficulties, VCEs should be synthesized with hard constraint on the region, where the changes are allowed to appear, while leaving the rest of the image unchanged. For example, consider the image XR with regions of the bird's head (R,)) and body (Rg) overlayed. Constraining the VCEs to introduce changes only to predetermined re- gions leads to two distinct explanations, XR, and XR,: of why the decision changes to bul- bul. By isolating the modified factors, the ex- planatory process greatly simplifies one can now state with certainty that f's new predic- tion is based either on the modified feathers (XRa) or the changed characteristics of its head (XR;). Region-constrained VCEs (RVCEs) al- low, therefore, to reason about the model's thought process a causal and principled man- ner, mitigating the potential confirmation bias and clarifying the explanatory process.
Jüny /[bilball swvoe) 0.97 Absulute
VCB
Xyos
[balbol xx.) 0.00
RVCR
Xn.
f[ballol xxx) 0.99
Xila
Figure I: Previous methods create VCEs with un- constrained changes, making virtually impossi- ble to understand the decision-making process of a model. We propose region-constrainet VCEs, establishing new paradigm for comprehensible and actionable explanatory process.
By putting RVCEs in the spotlight, our work establishes frontiers in the field of VCE genera- tion. First, we define the objective of finding RVCEs as solving a conditional inpainting task. By building top of the Image-to-Image Schrödinger Bridge (ISSB, Liu et al. (2023a)) approach and adapting to the classifier guidance scheme, we develop an efficient algorithm which synthesizes RVCES with extreme realism, sparsity and closeness to the original image. Specifically,
set new quantitative state-of-the-art (SOTA) on ImageNet (Deng et al.) 2009) with up to times better FID and times better sFID (realism), up 2 times higher COUT (sparsity), and match or exceed 3 (similarity) and Flip Rate (efficiency) achieved by previous methods. Through large-scale experiments, we demonstrate that, besides fully automated way of synthesizing meaningful and highly interpretable RVCEs, our approach, Regionn-oosttrained Counterfactual Schrödinger Bridge (RCSB), allows infer causally about the model's change in prediction and enables the user to actively interact with the explanatory process by manually defining the region of interest. More- over, our results highlight the importance of RVCEs in future research, indicating potential pitfalls of unconstrained methods that could lead drawing misleading conclusions.
2 BACKGROUND & RELATED WORK
In this section, we introduce the necessary background knowledge connected with score-based gen- erative models (SGMs) and I'S which forms the foundation of our method. We then present an overview of recent methods for VCE generation based on SGMs. For an extended literature review and detailed description of the theoretical basis, please refer the Appendix.
SGM. Following the work of Song et al. (202)), SGMs can be constructed through the framework of stochastic differential equations (SDEs), where samples from complex distribution po (e.g., natural images) are mapped to Gaussian distribution P1, while the model is trained to reverse this mapping. Formally, converting data noise is performed by following the forward SDE (Eq. 11a while denoising happens through the reverse SDE (Eq. (Ib) Anderson (982)))
dx; dt /B.dw.
(1a)
dx; = (x, - 8, V log p t)) + B,dw. where x, the noisy version of a clean image x E R" for some n E N at timestep and w denote the Wiener process and its reversed (in time) counterpart, F,(x,) R
(lb)
E [0,
Rd is
2

---

Published as a conference paper at ICLR 2025
the drift coefficient, 3, E R is the diffusion coefficient and Vx, log t) is the score function. An SGM Se. where 9 denotes the model's ' parameters, is trained to approximate the score, i.e, t)) Xx, log p(x,, t). During sampling, denoising begins from pure noise X1 p1 and follows some discretized version of Eq. (Tb) with the approximate score $0-
SGMs can also be adapted to conditional generation, where y represents the conditioning variable. In this case, the score Xx, log (x, t) is replaced by Vx, log p(x,; & y). which can be decomposed with Bayes' Theorem into Xx, log p(x,, y Vx, log p(x,( t) + V x, log p(y Xe: (). While Vx, log 1) can be approximated with an already trained Sa. Xx, logp(y Xx, () must be mod- eled additionally. For y representing class labels, p(y xx. can be approximated with an auxiliary iime-dependent classifier P xt,t) trained on noisy images {xL () Incorporating Ps into the sampling process termed as classifier guidance (CG), and can be strengthened (or weakened) with guidance scale through Xx, logp(x,, 1)+ + Xx, log ply x,, Therefore, class-conditional sampling SGMs amounts to additionally maximizing the likelihood (y x,, t) of the classi- fier throughout the generative process to arrive at images from the data manifold, which resemble (according to PΦ) instances of a specific class, We emphasize this fact here for further reference.
The framework of PSB ex- tends SGMs to P1 representing an arbitrary data distribution. training. FSB requires paired data,
PSB.
t 0
0.25
0.5
0.75
For
1-BB
egg. in the form of clean and par-
tially masked samples for inpainting, where it learns to infill the miss- ing parts. While SGMs can also be adapted to solve inverse prob-
SGM
lems like inpainting, I'SB maps these samples directly (see Fig. - for a comparison of their generative tra- jectories). Therefore, PSB follows the same theoretical paradigm, where sampling is achieved by discretizing Eq. (Ib) and using a score approximator sv. but the generative process begins from a corrupted (e.g., masked) image instead of pure noise. Hence, I'BB can also be adapted to conditional gen- eration in the same manner as SGMs, especially for class-conditioning with auxiliary classifier. Importantly, a special case of ISB follows an optimal transport ordinary differential equation (OT- ODE) when 3r + 0, eliminating stochasticity beyond the initial sampling step (see Appendix). We utilize the OT-ODE version of SB in our implementation.
Figure 2: Generative trajectories of IBSB and SGM. Inter- mediate images of ISB are much closer to the data mani- fold.
SGM-based VCEs. The initial approach of adapting SGMs to VCE generation, DIME (Jeanneret jet al. 2022), obtains the classifier's gradient by mapping the noised image to its clean version at each step through the reverse process. /Augustin al. (2022) (DVCE) incorporate the gradient of robust classifier and a cone projection scheme. Jeanneret et al. (2023) (ACE) decompose the VCE generation into pre-explanation construction and refinement using RePaint (Lugmayr et al. 2022).
Jeanneret et al. (2024) utilize foundation model, Stable Diffusion (SD, Rombach et al. (2022)). to
generate VCEs black-box scenario. |Faid al.] (2023) (LDCE) Modzkus al. (2024) utilize
Latent Diffusion Models (LDMs), including SD, in white-box context. WWng et al. (2024) propose
FastDiME to accelerate the generation process in a shortcut learning scenario. Also in black-box
context, Sobieski & Biecek (2024) utilize Diffusion Autoencoder (Preechakul et al. 2022) to find semantic latent directions that globally flip the classifier's decision. Augusttn ct al. (2024) also make use of SD in various contexts, including classifier disagreement and neuron activation besides VCEs. While ACE and FastDiME also assume constraints to some regions, those are always classifier- dependent. In this work, consider more general definition provided in the next section.
3 METHOD
In this section, we describe the details of our approach, beginning with the formulation of RVCEs as solutions to conditional inpainting task. Next, we motivate the use of I'SB as an effective prior for synthesizing meaningful RVCEs and follow with a series of steps that better align the gradients of a standard classifier w.r. corrupted images from its generative trajectory. We conclude with a description of the automated region extraction method, forming the basis of our algorithm.
3

---

Published as a conference paper at ICLR 2025
Tweedie's
ADAM stabilization
Adaptive
Trajectory
Factual
Region
Naive
estimate
normalization
truncation
46.4
27.5 (-18.9)
23.1 (-4.4)
20.2 (-2.9)
16.1 (-4.1)
Figure 3: Series of proposed improvements to better align the gradient's of the classifier of interest with the generative trajectory. Changes to the factual image are constrained to the indicated region. Subsequent images illustrate the influence of each new adaptation. Numbers below images corre- spond to FID (L) values obtained in a larger-scale experiment (for details, see Appendix).
RVCEs through conditional inpainting, We define the problem of finding RVCEs for the classifier I from a given image x", a region R and target class label y, where arg max,/ f'y' xx) # as the task of sampling from
p(x arg max f(y' y, 1 (1-
where R a binary mask with indicating the region which is not restricted to depend on the classifier f. Intuitively, sampling from Eq. (2) means obtaining x with the complement of R. un- changed and the content of R modified in a way that changes the decision of f to y, i.e, performing inpainting with additional condition coming from the classifier f-
Synthesizing meaningful RVCEs. Looking at Eq. (2), one quickly realizes that obtaining seman- tically meaningful RVCEs requires maximizing the likelihood [ x) of the classifier while in- painting R with content that keeps x in the data manifold. These conditions greatly resemble the CG scheme the context of IBS since the score estimate s& serves as an effective prior for gen- erating in manifold infills, while the likelihood Ps( | x) of an auxiliary classifier is maximized to ensure that Ps predicts them as instances of y. Moreover, I²SB maps masked images directly to clean samples, leaving the content outside R unchanged in the final image.
The above arguments suggest that inserting f in place of Pe should function as an effective mech- anism for sampling meaningful RVCEs. However, fundamental drawback of this naive approach is that, throughout the generative process. f's gradients originate from evaluating it on images with highly noised infills inside R. (see Fig. D. Such corrupted images are far from what f observed during training, hence leading to a misalignment of its gradients with the correct trajectory and generation of oft-of-manifold samples. Similar issue has been identified by previously mentioned SGM-based methods for VCEs, which can be generally unified as attempts to replace the auxiliary classifier pa with f in the CG scheme in SGMs and correct f's gradients. Following Fig. 2 one should expect the misalignment in these methods to be of great extent, the generative trajectory consists of highly noised images, leaving no meaningful content for f to provide accurate gradients. There, as shown in Fig- 2 ISSB provides crucial advantage, which stems from its generative tra- jectory being much closer to the data manifold. Moreover, by using 1'SB, f is able to effectively utilize the readily available context outside R. Hence, in the following, we focus on reducing the misalignment problem caused by the noised content inside R, in the end arriving at a highly effective algorithm for meaningful RVCEs.
Aligning the gradients. We propose to adapt the gradients of f to properly align with the generative trajectory of PSB through series of incremental steps. To provide the intuition standing behind the introduction of each consecutive improvement, Fig. 3 provides an example RVCE task, where the factual image depicts a zebra correctly predicted by the model (ResNet50 (He et al. 2016)), and the goal is to change the decision to 'sorrel'. We set the region constraint to include the entire animal to make the task challenging enough and verify the improvements quantitatively through a large-scale experiment with around 2000 images. For each step. compute FID between the RVCEs and original images to assess their realism. For details on the experimental setup, see Appendix.
Naive. We first verify that naively plugging f in place of p& does not provide meaningful results. Indeed, as shown in Fig. B the method struggles to include the information from f. The unrealistic infill also suggests that the classifier's signal negatively influences the score from ISSB
4

---

Published as a conference paper at ICLR 2025
Tweedie's formula. To begin with closing the gap between the data manifold and the generative trajectory, we refer to a classic result of Tweedic's ' formula (Robbins) 1992 Chung et al., 2022
Weng et al., 2024), which states that denoised estimate of the final image at step can be achieved by computing the posterior expectation
xx(xx) == E x x, + of t)),
(3)
where of = S6 3, For visual differences between x; and X (xx)),, see Appendix. Crucially, one has access to approximate (x.) at every step t by utilizing ISSB as the approximate score. Replacing Vx, log f(y xr) with Vx, log f'y 8o(xx)) brings the inputs of / much closer to what it expects, improving the conditional inpainting process as indicated by Fig. which now shows structure resembling a sorrel and a much smaller FID.
ADAM stabilization. Despite utilizing the Tweedie's estimate, we observed the norms of f's gradi- ents to have a very noisy tendency throughout the generation process, pointing out a possible cause for visible artifacts and the missing parts of the animal. Hence, we propose to smooth out the gra- dients by applying the ADAM update rule at each step (Vaeth et al. [024 Kingmal (2019), to which we simply refer as ADAM stabilization. Figure 3 indicates that this modification allows for filling in the missing parts of the sorrel and further lowering FID.
Adaptive normalization. Incorporating ADAM stabilization required greatly lowering the guid- ance scale to values on the order of 1e-2, as using standard s = 1 led to extreme artifacts. This phenomenon suggested that the step size could also be adjusted throughout the generation process. While we initially experimented with various types of schedulers (see Appendix), using adaptive normalization has empirically proven to be the most effective approach. Specifically, at the begin- ning of the conditional inpainting process, we register the norm of the first encountered gradient of the log-likelihood of S. We then use it as a normalizing constant for each subsequent gradient, meaning that the generation begins with gradient of unit norm. This simple modification not only further lowered FID, but also reduced the final visible artifacts and improved color balance (Fig. 3
Trajectory truncation. Up until this point, we relied soiely on the ability of ISB and the classifier's signal to correctly infill the missing regions with semantically meaningful content, with no knowl- edge of the structure of the missing objects. Since a possible infill of the region is always available from the original image, one can begin the inpainting process from some intermediate step instead of the final one. This intervention allows for mixing the available information with the one coming from the elassifier, and gives direct control over the preservation of the original content. As our ap- proach does not bias the conditional score with signal from any additional losses (like Learned Per- ceptual Image Patch Similarity (LPIPS Zhang et al. (2018)) or l2 in other works), we can fully rely on the conceptual compression of PSB, similarly to SGMs (Ho et al. 2020). which decomposes the generation process into initial phases responsible for the overall structure of objects and later ones re- sponsible for small details. showecsss the effect of using this trajectory truncation (T) at the 0.4 level, meaning that the infilling process starts from 1 = T T. where T denotes the final timestep. Understandably, trajectory truncation greatly lowers the FID score, much more informa- tion is available from the very beginning of the process, and introduces much more sub- tle changes to the image. We explore the ef-
Image
Attribution map
Extracted region
Naive
feet of manipulating r further in the Appendix, showing that it functions as a very interpretable mechanism for controlling the content preser- vation.
Ours
Automated region extraction. While the introduced algorithmical improvements effec- tively incorporate the classifier's signal into the inpainting process, they do not address the issue of predetermining the region for the resulting explanation. To this end, the optimal strategy would be fully automated and focus on regions that are both important to the classifier's predic- tion and point to semantically meaningful con- cepts. This description closely resembles the
Figure 4: Example region obtained with our auto- mated region extraction. Instead of directly bina- rizing an attribution map (upper row), we amplify the focus on semantic concepts (bottom row) with a simple approach based on grid cells.
5

---

Published as a conference paper at ICLR 2025
role of visual attribution methods, which assign importance values to pixels based on their relevance to the model's output (Holzinger et al. 2022). Figure 4 shows an example attribution map obtained with Integrated Gradients (IG, (Sundararajan ResNet50 model. Perceptually, highest attributions are focused around the squirrel's head. To ex- tract region from such attributions, one threshold them to cover specific fraction a of the total image area. However, after binarizing the attributions with a = 0.05, we observe that the resulting region highly scattered, losing focus from semantic concepts. To address this issue, we divide the image into a grid of square cells of size of the absolute pixel attributions inside Figure shows that this postprocessing mechanism (here with 16) greatly amplifies the focus of the resulting map. By thresholding it with a 0.05, we observe the extracted region to focus solely on the squirrel's 's head. This leads to fully automated strategy for obtaining regions that are both aligned with semantically meaningful concepts and based on pixels that are important for the classifier.
al. (2017) method for the squirrel prediction of
where each cell receives the value equal to the sum
We term the final version of the algorithm which combines all of the aforementioned improvements with the automated region extraction as RCSB. For the pseudocode of the entire procedure, see Appendix. We include implementation at httts://gtthbb.com//bbbeeeiiiiip
EXPERIMENTS
Following previous works for VCEs on Ima- geNet, we base the quantitative evaluation on 3 challenging main VCE generation tasks: Ze- bra- Sorrel, Cheetah Cougar, Egyptian Cat Persian Cat, where each task requires creating VCEs for images from both classes and flipping the decision to their counterparts. We treat it general benchmark for evaluat- ing the effectiveness of RCSB in various see- narios. We use FID (L) and sFID (4) to assess realism (Heusel et al. 2017), S (1) for repre- sentation similarity (Chen & He (202], COUT E [- (t) (Khorram & Fuxin 2022) for sparsity and Flip Rate (FR) (1) for efficiency. For qualitative examples, we extend the main tasks with large array of other tasks, which we show throughout the paper and the Ap- pendix, where more details regarding the exper- imental setup and the metries description can be found.
Method
HDD
sFID
COUT
Zebra Sorrel
ACE 4
84.5
122.7
0.92
-0.45
47.0
ACE la
67.7
-0.25
LDCE-cll
84.2
107.2
0.78
-0.06
88.0
LDCE-txt
82.4
107.2
0.71
-0.21
81.0
DVCE
33.1
43.9
0.62
-0.21
57.8
RCSB
13.0
20.4
0.82
0.70
99.7
RCSB
9.51
17.4
0.86
0.72
97.4
RCSB
8.0
16.2
0.88
0.74
94.7
Cheetah
Congar
ACE l
70.2
100.5
0.91
0.02
77.0
ACE lg
74.1
102.5
0.85
0.12
95.0
LDCB-cls
71.0
91.8
0.62
0.51
100.0
LDCE-sxt
91.2
117.0
0.59
0.34
98.0
DVCE
16.9
51.1
0.49
RCSB
30.2
39.2
0.87
0.79
100.0
RCSB
23.4
32.4
0.90
0.85
99.9
RCSB
17.2
26.6
0.92
0.92
100.0
Egyption Cat Persiam Cat
ACE I
93.6
156.7
0.85
0.25
85.0
ACE lg
107.3
160.4
0.78
0.34
97.0
LDCE-els
102.7
140.7
0.63
0.52
RCSB sets new SOTA for VCEs. We first ver- ify that synthesizing RVCEs with RCSB leads to new SOTA in VCE generation. quan-- titatively compares RCSB with recent SOTA approaches to VCEs on ImageNet. Our RVCES are much more realistic (at least 2 - 4x de- crease in FID and sFID), stay close to orig- inal images (match or exceed best values of $3) and almost always flip the model's 's decision (FR ~ 1.0). RCSB also solves a long-standing challenge of achieving extremely sparse expla- nations on ImageNet, especially on Zebra Sorrel task. While all other methods fail to achieve nonnegative values, RCSB approaches the upper bound of COUT. Our method is clearly the most balanced, as it does not struggle on any specific metric like, e.g., DVCE on S3 In the A Appendix, show that is also the most computationally efficient.
L.DCEaxt
121.7
162.4
0.61
0.56
99.0
DVCE
46.6
59.2
0.59
0.60
98.5
RCSB
41.1
0.82
RCSB
31.3
48.1
0.84
0.87
100.0
RCSB-4
23.0
40.0
0.87
0.92
100.0
Table 1:
Quantitative comparison with SOTA.
RCSB outperforms previous methods by a large margin across all metrics. The best results ob- tained with A(a = 0.1.c = 4,8 = 3, 0.6), but the superiority is clear for various configura- tions, including B(a = 0.2, 4, 1.5,T 0.6), C(a 0.3, 4, 1.5, 0.6).

---

Published as a conference paper at ICLR 2025
Limpkin Flamingo
Guacamole Cabbage
Mashed Potato - Cauliflower
Common Iguana + American Chameleon
Green Mamba Indian Cobra
American Chameleon + Common Iguana
Tench Goldfish
Cougar Lynx
Figure 5: Qualitative examples obtained with RCSB using automated region extraction. Each task of the form predicted class -> target class shows the factual image, the extracted region and the RVCE obtained with RCSB.
Figure shows example explanations obtained with RCSB, greatly highlighting the importance of synthesizing RVCEs instead of standard VCEs. Our region extraction approach is able to precisely localize semantic concepts responsible for the model's decision. For example, in the Guacamole Cabbage task, RCSB detects the guacamole bowl in the background and. guided by the classifier, infills with cabbage while leaving the rest of the image unchanged. RCSB capable of performing wide range of editing tasks with various levels of difficulty, beginning with textural and color- Cauliflower) to partially changing the object's
edits (e.g., Tench - Goldfish, Mashed Potato structure (e.g., Limpkin - Flamingo) to infilling the region with new, realistically looking concepts (e.g., Cougar + Lynx, Green Mamba > Indian Cobra, Cougar + Lynx). Most importantly, thanks to the region constraint, our RVCEs allow for greatly limiting the potential factors that influenced the model's decision, making the explanations much more interpretable.
RCSB allows for causal inference about the model's reasoning. Drawing definite conclusions about the model's reasoning from an unconstrained VCE is not possible, as one cannot be certain that modifying potentially irrelevant factors did not in fact influence the prediction. RVCEs overcome this limitation when constrained on the region connected with the sole factor of interest, e.g., the body of an animal in a species prediction task. To adapt RCSB such scenario, we replace the automated region extraction method with foundation sil-to--ojjcc--semeettta
model | Using the class name from a given task the text prompt allows us obtain highly precise segmentation masks of the relevant objects, enabling the identification of the cause behind the model's prediction change based solely on factors related to object of interest.
We first quantitatively assess that RCSB is capable of utilizing regions provided by generic object detector at scale. T() shows the results of this evaluation together with the used text prompt. Here, the metrics are computed by first discarding images with a mask that covers area larger than
II Language Segment Anything (1 AAM combines Segment Anything Model Kirillov al. GroundingDINO (Liu al. 2023b) allow object segmentation from text prompts.
2023 with

---

Published as a conference paper at ICLR 2025
Metric
FID
sFID
s
COUT
FR
FID
sFID
$a
COUT
FR
FID
sFID
$a
COUT
FR
Task
Zehra Sorrel
Cheetah Cougar
Egyptian Cat Persion
A
Exacl regions obtained with LangSAM and prompts: zebra / horse, cheetah cougur, cal respectively
Values
32.8
41.5
0.87
0.74
98.9
37.2
50.6
0.91
0.84
99.4
52.0
82.8
0.81
0.84
99.2
B
Regions hased on freeform masks with the area in the indicated range
10
20%
6.7
15.0
0.85
0.85
87.6
9.0
19.1
0.89
0.72
96.6
12.4
29.6
0.80
0.73
96.9
20
30%
7.8
15.8
0.84
0.53
92.2
11.6
21.3
0.88
0.71
99.6
17.7
34.0
0.78
0,74
99.3
C
Ahlation study with adupiarions of other inpuinting algorithmss
RePaint
63.8
76.0
0.55
0.77
99.3
129.3
144.2
0.50
0.77
99.0
148.7
175.2
0.38
0.76
99.5
MCG
43.2
55.6
0.73
0.45
96.0
76.6
91.4
0.74
0.64
100.0
93.7
117.5
0.62
0.65
99.9
DDRM
49.4
99.6
68.4
0.72
0.76
100.0
73.0
0.76
100.0
Table 2: Quantitative results from various experiments. A: regions extracted from LangSAM with text prompt connected to the initial class name. B: regions based on freeform masks that cover the fraction of the total area from the indicated range. C: automatically extracted regions used with adaptations of other inpainting algorithms.
40%. Despite I'SB being trained on masks covering at most 30% of the image area, we observed that it generalizes well beyond this threshold with 40% starting to pose a challenge. Crucially, despite the regions being agnossic and hence not necessarily focused on the most influential pixels, Table A) indicates that RCSB is versatile enough to maintain most of the performance from the automated approach. The efficiency, sparsity and representation similarity of the obtained RVCEs remain very close to the values achieved by the closest configuration (in terms of hyperparameters) from Table [ the region area is often close to exceeds 30%. The slight increase in FID and sFID stems mainly from the regions covering complex objects, whose modification may naturally move RVCEs further from original data at a distribution level, and lower number of images used for these metrics' computation (as both are sensitive to sample size) due to the rejection of samples from the area constraint.
Regions that contain exactly the objects of interest provide novel insights about the model's reason- ing. For example, consider the Lemon + Orange task from Fig. 6 where the lemons were correctly identified by the ResNet50 model. One would require the VCE for this task to indicate the sole determining factor of why lemons and not oranges'. However, with unconstrained VCEs, this iden- tification process quickly becomes incomprehensible due to small changes added to each object in the image, such as other fruits. By constraining VCEs to the region occupied by the lemons, the reasoning process can be disentangled and simplified, as one can now look for this factor in the modifications of the lemons only. In this case, RCSB is guided by the classifier to change the image in way that is consistent with human intuition.
RVCEs also allow for clarifying the model's decision-making when its reasoning is not initially understandable. In the Volcano - Seashore task, the image shows both objects, while the model predicts it as the former. Applying RCSB to the exact region of the seashore results in a RVCE that changes the model's decision when the water's color becomes more light blue and structures like stones start to appear. Hence, one is able to hetter understand what the model actually identifies as a seashore. In other examples, the method introduces class-specific characteristies when the changes are constrained precisely and exclusively to the object of interest, ensuring the receiver about the general cause of the model's decision change. Such cases are also especially relevant when the generative model used to synthesize explanations is prone to systematic errors like, e.g., SGMs struggling with correctly generating hands. In the Night Snake - Kingsnake task, this error can be bypassed with the region constraint by not allowing the generative model to affect anything other than the animal, hence alleviating the evaluation of the classifier on mut-of-manifole samples.
Discovering complex patterns with interactive RVCEs. Despite the impressive capabilities of deep models in object localization, the receiver of the explanation may be interested in testing the model for highly abstract and complex concepts that cannot be localized automatically and must be provided manually by the user. We begin with verifying the capability of RCSB in generating RVCEs based on user-defined regions by simulating such scenario at scale. Specifically, we ran- domly match images from the main tasks with regions given by the 10% - 20% and 20% 30% freeform masks from the ISB training data (Saharia et al. 2022). We argue that this serves very
8

---

Published as a conference paper at ICLR 2025
Night snake
Kingsnake (smake)
Lemon Orange (lemon)
Tench Goldfish (fish)
Kit fox + Arctic fox (fox)
Flat-coated Curly-coated (dog)
Volcano Seashore (seashore)
Figure 6: Qualitative examples obtained with RCSB using exact regions extracted from Lang using text prompt of the predicted class. For each task of the form predicted class - target class, factual image together with the used region and the resulting RVCE shown. The used text prompts are emphasized.
challenging benchmark, since the algorithm's access the most influential pixels (for the classifier) might often be very restricted.
Despite the task's difficulty, quantitative results from Table ( highlight the versatility of RCSB, which is able to effectively utilize the restricted resources to influence the classifier's prediction. While S COUT and FR not significantly different from previous results, we observe decrease in FID and sFID, indicating higher realism and closeness to the data distribution. This is largely due the fact that freeform masks are often not connected entire complex objects and do not contain the pixels most important to the classifier. Hence, RCSB may often leave large portions of the regions unchanged, which boosts the realism evaluation.
To allow for true interaction of the user with the explanatory process, we implement simple inter- face that allows for manual image segmentation using brush-like cursor. F shows example results, where we manually predefine the regions different images. This exploration gives impor- tant insights about the added value provided by RVCEs. In the Cat - Tiger task, we discover that the classifier's decision can be flipped by independently modifying either the cat's paws snout, in both cases introducing a tiger's coloration. Similarly in the Aretic Fox - Red Fox task, choos- ing either the ears and muzzle paws and stomach allows for changing the model's decision with the features of a red fox. User-defined regions also allow to discover unusual reasoning pat- terns of the model. In the Cucumber - Zucchini task, the model's decision be influenced by modifying only one of the cucumbers to zucchini, leaving the other unchanged. This observation connects with recent positions on the topic of contextual and spatial understanding of predictive models (Tomaszewska & Biecek 2024), providing new rationale in further exploring how image classifiers actually reason.
Ablating RCSB's components. We empirically verified that combining our novel guidance mecha- nism with the FBSB prior leads to highly effective RVCEs. better understand the benefits provided by each component of our framework, we perform an ablation study, where we adapt the proposed improvements to SGM-based inpainters, aiming to assess the influence of the guidance scheme and PSB in isolation. Specifically, we pick RePaint (Lugmayr et al. (022). one of the first adaptations of SGMs to inpainting, MCG (Chung et al. 2022) and DDRM Kawar et al. 2022). two different adaptations of SGMs to linear inverse problems, which also include inpainting. We manually tune our guidance scheme to each method on a small subset images and repeat the same evaluation protocol with the automated region extraction method (see Appendix for details of each adaptation).

---

Published as a conference paper at ICLR 2025
Cat + Tiger
Arctic Fox Red Fox
Cucumber Zuochinii
Cat Tiger
Hornbil
Toucan
Jay Indigo Bunsing
Figure 7: Qualitative examples obtained with RCSB from user-defined regions. For each task of the form predicted class + target class, a factual image together with the provided regions are shown. Arrows point to RVCEs obtained by modifying only the indicated region. As these methods are much less compute-efficient, we cap their computational budget on each task to 24 A100 GPU hours.
Table C)) shows the results of the ablation study. Despite the fact that the used methods were never explicitly trained for inpainting, combining them with our guidance mechanism and region extrac- tion allows for matching or even exceeding previous SOTA. For example, all adaptations achieve very high sparsity, almost always flip the classifier's decision and keep the explanation close to the original. This indicates the benefits of utilizing only the pixels from the extracted region and proper utilization of the classifier's 's gradients without biasing them with additional components like LPIPS or le loss. RCSB differentiates itself from the adaptations with a much higher realism of the obtained RVCEs (significantly lower FID and sFID), more balanced results and much smaller computational burden, e.g. 24x less NFES than RePaint. These benefits stem from the PSB prior, which trained to map corrupted images directly to clean samples and the resulting trajectory being much closer to the data manifold, allowing the classifier to more effectively influence the inpainting process.
DISCUSSION & LIMITATIONS
RVCEs offer new perspective on the concept of VCEs, with RCSB effectively demonstrating their versatility in various scenarios that have not been explored in previous work. In view of this, enfore- ing a hard region novel challenges and raises important questions. For instance, the explanations do not reveal changes in the interactions between different objects in the image that influence the model's decision. Fur- thermore, due the absence of ground truth, verifying the actual reasoning of a model based on the explanation remains difficult, even if RVCEs appear intuitive. Additionally, the evaluation process of RVCEs may be skewed by the preservation of large portion of the original pixels (e.g., FID).
-aaaaintttttentallly chosen independently of the predictive moodel-introduces
We address several of these issues in the Appendix, including two user studies on the general useful- ness of RVCEs, the possibility of interacting with the explanatory process, and their informativeness regarding model misclassifications. We also present extended qualitative and quantitative results for other classifiers, datasets, and attribution methods, empirical demonstrations of some of RCSB's capabilities (e.g., shape changes), and other key aspects. We believe that the limitations of RVCEs and RCSB offer valuable directions for future research.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was financially supported by the Polish National Centre for Research and Development (NCBiR, xLungs grant no. INFOSTRATEEG-000222011 provided by the Laboratory of Bioinformatics and Computational Genomics and the High Perfor- mance Computing Center of the Faculty Mathematics and Information Science, Warsaw Univer- sity of Technology.
The computational resources were
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models, volume 12. Stochastic Processes and their Applications, Elsevier, 1982.
Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explanations. In Advances in Neural Information Processing Systems, 2022.
Maximilian Augustin, Yannic Neuhaus, and Matthias Hein. Analyzing and explaining image clas- sifiers via diffusion guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Hobus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. In PloS one, 2015.
Przemyslaw Biecek and Wojeiech Samek. Position: Explain to question not to justify. Proceed- ings of the 4 International Conference on Machine Learning, volume 235, pp. 3996 4006, 21-27 Jul 2024.
Valentyn Boreiko, Maximilian Augustin, Francesco Croce, Philipp Berens, and Matthias Hein. Sparse visual counterfactual explanations in image space. DAGM German Conference on Pattern Recognition, pp. 133-148. Springer, 2022.
Qiong Cao, Li Shen, Weidi Xie, Omkar Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference automatic face & gesture recognition (FG 2018), pp. 67-74. IEEE, 2018.
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image clas- sifiers by counterfactual generation. International Conference on Learning Representations, 2019.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15750-157555 15758, 2021.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems, 2022.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mecann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. The Eleventh Interna- tional Conference on Learning Representations, 2023a.
Hyungjin Chung, Jeongsol Kim, and Jong Chul Ye. Direct diffusion bridge using data consistency for inverse problems. In Thirry-seventh Conference Neural Information Processing Systems, 2023b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scall scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 255.255. IEFF, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. In JEEE Signal Processing Magazine, 2012.
11

---

Published as a conference paper at ICLR 2025
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans image synthesis. In Advances in neural information processing systems, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Logan Engstrom, Andrew Ilyas, Hadi Salman, Shibani Santurkar, and Dimitris Tsipras. Robustness (python library). In
2019.
Karim Farid, Simon Schrodi, Max Argus, and Thomas Brox. Latent diffusion counterfactual expla- nations. In arXiv preprint arXiv:2310.06668, 2023.
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. Counterfactual visual explanations. In International Conference on Machine Learming, pp. 2376-2384. PMLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pailern recognition, pp- 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. In Advances in neural information processing systems, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in neural information processing systems, 2020.
Andreas Holzinger, Anna Saranti, Christoph Molnar, Przemyslaw Biecek, and Wojeiech Samek. Explainable AI methods-a brief overview. In International workshop on extending explainable Al beyond deep models and classifiers, pp. 13-38. Springer, 2022.
Gao Huang. Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinherger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4708, 2017.
Paul Jacob, Éloi Zablocki, Hedi Ben- Younes, Mickaël Chen, Patrick Pérez, and Matthieu Cord. Steex: steering counterfactual explanations with semantics. In European Conference on Computer Vision, pp. 387-403. Springer, 2022.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Diffusion models for counterfactual explana- tions. In Proceedings of the Asian Conference on Computer Vision, pp. 858-876, 2022.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Adversarial counterfactual visual explana- tions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16425-16435, 2023.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Text-to-image models for counterfactual explanations: black-box approach. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 4757-4767, 2024.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for im- proved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
Saeed Khorram and Li Fuxin. Cycle-consistent counterfactuals by latent transformations. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10203- 10212, 2022.
Ba Jimmy Kingma, Diederik P. Adam: A method for stochastic optimization. In International Conference Learning Representations, 2019.
12

---

Published as a conference paper at ICLR 2025
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan- Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference Computer Vision, pp.
2023.
Andreas Kirsch. An introduction to the mathematical theory of inverse problems. Applied mathe- matical sciences. Springer. New York, 2nd ed edition, 2011.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang. Bilal Alsallakh, Jonathan Reynolds, Alexander Mclnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, et al. Captum: A uni- fied and generic model interpretability library for pytorch. In arXiv preprint arXiv:2009.0786, 2020.
Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T Freeman, Phillip Isola, Amir Globerson, Michal Irani, et al. Explaining in style: Training a gan to explain a classifier in stylespace. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 693-702, 2021.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basse learning applied to document recog- nition. In Proceedings of the IEEE, 1998.
Guan-Horg Liu, Arash Vahdat, De-An Huang, Evangelos Theodorou, Weili Nie, and Anima Anandkumar. 12SB: Image-to-Image Schrödinger Bridge. In International Conference on Ma- chine Learning, pp.
PMLR, 2023a.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In arXiv preprint arXiv:2303. 054499 2023b.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Zhuang Liu, Hanzi Mao, Chao- Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 1 11976-1 1996, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Pro- ceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
Robert J. McCann. A convexity principle for interacting gases. In Advances Mathematics, 1997.
Franz Motzkus, Christian Hellert, and Ute Schmid. Cola-dce- concep--uudedd latent diffusion coun- terfactual explanations. In arXiv preprint arXiv:2406.01664 2024.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In Advances in neurul information processing systems, 2019.
Gabriel Peyré and Marco Cuturi. Computational optimal transport: With applications to data sci- ence. Foundations and Trends Machine Learning, 1 (1(5-))3355-677, 2019.
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasom Suwajanakorn Dif- fusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 10619-10629, 2022.
13

---

Published as a conference paper at ICLR 2025
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PMLR, 2021.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1134-1144, 2016.
Herbert E Robbins. An empirical Bayes approach to statisties. In Breakthroughs in Statistics: Foundations and basic theory, 1992.
Pau Rodríguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Char- lin, and David Vazquez. Beyond trivial counterfactual explanations with diverse valuable ex- planations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1056-1065, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and puttern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Jonathan Scarlett, Reinhard Heckel, Miguel RD Rodrigues, Paul Hand, and Yonina C Eldar. Theo- retical perspectives on deep learning methods in inverse problems. In IEEE journal on selected areas in information theory, 2022.
Erwin Schrödinger. Sur la théorie relativiste de l'électron et I' l'interprétation de la mécanique quan- tique. In Annales de l'institut Henri Poincaré, volume 2, pp. 269-310, 1932.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems, 2022.
Lisa Schut, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, al. Gener- ating interpretable counterfactual explanations by implicit minimisation of epistemic and alcatoric uncertainties. In International Conference on Artificial Intelligence and Statistics, pp. 1756-1764.
PMLR, 2021.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: visual explanations from deep networks via gradient-based local- ization. In International journal of computer vision, 2020.
Sheng-Min Shih, Pin-Ju Tien, and Zohar Karnin. Ganmex: One-vs-one attributions using gan-based model explainability. In International Conference on Machine Learning, pp. 9592-9602, PMLR, 2021.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. International conference on machine learning, pp. 3145- 3153. PMLR, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference Learning Representations, 2015.
14

---

Published as a conference paper at ICLR 2025
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. 2nd International Conference on Learning Representations, Workshop Track Proceedings, 2014.
Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by pro-
gressive exaggeration. In International Conference on Learning Representations, 2020.
Bartlomiej Sobieski and Przemystaw Biecek. Global counterfactual directions. In European Con- ference on Computer Vision, 2024.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In International Conference on Learning Representations, Workshop Track Proceedings, 2015.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328. PMLR, 2017.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2818-2826, 2016.
Jayaraman Thiagarajan, Vivek Sivaraman Narayanaswamy, Deepta Rajan, Jia Liang, Akshay Chaudhari, and Andreas Spanias. Designing counterfactual generators using deep model inver- sion. In Advances in Neural Information Processing Systems, 2021.
Rui Tian, Zuxuan Wu, Qi Dai, Han Hu, and Yu-Gang Jiang. Deeper insights into the robustness of vits towards common corruptions. In arXiv preprint arXiv:2204. 12143, 2022.
Paulina Tomaszewska and Przemystaw Biecek. Position paper: Do not explain (vision models) without context. In International Conference on Machine Learning, 2024.
Philipp Vaeth, Alexander M Fruchwald, Benjamin Paassen, and Magda Gregorova. Gradcheck: Analyzing classifier guidance gradients for conditional diffusion sampling. In arXiv preprint arXiv:2406. 77399, 2024.
Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by pro- totypes. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pp. 650-665. Springer, 2021.
Nina Weng, Paraskevas Pegios, Aasa Feragen, Eike Petersen, and Siavash Bigdeli. Fast diffusion- based counterfactuals for shorteut removal and generation. In European Conference on Computer Vision, 2024.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Swilzerland, September 6-12, 2014, Proceedings, Part / 13, pp. 318-833. Springer, 2014.
Mehdi Zemni, Mickaël Chen, Éloi Zablocki, Hédi Ben- YounYs, Patrick Pérez, and Matthieu Cord. Octet: Object-aware counterfactual explanations. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 15062-15071, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pallern recognition, pp. 586-595, 2018.
Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. In Inter- national Conference on Learning Representations, 2018.
15

---

Published as a conference paper at ICLR 2025
CONTENTS
Introduction
Background Related Work
Method
4 Experiments
5
Discussion & Limitations
10
Pseudocode
18
Limitations
19
Extended Background & Related Work
20
D Comparison to previous works
21
Extended Method
22
Additional figures
22
Incorporating the classifier's signal
22
Analytic posterior and OT-ODE
23
Extended Experiments
23
Details of individual experiments
24
F2
Metrics description
24
Adaptation of other inpainting algorithms
25
F.3.1
Manifold Constrained Gradient (MCG, Chung et al. (2022))
25
E33.2
Denosing Diffusion Restoration Models (DDRM, Kawar et (2022)
25
E3..3
RePaint (Lugmayr et al. [0222
26
F4
Schedulers for guidance scale
26
F5
Quantitative evaluation of other attribution methods
26
F.6
Diversity assessment
27
F7
Computational efficiency assessment
27
F8
Additional quantitative results
28
F9
Freeform masks
28
F
10
Unintuitive classes
28
Shape modification
29
:r..............................
Lower-level attributions
29
Other classifiers
30
14
Other benchmarks
30
16

---

Published as a conference paper at ICLR 2025
User studies
32
Usefulness and interaction
32
G.2 II: Understanding model's s failures
32
Qualitative examples
33
The Appendix is structured as follows. Appendix A shows pseudocode for both PSB (stochastic and deterministic version) and our RCSB. Appendix B delves deeper into the possible limitations of RVCES and RCSB. Appendix(Cíneludde and extensive literature review regarding topies connected with work. Appendix A Comparee our approach to prior methods for VCE generation in detail. Appendix shows additional figures from the method's description, considerations regarding the incorporation of the classifier's signal into FSB and more detailed derivation of the OT-ODE version. Appendix Dactiixxxexxtte eexeends our experimen- tal evaluation with details about the setup, additional results regarding, e.g., efficiency and diversity, other datasets and classifiers, and concludes with details about the adaptation of different inpainting algorithms. Appendix provides details about the conducted user studies. Appendix [H provides qualitative examples for additional classifiers, showing the versatility of RCSB, together with more RVCEs for the ResNet50 model.
additional background knowledge connected with 1-BB
17

---

Published as a conference paper at ICLR 2025
A PSEUDOCODE
Algorithm Standard SB Generation
1: Input: x P1 (xN), trained sy (+)
2: for n = .N to 1 do
3: Predict xo(x.)) using Sy (Xn: (n)
4: Xn-1 p(xx-1 80, X,) according to DDPM
5: end for
6: return Xo
Algorithm OT-ODE I'SB Generation
Input: x ~ Pı (xx), trained sy (+.+)
2: for N to 1 do
3: Predict &x(x)) using Sw (Xn; (n)
4: Xx-1 12 + p
5: end for
6: return X9
Algorithm 3 RCSB
1: Input: Number of steps N, binary region mask R. trajectory truncation classifier scale s,
input image x', trained ( (1, :), trained classifier f(y +), target class y
2:
R. @ x + R z, where N 0, I)
3: Discretize truncated timeline = to < h .... < IN =
4: xNv q(xx X1)
# sample from analytic posterior (Eq. 15
5: for = N to 1 do
6: Predict Re (x,, using Sy (x,: fm)
7: gn Xx. log fl Xx)
8:
ga = ADAM(g.)
9:
if
N then
9 1x
register norm of the first gradient
10:
end if
11:
Xn = X, + &n
12: Xn-1 Mnn 180 + An 8n
end for
14:
return X
Algorithm 4 ADAM Update Rule
1: Input: Gradient at step gn. hyperparameters a, 51, B2 (set to PyTorch Paazke et al. 2019 defaults)
2: m, 8,m,- -1 + (1
31 31)g.
# update biased first moment estimate update biased second moment estimate compute bias-corrected first moment compute bias-corrected second moment
3: V, = = +
32)g
4: m. -mn// (1 34)
5: v, = =vn/( B
6: ga = amm//
Vn +
# update gradient # return updated gradient
7: return g..
18

---

Published as a conference paper at ICLR 2025
0.4
0.6
0.8
(x()))
X
Figure 8: Visual difference between x, and its corresponding Tweedie's 's estimate )x) different timesteps.
across
0.1
Figure 9: Influence of manipulating T on the final RVCE obtained with the region shown in Fig.
B LIMITATIONS
Despite setting quantitative records, approach comes with natural limitations that must be mentioned. From the user perspective, RVCEs generated with RCSB, especially when the region visually appealing and extracted independently from the classifier, may lead overconfidence about the model's decision-making process. As an example, consider the bird's image from Fig. [ and the RVCE based on the region of its head. Because our approach allows for changing the model's 's decision by using solely the area of the head, one could interpret that the new features are explicitly and exclusively responsible for the new prediction. However, the situation may be much more complex. For example, the appearance of some specific features of the head, e.g., red eyc, may influence the model conditionally on some other features which were already present in the image. Since ground truth to counterfactual explanation exists (at least in non-synthetic scenarios), the exact relationships between different aspects of the image remain unknown. Hence, must be careful when interpreting any kind of counterfactual explanation, especially RVCEs, to not draw incorrect and misleading conclusions.
RVCE, does not mean that the model not relying on some unintuitive shortcuts or spurious correlations which were not modified in the process. Hence, one has remember that VCEs aim at identifying the minimal semantic change required to change the model's decision, and are not guaranteed to modify every feature that the model relies on. These aspects highlight the importance and need of principled evaluation measures for this kind of sample- masple-based explanations. While attribution methods have been heavily addressed in this context in recent years, the evaluation of VCEs remains very difficult challenge.
Moreover, modifying given image counterfactual explanation, in particular
In terms of practical limitations, our experiments are based on an I2SB model trained for an inpaint- ing task based on freeform masks with coverage of 20%-30% of the total area. While the model generalizes well to larger fractions of the total image area on the order of 40%-45%, its performance deteriorates above this threshold. From a theoretical point of view, the I2SB algorithm is not limited to any particular upper bound on the total area, but demonstrating that is possible to obtain good performance total area of the order of, e.g., 90%, remains open research question that has important implications for our work. We believe this to be an interesting direction for future research with many possible extensions.
19

---

Published as a conference paper at ICLR 2025
C EXTENDED BACKGROUND & RELATED WORK
Visual counterfactual explanations. In recent years, increasing attention is being paid to syn- thesizing VCEs for image classifiers (Goyal et al. 2019 Schut et al. 2021). ). These explanations aim at elucidating the model's reasoning by modifying the input image in a semantically minimal
and meaningful way while flipping its prediction. Utilizing generative models for this task has his- torically proven to be very effective (Chang et al.] 2019; [Singla et al., 2020: Lang et al.] 2021]) Non-SGM-based methods include works like Thiagarajan et al. (2021) which builds on the concept of deep inversion, OCTET from Zemni et al. (2023) focusing on VCES for complex scenes and more
examples built on top of generative models (Rodriguez et al., 202 Jacob et al.) 2022 Shih et al.,
2021
Zhao et al. 2018: Van Looveren & Klaise| 2021).
While offering impressive results (Farid et al.) 2023] Jeanneret al.) 2024 Augustin et al. 2024
Motzkus et al., (024), we argue that utilizing general foundation models like SD in the VCE gen- eration task may cause misleading conclusions, since the explained classifiers are trained on much smaller datasets than the generative model. For example, about 1 million images from ImageNet (Deng et al.] (209)) are used to train the classifier, while SD is trained on billion images from LAION-5B (Schuhmann et al. 2022). This discrepancy may naturally lead to SD symthesizing alistically looking variations of a given image that flip the classificr's decision but simultancously include semantic attributes never observed by the predictive model during training. Therefore, one may question the counterfactual nature of the explanation, as the classifier should not be expected to correctly treat attributes that it was never close observing. Hence, in this work, we focus on generative models trained on the same data as the classifier of interest. This way, we can study the behavior of predictive models in a faithfull manner, which is an open challenge for XAI community Biecek & Samek (2024).
Inverse problems. Inverse problems (Kirsch 2011) are defined as the task of recovering an un- known signal x based on a measurement related via a measurement model H through y = Hxx,, where H is not necessarily required to be linear or bijective. Hence, for given measurement y, there may exist probability distribution over possible solutions p(x | y H(x)). One special case of an inverse problem is image inpainting. where the missing area of an image, indicated by the mask M, must be infilled using the available context. The measurement model is then defined as H(x) = M @ x, where @ denotes an element-wise product and M a binary mask.
In recent years, deep learning methods have proven to be very effective at solving various kinds of inverse problems (Scarlett et al. 2022). Recently, utilizing generative methods, especially SGMs, established itself as the new SOTA approach in the image domain. One way of adapting SGMs to inverse problems is through conditional generation, where the conditional score can be derived with the measurement model. Many additional techniques, such as data consistency (Chung al.) 2023b), manifold constraint (Chung et al. 2022 2023a) and others (Kawar et al. 2022). are further utilized to improve this adaptation.
Image-to-Image Schrödinger Bridges. A much harder but possibly also much more effective ap- proach is to learn direct mappings between the distribution of signals x Po and measurements ~ Pi instead of adapting pretrained models. In this line of research, Liu et al. (2023a) propose to learn such mappings by constructing tractable subclass of Schrödinger bridges (SBs, (Schöddinger (1932)) ) termed Image-to- Image Schrödinger Bridges (I²SBs). The SB is an entropy entropy-regularizee op- timal transport model, which, resembling the framework of SGMs, considers the following forward and backward SDEs:
dx; (x+()) + 3r Vx, log (x,, d) + v B/dw,
(4a)
dx; (x. - 3, Xx, log T (x, dt + / /,dw,
(4b)
Similarly to SGMs, the marginal densities of Eqs. 4a and 4b) equivalent. The functions
T T E (Rd [o, 1)) represent time-varying energy potentials and are additionally constrained to solve the following partial differential equations
* (x).)
1. V
St
(5a)
L ().
V (FF) BAA
8t
s.t. V (Xo, 0) T ()=
4x 1) (x, (x)
(5b)
20

---

Published as a conference paper at ICLR 2025
In general, numerically solving Eqs. nonlinear terms Vr T being coupled via Eq. (5b). However, with Theorem 3.1, Liu et al. (2023a)
(4a) and (4b) is much more difficult compared to SGMs due to
show an important connection between the frameworks of SBs and SGMs. We repeat it here explic- itly for later reference.
Theorem (Reformulating SB drifts as score functions (Liu et al. 2023a)) If V, fulfill the constraints given by Eq. 5. then Vx, log (x,,t), Vx, log v (x,, t) are the score functions of the following linear SDEs, respectively
dx;- Fx(xx)dl++ BBdw, Xo (1,0),
dx, = Fx(x,)dt++ + B.dWw X1 x1~ ~ ( (;,1)
(7)
Crucially, Theorem 1 states that, while T v are not in general assumed to be valid probability distributions. it is true that V log T = V log e and V Vlog ve = V log log for ب _ reppeseenting the densities of the respective SDEs. Following this theoretical result, Liu al. (2023a) also show a principled approach for approximating Xx, log (x,, t) with a neural network Sg. In essence, these results allow train direct inverse problem solvers with the use of paired data, where Po represents clean data distribution and pl the distribution of its corrupted measurements. Additionally, Liu et al.
(2023a) show how ISSB connects with flow-based optimal transport (OT) (Peyré & Cuturi 2019
McCann 1997}. where assuming that 3 + O leads to an ordinary differential equation (ODE) dx, (x xo)dt that provides deterministic mapping with the use of sy estimate. In practive, this is achieved by eliminating the noise from the intermediate sampling steps (see Algorithm 2)
Visual attribution methods. The very first works in the current era of Explainable Artificial In- telligence (XAI) were concerned with providing explanations of the model's decision through vi- sual heatmaps, which highlighted pixels considered important the its prediction. One of the first approaches by (Simonyan et al. (2014) proposed simple backpropagation of the model's out- put w.r.t. the input, indicating the direction of its greatest ascent in the pixel space, often termed as Saliency. More sophisticated approaches emerged in the following years, where techniques like Layer-wist wise Relevance Propagation (LRP, Bach et al. (2015). Integrated Gradients (IG, Sunddrara- jan et al. (2017)) DeepLift and Input x Gradient (Shrikumar al. (017)) Guided Backpropagation (GuidedBackprop.. Springenberg et al. (2015), GradCAM (Selvaraju et al. 2020), Deconvolution (Zeiler & Fergus 2014) and others that utilize the gradient of the neural network promised to indi- cate more semantically meaningful concepts in less 'noisy" manner. Concurrent line of research methods assumed a more general black-box scenario, where explanations could be provided for a broader class of models. There, methods like Occlusion (Zeiler & Fergus, 2014]. Local Interpretable Model-agnostic Explanations (LIME, Ribeiro et al. (2016)), SHapley Ad- ditive exPlanations (SHAP, Lundberg & Lee (2017)) and its variations quickly advanced the state- of-the-art. In this work, we utilize their unified implementations provided by the Captum package (Kokhlikyan et al., 2020).
about perturbation-base
D COMPARISON TO PREVIOUS WORKS
Due to space limits, we only briefly mention the previous diffusion -based approaches to VCE gen- eration in Section 2 In the following, we provide more details about their theoretical formulations and how they relate to our approach.
Most of the previously published diffusion-basse approaches to VCEs rely on the conditional re- verse process obtained by replacing the unconditional score Xx, log p(x,,t) t) in Eq. (Ib) with the conditional variant of the form Vx, log p(x,, t) + Vx, log p(y X2, x,, This formulation allows one to incorporate the classifier- based conditioning in various forms. Jeanneret et al. (202)) (DiME) pro- pose to approximate the likelihood score Vx, log p(y xx, t) by fully denoising the image at step t and using it to obtain the classifier's gradient leading to quadratic complexity with respect to the total number of timesteps. The work of Augustin et al. (002)) (DVCE) instead propose to regularize the gradients of the explained classifier with the one coming from robustly trained classification net- work. Specifically, the gradient of the former is projected onto a cone around the direction indicated by the gradient of the latter with some predetermined angle. The follow-up work of Jeanneret et al.
21

---

Published as a conference paper at ICLR 2025
(ACE) proposes split the explanation generation into two distinct phases, with the first one responsible for generating the pre-explanation, and the second one performing processss..... The former combines standard diffusion denoising with PGD attack performed with the use of the classifier on x, for each 1. Then, the latter phase computes the absolute difference between the original image and the pre-explanation, and extracts binary mask by thresholding this difference. Next, it uses the RePaint (Lugmayr et al. (022) algorithm to unconditionally inpaint the masked region beginning from some intermediate timestep. To address the low efficiency of DiME, Weng et al. (2024) (FastDiME) propose to improve the conditioning process by utilizing at each timestep t the gradient of the classifier with respect to the Tweedie's estimate. By default, their approach performs dynamic masking throughout the generation process which indicates the region modified at each timestep. Moreover, the authors propose two 2-step extensions of their approach, namely FastDiME-2 and FastDiME-2+, that first perform either standard FastDiME or FastDiME without dynamic masking, then extract the most differing regions and utilize the binary mask resulting from the largest changes to conditionally inpaint it with the use of the classifier.
2023
Importantly, ACE and FastDiME bear resemblence to our approach in terms of performing some variant of conditional inpainting. Specifically, ACE first combines the classifier with diffusion de- noising to extract a region important to the predictive model, but inpaints it unconditionally. Fast- DiME (and its variants) also find region that is regarded important the classifier, but inpaint it conditionally. Our approach solves a more general problem by synthesizing RVCEs which do not assume any dependence of the predetermined region to the classifier of interest, which we show through a large array of experiments with regions coming from sources like automated segmenta- tion or user interaction. Moreover, while all of the previously mentioned works are concerned with adapting standard SGMs to either guided generation or inptainting, we show how more general class of models (tractable Schrödinger Bridges) can be adapted to the problem of conditional in- painting with the use of a classifier. Additionally, we propose series of improvements that better align its gradients with the generative trajectory that were not previously present this line of re- search connected to XAI, such ADAM stabilization or adaptive normalization. Finally, to the best of our knowledge, our approach is the first one to show that guidance can be performed with the signal coming solely from the classifier of interest, omitting the usage of additional proxy measures (like le loss or LPIPS) that maintain similarity to the original image. It it also important to higlight that, while very simple, automated region extraction approach was also not present in this line of research and, through experimental evaluation, was shown to provide highly interpretable and intuitive regions.
E EXTENDED METHOD
E.1 ADDITIONAL FIGURES
We provide illustrative examples for the visual differences between x, and the Tweedic's estimate 8o in Fig. 8 For the effect of manipulating the hyperparameter, see Fig.
E.2 INCORPORATING THE CLASSIFIER'S 'S SIGNAL
In the following, we explicitly define the DDPM (Ho al. 2020) sampler mentioned in Algorithm and elaborate on the exact way of incorporating the classifier's 's gradients into the generation process of PSB.
Denote by [t.)ee(o....N) N) the discrete sequence of timesteps of length N such that 0 = to < f1 < IN = 1. By o = fo" dr and 0² = St' B,dT, we denote the variances accumulated from each side. Additionally, let a² = -str,, 3 be the variance accumulated between two consecutive timesteps. For ease of notation, we define 1n- and f/n as
Mn-1
(8)
a n-
n
0²
An-
(9)
a
02
22

---

Published as a conference paper at ICLR 2025
With that, we can define the DDPM posterior sampler as
Xn-1 ~ p(xn-11 Xn),
(10)
0 2n-10 Xn-1 N/ 1n-1180 + 1n, a o on-1 11
(11)
where Xo = (x,,) denotes the Tweedie's 's estimate obtained with Sy, i.e,, a trained S²SB.
When using the OT-ODE version of PSB. we replace sampling from the posterior with a determin- istic version by following the mean, which yields the update rule
xn- Pa-1Xp + 1
(12)
By converting the Tweedic's estimate to the conditional score using Eq. 3 and applying Bayes' Theorem, are left with
X = 1n-1 (x, + a V. x. logp(xn, y)) + P
(13)
Wn-1 (xn, o V x. logp(xn, n) o. Vx,
x, logply Xn, Xx, n)) Pn-1
where Vx,, p(x,, n2) can be approximated by standard IB network trained on the task of inpainting. By manipulating Eq. (13) further, one can arrive at the following update rule
Xn-1 = -nn-1 (x,, + oh V x.. logp(xn, m)) + Hnn 10 logply Xn, n) + Mnn-xx,
Mn-1 (x,, + o V x. logp(x,,,) n)) + fn-11 1n--10 Vx, logp(y Xn. n) xn (14)
An-1
=nn-1 (Xn. + of x,, (og,(xx,,))) jun11 (Gn Vx,, logp(y Xn, n) + xn.
Here, we explicitly define the time-dependent coefficient Cn. While plugging Vx,, log fly x,) in place of Vx., log ply Xa,n) n) in Eq. (33 is the most intuitive, we empirically verified that replac- ing en vx ply ply n) with Vx. log foy x.,) leads to more semantically meaningful results. Practically, this can be explained by fn-1 achieving its highest values at the end of the generation process, effectively incorporating the classifier's signal to the highest extent in the final steps of the generation. Since we are interested in influencing the generative trajectory with the classifier f along the entire process (and possibly decreasing its influence to the greatest possible extent in the final timesteps to avoid adversarial changes), it seems intuitive that incorporating f into Eq. (14) allows for obtaining more meaningful RVCEs. This is due to fr, = 1 pa, meaning that the classifier's signal is amplified in the beginning of the generation and decreased in the final steps. This interven- tion also explains the effectiveness of the introduced improvements, as they break the independence of the classifier's signal between consecutive steps, practically incorporating the time-dependent coefficient G, into the gradient alignment.
E.3 ANALYTIC POSTERIOR AND OT-ODE
Following the original work of Liu et al. (202aa) (FSB), the analytic posterior from the forward stochastic process, which governs the mapping between given boundary pair (xo, x1), is defined
as
[xx4/x0 x1) = N (m(xo, x1, = Xu + t(x, xo), E, = at - -))I.
(15)
where by default a = 1. To arrive at the OT-ODE version of 1SBB, one must use - 0, effectively reducing q to a Dirac delta distribution centered at (x X1, t).
F EXTENDED EXPERIMENTS
We follow the evaluation protocol from previous works for VCEs on Imagenet, which, for a given task, uses all images from the training subset correctly predicted by the evaluated model. For ResNet50, this results in around 2000 images per task. We extract the results of other methods from the work of Farid et al. (2023). except the DVCE method (Augustin et al. (022), which eval- utes with a protocol that we were not able to fully reproduce. Hence, to ensure fair comparison, we adapted the implementation of DVCE to our evaluation. Specifically, utilize the multiple-norm
23

---

Published as a conference paper at ICLR 2025
robust ResNet50 from the work of Borriko al. (2022) which the authors of DVCE propose as default, to achieve VCEs for the ResNet50 model. In terms of hyperparameters, we fine-tuned them with grid search on subset of Sebra-Sorrel task and used = 18.0 the guidance scale for the non-robust ResNet50, since it performed the best.
For SB, we utilize the original checkpoint from Liu et al. (2023a) trained 20 30% freeform masks from Saharia et al. (022).. While the checkpoint trained on the 10 - 20% variant is also available and verified to work within our framework, we discovered that the former generalizes well to smaller area values. Hence, for the sake of simplicity, we utilize the 20 - 30% version only. By default, we use NFE=100, which we explored the most, but lower NFE regimes provided promising initial results. For the automated region extraction, we use IG by default, but evaluate 10 other attribution methods Appendix F5
F. DETAILS OF INDIVIDUAL EXPERIMENTS
Fig. Each improvement, together with the naive approach, is evaluated on the zebra-sorrel task with around 2000 images from ImageNet training set (following the protocol from the main exper- imental evaluation). Each image is initially predicted as either zebra or sorrel by the ResNet50 (He al. 2016) model and the decision must be flipped to the opposite class. FID is computed between the obtained explanations and original images. The hyperparameter values used for all improve- ments are: a = 0.3, s = 1.0 (except ADAM stabilization with s = 1e - 2). c = 16, T = 1.0 (except trajectory truncation, where
0.6).
Table 2 A: For all tasks, we use s = 1.5 and r = 0.4 (to better preserve the original content). and not
As we cannot control the area of masks provided by LangSAM, hyperparameters applicable in this scenario. Images with masks covering area greater than 40% of the total image are discarded from the evaluation ensure that we only the 10% 20% experiment uses configuration B from Table [ while the 20% - 30% experiment uses contiguration C. Hyperparameters a and e are not applicable, since masks are provided auto- matically from the mentioned dataset. C: Each inpainting algorithm is given a 24 A100 GPU hours time budget, resulting around 2000 images for DDRM, 800 images for MCG and 400 images for RePaint on each task. Details of their adaptations are provided separately in Appendix F3
meaningful RVCEs. B: Across all tasks,
F.2 METRICS DESCRIPTION
In the following, we provide detailed description of each metric used in the quantitative evaluation.
FID and sFID (realism). Following works on image synthesis, measuring the realism of the ob- tained explanations at a distribution level is often done with FID and sFID (Heusel et al., 2017).
Specifically, FID compares a set of real (r) and generated (g. in this case, the explanations) images
by first extracting their corresponding features from the Inception network (Szegedy et al. 2016) and then computing
FID
||, - 11 + Tr 1E,+E, (E,E,))
(16)
where prr Bg are the mean vectors and E,, Eg are the covariance matrices of the respective distri- butions in the feature space. As comparing original images with their edited versions (e.g., expla- nations) may bias the metric with original pixels mostly unchanged, artificially boosting the realism evaluation, sFID first divides the sets into folds and averages FID over the independent counterparts.
S (representation similarity).
Explanations should also resemble original images from a representation respective. Here, follow- ing the work of Jeanneret et al. (2023), we compute average SimSiam Similarity (S$) over a set of original images and the resulting counterfactuals. Specifically, S utilizes SimSiam network (Chen & Hc. 202T). which encodes both the factual and counterfactual images into their respective representations ry, ref and computes the cosine similarity
S
rf Tef
(17)
|rylla
||affl22
COUT (sparsity). In the context of VCEs, sparsity is understood as perturbing a minimal number of pixels to flip the model's decision. To quantify this criterion, the COUnterfactual Transition (COUT)
24

---

Published as a conference paper at ICLR 2025
metric computes
COUT AUPC, - y
AUPC,
1 M
Σ (f(k |x(m))= f(k: 1m)+
(18)
where x()) is the factual image, x (M9 the resulting VCE, and y", y are the class labels predicted by f for x0)), ((M respectively. In practice, COUT measures how fast the classifier's decision changes when interpolating between the original and the explanation, but the interpolation is defined as inserting pixels to the original image according to the extent (absolute value) change observed in average over set of samples.
the VCE through .M steps. COUT is typically reported
Flip Rate (efficiency). major criterion for a VCE method is its efficiency, understood as the ability effectively flip the model's decision. For set of triplets {x], X2, Ms ) where x is the original image and x; is the resulting VCE targeted to flip f's decision to yt. Flip Rate (FR) is defined as the fraction of cases which correctly flipped the decision the target class, i.e..
Σ (arg max f(y xx) - yz).
FR =
(19)
V
F.3 ADAPTATION OF OTHER INPAINTING ALGORITIIMS
In this subsection, we describe the adaptation of each inpainting algorithm from our ablation study. For each method, we follow the notation from its corresponding original work to make the descrip- tion easier to follow.
F MANIFOLD CONSTRAINED GRADIENT (MCG, CHUNG AL.. (2022)
MCG iteratively denoises (inpaints) the missing parts with the following two-step update (Fquations
14 and 15, Chung et al. (022
X/_1 = f(xi, So)
[W( - (-o((+x))) (gx(()) ~~0((,11
(20)
Oxt
Xi-t = Ax'-1
(21)
where Eq. (20) is a manifold constraint update and Eq. 21 is data consistency step. As described by the authors, both steps are crucial ensure that the gradient of the measurement term stays on the manifold and to deal with the potential deviation from the measurement consistency.
Since f(x,, se) implicitly predicts the mean He and variance d, each step f. related to the under- lying SDE dynamics, we apply our guidance scheme by modifying the original
f(x,, sa) Mr + oiz, z ~~~00 10011
(22)
by adding properly scaled (according to the relationship between the likelihood score and mean) conditioning:
f'(x; So) 14 + 0, of Br N~ N(0,1
(23)
where gn and g obtained described in Algorithm
F.3.2 DENOSING DIFFUSION RESTORATION MODELS (DDRM, KAWAR ET AL. 2022)
DDRM considers the SVD decomposition of the measurement model matrix H in the linear noisy inverse problem
y Hx T, NO.
(24)
25

---

Published as a conference paper at ICLR 2025
where By is the standard divination of the measurement noise. For the task of inpaiting, the H matrix a diagonal matrix with either 0 or on the diagonal indicating available and missing pixels. Hence, its SVD decomposition simplifies to using identity matrices in place of U and V.
The main contribution of DDRM that it provides a way to include the information from that de- composition and observation y into the generative process, which the authors summarize in Equa- tions? 7 and in the original work. The method uses trained denoising network to obtain a prediction of Xo timestep denoted Xa,r. In order include the additional information from the classifier into the DDRM framework, we modify that prediction to include the model's gradients by replacing the update rule
X V X0,
(25)
with
V (xo
(26)
9
where I and are obtained as described in Algorithm 3
F.3.3 REPAINT (LUGMAYR ET AL. 2022
RePaint performs task inpainting by modifying the standard denoising process, where, at each timestep I, the network's input is composed of noised pixels known from the original input sampled directly from and the unknown noisy pixels predicted by the network in the previous timestep. Additionally, to harmonize the two parts of the image, RePaint samples 2; directly from ((1%/2;;1) and repeats the forward procedure. By default, this resampling scheme is repeated 20 times for each of the standard diffusion steps.
In order to incorporate the information from the classifier, we modify the unconditional mean of the posterior pa in the denosing step to conditional one, effectively replacing the mean predictor
eo (x;
(77)
shown in the 7-th step of Algorithm 1 from the original work Luggmarr al. 2022 with
Eo (X,, +
(28)
where ÿ and y are obtained as described in Algorithm 3
F.4 SCUEDULERS FOR GUIDANCE SCALE
Figure 10 visualizes example schedulers used throughout the development of method. Our adaptive normalization technique empirically outperformed all tested schedulers.
0.8
1.0
Guidance stale
2.0
6.4
0.5
Timeslep
(a) Interval scheduler
(b) Exponential scheduler
(c) Gaussian scheduler
Figure 10: Visualization of more complex schedulers used throughout the development of method.
F.5 QUANTITATIVE EVALUATION OF OTHER ATTRIBUTION METHODS
To pick a default attribution method for RCSB, we evaluated it on the Zebra-Sorrel using the RCSB hyperparameter configuration for 11 different attribution methods shown in Table 3 Based on these results, we chose Integrated Gradients (Sundararajan et al., (017)) as the default, since provides the most balanced performance.
26

---

Published as a conference paper at ICLR 2025
Zebra Sorrel
method
5
LRP
7.5
15.5
0.87
0.62
93.6
InpuxXGradient
16.8
0.87
0.73
97.8
DecpLift
9.2
17.0
0.87
0.73
97.9
Integrated Gradients
17.4
Gradienthhap
10.5
18.5
0.87
0.74
97.4
LIME
12.9
20.7
0.85
0.55
88.4
GuidedBackprop
13.8
21.49
0.86
0.72
96.5
Occlusion
13.9
21.7
0.86
0.50
86.0
GraCCAM
14.1
22.15
0.85
0.52
87.1
Guid:&GradCAM
15.1
22.5
0.86
0.71
96.1
Saliency
15.2
23.0
0.86
Table Quantitative evaluation of attribution methods (described in Appendix C on the Zebra- Sorrel task following evaluation protocol.
F.6 DIVERSITY ASSESSMENT
RCSB utilizes the OT-ODE version of the SB, which provides a deterministic mapping between the noisy image and the resulting RVCE. The source of randomness comes from the Gaussian noise inserted into the image in the place of missing pixels the beginning of the inpainting process. In order to examine the diversity of the generated RVCEs, we followed the evaluation procedure from the work of Jeanneret al. 2023). In essence, we compute the mean pair-wise LPIPS metric between two runs with different seeds (used in generation of the Gaussian noise) for our three main configurations of hyperparameters RCSBA, RCSB and RCSBC, For each run, 256 RVCEs were generated. Results are shown in Table g Naturally, decreasing the area hyperparameter limits the extent of possible changes, leading to decrease in diversity, Picking 0.3 results in diversity comparable to values reported by previous works, e.g., Jeanneret et al. 2023).
Zebra Sorrel
Cheetab Cougar
Egyptian Cat Persian Cat
RCSB A
0.067
0.060
0.065
RCSB RCSB
0.096
0.129
0.140
0.137
Table 4: Diversity evaluation using 256 images for each task from experimental protocol.
F.7 COMPUTATIONAL EFFICIENCY ASSESSMENT
What connects previous SOTA SGM-based methods with our work is the use of large U-Net (Ron- neberger et al.] 2015) checkpoints for the denoising network with the number of hyperparameters far exceeding (e.g., 10x) the size of the utilized classifier, effectively dominating the computational burden. Hence, to ensure fair comparison, Table shows the number of Neural Function Evalua- tions (NFEs) used by each method to produce single explanation, divided into a. model (U-Net, classifier and other) b. and forward / backward passes, where the backward pass is around 2x more computationally demanding than the forward pass. Importantly, this comparison eliminates the dif- ferences stemming from the utilized hardware and optimality of the implementation, and is fair as each method consumes virtually the same amount of GPU memory. One exception to the use of the standard U-Net model is the LDCE method Farid et al. (2023). which applies it in the latent space of an autoencoder. However, as each latent U-Net step also requires decoding the image with the decoder. the computational demand stays similar to the standard approach.
As indicated by Table 5 RCSB is the most efficient approach, both terms of balancing the use of the U-Net and the classifier, and the number of forward/backwant passes. Importantly, the other category shows non-zero numbers only for the DVCE method, which additionaly uses the gradients of robust classifier in the generation process. The high number of forward/backwan passes through the classifiers in DVCE stems from applying them to a set augmented versions of x, at each timestep.
27

---

Published as a conference paper at ICLR 2025
NFE
Inpainting method
U-Net
Classifier
Other
Forward
backward
forward
buckward
RCSB
100
100
100
0
LDCE
191
191
191
191
0
0
DVCE
200
200
1600
1600
1600
1600
ACE
520
500
25
25
0
DDRM
200
200
200
200
0
0
MOG
1000
1000
RePaint
2110
2410
2110
2110
0
0
Table 5: Number of NFEs for each respective method with details about the model type and for- ward/backward passes.
F.8 ADDITIONAL QUANTITATIVE RESULTS
For most visually appealing results, we found a 0.1 0.15, c Rs 16, T 0.3 0.6 and =s 2 3 to perform the best. These hyperparameters were used to create RVCEs for Fig. 5 To assess that the performance
these configurations does not deviate from best configurations of Table 1. we followed the same evaluation protocol on the most challenging Zebra-Sorrel task and include the results in Table 6 Crucially, the performance stays virtually the same when comparing with Table
Area
Cell size
Guidance scale
Trajectory truncation
FID
sFID
COUT
FR
0.2
10.1
18.4
0.92
0.79
95.8
3.0
0.3
10.7
19.0
0.91
0.76
95.0
0.4
10.8
18.9
0.90
0.74
94.3
0.2
11.0
19.2
0.91
0.81
97.0
3.5
0.1
11-4
19.4
0.89
0.77
96.0
0.2
11.7
19.8
0.91
0.83
97.2
4.0
0.4
12.3
20.2
0.88
0.79
96.77
0.3
12.4
20.4
0.89
0.80
96.2
0.4
11.2
19.2
0.89
0.77
97.7
12.1
0.6
12.7
20.4
0.86
0.70
94.8
0.15
0.4
13.5
21.3
0.87
0.82
99.5
3.0
0.5
13.9
21.7
0.86
0.80
99.2
0.6
14.2
0.85
0.78
98.6
15.3
4.0
0.5
15.5
23.1
0.85
0.82
99.7
0.6
15.8
23.5
0.83
0.81
99.4
Table 6: Quantitative results for hyperparameters that provide the most visually appealing results.
F.9 FREEFORM MASKS
Figure 11 presents example RVCEs for the sorrel zebra task from the experiments based freeform masks with quantitative results T)) We observe that RCSB focuses on modifying the intersection of the randomly assigned mask with features that should intuitively be important to the classifier, while leaving the unimportant parts, like background and sky, mostly unchanged. This provides additional justification for high performance of RCSB despite not exclusively focusing the most important regions.
F.10 UNINTUITIVE CLASSES
shows RVCEs obtained for unintuitive class pairings. Interestingly, RCSB is able to largely preserve the realism of the explanations, while providing unusual compositions of objects, e.g. placing maltese dog in place of cauliflower.
Figure 12
28

---

Published as a conference paper at ICLR 2025
F.11 SHAPE MODIFICATION
While VCEs, and RVCEs in particular, are focused on providing the minimal semantic change that modifies the classifier's s decision, one may be interested in obtaining new content that deviates further from the original. Figure [ example RVCEs focused on modifying the original image to a larger extent, leading to shape and contour changes. By increasing the area (a) of the region constraint and the trajectory truncation (r), RCSB allows for weaker preservation of the original content, leading to more visible changes. Modifying the shape of the original objects is possible in various scenarios. For example, in the white stork + black stork task, despite the fact that the bird's color is the dominant differentiating feature, guiding RCSB with the classifier of interest can also lead to large shape changes. This is also visible in the vulture - flamingo task, where the bird's legs appear thinner and longer, while its modified head points to an opposite direction. Moreover, ImageNet contains task that are mostly characterized by shape differences rather than color or texture changes. Figure 13 shows that the primary characteristic of a pretzel can be easily modified to change the model's decision to bagel. The same can be seen when changing the decision from hatchet to hammer, where the latter's back part becomes longer than the original. Moreover, objects like paperknife and spoon can be easily modified with RCSB be predicted as spoon and ladle respectively through realistic shape changes.
To assess the effectiveness of RCSB in tasks, where the classifier's s decision should mostly depend on the objects shape rather than texture or color, we evaluate it on another three tasks: pretzel-tbagel, hatchet-hanmel and paperknife- wooden spoon. To compare with previous SOTA on ImageNet, we evaluate DVCE in the same scenario. The hyperparameters of both methods and experimental details follow those from the main evaluation protocol in Table [I] Results from Table 7 show that, despite the different nature of the considered tasks, RCSB preserves its performance and advantage over DVCE from Table I
Pretzel
Rogel
Hatchet
Hammer
Paperknile
Wooden spoon
Method
FED
sFID
COUT
FR
FID
sFID
$3
COUT
FID
sFID
$²
COUT
FR
DVCE
34.3
43.9
0.59
0.37
77.4
31,2
39.8
0.66
0.43
92.8
29.1
35.4
0.69
0.41
88.2
RCSB
11.4
22.9
0.86
0.84
97.2
9.8
15.4
0.91
0.89
91.8
9.9
182
0.36
0.88
98.9
Table 7: Quantitative results for tasks focused on characteristics connected to shape instead of tex- ture or color. RCSB is compared to DVCE, which is regarded as current SOTA ImageNet.
We also address the topic of shape modification further in Appendix F.14, where RCSB is evaluated on the MNIST dataset. In grayscale handwritten digits. individual classes primarily identified based on the shape of samples, hence serving as a proper proof-of-concept benchmark for evaluating the understanding of shapes by the method.
F.12 LOWER-LEVEL ATTRIBUTIONS
To verify whether RCSB is able to effectively synthesize RVCEs based pixels that are considered less important to the classifier, we extract the regions using our automated approach by first zeroing out the absolute attributions above some quantile q before converting it to a binary mask with the approach mentioned at the end of Section
Table g shows the results of this experiment performed on the three main tasks from Ima- geNet, where we follow the default protocol from the main part of this paper and use the RCSB hyperparameter configuration from Table [ We evaluate the RVCEs resulting from E {0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}. where the extracted region covers 10% of total image area. Crucially, the presented results showcase that RCSB's is largely preserved despite using pixels which should influence the classifier much more weakly. FID and sFID both decrease monoton- ically with respect to q, which indicates that an increasing number of pixels from the background gets modified, resulting in a smaller number of changed data characteristics. S remains mostly unchanged, meaning that the representational similarity is not influenced by varying q. Intuitively, both COUT and FR also decrease when picking smaller q. This is because RCSB is not able to utilize the most influential pixels for the classifier, which makes the task harder, thus lowering both the sparsity (changing most influential pixels) and efficiency of flipping the model's decision.
29

---

Published as a conference paper at ICLR 2025
Zehra Sorrel
Cheetab Cougar
Egyptian Cat - Persian Cat
Metric
FID
<FID
5
COUT
FR
FTD
<FTD
5
COUT
FR
FID
.RTD
5*'
COUT
FR
0.9
5.2
13.8
0.57
0.62
91.6
6.1
17.1
0.92
0.88
99.8
13.3
31.4
0.86
0.89
99.9
38.0
4.4
0.80
99.1
11.0
28.6
0.84
99.3
12.7
85.7
3.5
0.74
26.9
98.5
0.6
3.6
12.4
0.89
0.45
81.4
2.9
14.6
0.92
0.68
91.3
7.8
25.5
0.85
0.73
96.1
0.5
3.6
12.4
0.89
0.40
78.5
2.7
14.5
0.92
0.65
93.1
6.9
24.7
0.86
0.69
94.6
12.3
71.7
2.8
0.92
0.63
93.4
6.6
24.3
0.65
95.1
12.6
0.40
79.2
3.1
0.92
0.69
95.7
25.3
0.67
94.2
Table Quantitative results for lower-level attributions on three main tasks from ImageNet. Here, q denotes the value of the quantile above which absolute attributions are zeroed out before extracting a region with 10% area coverage.
F.13 OTHER CLASSIFIERS
In Table 9 provide quantitative results for other classifiers mentioned at the end of the experi- mental evaluation. The experimental setting follows the same protocol as the one from Table and and we use the RCSB configuration. Note that the results are consistent with those for ResNet50 in Table for all classifiers except the robust Madry ResNet50. Since we use single hyperparameter configuration, this probably due to the different nature of the model, and the results could be easily improved by tuning specific configuration for it.
Zebra Sorrel
Classifer
FID
sFID
COUT
FR
ChipeerSShot
4.13
12.76
0.90
0.93
100.0
ConsNeXiBase
15.69
23.55
0.84
99.8
MasrtRooNet50
47.49
55.22
0.65
-0.19
36.2
RBDeIT
10.00
17.76
0.83
0.70
94.0
RAXCIT
16.04
23.45
0.79
0.46
83.6
SwinB
3.20
12.19
0.94
0.50
$8.0
VGGI6
15.39
0.81
VGGI6 BN
5.44
13.53
0.91
0.87
99.9
VITB16
8.00
16.84
0.86
0.80
98.9
Table 9: Quantitative results for other classifiers evaluated on the most challenging (out of the three considered) zebra sorrel task from ImageNet.
F.14 OTHER BENCIMARKS
We extend the evaluation of RCSB with three additional datasets: CelebA-HQ Karras et al. 2018 with 000 samples of 256 x 256 resolution face images, CelebA (Liu et al. 2015) with around 200 000 samples of 128 x 128 resolution face images, and MNIST (Deng) 2012) with 70 000 samples of 32 x 32 resolution images of handwritten digits. The first two datasets are chosen to compare with all previously published diffusion-based approaches that did not evaluate ImageNet, effectively complementing our experimental results. While both of these datasets contain face images, they pose unique challenges, since CelebA-HQ contains much less samples (~ 6x) which are also of higher resolution, making the predictive tasks very different. The MNIST dataset was chosen to provide additional proof of the versatility of RCSB. While this dataset no longer in active use as an evaluation benchmark, we include it here to show that even the data of much different nature and resolution, RCSB is still able to provide meaningful and informative RVCEs that must focus on modifying the shape of a digit, since there is no notion of 'texture' present in the data.
For both CelebA and CelebA-HQ, we follow previous works and provide explanations for the DenseNet121 (Huang et al., 2017) model trained in multilabel scenario consisting of 40 distinct attributes. We consider two tasks evaluated prior works, i.c., flipping the smile and age classes to the opposite prediction. For MNIST, we train LeNet (Lecun et al. 1998) from scratch using the default training and validation splits. We note that, for each considered dataset, I2SB must be trained independently for the task of inpainting (on 20% 30% freeform masks). We start its training using a pretrained diffusion checkpoint a. from Lugmayr et al. (2022) for CelebA B-Q with default train- ing hyperparameters from Liu et al. (2023a) and 40 000 iterations, b. from Jeanneret al. (0022)
30

---

Published as a conference paper at ICLR 2025
for CelebA with default training hyperparameters from Liu et al. (2023a) and 100 000 iterations and c. from scratch on MNIST.
To provide more comprehensive comparison, also adapt the implementation of DiME (Jean- neret et al, 2022) to ImageNet and FastDiME to both CelebA-HQ and ImageNet. We first tune the hyperparameters of both methods with a large grid search uate the best configuration using the standard protocol for ImageNet (i.e., the same as our method) and 2048 samples from CelebA-HQ. Both DiME and FastDiME are implemented with the of the same checkpoints that the training of 12SB starts from, i.e. from the work of Lugmayr et al. (2022) for CelebA-HQ and Dhariwal & Nichol (2021 for ImageNet.
small subset of images, and then eval-
Method
Zebra Surrel
DIME FasDiME
222.85
243. 16
0.19
-0.31
0.0
96.48
103.45
0.22
-0.44
RCSB
8.0
16.2
0.88
0.74
94.7
Cheetah Cougar
DIME
268 22
291.99
0.11
-0.16
0.0
FasIDiME
133.01
141.12
0.12
-0.11
18.0
RCSB
17.2
26.6
0.92
0.92
100.0
Egyptian Cat Persian Cat
DIME
322.79
352.08
0.44
-0.05
FastDiME
193.63
207. 12
0.10
0.01
20.0
RCSB
23.0
40.0
0.87
0.92
100.0
Table 10: Quantitative results on the ImageNet dataset for DiME and FastDiME.
In terms of evaluation measures, we follow previous works (Jeanneret et al. 2022; 2023 Weng et al., 2024 and utilize FVA, FS (Cao et al. 2018), MNAC (Rodríguez et al., 2021) and CD (Jeanneret
al. 2022 in addition metrics used on ImageNet.
Tables and show the quantitative results achieved by all considered methods on CelebA and CelebA-HQ. Importantly, RCSB is able to outperform the current SOTA in many cases. For exam- ple, it provides new records for COUT, MNAC and FR on all considered (dataset, class) pairs. While high COUT and FR are expected based the method's performance on ImageNet, where it is able to efficiently generate very sparse RVCEs (with respect to the classifier), low MNAC additionaly small subset of face attributes and leaves others mostly unmodified. Our approach is also able to obtain very low FID and sFID values, often performing worse than ACE only, which indicates that the obtained explanations preserve the realism of the original samples. We include example RVCEs obtained with RCSB on CelebA-HQ in Fig. 14
shows that RCSB focuses
Suile
FID
COUT
FVA
DIVE DyvgAoo
20.4
33.8
36.5
73.4
4.63
2.34
30.9
52.2
4,27
10.3
96.9
11.8
97.3
ACTI dg
1:21
3.97
9099
0.89
2.84
1.23
0.78
97.6
1.48
4.13
92A
0.75
1:30
2.94
0.72
96.2
ACB 4g
1.90
4.56
90.0
0.83
2.37
1.56
0.2
66.5
2.06
4.62
92A
0.80
3.84
286
0.56
17.5
DIME
3.17
4.59
98.3
0.73
3.72
2.30
0.3
97.0
4.15
5.89
85.3
0007
1
3.27
0.68
SAL0
PastiM::
4.1%
6.15
9938
0.26
3.12
1.
0.14
433
0.76
99.3
0.74
3.65
3.00
0.56
98.6
FanDsME-2
3.33
3
99.6
0.75
RaIDME-
3.24
09.9
2.91
2,0
0.41
98.9
3
99.7
0.70
Table 11: Quantitative results on the CelebA dataset. We extract the results of other methods
RRodriuuez et al. 2021 Jacob et al., 2022 Jeanneret et al. 2022 2023) from the work Wenn
et al. (2024).
Regarding the MNIST dataset, we provide example RVCEs obtained with RCSB using the auto- mated region extraction for various tasks in Fig. 15 Crucially, the presented samples show that, despite the complexity of the detected regions, RCSE able to properly modify the shape of the ini- tial digit change the classifier's decision. As can be observed, this is performed through hybrid
31

---

Published as a conference paper at ICLR 2025
Age
Mrlhed
FID
FVA
FS
MNACC
CD
FR
FID
4111
PVA
BS
MINAC
CDUT
FR
DAVE
10510
157
7,41
100.5
12.1
6.76
STREX
21.9
92.5
5.21
26-5
96.0
5.67
32.
103.0
633
22.0
1630
0.88
26.2
LOCE
25.8
59.1
0.76
1.68
14.3
25.6
FaIDMB-2
16.51
31.4
99.9
0.87
1.43
6.16
0.28
87.1
26.0
403
995
0.61
3.15
4.36
0.31
92.6
RC53
3,04
20.0
1000
0.53
1.22
3.22
0.53
96.9
4.92
21.3
1000
0.96
1.47
5.16
0.80
99.4
Table 12: Quantitative results on the CelebA-HQ dataset. We extract the results of other methods from the work of Farid et al. (2023) and Jeanneret al. (2023) except FastDiME which we imple- ment and evaluate ourselves.
approach, where some parts of the initial digit remain the same, while new parts appear to either combine the existing elements of the digit or create entirely new ones.
G
USER STUDIES
The main goal of VCEs, and RVCEs in particular, is to explain the model's reasoning humans. This capability can be evaluated from various perspectives. In the following, we provide a detailed analysis of two independently conducted user studies, with the first one focused on the general usefulness of RVCEs in understanding the model's -aainng and the potential benefits stem- ming from a possible interaction of humans with the explanation creation process, and the second one concerned with specific use-case, where RVCEs are used to inform the user about the causes of model's miselassification and what must be changed in a given image for it to predict correctly. For both studies, the model of interest is the ResNet50 used in main experimental evaluation and the samples are extracted from ImageNet.
G.1 I: USEFULNESS AND INTERACTION
In this study, 15 participants with background knowledge in machine learning (at the level of MSc studies, not aware of the research conducted for this paper) were presented with comparisons of VCEs and RVCEs for the same factual images, together with absolute differences between the orig- inal and the explanation. An example of such comparison is included in Fig. [6 The participants were asked about which type explanation is more useful in understanding the model's 's decision- making. Here, 86.6% answered in favor of RVCEs. Moreover, each user was asked provide the reasons for their judgement. The answers generally focused on the semantic change being more localized, easier to interpret and better aligned with human intuition.
The second part of this study focused on evaluating the added value provided by the possibility of interacting with the explanation creation process through manual region specification. First, the participants were shown default interaction with VCEs, i.e., the original image and its VCE were presented to users with no possibility of interacting with Then, the participants were presented with the process of manual region specification for which an RVCE was generated, and offered the possibility to provide the region themselves. After that, each participant was asked whether the in- teractive process may be helpful and more useful in obtaining better understanding of the model's reasoning than the standard scenario. There 93.3% of participants answered in favor of the inter- active process. They were also asked to justify their choice, and the answers generally focused on the possibility of verifying regions that align with human understanding and incorporating domain experts that would be able to more thoroughly analyze the model.
In both parts, RVCEs were extracted from the figures in the main part of the manuscript and DVCE was used generate standard VCEs, since this method provided the best quantitative results prior to our work.
G.2 H: UNDERSTANDING MODEL'S FAILURES
In this study, a different group of 11 participants with background knowledge in machine learning similar level took part in evaluating whether RVCEs help in identifying the reasons for model's
32

---

Published as a conference paper at ICLR 2025
misclassifications. The study consisted of a general introduction to the problem of explaining deep classifiers with VCEs, the concept of RVCEs and the two-fold goal of the study: to understand why given model misclassifies the image and to identify the minimal semantic change required to cor- rect the prediction. Then, the participants took part in 5 variations the same experiment, where in each case different misclassification the factual image, the initially predicted class, the correct class and two sets of images, each rep- resenting instances of one of the two classes taken randomly from the web (see Fig. 17 left) for an example introduction to the experiment). Then, each participant observed the original image, the region constraint from our automated approach and the resulting series of RVCEs (see Fig. righht) for example). After each of the 5 variants of this experiment, the participants were asked whether they were able to identify the semantic features that the model lacked in initial prediction and that lead to correcting the decision once they appeared on the image. The response was, on average, positive in 80.02% of the cases with 14.56% standard deviation. In each case, they were also asked to describe these features. Here, the answers almost always aligned with what the RVCEs were introducing to the image. After all experiments, the participants were asked if RVCES are able to indicate the semantic features that were missing in the beginning for the model to predict correctly (90.9% positive answers), whether they better understood the initial misclassification (90.9% pos- itive answers) and if they judge RVCE as useful tool in explaining the model's decision-making (100% positive answers).
shown. The experiment began with presentation of
The conducted user studies highlight that the concept of RVCEs and the application RCSB their generation is preferred by the users in comparison to standard VCEs. Our explanations are found to be more useful and helpful in explaining the model's decision-making. The possibility of interacting with the explanation creation is also enjoyed and recognized to improve the explanatory process by almost all participants. Moreover, our method helped the users in obtaining better understanding of the classifier's 's failure cases and its potential causes.
H QUALITATIVE EXAMPLES
We provide additional qualitative examples for different scenarios. Results for other classifiers,
obtained with the automated region extraction, are depicted in Fig. (VGG16, Simonyan & Zis-
serman 2015)). ). Fig. 22 (VGG16 with Batch Normalization (BN), Simonyan & Zisserman (0155) ).
Fig. 23 ConvNeXt Base, Liu ct al. (2022)). Fig. 24 (ViT-B/16, Dosovitskiy et al. (2021), ), Fig. 25
(SwinB, Liu al. (2021)
Fig. 26 (robust Madry ResNet50, Engstrom et al. (019)) Fig.
bust Tian DeiT. Jan et al. (2022).
2).. Fig. 28 (zero-shot CLIP classifier, Radford et al. (2021). ). We
used a 0.2, r 0.7, 16,
1.5 universally across all additional classifiers, showcasing the
versatility of RCSB. Additional examples for ResNet50 are shown in Figs. 29 to 32 (automated re-
gion extraction with different hyperparameters), Fig. 33 (exact regions from LangSAM) and Fig.
(user-defined regions).
33

---

Published as a conference paper at ICLR 2025
Sorrel
Zebra
Image
Region
Explanation
Image
Region
Explanation
Example RVCEs obtained in the sorrel >> zebra task from the experiments based on freeform masks with quantitative results in Table BB) The columns show the factual image, the region to which changes are constrained and the resulting RVCE.
Figure 11:
Canliflower Maltese dog
Honeycomb
Hamster
Missile
Hamster
Purse Hamster
Sunglasses
Hamster
Figure 12: RVCEs for unusual class pairings. Each task of the form initial class target class depicts factual image, the region constraint and the resulting explanation respectively.
34

---

Published as a conference paper at ICLR 2025
White stork + Black stork
Vulture + Flamingo
Pretzel + Bagel
Hatchet + Hammer
Paperknife
Wooden spoon
Spoon
Ladle
Figure 13: RVCEs focused on modifying the shape of objects within the region constraint obtained with the automated extraction approach. By picking larger area (a) and trajectory truncation (r), the preservation of the original content can be reduced, leading to more diverse infills that change classifier's 's decision.
35

---

Published as a conference paper at ICLR 2025
Smiling + Not smiling
Not smiling
+ Smiling
Young
Old
Old + Young
Figure 14: Qualitative examples of RVCEs obtained with RCSB in smiling+ smiling and young-vold tasks on CelebA-HQ.
8 -
9 9-+3 3
8
8
3
9
g
3
3->
-
8
4
4
q
6 5
7-1
6
6
S
7
7
Figure 15: Qualitative examples of RYCEs obtained with RCSB in various higit-flippin tasksk on the MNIST dataset.
36

---

Published as a conference paper at ICLR 2025
Type A
Type B
Image
Explanation
Difference
Image
Explanation
Difference
Prediction: jay
Prediction: bulbul
Prediction: jay
Prediction: bulbul
Difference
Difference
Prediction: guacamole
Prediction: cabbage
Prediction: guacamole
Prediction: cabbage
Figure 16: An example comparison of VCEs and RVCEs presented to participants of the user study.
Juno Brumbling
Preck disse
Figure 17: An example introduction (left) and evaluation (right) from one of the experiments con- ducted in the II user study.
37

---

Published as a conference paper at ICLR 2025
Original
Limpkin + Flamingo
Ours
DIME
FistDiME
LDCE
DVCE
ACE
Figure 18: Qualitative comparison to other VCE generation methods: DiME Jeanneret al. 2022 FastDiME (Weng et al. 2024), LDCE (Farid et al. 2023). DVCE (Augustin et al. 2022) and ACE (Jeanneret al. 2023). For each explanation, the absolute difference between the factual image is additionaly provided.
38

---

Published as a conference paper at ICLR 2025
Original
Mashed Potato Cauliflower
Ours
DIME
FistDiME
LDCE
DVCE
ACE
Original
Cougar + Lynx
Ours
DiME
FistDiME
LDCE
DVCE
ACE
Figure 19: Qualitative comparison other VCE generation methods: DiME (Jaannrett al. 2022 FastDiME (Weng et al. 2024), LDCE (Farid et al. 2023), DVCE (Augustin et al. 2022) and ACE (Jeanneret al. 2023). For each explanation, the absolute difference between the factual image is additionaly provided.
39

---

Published as a conference paper at ICLR 2025
Original
Green Mamba + Indian Cobra
Ours
DIME
FastDiME
LDCE
DVCE
ACE
Figure 20: Qualitative comparison other VCE generation methods: DiME (Jaanneet al. 2022 FastDiME (Weng et al. 2024), LDCE (Farid et al. 2023). DVCE (Augustin et al. 2022) and ACE (Jeanneret al. (023). For each explanation, the absolute difference between the factual image is additionaly provided.
40

---

Published as a conference paper at ICLR 2025
Lemon Orange
Arctic fox Red fox
Cucumber + Zucchini
Guacamole Head cabbage
Ptarmigan Peacock
Broccoli + Head cabbage
Figure 21: Extended qualitative evaluation of automated region extraction for VGGI6 (Simonyan & Zisserman 2015) classifier. For each task, factual image shown on the left with the used region in the middle and the generated RVCE on the right.
Lemon
Orange
Arctic fox Red fox
Cucumber + Zucchini
Head cabbage
Canliflower
Cougar + Lynx
Hornbill Toucan
Figure 22: Extended qualitative evaluation of automated region extraction for VGG16BN (Simonyan
& Zisserman 2015) classifier. For each task, factual image is shown the left with the used region in the middle and the generated RVCE on the right.
41

---

Published as a conference paper at ICLR 2025
Lemon Orange
Lion Tiger
Cucumber + Zucchini
Kit fox
Arctic fox
Congar + Lynx
Head cabhage - Cauliflower
Figure 23:
Extended qualitative evaluation of automated region extraction for ConvNeXt Base (Luu al. 2022) classifier. For each task, factual image shown the left with the used region in the middle and the generated RVCE on the right.
Cougar + Lynx
Cucumber Zucchini
Ptarmigan Peacuck
Kit fox Arctic fox
Arctic fox + Red fox
Carbonara - Guacamole
Figure 24:
Extended qualitative evaluation of automated region extraction for ViTB (Dosovisskiy
al. 2021) classifier. For each task, factual image shown the left with the used region in the middle and the generated RVCE on the right.
42

---

Published as a conference paper at ICLR 2025
Cougar + Lynx
Cucumber Zucchini
Hornbill Toucan
Head cabbage
Canliflower
Arctic fox + Red fox
Guacamole Head cabbage
Figure 25: Extended qualitative evaluation of automated region extraction for SwinB (Liu et al. 2021) classifier. For each task, factual image is shown on the left with the used region the middle and the generated RVCE on the right.
Lemon Orange
Cougar Lynx
Cucumber + Zucchini
Kit fox Arctic fox
Fox squirrel
Guinea pig
Horbbill Toucan
Figure 26: Extended qualitative evaluation of automated region extraction for Madry ResNer50 [Engstrom et al. 2019) l2-norm robust classifier. For each task, factual image is shown the left with the used region in the middle and the generated RVCE on the right.
43

---

Published as a conference paper at ICLR 2025
Lemon Orange
Lion Tiger
PHOTOS COM
Cucumber + Zucchini
Kit fox Arctic fox
Head cabbage + Cauliflower
Cougar + Lynx
Figure 27: Extended qualitative evaluation of automated region extraction for Tian DeiT (Tian et al. 2022) corruption robust classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Lemon Orange
Arctic fox Red fox
Cougar - Lynx
Hornbill Toucan
Common iguana
American chameleon
Limpkin Flamingo
Figure 28: Extended qualitative evaluation of automated region extraction for CLIP ViT-B/32 (Rad-
ford al. 2021) zero-shot classifier. For each task, factual image shown on the left with the used region in the middle and the generated RVCE on the right.
44

---

Published as a conference paper at ICLR 2025
Cheetah + Cougar
Persian Cat - Egyptian Cat
Figure 29: Extended qualitative evaluation of automated region extraction with 4, 0.1 for the ResNe150 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE the right.
Braccoli + Canliflower
Cougar + Cheetah
Bullfrog + Terpian
Goose + Black swan
Figure 30: Extended qualitative evaluation of automated region extraction with 4, 0.3 for ResNet50 classifier. For each task, factual image shown on the left with the used region middle and the generated RVCE on the right.
45

---

Published as a conference paper at ICLR 2025
Cheetah + Cougar
Goose - Black swan
Figure 31: Extended qualitative evaluation of automated region extraction with 4, a 0.2 for the ResNet50 classifier. For each task, factual image shown on the left with the used region the middle and the generated RVCE on the right.
46

---

Published as a conference paper at ICLR 2025
Green mamba + Indian cobra
Duck + Goose
Arctic fox + Red fox
Wooden spoon Ladle
Lemon + Orange
Cougar Lynx
Figure 32:
Extended qualitative evaluation of automated region extraction with c 8, a 0.2 for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
47

---

Published as a conference paper at ICLR 2025
Great white shark + Hammerhead shark (shark)
Congar + Cheetah (animal)
Rabbit Angora rabbit (rabbit)
Fly Bee (fly)
Rock python - Boa constrictor (snake)
Night snake
King snake (snake)
Figure 33: Extended qualitative evaluation of exact regions obtained with LangSAM for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Bee cater Macaw
Vulture Flamingo
Figure 34: Extended qualitative evaluation of user-defined regions for the ResNet50 classifier. For each task, factual image is shown the left with the used region in the middle and the generated RVCE(s) on the right.
48

---

Published as a conference paper at ICLR 2025
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models, volume 12. Stochastic Processes and their Applications, Elsevier, 1982.
Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explanations. In Advances in Neural Information Processing Systems, 2022.
Maximilian Augustin, Yannic Neuhaus, and Matthias Hein. Analyzing and explaining image clas- sifiers via diffusion guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, 2024.
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Rober Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. In PloS one, 2015.
Przemyslaw Biecek and Wojciech Samek. Position: Explain to question not to justify. Proceed- ings of the s1st International Conference on Machine Learning, volume 235, pp. 3996-4006, 21-27 Jul 2024.
Valentyn Boreiko, Maximilian Augustin, Francesco Croce, Philipp Berens, and Matthias Hein.
Sparse visual counterfactual explanations in image space. In DAGM German Conference Pattern Recognition, pp. 133-148. Springer, 2022.
Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018), pp. 67-74. IEEE, 2018.
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image clas- sifiers by counterfactual generation. In International Conference on Learning Representations, 2019.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15750-15758, 2021.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems, 2022.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh Interna- tional Conference on Learning Representations, 2023a.
Hyungjin Chung, Jeongsol Kim, and Jong Chul Ye. Direct diffusion bridge using data consistency for inverse problems. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-255. IEEE, 2009.
Li Deng. The mnist database of handwritten digit images for machine learning research. In IEEE Signal Processing Magazine, 2012.
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. In Advances in neural information processing systems, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Logan Engstrom, Andrew Ilyas, Hadi Salman. Shibani Santurkar, and Dimitris Tsipras. Robustness (python library). In https://githhb.cco http:///ghhub..com/Mt//////// wwtp:///woooooo//aaaaaaaaaa/e//e atbbba/unssrrrlrrrrrnssssssssss 2019.
49

---

Published as a conference paper at ICLR 2025
Karim Farid, Simon Schrodi, Max Argus, and Thomas Brox. Latent diffusion counterfactual expla- nations. In arXiv preprint arXiv:2310.06668, 2023.
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. Counterfactual visual explanations. In International Conference on Machine Learning, pp. 2376-2384. PMLR, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. In Advances in neural information processing systems, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in neural information processing systems, 2020.
Andreas Holzinger, Anna Saranti, Christoph Molnar, Przemyslaw Biecek, and Wojeiech Samek. Explainable AI methods-a brief overview. In International workshop on extending explainable Al beyond deep models and classifiers, pp. 13-38. Springer, 2022.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
2017.
Paul Jacob, Éloi Zablocki, Hedi Ben- Younes, Mickaël Chen, Patrick Pérez, and Matthieu Cord. Steex: steering counterfactual explanations with semantics. In European Conference Computer Vision, pp. 387-403. Springer, 2022.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Diffusion models for counterfactual explana- tions. In Proceedings of the Asian Conference on Computer Vision, pp.
2022.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Adversarial counterfactual visual explana- tions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16425-16435, 2023.
Guillaume Jeanneret, Loic Simon, and Frédéric Jurie. Text-to-image models for counterfactual explanations: black-box approach. In Proceedings of the IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 4757-4767, 2024.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for im- proved quality, stability, and variation. In International Conference on Learning Representations, 2018.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. In Advances in Neural Information Processing Systems, 2022.
Saeed Khorram and Li Fuxin. Cycle-consistent counterfactuals by latent transformations. In Pro- ceedings of the IEFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10203- 10212, 2022.
Ba Jimmy Kingma, Diederik P. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2019.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafionn, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pp. 4015-4026, 2023.
Andreas Kirsch. An introduction the mathematical theory of inverse problems. Applied mathe- matical sciences. Springer, New York, 2nd ed edition, 2011.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya. Siqi Yan, al. Captum: A uni- fied and generic model interpretability library for pytorch. In arXiv preprint arXiv:2009.0889 2020.
50

---

Published as a conference paper at ICLR 2025
Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William Freeman, Phillip Isola, Amir Globerson, Michal Irani, et al. Explaining in style: Training a gan to explain a classifier in stylespace. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 693-702, 2021.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-baid badient-baase learning applied to document recog- nition. In Proceedings of the IEEE, 1998.
Guan-Homg Liu, Arash Vahdat, De-An Huang, Evangelos Theodorou, Weili Nic, and Anima Anandkumar. 12SB: Image-to-Image Schrödinger Bridge. In International Conference Ma- chine Learning, pp. 22042-22066 PMLR, 2023a.
Shilong Liu, Zhaoyang Zeng. Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In arXiv preprint arXiv:2303.055 00:2333,. 2023b.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference computer vision, pp. 10012-10022, 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1 11976-11988 1996, 2022.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 11461-11471, 2022.
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Pro- ceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
Robert J. McCann. A convexity principle for interacting gases. In Advances in Mathematics, 1997.
Franz Motzkus, Christian Hellert, and Ute Schmid. Cola-dce- concept-guided latent diffusion coun- terfactual explanations. In arXiv preprint 2406:22000.11449 2024.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In Advances in neural information processing systems, 2019.
Gabriel Peyré and Marco Cuturi. Computational optimal transport: With applications to data sci- ence. Foundations and Trends in Machine Learning. 11(5-6):355-600 2019.
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasor Suwajanakom. Dif- fusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10619-10629, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763, PMLR, 2021.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Herbert Robbins. An empirical Bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory, 1992.
51

---

Published as a conference paper at ICLR 2025
Pau Rodríguez, Massimo Caccia, Alexandre Lacoste, Lee Zamparo, Issam Laradji, Laurent Char- lin, and David Vazquez. Beyond trivial counterfactual explanations with diverse valuable ex- planations. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp- 1056-1065, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, part III 18, pp. 234-241. Springer, 2015.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Jonathan Scarlett, Reinhard Heckel, Miguel RD Rodrigues, Paul Hand, and Yonina C Eldar. Theo- retical perspectives on deep learning methods in inverse problems. In IEEE journal on selected areas in information theory, 2022.
Erwin Schrödinger. Sur théorie relativiste de l'électron et l'interprétation de la mécanique quan- tique. In Annales de I' l'institul Henri Poincaré, volume 2. pp. 269-310, 1932.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems, 2022.
Lisa Schut, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. Gener- ating interpretable counterfactual explanations by implicit minimisation of epistemic and aleatoric uncertainties. In International Conference Artificial Intelligence and Statistics, pp. 1756-1764 PMLR, 2021.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: visual explanations from deep networks via gradient- local- ization. In International journal of computer vision, 2020.
Sheng-Min Shih, Pin-Ju Tien, and Zohar Karnin. Ganmex: One-vs-one attributions using gan-based
model explainability. In International Conference on Machine Learning, pp. 9092999999999 9602. PMLR, 2021.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference on machine learning, pp. 3145- 3153. PMLR, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference Learning Representations, 2015.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In 2nd International Conference on Learning Representations, Workshop Track Proceedings, 2014.
Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by pro- gressive exaggeration. In International Conference on Learning Representations, 2020.
Bartlomiej Sobieski and Przemyslaw Biecek. Global counterfactual directions. In European Con- ference Computer Vision, 2024.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Scsee-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021.
52

---

Published as a conference paper at ICLR 2025
Jost Tobias Springenberg. Alexey Dosovitskiy. Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. International Conference on Learning Representations, Workshop Track Proceedings, 2015.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328, PMLR, 2017.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Jayaraman Thiagarajan, Vivek Sivaraman Narayanaswamy, Deepta Rajan, Jia Liang, Akshay Chaudhari, and Andreas Spanias. Designing counterfactual generators using deep model inver- sion. In Advances in Neural Information Processing Systems, 2021.
Rui Tian, Zuxuan Wu, Qi Dai, Han Hu, and Yu-Gang Jiang. Deeper insights into the robustness of vits towards common corruptions, In arXiv preprint vrXiv:220.112143 2022.
Paulina Tomaszewska and Przemystaw Biecek. Position paper: Do not explain (vision models) without context. In International Conference on Machine Learning. 2024.
Philipp Vaeth, Alexander M Fruehwald, Benjamin Paassen, and Magda Gregorova. Gradcheck: Analyzing classifier guidance gradients for conditional diffusion sampling. In arXiv preprint arXiv:2406. 17399, 2024.
Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by pro- totypes. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 650-665. Springer, 2021.
Nina Weng, Paraskevas Pegios, Aasa Feragen, Eike Petersen, and Siavash Bigdeli. Fast diffusion- based counterfactuals for shortcut removal and generation. In European Conference on Computer Vision, 2024.
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pp. 818-833. Springer, 2014.
Mehdi Zemni, Mickaël Chen, Éloi Zablocki, Hédi Ben- Younes, Patrick Pérez, and Matthieu Cord. Octet: Object-aware counterfactual explanations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15062115071, 2023.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. In Inter- national Conference on Learning Representations, 2018.
53


Reviews:
[{"review_id": "BRfkaFHFJA", "reviewer": "Reviewer_nGZZ", "summary": "This paper addresses the task of Visual Contour Factual Explanations (VCE) by introducing regional constraints to enhance the specificity and clarity of the explanations. Additionally, it proposes a method, RCSB, to generate counterfactual results based on region-specific information. The effectiveness of the proposed approach is evaluated on both generated and manually designed regions.", "questions": "See strengths and weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly written, with most details of the proposed method presented effectively. It includes numerous qualitative results that demonstrate the method’s effectiveness and clearly define the problem.", "weaknesses": "1. The paper demonstrates that the proposed method achieves better performance when a semantic region constraint is applied. However, it would be helpful to explore the effects when the region is not semantically relevant—such as when using a randomly selected area with similar shape and size—to understand how critical semantic relevance is to the method’s effectiveness.\n\n\n\n2. In Table 1, three configurations of the proposed method are presented as settings A, B, and C. While it appears these configurations differ based on specific hyperparameter choices, could you provide additional context or rationale for highlighting these particular settings? Are there theoretical or practical reasons to show them in this table?\n\n\n\n3. The results for LangSAM-based regions have a considerably higher FID score than those for other regions, suggesting that this style of regions may pose unique challenges. Could you clarify why this region is so different or particularly difficult for the method? Additional explanation on this would provide valuable insight into the model’s limitations.\n\n\n\n4. The paper briefly mentions \"confirmation bias,\" but could you expand on how this bias is defined in the original problem and mitigated by the proposed method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dgqqnBVFIt", "reviewer": "Reviewer_2FX9", "summary": "The authors propose a method for generating region-constrained visual counterfactual explanations, arguing that the region constraint leads to improved interpretability of the resulting counterfactual explanations (compared to unconstrained CEs). For this purpose, they leverage i) Image-to-Image Schrödinger Bridges, ii) automatically determined regions of interest, using either classical visual attribution methods or a text-to-object segmentation foundation model (LangSAM), and iii) several tweaks for stabilizing classifier guidance of the diffusion process. The proposed method outperforms several baselines on previously identified particularly challenging counterfactual questions on ImageNet.", "questions": "In addition to my questions & suggestions presented above under 'Weaknesses', just a few minor remarks:\n- In Fig. 1, what is the \"recent SOTA method\" used? \n- In the caption of Fig. 2, the authors write that \"Intermediate images of I2SB are much closer to the data manifold.\" - I assume that by this, the authors mean that most of the image is 'natural' throughout the whole generation procedure? I was a bit confused here at first, because the trajectory of the cat's face looks very similar in both rows (as would be expected).\n- Did the authors consider using the *inverse* of their mask (i.e., only allowing the model to change the background) for assessing a model's robustness to shortcut features?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is very well-written overall and generally a pleasure to read; the various technical aspects of the method are very well-motivated, rigorously and clearly presented, and demonstrated to be useful in ablation experiments.\n- Region-constrained counterfactual explanations are certainly very useful (although I would argue that this approach is not completely novel, cf. my comments below) and I believe the explicit problem formulation and framing here can help move this field forward. The extensive and clearly explained examples are also much appreciated.\n- The I2SB-based conditional inpainting approach appears to perform very well.\n- An interesting approach for (ADAM-derived) gradient stabilization and normalization in classifier-guided diffusion", "weaknesses": "**The relationship to several prior works should be discussed more comprehensively**, and possibly some of these methods should be included as baselines. The explicitly novel contributions should be highlighted more clearly.\n- Both [Jeanneret et al. (2023)](https://openaccess.thecvf.com/content/CVPR2023/papers/Jeanneret_Adversarial_Counterfactual_Visual_Explanations_CVPR_2023_paper.pdf) and [Weng et al. (2024)](https://link.springer.com/chapter/10.1007/978-3-031-73016-0_20) also use automatically determined region constraints and conditional inpainting to obtain region-constrained counterfactual explanations. In several places it reads as if the authors claim novelty on region-constrained CEs *per se*, or the use of conditional inpainting for this purpose.\n- Concerning the challenge of obtaining meaningful gradients with respect to noisy (intermediate) images, as discussed on page 4, this has been addressed before in various approaches such as DiME (by computing gradients with respect to denoised images instead, which is computationally expensive). Both FastDiME and [He et al.](https://openreview.net/forum?id=o3BxOLoxm1) have (more?) efficient implementations of the Tweedie approach. \n- [Karunratanakul et al. (2023)](https://openaccess.thecvf.com/content/ICCV2023/papers/Karunratanakul_Guided_Motion_Diffusion_for_Controllable_Human_Motion_Synthesis_ICCV_2023_paper.pdf) is also very related and contains many of the same ingredients presented here. Again, the precise differences should be discussed.\n\n**For SOTA claims, more extensive experiments and standard benchmarks would be needed.** \n- Performance is only evaluated quantitatively on three particular tasks previously identified to be particularly challenging for competitor methods. It seems unfair to specifically compare on the case where baselines are known to perform poorly, but not on the ones where they perform well. More generally, a SOTA claim for VCEs would require more extensive experiments. E.g., a comparison on CelebA-smile would already be interesting.\n- As noted above, I would suggest adding several more baseline methods such as DiME, FastDiME, and possibly combinations of these methods. E.g. it should be relatively straightforward to implement an I2SB version of FastDiME. This would help further disentangle the benefits provided by the different method components (I2SB, different auto-masking compared to prior work, gradient estimation & stabilization) presented here.\n\n**Overclaims regarding trustworthiness and 'causality' of the resulting explanations**. While I am generally on board concerning the utility of region-constrained VCEs, the authors do go quite a bit overboard in their claims about them, I believe.\n- If a VCE is constrained to only change a specific part of the image, this may yield to more intuitively appealing explanations, but the model might actually still be looking at weird shortcut features in other parts of the image (outside the mask). One would then miss this crucial information. This could happen in the proposed method e.g. due to non-localized shortcuts (e.g. [Wang et al.](https://openaccess.thecvf.com/content/ICCV2023/html/Wang_What_do_neural_networks_learn_in_image_classification_A_frequency_ICCV_2023_paper.html)), where every individual pixel of the background would receive a low importance score. It would also miss weak shortcut influence, where the shortcut doesn't completely dominate predictions but still subtly skews confidences scores, such as in racial/ethnic shortcuts in medical imaging ([Zou et al.](https://www.science.org/doi/full/10.1126/science.adh4260), [Yang et al.](https://www.nature.com/articles/s41591-024-03113-4)). Personally, I would be *very* careful with explanation methods that explicitly constrain the explanation to those parts of an image that we believe to be useful a priori. (Cf. e.g. the case study in Fig. 5 in regard to which it is claimed that \"RCSB allows for causal inference about the model’s reasoning\" - but if we constrain explanations to LangSAM segmentations, we might miss that the classification model is actually primarily looking elsewhere.) The whole setup could thus be seen as strongly *increasing* the risk for confirmation bias, which the authors claim to combat using their method. Also cf. the example in Fig. 1, where it could actually be considered very informative that the counterfactual explanation changes the copyright caption (the authors mention this as a flaw), because it might indicate potential shortcut learning! As a final example in this regard, the authors write that \"In this case, RCSB allows for increasing trust in the model, as making the lemons more orange correctly modifies its decision.\" - but again, this trust is unwarranted: the importance of the object color might in fact be negligible compared to the importance of some background shortcut feature.\n- Relatedly, the relative importance of different features is lost in region-constrained VCEs. The fact that one *can* change the head of a bird such that it is classified differently does not mean that these specific properties of the head are actually important to the model's classification. Again, this severely limits any claims of \"inferring causally about the model's predictions\" - we still don't know which features were actually important for the classification.\n\nTo reiterate, I am generally sympathetic to the approach pursued here. However, the discussion of its merits and limitations (!) should be much more nuanced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FPbafbr8QX", "reviewer": "Reviewer_PMZS", "summary": "This manuscript studies the problem of visual counterfactual explanations (VCEs) and argues that region is important. It is an extension of the visual counterfactual explanation by adding a hard region constraint. The method is based on a recently developed diffusion-based image-to-image translation method, named Image-to-Image Schrodinger Bridge ($I^2$SB). The authors discover a good way to inject the gradient from the classifier into the generation process of $I^2$SB and make the $I^2$SB generate more plausible images that change the prediction of the classifier. It gives us a more flexible way to understand the model prediction process.", "questions": "- I'm wondering what will be generated if we force the label of the generated image to a very different class. For example, if we want to change the output of the model given the image of an animal to the class of a dining table, what does the generated image look like?\n\n- There are some cases in which the model indeed makes use of unusual visual cues to predict the label of the image, like the background. We may also want to know that the model has learned such a prior from the dataset (for example, the model links the snow background to the wolf due to dataset bias). How can the proposed method discover such cases?\n\n- The visualization shown in Figure 5 is interesting. At the same time, I noticed that after changes, several images changed the image's attributes significantly toward the target class (like the Flamingo and Cauliflower cases). I'm wondering what can we learn from these changes to understand the model's behaviors, as the model's prediction on these changed samples is very reasonable and as expected.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Bring the region constraint to the counterfactual example generation somehow makes sense to better align with human perception.\n\n- The authors provide a way to better inject the guidance from the gradient of the classifier f into the generation process of the inpainting model. From the visual examples, we can indeed observe a much higher quality.", "weaknesses": "- As the regions are the core input of the proposed method, how to define/find the regions is crucial. The method uses either the attribution map generation or the grounding method. Will these methods introduce additional bias in the understanding of the classifier? \n\n- From my perspective, the proposed method is more like just an image painting with specific guidance, where the guidance comes from the gradients of the classification model. The quality of the explanation ability somehow entangles the ability of the classifier itself and the inpainting model. The quality (FID, etc,) of the generated samples is also highly entangled with the base inpainting models. This might be a problem if we want to understand the behaviors of the classifier.\n\n- The visual counterfactual explanation is similar to the problem of attribution map generation, where the goal is to detect the regions that have the most influence on the prediction of the model either positively or negatively. Actually, one of the region generation methods is attribution map generation (IG). If we use the attribution map generation method, we can also generate some examples with high FID while with different predicted labels. I wondered what's the core difference between these two setups.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rPYnWdpKrN", "reviewer": "Reviewer_p98R", "summary": "This paper targets at the task of VCEs and defines more refined task with region constraints to gain more precise explanations. Further, it also proposes the method RCSB for the newly defined task, which is built upon the I^2SB and solves the problem as a conditional inpainting task. The idea is simple and the experiments demonstrates its effectiveness. Nevertheless, the technical novelty is relatively limited.", "questions": "1. The paper claimed that the definition of VCEs has potential confirmation bias. However, it is not deeply investigated. Besides, the teaser example in Figure 1 also cannot indicate such confirmation bias.\n2. There is no direct qualitative comparison between different methods, in Figure 5 and 6.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The newly formulated RVCEs is reasonable. \n2. The presentation is good and the paper is easy to follow.\n3. The experiments demonstrates the effectiveness of the proposed method.", "weaknesses": "1. The technical novelty is relatively limited. It formulates the task as a conditional inpainting problem and solves it by applying $I^2SB$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1LcHbYE7MX", "reviewer": "Reviewer_Rswz", "summary": "This paper presents a method for creating visual counterfactual explanations for classification tasks such as ImageNet classification. These explanations illustrate how an image could be modified to change the classification output. The approach is based on a combination of existing methods which select the most relevant parts of the image to change the decision and then mask and inpaint these regions to change the classification decision. A major contribution of the proposed method is limiting the masking/inpainting to a small connected spatial region in the image rather than allowing changes everywhere. The method outperforms other approaches on a set of metrics which aim to balance various factors desirable for visual counterfactuals (namely, changing the model’s prediction while keeping the output image similar to the input, keeping changes sparse, and ensuring the output is realistic-looking).", "questions": "- The comparison models in the main experiment (Table 1) are not defined anywhere in the paper -- the abbreviations used in the table are never used in the text, and the table has no citations to tell you what the abbreviations might mean. I think I was able to work out what these models are through Googling but they should be named in the paper (though it's worth noting that there is another XAI method called \"ACE\").\n- Since the evaluation dataset seems to be restricted to class pairs that favor the proposed method (cases where the two classes have the same shape but different surface color/texture), I'd be curious to know how the proposed model compares on a wider set of counterfactual pairs. Does the proposed method outperform ACE, etc. on arbitrary counterfactual pairings, where the key differences between the classes might include shape changes?\n- Can this method be helpful for users to understand adversarial attacks (if the input image is a adversarial image, can the counterfactuals help explain why the model saw the image as the wrong category)?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed method outperforms similar approaches on the proposed metrics, at least on a very small evaluation dataset (3 class pairs).\n- The writing is very clear, and the results are very well organised and presented, with well-designed figures to illustrate the model's performance.\n- The paper shows some interesting possible applications, such as allowing users to target specific regions of an image to see those regions would need to change to change the model's decision.", "weaknesses": "- The main contribution of the paper (limiting the inpainting to a specific region) also seems like a weakness, since the model cannot capture cases where the differences between two classes are changes in shape. (For example, this is shown the American chameleon <-> common iguana examples -- the main difference between the two types of lizard is the overall shape/size, but the model can only show how the texture/color differs between these classes.)\n- The intro says that the goal of generating counterfactuals is to explain the decision-making process of an image classifier to a user, but it's not clear whether the proposed model is better than any other model at achieving that result. This approach performs best on the metrics, but simply maximizing performance across metrics seems like it would result in a \"counterfactual\" image that flips the classifier's decision with almost no visible changes to the image, which wouldn't be a useful counterfactual for human users (as they wouldn't be able to tell what changed).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yqQlio5M5U", "reviewer": "Reviewer_TMN9", "summary": "The authors propose a method for visual counterfactual explanation which is based upon methods like latent diffusion counterfactual explanation by Farid et al. and modified the previous approaches by adding a spatial (region) constraint, which allows more precise target generation of new images containing features of the target class. Thereby, also an external classifier is used as guidance for the generation process, e.g. for a stable diffusion model. Furthermore, the authors added optimization techniques to overcome limitation with respect to generated artefacts, color balance and realism (measured via FID score). Furthermore, the authors introduced also an approach for generating the regions automatically to enhance the usability of the method. In the evaluation, the authors show improved realism via FID score, effectiveness via flip rate, similarity and sparsity in comparison to current state-of-the-art methods (e.g. ACE or LDCE). In addition, qualitative results are presented showing the applicability of the method for e.g. discovering complex patterns. The authors claim finally that they could improve on existing methods by a large margin.", "questions": "Apart from the recommendations in the weaknesses section I have the following questions and suggestions:\n- Highlight the possible use cases of the method (e.g. class confusion) more so that the value of the method can be better understood\n- Avoid using emphasizing phrases, e.g. by a large margin, and not bringing a quantitative results or at least a reference\n- To many details and efforts with respect to the evalution are in the appendix weakening the main paper. I would suggest to include more details in the main paper from the additional evaluation results on different generative models and classifiers and reducing some of the visuals in the main paper + some very detailed analysis of the shown images and come up with a more condensed evaluation but including all the insights.\n- What are the limitations of the method?\n- It was not clear to me If there are not many possible image modifications for one sample yielding into a class change. How is that restricted to find the “best” modification and what are the optimal criteria?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Well written text that is good to follow as a reader\n- Despite there a different attempt to condition the generation process, the idea is novel enough when including the optimization steps of the method bringing XAI methods\n- The theoretic description of the method is well formulated with respect to quality and clarity\n- The structure of the evaluation has a high quality", "weaknesses": "- Only tested in a limited dataset, the authors should consider applying the method also on different datasets or domains\n- From the results in the main paper, it could be confirmed that the method works on different generative models and external classifiers but claimed that results are available in the appendix. Therefore, I encourage the authors to include at least the numeric results in the main paper, so that a reader can obtain insights on the performance for different domains and models.\n- The use of FID score is misleading the comparison to other methods, that do not limit the generation process by regions, since the limitation to certain spatial regions automatically reduced the FID score significantly. The authors should at least consider this limitation and think of better comparison, e.g. maybe a region-based FID score.\n- In the paper the authors state that the method helps gaining more clear visual explanations where even a user can interact with the method to optimize the results. This is true with respect to how the method works, but no study with real users was performed to evaluate if such statement holds true. Either this claim should be removed or validated by a (small) user study.\n- The other SOTA methods, e.g. ACE and DVCE are not really introduced, nor cited and not compared from a theoretical point of view. At least they must be cited and introduced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)