Rethinking Visual Counterfactual Explanations Through Region Constraint

## Abstract

Abstract Visual counterfactual explanations (VCEs) have recently gained immense popularity as a tool for clarifying the decision-making process of image classifiers. This trend is largely motivated by what these explanations promise to deliver – indicate semantically meaningful factors that change the classifier’s decision. However, we argue that current state-of-the-art approaches lack a crucial component – the region constraint – whose absence prevents from drawing explicit conclusions, and may even lead to faulty reasoning due to phenomenons like confirmation bias. To address the issue of previous methods, which modify images in a very entangled and widely dispersed manner, we propose region-constrained VCEs (RVCEs), which assume that only a predefined image region can be modified to influence the model’s prediction. To effectively sample from this subclass of VCEs, we propose Region-Constrained Counterfactual Schrödinger Bridges (RCSB), an adaptation of a tractable subclass of Schrödinger Bridges to the problem of conditional inpainting, where the conditioning signal originates from the classifier of interest. In addition to setting a new state-of-the-art by a large margin, we extend RCSB to allow for exact counterfactual reasoning, where the predefined region contains only the factor of interest, and incorporating the user to actively interact with the RVCE by predefining the regions manually.

## 1 Introduction

Visual counterfactual explanations (VCEs) aim at explaining the decision-making process of an image classifier by modifying the input image in a semantically meaningful and minimal way so that its decision changes. Over time, they have become an independent research direction with the latest methods presenting impressive and visually appealing results. Nevertheless, in this work we show that they possess a fundamental flaw at a conceptual level – the lack of *region constraint* and its proper utilization.

Consider the image $\mathbf{x}^{*}$ in Fig. 1, which the classifier $f$ correctly predicts to be a jay. In essence, VCEs focus on semantically editing $\mathbf{x}^{*}$ so that the prediction of $f$ changes to some target class – bulbul in this case – hence providing an answer to a specific *what-if* question, through which the model’s reasoning is explained. Consider now an example VCE for $\mathbf{x}^{*}$, denoted as $\mathbf{x}_{\text{VCE}}$, obtained with a recent state-of-the-art (SOTA) method. While $\mathbf{x}_{\text{VCE}}$ is successful at changing the prediction of $f$ and can be considered both realistic and semantically close to $\mathbf{x}^{*}$, answering *why* $f$ now predicts it as a bulbul is close to impossible.

Figure: Figure 1: Previous methods create VCEs with unconstrained changes, making it virtually impossible to understand the decision-making process of a model. We propose *region-constrained* VCEs, establishing a new paradigm for comprehensible and actionable explanatory process.
Refer to caption: x1.png

The algorithm simultaneously modifies the bird’s head and feathers, changes the texture of the branch and even modifies the copyright caption. The entanglement and dispersion of introduced changes hence leaves the question unanswered. We argue that to circumvent these fundamental difficulties, VCEs should be synthesized with a hard constraint on the *region*, where the changes are allowed to appear, while leaving the rest of the image unchanged. For example, consider the image $\mathbf{x}_{\mathbf{R}}^{*}$ with regions of the bird’s head ($\mathbf{R}_{1}$) and body ($\mathbf{R}_{2}$) overlayed. Constraining the VCEs to introduce changes *only* to predetermined regions leads to two distinct explanations, $\mathbf{x}_{\mathbf{R}_{1}}$ and $\mathbf{x}_{\mathbf{R}_{2}}$, of why the decision changes to bulbul. By isolating the modified factors, the explanatory process greatly simplifies – one can now state with certainty that $f$’s new prediction is based either on the modified feathers ($\mathbf{x}_{\mathbf{R}_{2}}$) or the changed characteristics of its head ($\mathbf{x}_{\mathbf{R}_{1}}$). Region-constrained VCEs (RVCEs) allow, therefore, to reason about the model’s thought process in a *causal* and principled manner, mitigating the potential *confirmation bias* and clarifying the explanatory process.

By putting RVCEs in the spotlight, our work establishes new frontiers in the field of VCE generation. First, we define the objective of finding RVCEs as solving a conditional inpainting task. By building on top of the Image-to-Image Schrödinger Bridge (I^2SB, Liu et al. ()) approach and adapting it to the classifier guidance scheme, we develop an efficient algorithm which synthesizes RVCEs with extreme realism, sparsity and closeness to the original image. Specifically, we set a new quantitative state-of-the-art (SOTA) on ImageNet (Deng et al., ) with up to 4 times better scores in FID and 3 times better sFID (realism), up to 2 times higher COUT (sparsity), and match or exceed S^3 (similarity) and Flip Rate (efficiency) achieved by previous methods. Through large-scale experiments, we demonstrate that, besides a fully automated way of synthesizing meaningful and highly interpretable RVCEs, our approach, *Region-constrained Counterfactual Schrödinger Bridge* (RCSB), allows to infer causally about the model’s change in prediction and enables the user to actively interact with the explanatory process by manually defining the region of interest. Moreover, our results highlight the importance of RVCEs in future research, indicating potential pitfalls of unconstrained methods that could lead to drawing misleading conclusions.

## 2 Background & Related Work

In this section, we introduce the necessary background knowledge connected with score-based generative models (SGMs) and I^2SB, which forms the foundation of our method. We then present an overview of recent methods for VCE generation based on SGMs. For an extended literature review and detailed description of the theoretical basis, please refer to the Appendix.

SGM. Following the work of Song et al. (), SGMs can be constructed through the framework of stochastic differential equations (SDEs), where samples from a complex distribution $p_{0}$ (*e.g*., natural images) are mapped to a Gaussian distribution $p_{1}$, while the model is trained to reverse this mapping. Formally, converting data to noise is performed by following the *forward* SDE (Eq. 1a), while denoising happens through the *reverse* SDE (Eq. 1b, Anderson ()):

$$ $\displaystyle\mathrm{d}\mathbf{x}_{t}$ $\displaystyle=\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}})\mathrm{d}t+\sqrt{\beta_{ t}}\mathrm{d}\mathbf{w},$ (1a) $\displaystyle\mathrm{d}\mathbf{x}_{t}$ $\displaystyle=(\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}})-\beta_{t}\nabla_{ \mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t)})\mathrm{d}t+\sqrt{\beta_{t}}\mathrm{d }\mathbf{\bar{w}},$ (1b) $$

where $\mathbf{x}_{t}$ is the noisy version of a clean image $\mathbf{x}\in\mathbb{R}^{n}$ for some $n\in\mathbb{N}$ at timestep $t\in[0,1]$ , $\mathbf{w}$ and $\bar{\mathbf{w}}$ denote the Wiener process and its reversed (in time) counterpart, $\mathbf{F}_{t}(\mathbf{x}_{t}):\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ is the *drift* coefficient, $\beta_{t}\in\mathbb{R}$ is the *diffusion* coefficient and $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t)}$ is the *score function*. An SGM $\mathbf{s}_{\bm{\theta}}$, where $\bm{\theta}$ denotes the model’s parameters, is trained to approximate the score, *i.e*., $\mathbf{s}_{\bm{\theta}}(\mathbf{x}_{t},t)\approx\nabla_{\mathbf{x}_{t}}\log{p
(\mathbf{x}_{t},t)}$. During sampling, denoising begins from pure noise $\mathbf{x}_{1}\sim p_{1}$ and follows some discretized version of Eq. 1b with the approximate score $\mathbf{s}_{\bm{\theta}}$.

SGMs can also be adapted to *conditional* generation, where $\mathbf{y}$ represents the conditioning variable. In this case, the score $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t)}$ is replaced by $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t\mid\mathbf{y})}$, which can be decomposed with Bayes’ Theorem into $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t\mid\mathbf{y})}=\nabla_{\mathbf
{x}_{t}}\log{p(\mathbf{x}_{t},t)}+\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{y}\mid
\mathbf{x}_{t},t)}$. While $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t)}$ can be approximated with an already trained $\mathbf{s}_{\bm{\theta}}$, $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{y}\mid\mathbf{x}_{t},t)}$ must be modeled additionally. For $\mathbf{y}$ representing class labels, $p(\mathbf{y}\mid\mathbf{x}_{t},t)$ can be approximated with an auxiliary time-dependent classifier $p_{\bm{\phi}}(\mathbf{y}\mid\mathbf{x}_{t},t)$ trained on noisy images $\{\mathbf{x}_{t}\}_{t\in[0,1]}$. Incorporating $p_{\bm{\phi}}$ into the sampling process is termed as *classifier guidance* (CG), and can be strengthened (or weakened) with *guidance scale* $s$ through $\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t},t)}+s\cdot\nabla_{\mathbf{x}_{t}}
\log{p(\mathbf{x}_{t},t\mid\mathbf{y})}$. Therefore, class-conditional sampling in SGMs amounts to additionally maximizing the likelihood $p_{\bm{\phi}}(y\mid\mathbf{x}_{t},t)$ of the classifier throughout the generative process to arrive at images from the data manifold, which resemble (according to $p_{\bm{\phi}}$) instances of a specific class. We emphasize this fact here for further reference.

Figure: Figure 2: Generative trajectories of I^2SB and SGM. Intermediate images of I^2SB are much closer to the data manifold.
Refer to caption: x2.png

I^2SB. The framework of I^2SB extends SGMs to $p_{1}$ representing an *arbitrary* data distribution. For training, I^2SB requires paired data, *e.g*., in the form of clean and partially masked samples for inpainting, where it learns to infill the missing parts. While SGMs can also be adapted to solve inverse problems like inpainting, I^2SB maps these samples *directly* (see Fig. 2 for a comparison of their generative trajectories). Therefore, I^2SB follows the same theoretical paradigm, where sampling is achieved by discretizing Eq. 1b and using a score approximator $\mathbf{s}_{\bm{\psi}}$, but the generative process begins from a corrupted (*e.g*., masked) image instead of pure noise. Hence, I^2SB can also be adapted to conditional generation in the same manner as SGMs, especially for class-conditioning with an auxiliary classifier. Importantly, a special case of I^2SB follows an optimal transport ordinary differential equation (OT-ODE) when $\beta_{t}\rightarrow 0$, eliminating stochasticity beyond the initial sampling step (see Appendix). We utilize the OT-ODE version of I^2SB in our implementation.

SGM-based VCEs. The initial approach of adapting SGMs to VCE generation, DiME (Jeanneret et al., ), obtains the classifier’s gradient by mapping the noised image to its clean version at each step through the reverse process. Augustin et al. () incorporate the gradient of a robust classifier and a cone projection scheme. Jeanneret et al. () decompose the VCE generation into pre-explanation construction and refinement using RePaint (Lugmayr et al., ). Jeanneret et al. () utilize a foundation model, Stable Diffusion (SD, Rombach et al. ()), to generate VCEs in a black-box scenario. Farid et al. () and Motzkus et al. () utilize Latent Diffusion Models (LDMs), including SD, in a white-box context. Weng et al. () propose FastDiME to accelerate the generation process in a shortcut learning scenario. Also in black-box context, Sobieski & Biecek () utilize a Diffusion Autoencoder (Preechakul et al., ) to find semantic latent directions that globally flip the classifier’s decision. Finally, Augustin et al. () also make use of SD in various contexts, including classifier disagreement and neuron activation besides VCEs.

## 3 Method

In this section, we describe the details of our approach, beginning with the formulation of RVCEs as solutions to conditional inpainting task. Next, we motivate the use of I^2SB as an effective prior for synthesizing meaningful RVCEs and follow with a series of steps that better align the gradients of a standard classifier w.r.t corrupted images from its generative trajectory. We conclude with a description of the automated region extraction method, forming the basis of our algorithm.

RVCEs through conditional inpainting. We define the problem of finding RVCEs for the classifier $f$ from a given image $\mathbf{x}^{*}$, a region $\mathbf{R}$ and *target class* label $y$, where $\operatorname*{arg\,max}_{y^{\prime}}f(y^{\prime}\mid\mathbf{x}^{*})\neq y$, as the task of sampling from

$$ $p(\mathbf{x}\mid\operatorname*{arg\,max}_{y^{\prime}}f(y^{\prime}\mid\mathbf{x })=y,(\mathbf{1}-\mathbf{R})\odot\mathbf{x}=(\mathbf{1}-\mathbf{R})\odot \mathbf{x}^{*}),$ (2) $$

where $\mathbf{R}$ is a binary mask with $1$ indicating the region. Intuitively, sampling from Eq. 2 means obtaining $\mathbf{x}$ with the complement of $\mathbf{R}$ unchanged and the content of $\mathbf{R}$ modified in a way that changes the decision of $f$ to $y$, *i.e*., performing inpainting with additional condition coming from the classifier $f$.

Figure: Figure 3: Series of proposed improvements to better align the gradient’s of the classifier of interest with the generative trajectory. Changes to the *factual* image are constrained to the indicated *region*. Subsequent images illustrate the influence of each new adaptation. Numbers below images correspond to FID ($\downarrow$) values obtained in a larger-scale experiment (for details, see Appendix).
Refer to caption: x3.png

Synthesizing meaningful RVCEs. Looking at Eq. 2, one quickly realizes that obtaining semantically meaningful RVCEs requires maximizing the likelihood $f(y\mid\mathbf{x})$ of the classifier while inpainting $\mathbf{R}$ with content that keeps $\mathbf{x}$ in the data manifold. These conditions greatly resemble the CG scheme in the context of I^2SB, since the score estimate $\bm{s}_{\bm{\psi}}$ serves as an effective prior for generating in-manifold infills, while the likelihood $p_{\bm{\phi}}(y\mid\mathbf{x})$ of an auxiliary classifier is maximized to ensure that $p_{\bm{\phi}}$ predicts them as instances of $y$. Moreover, I^2SB maps masked images *directly* to clean samples, leaving the content outside $\mathbf{R}$ unchanged in the final image.

The above arguments suggest that inserting $f$ in place of $p_{\bm{\phi}}$ should function as an effective mechanism for sampling meaningful RVCEs. However, a fundamental drawback of this *naive* approach is that, throughout the generative process, $f$’s gradients originate from evaluating it on images with highly noised infills inside $\mathbf{R}$ (see Fig. 2). Such corrupted images are far from what $f$ observed during training, hence leading to a *misalignment* of its gradients with the correct trajectory and generation of out-of-manifold samples. Similar issue has been identified by previously mentioned SGM-based methods for VCEs, which can be generally unified as attempts to replace the auxiliary classifier $p_{\bm{\phi}}$ with $f$ in the CG scheme in SGMs and *correct* $f$’s gradients. Following Fig. 2, one should expect the misalignment in these methods to be of great extent, as the generative trajectory consists of highly noised images, leaving no meaningful content for $f$ to provide accurate gradients. There, as shown in Fig. 2, I^2SB provides a crucial advantage, which stems from its generative trajectory being *much closer* to the data manifold. Moreover, by using I^2SB, $f$ is able to effectively utilize the readily available context outside $\mathbf{R}$.
Hence, in the following, we focus on reducing the misalignment problem caused by the noised content inside $\mathbf{R}$, in the end arriving at a highly effective algorithm for meaningful RVCEs.

Aligning the gradients. We propose to adapt the gradients of $f$ to properly align with the generative trajectory of I^2SB through a series of incremental steps. To provide the intuition standing behind the introduction of each consecutive improvement, Fig. 3 provides an example RVCE task, where the factual image depicts a zebra correctly predicted by the model (ResNet50 (He et al., )), and the goal is to change the decision to ‘sorrel’. We set the region constraint to include the entire animal to make the task challenging enough and verify the improvements quantitatively through a large-scale experiment with around $2000$ images. For each step, we compute FID between the RVCEs and original images to assess their realism. For details on the experimental setup, see Appendix.

Naive. We first verify that naively plugging $f$ in place of $p_{\bm{\phi}}$ does not provide meaningful results. Indeed, as shown in Fig. 3, the method struggles to include the information from $f$. The unrealistic infill also suggests that the classifier’s signal negatively influences the score from I^2SB.

Tweedie’s formula. To begin with closing the gap between the data manifold and the generative trajectory, we refer to a classic result of Tweedie’s formula (Robbins, ; Chung et al., ), which states that a *denoised estimate* of the final image at step $t$ can be achieved by computing the posterior expectation

$$ $\hat{\mathbf{x}}_{0}(\mathbf{x}_{t}):=\mathbb{E}[\mathbf{x}_{0}\mid\mathbf{x}_ {t}]=\mathbf{x}_{t}+\sigma^{2}_{t}\nabla_{\mathbf{x}_{t}}\log{p(\mathbf{x}_{t} ,t)},$ (3) $$

where $\sigma^{2}_{t}=\int_{0}^{t}\beta_{\tau}\mathrm{d}\tau$. For visual differences between $\mathbf{x}_{t}$ and $\hat{\mathbf{x}}_{0}(\mathbf{x}_{t})$, see Appendix. Crucially, one has access to approximate $\hat{\mathbf{x}}_{0}(\mathbf{x}_{t})$ at every step $t$ by utilizing I^2SB as the approximate score. Replacing $\nabla_{\mathbf{x}_{t}}\log{f(y\mid\mathbf{x}_{t})}$ with $\nabla_{\mathbf{x}_{t}}\log{f(y\mid\hat{\mathbf{x}}_{0}(\mathbf{x}_{t}))}$ brings the inputs of $f$ much closer to what it expects, improving the conditional inpainting process as indicated by Fig. 3, which now shows a structure resembling a sorrel and a much smaller FID.

ADAM stabilization. Despite utilizing the Tweedie’s estimate, we observed the norms of $f$’s gradients to have a very noisy tendency throughout the generation process, pointing out a possible cause for visible artifacts and the missing parts of the animal. Hence, we propose to smooth out the gradients by applying the ADAM update rule at each step (Vaeth et al., ; Kingma, ), to which we simply refer as ADAM stabilization. Figure 3 indicates that this modification allows for filling in the missing parts of the sorrel and further lowering FID.

Adaptive normalization. Incorporating ADAM stabilization required greatly lowering the guidance scale to values on the order of $1\mathrm{e}{-2}$, as using standard $s=1$ led to extreme artifacts. This phenomenon suggested that the step size could also be adjusted throughout the generation process. While we initially experimented with various types of schedulers (see Appendix), using *adaptive normalization* has empirically proven to be the most effective approach. Specifically, at the beginning of the conditional inpainting process, we register the norm of the first encountered gradient of the log-likelihood of $f$. We then use it as a normalizing constant for each subsequent gradient, meaning that the generation begins with gradient of unit norm. This simple modification not only further lowered FID, but also reduced the final visible artifacts and improved color balance (Fig. 3).

Trajectory truncation. Up until this point, we relied solely on the ability of I^2SB and the classifier’s signal to correctly infill the missing regions with semantically meaningful content, with no knowledge of the structure of the missing objects. Since a possible infill of the region is always available from the original image, one can begin the inpainting process from some *intermediate* step instead of the final one. This intervention allows for mixing the available information with the one coming from the classifier, and gives direct control over the preservation of the original content. As our approach does not bias the conditional score with signal from any additional losses (like Learned Perceptual Image Patch Similarity (LPIPS Zhang et al. ()) or $l_{2}$ in other works), we can fully rely on the *conceptual* compression of I^2SB, similarly to SGMs (Ho et al., ), which decomposes the generation process into initial phases responsible for the overall structure of objects and later ones responsible for small details. Figure 3 showcases the effect of using this *trajectory truncation* ($\tau$) at the $0.4$ level, meaning that the infilling process starts from $t=\tau\cdot T$, where $T$ denotes the final timestep.

Figure: Figure 4: Example region obtained with our automated region extraction. Instead of directly binarizing an attribution map (upper row), we amplify the focus on semantic concepts (bottom row) with a simple approach based on grid cells.
Refer to caption: x4.png

Understandably, trajectory truncation greatly lowers the FID score, as much more information is available from the very beginning of the process, and introduces much more subtle changes to the image. We explore the effect of manipulating $\tau$ further in the Appendix, showing that it functions as a very interpretable mechanism for controlling the content preservation.

Automated region extraction. While the introduced algorithmical improvements effectively incorporate the classifier’s signal into the inpainting process, they do not address the issue of predetermining the region for the resulting explanation. To this end, the optimal strategy would be fully automated and focus on regions that are both important to the classifier’s prediction and point to semantically meaningful concepts. This description closely resembles the role of visual attribution methods, which assign importance values to pixels based on their relevance to the model’s output (Holzinger et al., ). Figure 4 shows an example attribution map obtained with Integrated Gradients (IG, Sundararajan et al. ()) method for the squirrel prediction of a ResNet50 model. Perceptually, highest attributions are focused around the squirrel’s head. To extract a region from such attributions, one can threshold them to cover a specific fraction $a$ of the total image area. However, after binarizing the attributions with $a=0.05$, we observe that the resulting region is highly scattered, losing focus from semantic concepts. To address this issue, we divide the image into a grid of square cells of size $c\times c$, where each cell receives the value equal to the sum of the absolute pixel attributions inside it. Figure 4 shows that this postprocessing mechanism (here with $c=16$) greatly amplifies the focus of the resulting map. By thresholding it with $a=0.05$, we observe the extracted region to focus solely on the squirrel’s head. This leads to a fully automated strategy for obtaining regions that are both aligned with semantically meaningful concepts and based on pixels that are important for the classifier.

We term the final version of the algorithm which combines all of the aforementioned improvements with the automated region extraction as RCSB. For the pseudocode of the entire procedure, see Appendix. We include our implementation at [https://github.com/sobieskibj/rcsb](https://github.com/sobieskibj/rcsb).

## 4 Experiments

**Table 1: Quantitative comparison with SOTA. RCSB outperforms previous methods by a large margin across all metrics. The best results are obtained with $A(a=0.1,c=4,s=3,\tau=0.6)$, but the superiority is clear for various configurations, including $B(a=0.2,c=4,s=1.5,\tau=0.6)$, $C(a=0.3,c=4,s=1.5,\tau=0.6).$**
| Method | FID | sFID | S^3 | COUT | FR |
| --- | --- | --- | --- | --- | --- |
| Zebra – Sorrel |  |  |  |  |  |
| ACE $l_{1}$ | $84.5$ | $122.7$ | $\mathbf{0.92}$ | $-0.45$ | $47.0$ |
| ACE $l_{2}$ | $67.7$ | $98.4$ | $\underline{0.90}$ | $-0.25$ | $81.0$ |
| LDCE-cls | $84.2$ | $107.2$ | $0.78$ | $-0.06$ | $88.0$ |
| LDCE-txt | $82.4$ | $107.2$ | $0.71$ | $-0.21$ | $81.0$ |
| DVCE | $33.1$ | $43.9$ | $0.62$ | $-0.21$ | $57.8$ |
| RCSB^C | $13.0$ | $20.4$ | $0.82$ | $0.70$ | $\mathbf{99.7}$ |
| RCSB^B | $\underline{9.51}$ | $\underline{17.4}$ | $0.86$ | $\underline{0.72}$ | $97.4$ |
| RCSB^A | $\mathbf{8.0}$ | $\mathbf{16.2}$ | $0.88$ | $\mathbf{0.74}$ | $\underline{94.7}$ |
| Cheetah – Cougar |  |  |  |  |  |
| ACE $l_{1}$ | $70.2$ | $100.5$ | $\underline{0.91}$ | $0.02$ | $77.0$ |
| ACE $l_{2}$ | $74.1$ | $102.5$ | $0.88$ | $0.12$ | $95.0$ |
| LDCE-cls | $71.0$ | $91.8$ | $0.62$ | $0.51$ | $\underline{100.0}$ |
| LDCE-txt | $91.2$ | $117.0$ | $0.59$ | $0.34$ | $98.0$ |
| DVCE | $46.9$ | $54.1$ | $0.70$ | $0.49$ | $99.0$ |
| RCSB^C | $30.2$ | $39.2$ | $0.87$ | $0.79$ | $\underline{100.0}$ |
| RCSB^B | $\underline{23.4}$ | $\underline{32.4}$ | $0.90$ | $\underline{0.85}$ | $99.9$ |
| RCSB^A | $\mathbf{17.2}$ | $\mathbf{26.6}$ | $\mathbf{0.92}$ | $\mathbf{0.92}$ | $\mathbf{100.0}$ |
| Egyptian Cat – Persian Cat |  |  |  |  |  |
| ACE $l_{1}$ | $93.6$ | $156.7$ | $\underline{0.85}$ | $0.25$ | $85.0$ |
| ACE $l_{2}$ | $107.3$ | $160.4$ | $0.78$ | $0.34$ | $97.0$ |
| LDCE-cls | $102.7$ | $140.7$ | $0.63$ | $0.52$ | $99.0$ |
| LDCE-txt | $121.7$ | $162.4$ | $0.61$ | $0.56$ | $99.0$ |
| DVCE | $46.6$ | $59.2$ | $0.59$ | $0.60$ | $98.5$ |
| RCSB^C | $41.1$ | $56.3$ | $0.79$ | $0.82$ | $\underline{100.0}$ |
| RCSB^B | $\underline{31.3}$ | $\underline{48.1}$ | $0.84$ | $\underline{0.87}$ | $\underline{100.0}$ |
| RCSB^A | $\mathbf{23.0}$ | $\mathbf{40.0}$ | $\mathbf{0.87}$ | $\mathbf{0.92}$ | $\mathbf{100.0}$ |

Following previous works for VCEs on ImageNet, we base the quantitative evaluation on 3 challenging main VCE generation tasks: Zebra – Sorrel, Cheetah – Cougar, Egyptian Cat – Persian Cat, where each task requires creating VCEs for images from both classes and flipping the decision to their counterparts. We treat it as a general benchmark for evaluating the effectiveness of RCSB in various scenarios. We use FID ($\downarrow$) and sFID ($\downarrow$) to assess realism (Heusel et al., ), S^3 ($\uparrow$) for representation similarity (Chen & He, ), COUT $\in[-1,1]\ (\uparrow)$ (Khorram & Fuxin, ) for sparsity and Flip Rate (FR) ($\uparrow$) for efficiency.
For qualitative examples, we extend the main tasks with a large array of other tasks, which we show throughout the paper and the Appendix, where more details regarding the experimental setup and the metrics description can be found.

RCSB sets new SOTA for VCEs. We first verify that synthesizing RVCEs with RCSB leads to new SOTA in VCE generation. Table 1 quantitatively compares RCSB with recent SOTA approaches to VCEs on ImageNet. Our RVCEs are much more realistic (at least $2-4\times$ decrease in FID and sFID), stay close to original images (match or exceed best values of S^3) and almost always flip the model’s decision ($\text{FR}\approx 1.0$). RCSB also solves a long-standing challenge of achieving extremely *sparse* explanations on ImageNet, especially on Zebra – Sorrel task. While all other methods fail to achieve nonnegative values, RCSB approaches the upper bound of COUT. Our method is clearly the most balanced, as it does not struggle on any specific metric like, *e.g*., DVCE on S^3. In the Appendix, we show that it is also the most computationally efficient.

Figure 5 shows example explanations obtained with RCSB, greatly highlighting the importance of synthesizing RVCEs instead of standard VCEs. Our region extraction approach is able to precisely localize semantic concepts responsible for the model’s decision. For example, in the Guacamole $\rightarrow$ Cabbage task, RCSB detects the guacamole bowl in the background and, guided by the classifier, infills it with cabbage while leaving the rest of the image unchanged. RCSB is capable of performing a wide range of editing tasks with various levels of difficulty, beginning with textural and color-based edits (*e.g*., Tench $\rightarrow$ Goldfish, Mashed Potato $\rightarrow$ Cauliflower) to partially changing the object’s structure (*e.g*., Limpkin $\rightarrow$ Flamingo) to infilling the region with new, realistically looking concepts (*e.g*., Cougar $\rightarrow$ Lynx, Green Mamba $\rightarrow$ Indian Cobra, Cougar $\rightarrow$ Lynx). Most importantly, thanks to the region constraint, our RVCEs allow for greatly limiting the potential factors that influenced the model’s decision, making the explanations much more interpretable.

Figure: Figure 5: Qualitative examples obtained with RCSB using automated region extraction. Each task of the form *predicted class* $\rightarrow$ *target class* shows the factual image, the extracted region and the RVCE obtained with RCSB.
Refer to caption: x5.png

RCSB allows for causal inference about the model’s reasoning. Drawing definite conclusions about the model’s reasoning from an unconstrained VCE is not possible, as one cannot be certain that modifying potentially irrelevant factors did not in fact influence the prediction. RVCEs overcome this limitation when constrained on the region connected with the sole factor of interest, *e.g*., the body of an animal in a species prediction task. To adapt RCSB to such scenario, we replace the automated region extraction method with a foundation text-to-object-segmentation model (^1^11Language Segment Anything ([LangSAM](https://github.com/paulguerrero/lang-sam)) combines Segment Anything Model (Kirillov et al., ) with GroundingDINO (Liu et al., ) to allow object segmentation from text prompts.). Using the class name from a given task as the text prompt allows us to obtain highly precise segmentation masks of the relevant objects, enabling the identification of the cause behind the model’s prediction change based solely on factors related to the object of interest.

We first quantitatively assess that RCSB is capable of utilizing regions provided by a generic object detector at scale. Table 2(A) shows the results of this evaluation together with the used text prompt. Here, the metrics are computed by first discarding images with a mask that covers area larger than $40\%$. Despite I^2SB being trained on masks covering at most $30\%$ of the image area, we observed that it generalizes well beyond this threshold with $40\%$ starting to pose a challenge. Crucially, despite the regions being classifier-agnostic and hence not necessarily focused on the most influential pixels, Table 2(A) indicates that RCSB is versatile enough to maintain most of the performance from the automated approach. The efficiency, sparsity and representation similarity of the obtained RVCEs remain very close to the values achieved by the closest configuration (in terms of hyperparameters) from Table 1, as the region area is often close to or exceeds $30\%$. The slight increase in FID and sFID stems mainly from the regions covering complex objects, whose modification may naturally move RVCEs further from original data at a distribution level, and a lower number of images used for these metrics’ computation (as both are sensitive to sample size) due to the rejection of samples from the area constraint.

**Table 2: Quantitative results from various experiments. A: regions extracted from LangSAM with text prompt connected to the initial class name. B: regions based on freeform masks that cover the fraction of the total area from the indicated range. C: automatically extracted regions used with adaptations of other inpainting algorithms.**
| Metric | FID | sFID | S^3 | COUT | FR | FID | sFID | S^3 | COUT | FR | FID | sFID | S^3 | COUT | FR |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Task | Zebra – Sorrel | Cheetah – Cougar | Egyptian Cat – Persian Cat |  |  |  |  |  |  |  |  |  |  |  |  |
| A | Exact regions obtained with LangSAM and prompts: zebra / horse, cheetah / cougar, cat respectively |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| Values | 32.8 | 41.5 | 0.87 | 0.74 | 98.9 | 37.2 | 50.6 | 0.91 | 0.84 | 99.4 | 52.0 | 82.8 | 0.81 | 0.84 | 99.2 |
| B | Regions based on freeform masks with the area in the indicated range |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| $10-20\%$ | 6.7 | 15.0 | 0.85 | 0.85 | 87.6 | 9.0 | 19.1 | 0.89 | 0.72 | 96.6 | 12.4 | 29.6 | 0.80 | 0.73 | 96.9 |
| $20-30\%$ | 7.8 | 15.8 | 0.84 | 0.53 | 92.2 | 11.6 | 21.3 | 0.88 | 0.71 | 99.6 | 17.7 | 34.0 | 0.78 | 0.74 | 99.3 |
| C | Ablation study with adaptations of other inpainting algorithms |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
| RePaint | 63.8 | 76.0 | 0.55 | 0.77 | 99.3 | 129.3 | 144.2 | 0.50 | 0.77 | 99.0 | 148.7 | 175.2 | 0.38 | 0.76 | 99.5 |
| MCG | 43.2 | 55.6 | 0.73 | 0.45 | 96.0 | 76.6 | 91.4 | 0.74 | 0.64 | 100.0 | 93.7 | 117.5 | 0.62 | 0.65 | 99.9 |
| DDRM | 42.5 | 49.4 | 0.69 | 0.72 | 99.6 | 60.5 | 68.4 | 0.72 | 0.76 | 100.0 | 59.2 | 73.0 | 0.63 | 0.76 | 100.0 |

Regions that contain *exactly* the objects of interest provide novel insights about the model’s reasoning. For example, consider the Lemon $\rightarrow$ Orange task from Fig. 6, where the lemons were correctly identified by the ResNet50 model. One would require the VCE for this task to indicate the sole determining factor of ’why lemons and not oranges’. However, with unconstrained VCEs, this identification process quickly becomes incomprehensible due to small changes added to each object in the image, such as other fruits. By constraining VCEs to the region occupied by the lemons, the reasoning process can be disentangled and simplified, as one can now look for this factor in the modifications of the lemons only. In this case, RCSB allows for increasing trust in the model, as making the lemons more orange correctly modifies its decision.

RVCEs also allow for clarifying the model’s decision-making when its reasoning is not initially understandable. In the Volcano $\rightarrow$ Seashore task, the image shows both objects, while the model predicts it as the former. Applying RCSB to the exact region of the seashore results in a RVCE that changes the model’s decision when the water’s color becomes more light blue and structures like stones start to appear. Hence, one is able to better understand what the model *actually* identifies as a seashore. In other examples, the method introduces class-specific characteristics when the changes are constrained *precisely* and *exclusively* to the object of interest, ensuring the receiver about the general cause of the model’s decision change. Such cases are also especially relevant when the generative model used to synthesize explanations is prone to systematic errors like, *e.g*., SGMs struggling with correctly generating hands. In the Night Snake $\rightarrow$ Kingsnake task, this error can be bypassed with the region constraint by not allowing the generative model to affect anything other than the animal, hence alleviating the evaluation of the classifier on out-of-manifold samples.

Discovering complex patterns with interactive RVCEs. Despite the impressive capabilities of deep models in object localization, the receiver of the explanation may be interested in testing the model for highly abstract and complex concepts that cannot be localized automatically and must be provided manually by the user. We begin with verifying the capability of RCSB in generating RVCEs based on user-defined regions by simulating such scenario at scale. Specifically, we randomly match images from the main tasks with regions given by the $10\%-20\%$ and $20\%-30\%$ freeform masks from the I^2SB training data (Saharia et al., ). We argue that this serves as a very challenging benchmark, since the algorithm’s access to the most influential pixels (for the classifier) might often be very restricted.

Figure: Figure 6: Qualitative examples obtained with RCSB using *exact* regions extracted from LangSAM using text prompt of the predicted class. For each task of the form *predicted class* $\rightarrow$ *target class*, a factual image together with the used region and the resulting RVCE are shown. The used text prompts are *emphasized*.
Refer to caption: x6.png

Despite the task’s difficulty, quantitative results from Table 2(B) highlight the versatility of RCSB, which is able to effectively utilize the restricted resources to influence the classifier’s prediction. While S^3, COUT and FR are not significantly different from previous results, we observe a decrease in FID and sFID, indicating higher realism and closeness to the data distribution. This is largely due to the fact that freeform masks are often not connected to entire complex objects and do not contain the pixels most important to the classifier. Hence, RCSB may often leave large portions of the regions unchanged, which boosts the realism evaluation.

Figure: Figure 7: Qualitative examples obtained with RCSB from *user-defined* regions. For each task of the form *predicted class* $\rightarrow$ *target class*, a factual image together with the provided regions are shown. Arrows point to RVCEs obtained by modifying only the indicated region.
Refer to caption: x7.png

To allow for true interaction of the user with the explanatory process, we implement a simple interface that allows for manual image segmentation using a brush-like cursor. Figure 7 shows example results, where we manually predefine the regions on different images. This exploration gives important insights about the added value provided by RVCEs. In the Cat $\rightarrow$ Tiger task, we discover that the classifier’s decision can be flipped by independently modifying either the cat’s paws or snout, in both cases introducing a tiger’s coloration. Similarly in the Arctic Fox $\rightarrow$ Red Fox task, choosing either the ears and muzzle or paws and stomach area allows for changing the model’s decision with the features of a red fox. User-defined regions also allow to discover unusual reasoning patterns of the model. In the Cucumber $\rightarrow$ Zucchini task, the model’s decision can be influenced by modifying only one of the cucumbers to zucchini, leaving the other unchanged. This observation connects with recent positions on the topic of contextual and spatial understanding of predictive models (Tomaszewska & Biecek, ), providing new rationale in further exploring how image classifiers *actually* reason.

Ablating RCSB’s components. We empirically verified that combining our novel guidance mechanism with the I^2SB prior leads to highly effective RVCEs. To better understand the benefits provided by each component of our framework, we perform an ablation study, where we adapt the proposed improvements to SGM-based inpainters, aiming to assess the influence of the guidance scheme and I^2SB in isolation. Specifically, we pick RePaint (Lugmayr et al., ), one of the first adaptations of SGMs to inpainting, MCG (Chung et al., ) and DDRM (Kawar et al., ), two different adaptations of SGMs to linear inverse problems, which also include inpainting. We manually tune our guidance scheme to each method on a small subset of images and repeat the same evaluation protocol with the automated region extraction method (see Appendix for details of each adaptation). As these methods are much less compute-efficient, we cap their computational budget on each task to 24 A100 GPU hours.

Table 2(C) shows the results of the ablation study. Despite the fact that the used methods were never explicitly trained for inpainting, combining them with our guidance mechanism and region extraction allows for matching or even exceeding previous SOTA. For example, all adaptations achieve very high sparsity, almost always flip the classifier’s decision and keep the explanation close to the original. This indicates the benefits of utilizing only the pixels from the extracted region and a proper utilization of the classifier’s gradients without biasing them with additional components like LPIPS or $l_{2}$ loss. RCSB differentiates itself from the adaptations with a much higher realism of the obtained RVCEs (significantly lower FID and sFID), more balanced results and much smaller computational burden, *e.g*. $24\times$ less NFEs than RePaint. These benefits stem from the I^2SB prior, which is trained to map corrupted images directly to clean samples and the resulting trajectory being much closer to the data manifold, allowing the classifier to more effectively influence the inpainting process.

For extended quantitative and qualitative results, including RVCEs obtained for another 5 non-robust, 2 robust and a zero-shot CLIP classifier (Radford et al., ), and evaluation of 10 other attribution methods, we refer to the Appendix.

## 5 Conclusions

Our work advances the SOTA in VCE generation by constraining the explanations to differ from the factual image exclusively within a predetermined region. RCSB is not only very effective in sampling such explanations, proven by new quantitative records, but also showcases the novel capabilities for explaining image classifiers enabled by RVCEs. Specifically, to properly reason about the model’s decision-making process, one must ensure that the potential confounding factors are limited to the greatest possible extent. RVCEs obtained with RCSB allow to do that in a wide range of scenarios, ranging from a fully automated approach to incorporating the user directly into the interactive explanation creation process. Our work establishes a new paradigm for explaining image classifiers in a much more principled manner, allowing the receiver to infer causally about the model’s reasoning.

## Appendix A Pseudocode

Figure: Algorithm 1 Standard I^2SB Generation

Figure: Algorithm 2 OT-ODE I^2SB Generation

Figure: Algorithm 3 RCSB

Figure: Algorithm 4 ADAM Update Rule

Figure: Figure 8: Visual difference between $\mathbf{x}_{t}$ and its corresponding Tweedie’s estimate $\hat{\mathbf{x}}_{0}(\mathbf{x}_{t})$ across different timesteps.
Refer to caption: x8.png

Figure: Figure 9: Influence of manipulating $\tau$ on the final RVCE obtained with the region shown in Fig. 3.
Refer to caption: x9.png

## Appendix B Extended Background & Related Work

Visual counterfactual explanations. In recent years, increasing attention is being paid to synthesizing VCEs for image classifiers (Goyal et al., ; Schut et al., ). These explanations aim at elucidating the model’s reasoning by modifying the input image in a semantically minimal and meaningful way while flipping its prediction. Utilizing generative models for this task has historically proven to be very effective (Chang et al., ; Singla et al., ; Lang et al., ). Non-SGM-based methods include works like Thiagarajan et al. () which builds on the concept of deep inversion, OCTET from Zemni et al. () focusing on VCEs for complex scenes and more examples built on top of generative models (Rodríguez et al., ; Jacob et al., ; Shih et al., ; Zhao et al., ; Van Looveren & Klaise, ).

While offering impressive results (Farid et al., ; Jeanneret et al., ; Augustin et al., ; Motzkus et al., ), we argue that utilizing general foundation models like SD in the VCE generation task may cause misleading conclusions, since the explained classifiers are trained on much smaller datasets than the generative model. For example, about 1 million images from ImageNet (Deng et al., ) are used to train the classifier, while SD is trained on 5 billion images from LAION-5B (Schuhmann et al., ). This discrepancy may naturally lead to SD synthesizing realistically looking variations of a given image that flip the classifier’s decision but simultaneously include semantic attributes never observed by the predictive model during training. Therefore, one may question the *counterfactual* nature of the explanation, as the classifier should not be expected to correctly treat attributes that it was never close to observing. Hence, in this work, we focus on generative models trained on the same data as the classifier of interest. This way, we can study the behavior of predictive models in a *faithfull* manner, which is an open challenge for XAI community Biecek & Samek ().

Inverse problems. Inverse problems (Kirsch, ) are defined as the task of recovering an unknown signal $\mathbf{x}$ based on a measurement $\mathbf{y}$ related via a measurement model $\mathbf{H}$ through $\mathbf{y}=\mathbf{H}(\mathbf{x})$, where $\mathbf{H}$ is not necessarily required to be linear or bijective. Hence, for a given measurement $\mathbf{y}$, there may exist a probability distribution over possible solutions $p(\mathbf{x}\mid\mathbf{y}=\mathbf{H}(\mathbf{x}))$. One special case of an inverse problem is image inpainting, where the missing area of an image, indicated by the mask $\mathbf{M}$, must be infilled using the available context. The measurement model is then defined as $\mathbf{H}(\mathbf{x})=\mathbf{M}\odot\mathbf{x}$, where $\odot$ denotes an element-wise product and $\mathbf{M}$ is a binary mask.

In recent years, deep learning methods have proven to be very effective at solving various kinds of inverse problems (Scarlett et al., ). Recently, utilizing generative methods, especially SGMs, established itself as the new SOTA approach in the image domain. One way of adapting SGMs to inverse problems is through conditional generation, where the conditional score can be derived with the measurement model. Many additional techniques, such as data consistency (Chung et al., ), manifold constraint (Chung et al., ; ) and others (Kawar et al., ), are further utilized to improve this adaptation.

Image-to-Image Schrödinger Bridges. A much harder but possibly also much more effective approach is to learn *direct* mappings between the distribution of signals $\mathbf{x}\sim p_{0}$ and measurements $\mathbf{y}\sim p_{1}$ instead of adapting pretrained models. In this line of research, Liu et al. () propose to learn such mappings by constructing a tractable subclass of Schrödinger bridges (SBs, Schrödinger ()), termed Image-to-Image Schrödinger Bridges (I^2SBs). The SB is an entropy-regularized optimal transport model, which, resembling the framework of SGMs, considers the following forward and backward SDEs:

$$ $\displaystyle\mathrm{d}\mathbf{x}_{t}$ $\displaystyle=(\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}})+\beta_{t}\nabla_{ \mathbf{x}_{t}}\log\bm{\Psi}(\mathbf{x}_{t},t))\mathrm{d}t+\sqrt{\beta_{t}} \mathrm{d}\mathbf{w},$ (4a) $\displaystyle\mathrm{d}\mathbf{x}_{t}$ $\displaystyle=(\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}})-\beta_{t}\nabla_{ \mathbf{x}_{t}}\log\widehat{\bm{\Psi}}(\mathbf{x}_{t},t))\mathrm{d}t+\sqrt{ \beta_{t}}\mathrm{d}\mathbf{\bar{w}},$ (4b) $$

Similarly to SGMs, the marginal densities of Eqs. 4a and 4b are equivalent. The functions $\bm{\Psi},\widehat{\bm{\Psi}}\in C^{2,1}(\mathbb{R}^{d},[0,1])$ represent time-varying energy potentials and are additionally constrained to solve the following partial differential equations

$$ $\displaystyle\begin{cases}\frac{\bm{\Psi}(\mathbf{x}_{t},t)}{\partial t}=- \nabla\bm{\Psi}^{\top}\mathbf{F}-\frac{1}{2}\beta\Delta\bm{\Psi}\\ \frac{\widehat{\bm{\Psi}}(\mathbf{x}_{t},t)}{\partial t}=-\nabla\cdot(\widehat {\bm{\Psi}}\mathbf{F})+\frac{1}{2}\beta\Delta\widehat{\bm{\Psi}}\end{cases}$ (5a) $\displaystyle\text{s.t. }\bm{\Psi}(\mathbf{x}_{0},0)$ $\displaystyle\widehat{\bm{\Psi}}(\mathbf{x},0){=}p_{0}(\mathbf{x}),\bm{\Psi}( \mathbf{x},1)\widehat{\bm{\Psi}}(\mathbf{x},1){=}p_{1}(\mathbf{x})$ (5b) $$

In general, numerically solving Eqs. 4a and 4b is much more difficult compared to SGMs due to nonlinear terms $\bm{\Psi},\widehat{\bm{\Psi}}$ being coupled via Eq. 5b. However, with Theorem 3.1, Liu et al. () show an important connection between the frameworks of SBs and SGMs. We repeat it here explicitly for later reference.

###### Theorem 1 (Reformulating SB drifts as score functions (Liu et al., 2023a ) )

If $\widehat{\bm{\Psi}},\bm{\Psi}$ fulfill the constraints given by Eq. 5, then $\nabla_{\mathbf{x}_{t}}\log\widehat{\bm{\Psi}}(\mathbf{x}_{t},t),\nabla_{
\mathbf{x}_{t}}\log\bm{\Psi}(\mathbf{x}_{t},t)$ are the score functions of the following linear SDEs, respectively

$$ $\displaystyle\mathrm{d}\mathbf{x}_{t}=\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}}) \mathrm{d}t+\sqrt{\beta_{t}}\mathrm{d}\mathbf{w},\quad\mathbf{x}_{0}\sim \widehat{\bm{\Psi}}(\cdot,0),$ (6) $\displaystyle\mathrm{d}\mathbf{x}_{t}=\mathbf{F}_{t}(\mathbf{\mathbf{x}_{t}}) \mathrm{d}t+\sqrt{\beta_{t}}\mathrm{d}\mathbf{\bar{w}},\quad\mathbf{x}_{1}\sim \bm{\Psi}(\cdot,1).$ (7) $$

Crucially, Theorem 1 states that, while $\widehat{\bm{\Psi}},\bm{\Psi}$ are not in general assumed to be valid probability distributions, it is true that $\nabla\log{\widehat{\bm{\Psi}}}=\nabla\log{p^{\ref{eq:i2sb_forward_sde}}}$ and $\nabla\log{\bm{\Psi}}=\nabla\log{p^{\ref{eq:i2sb_reverse_sde}}}$ for $p^{\ref{eq:i2sb_forward_sde}},p^{\ref{eq:i2sb_reverse_sde}}$ representing the densities of the respective SDEs. Following this theoretical result, Liu et al. () also show a principled approach for approximating $\nabla_{\mathbf{x}_{t}}\log\widehat{\bm{\Psi}}(\mathbf{x}_{t},t)$ with a neural network $\mathbf{s}_{\bm{\psi}}$. In essence, these results allow to train *direct* inverse problem solvers with the use of paired data, where $p_{0}$ represents a clean data distribution and $p_{1}$ the distribution of its corrupted measurements. Additionally, Liu et al. () show how I^2SB connects with flow-based optimal transport (OT) (Peyré & Cuturi, ; McCann, ), where assuming that $\beta_{t}\rightarrow 0$ leads to an ordinary differential equation (ODE) $\mathrm{d}\mathbf{x}_{t}=\mathbf{v}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0})
\mathrm{d}t$ that provides a deterministic mapping with the use of $\mathbf{s}_{\bm{\psi}}$ estimate. In practive, this is achieved by eliminating the noise from the intermediate sampling steps (see Algorithm 2).

Visual attribution methods. The very first works in the current era of Explainable Artificial Intelligence (XAI) were concerned with providing explanations of the model’s decision through visual heatmaps, which highlighted pixels considered important to the its prediction. One of the first approaches by Simonyan et al. () proposed simple backpropagation of the model’s output w.r.t. the input, indicating the direction of its greatest ascent in the pixel space, often termed as Saliency. More sophisticated approaches emerged in the following years, where techniques like Layer-wise Relevance Propagation (LRP, Bach et al. ()), Integrated Gradients (IG, Sundararajan et al. ()), DeepLift and Input $\times$ Gradient (Shrikumar et al., ), Guided Backpropagation (GuidedBackprop, Springenberg et al. ()), GradCAM (Selvaraju et al., ), Deconvolution (Zeiler & Fergus, ) and others that utilize the gradient of the neural network promised to indicate more semantically meaningful concepts in a less ’noisy’ manner. Concurrent line of research about perturbation-based methods assumed a more general black-box scenario, where explanations could be provided for a broader class of models. There, methods like Occlusion (Zeiler & Fergus, ), Local Interpretable Model-agnostic Explanations (LIME, Ribeiro et al. ()), SHapley Additive exPlanations (SHAP, Lundberg & Lee ()) and its variations quickly advanced the state-of-the-art. In this work, we utilize their unified implementations provided by the Captum package (Kokhlikyan et al., ).

###### Theorem 1 (Reformulating SB drifts as score functions (Liu et al., 2023a ) )

## Appendix C Extended Method

### C.1 Additional figures

We provide illustrative examples for the visual differences between $\mathbf{x}_{t}$ and the Tweedie’s estimate $\hat{\mathbf{x}}_{0}(\mathbf{x}_{t})$ in Fig. 8. For the effect of manipulating the $\tau$ hyperparameter, see Fig. 9

### C.2 Incorporating the classifier’s signal

In the following, we explicitly define the DDPM (Ho et al., ) sampler mentioned in Algorithm 1 and elaborate on the exact way of incorporating the classifier’s gradients into the generation process of I^2SB.

Denote by $\{t_{i}\}_{i\in\{0,...,N\}}$ the discrete sequence of timesteps of length N such that $0=t_{0}<t_{1}<\cdots<t_{N}=1$. By $\sigma^{2}_{n}=\int_{0}^{t_{n}}{\beta_{\tau}d\tau}$ and $\bar{\sigma}^{2}_{n}=\int_{t_{n}}^{1}{\beta_{\tau}d\tau}$, we denote the variances accumulated from each side. Additionally, let $\alpha^{2}_{n-1}=\int_{t_{n-1}}^{t_{n}}{\beta_{\tau}d\tau}$ be the variance accumulated between two consecutive timesteps. For ease of notation, we define $\mu_{n-1}$ and $\bar{\mu}_{n-1}$ as

$$ $\displaystyle\mu_{n-1}$ $\displaystyle=\frac{\alpha^{2}_{n-1}}{\alpha^{2}_{n-1}+\sigma^{2}_{n-1}},$ (8) $\displaystyle\bar{\mu}_{n-1}$ $\displaystyle=\frac{\sigma^{2}_{n-1}}{\alpha^{2}_{n-1}+\sigma^{2}_{n-1}}.$ (9) $$

With that, we can define the DDPM posterior sampler as

$$ $\displaystyle\mathbf{x}_{n-1}$ $\displaystyle\sim p(\mathbf{x}_{n-1}|\hat{\mathbf{x}}_{0},\mathbf{x}_{n}),$ (10) $\displaystyle\mathbf{x}_{n-1}$ $\displaystyle\sim\mathcal{N}\left(\mu_{n-1}\hat{\mathbf{x}}_{0}+\bar{\mu}_{n-1 }\mathbf{x}_{n},\frac{\alpha^{2}_{n-1}\sigma^{2}_{n-1}}{\alpha^{2}_{n-1}+ \sigma^{2}_{n-1}}I\right),$ (11) $$

where $\hat{\mathbf{x}}_{0}=\hat{\mathbf{x}}_{0}(\mathbf{x}_{n})$ denotes the Tweedie’s estimate obtained with $\mathbf{s}_{\bm{\psi}}$, *i.e*., a trained I^2SB.

When using the OT-ODE version of I^2SB, we replace sampling from the posterior with a deterministic version by following the mean, which yields the update rule

$$ $\mathbf{x}_{n-1}=\mu_{n-1}\hat{\mathbf{x}}_{0}+\bar{\mu}_{n-1}\mathbf{x}_{n}.$ (12) $$

By converting the Tweedie’s estimate to the conditional score using Eq. 3 and applying Bayes’ Theorem, we are left with

$$ $\begin{split}\mathbf{x}_{n-1}&=\mu_{n-1}\left(\mathbf{x}_{n}+\sigma^{2}_{n} \nabla_{\mathbf{x}_{n}}\log{p(\mathbf{x}_{n},n\mid\mathbf{y})}\right)+\bar{\mu }_{n-1}\mathbf{x}_{n}\\ &=\mu_{n-1}\left(\mathbf{x}_{n}+\sigma^{2}_{n}\nabla_{\mathbf{x}_{n}}\log{p( \mathbf{x}_{n},n)}+\sigma^{2}_{n}\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{y}\mid \mathbf{x}_{n},n)}\right)+\bar{\mu}_{n-1}\mathbf{x}_{n},\end{split}$ (13) $$

where $\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{x}_{n},n)}$ can be approximated by a standard I^2SB network trained on the task of inpainting. By manipulating Eq. 13 further, one can arrive at the following update rule

$$ $\begin{split}\mathbf{x}_{n-1}&=\mu_{n-1}\left(\mathbf{x}_{n}+\sigma^{2}_{n} \nabla_{\mathbf{x}_{n}}\log{p(\mathbf{x}_{n},n)}\right)+\mu_{n-1}\sigma^{2}_{n }\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{y}\mid\mathbf{x}_{n},n)}+\bar{\mu}_{n-1 }\mathbf{x}_{n}\\ &=\mu_{n-1}\left(\mathbf{x}_{n}+\sigma^{2}_{n}\nabla_{\mathbf{x}_{n}}\log{p( \mathbf{x}_{n},n)}\right)+\bar{\mu}_{n-1}\left(\frac{\mu_{n-1}\sigma^{2}_{n}}{ \bar{\mu}_{n-1}}\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{y}\mid\mathbf{x}_{n},n)} +\mathbf{x}_{n}\right)\\ &=\mu_{n-1}\left(\mathbf{x}_{n}+\sigma^{2}_{n}\nabla_{\mathbf{x}_{n}}\log{p( \mathbf{x}_{n},n)}\right)+\bar{\mu}_{n-1}\left(c_{n}\nabla_{\mathbf{x}_{n}} \log{p(\mathbf{y}\mid\mathbf{x}_{n},n)}+\mathbf{x}_{n}\right).\end{split}$ (14) $$

Here, we explicitly define the time-dependent coefficient $c_{n}$. While plugging $\nabla_{\mathbf{x}_{n}}\log{f(\mathbf{y}\mid\mathbf{x}_{n})}$ in place of $\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{y}\mid\mathbf{x}_{n},n)}$ in Eq. 13 is the most intuitive, we empirically verified that replacing $c_{n}\nabla_{\mathbf{x}_{n}}\log{p(\mathbf{y}\mid\mathbf{x}_{n},n)}$ with $\nabla_{\mathbf{x}_{n}}\log{f(\mathbf{y}\mid\mathbf{x}_{n})}$ leads to more semantically meaningful results. Practically, this can be explained by $\mu_{n-1}$ achieving its highest values at the end of the generation process, effectively incorporating the classifier’s signal to the highest extent in the final steps of the generation. Since we are interested in influencing the generative trajectory with the classifier $f$ along the entire process (and possibly decreasing its influence to the greatest possible extent in the final timesteps to avoid adversarial changes), it seems intuitive that incorporating $f$ into Eq. 14 allows for obtaining more meaningful RVCEs. This is due to $\bar{\mu}_{n}=1-\mu_{n}$, meaning that the classifier’s signal is amplified in the beginning of the generation and decreased in the final steps. This intervention also explains the effectiveness of the introduced improvements, as they break the independence of the classifier’s signal between consecutive steps, practically incorporating the time-dependent coefficient $c_{n}$ into the gradient alignment.

### C.3 Analytic posterior and OT-ODE

Following the original work of Liu et al. () (I^2SB), the analytic posterior from the forward stochastic process, which governs the mapping between a given boundary pair $(\mathbf{x}_{0},\mathbf{x}_{1})$, is defined as

$$ $q(\mathbf{x}_{t}|\mathbf{x}_{0},\mathbf{x}_{1})=\mathcal{N}\Big{(}\bm{\mu}( \mathbf{x_{0}},\mathbf{x_{1}},t)=\mathbf{x}_{0}+t(\mathbf{x}_{1}-\mathbf{x}_{0 }),\mathbf{\Sigma}_{t}=\alpha t(1-t)\mathbf{I}\Big{)},$ (15) $$

where by default $\alpha=1$. To arrive at the OT-ODE version of I^2SB, one must use $\alpha\rightarrow 0$, effectively reducing $q$ to a Dirac delta distribution centered at $\bm{\mu}(\mathbf{x_{0}},\mathbf{x_{1}},t)$.

## Appendix D Extended Experiments

We follow the evaluation protocol from previous works for VCEs on Imagenet, which, for a given task, uses all images from the training subset correctly predicted by the evaluated model. For ResNet50, this results in around 2000 images per task. We extract the results of other methods from the work of Farid et al. (), except the DVCE method (Augustin et al., ), which evalutes with a protocol that we were not able to fully reproduce. Hence, to ensure fair comparison, we adapted the implementation of DVCE to our evaluation. Specifically, we utilize the multiple-norm robust ResNet50 from the work of Boreiko et al. (), which the authors of DVCE propose as default, to achieve VCEs for the ResNet50 model. In terms of hyperparameters, we fine-tuned them with grid search on a subset of Zebra–Sorrel task and used $s=18.0$ as the guidance scale for the non-robust ResNet50, since it performed the best.

For I^2SB, we utilize the original checkpoint from Liu et al. () trained on $20-30\%$ freeform masks from Saharia et al. (). While the checkpoint trained on the $10-20\%$ variant is also available and verified to work within our framework, we discovered that the former generalizes well to smaller area values. Hence, for the sake of simplicity, we utilize the $20-30\%$ version only. By default, we use NFE=100, which we explored the most, but lower NFE regimes provided promising initial results. For the automated region extraction, we use IG by default, but evaluate 10 other attribution methods in Section D.5.

### D.1 Details of individual experiments

Fig. 3: Each improvement, together with the naive approach, is evaluated on the zebra-sorrel task with around 2000 images from ImageNet training set (following the protocol from the main experimental evaluation). Each image is initially predicted as either zebra or sorrel by the ResNet50 (He et al., ) model and the decision must be flipped to the opposite class. FID is computed between the obtained explanations and original images. The hyperparameter values used for all improvements are: $a=0.3$, $s=1.0$ (except ADAM stabilization with $s=1e-2$), $c=16$, $\tau=1.0$ (except trajectory truncation, where $\tau=0.6$).

Table 2: A: For all tasks, we use $s=1.5$ and $\tau=0.4$ (to better preserve the original content). As we cannot control the area of masks provided by LangSAM, hyperparameters $a$ and $c$ are not applicable in this scenario. Images with masks covering area greater than $40\%$ of the total image are discarded from the evaluation to ensure that we only use meaningful RVCEs. B: Across all tasks, the $10\%-20\%$ experiment uses configuration $B$ from Table 1, while the $20\%-30\%$ experiment uses configuration $C$. Hyperparameters $a$ and $c$ are not applicable, since masks are provided automatically from the mentioned dataset. C: Each inpainting algorithm is given a 24 A100 GPU hours time budget, resulting in around 2000 images for DDRM, 800 images for MCG and 400 images for RePaint on each task. Details of their adaptations are provided separately in Section D.3.

### D.2 Metrics description

In the following, we provide detailed description of each metric used in the quantitative evaluation.

FID and sFID (realism). Following works on image synthesis, measuring the realism of the obtained explanations at a distribution level is often done with FID and sFID (Heusel et al., ). Specifically, FID compares a set of real (*r*) and generated (*g*, in this case, the explanations) images by first extracting their corresponding features from the InceptionV3 network (Szegedy et al., ) and then computing

$$ $\text{FID}=||\bm{\mu}_{r}-\bm{\mu}_{g}||^{2}+\text{Tr}\left(\Sigma_{r}+\Sigma_ {g}-2\left(\bm{\Sigma}_{r}\bm{\Sigma}_{g}\right)^{1/2}\right),$ (16) $$

where $\bm{\mu}_{r},\bm{\mu}_{g}$ are the mean vectors and $\bm{\Sigma}_{r},\bm{\Sigma}_{g}$ are the covariance matrices of the respective distributions in the feature space. As comparing original images with their edited versions (*e.g*., explanations) may bias the metric with original pixels mostly unchanged, artificially boosting the realism evaluation, sFID first divides the sets into folds and averages FID over the independent counterparts.

S^3 (representation similarity).

Explanations should also resemble original images from a representation respective. Here, following the work of Jeanneret et al. (), we compute average SimSiam Similarity (S^3) over a set of original images and the resulting counterfactuals. Specifically, S^3 utilizes a SimSiam network (Chen & He, ), which encodes both the factual and counterfactual images into their respective representations $\mathbf{r}_{f}$, $\mathbf{r}_{cf}$ and computes the cosine similarity as

$$ $\text{S}^{3}=\frac{\mathbf{r}_{f}\cdot\mathbf{r}_{cf}}{||\mathbf{r}_{f}||_{2} \cdot||\mathbf{r}_{cf}||_{2}}.$ (17) $$

COUT (sparsity). In the context of VCEs, sparsity is understood as perturbing a minimal number of pixels to flip the model’s decision. To quantify this criterion, the COUnterfactual Transition (COUT) metric computes

$$ $\begin{split}\text{COUT}=&\text{AUPC}_{y}-\text{AUPC}_{y^{*}},\\ \text{AUPC}_{k}=&\frac{1}{M}\sum_{m=0}^{M-1}\frac{1}{2}(f(k\mid\mathbf{x}^{(m) })-f(k\mid\mathbf{x}^{(m+1)}))\end{split}$ (18) $$

where $\mathbf{x}^{(0)}$ is the factual image, $\mathbf{x}^{(M)}$ the resulting VCE, and $y^{*},y$ are the class labels predicted by $f$ for $\mathbf{x}^{(0)},\mathbf{x}^{(M)}$ respectively. In practice, COUT measures how fast the classifier’s decision changes when interpolating between the original and the explanation, but the interpolation is defined as inserting pixels to the original image according to the extent (absolute value) of change observed in the VCE through $M$ steps. COUT is typically reported as an average over a set of samples.

Flip Rate (efficiency). A major criterion for a VCE method is its efficiency, understood as the ability to effectively flip the model’s decision. For a set of triplets $\{\mathbf{x}^{*}_{i},\mathbf{x}_{i},y_{i}\}_{i=1}^{I}$, where $\mathbf{x}^{*}_{i}$ is the original image and $\mathbf{x}_{i}$ is the resulting VCE targeted to flip $f$’s decision to $y_{i}$, Flip Rate (FR) is defined as the fraction of cases which correctly flipped the decision to the target class, *i.e*.,

$$ $\text{FR}=\frac{1}{I}\sum_{i=1}^{I}\mathbf{1}(\operatorname*{arg\,max}_{y^{ \prime}}f(y^{\prime}\mid\mathbf{x}_{i})=y_{i}).$ (19) $$

### D.3 Adaptation of other inpainting algorithms

In this subsection, we describe the adaptation of each inpainting algorithm from our ablation study. For each method, we follow the notation from its corresponding original work to make the description easier to follow.

#### D.3.1 Manifold Constrained Gradient (MCG, Chung et al. ( 2022 ) )

MCG iteratively denoises (inpaints) the missing parts with the following two-step update (Equations 14 and 15, Chung et al. ()):

$$ $\displaystyle\mathbf{x}_{t-1}^{\prime}$ $\displaystyle=\mathbf{f}(\mathbf{x_{i}},\mathbf{s_{\theta}})-\frac{\partial}{ \partial\mathbf{x_{t}}}\lVert\textbf{W}(\textbf{y}-\textbf{H}\mathbf{\hat{x}_{ 0}}(\mathbf{x_{t}})\rVert_{2}^{2}+g(\mathbf{x_{t}})\mathbf{z},\quad\mathbf{z} \sim\mathcal{N}(0,\textbf{I})$ (20) $\displaystyle\mathbf{x_{i-t}}$ $\displaystyle=\textbf{A}\mathbf{x^{\prime}}_{t-1}+\textbf{b}$ (21) $$

where Eq. 20 is a manifold constraint update and Eq. 21 is a data consistency step. As described by the authors, both steps are crucial to ensure that the gradient of the measurement term stays on the manifold and to deal with the potential deviation from the measurement consistency.

Since $\mathbf{f}(\mathbf{x}_{t},\mathbf{s}_{\bm{\theta}})$ implicitly predicts the mean $\mu_{t}$ and variance $\sigma_{t}$ at each step $t$, related to the underlying SDE dynamics, we apply our guidance scheme by modifying the original

$$ $\mathbf{f}(\mathbf{x_{i}},\mathbf{s_{\theta}})=\mu_{t}+\sigma_{t}\mathbf{z}, \quad\mathbf{z}\sim\mathcal{N}(0,\textbf{I})$ (22) $$

by adding properly scaled (according to the relationship between the likelihood score and mean) conditioning:

$$ $\mathbf{f^{\prime}}(\mathbf{x_{i}},\mathbf{s_{\theta}})=\mu_{t}+\sigma_{t} \mathbf{z}+s\cdot\sigma_{t}^{2}\frac{\bar{\mathbf{g}}_{n}}{g},\quad\mathbf{z} \sim\mathcal{N}(0,\textbf{I})$ (23) $$

where $\mathbf{g}_{n}$ and $g$ are obtained as described in Algorithm 3.

#### D.3.2 Denosing Diffusion Restoration Models (DDRM, Kawar et al. ( 2022 ) )

DDRM considers the SVD decomposition of the measurement model matrix $\mathbf{H}$ in the linear noisy inverse problem

$$ $\mathbf{y}=\mathbf{H}\mathbf{x}+\sigma_{\bm{y}}\mathbf{z},\mathbf{z}\sim \mathcal{N}(0,\textbf{I}),$ (24) $$

where $\sigma_{\bm{y}}$ is the standard divination of the measurement noise.
For the task of inpaiting, the $\mathbf{H}$ matrix is a diagonal matrix with either 0 0 or $1$ on the diagonal indicating available and missing pixels. Hence, its SVD decomposition simplifies to using identity matrices in place of $\mathbf{U}$ and $\mathbf{V}$.

The main contribution of DDRM is that it provides a way to include the information from that decomposition and observation $\mathbf{y}$ into the generative process, which the authors summarize in Equations 7 and 8 in the original work. The method uses a trained denoising network to obtain a prediction of $\mathbf{x}_{0}$ at timestep $t$, denoted as $\color[rgb]{.5,0,.5}{\mathbf{x}_{\theta,t}}$. In order to include the additional information from the classifier into the DDRM framework, we modify that prediction to include the model’s gradients by replacing the update rule

$$ $\color[rgb]{.5,0,.5}\mathbf{\bar{x}}_{\theta,t}\color[rgb]{0,0,0}=V^{T}\color[ rgb]{.5,0,.5}\mathbf{x}_{\theta,t}$ (25) $$

with

$$ $\color[rgb]{.5,0,.5}\mathbf{\bar{x}}_{\theta,t}\color[rgb]{0,0,0}=V^{T}(\color [rgb]{.5,0,.5}\mathbf{x}_{\theta,t}\color[rgb]{0,0,0}+s\frac{\bar{g}}{g}),$ (26) $$

where $\bar{g}$ and $g$ are obtained as described in Algorithm 3.

#### D.3.3 RePaint (Lugmayr et al., 2022 )

RePaint performs the task of inpainting by modifying the standard denoising process, where, at each timestep $t$, the network’s input is composed of noised pixels known from the original input sampled directly from $q$ and the unknown noisy pixels predicted by the network in the previous timestep. Additionally, to harmonize the two parts of the image, RePaint samples $x_{t}$ directly from $q(x_{t}|x_{t-1})$ and repeats the forward procedure. By default, this resampling scheme is repeated $20$ times for each of the standard diffusion steps.

In order to incorporate the information from the classifier, we modify the unconditional mean of the posterior $p_{\theta}$ in the denosing step to conditional one, effectively replacing the mean predictor

$$ $\bm{\varepsilon}_{\bm{\theta}}(X_{t},t)$ (27) $$

shown in the $7$-th step of Algorithm $1$ from the original work (Lugmayr et al., ) with

$$ $\bm{\varepsilon}_{\bm{\theta}}(X_{t},t)+s\cdot\sigma^{2}_{t}\frac{\bar{g}}{g}$ (28) $$

where $\bar{g}$ and $g$ are obtained as described in Algorithm 3.

### D.4 Schedulers for guidance scale

Figure 10 visualizes example schedulers used throughout the development of our method. Our adaptive normalization technique empirically outperformed all tested schedulers.

Figure: (a) Interval scheduler
Refer to caption: x10.png

### D.5 Quantitative evaluation of other attribution methods

To pick a default attribution method for RCSB, we evaluated it on the Zebra–Sorrel using the RCSB^B hyperparameter configuration for $11$ different attribution methods shown in Table 3. Based on these results, we chose Integrated Gradients (Sundararajan et al., ) as the default, since it provides the most balanced performance.

**Table 3: Quantitative evaluation of 11 attribution methods (described in Appendix B) on the Zebra–Sorrel task following our evaluation protocol.**
| Zebra – Sorrel |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Attribution method | FID | sFID | S^3 | COUT | FR |
| LRP | $\mathbf{7.5}$ | $\mathbf{15.5}$ | $\mathbf{0.87}$ | $0.62$ | $93.6$ |
| InputXGradient | $\underline{9.0}$ | $\underline{16.8}$ | $\underline{0.87}$ | $0.73$ | $\underline{97.8}$ |
| DeepLift | $9.2$ | $17.0$ | $0.87$ | $0.73$ | $\mathbf{97.9}$ |
| Integrated Gradients | $9.5$ | $17.4$ | $0.86$ | $0.72$ | $97.4$ |
| GradientShap | $10.5$ | $18.5$ | $0.87$ | $\underline{0.74}$ | $97.4$ |
| LIME | $12.9$ | $20.7$ | $0.85$ | $0.55$ | $88.4$ |
| GuidedBackprop | $13.8$ | $21.49$ | $0.86$ | $0.72$ | $96.5$ |
| Occlusion | $13.9$ | $21.7$ | $0.86$ | $0.50$ | $86.0$ |
| GradCAM | $14.1$ | $22.15$ | $0.85$ | $0.52$ | $87.1$ |
| GuidedGradCAM | $15.1$ | $22.5$ | $0.86$ | $0.71$ | $96.1$ |
| Saliency | $15.2$ | $23.0$ | $0.86$ | $\mathbb{0.75}$ | $98.4$ |

### D.6 Diversity assessment

RCSB utilizes the OT-ODE version of the I^2SB, which provides a deterministic mapping between the noisy image and the resulting RVCE. The source of randomness comes from the Gaussian noise inserted into the image in the place of missing pixels at the beginning of the inpainting process. In order to examine the diversity of the generated RVCEs, we followed the evaluation procedure from the work of Jeanneret et al. (). In essence, we compute the mean pair-wise LPIPS metric between two runs with different seeds (used in generation of the Gaussian noise) for our three main configurations of hyperparameters RCSB^A, RCSB^B and RCSB^C. For each run, $256$ RVCEs were generated. Results are shown in Table 4. Naturally, decreasing the area hyperparameter limits the extent of possible changes, leading to a decrease in diversity. Picking $a=0.3$ results in diversity comparable to values reported by previous works, *e.g*., Jeanneret et al. ().

**Table 4: Diversity evaluation using $256$ images for each task from our experimental protocol.**
|  | Zebra – Sorrel | Cheetah – Cougar | Egyptian Cat – Persian Cat |
| --- | --- | --- | --- |
| RCSB^A | $0.067$ | $0.060$ | $0.065$ |
| RCSB^B | $0.092$ | $0.096$ | $0.095$ |
| RCSB^C | $0.129$ | $0.140$ | $0.137$ |

### D.7 Computational efficiency assessment

What connects previous SOTA SGM-based methods with our work is the use of large U-Net (Ronneberger et al., ) checkpoints for the denoising network with the number of hyperparameters far exceeding (*e.g*., $10\times$) the size of the utilized classifier, effectively dominating the computational burden. Hence, to ensure fair comparison, Table 5 shows the number of Neural Function Evaluations (NFEs) used by each method to produce a single explanation, divided into a. model (*U-Net*, *classifier* and *other*) b. and *forward* / *backward* passes, where the backward pass is around $2\times$ more computationally demanding than the forward pass. Importantly, this comparison eliminates the differences stemming from the utilized hardware and optimality of the implementation, and is fair as each method consumes virtually the same amount of GPU memory. One exception to the use of the standard U-Net model is the LDCE method of Farid et al. (), which applies it in the latent space of an autoencoder. However, as each latent U-Net step also requires decoding the image with the decoder, the computational demand stays similar to the standard approach.

As indicated by Table 5, RCSB is the most efficient approach, both in terms of balancing the use of the U-Net and the classifier, and the number of forward/backward passes. Importantly, the *other* category shows non-zero numbers only for the DVCE method, which additionaly uses the gradients of a robust classifier in the generation process. The high number of forward/backward passes through the classifiers in DVCE stems from applying them to a set of $16$ augmented versions of $\mathbf{x}_{t}$ at each timestep.

**Table 5: Number of NFEs for each respective method with details about the model type and forward/backward passes.**
| NFE |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Inpainting method | U-Net | Classifier | Other |  |  |  |
| forward | backward | forward | backward | forward | backward |  |
| RCSB | $100$ | $100$ | $100$ | $100$ | 0 0 | 0 0 |
| LDCE | $191$ | $191$ | $191$ | $191$ | 0 0 | 0 0 |
| DVCE | $200$ | $200$ | $1600$ | $1600$ | $1600$ | $1600$ |
| ACE | $520$ | $500$ | $25$ | $25$ | 0 0 | 0 0 |
| DDRM | $200$ | $200$ | $200$ | $200$ | 0 0 | 0 0 |
| MCG | $1000$ | $1000$ | $1000$ | $1000$ | 0 0 | 0 0 |
| RePaint | $2410$ | $2410$ | $2410$ | $2410$ | 0 0 | 0 0 |

### D.8 Additional quantitative results

For most visually appealing results, we found $a\approx 0.1-0.15$, $c\approx 8-16$, $\tau\approx 0.3-0.6$ and $s\approx 2-3$ to perform the best. These hyperparameters were used to create RVCEs for Fig. 5. To assess that the performance of these configurations does not deviate from best configurations of Table 1, we followed the same evaluation protocol on the most challenging Zebra–Sorrel task and include the results in Table 6. Crucially, the performance stays virtually the same when comparing with Table 1.

**Table 6: Quantitative results for hyperparameters that provide the most visually appealing results.**
| Area $a$ | Cell size $c$ | Guidance scale $s$ | Trajectory truncation $\tau$ | FID | sFID | S^3 | COUT | FR |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| $0.1$ | $16$ | $3.0$ | $0.2$ | $10.1$ | $18.4$ | $0.92$ | $0.79$ | $95.8$ |
| $0.3$ | $10.7$ | $19.0$ | $0.91$ | $0.76$ | $95.0$ |  |  |  |
| $0.4$ | $10.8$ | $18.9$ | $0.90$ | $0.74$ | $94.3$ |  |  |  |
| $3.5$ | $0.2$ | $11.0$ | $19.2$ | $0.91$ | $0.81$ | $97.0$ |  |  |
| $0.4$ | $11.4$ | $19.4$ | $0.89$ | $0.77$ | $96.0$ |  |  |  |
| $0.3$ | $11.6$ | $19.7$ | $0.90$ | $0.79$ | $96.2$ |  |  |  |
| $4.0$ | $0.2$ | $11.7$ | $19.8$ | $0.91$ | $0.83$ | $97.2$ |  |  |
| $0.4$ | $12.3$ | $20.2$ | $0.88$ | $0.79$ | $96.7$ |  |  |  |
| $0.3$ | $12.4$ | $20.4$ | $0.89$ | $0.80$ | $96.2$ |  |  |  |
| $0.15$ | $16$ | $2.0$ | $0.4$ | $11.2$ | $19.2$ | $0.89$ | $0.77$ | $97.7$ |
| $0.5$ | $12.1$ | $20.0$ | $0.88$ | $0.74$ | $96.5$ |  |  |  |
| $0.6$ | $12.7$ | $20.4$ | $0.86$ | $0.70$ | $94.8$ |  |  |  |
| $3.0$ | $0.4$ | $13.5$ | $21.3$ | $0.87$ | $0.82$ | $99.5$ |  |  |
| $0.5$ | $13.9$ | $21.7$ | $0.86$ | $0.80$ | $99.2$ |  |  |  |
| $0.6$ | $14.2$ | $21.7$ | $0.85$ | $0.78$ | $98.6$ |  |  |  |
| $4.0$ | $0.4$ | $15.3$ | $22.9$ | $0.86$ | $0.84$ | $99.6$ |  |  |
| $0.5$ | $15.5$ | $23.1$ | $0.85$ | $0.82$ | $99.7$ |  |  |  |
| $0.6$ | $15.8$ | $23.5$ | $0.83$ | $0.81$ | $99.4$ |  |  |  |

## Appendix E Qualitative examples

We provide additional qualitative examples for different scenarios. Results for other classifiers, obtained with the automated region extraction, are depicted in Fig. 11 (VGG16, Simonyan & Zisserman ()), Fig. 12 (VGG16 with Batch Normalization (BN), Simonyan & Zisserman ()), Fig. 13 (ConvNeXt Base, Liu et al. ()), Fig. 14 (ViT-B/16, Dosovitskiy et al. ()), Fig. 15 (SwinB, Liu et al. ()) , Fig. 16 (robust Madry ResNet50, Engstrom et al. ()), Fig. 17 (robust Tian DeiT, Tian et al. ()), Fig. 18 (zero-shot CLIP classifier, Radford et al. ()). We used $a=0.2,\tau=0.7,c=16,s=1.5$ universally across all additional classifiers, showcasing the versatility of RCSB. Additional examples for ResNet50 are shown in Figs. 21, 22, 20 and 19 (automated region extraction with different hyperparameters), Fig. 23 (exact regions from LangSAM) and Fig. 24 (user-defined regions).

Figure: Figure 11: Extended qualitative evaluation of automated region extraction for VGG16 (Simonyan & Zisserman, ) classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x13.png

Figure: Figure 12: Extended qualitative evaluation of automated region extraction for VGG16BN (Simonyan & Zisserman, ) classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x14.png

Figure: Figure 13: Extended qualitative evaluation of automated region extraction for ConvNeXt Base (Liu et al., ) classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x15.png

Figure: Figure 14: Extended qualitative evaluation of automated region extraction for ViTB16 (Dosovitskiy et al., ) classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x16.png

Figure: Figure 15: Extended qualitative evaluation of automated region extraction for SwinB (Liu et al., ) classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x17.png

Figure: Figure 16: Extended qualitative evaluation of automated region extraction for Madry ResNet50 (Engstrom et al., ) $l_{2}$-norm robust classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x18.png

Figure: Figure 17: Extended qualitative evaluation of automated region extraction for Tian DeiT (Tian et al., ) corruption robust classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Refer to caption: x19.png

Figure: Figure 18: Extended qualitative evaluation of automated region extraction for CLIP ViT-B/32 (Radford et al., ) zero-shot classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x20.png

Figure: Figure 19: Extended qualitative evaluation of automated region extraction with $c=4$, $a=0.1$ for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x21.png

Figure: Figure 20: Extended qualitative evaluation of automated region extraction with $c=4$, $a=0.3$ for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x22.png

Figure: Figure 21: Extended qualitative evaluation of automated region extraction with $c=4$, $a=0.2$ for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE on the right.
Refer to caption: x23.png

Figure: Figure 22: Extended qualitative evaluation of automated region extraction with $c=8$, $a=0.2$ for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Refer to caption: x24.png

Figure: Figure 23: Extended qualitative evaluation of exact regions obtained with LangSAM for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Refer to caption: x25.png

Figure: Figure 24: Extended qualitative evaluation of user-defined regions for the ResNet50 classifier. For each task, factual image is shown on the left with the used region in the middle and the generated RVCE(s) on the right.
Refer to caption: x26.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See strengths and weaknesses.
soundness: 3
strengths: The paper is clearly written, with most details of the proposed method presented effectively. It includes numerous qualitative results that demonstrate the method’s effectiveness and clearly define the problem.
summary: This paper addresses the task of Visual Contour Factual Explanations (VCE) by introducing regional constraints to enhance the specificity and clarity of the explanations. Additionally, it proposes a method, RCSB, to generate counterfactual results based on region-specific information. The effectiveness of the proposed approach is evaluated on both generated and manually designed regions.
weaknesses: 1. The paper demonstrates that the proposed method achieves better performance when a semantic region constraint is applied. However, it would be helpful to explore the effects when the region is not semantically relevant—such as when using a randomly selected area with similar shape and size—to understand how critical semantic relevance is to the method’s effectiveness.



2. In Table 1, three configurations of the proposed method are presented as settings A, B, and C. While it appears these configurations differ based on specific hyperparameter choices, could you provide additional context or rationale for highlighting these particular settings? Are there theoretical or practical reasons to show them in this table?



3. The results for LangSAM-based regions have a considerably higher FID score than those for other regions, suggesting that this style of regions may pose unique challenges. Could you clarify why this region is so different or particularly difficult for the method? Additional explanation on this would provide valuable insight into the model’s limitations.



4. The paper briefly mentions "confirmation bias," but could you expand on how this bias is defined in the original problem and mitigated by the proposed method?

---

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: In addition to my questions & suggestions presented above under 'Weaknesses', just a few minor remarks:
- In Fig. 1, what is the "recent SOTA method" used? 
- In the caption of Fig. 2, the authors write that "Intermediate images of I2SB are much closer to the data manifold." - I assume that by this, the authors mean that most of the image is 'natural' throughout the whole generation procedure? I was a bit confused here at first, because the trajectory of the cat's face looks very similar in both rows (as would be expected).
- Did the authors consider using the *inverse* of their mask (i.e., only allowing the model to change the background) for assessing a model's robustness to shortcut features?
soundness: 4
strengths: - The paper is very well-written overall and generally a pleasure to read; the various technical aspects of the method are very well-motivated, rigorously and clearly presented, and demonstrated to be useful in ablation experiments.
- Region-constrained counterfactual explanations are certainly very useful (although I would argue that this approach is not completely novel, cf. my comments below) and I believe the explicit problem formulation and framing here can help move this field forward. The extensive and clearly explained examples are also much appreciated.
- The I2SB-based conditional inpainting approach appears to perform very well.
- An interesting approach for (ADAM-derived) gradient stabilization and normalization in classifier-guided diffusion
summary: The authors propose a method for generating region-constrained visual counterfactual explanations, arguing that the region constraint leads to improved interpretability of the resulting counterfactual explanations (compared to unconstrained CEs). For this purpose, they leverage i) Image-to-Image Schrödinger Bridges, ii) automatically determined regions of interest, using either classical visual attribution methods or a text-to-object segmentation foundation model (LangSAM), and iii) several tweaks for stabilizing classifier guidance of the diffusion process. The proposed method outperforms several baselines on previously identified particularly challenging counterfactual questions on ImageNet.
weaknesses: **The relationship to several prior works should be discussed more comprehensively**, and possibly some of these methods should be included as baselines. The explicitly novel contributions should be highlighted more clearly.
- Both [Jeanneret et al. (2023)](https://openaccess.thecvf.com/content/CVPR2023/papers/Jeanneret_Adversarial_Counterfactual_Visual_Explanations_CVPR_2023_paper.pdf) and [Weng et al. (2024)](https://link.springer.com/chapter/10.1007/978-3-031-73016-0_20) also use automatically determined region constraints and conditional inpainting to obtain region-constrained counterfactual explanations. In several places it reads as if the authors claim novelty on region-constrained CEs *per se*, or the use of conditional inpainting for this purpose.
- Concerning the challenge of obtaining meaningful gradients with respect to noisy (intermediate) images, as discussed on page 4, this has been addressed before in various approaches such as DiME (by computing gradients with respect to denoised images instead, which is computationally expensive). Both FastDiME and [He et al.](https://openreview.net/forum?id=o3BxOLoxm1) have (more?) efficient implementations of the Tweedie approach. 
- [Karunratanakul et al. (2023)](https://openaccess.thecvf.com/content/ICCV2023/papers/Karunratanakul_Guided_Motion_Diffusion_for_Controllable_Human_Motion_Synthesis_ICCV_2023_paper.pdf) is also very related and contains many of the same ingredients presented here. Again, the precise differences should be discussed.

**For SOTA claims, more extensive experiments and standard benchmarks would be needed.** 
- Performance is only evaluated quantitatively on three particular tasks previously identified to be particularly challenging for competitor methods. It seems unfair to specifically compare on the case where baselines are known to perform poorly, but not on the ones where they perform well. More generally, a SOTA claim for VCEs would require more extensive experiments. E.g., a comparison on CelebA-smile would already be interesting.
- As noted above, I would suggest adding several more baseline methods such as DiME, FastDiME, and possibly combinations of these methods. E.g. it should be relatively straightforward to implement an I2SB version of FastDiME. This would help further disentangle the benefits provided by the different method components (I2SB, different auto-masking compared to prior work, gradient estimation & stabilization) presented here.

**Overclaims regarding trustworthiness and 'causality' of the resulting explanations**. While I am generally on board concerning the utility of region-constrained VCEs, the authors do go quite a bit overboard in their claims about them, I believe.
- If a VCE is constrained to only change a specific part of the image, this may yield to more intuitively appealing explanations, but the model might actually still be looking at weird shortcut features in other parts of the image (outside the mask). One would then miss this crucial information. This could happen in the proposed method e.g. due to non-localized shortcuts (e.g. [Wang et al.](https://openaccess.thecvf.com/content/ICCV2023/html/Wang_What_do_neural_networks_learn_in_image_classification_A_frequency_ICCV_2023_paper.html)), where every individual pixel of the background would receive a low importance score. It would also miss weak shortcut influence, where the shortcut doesn't completely dominate predictions but still subtly skews confidences scores, such as in racial/ethnic shortcuts in medical imaging ([Zou et al.](https://www.science.org/doi/full/10.1126/science.adh4260), [Yang et al.](https://www.nature.com/articles/s41591-024-03113-4)). Personally, I would be *very* careful with explanation methods that explicitly constrain the explanation to those parts of an image that we believe to be useful a priori. (Cf. e.g. the case study in Fig. 5 in regard to which it is claimed that "RCSB allows for causal inference about the model’s reasoning" - but if we constrain explanations to LangSAM segmentations, we might miss that the classification model is actually primarily looking elsewhere.) The whole setup could thus be seen as strongly *increasing* the risk for confirmation bias, which the authors claim to combat using their method. Also cf. the example in Fig. 1, where it could actually be considered very informative that the counterfactual explanation changes the copyright caption (the authors mention this as a flaw), because it might indicate potential shortcut learning! As a final example in this regard, the authors write that "In this case, RCSB allows for increasing trust in the model, as making the lemons more orange correctly modifies its decision." - but again, this trust is unwarranted: the importance of the object color might in fact be negligible compared to the importance of some background shortcut feature.
- Relatedly, the relative importance of different features is lost in region-constrained VCEs. The fact that one *can* change the head of a bird such that it is classified differently does not mean that these specific properties of the head are actually important to the model's classification. Again, this severely limits any claims of "inferring causally about the model's predictions" - we still don't know which features were actually important for the classification.

To reiterate, I am generally sympathetic to the approach pursued here. However, the discussion of its merits and limitations (!) should be much more nuanced.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - I'm wondering what will be generated if we force the label of the generated image to a very different class. For example, if we want to change the output of the model given the image of an animal to the class of a dining table, what does the generated image look like?

- There are some cases in which the model indeed makes use of unusual visual cues to predict the label of the image, like the background. We may also want to know that the model has learned such a prior from the dataset (for example, the model links the snow background to the wolf due to dataset bias). How can the proposed method discover such cases?

- The visualization shown in Figure 5 is interesting. At the same time, I noticed that after changes, several images changed the image's attributes significantly toward the target class (like the Flamingo and Cauliflower cases). I'm wondering what can we learn from these changes to understand the model's behaviors, as the model's prediction on these changed samples is very reasonable and as expected.
soundness: 3
strengths: - Bring the region constraint to the counterfactual example generation somehow makes sense to better align with human perception.

- The authors provide a way to better inject the guidance from the gradient of the classifier f into the generation process of the inpainting model. From the visual examples, we can indeed observe a much higher quality.
summary: This manuscript studies the problem of visual counterfactual explanations (VCEs) and argues that region is important. It is an extension of the visual counterfactual explanation by adding a hard region constraint. The method is based on a recently developed diffusion-based image-to-image translation method, named Image-to-Image Schrodinger Bridge ($I^2$SB). The authors discover a good way to inject the gradient from the classifier into the generation process of $I^2$SB and make the $I^2$SB generate more plausible images that change the prediction of the classifier. It gives us a more flexible way to understand the model prediction process.
weaknesses: - As the regions are the core input of the proposed method, how to define/find the regions is crucial. The method uses either the attribution map generation or the grounding method. Will these methods introduce additional bias in the understanding of the classifier? 

- From my perspective, the proposed method is more like just an image painting with specific guidance, where the guidance comes from the gradients of the classification model. The quality of the explanation ability somehow entangles the ability of the classifier itself and the inpainting model. The quality (FID, etc,) of the generated samples is also highly entangled with the base inpainting models. This might be a problem if we want to understand the behaviors of the classifier.

- The visual counterfactual explanation is similar to the problem of attribution map generation, where the goal is to detect the regions that have the most influence on the prediction of the model either positively or negatively. Actually, one of the region generation methods is attribution map generation (IG). If we use the attribution map generation method, we can also generate some examples with high FID while with different predicted labels. I wondered what's the core difference between these two setups.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. The paper claimed that the definition of VCEs has potential confirmation bias. However, it is not deeply investigated. Besides, the teaser example in Figure 1 also cannot indicate such confirmation bias.
2. There is no direct qualitative comparison between different methods, in Figure 5 and 6.
soundness: 3
strengths: 1. The newly formulated RVCEs is reasonable. 
2. The presentation is good and the paper is easy to follow.
3. The experiments demonstrates the effectiveness of the proposed method.
summary: This paper targets at the task of VCEs and defines more refined task with region constraints to gain more precise explanations. Further, it also proposes the method RCSB for the newly defined task, which is built upon the I^2SB and solves the problem as a conditional inpainting task. The idea is simple and the experiments demonstrates its effectiveness. Nevertheless, the technical novelty is relatively limited.
weaknesses: 1. The technical novelty is relatively limited. It formulates the task as a conditional inpainting problem and solves it by applying $I^2SB$.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - The comparison models in the main experiment (Table 1) are not defined anywhere in the paper -- the abbreviations used in the table are never used in the text, and the table has no citations to tell you what the abbreviations might mean. I think I was able to work out what these models are through Googling but they should be named in the paper (though it's worth noting that there is another XAI method called "ACE").
- Since the evaluation dataset seems to be restricted to class pairs that favor the proposed method (cases where the two classes have the same shape but different surface color/texture), I'd be curious to know how the proposed model compares on a wider set of counterfactual pairs. Does the proposed method outperform ACE, etc. on arbitrary counterfactual pairings, where the key differences between the classes might include shape changes?
- Can this method be helpful for users to understand adversarial attacks (if the input image is a adversarial image, can the counterfactuals help explain why the model saw the image as the wrong category)?
soundness: 3
strengths: - The proposed method outperforms similar approaches on the proposed metrics, at least on a very small evaluation dataset (3 class pairs).
- The writing is very clear, and the results are very well organised and presented, with well-designed figures to illustrate the model's performance.
- The paper shows some interesting possible applications, such as allowing users to target specific regions of an image to see those regions would need to change to change the model's decision.
summary: This paper presents a method for creating visual counterfactual explanations for classification tasks such as ImageNet classification. These explanations illustrate how an image could be modified to change the classification output. The approach is based on a combination of existing methods which select the most relevant parts of the image to change the decision and then mask and inpaint these regions to change the classification decision. A major contribution of the proposed method is limiting the masking/inpainting to a small connected spatial region in the image rather than allowing changes everywhere. The method outperforms other approaches on a set of metrics which aim to balance various factors desirable for visual counterfactuals (namely, changing the model’s prediction while keeping the output image similar to the input, keeping changes sparse, and ensuring the output is realistic-looking).
weaknesses: - The main contribution of the paper (limiting the inpainting to a specific region) also seems like a weakness, since the model cannot capture cases where the differences between two classes are changes in shape. (For example, this is shown the American chameleon <-> common iguana examples -- the main difference between the two types of lizard is the overall shape/size, but the model can only show how the texture/color differs between these classes.)
- The intro says that the goal of generating counterfactuals is to explain the decision-making process of an image classifier to a user, but it's not clear whether the proposed model is better than any other model at achieving that result. This approach performs best on the metrics, but simply maximizing performance across metrics seems like it would result in a "counterfactual" image that flips the classifier's decision with almost no visible changes to the image, which wouldn't be a useful counterfactual for human users (as they wouldn't be able to tell what changed).

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: Apart from the recommendations in the weaknesses section I have the following questions and suggestions:
- Highlight the possible use cases of the method (e.g. class confusion) more so that the value of the method can be better understood
- Avoid using emphasizing phrases, e.g. by a large margin, and not bringing a quantitative results or at least a reference
- To many details and efforts with respect to the evalution are in the appendix weakening the main paper. I would suggest to include more details in the main paper from the additional evaluation results on different generative models and classifiers and reducing some of the visuals in the main paper + some very detailed analysis of the shown images and come up with a more condensed evaluation but including all the insights.
- What are the limitations of the method?
- It was not clear to me If there are not many possible image modifications for one sample yielding into a class change. How is that restricted to find the “best” modification and what are the optimal criteria?
soundness: 3
strengths: - Well written text that is good to follow as a reader
- Despite there a different attempt to condition the generation process, the idea is novel enough when including the optimization steps of the method bringing XAI methods
- The theoretic description of the method is well formulated with respect to quality and clarity
- The structure of the evaluation has a high quality
summary: The authors propose a method for visual counterfactual explanation which is based upon methods like latent diffusion counterfactual explanation by Farid et al. and modified the previous approaches by adding a spatial (region) constraint, which allows more precise target generation of new images containing features of the target class. Thereby, also an external classifier is used as guidance for the generation process, e.g. for a stable diffusion model. Furthermore, the authors added optimization techniques to overcome limitation with respect to generated artefacts, color balance and realism (measured via FID score). Furthermore, the authors introduced also an approach for generating the regions automatically to enhance the usability of the method. In the evaluation, the authors show improved realism via FID score, effectiveness via flip rate, similarity and sparsity in comparison to current state-of-the-art methods (e.g. ACE or LDCE). In addition, qualitative results are presented showing the applicability of the method for e.g. discovering complex patterns. The authors claim finally that they could improve on existing methods by a large margin.
weaknesses: - Only tested in a limited dataset, the authors should consider applying the method also on different datasets or domains
- From the results in the main paper, it could be confirmed that the method works on different generative models and external classifiers but claimed that results are available in the appendix. Therefore, I encourage the authors to include at least the numeric results in the main paper, so that a reader can obtain insights on the performance for different domains and models.
- The use of FID score is misleading the comparison to other methods, that do not limit the generation process by regions, since the limitation to certain spatial regions automatically reduced the FID score significantly. The authors should at least consider this limitation and think of better comparison, e.g. maybe a region-based FID score.
- In the paper the authors state that the method helps gaining more clear visual explanations where even a user can interact with the method to optimize the results. This is true with respect to how the method works, but no study with real users was performed to evaluate if such statement holds true. Either this claim should be removed or validated by a (small) user study.
- The other SOTA methods, e.g. ACE and DVCE are not really introduced, nor cited and not compared from a theoretical point of view. At least they must be cited and introduced.

Meta-review: The paper initially received mixed reviews, 656653. There were a number of major issues raised:

1. what is the effect if the region is not semantically relevant? [nGZZ]
2. What is the purpose of the three settings A, B, and C [nGZZ]
3. Why do LangSAM-based regions have higher FID, and why is it so difficult for the proposed method? [nGZZ]
4. how is confirmation bias mitigated by the proposed method? [nGZZ, p98R]
5. missing discussion about prior works [2FX9]
6. needs more evaluation (both datasets and baselines) [2FX9]
7. overclaims need to be tempered [2FX9]
8. there is a risk when constraining the explanation to semantically meaningful variations that the explanation will not show the actual decision process [2FX9]
9. does using attribution maps or grouping methods bias the understanding of the classifier? [PMZS]
10. the explanation ability entangles the ability of the classifier and that of the pinpointing model, which is problematic for understanding the classifier [PMZS]
11. difference between attribution map generation (IG) and visual counterfactuals is unclear. [PMZS]
12. what happens when we force the label to a different class? [PMZS]
13. can the method discover cases where unusual visual cues are used, e.g., the background context is use due to dataset bias? [PMZS]
14. How to interpret some examples in Fig 5 where the model's prediction is reasonable and expected? [PMZS]
15. Limited technical novelty [p98R]
16. Cannot capture changes in shape due to inpainting in a specific region. Missing results on less favorable test cases with shape changes [Rswz]
17. unclear whether the proposed method is better, since the best counterfactual image would flip the decision with minimal visible change (which is not quite useful) [Rswz]
18. can it help explain adversarial attacks? [Rswz]
19. FID score is misleading because the proposed method only changes a region [TMN9]
20. missing user study [TMN9]
21. need more use cases of the method to show its usefulness [TMN9]

The authors wrote a response. During the discussion period, 2FX9, nGZZ, p98R were satisfied with the response. TMN9 was also positive, but noted that Point 19 was not properly addressed: "Still, the paper should name that limitation of the metric at least and could have thought of a region-based metric or similar."

PMZS was mostly satisfied but noted there was still a concern about the new user study, which 2FX9 also noted:

- PMZS: "However, I still hold one concern regarding to the root of the method. From the user study, we can conclude that the method can help person to find the most influence region aligned with human's perception. It is doubtful if the method can explain the wrong behavior of the model especially when the model's prediction is influence by some factors similar to adversarial attach. But this might be a problem beyond this manuscript."
- 2Xf9: "I do have some concerns about the interpretation of the newly added user study. .... the constraining of explanations to semantically meaningful variations may induce trust on the user's side that is unwarranted. This is actually reflected well in what the authors write about their user study: "The answers generally focused on the semantic change being more localized, easier to interpret and better aligned with human intuition". The fact that an explanation is 'aligned with human intuition' should be completely irrelevant for evaluating the utility of an explanation method: if the model does something very weird, I want to see that, and not an explanation that looks intuitively pleasing but does not reflect the model's actual decision process well. As far as I understand it, there was no evaluation in this study of whether the users actually understood the model's decision process better, or whether they just felt they did? As it stands then, the user study seems to show that users like these explanations, but not whether the explanations are any good in the sense that they really help them understand the model (better)."
 
Regarding this concern, the AC noted that it is mainly about the user study where regions are defined by users. For the fully-automated version, then the region limitation inherits the limitations of the attribution method employed.

Finally, Rswz raised concerns about the limitation of the method to explain shape changes or adversarial attacks. The AC thought that the authors responsed well to these questions (Rswz did not participate)

Overall, the paper received positive support with updated scores 866655 with reviewers noting the potential usefulness and good performance, despite its limitations. These limitations are interesting avenues for future work.  The AC agreed and recommends accept. The paper should be further edited to address the discussion points. In particular a summary of the above  limitations should be included in the main paper (not the supplemental), for full disclosure.

Decision: Accept (Poster)