LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
Authors: Zhuo Cao
Xuan Zhao
Lena Krieger
Hanno Scharr
Ira Assent
Venue: NeurIPS 2025 poster ()
Keywords: XAI; Counterfactual Explanation; AI4Science; Flow Matching
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: LeapFactual is a model-agnostic algorithm that generates reliable, actionable counterfactual explanations using conditional flow matching, overcoming key limitations of existing methods in high-stakes and human-in-the-loop domains.

Abstract: The growing integration of machine learning (ML) and artificial intelligence (AI) models into high-stakes domains such as healthcare and scientific research calls for models that are not only accurate but also interpretable. Among the existing explainable methods, counterfactual explanations offer interpretability by identifying minimal changes to inputs that would alter a model’s prediction, thus providing deeper insights. However, current counterfactual generation methods suffer from critical limitations, including gradient vanishing, discontinuous latent spaces, and an overreliance on the alignment between learned and true decision boundaries. 
To overcome these limitations, we propose LeapFactual, a novel counterfactual explanation algorithm based on conditional flow matching. LeapFactual generates reliable and informative counterfactuals, even when true and learned decision boundaries diverge. LeapFactual is not limited to models with differentiable loss functions. It can even handle human-in-the-loop systems, expanding the scope of counterfactual explanations to domains that require the participation of human annotators, such as citizen science. We provide extensive experiments on benchmark and real-world datasets highlighting that LeapFactual generates accurate and in-distribution counterfactual explanations that offer actionable insights. We observe, for instance, that our reliable counterfactual samples with labels aligning to ground truth can be beneficially used as new training data to enhance the model. The proposed method is diversely applicable and enhances scientific knowledge discovery as well as non-expert interpretability.

LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching
Zhuo Cao
Xuan Zhao Lena Krieger 1.2- Hanno Scharr'
Ira Assent .3
IAS-8, Forschungszentrum Jülich, Germany
Munich Center for Machine Learning (MCML), LMU Munich, Germany 3 Aarhus University, Denmark {z. cao, xu. zhao, 1.krieger, h. h.scharr, i afssent)fz--juelll de
Abstract
The growing integration of machine learning (ML) and artificial intelligence (AI) models into high-stakes domains such as healtheare and scientific research calls for models that are not only accurate but also interpretable. Among the existing ex- plainable methods, counterfactual explanations offer interpretability by identifying minimal changes to inputs that would alter model's prediction, thus providing deeper insights. However, current counterfactual generation methods suffer from critical limitations, including gradient vanishing, discontinuous latent spaces, and an overreliance on the alignment between learned and true decision boundaries. To overcome these limitations, we propose LEAPFACTUAL, novel counterfactual explanation algorithm based on conditional flow matching. LEAPFACTUAL gener- ates reliable and informative counterfactuals, even when true and learned decision boundaries diverge. Following a model-agnostic approach, LEAPFACTUAL is not limited to models with differentiable loss functions. It can even handle human-in- the-loop systems, expanding the scope of counterfactual explanations to domains that require the participation of human annotators, such citizen science. We pro- vide extensive experiments on benchmark and real-world datasets highlighting that LEAPFACTUAL generates accurate and in-distribution counterfactual explanations that offer actionable insights. We observe, for instance, that our reliable counterfac- tual samples with labels aligning to ground truth can be beneficially used new training data to enhance the model. The proposed method is diversely applicable and enhances scientific knowledge discovery as well as non-expert interpretability. The code is available on https: //github. com/ /tt.:caarrr/LLspacttuaalllllllll
Introduction
The widespread adoption of machine learning (ML) and Artificial Intelligence (AI) models in high-stakes domains such as healthcare [T-7]] or scientific research [+-10] demands not only high performance but also transparency, reliability, and interpretability. In the context of scientific discovery, where reproducibility, verifiability, and deep understanding of underlying mechanisms paramount, opaque "black-box" models can hinder progress and erode trust in computational findings. Without interpretability, the outputs of ML models remain inscrutable, limiting their utility for hypothesis generation, experimental validation, and the advancement of scientific knowledge. In response, a variety of explainability methods have emerged to illuminate the inner workings of black-box models. Among the most common are gradient-bases methods [T] and verturbation-basee techniques [3] These methods generate map of the regions that contribute most to the decision by either taking the backpropagation through the neural network or perturbing the input data. While
These authors contributed equally.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

these methods offer insights on the location of the class-related features, they do not expose residual features (e.g., pattern in the location) or guide practitioners toward meaningful model refinements.
Counterfactual explanation (CH) has recently gained attention as a complementary. more informative alternative. By answering 'What would have needed be different for the outcome to be different"' counterfactuals expose decision boundaries more clearly and can also be used to generate new, informative training samples that improve generalization and fairness [3] 7 Despite their promise, existing counterfactual generation algorithms face significant limitations. Many struggle with gradient vanishing [4 or discontinuous latent spaces [9 [24] leading uninformative or unrealistic counterfactuals (see Sections 22.1 and 3.1.. Moreover, CEs are typically located near the decision boundary. While this aligns with the definition of a counterfactual, it often fails lie within the data distribution, thereby failing to accurately represent it (see Figure T.. Our algorithm generates counterfactual samples that remain within the distribution, a characteristic we refer to as reliability.
Addressing the aforementioned limitations is essential to realize the full potential of CEs for inter- pretability and model enhancement.
To address these challenges, our main contributions are:
We introduce LEAPFACTUAL, novel, reliable CE algorithm based on conditional flow matching. retains the strengths of existing approaches while overcoming several critical shortcomings and handles scenarios where the true decision boundary deviates from the learned model.
We propose the CE-CFM training objective, and provide theoretical motivation showing that flow matching naturally disentangles the class-related information from residual information. This clarifies how important features isolated for counterfactual generation.
We validate the effectiveness of LEAPFACTUAL diverse benchmark datasets from different do- mains, including astrophysics, to demonstrate its upplicability and performance across disciplines.
We generate counterfactuals using non-differentiabbl models, such as human annotators, to illustrate that LEAPFACTUAL is model-agnostin which significantly expands the applicability of CE to fields such as citizen science.
2 Preliminary and Related Work
We denote random variables with capital letters (e.g., X, Y. Z) and sets with script letters (c.g., X. V. Z). The realizations of variables are shown with lowercase letters (e.g. x. y, 2). The probability den- sity function of a distribution IP()) represented by p(z) or q(z). The expectation with respect to p(x) denoted as Eptt) [-]- We use lower- case letters, such as f. g. and I, to represent functions. The function of a neural network with parame- ters 0 is represented as fa. Loss functions are denoted as L.
B
8
2
6
X
8
B
Figure 1: (Left) Conceptual illustration of ID data embedded in 2D space. Blue and red circles denote data of MNIST classes (8 and 6, resp.), with the dotted line representing the data manifold. The true decision boundary is x, while the solid line is the learned boundary. The background color indicates the output of the binary classifier. Adversarial attack V. CGM- generated CE A, and the starting point (Right) Conceptual multi-class 3D exam- ple. Each color represents MNIST digit class. A sample from the blue cluster is transported along the olive path reach the red cluster, forming a CE. The purple line shows the large distance between CE and the source, highlighting gradient van- ishing issues. More details can be found in Section 3.1
2.1 Counterfactual Explanation (CE)
CE A examples originate from
explanations (CEs) 25 26 modify an input data point in semantically meaningful way to produce a similar counterpart with a different target label, offer- ing deeper insight into model decisions. important features, but also provide actionable changes 3 77) making them especially useful in
Counterfactual
Unlike other explainability methods, CEs not only emphasize

---

domains requiring transparency. Recently, generative models are the standard paradigm to produce in-distribution counterfactuals and avoid adversarial artifacts (see Section There are two paradigms depending on the usage of generative models: Optimization (Opt)- bhased methods and conditional generative model (CGM)-based methods (see Figure?).
Formally, given classifier fo R2 + [0. 1]" and an input x ~ P(X), the general objective for an Opt-based method [1 is find counterfactual 2CE by minimizing: LCE(#CE) CcIs(fo(res) Me) + AC (z, ,CE), where LCLS encourages the classifier fo to predict the target label He and L enforces similarity to z, with À balancing the two terms. Note that overemphasis on similarity can yield adversarial examples ((27/291) since the data ambient dimension is usually higher than the intrinsic dimension [30, [1] (see also Figure DT) A common solution to prepend
generative model 96 (c.g., VAE [32.. GAN 33 to fo to ensure the modification is in in-distribution. This reformulates the loss to:
Lou(ZcE) = CeLs(folo 9e) + (D (o).))))))
where z is the latent vector corresponding to the input from a VAE encoder or GAN inversion.
In contrast, CGM-based methods [19 24 integrate the information encoded in the classifier directly into the generative model, by training conditioned on the output of the classifier.
Among various data modalities, generating CEs for visual data remains particularly challenging due to the high input dimension, Note, while our method focuses on visual data, can be generalized to other data types. In the following, we summarize related work on visual counterfactual explanations regarding Opt-based and CGM-based methods.
Optimization (Opt)-hased Methods Opt-based methods prepend a gener- ative model before the classifier be studied, and optimize the latent vec- tor corresponding to the input image towards the target label to generate CEs. REVISE [14] is gradient-baad based method that samples from a genera- tive model's latent space to find min- imal changes altering predictions. In- stead of directly optimizing latent vec-
Classifier
Fothened representation
Structured representation
Optimizztion
Flow
Opt-based
CCif-basedd
LoopPactual
Figure Overview of architectures: Opt-based models, CGM-based models and LEAPFACTUAL. Trapezoids repre- sent generative models. Opt-based and CGM-based methods employ only flattened or structured representation (see Sec- tion 3.1
while LEAPFACTUAL combines both.
tors, Goetschalckx et al. [15] learn latent directions by differentiating through both generator and classifier. Other works consider training linear Support Vector Machines latent space to control facial attributes [6 or using GAN for image editing, guided by gradients to generate target-class images with minimal changes [17.. Dombrowski et al. 18) propose theoretically grounded approach optimizing in the latent space of Normalizing Flows.
Conditional Generative Model (CGM)-based Methods CGGM-based methods incorporate the output of the classi- fier as condition of the generative model, by replacing the condition counterfactual samples are generated. Saman- gouei et al. [19 jointly train classifier-specific encoders with a GAN to produce reconstructions, counterfactuals, and modification masks. To ensure realisitic counterfactu- als, Singla et al. (20 use a GAN conditioned on classifier predictions with an encoder. DCEVAE [22] generates counterfactualls conditioned on input and target label. Other directions include training StyleGAN
Table 1:
Key characteristics of proposed method and baseline methods.
Opt
CGM
Ours
Continuous latent space No gradient vanishing
/
x
x
/
Pre-trained generator
/
x
Model-agnostic Reliable
x
X
x
with classificr-specifie style spaces for counterfactual manipulation [2]] employing Normalizing
Flows with a Gaussian mixture latent space and an information bottleneck to disentangle class information 244) and generating residuals added to the input to flip classifier decisions 21].
Unfortunately, both paradigms face significant drawbacks: Opt-based approaches enable continuous latent space exploration but face gradient vanishing and require differentiable classifiers; CGM-based methods mitigate vanishing gradients and offer structured representations but lack continuous latent spaces and often need separate generative models per classifier. The characteristics are summarized
3

---

in Table II The proposed method LEAPFACTUAL overcomes these issues and considers the mismatch between the learned and true decision boundaries to generate reliable CEs (see Section 3.1
2.2 Flow Matching
The development of flow-based generative models dates back to Rezende et al. 2015 [4 which introduced Normalizing Flows an invertible and efficiently differentiable mapping between fixed distribution and the data distribution. Initially, these mappings were constructed as static compositions of invertible modules. Later, continuous normalizing flows (CNFs) were introduced, leveraging neural ordinary differential equations (ODEs) to model these transformations dynamically [35 However, CNFs face significant challenges in training and scalability, particularly when applied to large datasets [75. More recently, many works [3 42 demonstrated that CNFs could be
trained using an alternative approach: regressing ODE's s drift function, similar to how diffusion models trained. This method, known flow matching (FM), has been shown to improve sample quality and stabilize CNF training [38]. Initially, FM assumed a Gaussian source distribution, but subsequent generalizations have extended its applicability to more complex manifolds [S)]. arbitrary source distributions [44]. and couplings between source and target samples derived from input data inferred via optimal transport. In this work, build upon I-CFM, the theoretical framework established by Tong et al.
Formally, flow matching source distribution Po to a target P1. The dynamics
38
learns time-dependent vector field (z)2) whose flow pushes
d= (z)
induce probability path Pt with Pt=o Pn and pt=1 = P1- In conditional flow matching (CFM), Pe(2) is represented as a mixture over conditional paths pe (2 h) with == (Za, 21) [42]. Under independent coupling, (h)) = ((=0)g(21) the conditional path and target vector field are
(=(= h) A(= (1=(111 =) + 121, T²T),
",(= h) 21 20:
which defines I-CFM via a Gaussian flow with fixed variance I²I.
In practice, neural network (t, 2) is trained to regress the target field ue(z h) specified in Equation (2). Coneretely, we sample
~ Unif[o, 1], (z0. ~ ((())((())((3))) 2 p-<pec-l
and minimize regression loss that uy (t, (t, 2) v ue(z ht.. In I-CFM the target simplifies to the constant vector ž1 20, independent of and 2; the conditioning h and the sampling of z ensure the model learns the correct field along the path. At inference. integrating the learned field produces the flow map that transports samples from Po to pr. Additional training details are provided in Section 3.2.1
3 LEAPFACTUAL: A Counterfactual Explanation Algorithm
We motivate our approach by analysing the latent space of generative models in Section 3.1 We then demonstrate the flexibility of CFM in Section 3.2 and introduce our new algorithm LEAPFACTUAL.
3.1 Analysis of Latent Spaces
It is important to distinguish the latent representations used in unconditional generative models, employed by Opt-based methods, and conditional generative models, used CGM-based methods. Let us assume latent vector z that is defined to be the result of mapping function m, that maps class-related (C) and residual features (R) to the latent space: 2 m(c,r). For instance, in the MNIST dataset, C denotes the digit identity, while R captures the handwriting style. Ideally, CE modifies only C, resulting in žCr = mi(CCF, r). In unconditional generative models, latent space is flattened (Figure 3 left), meaning both C' and R are entangled in Z. As result, modifying inevitably affects both components, making it difficult to isolate class-related changes. Additionally, such models trained to densely populate the latent space to cover the full data distribution

---

(Figure 1 right), which increases optimization complexity. Conversely, CGMs employ structured representations ( middle), where class-related features C' are provided as externa conditions. and the latent vector Z encodes only the residual information R. In this case, = ECE, enabling CF generation by simply altering c while keeping r fixed. However, this structure leads to discontinuous latent space, i.e., latent spaces of different classes are separated. This makes interpolation between samples non-trivial.
These limitations are illustrated in Figure I On the left, we depict data points on 1D intrin- sic manifold embedded in 2D space. Gener- ative models constrain CE samples to remain in-distribution, thereby avoiding adversarial ar- tifacts V. Opt-based methods, CEs & are ob- tained by optimizing the objective in Section [21 Typically, the optimization stops after crossing the learned decision boundary, without reaching the true decision boundary x. This results in uninformative and potentially misleading CEs, e.g.. in the MNIST case, a CE classified as '6' may fail to exhibit key features of a genuine '6' More generally speaking, CEs close to the learned boundary are often not typical examples of their class. Further, samples between the learned and true decision boundaries are critical for improving model performance because they provide insights into the class features, which are unexplored under the Opt-based paradigm. This problem becomes more severe in multi-class scenarios (Figure [ right), where the latent space is fully occupied and CEs may be far from the original input and pass through regions corresponding other classes, making the gradient vanishing problem more significant. As result, most Opt-based approaches are either limited to binary classification or require decomposition into binary subproblems. CGM-based methods alleviate some of these issues through structured and disentangled representations. However, they do not guarantee that the generated CE A is close to the actual decision boundary. Moreover, their discontinuous latent spaces make reversing from the CE sample towards the decision boundary infeasible.
Figure 3: (Left) Flattened latent representation, (Middle) structured latent representation, and (Right) proposed latent representation, where X, Z, C, and R represent input, the corresponding latent variable, class-related information, and resid- ual information. The distributions connected by flow matching are Zu and Z1.
In addition, both Opt-based and (GM-based methods have practical limitations. Optimization-basee approaches require a differentiable loss and are vulnerable to gradient decay in the generative model or classifier. CGM-based methods, on the other hand, must retrain the generative model for each new classifier - a costly and complex process - and demand careful architectural design 45 to ensure that conditioning information is preserved during training.
LEAPFACTUAL combines the advantages of both paradigms while mitigating the drawbacks. To achieve this, propose mapping between flattened and structured latent spaces by introducing a new latent dimension (Figures []] to right). Progressing along this axis removes class-related information, while reversing the direction re-injects it. Below, we present the algorithm in detail.
3.2 LeapFactual
In Section 3.2.1 we introduce the CE-CFN training objective and its theoretical justification for bridging the flattened and structured representations. Then, in Section 3.2.2 we show how a flow matching model trained with the CE-CFM objective can generate high-quality and reliable CEs.
3.2.1 Training Phase
We aim to leverage flow matching to bridge the gap between flattened and structured latent represen- tations, as illustrated in the left and middle columns of Figures and However, this integration is not straightforward. As shown in the right column of the figures, our approach introduces a flow between Zu (structured encoding) and Z1 (flattened encoding). mediated by their shared parent R. This design violates the independent coupling assumption inherent to the I-CFM framework, necessitating a modification of the original formulation. Given an input image X, a classifier fo extracts class-relevant information C, from which the residual R can be subtracted. In MNIST digit classification, C captures the digit identity, while corresponds the writing style. Once R inferred from samples drawn from the data distribution X and their corresponding class predictions
5

---

C, the latent variables Zu and Z1 become d-separated 46]; i.e., they are conditionally independent given their only common parent R.
To address this, we redefine the conditioning term in I-CFM to h == (20, =1.)). Hereby, 21, ~ q(21 c) denotes the latent vectors specific the class c (x) being predicted by the model, 20 q( 22) is randomly sampled from a Gaussian distribution.
The joint distribution can be expressed q(h) ((z))q(Z1 c) (derivation in Appendix A.I.). To construct the probability path pz(z h) and the corresponding vector field uz h) as outlined in
Equation 2 we sample from ((2)) and q(21 c). The samples are then inserted into the equation, thereby defining a transition from a Gaussian prior q(20) to the class saass-se distribution q(2) | c).
To generalize across classes, we condition the network the predicted label fx)),, leading to our Counterfactual Explanation - Conditional Flow Matching (CE-CFM) objective:
CCE-CEM ()) == (1) A) I z. c) uz(2 (21 hhll?
where 0g is a neural network parameterized by v. Unlike the original I-CFM formulation 42) the CE-CFM objective explicitly incorporates the classifier output e into ug. Architecturally, this allows single network to represent multiple class-specifie flows From an inffommatton-thhorreee perspective, the Gaussian nature of Zo ~ ((z)) acts as an information bottleneck, compressing the content of Z1. By including class information C as condition, the network can more efficiently discard related features during compression:
Theorem 1.
Let Zi denote the original representation and Zo its counterpart obtained via flow matching conditioned on class information C. Then, Zo is compressed repre- sentation of Zi and the optimized information loss incurred through this transformation corresponds exactly to the class-related information C that is provided as a condition.
Proof. in Appendix A.2
Once the model is trained, Zo and Z1 are linked through invertible transformations: a backward integral 1 (t, 2, e) dt and a forward integral f-=0 By 2, c) dt. The backward pass effectively removes class-relevant information (compression), while the forward pass reintroduces it (reconstruc- tion). For conceptual clarity, we refer to these two operations as the lifting and landing transports.
3.2.2 Explaining Phase
In principle, any generative model can be used to realize the proposed mapping- for example, a VAE encoder/decoder pair, GANs with inversion, or normalizing flows, which are inherently invertible. However, with flow matching, we achieve significantly greater flexibility, enabling not only information replacement but also more nuanced operations such as blending and injection.
We visualize the transport processes in Figure 4 Here, the latent space position is defined as z = Cp + Tps where eo E (=00255²² denotes the four center positions, and rp E (-0.25, 0.25)² denotes the relative position. The relative position is constrained so that lies within the squares around the centers.
Information Replacement It can be observed from Figure (a) that multiple points from different classes in Zj are mapped to the same black point in Zo. indicating that Zo is a compressed represen- tation of Z1, which complies Theorem I It is also evident in Figure (b), where we fix the relative position Tp while varying the center positions Cp. yielding the source points Z1,blue O, yellol O and
Z1.green O. During the lifting transport, center information is discarded, resulting in similar transported points: 20,blue R yelyllow R 20.green- Consequently, the center information of the source points is fully replaced by the target label Gred during the landing transport. and all points
map to locations around 21.red O. This process produces a global counterfactual [47]. where counterfactuals from different classes converge to the same result when their intra-class (residual) information is the same and targeting the same class, failing to explore inter-class counterfactuals.
Information Blending For effective model explanations, local counterfactuals must preserve both intra- and inter-class relationships. Instead of fully replacing class-related features, meaningful counterfactuals blend source (with subscript s) and target (with subscript i) features. As shown in Figure 4 (c), blended points .oe interpolate between source and target: z ac, + (1 a)c,

---

(a)
(b)
(e)
0.0
0.0
0.5
0.5
0.2
-0.
0.5
0.5
0.0
(c)
(d)
0.0
0.0
-0.2
1.0
1,0
0.5
0.5
-0.4
-0.5
0.0y
-0.5
0.0,
0.0
0.0
0.5-0.5
0.5-0.5
-0.4
-0.2
0.0
0.2
0.4
Figure 4: Toy experiments illustrating lifting and landing transports. The vertical axis represents t. Hollow circles denote source points; filled circles or triangles indicate transported points. (a) Blue source points are lifted to / O, removing center position information, which is replaced by the red class during landing. (b) Center information from green, blue, and yellow sources overwritten by the red class. (c) Transported points blend center information from source (green, blue, red) and target (red); dashed lines show interpolation. (d) Target information is injected even when source and target are both red; dashed lines show interpolation. (e) At 1: filled circles mark blended points, triangles mark injected ones; dashed and dotted lines denote blending and injection interpolations.
with a < 1. This achieved by adjusting the transport step size y in ʃ yuu dt: y = yields full replacement, while fo can track the class label at each step. This is useful in multi-class tasks where class identity may change mid-transport. For instance, in Figure (e), the blue point transitions through the yellow region before reaching the target, deviating from the dashed interpolation path.
enables blending. Blending can be applied progressively. and the classifier
Information Injection Stopping transport when the classifier prediction matches the target label may be insufficient, as the learned boundary may not reflect the true one (Section 3.1) To refine the result, target class information must be explicitly injected. Conditional lifting removes source class information, which cancels the effect of injection when source and target classes coincide (see red points OO in Figure (b), (c)). To prevent cancellation, we use a smaller step size for the lifting than for the landing transport. As shown in Figure 4 (d), where "Ma O, "Maad 1, the target red point follows: 2 = (1 aa + (1 a)r, amplifying target class information when C. = Ce = Cred- The relative position is also adjusted to maintain in-distribution samples. As shown in Figure (e), the progressive injection after blending maximizes target class information.
Algorithm LEAP
Input:
Flow matching model ve trained with CE- CE-CFM objective, source point Zsource (at t=1), current label ye. target label őe. step sizes Miin Mand
Step 1:
Lift
From Zi to 7
2d (t) Zsource + 1 munul(T 2 yc) dr, Le [0,
<- 2 (0)
Step Land
From Zo Zi
9c (t) = Zun + fo Nmaaull, (80(7), d6) dr, te [0
Fland t- 20 (1)
Output: Transported point Sland
define the combination lifting transport and landing transport as a LEAP (see Algorithm[T) TD. Specifically, a Blending Leap uses step sizes mh "b.in = -hland < 1, while an Injection Leap sets Finn < Naaan Based on these, we introduce our Algorithm LEAPFACTUAL. which combines No blending leaps and N; injection leaps. Generally, small step size is preferred
Algorithm
7

---

Algorithm 2 LEAPPACTUAL
Input:
Source point x, target label ye. classifier fo. generative model 9g. flow matching model by trained with
CE-CFM objective
Hyperparameters: Number of blending leaps N,, blending step size 7b, number of injection leaps N; injection
step sizes "N. in "idand
Step
Preprocessing
z + 96 (x)
B Acquiring & via VAE encoder or GAN inversion Deetermining the current label Generating
Uc fe(g
Step Information Blending
for = 0 to N₁- do
4- LEAP((r:: 2, Mes De: 70, 76) y/: Lo [ga(2))
Blending the source and target classes information
end for
Step 3: (Optional) Information Injection
> Generating Reliable CE
for = to N₁ 11-- do
+ LEAP(UE: 2, Ves De: Wilift Thllnd) V fo (gs(
Injecting the target class information
end for Step 4: Postprocessing
TCE (z)
Output: Transported point ree
for higher precision, although this may require larger number of leaps and consequently higher inference time. The number of steps depends on the dataset and specific target classes. However, when blending only, the algorithm will automatically stop as soon as the target class is reached due to information replacement. Generally, starting with small step size and large number of steps is suggested.
Experiments
In the following sections, we demonstrate the performance of LEAPFACTUAL across diverse datasets and experiment setups. We differentiate between LEAPFACTUAL and LEAPFACTUAL_ R, the latter one additionally includes information injection. We refer to CEs generated with LEAPFACTUAL R as reliable CEs. In Section 4.1 we compare our method with kate-of-theeart methods and demonstrate the interpretability of our CEs using the Morpho-MNIST dataset [48). In Section 4.2
we show
that reliable CEs enhance classification performance using the Galaxy 10 DECaLS dataset 49 50].
Finally, in Section [4.) we use the FFHQ[S1] dataset and StyleGAN3[52] to illustrate scalability and applicability to nonfdifferentiable methods. We evaluate performance based on correctness, Area Under the ROC Curve (AUC) and Accuracy (ACC) using the desired target label as ground truth, and similarity, measures depend on the individual experiments. More experimental details in Appendix
4.1 Quantitative Assessment
In this section, we compare the performance of our method against two competitors: the Opt-based and CGM-based methods. Morpho-MNIST (48) provides a modified version of MNIST specifically designed to benchmark representation learning. We evaluate similarity as the Absolute Relative Error between morphological properties of counterfactual and reconstructed samples (see Appendix B.2.1 To ensure a fair comparison, we use the same VAE model for both the Opt-based and proposed methods, and apply only minimal modifications to the CGM-based method (refer to Appendix B). We use 4-yeer MLP as the flow matching model and explain a simple MLP classifier, with test accuracy 96. 96.58%. Implementation details are provided in Appendix
As shown in Table the Opt-based method yields the poorest performance across both correctness and similarity metries. This is primarily due gradient vanishing, which causes counterfactual trajectories to halt before reaching the target decision region. This issue is clearly visible in Figure 5 where many counterfactuals fail resemble typical samples from the target class. The CGM-based method performs significantly better in terms of correctness but introduces greater distortion compared the proposed approach. These observations are consistent with the discussion Section 3.1

---

Table Results on Morpho-MNIST
MNNIST for runs across different methods (columns) and evaluation metrics (rows). First rows report Accuracy (ACC) and Area Under the ROC Curve (AUC), while the remaining rows show mean and standard error of Absolute Relative Error (D) morphological properties. The st,, 2nd, and 3rd best performances are indicated by bold, underline, and italic.
Metric
Opt-based
CGM-based
LeapFactual
LeapFactual__
ACC
0.8277=0.0071
0.9422=0.00k
0.9868+0.0022
0.990620.0016
AUC
0.881440.000
0.9979#0.000 0002
0,9990+0.0000
0.999940.00m
D(Area)
0.2475=0.004f
0.2556=0.0083
0.1665=90.00%
0.2295+0.0058
D(Length)
0.2472+0.0033
0.2999=0.001
0.2128:0.000
0,2732=0,002 0029
D(Slant)
4.3667=0.2430
3.4613=0. 1783
2.5026
3.3144+0 03352
D(Thickness)
0.1724+000033
0.0859+0.000
0.0807
0.0895 0020
D(Width)
0.2912=0.002
0.3069#0.000
0.2061=0.003
0.2784+0.0033
D(Height)
0.0620+0.000
0.0288#0. 0003
0.0265=0.0007
0.0303+0.0007
Target
0
1
3
7
8
9
In comparison, LEAPFACTUAL demon- strates superior performance in both cor- rectness and similarity. It generates rectly classified counterfactual samples while minimally modifying the origi- nal inputs. The information injection LEAPFACTUAL R further improves correctness without compromising sim- ilarity. As shown in Figure [ our methods produce realistic counterfac- tual samples that modify only the most important class-related features, preserv- ing digit style. This selective modifica- tion is particularly beneficial for distin- guishing similar classes (e.g.. vs. or 5 vs. 8), thereby enhancing interpretabil- ity. More qualitative comparisons in Ap- pendix B
Input
7
2
0
4
9
S
9
Reconstruction
7
2
0
4
4
9
9 9 9 9
Opt-based
7
2
0
4
4
4
2
R
8 8 8
CGM based
0
1
2
3
4
5
5
7
LespFactual
0
L
9
4
5
4
9
LeapFactual
0
2
3
4
5
6
8
9
Figure 5: Qualitative comparison of counterfactual samples across different methods (rows) and target labels (columns). The first two rows are randomly sampled target labels and inputs, followed by the reconstructed images.
4.2 Model Improvement
We show the advantages of reliable CEs in model training on the Galaxy 10 DECaLS dataset [49] 550) a 10-class galaxy morphology classification task. In the experiment, we fix Nb 250, 0.1,
Merging
Cigar
Min 1, and Ialdd 1.025 based on ablation results in Appendix B
We train a weak classifier on 20% of the dataset and use second VGG model architecture trained on 100% as baseline. We generate standard and liable CEs (depicted in Figure (6) regarding all classes other than the original for each image in the training subset, resulting in two auxiliary datasets assuming CE target labels as ground truth. We then blend varying fractions of these auxiliary datasets with the original training subset, used to train the weak classifier (20%), to evaluate their impact on the models classification performance.
Figure 6: Top to bottom: Input images, CEs, and reliable CEs. (Left) Merging to Round galaxy, (Right) Round to Cigar galaxy.
Table 3: Model performance across runs with different fractions (second row) of standard and reliable CEs added to training set. Baselines trained with 20% and 100% of training set.
0.0000
0.0016
0
(1970 0.0013
0.0062
(19738
0.9776
0.9748
0.9793 0.005

---

When standard CEs are blended into the training data, model performance declines as the proportion of CEs increases, shown T contrast, using reliable CEs leads improvements in both accuracy and AUC with increasing fraction. This is because the standard CEs are on the learned decision boundary of the weak classifier, while reliable CEs align closely with the true decision boundary (see Appendix [ Thus, they can serve as augmentation strategy for small datasets or imbalanced classes, thereby addressing fairness and reducing the underrepresentation of minorities. Further, reliable CEs can improve model validation by keeping class-unrelated features similar to the original input, improving expressiveness and helping to identify shortcut learning [53 We will further investigate these promising results in future work.
4.3 Generalization
In this experiment, we demonstrate that the proposed method is highly scalable and applicable to non- differentiable classifiers, such as human annotators. To simulate this scenario, we employ a pretrained CLIP model [55] as a proxy for human judgment. The model is used classify images either Smiling Face Face.
Table 4: Quantitative results for FFHQ using 1,024 samples. Last row reports results for randomly paired images.
No
ACC
AUC1
SSIM
PSNR1
LPIPS]
5
0.706
0.706
0.564
18.036
0.149
The images are generated by a StyleGAN3 model [52] pretrained the FFHQ dataset [51] (1024? pixels). In this setting, existing Opt- based and CGM- based methods infeasible due to their reliance differentiability retraining.
10
0.957
0.957
0.538
17.126
0.171
15
0.982
0.982
0.531
16.946
0.176
20
0.993
0.993
0.525
16.833
0.180
0.070
8.848
0.555
For the flow matching model, we train ID U-Net (56) for 120 epochs. A total of 20K images are randomly sampled from StyleGAN3 and projected into the w-space [5I], which serves as input. The corresponding predicted labels from the CLIP model are used as conditional inputs.
In Figure 7 rows and 2 illustrate image transformations from Face to Smiling Face, and vice versa, using our method. Note, even subtle differ- ences in facial features lead changes in classification outcomes. Compar- ing rows and 3 in Figure 7 we ob- serve that the target feature -facial expression- is further accentuated, providing a clear visual explanation. Table presents accuracy and similar- ity metrics for various values of No.
Figure 7: Top to bottom: randomly sampled input images, CEs, and reliable CEs. (Left) Face Smiling Face. (Right) Smiling Face to Face.
5 Conclusion
Our work addresses key limitations in existing counterfactual explanation methods by introducing LEAPFACTUAL, novel algorithm comprising the CE-CFM training objective and explanation procedure. Our analysis shows that LEAPFACTUAL offers a flexible mechanism for generating counterfactual explanations, capable of producing not only high-quality but also reliable results, even in the presence of discrepancies between true and learned decision boundaries. While our current focus is on visual data, the method is broadly applicable to other data modalities as well.
Limitations & Future Work A limitation arises from the dimensionality of the latent space. In high-dimensional settings. such as those involving diffusion models or normalizing flows, the flow matching model cost increases significantly, leading to reduced training efficiency. Future work includes improving LEAPFACTUAL by replacing I-CFM with more efficient approaches like OT- CFM [2]. However, this improvement is not trivial, since the application of OT-CFM requires modifications to the transport map on the classifiers' predictions. Furthermore, we see potential for future work exploring latent space entanglement and challenges associated with imbalanced datasets.
10

---

Acknowledgement This work was partially funded by project W2/W3-108 Initiative and Network- ing Fund of the Helmholtz Association. We are grateful Andrew Alexander Draganov for his constructive discussions and thorough proofreading, which substantially enhanced the clarity and presentation of this work.
References
[1] Silvan Mertes, Tobias Huber, Katharina Weitz, Alexander Heimerl, and Elisaheth André. GAN- terfactual- Counterfactual Explanations for Medical Non- Von-Experts using Generative Adversarial Learning. Frontiers in Artificial Intelligence, 5:825565, 2022.
[2] Jayaraman J Thiagarajan, Kowshik Thopalli, Deepta Rajan, and Pavan Turaga. Training calibration-based counterfactual explainers for deep learning models medical image analysis. Scientific Reports, 12(1))597, 2022.
[3] Mattia Prosperi, Yi Guo, Matt Sperrin, James S Koopman, Jae S Min, Xing He, Shannan Rich, Mo Wang, lain E Buchan, and Jiang Bian. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nature Machine Intelligence. 2(7):369-3755 2020.
[4] Stephen Pfohl, Tony Duan, Daisy Yi Ding, and Nigam Shah. Counterfactual reasoning for fair clinical risk prediction. In Machine Learning for Healthcare Conference, pages 325-358. PMLR, 2019.
[5] Hao Huang, Emetis Niazmand, and Maria-Esther Vidal. Hybrid AI Approach for Counterfactual Prediction over Knowledge Graphs for Personal Healthcare. Artificial Intelligence and Data Science for Healthcare: Bridging Data-Centric Al and People- Coople-Centric Healthcare, 2024. URL
https:///penxeviee..etee/ee weeerrrummddd-eee-eeeeee bexdbexxggggRgR
[6] Toygar Tanyel, Serkan Ayvaz, and Bilgin Keserci. Beyond known reality: Exploiting counter- factual explanations for medical research. arXiv preprint arXiv:2307.02131, 2023.
[7] Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counterfactual explanations to interventions. In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Transparency, pages 353-362, 2021.
[8] Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47-600 2023.
[9] Annalisa Bracco, Julien Brajard, Henk A Dijkstra, Pedram Hassanzadeh, Christian Lessig, and Claire Monteleoni. Machine learning for the physics of climate. Nature Reviews Physics, 7(1): 6-20, 2025.
[10] Zhuo Cao, Lena Krieger, Hanno Scharr, and Ira Assent. Galaxy Morphology Classification with Counterfactual Explanation. In Machine Learning and the Physical Sciences Workshop @ NeurIPS, 2024.
[11) Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual Explanations from Deep Networks via Gradient- based Localization. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 618-626, 2017.
[12] Scott M Lundberg and Su-In Lec. A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30, 2017.
[13] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1 1135-1144, 2016
[14] Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems. arXiv preprint 1oXX::1907.09625. 2019.

---

[15] Lore Goetschalekx, Alex Andonian, Aude Oliva, and Phillip Isola. GANalyze: Toward Visual Definitions of Cognitive Image Properties. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 5744-5753, 2019.
[16] Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(4):2004-20) 20080 2020.
[17] Shusen Liu, Bhavya Kailkhura, Donald Loveland, and Yong Han. Generative Counterfactual Introspection for Explainable Deep Learning. In IEEE Global Conference on Signal and Information Processing (GlobalSIP), pages 1-5. IEEE, 2019.
[18] Ann-Kathrin Dombrowski, Jan Gerken, Klaus-Robert Müller, and Pan Kessel. Diffeomorphic Counterfactuals with Generative Models. IEEE Transactions Pattern Analysis and Machine Intelligence, 2023.
[19] Pouya Samangouei, Ardavan Saeedi, Liam Nakagawa, and Nathan Silberman. ExplainGAN: Model Explanation via Decision Boundary Crossing Transformations. In Proceedings of the European Conference on Computer Vision (ECCV), pages 666-681, 2018.
[20] Sumedha Singla, Brian Pollack, Junxiang Chen, and Kayhan Batmanghelich. Explanation by Progressive Exaggeration. arXiv preprint 1rXiv:1911.00488 2019.
[21] Daniel Nemirovsky, Nicolas Thicbaut, Ye Xu, and Abhishek Gupta. CounteRGAN: Gener- ating Realistic Counterfactuals with Residual Generative Adversarial Nets. arXiv preprini arXiv:2009.03 199, 2020.
[22] Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, and Il-Chul Moon. Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8128 8136, 2021.
[23] Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T Freeman, Phillip Isola, Amir Globerson, Michal Irani, et al. Explaining in style: train- ing a GAN to explain classifier in stylespace. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 693-702, 2021.
[24] Frederik Hvilshøj, Alexandros Iosifidis, and Ira Assent. ECINN: Efficient Counterfactuals from Invertible Neural Networks. arXiv preprint arXiv:2103. 13701, 2021.
[25] Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 607-617, 2020.
[26] Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harv. JL & Tech. 31:841, 2017.
[27] Christian Szegedy, Wojciech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 1312,6199 2013.
[28] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adver- sarial Examples. arXiv preprint arXiv:1412.6572. 2014.
[29] Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The Limitations of Deep Learning in Adversarial Settings. In 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pages 372-387. IEEE, 2016
[30] Piotr Tempczyk, Rafat Michaluk, Lukasz Garncarkk Przemyslaw Spurek, Jacek Tabor, and Adam Golinski. LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihood. In International Conference on Machine Learning, pages 21205-21233 PMLR, 2022.
[31] Jan Pawel Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. Diffusion Models Encode the Intrinsic Dimension of Data Manifolds. In International Conference on Machine Learning. PMLR, 2024.
12

---

[32] Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes, 2013.
[33] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Nets. Advances in Neural Information Processing Systems, 27, 2014.
[34] Danilo Rezende and Shakir Mohamed. Variational Inference with Normalizing Flows. In International Conference on Machine Learning, pages 1530-1538. PMLR, 2015.
[35] Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems, 31, 2018.
[36] Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models. arXiv preprint arXiv: 18100013,, 2018.
[37] Derek Onken, Samy Wu Fung. Xingjian Li, and Lars Ruthotto. OT-Flow: Continuous Normalizing Flows via Optimal Transport. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 9223-9223 223-9222, 2021.
Fast and Accurate
[38] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow Matching for Generative Modeling. arXiv preprini arXiv:2210.02747, 2022.
[39] Michael S Albergo and Eric Vanden- Eijnden. Building Normalizing Flows with Stochastic Interpolants. arXiv preprint (tt.::2209.155711. 2022.
[40] Qiang Liu. Rectified Flow: A Marginal Preserving Approach to Optimal Transport. arXiv preprint arXiv: 14577, 2022.
[41] Alexander Tong, Nikolay Malkin, Kilian Fatras, Lazar Atanackovic, Yanlei Zhang, Guillaume Huguet, Guy Wolf, and Yoshua Bengio. Simulation-Free Schrödinger Bridges via Score and Flow Matching. arXiv preprint 2307.03672, 2023.
[42] Alexander Tong, Kilian Fatras, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector- Brooks, Guy Wolf, and Yoshua Bengio. Improving and Generalizing Flow-based Generative Models with Minibatch Optimal Transport. Transactions on Machine Learning Research, pages 1-34, 2024.
[43] Ricky TQ Chen and Yaron Lipman. Flow Matching on General Geometries. arXiv preprint arXiv:2302. 03660, 2023.
[44] wram-dlexandre Pooladian, Heli Ben Ben-Hamu, Carles Domingo-Enrich Brandon Amos, Yaron Lipman, and Ricky TQ Chen. Multisample Flow Matching: Straightening Flows with Minibatch Couplings. In International Conference Machine Learning, pages 28100-28127. PMLR, 2023.
[45] Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, and Marc' Aurelio Ranzato. Fader Networks: Manipulating Images by Sliding Attributes. Advances in Neural Information Processing Systems, 30, 2017.
[46] Dan Geiger, Thomas Verma, and Judea Pearl. d-separation: From theorems to algorithms. In Machine Intelligence and Pattern Recognition, volume 10, pages 139-148. Elsevier, 1990.
[47] Arnaud Van Looveren and Janis Klaise. Interpretable Counterfactual Explanations Guided by Prototypes. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pages 650-665. Springer, 2021.
[48] Daniel C Castro, Jeremy Tan, Bernhard Kainz, Ender Konukoglu, and Ben Glocker. Morpho- MNIST: Quantitative Assessment and Diagnostics for Representation Learning. Journal of Machine Learning Research, 20(178):1-2) 1-29, 2019.
[49] Arjun Dey, David J Schlegel, Dustin Lang. Robert Blum, Kaylan Burleigh, Xiaohui Fan, Joseph R Findlay, Doug Finkbeiner, David Herrera, Stéphanie Juneau, et Overview of the DESI Legacy Imaging Surveys. The Astronomical Journal, 167(5):168, 2019.
13

---

[50] Kyle W Willett, Chris Lintott, Steven P Bamford, Karen L Masters, Brooke D Simmons, Kevin RV Casteels, Edward M Edmondson, Lucy Fortson, Sugata Kaviraj. William C Keel, et al. Galaxy 7oo 2: detailed morphological classifications for 304 122 galaxies from the Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Sociery, 435(4):2835-2866) 2013.
[51] Tero Karras, Samuli Laine, and Timo Aila. A Style-Based Generator Architecture for Generative Adversarial Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401-4410, 2019.
[52] Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-Free Generative Adversarial Networks. Advances in Neural Information Processing Systems, 34:852-863, 2021.
[53] Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojeiech Samek, and Klaus-Rober Müller. Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications, 10(1):1096 2019.
[54] Robert Geirhos, Morn-Herrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 1):665 673, 2020.
[55] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models From Natural Language Supervision. In International Conference on Machine Learning, pages 8748-8763. PMLR, 2021.
[56] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and Improving the Training Dynamics of Diffusion Models. In Proceedings of the IEEÉ/CVF Conference on Computer Vision and Pattern Recognition, pages 24174-24184 2024.
[57] Luke Ditria and Tom Drummond. Long-Term Prediction of Natural Video Sequences with Robust Video Predictors. arXiv preprint arXiv:2308. 11079, 2023.
[58] Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein Auto- Encoders. In International Conference on Learning Representations, 2018.
[59] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4): 600-612, 2004.
[60] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The Unrea- sonable Effectiveness of Deep Features Perceptual Metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction reflect the paper's contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitation of the work is discussed the last section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how reliable the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
The proof and assumptions are provided in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The code is available. The details of the experiments are provided in the appendix.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: All the datasets used in the experiments are publicly available. The codes that produce synthetic data and reproduce the experiments are provided in supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips.c//pppp
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The experimental setting and details are provided the main paper and the supplementary material.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide uncertainties for the first and second experiments. For the third experiment, we do not provide error bars due to the high computational cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The computer resources used for the experiments are stated in main paper and supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httts:://neurip::: httpa//rpp.ppcpp...../ccccccaaaa
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The potential societal impact discussed in appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper proposes explanation method. No pretrained model dataset released.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
All the code, data used in the paper are publicly available and cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The details of the code and model are provided in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowd sourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

A Proofs
A.1
Conditional Independence
Proof. We define the joint latent variable representation as h = (zo. c, c, where I denotes the input image and с fo(x) is the corresponding classifier output. Here, 21 represents the latent encoding of z, and is sampled from a Gaussian prior. The distribution over / can be expressed as:
(h) = q(20, 21 c,
(220, 21, 2,2
((c,z)
a(r x,2) () rr,c)
= =(0)) ((21 (c)
The transition from the second the third line relies the conditional dependencies illustrated in Figure 3] From the third to the fourth line, we use the fact that the auxiliary variable is fully determined by and c. Furthermore, since zi is encoded representation of assume that all relevant information in æ is captured by 21. Consequently, the conditioning on æ can be omitted in the final expression. For the same reason, remove the x--conitioning in the representation and define h = (20, 21 c).
A.2 Mutual Information
Proof. We adopt the information bottleneck framework, which aims to find representation that balances compression of the input X with retention of information relevant to the target label Y. The objective
min
T(X: BI(Z;Y),
where [Xx; Z) quantifies compression and I(Z;Y) quantifies predictive power, with 3 as the trade- off coefficient.
In our case, Z1 is lifted to Zo conditioned on the class C. The goal of lifting is to minimize T(Zo C: Z1), effectively compressing Zı into Zo while conditioning class-related information. Conversely, the landing transport reconstructs Z₁ from Zo conditioned on C, aiming to maximize I[Z;; Z C). Since the transport is invertible, assume 3 = 1.
The information loss is expressed as:
min T(%o C; Z1) 71 (C)
min 11,,,, 1(21, - (Zo,, C) 1(Zo, C)]
min T(Zo. C) C(Z,C)
C.
where si parameterizes the vector field used in flow matching. In the last two steps, we notice from Figure3Jthat the minimum mutual information between Zo and C is zero because Zo only depends on R in optimal setting; the maximum mutual information between Zi and C' C because Zi contains both and R. This derivation confirms that the information removed during lifting corresponds precisely to C. the class-related information.
22

---

B Experiments
The following section provides more details regarding experiments including the implementation (Section B.1 and evaluation metrics (Section B.2).
B.1
Implementation Details
This section provides more details regarding compute resources, datasets and hyperparameters.
B.1.1 Technical Details
We performed our experiments on a single node a GPU server, which includes one NVIDIA A100 with 80GB of VRAM, and an AMD EPYC 7742 with 1TB RAM shared with the other nodes of the server.
B.1.2 Quantitative Assessment
Morpho-MNIST Morpho-MNIST 48 is modified version of MNIST. The dataset is designed as benchmark dataset aimed at quantitatively assessing representation learning.
Generative Models The VAE model consists of encoder and decoder. The encoder architecture
is defined as follows: Conv(3, 8, 3, 2, 1) - BN - LeakyRe - Conv(8, 16, 3, 2, 1) + BN + Lea.y + C16 32, 2, 1) - BN - (eu0)) BN + LeakyReLU(0.2) + (12c, 64), where the final linear layer produces a 64-dimensional vector, which split into the mean vector f and the standard deviation vector The decoder mirrors the encoder architecture and includes an additional 1 convolutional layer with a Sigmoid activation at the end reconstruct the input data.
C3nv(32. 32, 2, 1)
The model is trained for 100 epochs using the Adam optimizer with a learning rate of 5 x 10-3 and a batch size of 256. The loss function combines the mean squared error (MSE) reconstruction loss and the Kullback-Leibler divergence (KLD), with weighting ratio of 4,000 between the MSE and the KLD terms.
Opt-based Method with a learning rate of 0.2 for 1,000 epochs. The hyperparameter set to 0.0006 to mitigate gradient vanishing.
For each batch of input, we optimize Equation [ using the Adam optimizer
We also experimented with ReLU activation functions in the VAE architecture. However, the results show that the counterfactual explanations (CEs) remain unchanged due to gradient decay. Consequently, we retain the LeakyReLU activation with a negative slope of 0.2 throughout the experiment.
CGM-based Method To incorporate conditional information, we extend the input dimension of the linear layer at the end of the encoder and the linear layer the beginning of the decoder from 128 to 138. This allows the model to accept 10-dimensional one-hot encoded class label from the Morpho-MNIST dataset as the condition input. The training procedure remains the same as that of the standard VAE model.
LeapFactual The architecture of the flow network defined as follows: Linear(32 10, 64) + SiLU + Linear(64, 64) + SiLU + Linear(64, 64) - SiLU + Linear(64, 32), where the input dimensions 32, 1, and 10 correspond the latent vector from the VAE, a time conditioning variable, and one-hot encoded class label, respectively. The flow matching noise parameter a is set to 0. The model trained using the Adam optimizer with a learning rate of 0.005 and batch size of 256 for 30 epochs, taking approximately 80 seconds to complete.
For this experiment, the hyperparameters of LEAPFACTUAL set follows: 7h 0.1 and N,, 15 for blending, and 7. iif = 0, land = 0.1, and N; for injection.
Additional qualitative results from LEAPFACTUAL provided in Figure
23

---

2
2
3
4
5 5
4
6 6
7 7
3 8
9
9
9
6
9
2
0
3
4
5
5
5
9
6 6
7 7
7
8
9
6
6
2
2
5
0
0
5
7
7
0
8
8
1
3
3
2
333
7
2
5
7
6 6
7
2
8
8
1
9 9 9 9 9 9 9
7
4
2
3
5
5
5
16626
6
7
4
8
8
4
9 9 9 7
6
3
5
2
2
3
3
6
4
4
0
4
7 7 6
6
9 8
5
8
/
9
2
2
3
3
3
4
4
4
5
5
6
6
477
8
8
O
0
1
2
2
2
7
3
3
4
7
5 5
3
366277
277
8 8
7
6
2
2
7
3
3
8
4
4
4
5
5
7
6 6
37
6 8
/
6
2
2
3
3
4
4
6
6
6 8
9
Figure 8: Morpho-MNIST different input (rows) and target labels (columns). In each column, the input, CEs, and reliable CEs are shown from left to right.
Table 5: Galaxy 10 ablation experiment of No with fixed 7h 0.1
No
ACC1
AUC1
ACC,
AUC,1
SSIM
PSNR1
LPIPS.
100
0.9718
0.9855
0.4868
0.8798
0.8579
22.8416
0.0641
150
0.9772
0.9884
0.4862
0.8829
0.8562
22.3836
0.0655
200
0.9814
0.9896
0.4880
0.8832
0.8551
22.1526
0.0663
250
0.9832
0.9901
0.4892
0.8836
0.8544
21.9950
0,0668
300
0.9832
0.9898
0.4874
0.8835
0.8538
21.8846
0.0673
B.1.3
Model Improvement
This experiment deals with Model improvement.
Dataset The Galaxy 10 DECaLS dataset is publicly available: Galaxy DECaLS It includes around 18,000 colored images and deals with a 10-classes galaxy morphology classification task. The dataset is split into training and test sets with a fraction of 90% and 10%. The pre- pre-processing includes random rotation augmentation, cropping to the center 150 x 150 pixels, and resizing to 128 128.
VAE Model activation functions replaced by LeakyReLU. For latent space regularization, we substitute the KL divergence penalty with the Maximum Mean Discrepancy (MMD) 588.
The architecture of the VAE is adapted from Ditria and Drummond [57], with all ELU
The architecture of the flow network is defined as follows: Linear(64 + 1 10, 256) + SiLU - L256 256) - SiLU + Linear(256, 256) - SiLU Linear(256, 256) - SiLU - Linear(256, 256) + SiLU + Li56ar(256,, 64), where the input dimensions 64, 1, and 10 correspond to the latent vector from the VAE, the time conditioning variable, and the one-hot encoded class label, respectively. The flow matching noise parameter is set to O. The model is trained using the Adam optimizer with a learning rate of 10-4 batch size of 256, and for a total of 500 epochs. Training takes approximately 2 hours.
Flow Model
Ablation Study We begin with an ablation study to identify the optimal value of N₂ for blending, while keeping 7h = 0.1 fixed. As shown in Table the best performance is achieved with N₂ 250. Next, we fix N₁ = 250 and ? = 0.1, and proceed to tune the injection hyperparameters.
To evaluate reliable counterfactual CEs, we train two VGG16 classifiers using 20% and 100% of the dataset, referred to as the weak and strong classifiers, respectively. Their test accuracies are approximately 81% and 85%. We generate both standard and reliable CEs from the weak classifier and evaluate them using accuracy (ACC) and area under the curve (AUC) metrics, as assessed by the strong classifier. These denoted ACCr and AUCr, respectively. These two metrics used to select the best injection hyperparameters for generating reliable CEs.
To tune the injection parameters, we first fix the difference "ilannd Vilin and vary Tim followed by optimizing Tilannd- As shown Figure 9 the best performance obtained with 7ina and
ladd 1.025.

---

ACC,
0.6
AUC,
0.95
0.90
0.5
1.0
1.025 1.050 1.075 1.100
Vi.m
Yuand
Figure 9: Hyperparameter ablation (Left) Ablation of Ni with Viland Nin 0.025. (Right)
Ablation of Milad with "i, = 1. Line color intensity increases with N; steps of 10, starting at 10.
Finally, we observe the effect N,. In Table 6 we can find that reliable metrics improve with injection steps. Notably, ACC, rises from 47% 82% without degrading the weak classifier's accuracy. This confirms that the standard CE lying around the weak classifier's decision boundary, while reliable CE is more aligned with the true decision boundary.
More qualitative results are shown in Figure 10
Table 6: Quantitative results for Galaxy 10 reliable CEs. Uncertainties calculated across 10 runs.
Metric
N;
N;
N; 15
Ni 25
Ni 35
ACC1
0,9821 a 0.0008
0.9439 a: 0.0017
0.9599 a: 0.0011
0,9612 a: 0.0012
0.9545 + 0.0011
AUCT
0,9897 0.0003
0.9934
0.0003
0.9973 1 0.0001
0,9975
0.0001
0.9974 = 0.0001
ACC,-
0.4738
0.0034
0.6620
0.0040
0.7899 0.0023
0.8171
0.0028
0.8135 1 0.0031
AUG,
0.8797
0.0012
0,94p4 0.00n9
0.9740 + 0.0006
0,9815
0.0005
0.9832 + 0.0005
SSIM
0.8577
0.0007
0.8255 0.0009
0.7774 0.0010
0.7417 0.0010
0.7122 0.0010
PSNR
22.4959
0.0778
21.0064
0.0826
19.2938 * 0.0768
18.0882
0.0787
17,0410 = 0.0733
LPIPS]
0.0651
0.0005
0.0006 0.0821
0.1095 0.0006
0.1324 0.0007
0.1524
B.1.4 Generalization
This section deals with the experiment setup regarding the generalization experiment.
CLIP We CLIP model with Vision Transformer (ViT-B/32) the image encoder. The data transferred from [-1, 1] to [O, I] before being fed into the transformation of the CLIP model. The model is trained to differentiate between face and smiling face. We determine the labels based on the higher score among Smiling face and Face predicted by the CLIP mode.
StyleGAN3 We use StyleGAN3 [52 pretrained on the FFHQ dataset (checkpoint name stylegan3- r-ffhq- 1024x 1024x1024.pkx from here). The noise mode is set to 'const' and the truncation __si is set to 1. randomly sample from a Gaussian distribution and project them to w-space and image space the mapping function and decoder.
LeapFactual In this experiment, we use ID U-Net [56] as the flow matching model. To train the model, we first sample 20,000 random latent vectors from Gaussian distribution and map them the w-space as the input. Then we use the classifier output of corresponding images as the condition of the U-Net. We use Adam optimizer to train the model for 120 epochs with a batch size of 32, learning rate of 2 x 10-4 and a weight decay of 1 x 10-5, Training is performed on a single NVIDIA A100 GPU and takes approximately 22 hours.
The hyperparameters for the blending parameters are % = 0.8 and N₂ = 10. For information injection, set Rann = 0.8, "idm 0.83, and N, 5. The flow matching is 10-4
B.2 Evaluation Metrics
This section introduces the evaluation metrics employed for the evaluation of our experiments.
25

---

B.2.1 Morphometries
proposed the Morpho-MNIST dataset as well as measurable properties known as morphometrics evaluate differences regarding characteristics like stroke thickness and area, length, width, height and slant of the depicted digits. The assessment of these metrics begins with first binarizing the image, followed by measuring various features about the resulting skeleton including: the length of the skeleton (length), average distance between the center of skeleton and its closest borders (thickness), horizontal shearing angle (slant) and the dimensions of a bounding box (width, height and area).
48
We report the absolute of the relative error regarding the morphometries:
Absolute Relative Error Absolute Error
True Value
with the absolute error being the difference between measured and true value. We employ the absolute value, since the angle reported for slant may be negative.
We used the official implementation available online:
Morpho-MNIST git For more details, please
refer to 48
B.2.2 Metries
We assess the correctness of the generated CE employing accuracy (ACC) and area under the Receiver Operating Curve (AUROC), using the implementations provided by torchmetrics For both metrics higher is better. While accuracy provides a quick overview of how often the model provides the correct prediction (correct predicitons / total predictions). Since AUROC denoted the area under the curve between true positive rate and false positive rate, it provides deeper insight and is favourable especially when the dataset imbalanced.
To evaluate the similarity between original image and CE, we employ three well-known Perceptual Similarity Metrics working on different levels. Including Structural Similarity Index (SSIM) [59), Learned Perceptual Image Patch Similarity (LPIPS) metric [60]. and Peak Signal-to-Noiss Ratio (PSNR).
SSIM evaluates similarity based on comparing three concepts: luminance, structure and contrast.
(2palHace + C1) (20 + C2)
SSIM(I, (02 1220 + C1) (o2 + 22 + C2)
The influence of luminace is included as the means of pixel values denoted as frr and Maae contrast is included with the standard deviations a₂ and dxer and finally the covariance azaes is employed to take account for structure.
PSNR, in contrast, is a more simple metric building the mean-squared error and comparing pixel-wises basis.
max (2r)2
PSNR(r, xCB) 10 log10
(5)
MSE(2, .CEE
Unlike PSNR or SSIM, LPIPS evaluates the similarity of two images based on deep features extracted from neural network, as feature extractor we set Squeezenet. The similarity is evaluated by com- paring layer-wise computed euclidean distances of the normalized images. Which are subsequently being accumulated distance between the two images, therefore lower LPIPS score resembles a higher similarity.
weighted-sum, the weights being learned. The score can then denotes the
More details can be found in the original publications. In our experiments, we use the implementations provided by ttrchmetrics setting Squeezenet as backbone network for LPIPS while keeping all other parameters at their default values.
C Broader Impacts
Our work introduces a method to generate realistic and reliable counterfactuals, aiming at inter- pretability. By demonstrating how input data has to be modified in order to obtain a different decision and providing actionable changes, we promote transparency and reduce of barriers the application of Machine Learning models.
26

---

Potential positive impacts of our approach may include, supporting non-expert users in understanding decisions, enhancing scientific knowledge discovery and enabling future works regarding fairness. Our method could be extended to generate counterfactual samples for underrepresented minorities in datasets. Additionally, counterfactual explanations can be utilized to investigate features contributing to the decision, thereby facilitating bias detection.
However, there risks for potential misuse, such as the creation of deep fakes. Our approach allows changing the input data towards specific classes, thus allowing to generate new data. Another concern may that a simplified representation of the decision decision-making process could mislead users. While we have no opportunity to mitigate the risk towards deep fakes, we advocate for the use of reliable CEs to minimize potential oversimplifications. These CEs ensure, that the generated counterfactual sample remain in the distribution of the data, avoiding out-of -ieoof--ittibution artifacts that could mislead the user.
27

---

Figure 10: In each group (two rows), the column with a single image the input, the remaining CEs (top) and reliable CEs (bottom). From top bottom, the input image are in the classes of Distributed, Merging, Round Smooth, In-between Round Smooth, Cigar Round Smooth, Barred Spiral, Unbarred Tight Spiral, Unbarred Loss Spiral, Edge-on without Bulge, Edge-on with Bulge
28


Reviews:
[{"review_id": "Bik9caWxtv", "reviewer": "Reviewer_rHeX", "summary": "The paper introduces LeapFactual, a counterfactual explanation algorithm. LeapFactual leverages conditional flow matching to generate robust and plausible counterfactuals by separating class-related features from residual information. Authors reformulate conditioning term from I-CFM to be class related. It addresses critical limitations of existing methods, such as gradient vanishing, discontinuous latent spaces, and reliance on learned decision boundaries. The method combines the strengths of optimization-based and generation-based approaches while remaining model-agnostic, enabling use with non-differentiable systems like human-in-the-loop workflows. Key contributions include:\n1. Proposing LeapFactual: A CFM-based algorithm that bridges flattened and structured latent spaces, allowing to produce realistic counterfactuals even when learned decision boundaries diverge from true ones.\n2. Theoretical grounding: The CE-CFM objective is theoretically justified, showing flow matching inherently disentangles class-related and residual information.\n3. Empirical validation: Extensive experiments on benchmark datasets: Morpho-MNIST, Galaxy10 DECaLS and real-world tasks demonstrate superior accuracy, interpretability, and utility in enhancing model performance via robust CEs used as training data.", "questions": "1. Authors compare their proposed method against defined baselines, but not against other published methods. Could you compare your method against other existing methods e.g. https://arxiv.org/pdf/2210.11841\n2. Could you explain why different metrics were used for different datasets? Additionally could you provide explanations on why different datasets were used to evaluate different aspects of the proposed method and not all 3 datasets for all aspects?\n3. How does computational cost scale with input resolution (high-res images like FFHQ)? A brief analysis of training/inference time trade-offs compared to baselines would strengthen claims of generalization.\n4. Authors mention non-differentiable models (human annotators). What safeguards prevent biases in such scenarios, and how does the method mitigate risks of propagating human errors into counterfactuals?", "limitations": "The authors acknowledge scalability issues in high-dimensional spaces but could expand on potential mitigation strategies via OT-CFM as mentioned in future work. Ethical impacts are briefly addressed, but further analysis of bias amplification or misuse scenarios would improve robustness. Overall, the limitations are candid and well-articulated.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Quality\n### Strength\n  1. The theoretical foundation, including *Theorem 1* on information compression via class conditioning, provides clear justification for the design of their CE-CFM training objective.  \n  2. Empirical validation is thorough: experiments on Morpho-MNIST, Galaxy 10 DECaLS, and FFHQ datasets demonstrate significant improvements over baseline Opt-based and CGM methods in terms of accuracy, AUC, and perceptual metrics like SSIM/PSNR.  \n  3. The method handles non-differentiable models (human-in-the-loop systems via CLIP).\n  4. Robust CEs improve model training accuracy when used as augmentations, addressing real-world dataset imbalance challenges.\n### Weaknesses:  \n- Scalability limitations in high-dimensional spaces are mentioned as a future work, but no ablation studies on computational efficiency are provided.  \n- While the authors discuss discrepancies between true/learned decision boundaries, they do not directly evaluate how often their method \"rescues\" misaligned decisions. This could be quantified with additional experiments.\n- Edge Cases Unexplored: Potential failure scenarios, such as noisy inputs or extreme class imbalances beyond tested datasets, are not addressed in experiments.  \n---\n## Clarity\n### Strengths:  \n  - The paper is well-organized and clearly structured. Figures effectively convey complex ideas. Figure 1 represents conceptual illustration of decision boundaries and Figure 4 shows flow transports.\n### Weaknesses\n- The authors incorrectly use the term 'robust' when describing counterfactual explanations that lie closer to the data manifold. According to [1], this property is more accurately termed 'plausibility.' In the established literature, robustness specifically refers to the stability of counterfactual explanations under small perturbations to either the input data or the underlying model [2, 3], which is conceptually distinct from the notion of plausibility.\n- Authors state that OPT-based methods face gradient vanishing problem, but this claim is unsupported.\n- Authors discuss structural and flattened representations, but the definition is unclear.\n- Section 3.2.1’s discussion of conditional flow matching requires deeper familiarity with I-CFM, leaving some gaps for non-experts in flow-based models.\n- Algorithm 1 would benefit from pseudocode details on how `BLENDINGLEAP` and `INJECTIONLEAP` are implemented.\n\n[1] Guidotti, Riccardo. \"Counterfactual explanations and how to find them: literature review and benchmarking.\" _Data Mining and Knowledge Discovery_ 38.5 (2024): 2770-2824.\n\n[2] Artelt, André, et al. \"Evaluating robustness of counterfactual explanations.\" _2021 IEEE symposium series on computational intelligence (SSCI)_. IEEE, 2021.\n\n[3] Jiang, Junqi, et al. \"Formalising the robustness of counterfactual explanations for neural networks.\" _Proceedings of the AAAI conference on artificial intelligence_. Vol. 37. No. 12. 2023.\n\n---\n## Significance\n### Strengths\n  - The work addresses a critical challenge in counterfactual explanations - generating robust samples within distribution while handling model discrepancies.\n  - Model-agnosticism enables broader adoption, including scenarios where retraining generative models per classifier is impractical.  \n### Weaknesses\n  - No\n\n---\n## Originality\n### Strengths\n- The integration of *conditional flow matching* for counterfactual generation is novel.\n- Combines structured latent spaces with continuous exploration.\n- Introduces distinct transport operations (lifting and landing) to decouple class-related vs. residual information, enabling blending/injection beyond simple replacement.\n### Weaknesses\n- No", "was_revised": true, "final_justification": "See the response to the authors.", "rebuttal": "Dear Reviewer,\n\nWe sincerely thank you for your feedback and time. We appreciate the positive feedback regarding significance and originality and the helpful constructive criticism, that we think in most cases can be integrated easily. To address your concerns, we conducted an extra experiment on the **tabular adult dataset** and appended it at the end. In the following, we address each concern and question point by point. \n\n## Quality weaknesses:\n\n**W1 (scalability limitations - no ablation on computational efficiency):**  We included reference training time in Appendix B.1.2, B.1.3, and B.1.4: The model for the MorphoMNIST, Galaxy, and FFHQ experiments takes about 80 secs, 2 hours, and 22 hours to train on a single A100 GPU, respectively. However, we emphasize that the bottleneck for the FFHQ experiment comes primarily from StyleGAN, which is a heavy generative model, taking relatively long to decode the latent vectors. The 1D-UNet model can also be improved to increase efficiency. We mention scalability limitations for high-dimensional spaces as future work, especially regarding diffusion models and normalizing flows, since the latent-space dimensions are as high as input images. Therefore, the training time is expected to be much longer.\n\n**W2 (rescue misaligned decisions - additional experiments):** We address differences in true and learned decision boundaries as stated, however directly measuring how often we ‘rescue’ samples due to misalignment seems not trivial. As an indirect measurement, we refer to the experiment in Appendix B.1.3 (Ablation Study) and the new adult experiment (please see below). The experiments show that the accuracy of the **strong** classifier increases when we increase the number of injection steps, indicating that the performance on rescuing samples from misalignment depends on the number of injection steps. \n\n**W3 (potential failure scenarios like noisy input or class imbalances):** We begin with addressing the edge case of noisy samples, which we understand as noisy data rather than noisy labels, unlike in Q4. If the generative model provided by the user is robust to noise, the latent space itself will be more or less the same. Since we apply our method on the latent space, our approach depends on the robustness of the generative model. Secondly, the new adult tabular data experiment shows that the method is robust to imbalanced data (20% high income vs. 80% low income) at least to some degree. More experiments are needed to measure this robustness, as mentioned in our future work section. We emphasize that common upsampling or downsampling strategies can be applied here to rescue the imbalanced data problem.\n\n## Clarity weaknesses:\n\n**W1 (incorrectly named robust instead of plausible):** Thank you for pointing this out. We agree that our definition of robustness is not favorable since it will lead to confusion. We want to stress the fact that the produced counterfactuals are dependable, thus plausibility does not really capture what we want to express. However, we agree with you that renaming is necessary, and therefore, we will update the notation in the camera-ready version to **reliable counterfactuals**. \n\n**W2 (OPT gradient vanishing - claim unsupported):** We understand that the claim seems to be unsupported. The gradient vanishing can be explained by the theory behind optimization-based methods. If we look at the loss function (Equation 1), we can find that it depends on the learned decision boundary, and the first summand becomes zero as soon as it reaches the “correct” class (regarding the learned decision boundary). Since we assume there to be an imperfect classifier, we see this leading to gradient vanishing. This can also be observed in Appendix B.1.3 (Ablation Study) and the adult experiment, where the accuracy of counterfactuals generated by the **weak classifier** is very low regarding the **strong classifier**. \n\n**W3 (structural and flattened representations, definition unclear)+ W4 (discussion of conditional flow matching) + W5 (pseudocode details for Blendingleap and Injectionleap):** Thank you for the hints, we will improve the definition of structural and flattened representation to be more precise and elaborate on conditional flow matching in the appendix of the camera-ready version. Similarly, the pseudocode will also be integrated in the appendix.\n\n## Questions:\n\n**Q1 (Compare with other baselines):** We focus on a fair comparison and therefore we want to compare against competitors with the same starting points, which are the models in our case. For all approaches, we compare against the paradigms to showcase the potential of our approach, rather than comparing specific models that belong to one of the approaches. To our knowledge, the referred work uses a classifier-guided diffusion model to generate CE, which is in the optimization-based paradigm. However, because generalizing to diffusion models would be our future work, we really appreciate the reviewer’s suggestion and will carefully read the paper.\n\n**Q2 (Different metrics and datasets):** The MorphoMNIST dataset is a specifically designed benchmark dataset that provides reliable metrics within its implementation. Unfortunately, these metrics can not be easily transferred to other datasets. The classifier accuracy for MorphoMNIST is already high, therefore the model improvement experiment for this dataset can not provide any additional information. For FFHQ, there is no ground truth label on whether the person is smiling or not; therefore, we can not do the model improvement experiment. For the generalization experiment, the MorphoMNIST and Galaxy data are trivial (or have been demonstrated to work), and we need a more challenging dataset.\n\n**Q3 (Computation cost):** We did include training time for the experiments in Appendix B.1.2 to B.1.4. The inference time is in the scale of seconds or minutes, depending on the problem difficulty. See also W1.\n\n**Q4 (Human error):** Thank you for bringing up this discussion. We assume that misalignment will always happen. Human annotation errors cause decision boundary misalignment. Our reliable counterfactuals mitigate this, as they do not strictly rely on the learned boundary. The Adult experiment shows the approach works even when labels are intentionally flipped.\n\n## Extra experiment\nWe conducted an additional experiment using the Adult dataset [1]. The dataset was split into training and test sets using an 80/20 ratio.\n\nWe trained a weak classifier with label noise, achieved an accuracy of 0.7093, and an AUC of 0.6874. The strong classifier trained on clean data reached 0.8546 accuracy and 0.9039 AUC.\n\nWe then trained a TVAE [2] generative model and a flow matching model conditioned on the weak classifier's predictions to generate counterfactuals (CE) and reliable CE (target labels = flipped predictions). Both classifiers evaluated the generated samples.\n\nTo measure the similarity, Categorical features were one-hot encoded, and numerical features were normalized to enable the Mean Squared Error (MSE) metric.\n\nFrom the results, we observe the following:\n- **Weak classifier's** accuracy increases with blending steps but drops sharply during injection.\n- **Strong classifier's** accuracy starts below the weak classifier's during blending but surges from ~73% to ~93% after 10 injection steps, reaching 97% in the end.\n\nThese findings demonstrate that:\n- The proposed method is effective on tabular data, provided the generative model is capable of capturing the underlying causal relationships.\n- The method is robust to imbalanced datasets (the Adult dataset contains 80% low-income and 20% high-income labels) and can handle noisy data (e.g., human annotation errors), as evidenced by the performance of the weak classifier trained with noisy labels.\n- The reliable counterfactual examples effectively resolve the misalignment between the learned and true decision boundaries, under the assumption that the strong classifier better approximates the true decision boundary than the weak classifier.\n\n| Metric        | N_b=30         | N_b=60         | N_b=90         | N_i=10         | N_i=20         | N_i=30         |\n|---------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|\n| **ACC↑**     | 0.7607 ± 0.0027         | 0.8355 ± 0.0026         | **0.8503 ± 0.0026**     | 0.6984 ± 0.0016         | 0.6686 ± 0.0020         | 0.6389 ± 0.0013         |\n| **AUC↑**     | 0.8522 ± 0.0014         | 0.8989 ± 0.0017         | **0.9082 ± 0.0016**     | 0.8070 ± 0.0011         | 0.7905 ± 0.0015         | 0.7713 ± 0.0011         |\n| **ACC_r↑** | 0.7232 ± 0.0017         | 0.7258 ± 0.0017         | 0.7276 ± 0.0019         | 0.9328 ± 0.0016         | 0.9655 ± 0.0009         | **0.9692 ± 0.0009**     |\n| **AUC_r↑** | 0.7841 ± 0.0019         | 0.7860 ± 0.0018         | 0.7874 ± 0.0021         | 0.9466 ± 0.0019         | 0.9747 ± 0.0011         | **0.9783 ± 0.0011**     |\n| **MSE↓**     | **0.0537 ± 0.0001**     | 0.0544 ± 0.0001         | 0.0546 ± 0.0001         | 0.0688 ± 0.0002         | 0.0717 ± 0.0002         | 0.0731 ± 0.0002         |\n\n*Metrics with mean ± standard error for 5 runs. Bold values represent the best-performing configuration for each metric. The first three columns (N_b) indicate performance changes across blending steps, while the last three columns (N_i) indicate performance changes during injection steps, which continue from the last blending step. $\\gamma_b = 0.1$, $\\gamma_{i,lift}=1$, and $\\gamma_{i,land}=1.1$. ACC and AUC are evaluated using the weak classifier, whereas ACC_r and AUC_r are evaluated using the strong classifier.*\n\n[1] Becker, B. & Kohavi, R. (1996). Adult [Dataset]. UCI Machine Learning Repository.\n\n[2] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. Modeling Tabular data using Conditional GAN. NeurIPS, 2019."}, {"review_id": "di87Knp9YV", "reviewer": "Reviewer_XFr6", "summary": "- This paper proposes a method for generating realistic counterfactual examples. Traditional counterfactual explanation methods, including optimization-based approaches and CGM-based methods using label-conditioned generative models, sometimes fail to produce realistic counterfactual examples. \n- The authors introduce a new counterfactual explanation method based on conditional flow matching. This approach trains a map between two types of latent representations: a \"flattened representation\", where class-related and residual features are entangled, and a \"structured representation\", where only residuals are entangled. By doing so, it aims to generate counterfactual examples that account for both.\n- Numerical experiments demonstrate that the proposed method successfully outputs natural images as counterfactual examples. The authors also show that using CEs generated by their method as new training samples can improve prediction performance.", "questions": "- The flow matching model uses a simple 4-layer MLP. This isn't inherently an issue, but could the computation time increase significantly if we use a large neural network for the flow matching model in your proposed method? Please discuss the computation time for training a flow-matching model. \n- In the numerical experiments, the proposed method was applied to image data, but I am interested in its applicability to tabular data. In tabular data where features have causal relationships, could the proposed method be used to generate causally aware counterfactual explanations?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Strengths\n\t- Research on generating realistic counterfactual examples is highly significant in explainable machine learning. This paper proposes a method to achieve this using a flow-matching model. The approach of using flow matching for counterfactual explanation is original, and I believe the research holds substantial value for the broader machine learning community.\n\t- The computational experiments validate the proposed method's effectiveness through multi-faceted and appropriate procedures. Specifically, the result that prediction performance improves when we use counterfactual examples generated by the proposed method as training samples is particularly interesting.\n\t- The paper itself is clearly written and easy to read.\n- Weaknesses\n\t- Many explanations rely on intuition based on specific examples, and the theoretical foundations for how the proposed method overcomes the limitations of existing research seem somewhat weak.\n\t- While not a fundamental weakness, the term \"Robust CE\" is used in various contexts within Counterfactual Explanations (CE) and Algorithmic Recourse (AR). For instance, (Dominguez-Olmedo et al., '22) defines \"Robust CE\" as robustness against uncertainty in input features. Given the diverse definitions of robustness in CE (see (Jiang et al., '24)), a more detailed explanation of what \"Robust CE\" signifies in this research and what \"robustness\" implies would be beneficial. \n\n\n(Dominguez-Olmedo et al., '22) Ricardo Dominguez-Olmedo, Amir H Karimi, and Bernhard Schölkopf. On the Adversarial Robustness of Causal Algorithmic Recourse. Proceedings of the 39th International Conference on Machine Learning, PMLR 162:5324-5342, 2022.\n\n(Jiang et al., '24) Junqi Jiang, Francesco Leofante, Antonio Rago, and Francesca Toni. Robust Counterfactual Explanations in Machine Learning: A Survey, https://doi.org/10.48550/arXiv.2402.01928. \n\n(Dominguez-Olmedo et al., '22) Ricardo Dominguez-Olmedo, Amir H Karimi, and Bernhard Schölkopf. On the Adversarial Robustness of Causal Algorithmic Recourse. Proceedings of the 39th International Conference on Machine Learning, PMLR 162:5324-5342, 2022.\n(Jiang et al., '24) Junqi Jiang, Francesco Leofante, Antonio Rago, and Francesca Toni. Robust Counterfactual Explanations in Machine Learning: A Survey, https://doi.org/10.48550/arXiv.2402.01928.", "was_revised": true, "final_justification": "The authors have sincerely addressed all of my concerns and questions. While I recommend they revise the manuscript to incorporate their responses, I believe the paper is suitable for publication.", "rebuttal": "Dear Reviewer,\n\nWe thank you for the thorough and helpful review. We appreciate the feedback and are happy to address it to improve our paper even more. To address your concerns, we conducted an extra experiment on the **tabular adult dataset** and appended it at the end. In the following, we will address weaknesses and questions individually.\n\n## Weaknesses:\n**W1 (explanations rely on intuition, theoretical foundations for how the proposed method overcomes the limitations of existing research seem somewhat weak):** Thank you for the feedback. We agree that it would be valuable to clarify this point more explicitly. In our revision, we will add a few sentences to state clearly that we address the limitations by applying flow matching on a flattened (OPT-based) representation and subsequently using it to obtain a structured (CGM-based) representation, which is then leveraged for counterfactual generation. This approach allows us to capitalize on the strengths of both paradigms while mitigating their respective drawbacks. We also provide the theoretical foundations of both paradigms and detail how their advantages are integrated in Section 3.1. Regarding the intuitive explanation, we view this as a key property and strength of counterfactual explanations (CE): CE offers users clear information that helps them intuitively grasp the underlying reasoning behind decisions. If there are additional points we have not yet addressed, we would greatly appreciate further feedback so that we can continue improving the paper.\n\n**W2 (\"Robust CE\"):** We see that the term “Robust CE” brings up confusion as it denotes other characteristics in different contexts. In our case we want to express that the generated counterfactuals are robust in the sense of alignment to the dataset. After reading the reviews we acknowledge the confusion and rename it to **reliable counterfactuals**, as this expresses the dependability of the method we are aiming at.\n\n## Questions:\n**Q1 (Computation time for a larger flow-matching model):** We agree that the training and inference time will also depend on the dataset and model applied. We included reference training time in Appendix B.1.2, B.1.3, and B.1.4. The model for the MorphoMNIST, Galaxy, and FFHQ experiments takes about 80 secs, 2 hours, and 22 hours to train on a single A100 GPU, respectively. However, we emphasize that the bottleneck for the FFHQ experiment highly comes from StyleGAN, which is a heavy generative model, taking relatively long to decode the latent vectors. The 1D-UNet model can also be improved to increase efficiency. The inference time is in the scale of seconds or minutes, depending on the problem's difficulty. \n\n**Q2 (Generalization data types):** Thank you for the suggestion. We did an extra experiment on the tabular adult data (see below). The experiment suggests that if the generative model is strong enough to capture the causal relation, we can generate reasonable CEs. \n\n## Extra experiment\nWe conducted an additional experiment using the Adult dataset [1]. The dataset was split into training and test sets using an 80/20 ratio.\n\nTo establish baseline classifiers, we first trained a **weak classifier** by introducing label noise—flipping the labels of a fraction of the training data. This model achieved a test accuracy of $0.7093$ and an AUC of $0.6874$. Subsequently, we trained a **strong classifier** on clean data without label noise, achieving a test accuracy of $0.8546$ and an AUC of $0.9039$.\n\nNext, we trained a TVAE [2] as our tabular data generative model using the full dataset. We then trained a flow matching model conditioned on the predictions of the weak classifier to generate counterfactual examples (CE) and reliable counterfactual examples, where the target labels corresponded to the flipped predictions. Both the weak and strong classifiers were used to evaluate the generated samples in terms of accuracy and AUC.\n\nFor the similarity metric, categorical variables were converted into one-hot representations, while numerical features were normalized. The mean squared error (MSE) was then computed to quantify similarity between the generated and original samples.\nThe results of the experiment are summarized in the table below.\n\nFrom the results, we observe the following:\n- The accuracy of the **weak classifier** improves progressively as the number of blending steps increases, but drops sharply upon performing information injection.\n- The accuracy of the **strong classifier**, in contrast, is initially lower than that of the weak classifier during the blending steps. However, it surges from approximately **73**% to **93**% after only 10 injection steps and ultimately reaches **97**%.\n- The **MSE** increases consistently as the number of blending and injection steps grows.\n\nThese findings demonstrate that:\n- The proposed method is effective on tabular data, provided the generative model is capable of capturing the underlying causal relationships.\n- The method is robust to imbalanced datasets (the Adult dataset contains 80% low-income and 20% high-income labels) and can handle noisy data (e.g., human annotation errors), as evidenced by the performance of the weak classifier trained with noisy labels.\n- The reliable counterfactual examples effectively resolve the misalignment between the learned and true decision boundaries, under the assumption that the strong classifier better approximates the true decision boundary than the weak classifier.\n\n\n| Metric        | N_b=30         | N_b=60         | N_b=90         | N_i=10         | N_i=20         | N_i=30         |\n|---------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|\n| **ACC↑**     | 0.7607 ± 0.0027         | 0.8355 ± 0.0026         | **0.8503 ± 0.0026**     | 0.6984 ± 0.0016         | 0.6686 ± 0.0020         | 0.6389 ± 0.0013         |\n| **AUC↑**     | 0.8522 ± 0.0014         | 0.8989 ± 0.0017         | **0.9082 ± 0.0016**     | 0.8070 ± 0.0011         | 0.7905 ± 0.0015         | 0.7713 ± 0.0011         |\n| **ACC_r↑** | 0.7232 ± 0.0017         | 0.7258 ± 0.0017         | 0.7276 ± 0.0019         | 0.9328 ± 0.0016         | 0.9655 ± 0.0009         | **0.9692 ± 0.0009**     |\n| **AUC_r↑** | 0.7841 ± 0.0019         | 0.7860 ± 0.0018         | 0.7874 ± 0.0021         | 0.9466 ± 0.0019         | 0.9747 ± 0.0011         | **0.9783 ± 0.0011**     |\n| **MSE↓**     | **0.0537 ± 0.0001**     | 0.0544 ± 0.0001         | 0.0546 ± 0.0001         | 0.0688 ± 0.0002         | 0.0717 ± 0.0002         | 0.0731 ± 0.0002         |\n\n*Metrics with mean ± standard error for 5 runs. Bold values represent the best-performing configuration for each metric. The first three columns (N_b) indicate performance changes across blending steps, while the last three columns (N_i) indicate performance changes during injection steps, which continue from the last blending step. The step sizes are $\\gamma_b = 0.1$, $\\gamma_{i,lift}=1$, and $\\gamma_{i,land}=1.1$. ACC and AUC are evaluated using the weak classifier, whereas ACC_r and AUC_r are evaluated using the strong classifier.*\n\n[1] Becker, B. & Kohavi, R. (1996). Adult [Dataset]. UCI Machine Learning Repository. \n\n[2] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. Modeling Tabular data using Conditional GAN. NeurIPS, 2019."}, {"review_id": "8lJW0UiF0k", "reviewer": "Reviewer_8euj", "summary": "LeapFactual is a new method for generating visual counterfactual explanations leveraging conditional flow matching on an input's latent representation. Counterfactual explanations promise to provide more insight into a classifier's decision, since they try to apply (minimal) changes to an input $x$, transforming it to $x_\\mathrm{CE}$, such that the classifier $f$ predicts target class $c^\\prime = f(x_\\mathrm{CE})$ instead of the original class $c = f(x)$. If the intrinsic dimension of the data is $<<$ than its ambient dimension, it is common to use a generative model $g$ to transform $x$ to its lower-dimensional latent representation $z = g^{-1}(x)$ and search an appropriate $z_\\mathrm{CE}$ in the latent space such that $x_\\mathrm{CE} = g(z_\\mathrm{CE})$.\nThe authors claim that with their flow-based approach to counterfactual generation, they mitigate the drawbacks of the two predominant approaches:\n1. Optimization-based approaches (OPT-based) that minimize an objective composed of the classifier loss term for the new _target_ class $c'$ and a distance term between $x_\\mathrm{CE}$ and $x$, aiming to steer $x \\to x_\\mathrm{CE}$ such that the predicted class label flips to the desired class while keeping $x$ and $x_\\mathrm{CE}$ similar. While enabling continuous latent space interpolation, these approaches seem to suffer from vanishing gradients near the decision boundary and necessitate differentiable classifiers.\n2. Conditional Generative Model-based approaches (CGM-based), which integrate the classifier information directly by training conditioned on classifier output. This offers structured representations of residual and class information but lacks a continuous latent space due to explicit conditioning on classifier outputs.\nThe authors achieve this by transforming class-specific latent representations $z_{1,c}$ to class-agnostic latent representations $z_0$ using independent conditional flow matching (I-CMF), a process they refer to as \"lifting\", and infusing class information back by inverting this flow and conditioning on the target class, which they refer to as \"landing\". To avoid a situation where $x_\\text{CE}$ is too close to the classifier boundary (which might not conform with the true boundary), the authors describe to _inject_ additional class information after solely _blending_ it before.\nThe authors demonstrate the efficacy of their method on the Morpho-MNIST dataset, which provides a quantitative approach to measuring distortions of class-related and residual information, showcase its usefulness in dataset augmentation using the Galaxy10DECaLS dataset, and its aptitude to setting where classifiers are non-differentiable, reporting positive results.", "questions": "1. As alluded to in the \"Strengths & Weaknesses\" section, I don't quite understand the trade-offs of choosing certain hyperparameters. For instance, in Figure 1, the MNIST example seems to suggest a single leap is sufficient, while in Figure 4, $N_b$ and $N_i$ are higher. For instance, is the choice of $N_b$ and $\\gamma_b$ truly independent, or is there an equivalence for certain combinations of parameters?\n2. I wonder how this method extends to other data, for instance, textual data, when generating counterfactuals. Are the adaptations as trivial as suggested by the authors?", "limitations": "Besides the applicability to other data as claimed by the authors, the limitations raised by the authors (primarily the high computational effort due to the CFM approach) seem comprehensive.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths, in no particular order:\n1. The authors were able to empirically demonstrate that their method produces counterfactuals that are reliably recognized by the classifier as belonging to the target class while reducing distortions of residual information. This is particularly clearly demonstrated in section 4.1 (Morpho-MNIST task).\n2. Their method allows for considerable customization via the various degrees of information replacement, blending, and injection, although some more clarity with regard to the impact of the corresponding hyper-parameters would be appreciated.\n3. Generally, the figures and diagrams add some clarity, although Figure 4 could be improved substantially.\n\n---\nWeaknesses, in no particular order:\n1. Figure 4 and the accompanying text is hard to read:\n\t1. In the text, the authors state \"that multiple points from different classes in $Z_0$ are mapped to the same point in $Z_1$, which confirms Theorem 1\". While this observation complies with the theorem, it does not _confirm_ it (this observation would also be true for the trivial mapping that maps all $z_0$ to the origin, which is not a helpful mapping in this case).\n\t2. The described phenomena in 4 (a)-(d) are hardly visible due to the small size of the subplots. I recommend adjusting the margins between subplots and removing or reducing the size of the tick labels.\n2. The method is parameterized by multiple hyper-parameters, $N_b$, $\\gamma_b$, $N_i$, and $\\gamma_{i,\\mathrm{lift}} < \\gamma_{i,\\mathrm{land}}$. The authors provide some insights into the impact of these hyper-parameters in B.1.3 for the \"Model Improvement\" task presented in section 4.2, however, it would be interesting to see more ablations for other datasets, including the Morpho-MNIST dataset (and potentially a non-visual dataset, see the following point), or heuristics on how to choose there parameters based on the problem at hand.\n3. The authors state that \"while our method focuses on visual data, it can be generalized to other data types\". However, the authors only provide empirical evidence of the efficacy of their method on visual data (besides the simple toy problem discussed in 3.2.2). I would appreciate an experiment that puts this statement to the test on some non-visual dataset, or remove it from the paper.\n4. The clarity of the text could be improved in multiple sections:\n\t1. In section 2.1, the authors choose to start multiple sentences with bare citation numbers (e.g., \"[14] introduced REVISE ...\" or \"[17] use a GAN ...\"), which does not aid clarity.\n\t2. in line 220, I would favor \"subtracting C from X to get R\" instead of \"deducted\"-", "was_revised": true, "final_justification": "The rebuttal satisfactorily addressed most of my concerns. The clarification on Figure 4, planned improvements to readability, and corrections to wording improve clarity. The explanation of hyperparameter trade-offs is helpful, though I still see value in adding more concrete heuristics or equivalence insights. The additional tabular dataset experiment meaningfully strengthens the claim of generalizability beyond visual data, though I would like clarification on whether one-hot categorical constraints are preserved in generated counterfactuals. Given that these remaining points are relatively minor compared to the strengths demonstrated, I maintain a positive overall assessment.", "rebuttal": "Dear Reviewer,\n\nWe thank you for your time and thorough review. We are very happy to see that most of the issues seem to be solved straightforwardly. To address your concerns, we conducted an extra experiment on the \\textbf{tabular adult dataset} and appended it at the end. In the following sections, we address weaknesses and questions individually.\n\n## Weakness:\n**W1 (Figure 4 - confirm Theorem 1, subplots too small):** Thank you for acknowledging that the included Figures and diagrams added to clarity. We see the issue with the size of the subplots in Figure 4 and will make use of the additional content page in the camera-ready version to extend the Figure. We appreciate the recommendation of how we can improve it.  In the sentence \"...which confirms Theorem 1\", we recognize that confirm is the wrong choice of word for the description, we will change it to “...which complies with Theorem 1.”\n\n**W2 (hyperparameter ablations for other datasets -> heuristics for choice):** Please see Q1 below.\n\n**W3 (generalization to other datatypes, either non-visual dataset or remove from paper):** Please see Q2 below.\n\n**W4 (Section 2.1 bare citation start + phrasing in line 220):** Thank you for pointing this out. We agree with you and see that the paper's clarity and readability will benefit from rephrasing the sentences in Section 2.1 as well as changing the wording to the suggested one in line 220.\n\n## Questions: \n**Q1 (hyperparameter choosing; Number of steps in Figure 1 vs. Figure 4):** We provide some hyperparameter ablations in the appendix (Table 5 & 6, and Figure 9, also Table 4 in the main text). The approach takes two types of hyperparameters: step size and number of steps. Generally, a small step size is preferred for higher precision, although this may require a larger number of steps and consequently a higher inference time. The number of steps depends on the dataset and specific classes you generate the counterfactuals for; therefore, we can not provide the optimal parameters for every scenario. However, when blending only, the algorithm will automatically stop as soon as the target class is reached due to information replacement. Generally, we suggest starting with a small step size and a large number of steps. For future works, we see possible improvement, including adaptive step size. Regarding the figures: Figure 1 is a conceptual figure; in reality, the latent space is high-dimensional and more complex. We will improve the caption of the Figure to avoid misunderstanding. \n\n**Q2 (generalization data types):**  Thank you for the suggestion. We have done an extra experiment on the tabular adult dataset. Please see below for details. \n\n## Extra experiment\nWe conducted an additional experiment using the Adult dataset [1]. The dataset was split into training and test sets using an 80/20 ratio.\n\nTo establish baseline classifiers, we first trained a **weak classifier** by introducing label noise—flipping the labels of a fraction of the training data. This model achieved a test accuracy of $0.7093$ and an AUC of $0.6874$. Subsequently, we trained a **strong classifier** on clean data without label noise, achieving a test accuracy of $0.8546$ and an AUC of $0.9039$.\n\nNext, we trained a TVAE [2] as our tabular data generative model using the full dataset. We then trained a flow matching model conditioned on the predictions of the weak classifier to generate counterfactual examples (CE) and reliable counterfactual (we changed the name from robust to reliable according to reviewers' suggestions)) examples, where the target labels corresponded to the flipped predictions. Both the weak and strong classifiers were used to evaluate the generated samples in terms of accuracy and AUC.\n\nFor the similarity metric, categorical variables were converted into one-hot representations, while numerical features were normalized. The mean squared error (MSE) was then computed to quantify similarity between the generated and original samples.\nThe results of the experiment are summarized in the table below.\n\nFrom the results, we observe the following:\n- The accuracy of the **weak classifier** improves progressively as the number of blending steps increases, but drops sharply upon performing information injection.\n- The accuracy of the **strong classifier**, in contrast, is initially lower than that of the weak classifier during the blending steps. However, it surges from approximately **73**% to **93**% after only 10 injection steps and ultimately reaches **97**%.\n- The **MSE** increases consistently as the number of blending and injection steps grows.\n\nThese findings demonstrate that:\n- The proposed method is effective on tabular data, provided the generative model is capable of capturing the underlying causal relationships.\n- The method is robust to imbalanced datasets (the Adult dataset contains 80% low-income and 20% high-income labels) and can handle noisy data (e.g., human annotation errors), as evidenced by the performance of the weak classifier trained with noisy labels.\n- The reliable (prev. robust) counterfactual examples effectively resolve the misalignment between the learned and true decision boundaries, under the assumption that the strong classifier better approximates the true decision boundary than the weak classifier.\n\n\n| Metric        | N_b=30         | N_b=60         | N_b=90         | N_i=10         | N_i=20         | N_i=30         |\n|---------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|-------------------------|\n| **ACC↑**     | 0.7607 ± 0.0027         | 0.8355 ± 0.0026         | **0.8503 ± 0.0026**     | 0.6984 ± 0.0016         | 0.6686 ± 0.0020         | 0.6389 ± 0.0013         |\n| **AUC↑**     | 0.8522 ± 0.0014         | 0.8989 ± 0.0017         | **0.9082 ± 0.0016**     | 0.8070 ± 0.0011         | 0.7905 ± 0.0015         | 0.7713 ± 0.0011         |\n| **ACC_r↑** | 0.7232 ± 0.0017         | 0.7258 ± 0.0017         | 0.7276 ± 0.0019         | 0.9328 ± 0.0016         | 0.9655 ± 0.0009         | **0.9692 ± 0.0009**     |\n| **AUC_r↑** | 0.7841 ± 0.0019         | 0.7860 ± 0.0018         | 0.7874 ± 0.0021         | 0.9466 ± 0.0019         | 0.9747 ± 0.0011         | **0.9783 ± 0.0011**     |\n| **MSE↓**     | **0.0537 ± 0.0001**     | 0.0544 ± 0.0001         | 0.0546 ± 0.0001         | 0.0688 ± 0.0002         | 0.0717 ± 0.0002         | 0.0731 ± 0.0002         |\n\n*Metrics with mean ± standard error for 5 runs. Bold values represent the best-performing configuration for each metric. The first three columns (N_b) indicate performance changes across blending steps, while the last three columns (N_i) indicate performance changes during injection steps, which continue from the last blending step. The step sizes are $\\gamma_b = 0.1$, $\\gamma_{i,lift}=1$, and $\\gamma_{i,land}=1.1$. ACC and AUC are evaluated using the weak classifier, whereas ACC_r and AUC_r are evaluated using the strong classifier.*\n\n[1] Becker, B. & Kohavi, R. (1996). Adult [Dataset]. UCI Machine Learning Repository.\n\n[2] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, Kalyan Veeramachaneni. Modeling Tabular data using Conditional GAN. NeurIPS, 2019."}]

Author rebuttal:

Meta-review: a) summary: The paper introduces a novel method for generating counterfactual explanations addressing the shortcomings of optimization CE methods (vanishing gradients) and other conditional generative methods (discontinuity of latent space). It modyfies the initial I-CFM formulation from [42] to be class conditioned and use it to remove and re-introduce class relevant information to the data through blending and optionally injection leaps (lifting and landing transports). The experiments explore the effectiveness of the method both quantiatively and qualitatively and show superior performance with respect to baselines.

(b) strengths: Technically strong paper with solid theoretical background. Convincing experimental evaluation. Flexibility of the method to operate even over non-differentiable classifiers including human-in-the-loop systems.

(c) weaknesses: Clarity of some statements and used terminology, high computational costs and bad scalability to high-dimensional spaces.

(d) reasons for accept: Explainability is an important topic with CE an increasingly popular approach. The paper takes a fresh look that is theoretically well grounded while addressing clearly stated shortcomings of previous methods. The experimental evaluation looks convincing. 

(e) review discussion: The reviewers appreciated the novelty and good theoretical grounding of the method with clear justification for the selected approach. They expressed some concerns about clarity of some statements/terms (e.g. robustness of CE, structural/flattened representation), missing support for some claims (e.g. vanishing gradients), computational costs and scalability, practical implementation details and hyperparameter setting. The authors addressed most of these very clearly and open within the rebuttal often by referring to existing appendix, added an experiment on tabular data, and acknowledged some existing limitations (mainly scalability to high-dimensional spaces) left for future work. The reviewers generally appreciated the answers and regarded the discussion as useful with positive outcomes.
Decision: Accept (poster)