A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimation
Authors: Wei Chen
Jiacheng Li
Shigui Li
Zhiqi Lin
Junmei Yang
John Paisley
Delu Zeng
Venue: ICLR 2026 Poster (2026)
Keywords: Density ratio estimation
minimum variance path principle
path optimization
Kumaraswamy mixture model
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: MVP analytically minimizes the path-dependent variance in score-based density ratio estimation, learning stable, accurate trajectories without heuristic path selection.

Abstract: Score-based methods are powerful  across machine learning, but they face a paradox: theoretically path-independent, yet practically path-dependent.
	We resolve this by proving that practical training objectives differ from the ideal, ground-truth objective by a crucial, overlooked term: the path variance of the score function.
	We propose the MVP (**M**imum **V**ariance **P**ath) Principle to minimize this path variance. 
	Our key contribution is deriving a closed-form expression for the variance, making optimization tractable. 
	By parameterizing the path with a flexible Kumaraswamy Mixture Model, our method learns data-adaptive, low-variance paths without heuristic manual selection. 
	This principled optimization of the complete objective yields more accurate and stable estimators, establishing new state-of-the-art results on challenging benchmarks and providing a general framework for optimizing score-based interpolation.

Published as a conference paper at ICLR 2026
A MINIMUM VARIANCE PATH PRINCIPLE FOR ACCU- RATE AND STABLE SCORE-BASED DENSITY RATIO Es- TIMATION
Wei Chen', Jiacheng Li,, Shigui Li,, Zhiqi Lin', Junmei Yang", John Paisley,, Delu Zeng25*
I School of Mathematics, South China University of Technology
School of Electronic and Information Engincering, South China University of Technology
Sccooo of Computer Science and Engineering, South China University of Technology
*Department of Electrical Engineering, Columbia University
'Department of Electrical and Computer Engineering, University of Waterloo
(mawei chen, ee i achnng 1i lishi gui, 20231 1 9891 92) @ma scut. edu cn, y junmei, dlzeng) ) scut. edu. en, jpa eeycol umbia edu
ABSTRACT
Score-based methods powerful across machine learning, but they face a paradox: theoretically path-independent, yet practically path-dependent. We resolve this by proving that practical training objectives differ from the ideal, ground-truth objective by a crucial, overlooked term: the path variance of the score function. We propose the MVP (Mimum Variance Path) Principle to minimize this path variance. Our key contribution is deriving a closed-form expression for the variance, making optimization tractable. By parameterizing the path with flexible Kumaraswamy Mixture Model, our method learns data-adaptive, low-variance paths without heuristic manual selection. This principled optimization of the complete objective yields more accurate and stable estimators, establishing new state-of-the-an results on challenging benchmarks and providing a general framework for optimizing score-based interpolation.
INTRODUCTION
Density Ratio Estimation (DRE) is fundamental machine learning task, underpinning applications such as f-divergence estimation (Chen et al., 2025c), Riesz regression (Kato, 2025b;c; 2026), aligning large language models (Higuchi & Suzuki, 2025), causal inference (Wang al., 2025), domain adaptation (Wang et al., 2023), and likelihood-free inference (Cobb et al., 2024). Classical methods problem where two densities exhibit low overlap or large
often fail under the "density-chasm discrepancy (Rhodes et al., 2020). A key advance is the rise of continuous, score-base based methods (Choi et al., 2022; et al., 2025; Chen et al., 2025c), which express the log-density ratio as a path integral of time-dependent score function along smooth interpolation between two densities.
Theoretically, these methods are path-invariant: integrating the score along any smooth path recovers the exact target (Choi et al., 2022). However, a practical paradox emerges in practice: with neural network approximation, performance becomes significantly path- aath-dependent, as shown in our prelimi- nary experiments (see Fig. la). This sensitivity mirrors challenges in broader probabilistic inference and generative modeling (Sahoo et al., 2024; Strasman et al., 2025).
We trace this paradox subtle but crucial gap between the ideal objective, which minimizes the true mean- squured error to the ground- truth score, and the tractable score- score-matching matching loss used in practice. The two objectives differ by path-dependent term that is typically ignored path is fixed (Choi et al., 2022). Our analysis reveals that this term is, in fact, the dominant factor driving performance differences across paths, akin the impact of noise schedules in generative modeling (Kingma et al., 2021; Guo & Schwing, 2025; Strasman et al., 2025).
constant when the
Our main contribution is close this gap. We prove that the missing term is precisely the path variance of the ground-truth score, quantity that must be minimized and cannot be treated as
"Corresooniigg author.

---

Published as a conference paper at ICLR 2026
@ Enwar
30 Folmeer
Path Variance LARGE 6-0 RUSE
Path Variance SMALL EOC.OORR End
[R] digor omeoic
(14 XP
(a) Estimates vary across path settings.
(b) Path with large variance.
(c) Path with small variance.
Figure I: (a) Preliminary experiments under various path settings. The left image denotes the ground
truth. (b-c) Probability paths between po and p with large and small variance, respectively.
constant (illustrated in Figs. lb and Ic and proved in Theorem 4.2). Building on this insight, we propose MVP (Mimum Variance Path) Principle, the first framework that directly optimize the complete, ideal objective. Our contributions are threefold:
We formally identify path variance as the missing term bridging practical and ideal objectives, and derive closed-form, tractable expressions for it under two widely used interpolants.
We parameterize the path with a flexible Kumaraswamy Mixture Model (KMM) and learn an optimal, data-adaptive path by directly minimizing the analytical path variance.
3. Our approach resolves the empirical paradox described above and removes the need for heuristic path selection, achieving state-of-ther of-the-att performance challenging benchmarks.
RELATED WORKS
Score-Based Density Ratio Estimation. DRE is a fundamental statistical problem with a rich history (Sugiyama al., 2012). Classical methods, such those based statistical divergences, like KLIEP (Sugiyama et al., 2008) or noise-contrastive principles (Gutmann & Hyvärinen, 2010; 2012), often falter when the supports of the two distributions have minimal overlap. which is challenge known as the density-chasm problem (Liu et al., 2017). A recent paradigm shift addresses this via divide-and-conquer strategies that bridge Po and P1- Starting with discrete methods like TRE (Rhodes et al., 2020), which decomposes the density ratio into product of simpler intermediate ratios. It was later extended to the continuous setting in DRE-cc (Choi al., 2022), linking DRE to score- based generative modeling through time-dependent score along a smooth path. Later work improved this framework, for example, via the dequantified diffusion bridge interpolant (DDBI) for better stability and weaker assumptions (Chen al., 2025c). Closely related is the work conditional probability paths (Yu et al., 2025), providing theoretical guarantees for DRE by bounding the estimation error in terms of path smoothness, which is closely related to our Lemma 4.1. Other notable advances include improving robustness to outliers (Nagumo & Fujisawa, 2024), mitigating overfitting (Kato Teshima, 2021), mapping distributions into simpler latent spaces (Choi et al., 2021; Wang et al., 2025), building direct DRE via low-variance secant alignment identity (Chen et al., 2025b) or Riesz regression (Kato, 2025a), and implicitly defining a probability path through endpoint densities (Kimura & Bondell, 2025; Yu et al., 2025). However, nearly all these score-bascd methods rely on fixed, hand-crafted interpolation paths. The crucial question of how to design an optimal, data-dependent path tailored to specific DRE task remains largely unexplored.
The Importance of Paths in Probabilistic Inference and Generative Modeling. A parallel evolution in the broader field of probabilistic inference (Paisley & Carin, 2009; Paisley al., 2012a; Xu & Zeng, 2024) and generative modeling (Shen et al., 2025; Chen et al., 2025a; Xu et al., 2025a) highlights the path (a.k.a. noise) schedule
critical design choice for capturing complex data structures (Li et al., 2023; 2025c; Lin et al., 2025), In conformal prediction, nested prediction sets are often built upon paths that guarantee calibrated uncertainty under distribution shift (Luo et al., 2026a;b; Bao et al., 2025a;b). In time-series forecasting, temporal paths and noise schedules directly govern how uncertainty propagates over multi-step horizons and how long-range dependencies are captured (Li et al., 2024a; 2025a;b). Furthermore, Gaussian processes inherently model functions as stochastic paths (Xu et al., 2025c; 2024; 2025b), with modern variants increasingly learning adaptive kemells and path densities handle non-stationarity and high-dimensional manifolds.
Despite these advances, early score-based generative models like DDPM (Ho et al., 2020) employed fixed heuristies such as the variance- frriacce-preeeving (VP) schedule (Song et al., 2021b) or the cosine schedule (Nichol & Dhariwal, 2021). It quickly became evident that these choices were non-trivial, with theoretical analyses showing how schedules directly impact model stability and performance
2

---

Published as a conference paper at ICLR 2026
(Song et al., 202 Sahoo al., 2024; Strasman et al., 2025). This realization spurred shift towards learnable, dettaddeenndntt paths, a trend prominent across the field, from variational diffusion models that learn schedules by maximizing variational bound (Kingma et al., 2021: Guo & Schwing, 2025), to continuous normalizing flows that optimize the trajectory of a governing ODE (Zhang et al., 2024). A highly relevant advancement is neural diffusion models (Bartosh et al., 2024), which generalize the forward process by enabling learnable, non-linear, efficient (Li et al., 2024b), and time-dependent transformations of the data. While powerful for generative tasks, this line of work primarily optimizes the variational bound (Hoffman et al., 2013; Paisley et al., 2012b) for density estimation. Furthermore, the problem of finding an optimal path to reduce the estimation error of density ratios has been analyzed in the context of annealing, demonstrating that even basic paths offer provable benefits over no annealing (Chehab et al., 2023). A complementary perspective from optimal transport (OT) has been used to find geometrically optimal paths, as seen in rectified flow (Liu et al., 2023) and other OT-iispired models (Du et al., 2022; Shaul et al., 2023; Albergo et al., 2023). This body of work establishes a clear principle: for optimal performance in coore-based models, the path should not be fixed but adapted to the data. While generative modeling has adopted this principle, DRE still has not.
Bridging the Gap: Our Contribution. We propose the first DRE framework that goes beyond fixed or heuristic paths by explicitly learning an optimal path through theoretically grounded objective. Our analysis shows that this objective (the path variance) is the missing term in the ideal sctre-matching loss that prior DRE methods have overlooked. By parameterizing the path with flexible KMM and optimizing it with its path variance, we convert the intractable functional search into a tractable, low-dimensional optimization. Our method, MVP, thus adapts the path schedule to the data distributions, achieving more accurate density ratio estimation.
3 BACKGROUND
Let Xo Po and X1 P1 be random variables in Id with p1 < Po- The goal of density ratio estimation (DRE) is to estimate r(z) Pa
(_(#) for a given sample æ using i.i.d. samples from po and
Pt. Conventional DRE methods often deteriorate when Pa and p1 are non-overlapping or significantly different (Gutmann & Hyvärinen, 2012; Liu et al., 2017).
Interpolants. To address this problem. Rhodes et al. (2020) proposed TRE, factorizing the direct ratio r into M ratios via intermediate variables Xm/M ~ Pm/M defined by
Xm/M = vI {mm/M)>xo + (m/M))x,,
0, 1, ...., M,
(1)
where B 10, 1] + [0, 1] is a monotone increasing schedule with ((0) = 0, 3() = 1. As M- + co, this discrete scheme converges the deterministic interpolant (DI) (Choi al., 2022)
x a(t)xo + =(t)x), Xo ~ Po, X1 ~ X1, ~ t.,t E 1].
(2)
where
is path schedule satisfying the boundary conditions (0) 3(1
1, (1) ((0) = O. Common choices satisfy a(t) + B(t)
or a(t)² B(t) = 1. Conditioning
on x1 = I and assuming Xo follows a standard Gaussian, the transition kernel admits a closed form
pa( =1)
(()):((()),,,))) (t)) Ia) (Vincent, 2011; Yu et al., 2025).
While the the Gaussian assumption on Xn Po(x:) is convenient, it may not hold for more general applications such as mutual information estimation. To relax this requirement and improve the stability of DRE-co, Chen et al. (2025c) extended DI to the dequantified diffusion bridge interpolant (DDBI) by adding noise term to the interpolation, creating a bridge process of the form
X = al)xx + (()xx + v((1 - l)7² + (a(1)2 + 3(1)))ez,
where z ~ ~N, Ia) is Gaussian noise, 7 2 0 is the noise factor, and 2 0 is a small constant. In this case, the closed-form transition kernel conditioned on æn and æ1 becomes p.(x: Zo, x1) = N (#r; (())a) + (() (1(1 - 1)>² + (a(1)² + ((t)))e)[a) Ta) (Yu et al., 2025; Chen et al., 2025c).
Density Ratio Estimation. For the discrete path in Eq. (1), the log-density ratio for given sample xe is expressed as a telescoping summation of intermediate log-density ratios,
log r (œ) = log (a:) - log = Σ log P(m+ 1)/ad (x)
M-1
(4)
Pm/M M (x)
=
which is Riemann sum approximation of the continuous representation (Choi et al., 2022).
3

---

Published as a conference paper at ICLR 2026
rx)) = log (a) log p2() / 3 lo & S. (x) t)dt.
(5)
Here, (x) t) - O log (x) is the time score, and s (a. 0e (()) is the data score. Note that (t) and (a) are labels indicating "time" and "data", not derivatives.
(t) To approximate s((), we train a time score model Eg by minimizing the ideal but intractable time
score matching objective, (0s (2)) (x. 1) =0 (xr, 1) Since s(0) is unknown, this loss is replaced by tractable alternative derived via integration by parts, known the sliced time score matching (STSM) (Choi et al., 2022) objective,
2
CSTSM (0) = 2 (m1))
(4) (x, 5z) (f) $e (xo. 0) 08 (x1, 1) -p((t)nit))) 22
(t)
(6)
where the first term enforces boundary conditions, 0,s8 is the time derivative, and p(t) a timestep
distribution over (0. 1) (see Appendix Sec. B.2.5 for details). This objective fully computable,
(t) and its derivative, and differs from LrSM by path-dependent constant. The depending only on 80
log-density ratio estimated by log f(x) So s0 (x, t).t.
MINIMUM PATH VARIANCE PRINCIPLE
Our proposed method, MVP, is designed to resolve an important discrepancy between the theory and practice of score-based DRE. In this section, we detail the core components of our method: the derivation the analytical path variance and our flexible KMM-based path parameterization.
4.1 THE PATH VARIANCE: AN OVERLOOKED TERM IN THE OBJECTIVE
Our analysis begins by relating the expected density ratio estimation error to the score-matching objective. We consider the estimation error,
A (xr) = log r(=) log f(x)? x ~ Pio
where the true and estimated log-density ratios are given by logr(x) = fo sxx t)dt and log f(x) Jo =g (ar, t)dt. The lemma below provides an explicit upper bound for this error.
Lemma 4.1. Let {pt 1e be a probability path connecting Po and P1. Under Assumption A.1, the expected estimation error admits the following upper bound:
ELL (
(S((E (app ())) m dt
Ep [A(x)] S
(8)
TuL
where e(x,, 1) = (2, (x, 1) is the error of the score model, the exponents satisfy and L is the Lipschitz constant in Assumption A.
1 n
1.
Relationship to Existing Error Bounds. Our derivation in Lemma 4.1 is conceptually aligned with prior theoretical analyses of path-based density estimation. For example, Yu et al. (2025) also obtain an error bound by controlling the path integral via smoothness assumptions (e.g., Lipschitz constants), highlighting the central role of path regularity. However, important differences remain. Their bound expressed in terms of statistical divergence between the true and estimated density ratios, whereas our Lemma 4. provides direct upper bound on the expected squared error of the log-ratio estimate. Moreover, the goals diverge: Yu al. (2025) leverage their bound motivate the LCTSM loss, while our analysis uses the general bound to decompose the total theoretical error and justify explicitly optimizing the previously unaddressed path variance term V.
The proof is presented in Sec. A.2. The case (n oc, 1) is particularly insightful, as it simplifies the bound be proportional the integrated squared error, So [e((m)<((x; ())de Ep(()x(()) (a) (e 2, which is exactly the score-matching loss, (O). This reveals that minimizing the TSM loss, LTSM (), is essential for achieving minimum estimation error.
We adopt the term "sliced" because the objective, derived through integration by parts, involves projecting onto random directions, technique known Sliced Score Matching (Song et al., 2020) in the generative modeling literature. Choi al. (2022) simply call this objective Time Score Matching.
4

---

Published as a conference paper at ICLR 2026
However, established in Choi et al. (2022), one cannot directly optimize LISM. Instead, a tractable objective like LSISM is used. The connection between them is an exact algebraic identity:
LISM () = LST + Ey.(t)) |0, lo
This identity reveals a crucial gap: Minimizing the tractable loss LSTSM does not account for the second term, Jo' Ep, (z) |å: log dtt. The main contribution of our work is to identify this overlooked term and propose a method to minimize it. We first present the following theorem.
Theorem 4.2 (Minimum Path Variance Principle), In the loss decomposition (Eq. (9)), the second- moment term is exactly the path variance V - So (a log pt (a)))dt of a probability path (prheeo,1)) Consequently, the expected estimation error of model sg
(t) is bounded by
E (p)(2) [a()) et (SSISM(8) + L'' Varp.tm) (O, logp((z))dtt
(10)
This establishes the Minimum Variance Path (MVP) principle: Minimizing the overall error bound requires jointly minimizing the tractable model loss LSTSM and the path variance V.
See Appendix Sec. A.3 for proof. This theorem reveals that path variance is not an auxiliary regularizer but the overlooked component of the ideal objective. Conventional approaches fix the path priori, treating V constant and ignoring its effect. Our key insight is that, by learning the path itself, we can explicitly minimize this term, not as a guarantee of a tighter bound, but as a heuristic that empirically improves estimation accuracy and stability.
Remark on the Lipschitz Constant I, in Theorem 4.2. Our principle of minimizing the path variance V also serves to indirectly control the Lipschitz constant that appears as a multiplicative factor in the error bound. We discuss the empirical nature of this relationship in Remark A.3.
Remark on Stabilizing LSTSM in Theorem 4.2. The score matching loss LSTSM is optimized with respect to at every training step. but its empirical estimate depends on the current path Pe through samples 2; (z)) To reduce the sensitivity of this estimate to aath-induced score instability, we adopt an alternating scheme: after each update of the path parameters (see Sec, 4.2) by minimizing the path variance V. we refresh the variance-based time sampler p(t) xc 1/(Varp, (Ō, log Pt) + +) (see Sec. B.2.5). This shifts sampling toward time steps with low score variance, reducing gradient noise and improving the reliability of the stochastic estimator of LSTSM- Although this procedure does not minimize LSISM with respect to the path, it stabilizes its optimization by controlling its variance dependence on pt. Thus, minimizing V indirectly facilitates a tighter overall error bound by ensuring stable and effective optimization of the model loss.
Closed-form Path Variance. Directly minimizing V is nontrivial since it depends on the unknown time score (*) (z. t) 0, log (x). We therefore derive analytical, computable forms of V that depend only on path coefficients and data distribution moments. Closed-form results obtained for both the DI (Eq. (2)) and the DDBI (Eq. (3)) interpolants, summarized in the following proposition.
Proposition 4.3. The path variance for the DI and DDBI interpolants, which depends on the path schedules (a, 3) = {ax((), (())) can be expressed in the following analyticul forms:
1. For the DI in Eq. (2) with Gaussian prior (o( = Nw; Ia):
Vorla, 31 39-1 --
[dá(())
{())
E (mn)) [| dt.
(11)
all)2
2. For the DDBI in Eq. (3) with noise schedule of = t(1 11) + (a(())² B(1)))e:
| 3(1)as 1
VDDBi (a, 8] 81 1/'
d (o))2 2 (of)2
E (po((s))p
+
dt,
(12)
of
The proofs for these results are provided in Appendix Sec. A.4. These analytical forms are the key technical contribution that enables the practical implementation of our Minimum Path Variance principle, as they provide direct, computable target for optimizing the path schedules.
5

---

Published as a conference paper at ICLR 2026
4.2 MINIMIZING THE PATH VARIANCE VIA KMM PARAMETERIZATION
The analytical expressions in Proposition 4.3 transform the problem of minimizing the path variance into a functional optimization problem over the space of valid path schedules (a, B). To solve this tractably. we introduce a flexible, low-dimensional parameterization for the path that satisfies the necessary boundary conditions and monotonicity constraints by construction. This transforms an infinite-dimensional functional optimization into low-dimensional parametric one, offering superior stability and tractability.
A Function Class with Built-in Constraints. A valid schedule (E(()) must satisfy the boundary conditions a()) = 1, (1) = 0 and be monotonically decreasing. Instead of enforcing these properties with external constraints, we select a function class that satisfies them inherently. We achieve this by defining based on the cumulative distribution function (CDF) of probability distribution defined on [o, 1]. A CDF, F(t). is by definition a monotonically increasing function from 0 to 1. Therefore, the transformation 1 F(t) yields a function that is monotonically decreasing from to O, matching our requirements for {a(t)
The Kumaraswamy Mixture Model (KMM). While a single distribution on [0, 1] (e.g., Beta or Kumaraswamy) could define valid path, its expressive power is limited, being unimodal. To construct a significantly more flexible function class, we employ a Kumaraswamy Mixture Model (KMM) (Kumaraswamy, 1980; Sindhu et al., 2015). The Kumaraswamy distribution is chosen over the more common Beta distribution for its simple and closed-form CDF, which allows for efficient and stable computation of derivatives. By taking a weighted sum of several Kumaraswamy components, we can create complex, multimodal probability density shapes, which in turn allows the path schedule have varying rates of change optimally minimize variance.
Formally, we define a K K-component KMM, parameterized by Φ = {wk, ak, by J via its probabil- ity density function (PDF) and CDF,
K
K
PΦ
Σ k=1
wk be,, (t Σ k=1
Wk [1 (1- -1--10t00 )0
(13)
where KS(I; 0, b) = abt 1-)001 is the Kumaraswamy PDF, wn are mixture weights satisfying Σ wh L, and ags, bk >> 0 are shape parameters.
Final Construction and Guaranteed Properties. We now define our learnable path schedule (1) using the KMM's CDF. The derivative (1) is simply the negative of the KMM's PDF,
(t) = 1 (t),), des()) (t).
(14)
This construction guarantees the following desired properties without external enforcement:
Boundary Conditions: (0) = 1 (0) 1 and (1 = 1 - (L) = 0.
Monotonicity: (1) is guaranteed to be monotonically decreasing since (1) 2 0.
Smoothness: (t) is infinitely differentiable on (0, 1), ensuring smooth probability flow.
The coupling schedule (t) and its derivative follow from the constraint selected below.
Bt(() = 1 - (t) B((() = (t) Spherical: as (t) (t) =1 (t(()= v1 11-apa as((),
Affine: a (1) + (t)
(15)
Bt(() V
(1)
(16)
(x(1)
Substituting into the path variance expressions in Proposition 4.3, we recast the functional optimiza- tion miite-dimensional problem over the KMM parameters Φ. namely minimizing Vla Ag).
Parameterization for Unconstrained Optimization. To perform gradient-based optimization for &. we must handle the constraints on the KMM parameters (wy > 0, Σ wk = 1, ag > b, > 0). We use a standard reparameterization technique, optimizing a set of unconstrained latent variables & = (wkk, az. b&) which are mapped to valid KMM parameters via softmax and softplus transformations,
wh
k 1==1 e0n
a = softplus(åg) = log(1 + cax), by = softplus(x.).
(17)

---

Published as a conference paper at ICLR 2026
The path parameters & and score model parame- ters θ are jointly optimized via the total objective Lioal CSTSM + >zV(a S3] (described in Appendix Sec. B). Details of path optimization given in Algorithm 2, and a PyTTorch implemen- tation is provided in Algorithm (see appendix). After training, MVP uses the same inference as con- ventional DRE methods shown in Algorithm 1.
Algorithm DRE with MVP: Inference.
Require: Sample 2, optimal parameters 0
and Φ², integration steps I.
1: 4 i/1,i i
{0, 11.,,.,,/ 7}.
2: 8/ 50 (x, (,),
(0) (0)
(0.
13.
3: log f(x) t
+ 1 8:
Ensure: f(x) cxp f(x)).
Algorithm 2 Unconstrained Optimization of the MVP Path.
Require: Num components K, initial latent params & = fdēg, āz. by 1 learning rate 1).
1: Sample a batch of N time steps {t,),= ~ p(t) (described in Appendix See. B.2.5).
2: Transform & to constrained KMM params & = {we, ak; by KKK using Eq. (17).
and their derivatives using Eqs. (14) to (16). 4: Approximate the path variance V(a So) via Monte Carlo using either Eq. (11) or Eq. (12).
3: Compute path schedules (44)
5: Compute gradients V Vaa. [6] via automatic differentiation.
6: Update latent params: e + Opt imi zerStep(d, V V 3p).n).
Ensure: Optimized latent parameters o² defining the path function as- and Sgg.
EXPERIMENTS
We conduct a comprehensive set of experiments to validate the effectiveness of our Minimum Path Variance principle and demonstrate the superior performance of our proposed method, MVP.
5.1 BASELINE SETTING AND EXPERIMENTAL. SETUP
Our evaluation is designed to cover wide range of tasks and distribution types. We follow the experimental protocol established by Choi et al. (2022); Chen et al. (2025c), employing two primary interpolant structures based on the task requirements. For standard density estimation tasks where Po can be assumed to be a simple distribution, we use the DI framework defined in Eq. (2). For more general tasks such as f-divergence and mutual information estimation, where this assumption is restrictive, we use the more robust DDBI framework from Eq. (3).
To evaluate our approach, we benchmark against a comprehensive suite of common, fixed path schedules representing the current standard in the DRE and generative modeling literatures. These baselines include the Linear (Albergo et al., 2023), VP (Ho et al., 2020; Song et al., 2021b), Cosine (Nichol & Dhariwal, 2021), Föllmer (Föllmer, 2005; Chen et al., 2024), and Trigonometric (Albergo et al., 2023) paths, with detailed formulations provided in Sec. B. In contrast to these pre-defined paths, our MVP path schedule utilizes a learnable path parameterized by KMM, as discussed in Sec. 4.2 above. The path is optimized by directly minimizing the analytical path variance expressions from Proposition 4.3, allowing to adapt to the specific data distributions of given task.
5.2 ABLATION STUDIES.
KMM Components. We first conduct an ablation study on the sensitivity of MVP the number of components K in the KMM path parameterization. Experiments are done under the most challenging settings for both tasks: mutual information (MI) estimation
Table Ablation study on the number of KMM components (K) on the most chal- lenging mutual information (MI) estima- tion (d 160) and density estimation (BSDS300) tasks. We use mean squared emor (MSE) negative log-likelihood (NLL). Lower is better. We recommend = 5.
= 160 and density estimation on BSDS300. As shown in Tab. 1, a single component (K = 1) is too restrictive and performs poorly, high- lighting the need for path flexibility. Performance improves markedly at K = 2 and reaches its peak around K = 5. Increasing the number of compo- nents further (K = 8) brings no benefit and may slightly hurt performance due to overfitting the path variance objective. Thus, MVP empirically remains
d 160 (MI Est. MI
40)
BSDS300 NLL
K
MSEJ
40.60 1 0.66
1.32
-48.51 1 0.65
40.55
1 0.60
1.09
-145.41 0.31
5
40.59
+ 0.55
1.02
-143.97 = 0.22
8
41.16 0.63
2.32
-143.60 + 0.45
robust, with K 5]. In practice, we set K 5, which achieves the best trade-off.
7

---

Published as a conference paper at ICLR 2026
Path Constraints. We treat the choice between the affine constraint (a(t) + B(t) = 1) and the spherical constraint (a(t)² + B(t)² = 1) as a hyperparameter. The ablation results reveal a clear, task-dependent pattem that provides practical guidance: 1) For density estimation tabular data (see Tab. 4), the affine constraint performs best on lower-dimensional cases (POWER, GAS), whereas the spherical constraint is superior on higher-dimensional and more complex cases (HEPMASS, MINIBOONE, BSDS300). This indicates that the optimal path geometry is closely linked to the intrinsic complexity of the data manifold. 2) For MI estimation, the result is different. On Gaussian tasks with high discrepancy but simple geometry (see Tab. 3), the affine constraint consistently delivers better accuracy and stability. By contrast, on more challenging MI tasks with pathological geometries, we found that the spherical constraint proves more robust, particularly for datasets with discontinuities (Additive Noise) or complex dependencies (Gamna-EXxonential) (See Tab. in appendix for these additional experiments.) These findings validate our treatment of the path constraint as a data-dependent hyperparameter rather than a fixed choice. In practice, the affine constraint is well-suited for distributions with high discrepancy but simple geometry, while the spherical constraint is preferable for complex, high-dimensional, or non-Gaussian settings.
5.3 S-DIVERGENCE ESTIMATION
Geometrically Pathological Distributions. We first test robustness on five mutual information (MI) estimation tasks that can be viewed as challenging f f-divergence problems. (For a formal MI definition, see Appendix Sec. B.4.) Following Czyž et al. (2023), these distributions exhibit concentrated densities, sharp discontinuities, heavy tails, and extreme correlations, all of which break standard estimator assumptions. Results in Tab. 2 show that fixed-path baselines degrade severely on these geometries, while MVP consistently achieves low error (see also Tab. in the appendix). On the Additive Noise task with discontinuities and the Gamma- Exxonnn-xxonentialll task with complex non-linear dependence, our method outperforms all baselines by a wide margin, confirming that path-variana variance optimization enables MVP to adapt to pathological structures effectively.
Table 2: MSE results on the Additive Noise (sharp discontinuities) and Gamma-Exponente Exponential (non- linear dependency) datasets. Across all correlation levels (top row of each sub-table), MVP achieves notably lower MSE than fixed-path baselines. Full results on 5 datasets given in Tab. 6 in appendix.
Path
Constraint
0.1
0.2
0.3
0.5
0.6
0.7
0.8
0.9
Linear
affine
0.3053
0.0541
0.0233
0.0465
0.0367
0.0714
0.0333
0.0307
0.0268
Cosine Föllmer
spherical
0. 4803
0,1086
0.0788
0.0969
0.0719
0.0867
0,1015
0.1204
0.1304
spherical
0.4731
0.1048
0.0510
0.0916
0.0864
0.1024
0,1226
0.0944
0.1115
Trigonometric
spherical
0.3615
0.0565
0.0508
0.0654
0.0596
0.0517
0.0502
0.0448
0.0603
VP
spherical
0.3455
0.0290
0.0303
0.0346
0.0210
0.0313
0.0426
0.0390
0.0123
MVP (ours)
affine
0. 0,2360
0.0199
0.0081
0,0167
0.0161
0.0216
0,0304
0.0155
0.0010
MYP (ours)
spherical
0.1016
0.0134
0.0078
0.0009
0.0138
0.0117
0.0196
0.0026
0.0029
Path
Constraint
1.0
1.1
1.2
1.4
1.5
1.6
1.7
1.8
Linear
affine
2. 2.616
0.7064
0.1686
0.1308
0.0513
0.0420
0.0051
0.01417
0.0066
Cosine
spherical
7.2466
1.0785
0.4875
0.2582
0.2802
0.3063
0.2936
0.1864
D.3438
Füllmer
spherical
0.4006
0.4275
0.6304
0.3741
0.2822
0.4662
0.2473
0.3838
0.3105
Trigonometric
spherical
10.4420
1.3841
0.2705
0,3854
0.0184
0.0081
0.1109
0.0103
0.0608
VP
spherical
2. 7080
1.3872
0.1116
0.1224
0.1320
0.1817
0.4407
0.0919
0.2507
MVP (ours)
affine
0.4785
1.5486
0.1137
0.1274
0.0915
0.0063
0.0032
0.0294
0.0069
MVP (ours)
spherical
0.2721
0.2304
0.0917
0.0434
0.0075
0.0187
0.0091
0.0030
0.0004
Hiigh-dimensional & High-discrepancy Distributions. We further evaluate MI estimation in a high-dimensional, high-discrepancy setting, designed to induce the "density-chasm" problem in DRE (Rhodes et al., 2020). Performance is measured by MI accuracy and MSE against the true value. As shown in Tab. 3, most fixed-path baselines collapse dimension d and MI increase. For example, Föllmer and Cosine paths perform reasonably at low d but diverge at = 160, MI= 40. In contrast, MVP with affine constraint maintains a low MSE of 1.02, outperforming the next best baseline by a large margin. These results highlight that adaptive path learning under our Minmimum Path Variance principle reliably bridges the density chasm in settings where heuristic fixed paths fail.
5.4 DENSITY ESTIMATION
To assess our model's ability to capture the true data distribution Pdata. we perform density estimation by expressing complex, possibly intractable data distribution Paam in terms of a simple base distribution Po (e.g., (0, Ia)) using the density ratio r r(x) Paaa Pama(ar)// (m)/po(<²). This yields log Paata (æ:)
8

---

Published as a conference paper at ICLR 2026
Table 3: Mutual information estimation under high-discrepancy settings (MI E {10, 20, 30, 403 nats). We report the estimated mutual information (mean + std) and MSE across different path settings and constraint types. Bolded MSE values indicate the best performance for each dimension. Our MVP path demonstrates superior performance in most high-discrepaney settings.
d 40 (MI 10) d 80 (MI
20) d 120 (MI 30) d 160 (MI 40)
Path
Constraint
Est. MI
MSE
Est. MI
MSE
Est. MI
MSE
Est. MI
MSE
Linear
Affine
9.7920.17
0.13
20.4420.31
0.44
30.1820.50
0.63
37.07
9.17
Cosine
Spherical
9.62+0.28
0.34
19.21=0.47
1.19
28. 28.60=0.71
2.97
33.95+0.71
37.91
Föllmer
Spherical
9.4110.22
0.54
17.94.0.46
4.75
25.31:0.70
23.06
31.5110.60
72.98
Trigonometric
Spherical
10.12#0.24
0.13
19.8820.47
0.47
33.37=0.74
12.41
41.74++.70
4.18
VP
Spherical
10.02..0.4
0.28
20.08
0.66
26. 10.57
15.53
33.47
43.60
MVP (ours)
Spherical
9.81
0.16
20.43
0.62
28.88 10.69
2.35
77.22.1072
8.79
MVP (ours)
Affine
10.0840.20
0.10
19.8020.36
0.34
30.22
0.58
40.5922.65
1.02
log r(x) + log (a) so that density estimation reduces to approximating r(x), and the log- likelihood estimated log Pdata (2:) log f(x:) log Pa()).
Structured and Multi-modal Datasets. We further evaluate MVP on six challenging datasets: circles, rings, pinwheel, 2spirals, checkerboard, and tree (Grathwohl et al., 2018; Karras et al., 2024), which feature disconnected components, intricate spirals, and discontinu- ous densities. As shown in Fig. 2, MVP yields sharper and more accurate density estimates on checkerboard and tree, and matches the best baseline (VP path) on the remaining datasets.
Figure 2: Density estimation on checkerboard and tree datasets. MVP successfully learns data- adaptive path tailored to the specific manifold of each dataset. Full results in Fig. 4 of appendix.
Full results are provided in Fig. 4 in the appendix. These results demonstrate the significant adapt- ability of MVP and help confirm that minimizing path variance enables MVP to adapt effectively to diverse data geometries without manual path selection.
Real-world Tabular Datasets. Finally, we evaluate MVP on five widely-used tabular benchmarks (Grathwohl et al., 2018), reporting test Negative Log-Likelihood (NLL), where lower is better. As shown in Tab. 4, MVP consistently outperforms all fixed-path baselines and achieves the-art results across all datasets. Two key observations note are: 1) On POWER and GAS, the affine constraint leads the best performance, whereas the spherical constraint is more effective the higher-dimensional datasets (HEPMASS, MINIBOONE, and BSDS300). This highlights the adaptability MVP data geometry. 2) The gains over strong baselines (e.g.. VP and trigonometric paths) are substantial, especially on BSDS300 300 where MVP improves NLL by more than 10 points, underscoring the advantage of explicitly minimizing path variance. These results validate our central claim, that adaptively selecting the path via MVP yields a deta-dependent path that aligns better with the underlying structure, leading to improved density estimation.
state-of-
Table 4: Test Negative Log-Likelihood (NLL) on five tabular datasets (lower is better). Our MVP path consistently outperforms all fixed-path baselines, achieving SOTA across the benchmarks.
Path
Constraint
POWER
GAS
HEPMASS MINIBOONE
BSDS300
Linear
Affine
0.1990.04
5.00.0.06
19.641..14
18.85
-98.37 20.48
Cosine
Spherical
0.55+0.03
-1.7220.07
21.6240.05
24.36+0.17
-75.39+1.17
Föllmer
Spherical
0.52+0006
-0.45.0.07
21.57
25.66+0.34
-82.87+0.0
Trigonometric
Spherical
-0.5440.15
-4.57=0.16
19.50 19.50=+0.05
20.27+10110
-89.41
VP
Spherical
-0.0440.08
-6.02+0.13
18.0640.16
18.25+0008
-131.90=0.30
MVP (ours)
Spherical
-0.15
-6.9240.21
18.01
17.81+4009
-143.9740.22
MVP (ours)
Affine
-0.81+0.08
6.9540.10
18.7940.16
17.99+20.10
-129.73=0.20

---

Published as a conference paper at ICLR 2026
5.5 ILLUSTRATION OF OPTIMIZED PATH SCHEDULES
We visualize the optimized schedules (() and (1) across various tasks to interpret the efficacy of the MVP principle, with detailed comparisons provided in Fig. and Sec. B.6. While Theorem 4.2 establishes a theoretical link between path variance and estimation error, we emphasize that our implementation treats the minimization of V principled heuristic rather than a strict minimization of the ground-truth eπror. This analytical proxy objective serves to encourage smoother path transi- tions and indirectly control the Lipschitz constant L. As illustrated in the optimized schedules for pathological and high-dimen nigh-dimensional distributions Figs. 5 and 6, the learned paths (red) significantly deviate from fixed baselines. Specifically, MVP tends to adopt a more gradual rate change near the boundaries E {0, 1). which serves heuristic to suppress instantaneous velocity spikes in the time score that often lead to numerical instability.
Path Function Comparisen (a(t))
Path Comparison (B(21)
1.0
1.0
0.8
0.6
0.6
0.4
Linear
meer VP5DE
VPSDE
0.2
Cosire
0.2
Cos
Fallmer
Folmmer
Trigonomatr MyP (ours)
Trigonometic MVP (ours)
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Time
Time t
(a) a; spherical.
(b) 8; spherical.
Figure 3: Optimized XMM-parameterizze path schedule for high-dimensional distributions.
This data-adeptive schedule is particularly effective for navigating the density-chasm encountered in high-dimensional settings. As shown in Figs. 3 and 6, while fixed schedules like Föllmer or cosine often traverse regions of vanishingly low density and subsequently diverge, the MVP path adapts its velocity to maintain more reliable score signals. The structural diversity of the learned schedules across structured manifolds Fig. 7 and necessity of the flexible KMM parameterization. By allowing the schedule to adjust its rate of change align with the specific manifold structure, MVP ensures that the probability flow is tailored the data distribution. This alignment directly facilitates the substantial improvements in NLL. and SOTA results reported in Tab. 4.
tabular datasets Fig. 8 further underscores the
CONCLUSIONS AND FUTURE WORKS
This paper addresses the key paradox in score-based density ratio estimation (DRE) that estimator performance strongly depends on the chosen path schedule while being theoretically path-invariant. Our analysis reveals that this discrepancy arises from overlooked path variance term in training objectives. Based on this observation, we established the Minimum Path Variance (MVP) principle, arguing that this term should be explicitly minimized rather than ignored. We then proposed MVP, a framework that derives a closed-form expression for path variance and directly optimizes it. By parameterizing the path with a Kumaraswamy Mixture (KMM) Model, MVP learns data- adaptive low-variance path. This eliminates the need for heuristic path selection and ensures closer alignment with the ideal score-matching objective. Experiments demonstrate that MVP achieves SOTA performance in estimation error and stability across challenging DRE benchmarks.
Future Works. The MVP principle is general and could benefit other score-based generative models by learning optimal noise schedules to improve sample quality and stability. Beyond accuracy, future research could focus on computational efficiency, for example by leveraging optimal transport- path formulations or adopting lighter parameterizations further accelerate training. We believe that explicitly optimizing the geometry of interpolation paths will continue to afford significant advances in density ratio estimation and related score-hased methods.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported part by grants from National Natural Science Foundation of China (52539005), the fundamental research program of Guangdong. China (2023A151501 1281), I), the China Scholarship Council (202306150167), Guangdong Basic and Applied Basic Research Foundation (2420210719000666) 1900000677 Science and Technology Research Project(2220001118666)
REFERENCES
Michael Albergo, Nicholas Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint arXiiv23333.08797 2023.
Jie Bao, Nicolo Colombo, Valery Manokhin, Suqun Cao, and Rui Luo. A review and comparative analysis of univariate conformal regression methods. In Fourteenth Symposium Conformal and Probabilistic Prediction with Applications (COPA 2025), pp. 282-304. PMLR, 2025a.
Jie Bao, Chuangyin Dang, Rui Luo, Hanwei Zhang, and Zhixin Zhou. Enhancing adversarial robust- ness with conformal prediction: A framework for guaranteed model reliability. In International Conference on Machine Learning, 2025b.
Grigory Bartosh, Dmitry Vetrov, and Christian A Naesseth. Neural diffusion models. In International Conference on Machine Learning, pp. 3073-3095. PMLR, 2024.
Omar Chehab, Aapo Hyvarinen, and Andrej Risteski. Provable benefits of annealing for estimat- ing normalizing constants: Importance sampling, noise-contrastive estimation, and beyond. In Advances in Neural Information Processing Systems, volume 36, pp. 45945-45970, 2023.
Wei Chen, Shian Du, Shigui Li, Delu Zeng, and John Paisley, Entropy-informed weighting channel normalizing flow for deep generative models. Pattern Recognition. pp. 112442, 2025a.
Wei Chen, Shigui Li, Jiacheng Li, Jian Xu, Zhiqi Lin, Junmei Yang, Delu Zeng. John Paisley, and Qibin Zhao. Diffusion secant alignment for score-based density ratio estimation. arXiv preprint arXiv:2509.04852. 2025b.
Wei Chen, Shigui Li, Jiacheng Li, Junmei Yang. John Paisley, and Delu Zeng. Dequantified diffusion- schrödinger bridge for density ratio estimation. In International Conference on Machine Learning. 2025c. URL https: openreview foet/forum? id= 2vyHCOcwsw.
Yifan Chen, Mark Goldstein, Mengjian Hua, Michael Samuel Albergo, Nicholas Matthew Boffi, and Eric Vanden-Fijnden Probabilistic forecasting with stochastic interpolants and föllmer processes. In International Conference on Machine Learning, pp. 6728-6756. PMLR, 2024.
Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, volume 161, pp. 172-182. PMLR. 27-30 Jul 2021. URL
mar...mlrr...... wtt.::gsss........eeeeeee wwa.eess/v66/choiiiie wwt.ch/hccaaahhc.. html.
Kristy Choi, Chenlin Meng, Yang Song, and Stefano Ermon. Density ratio estimation via in- finitesimal classification. In Proceedings of the International Conference on Artificial Intel- ligence and Statistics, volume 151, pp. 2552-2573. PMLR, 28-30 Mar 2022. URL https: // //proceedings mlr press / /tlllllll////l/// choi22a. html.
Adam Cobb, Brian Matejek, Daniel Elenius, Anirban Roy, and Susmit Jha. Direct amortized likelihood ratio estimation. In Proceedings of the AAA! Conference on Artificial Intelligence, pp. 20362-20369, 2024.
Pawel Czyž, Frederic Grabowski, Julia Vogt, Niko Beerenwinkel, and Alexander Marx. Beyond normal: On the evaluation of mutual information estimators. Advances in Neural Information Processing Systems, 2023.
Bin Dai and David Wipf. Diagnosing and enhancing vae models. In International Conference on Learning Representations, 2020.
11

---

Published as a conference paper at ICLR 2026
Shian Du, Yihong Luo, Wei Chen, Jian Xu, and Delu Zeng. To-flow: Efficient continuous normalizing flows with temporal optimization adjoint with moving speed. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12570-12580 2022.
Hans Föllmer. An entropy approach to the time reversal of diffusion processes. Stochastic
Differential Systems: Filtering and Control, pp. 156-163, Springer, 2005.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In International Confer- ence on Learning Representations, 2018.
Pengsheng Guo and Alexander Schwing. Variational rectified flow matching. In International Conference on Machine Learning, 2025.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In International Conference on Artificial Intelligence and Statistics, 2010.
Michael U Gutmann and Aapo Hyvärinen. Noise-contrastivg estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13 (2), 2012.
Rei Higuchi and Taiji Suzuki. Direct density ratio optimization: A statistically consistent approach aligning large language models. In International Conference on Machine Learning, 2025. URL https: openrevieww net forum?ide jDdaysR6bz.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020.
Matthew Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational inference.
The Journal of Machine Learning Research, 14(1):10031137,, 2013.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with a bad version of itself. In Advances in Conference on Neural Information Processing Systems, 2024. URL https:/ https::/openreviees /pwenreveeeeeeeeeeeeeeee net forum? iid=
bg6EVPVs3s.
Masahiro Kato. Riesz regression direct density ratio estimation. arXiv preprint arX1v:25/1.04568, 2025a.
Masahiro Kato. Scorematchingriessz Auto-dml with infinitesimal classification, 2025b. URL
https: /ttp://aaavvvv/eaeeeeeeeeeeaaava wwtp:////g/bbbbb///.o//s
Masahiro Kato. Semi-supervised treatment effect estimation with unlabeled covariates via generalized 08303.
riesz regression, 2025c. URL https:::t//............tt htt.hhppe//aaall..rrraa////////s
Masahiro Kato. Riesz representer fitting under bregman divergence: A unified framework for debiased machine learning, 2026. URI. htttttt//////xiii org/ abs 077552.
Masahiro Kato and Takeshi Teshima. Non-negative Bregman divergence minimization for deep direct density ratio estimation. In International Conference Machine Learning. 2021.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
Masanari Kimura and Howard Bondell. Density ratio estimation via sampling along generalized geodesics statistical manifolds. In Proceedings of The International Conference Artificial Intelligence and Statistics, volume 258, pp. 253-261, 2025. URL https: mlr. press v258 Bat.//iimura2555 html.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems, 2021.
12

---

Published as a conference paper at ICLR 2026
Lukas Kirchdorfer, Cathrin Elich, Simon Kutsche, Heiner Stuckenschmidt, Lukas Schott, and Jan M Köhler. Analytical uncertainty-base based loss weighting in multi-task learning. In DAGM German Conference Pattern Recognition, 2024.
Ponnambalam Kumaraswamy, A generalized probability density function for double-bounded random processes. Journal of hydrology, 46(1-2):79-88) 1980.
Jiacheng Li, Wei Chen, Yican Liu, Junmei Yang, Delu Zeng, and Zhiheng Zhou. Neural ordinary differential equation networks for fintech applications using internet of things. IEEE Internet of Things Journal, 2024a.
Jiacheng Li, Wei Chen, Yican Liu, Junmei Yang. Zhiheng Zhou, and Delu Zeng. Diffinformer: diffu- sion informer model for long sequence time-series forecasting, Expert Systems with Applications, pp. 129944, 2025a.
Jiacheng Li, Wei Chen, Yican Liu, Junmei Yang. Zhiheng Zhou, and Delu Zeng. Integrating ordinary differential equations with sparse attention for power load forecasting. IEEE Transactions on Instrumentation and Measurement, 2025b.
Shigui Li, Wei Chen, and Delu Zeng. Scire-solver: Accelerating diffusion models sampling by score-integrand solver with recursive difference. arXiv preprint arXiv:2308.07780 2023.
Shigui Li, Linzhang Lu. Xing Qiu, Zhen Chen, and Delu Zeng, Tighter bound estimation for efficient biquadratic optimization over unit spheres. Journal of Global Optimization, 90(2):323-353, 2024b.
Shigui Li, Wei Chen, and Delu Zeng. EVODiff: Entropy-aware variance optimized diffusion inference. The Annual Conference on Neural Information Processing Systems, 2025c. URL https:/ wpwpreeeeeeeweeee///e net/ wtw.eeeweneeeeteeteeeeeeeeeeeeee
Zhiqi Lin, Wei Chen, Jian Xu, Delu Zeng, and Min Chen. Reciprocalla-llie: Low-light image enhancement with luminance-aware reciprocal diffusion process. Neurocomputing, pp. 131438, 2025.
Song Liu, Akiko Takeda, Taiji Suzuki, and Kenji Fukumizu. Trimmed density ratio estimation. In Advances in Neural Information Processing Systems, 2017.
Xingchao Liu, Chengyue Gong, al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023.
Rui Luo, Jie Bao, Suqun Cao, Chuangyin Dang, and Zhixin Zhou Game-theoreti defenses for adversarially robust conformal prediction. Transactions on Machine Learning Research, 2026a.
Rui Luo, Jie Bao, Xiaoyi Su, Wen Jung Li, and Suqun Cao. Enhancing image-conditional coverage in segmentation: Adaptive thresholding via differentiable miscoverage loss. In International Conference on Learning Representations, 2026b. URL https: /ttp://///e/eeeeeeeeeeee new../e////e.. foorum? iddGddAiiess WJs
Ryosuke Nagumo and Hironori Fujisawa. Density ratio estimation with doubly strong robustness. In International Conference on Machine Learning. 2024.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, 2021.
John Paisley and Lawrence Carin. Hidden markov models with stick-breaking priors. IEEE transac- tions Signal Processing, 57(10):3905-3911 2009.
John Paisley, David Blei, and Michael Jordan. btick-breaking beta processes and the poisson process. In Artificial Intelligence and Statistics, pp. 850-858. PMLR, 2012a.
John Paisley, David M Blei, and Michael I Jordan. Variational bayesian inference with stochastic search. In International Conference on Machine Learning, 2012b.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. In
Advances in Neural Information Processing Systems, 2020.
13

---

Published as a conference paper at ICLR 2026
Subham Sahoo, Aaron Gokaslan, Christopher M De Sa, and Volodymyr Kuleshov. Diffusion models with learned adaptive noise. In Advances in Neural Information Processing Systems, 2024.
Neta Shaul, Ricky TQ Chen, Maximilian Nickel, Matthew Le, and Yaron Lipman. On kinetic optimal probability paths for generative models. In International Conference on Machine Learning, pp-
30883-30907. PMLR, 2023.
Yirong Shen, Lu GAN, and Cong Ling. Information theoretic learning for diffusion models with warm start. In The Annual Conference on Neural Information Processing Systems, 2025. URL https:/ fttts:://cperrevieee net
Tabassum Naz Sindhu, Navid Feroze, and Muhammad Aslam. Bayesian analysis kumaraswamy mixture distribution under different loss functions. Journal of Statistics, 22, 2015.
Yang Song. Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Proceedings of The Uncertainty in Artificial Intelligence mlr.
Conference, volume 115, pp. 574-584. PMLR, 2020. URL ptttpp::/proceeeeee press/v115/ser song22a html.
Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. In Advances in Neural Information Processing Systems, 2021a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference Learning Representations, 2021b.
Stanislas Strasman, Antonio Ocello, Claire Boyer, Sylvain Le Corff, and Vincent Lemaire. An analysis of the noise schedule for score-based generative models. Transactions on Machine Learning Research, 2025. ISSN 2835-8850 URL https: opwnreveeww.w.ee/////p/. net/ forum?
id=B1YIPa0Fx1.
Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul Von Bünau, and Motoaki Kawanabe. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60:699-746, 2008
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the bregman divergence: unified framework of density-ratio estimation. Annals of the Institute of Statistical Mathematics, 64: 100404401044, 2012.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computa- tion, 23(7):1661-1674 2011.
Haoxuan Wang. Zhiding Yu, Yisong Yue, Animashree Anandkumar, Anqi Liu, and Junchi Yan. Learning calibrated uncertainties for domain shift: a distributionally robust learning approach. In International Joint Conference on Artificial Intelligence, 2023.
Meilin Wang, Wei Huang, Mingming Gong, and Zheng Zhang. Projection pursuit density ratio estimation. In International Conference on Machine Learning, 2025.
Max Wasserman and Gonzalo Mateos. Stabilizing the kumaraswamy distribution. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https: www.rvveeeeveeeeeeeeeeeeee.eeeee
https:://pperrevieeee n
forum? iit-idd-baii ba
Jian Xu and Delu Zeng. Sparse variational student-t processss.. In Proceedings the AAAI Conference on Artificial Intelligence, volume 38, pp. 16156-16163, 2024.
Jian Xu, Delu Zeng, and John Paisley. Sparse inducing points in deep gaussian processes: Enhancing modeling with denoising diffusion variational inference. In International Conference on Machine Learning, 2024. URL https: http///ppeeeeeeeeeeeeeeeee net wtt.//oormiiio//moooo wawidd=iinnnA. 4T4AI00pM.
Jian Xu, Shian Du, Junmei Yang, Xinghao Ding, Delu Zeng, and John Paisley. Bayesian gaussian pro- cess ODEs via double normalizing flows. In The 28th International Conference on Artificial Intelli- gence and Statistics, 2025a. URL https: : http//ppervvveeeeeeeeeeeee net. fott://frrmmmiieee d T jxD b.
14

---

Published as a conference paper at ICLR 2026
Jian Xu, Shian Du, Junmei Yang, Qianli Ma, Delu Zeng, and John Paisley. Variational learn- ing of gaussian process latent variable models through stochastic gradient annealed importance sampling. In The 41st Conference Uncertainty in Artificial Intelligence, 2025b. URL https:: / /eppnrevieeeeeeeeeeeeeeeeeeeeeee ew net / Coruumid==bTeovvee
Jian Xu, Zhiqi Lin, Min Chen, Junmei Yang, Delu Zeng, and John Paisley. Fully bayesian differential gaussian processes through stochastic differential equations. Knowledge-Basd Baaedd Systems, 314: 113187, 2025c.
Hanlin Yu, Arto Klami, Aapo Hyvärinen, Anna Korba, and Omar Chehab. Density ratio estimation with conditional probability paths. International Conference on Machine Learning. 2025.
Dinghuai Zhang, Ricky TQ Chen, Cheng-Hao Liu, Aaron Courville, and Yoshua Bengio. Diffusion generative flow samplers: Improving learning signals through partial trajectory optimization. In International Conference on Learning Representations, 2024.
15

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of results, have made every effort to provide comprehensive account of our work. All experimental setups, including dataset pre-processing and hyperparameter settings, are detailed in the main paper (see Sec. and the appendix (see Sec. B). We provide clear pseudocode for our core algorithms, including the optimization of the path parameters (Algorithm 2) and the final density ratio estimation procedure (Algorithm 1), with the core of our PyTorch imple- mentation detailed in Algorithm 3. Furthermore, complete implementation of our method, MVP, along with scripts to replicate all experiments, will be made publicly available upon publication.
LLM DISCLAIMER
During the preparation of this manuscript, large language model (LLM) was used tool aid the research and writing process. The uses of the LLM were primarily for two purposes: (1) to assist in polishing the writing, including improving grammar, clarity, and phrasing of sentences; and (2) for retrieval and discovery, such as finding related work and summarizing existing literature to help situate our contributions within the broader academic context. The core ideas, theoretical derivations, experimental design, and interpretation of results were conducted by the authors.
ETHICS STATEMENT
This paper presents foundational work aimed at advancing the field of machine learning, specifically in the area of density ratio estimation. The research does involve human subjects, private data, or applications with direct ethical risks. While the direct societal impacts of this theoretical work are limited, we acknowledge that future extensions of our methods, particularly via our open-source codebase, could be applied in various domains. We encourage that any such application in sensitive contexts incorporate a comain-specifee ethical review to assess potential impacts.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
APPENDIX CONTENTS
Introduction
2 Related Works
3 Background
4 Minimum Path Variance Principle
4.1
The Path Variance: An Overlooked Term Objective
4.2
Minimizing the Path Variance via KMM Parameterization
5
Experiments
5.1
Baseline Setting and Experimental Setup
5.2
Ablation Studies.
5.3
f-divergence Estimation
5.4
Density Estimation
5.5
Illustration of Optimized Path Schedules
10
6
Conclusions and Future Works
10
Appendix
17
Assumptions and Proofs
19
A.I Assumptions
19
A.2 Proof of Lemma 4.1
19
A.3 Proof of Theorem 4.2
20
A.4 Proof of Proposition 4.3
21
A.5 Control of the Lipschitz Constant via Minimum Variance Path Principle
23
A.6 Proof of the Equivalence of Time Score Matching Objectives
B Experimental Settings and More Results
25
B.1 Formulations and Variances of Common Path Schedules
25
B.2
Experimental Settings for Objectives
26
B.2.1
Commonly Used Objectives
27
B.2.2
Total Objective for DRE with the DI
27
B.2.3
Total Objective for DRE with the DDBI
28
B.2.4
Adaptive Objective Weighting Trick
28
B.2.5
Variance-based Importance Sampling for Timesteps...
29
B.3
Experimental Settings for the Kumaraswamy Distribution
29
B.4
Experimental Settings and Results for f /ddivergence Estimation
32
B.5
Experimental Settings and Results for Density Estimation
34
17

---

Published as a conference paper at ICLR 2026
B.6 Illustration of Optimized Path Schedules
35
18

---

Published as a conference paper at ICLR 2026
A ASSUMPTIONS AND PROOFS
A.1 ASSUMPTIONS
Assumption A.1. For every æ, the map r log),T(æ)) is L-Lipschitz on [0, with a constant L independent of æ, and pr (x) > 0 for the a under consideration. Consequently, + log pr (x) is absolutely continuous, its (weak) derivative exists for almost every T E [0, 1]. and satisfies
0 lo (x)) - L,
a.e T E [0, I].
(18)
Assumption A.2. Under Assumption A. for any and any 0 S s the time integral of the time score is bounded by
(x log P. (x)
Or log pr (x)dr s S 10 lo)p s L(t-s). s).
(19)
A.2 PROOF OF LEMMA 4.1
Lemma 4.1. Let (pr he(u,1) be a probability path connecting po and Pt. Under Assumption A. the expected estimation error admits the following upper bound:
cath -
(f 1))) m dt
(x)
[A
nL
where e(xx t) (2, $g (x, t) is the error of the score model, the exponents satisfy 1 1 1. and L is the Lipschitz constant in Assumption A.I.
(1)
Proof. For simplicity, denote e(x, t) g(t t) 00 (x,t). t). We can express the density ratio estimation error A (x) by applying the Cauchy- Sehuwry inequality:
A (x) = llgg r(x) log f(x))
(0) 100 (xx 1)dl - 1// 10) (x, l)dl
1/'' (s00 1) - 20 (xx, de
= =1%- c(xr l)de
(20)
/ 1dtt) (fo t(x,,))dtt
(Cauchy-Schwart inequality)
<(xr, d).
Taking expectations over (x) and applying Tonelli's theorem (the integrand is nonnegative) gives: - -/(- Ey,(a) [c(æ, t) dt.
Ep) [xx) < E,, (ar) [/'' c(æ, t)aat -
(c(z, t)² 2 0)
(21)
Next, we rewrite the inner expectation with respect pi in terms of pe- Using the change of measure:
/
c(xx, t) (x)dx = Ep(()) ((2 (x())
Ep [c(x; t)
(x)
Ep((a))
[e(x,,) t)² exp (log P1 (x) (x))
<(x; t)² exp J. O, (ogp)
Bp,(a)
(22)
e < 1)2 exp (
< Ep((a)
10, lo)
(x)
< E Ep.(w)
<xx, 1)2 ((1-1))
(xx)
19

---

Published as a conference paper at ICLR 2026
where (+) follows since c(x,t)² 2 0 a.c., and (+*) follows under Assumption A.1.
Substituting Eq. (22) into Eq. (21) yields the bound
Ep (a) [a))] =
Bp,(m) [<(x, t)² e41-00) dt
L.
e ri(1 (1(1-1)) 1oll- Bpe(m)) [e(zx, t) dt
( (ulll-bal)) (fo
(FPriio (EEpccos) [c{) di
#
(23)
en 101--1
# (L. (Bppico) [<(ar, t)?!) m df
nL
where the second inequality follows from Hölder's inequality with 1 + 1 1. Different choices of (n, m) yield different bounds:
(n oc, 1):
Bp((a)A(())] e² J. t)
(24)
which preserves the training loss in its original integral form, at the cost of looser exponen- tial factor e².
(n = 2, m = 2) Schwarz):
22 ((=))²
( 3
(Exccoooll)) t)21) dt
(p) [[x)) -
(25)
which yields tighter L-dependent factor but replaces the training loss with an 1,2-type form.
General (n, m): one may also choose intermediate (n, m) to trade off between the looseness
of the exponential factor (Jo ent(I-tiat) 1/n and the form of the training loss norm.
This finishes the proof.
A.3 PROOF OF THEOREM 4.2
Theorem 4.2 (Minimum Path Variance Principle). In the loss decomposition (Eq. (9)), the second- moment term is exactly the path variance V - So Varp.( log P (x)))dt of a probability path {pe 1e Consequently, the expected estimation error of model s is bounded by
L (x)()) [Axx]] S e (ESTSM(()
Varp. (0, log (x))dt
(10)
This establishes the Minimum Variance Path (MVP) principle: Minimizing the overall error bound requires jointly minimizing the tractable model loss LSTSM and the path variance V.
Proof. We begin from the upper bound established in Lemma 4.1 (for the case 00, 1):
E (p)))))(())] S el S." Bppta) (c(x, 1)
(26)
where e(x, 1) = (x, 1) s0 (æ.() is the error of the score model. The integral term is precisely the ideal, but intractable, time score matching loss LTSM (0) (see Eq. (56)). The key insight, as established in Choi et al. (2022), is that the ideal loss can be exactly decomposed into two terms:
S.'' E'p.ees (<(2, jd (Os = LSTSM(0) + Bpcc) (s00
(27)
20

---

Published as a conference paper at ICLR 2026
where (x, - à log (x)).. For the second term, establish the variance identity:
(m)( (x,
(ar) (x0
+ (m)( (z)
(2,, t) + / (oz)))logpi(x)dee
(ar))(a))
2
Varp:(æ) (20) + (xpr(a))da
(28)
(2r, +
0e / ((())
(arp:(a))
(a)
(00) (xx,
since Oe px<)dx = 0,(1)= 0. This term is the path variance, which is explicitly path-dependent as it is determined by the time score (x.
Substituting Eqs. (27) and (28) into Eq. (26) completes the proof:
E (a) S e
(x) ()
c
(0) + Beccr Ber (a.)))
(29)
e
(OS + Byco) (Varpi(as (s()) ]]
cτ (STSM(() + / Varmi(a)) (z, dt
A.4 PROOF OF PROPOSITION 4.3
Proposition 4.3. The path variance V for the DI and DDBI interpolants, which depends the path
schedules (a, 3) = {a(t), (et)ot))) can be expressed in the following analytical forms:
1. For the DI in Eq. (2) with a Gaussian prior Po(x) (xx;0, Ia)::
300-f Volla, a| (0a
+ (2datt)) ax)) Ep [[|2]11]] dt.
(dā(())
((1)
(11)
a(t)2
2. For the DDBI in Eq. (3) with noise schedule of = t(1 tt)? + (a(t)) B(t)?)e:
| + B(t dl
d ()))²
E (po(ao)m(ar))
Vonei (a, 8]
(12)
2 (
of
Proof. Let (2, t) & 0 (z)):()) be the true time score. We provide the derivation for each case separately. The core strategy for both is apply the law of total variance to decompose Varp:(z) (O, (or) and then analyze each component.
Case (DI). For the DI case, the interpolant is X = a(t)xo + 3())x,, where Xo ~ Po(x)) Ni 0, Ia) and X1 (x)). The transition kernel for a given æ, is pe( xx)
N (z:; (()))1,(())21,
By the law total variance, we can decompose the variance term:
Varp.(a) (0, log (x))
Ep (w)) Var (0p log (x [x))]] + Varp, (wr)) (Bpi(elee)) (0, log (r( xi)))
(30)
21

---

Published as a conference paper at ICLR 2026
We first compute the inner expectation required for Term (II). The conditional time score is:
a 3()aa) 12
O (: æ1) = 0
log(2 - dllgge(t)
2
2a(t)²
i(())
+ a(l)
B(1)
d a(t) a(t)3
| 8(t)x,1 11
(x 1) Txx
(31)
a(t)²
(tt)
att) + a(t)
a(t)3 ||=1²² + B(t)
= æ1.
a(t)?
where we have defined the centered sample x B(t) Nz O, (a)
Using the moments [[] 0 and
||||=||]] = d the inner expectation simplifies to:
E ((rim)) [0 log Pe (x æx)] = -d ((() a(t) a(1)s
a(l) (da(t)))
3(1) a(t)² [| = O. (32)
Since the conditional expectation is zero for any 21, its variance, Term (II), is also zero. The total variance thus simplifies entirely to Term (I). Furthermore, because the conditional mean of the time score is zero, its conditional variance equals its conditional second moment:
Var,, (alma) (3g log (x xx1) = Ep, (m|
(O. log (3 px
x1))?
27
|| (attssllall
a(t) a(t)3
d a(t)
B(t) a(t)2
-E (m(o)))
+
a(t)
á(() 2
d [a(())²
á(1)²
((1)
B[|+2|+]+
2d
B B|||2113
B[(2 [[(=Tax "ax)))) )²]
a(t)3)
a(t))
a(t)*
att)
(33)
a(t) d(d + 2))a(t)+ +2 a(t)
a(t)²
a(t)
B(t)
a(t)²
2d
a(t)*
(da(t)))
a(t)
a(t)a (at) l?²)
a(1)²
3()2
(d(d + 2) d² 2ad2)
a(t) | ?
att)
2t)
{tt)
(t)) x 11.
at))
Here, we use the standard Gaussian moments: ||[|=||||| 2(d+2)a(t)*+ [(("" x1
and that all cross-terms involving odd moments of z are zero.
Finally, substituting this result into (I) and taking the expectation over pi (æ)) yields the total variance:
Varp.(w) (0, log Px)))) Epp (m,)
(arp.(=l= (0, lo x1))]
2dá
32
E (m))
||
112
af
at
(34)
2d
32
+
Ep (m))
(m))
||| 113]
Case 2 (DDBI). For the DDBI case, the interpolant is x, = a{{)x + 3(()x; + og with X Po-
X1 ~ P1, and noise schedule of = t(1 - t)?² + (a(t)² + ((t))).. The transition kernel conditioned on both endpoints (x æg. 31) = A (z: p4, of Ia). where μ4, att) + ((t)a1
The law of total variance is now applied by conditioning the pair (mo, m.
Varp.(w) (0p log lo (x)))
= E (po(so)pr(ar)) (ar))
Var (x)) .) (0, log pa (x æo, x x1))
Term
(35)
Var (p)pp(a))pp(m) (mi) (E (Ep, (ze mn 0 lo æo, x
(II)
22

---

M.) pr Published as a conference paper at ICLR 2026
Let b4 a(t)zu B(t)zz and jú å(t) + B(t)z11 The conditional time score is:
d
| Mell
1
0, lo wo wi) = 0
log (
2
20²
(36)
d
1 - Hill
(x
(0,) +
2 20
2(
of
Letting& μr o() with [|[|2|
do?, take the conditional expectation:
E||
TF[2] T irc
(p((ar)m) [0s log p(
To. [1)]
o +
(o +
2
of
(37)
d
do
(0,) =0
2
2
As in the DI case, the conditional expectation is zero, and thus Term (II) is zero and the total variance reduces the Term Similarly, the conditional variance equals the conditional second moment:
Varp, (m) (0.log Pt (x Zo, x1)) (ml (0.1og (2 To, x1)))
d
2( (2 (ot)
|=11
27
27
d2 (df)²
+E
(+)
20
o
(of)²
d (
d (ot)2
2 (of)² + (ot)²
1
B|||[||]+
Bl(z 1
4 (ot)2
4(07))
(o))²
& (
(oP) +
2))0
(**)
(a
4(07))
(o
d
(ld 2)
(ot)²
|/
+
4
4
(o
of
d + l + 2d)
(df)²
|
4
(o?)²
of
d (d))² (o))²
l(ā(t) + ((txx 12
(38)
of
where (+) holds because the cross-terms involving odd moments of the centered Gaussian z are zero
d² (#2)²
/ ||=||
2[a ?
and the rest term satisfies [[2AB] 2A B[B] 2 क with A ..
B
(**) holds
with the moments E|| ||]] d(d + ?) and the quadratic form TE(( (4,) = of 1
Taking the final expectation over Pu (zo) and pı (x11 gives the total variance:
Var (a)) (0) log pe d (0?)) (a?)²
+ Epo.pi
((()xo + $(1)x;
(39)
of
This completes the proof for the DDBI case.
L
A.S CONTROL OF THE LIPSCHITZ CONSTANT VIA MINIMUM VARIANCE PATH PRINCIPLE
Remark A.3. The connection between path variance Via, 3] and the Lipschitz constant L in As- sumption A. is primarily empirical and heuristic in nature, which is why we present it as a remark rather than formal proposition. In practice, minimizing V tends to produce smoother paths, and empirically correlates with reduced values of L.
Argument. Our heuristic is motivated by the observation that both V and the time score magnitude depend the path derivatives át!) and ((L). Specifically, the closed-form expressions for V in
23

---

Published as a conference paper at ICLR 2026
Proposition 4.3 are explicit functionals of squared path velocities (e.g.. d(t)?, ((t)? á((()
((t) Thus, minimizing V discourages paths with large instantaneous velocities.
Because our KMM parameterization yields smooth, bounded paths (see Eq. (14)), extreme velocity spikes are structurally unlikely. In such a regular function class, minimizing the integrated squared velocity V approximately suppresses the maximum path velocity, which in turn appears to help keep L from becoming excessively large in practice (see Assumption A. for detailed derivations).
This behavior is consistently observed our experiments: the optimized paths (see Figs. 5 to in Sec. B.6) are visibly smooth. We emphasize that this is a phenomenological observation, not theoretical guarantee, and further analysis of the precise relationship between path geometry and L remains an open question.
A.6 PROOF OF THE EQUIVALENCE TIME SCORE MATCHING OBJECTIVES
Proposition A.4. Let be the marginal distribution induced by the path schedule {a<(t), ((t)) For any fixed schedule, all three objectives share the same optimum: sg. sg. (a. t) = O log (e()).
Time Score Matching (TSM):
LTSM (0) (mm)).(
Or logg)(()))
2 50 (z t)
(2)
(40)
Conditional Time Score Matching (CTSM):
LCTSM (0) (pp()))( Or log pe( yg) - - (xt, t)
(41)
where is an instance of the conditioning variable y (y X1 for the DI, y (xo, x1) for the DDBI).
Sliced Time Score Matching (STSM):
LSTSM(0) (xp(t)p((aaa (2)) 0:s0 (xx t)+ (1) (2r, t)²
+ Liuundar,
(42)
where Lpoundury = E Epo(ao)) (D 0) - (0) (x1, 1)}
Proof. We prove that all three objectives are minimized when s/@ (xe, t) = Ô, log (xx;) by showing their differences are constants independent of O.
Part 1: Relating CTSM and TSM. We begin with the CTSM objective. For fixed i, the inner
expectation in CTSM can be expanded using the identity [B[2]] Var(z) BB[2]2
Ep(r) O log (x. y o (xx, t) 1
LeTSM( (0) = (pp()
(43)
For given and sample æ,, we analyze the inner expectation over y::
| log (ar,
8) =60² (xx,,) t)) t)
Ep(u)
|ah log (g) u( + p(xx,, () .60 (Zxz, 1)²)]
-p(())
(x)
(44)
[00, lo v) u( 1[] + |u(are,)) 1) -000 ( 1))?
Bp(y)
22
Varp(a)) (o, log (x) y)) + p( t) 5/ (zz, t)
where (*) holds under (1x E [ lo (xx
y)]. The key insight is that the conditional
expectation u( !) yields the marginal score:
(( () Ep(x) (0r log (ar
y)] 0, log (og.(((.)).
(45)
24

---

Published as a conference paper at ICLR 2026
Substituting this back yields the final relationship:
LCTSM( = (w.))).( log (a - (2),
(2e, 0) + (pe))p((es) [aaryy) (O, pag; 19))
(0)
Vend
(46)
Since Vound is independent of 0, both objectives share the same minimizer.
Part 2: Relating STSM and TSM. The STSM objective is derived via integration by parts from the TSM objective. The standard derivation shows that (Choi et al., 2022):
(0) (O) + ((..) |0 log Pt (c (xx,)²
(47)
Vnn
Since Vinarg is independent of 0 and is constant for a given path schedule, the optimum is shared. Finally, since (CTSM (0) LrMM ()) + const and (O) (0) + const, all three objectives are minimized by the same model (t) $g- (z) 1) = a log Pt (2)).
B EXPERIMENTAL SETTINGS AND MORE RESULTS
B. FORMULATIONS AND VARIANCES COMMON PATH SCHEDULES
This section provides the precise mathematical formulations for the common, fixed probability path schedules (a) (t)) and their time derivatives (á((), 3t)) used in our comparative experiments, alongside their corresponding analytical variance expressions derived from Proposition 4.3. For brevity in the variance expressions, we define the data-depent data-dependent constants Co ( [|| [3]
C = Ep (mm) [I] [ 2| and Co1 = Epo.py [x]] x1). The summary of path characteristics and associated variance properties is presented in Tab. 5.
Table 5: Summary of path characteristics and associated variance properties.
Path Schedule
Constraint
VDila, 31
VDDui 3]
Key Observation
Linear
a(l) + 8(t) 1 Divergent
Finite
DI is ill-posed; DDBI is stable.
VP
a(1)² + =3(1)2 Divergent
Finite
DI diverges as - Smoother path, but DI still diverges.
Cosine
a(t)² =B(t)2 Divergent
Finite
Föllmer
att)² -B(t)² 1 Divergent
Finite
Severe singularity in a(t) at t 1.
Trigonometric
at)) -(t)² Divergent
Finite
Singularity from tan" term at t = 1.
Linear Path. The linear interpolant, common baseline in stochastic interpolant literature (Albergo et al., 2023), defines a direct affine path satisfying (t) + (() = 1.
(tt) - A(t) t,
(48)
a(t) = --1 B(t) = 1.
The corresponding DI variance integral, Von( fo' (2d (2d
C1)(1 1) -2d?, diverges due to the
singularity at f For the DDBI case, the variance is given by:
VDDEI $] /' d (0f)2
Co + Ci
20o1
dt.
(49)
2 (0?)2
of
where the noise schedule is of = ((1 1) ((1 1)2 12)e.
25

---

Published as a conference paper at ICLR 2026
VP Path. This path is inspired by the variance preeeeeviiggggngggggg.. (VP) SDE schedule (Ho et al., 2020;
Song et al., 2021b), satisfying the spherical constraint att)² + B(t)² = 1. We use the standard diffusion constants Sn- 0.1 and 31 = 20.
a(t) = exp (-0.25/2(8) (31 (6)) - 0.51/3))
3(1) = a1-a(1);
(t)) = at))
(50)
a(t) = -0.5 ((3) Po)t 5b) (t))
8(t) (0.5(( - Bo)t + (o)).
Due to the term 1/a(1)² which diverges as I + 1, the DI variance is infinite. The DDBI variance is finite and given by:
+ 2a(t)) of where the spherical constraint simplifies the noise schedule of t(1 t)?² +
d (1 26)²
VDDBI (a, 8]
dt, (51)
(0
Cosine Path. The cosine schedule (Nichol & Dhariwal, 2021) also satisfies the spherical constraint.
is defined via a(t) cos
(++ 1+.
xt
/ cos
3 3) with 0.008.
ax({) vā(()
B(t) /I 11-aah a(t).
a(t)
a(1)
(52)
(t)
B(t) 2,(()*
(t(
The DI variance for this path is divergent. The DDBI variance takes the same functional form as the VP case, substituting the appropriate derivatives.
Föllmer Path. This path is derived from Föllmer's construction of a Brownian bridge (Föllmer, 2005; Chen al., 2024) and satisfies the spherical constraint.
all) = v1 - 12
3(1) L,
t
(53)
a(l)= = a(t)
B(1) =.
The variance, VDIla, 3 = fo (1-1-)2 + C1 1-(2)
2012
dt, diverges severely at = 1. The DDBI
variance is given by:
VODAI ja,3] 3] = S''
d f1-(1-21)
I-pr Co + Ci V1-12
21
Col
dt.
(54)
(ot)
tf
Trigonometric Path. This path follows a circular trajectory in the (a, B) space (Albergo et al., 2023), satisfying the spherical constraint a<t)² B(t)² 1.
a(t) cos
B(t sin (7
(55)
a(t)
π 3(t) 2
B(t) (t).
The DI variance, VDila, 3] (M 22 fo (2d (gt) + C.)dtt diverges due to the tangent term at t == 1. The DDBI variance takes the same functional form as the VP case.
B.2 EXPERIMENTAL SETTINGS FOR ORJECTIVES
Our experimental framework follows the theoretical analysis in Sec. 4.1. For each interpolant class (DI and DDBI), we have two primary objectives: 1) the minimization of the path variance V[a Sal to learn an optimal path schedule, and 2) the minimization of joint score matching loss to train the joint score model (a.() (x. - [.0
%9 (x,,
(#) (z, (), %g (æx, ()). Here, 8g
(w) and (t) are the data and 89
time score models, aiming to approximate the data score s(z) t) & Ö2p log (:( and time score (xx,t) & O log (xx). The superscripts (t) and (x) are labels indicating "time" and "data", not derivatives. We use these notations consistently throughout the paper. This section provides the precise mathematical formulations for these objectives implemented in our experiments.
26

---

Published as a conference paper at ICLR 2026
B.2.1 COMMONLY USED OBJECTIVES
In this section, we introduce three objectives for training the time score model.
Time Score Matching (TSM). To approximate the true marginal time score s0 (x.,), 1), the time (t) score model sg (x. 1) is trained by minimizing the ideal, but intractable, time score-matching loss:
LISM ()) (pp())(())) s(0) (x,,) 100 (ax, t)
(56)
Sliced Time Score Matching (STSM), Since the target score ( 1) is unknown, this objective cannot be optimized directly. A standard technique, as shown in Choi et al. (2022), is to apply integration by parts to derive a tractable objective. This equivalent training loss, which is termed as the sliced time score matching (STSM), given by:
LSTSM (0) 2E (200)
(
Co.
(1) 6 (21, 1) Bp(tiri
20 (ax, (xr,))+ (0) (z, (2, t)²
(57)
where the first term denote the boundary conditions of (t)
60 0,5² is the time derivative of s/ and p(t) is a timestep distribution over (0, 1) (see Sec. B.2.5 for details).
Conditional Time Score Matching (CTSM). In the limit of infinite data and model capacity, the time score model (e) exactly recovers the time score s(t) by minimizing the conditional time score matching (CTSM) objective (Yu et al., 2025):
LCTSM (0) = (t.))) O Po (2: y) 60
(t) (xx,, 1)
2
(58)
where p(t) is a timestep distribution (see Sec, B.2.5 for details), æz is induced from DI (see Eq. (2)) or DDBI (see Eq. (3)), and y is an instance of the conditioning variable y (y - x1 for DI, y & (Xo, x1) for DDBI).
We Use CTSM. As shown in Proposition A.4, all these three objectives are equivalent, guaranteeing that the learned model satisfies ag- (z, = Ô log (zr).. Hence, we adopt CTSM for training since (i) the conditional score available in closed form, (ii) no derivatives are required, and (iii) the score model is evaluated only once per batch.
(t)
B.2.2 TOTAL OBJECTIVE FOR DRE WITH THE DI
Path Variance Objective. For the DI setting, we learn the path parameters & of our KMM-based schedule by minimizing the analytical variance given in Proposition 4.3:
Voi Ja 3ol = - (d) (t)2
Bx())
min +
(t())² Bm((e))
[| [1]] dt.
(59)
Joint Score Matching Objective. We adopt the joint training strategy from Choi et al. (2022);
Yu et al. (2025), where the joint score network s/ (2.1) (.,,)((z,))) A [=(=e) (ar, t), (( t)] is trained to simultaneously predict both the conditional time score and the conditional data score. The network (t) thus outputs scalar time score $ < R and vector data score s(()) Rd
(t) The time score target is dD (ar,, 1; x1))) O, log p (æte x 1 given by:
a(l)
3(1)
p (r,, 11))
(4)
(||0o|12 d) a(t) a(t)
mo æ1
(60)
The data score target is spi (æt, t; (x)
(a))
A V
æ. log 1o (x 11). This is given by:
(a) spi
(z,,
Kt 8(t)
a(t)x0
200
(61)
a<1)²
a(t)²
aat)
27

---

Published as a conference paper at ICLR 2026
Thus, the conditional joint score matching objective (CJSM) is to minimize the sum of the mean squared errors for both scores:
(C E (xr)pp)(
sDi (xx,
(a.1)
t;
(x., t)
Sg (2x.1) (xx,,
(62)
where 100.0)(m (a,, 1;
[
(1) (a)) (#z; 11,, ) sDi (m,; (x); is the joint score target and (a.)) %
(mr;
Is (m) (zr, t), s0 (x,, f)] is the joint score model.
Total Objective. The total objective the weighted sum of the path variance objective and the joint score matching objective:
( (0, Φ) = CrIs (0) + A2 (a Aa).
+ =21,, =
(63)
where X and A are the weighting coefficients determined by the Soft Optimal Uncertainty Weighting (UW-SO) method (see Sec. 3.2.4 for details).
B.2.3 TOTAL OBJECTIVE FOR DRE WITH THE DDBI
Path Variance Objective. Similarly, for the DDBI setting, the KMM path parameters & are learned by minimizing the corresponding analytical variance from Proposition 4.3:
VDDBI (arp, (0pp,30== = 1/¹
(op)² 2 (0
+ (=)))))
(t)()) + Bst()
min
dl,
(64)
of
where of = =1 t)² 2 + (a: (f) (t) 2 0 is the noise factor, and e a small number.
Joint Score Matching Objective. Following the same joint training strategy, the network predicts both the conditional time and data scores. The target scores are conditioned on the endpoints (æo. æ1).
(1) The time score target is DDBI (2 1;00, x21) - Og log Pe(ar, Wo 21). given by:
07
| ot
(á(())zo + ((1)x))
(t) DDBI (xt. to
d
(6.5)
207
dt
The data score target is SonnI(at t: xD, x1) Væ, log Pt (xt
Tu, 21), which simplifies to:
2t - (a<(1) xp + (())a) 0+2 z
DDRI (at, t: Do, 2.)
(66)
o?
0f 5
In these expressions, of t(1 - t)2 2 (a(t)² + 3(t))) and x #: (a-(aat)a + 3(4)aa)
Similarly, the CJSM objective is to minimize the sum the mean squared errors for both scores:
(a.1) SDDBI (mt, 4; wo, 221) 80 (a..t) ( 1)
LCISM (egtttp..ta
(67)
where (x.1) SDBB (xr, t; Vo. a [s [s (x,, t; Vo. I.). SDBI (z,, t; zo x; is the joint score target and (t)
(x)
(mn,t) (xr, t) = |sp (xt, t), $g (x,, t)] is the joint score model. $g
Total Objective. Similarly, the total objective given by:
Luoal (,,00 Φb) X₁ (CIM(()) (0) + >22 Ja Bol, d + 22 =
(68)
B.2.4 ADAPTIVE OBJECTIVE WRIGHTING TRICK
We balance heterogeneous training objectives, including the path and the joint score matching objectives, using the soft optimal uncertainty weighting (UW-SO) method of Kirchdorfer et al. (2024), an analytical refinement of the uncertainty-based weighting framework of Kendall et al. (2018).
28

---

Published as a conference paper at ICLR 2026
UW-SO computes closed-foree weights every optimization step, eliminating the need for learned noise parameters while preserving the benefits of incertainty-awwl balancing.
Given J loss terms { compute their normalized weights
exp
T(Cytc)
A
(69)
Σラ= exp T[C)+e)
where ensures numerical stability and T 0 is softmax temperature (we set T = by default). The final training objective is goo-ggraieet weighted sum:
Liotal
sg()))c
(70)
where sq denotes the stop- -roddent operation, preventing the weights from affecting the backward pass. This yields parameter- free, stable, and easily reproducible loss combination strategy, which we adopt as the default in all experiments.
B.2.5 VARIANCE-BAN BASED IMPORTANCE SAMPLING FOR TIMESTEPS
Rather than sampling ~ (0, 1), we adopt variance-based importance sampler inspired by Song et al. (2021a); Yu et al. (2025), which improves training stability by focusing computational effort on time steps where the ground-truth score signal is more reliable (i.e., has lower variance).
We utilize this sampler to fundamentally simplify the training objective. We start with the objective containing the optimal inverse-variance weighting function, X(t) x 1/ (Varp.(æ) (sf*) (x, f)) + s).
and employ importance sampling by drawing time steps 1. ~ p(1). The resulting objective is proportional to:
LTSM x Expp(e)
X(t)
(x,
1 -0 1)
(71)
p(t)
By setting the sampling distribution p(t) to be directly proportional to this optimal weighting function:
p(t) X(t) Varp.(r) (s()) +
(72)
the importance sampling factor A(t) p(t)
becomes constant, 1
This results in an effectively uniform
weighting of the squared error term:
LTSM x 7 Exup(t)
22 x Empp(() -,,00) (2, (c.0))]- [10())))) -))))))) (xx 1)
(73)
This mathematically guarantees the benefits of optimal weighting while maintaining an objective free of explicit, non-unity weighting functions.
Analytical expressions of the variance are available for DI and DDBI (Proposition 4.3). In practice, we pre-compute variances on time grid and sample from the resulting categorical distribution. updating it periodically as the path schedule (t. evolves.
B.3 EXPERIMENTAL SETTINGS FOR THE KUMARASWAMY DISTRIBUTION
A Stable Kumaraswamy. The Kumaraswamy distribution suffers from severe numerical issues near the [0, 1] boundaries, where direct evaluation ts or (1 - -0) leads to catastrophic cancellation and unstable gradients (Wasserman & Mateos, 2025). To address this, we compute powers as,
exp (y log1p(= 1));
(74)
which preserves precision by avoiding subtraction of nearly equal numbers. This stable power function is applied the CDF and PDF (Eq. (13)), ensuring reliable gradients during training.
29

---

Published as a conference paper at ICLR 2026
Initialization Strategy for Mixture Components. To prevent component collapse and encourage specialization, we initialize the K Kumaraswamy components with modes spread across [0, 1]. For each Tą = K+T
k+0.5 the shape parameters (ak, bş) are set asymmetrically:
{ (1.5 + 0.5k, 3.0 + 2.0(K - -))
ik < 0.5,
(ak+ by))
(75)
(3.0 + 2.0k, 1.5 + 0.5(K - k))
ix 2 0.5.
This design produces right-skewed components (az < bb) for T < 0.5 and left-skewed components (ay by) for Tk 0.5, ensuring that their modes
1/ak
mode(k) bز
ak 1,b 1,
(76)
distributed across [0, 1J. The weights initialized uniformly, wh 1/K. Such diversity- oriented initialization accelerates convergence and avoids degenerate solutions (Dai & Wipf, 2020),
PyTorch implementation of MVP path with KMM parameterization. We also provide the PyTorch implementation of the proposed MVP in this paper, shown in Algorithm 3.
30

---

Published as a conference paper at ICLR 2026
Algorithm 3 PyTorch implementation of MVP path with KMM parameterization.
class MVPPath :on.oel uam
An implementation for computing the MVP path schedules alpha (t) and (et using a Kumaraswany Mixture Model (KMM)
def (self n_comp=4, constraint_ super () super()._nni__() sel self.constraint_typ.. self.eps - constraint_type, eps
eps=le-5)
Learnable parameters for the RMM, Sec. B.3 for details self. nn. Paraneter Storch.zeron (1, c_comp))
11
12
self.log_ä nn. on.Parameter (torch. (1, r_comp)]
13
self.log_b - nn. an.Paraeeter (torch.randn. (1, r_comp)]
def kmm_kkmmppp param (self):
16
"Applies transformations to get valid KMM parameters."
17
weights = sofftmax (self. (ss f.oggits, dim=-1)
(self. a 5. (fef b F softplus (self.log_b) # Ensures b
# Ensures
20
return weights, a, b
21
22
def sstabbe_pox (self, y): "Nueerically stable conputation of x"y for close 1." return torch.exp(y torch. 1x 1))
23
25
26
def kmm_odf (self, t, weights, a, b):
"Computes the KMM Cunulative Distribution Function."
t_a stables.a (t, a)
29
comp_cdfs
self..stable.poo
t_a, b)
30
return torch. sum (weights comp_cdfs, dim=-1, keepdin-True)
31
def
kmm (self, t, weights, a, b):
"Computes the KMM's Probability Density Function." t_a wtw.le..oollleooo...ee..o a) m_ = self stable..tbb a 1)
34
35
36
o = stableeoa t_a, b - 1)
comp_pdfs
t_amm one_m_t_a_bbmi
39
return torch. sum (weights comp_pdfs, dim=-l, keepdin-True)
40
41
def forward (self, t):
"Computes alpha, beta, and their derivatives at time t t.clamp (self. weights, a, b = self ge....ggt.kmmpparm. ()
43
self.eps)
44
45
46
# alpha is the inverted CDF; dot_alpha is the negative PDF Eq. (14) for details alpha self, km edf (t, weights, a, b) alpha -self, kmm (t, weights, a, b)
48
89
50
$ Beta and its derivative depend the constraint "spherical";
52
if self.constraint_tppo
53
# In this case, alpha (t) 2 + beta (t) - beta = torch. torch.sqrt dot beta = -alpha dot alpha / beta else: "affine"
54
sqrt (1 - alpha pow + self.eps)
55
57
# this case, alpha beta (t)
58
beta = alpha
59
dot_beta -dot_aiphe
return alpha, beta, dot_alpha, dot beta
61
31

---

Published as a conference paper at ICLR 2026
B.4 EXPERIMENTAL SETTINGS AND RESULTS FOR f-DIVERGENCE ESTIMATION
Mutual Information (MI) Estimation. MI quantifies the statistical dependence between two random variables x p(z) and y ~ q(y), and is defined as MI(x, y) = Ep(a.a) log pla.z) p(æ))(()
MI
estimation naturally reduces to DRE since y) involves the expectation of a log density ratio.
Details on Geometrically Pathological Distributions. To further probe the robustness of our method, we evaluate it suite of five MI estimation tasks involving geometrically pathological distributions. These tasks, inspired by the benchmark suite from Czyž et al. (2023), are specifically designed to challenge the underlying assumptions of many standard estimators.
Edge-Singular Gaussian. This task assesses model stability edge-case correlation
scenarios. We consider two joinlly Gaussian random variables, (x.y) ~ NN,, [[1, pl. (o, I]]) The mutual information, given by MI(x: = -0.5 -0.5log(1 - p challenging the extremes of the correlation coefficient p. As p p- 17, the distributions become nearly degenerate and collinear, leading to numerical divergence in many estimators. Conversely, as p - 0+, the dependency signal becomes vanishingly weak, testing the model's sensitivity and its ability to avoid vanishing gradients.
becomes numerically
2. Half-Cube Map. This distribution challenges models with heavy-tailed data. We begin with two correlated Gaussian variables and apply the homeomorphism half-cube(z) z z/3//sign(z) each. The resulting variables, x' half-cube(x) and y' = half-cube have significantly heavier tails than the original Gaussians. While this transformation preserves the mutual information, i.e., MI'x';'') MI(x; y). the distorted, heavy-tailed geometry can be difficult for non-parametric methods that rely on local density estimation.
3. Asinh Mapping. This task tests the model's ability to handle distributions with highly con- centrated densities. We start with two independent standard Gaussian random variables, and y. and apply the inverse hyperbolic sine transformation, asinh(z) log(z V2 I), to each. The resulting variables, x' = asinh(x) and y' = asinh(y), form our two distribu- tions. This mapping compresses the tails of the original Gaussian distributions into central region with extremely high curvature, creating sharp density peaks that can cause gradient instability and pose significant challenges for kernel-based methods.
4. Additive Noise. This scenario is designed to evaluate performance on distributions with fragmented support and sharp discontinuities. Let x be a random variable uniformly distributed on [0, 1], i.e.,
(0, 1), and let n be an independent noise variable, n u(-e, e). We define the second variable as = x + n. The resulting joint distribution is piccewise-constant and possesses sharp, non-differentiable boundaries. This structure violates the smoothness assumptions underlying many score-based estimators, providing a stringent test of model robustness. The true MI is given by MI(x;y) = log(2c) + 0.5 for 0.5.
5. Gaama--Exonnene This task features a complex, non-linear dependency structure. We define hierarchical relationship where the first variable x is drawn from a Gamma distribu- tion, X ~ Gamma(p, 1). The second variable y is then drawn from an Exponential distri- bution whose rate parameter given by the value of x, i.e., y x = x ~ Exponential The resulting joint distribution is highly asymmetric and non-Gaussian. The true mutual information is I [xx; y) = <p + 1) log(p), where & is the digamma function.
The results in Tab. 6 can be summarized as follows: (1) On the Edge--ingulla Gaussian task, MVP (both affine and spherical) attains the lowest or near-lowest MSE, especially under extreme correlations (p + 0, +1). On the Half-Cube Map task with heavy tails, the spherical constraint yields stable and accurate estimates, outperforming all baselines. (2) On the Asinh Mapping task with concentrated densities and sharp curvature, MVP again achieves the lowest error, while baselines such as VP struggle with density peaks. (3) The largest gains appear on the Additive Noise and Gamma-Exponenaa Exponential tasks. With sharp discontinuities and complex non-linear dependencies, fixed- path methods degrade severely, whereas MVP, particularly with the spherical constraint, achieves dramatically lower MSE, often by order of magnitude.
32

---

Published as a conference paper at ICLR 2026
Overall, no fixed path is uniformly effective across all five benchmarks, with performance varying by data geometry. In contrast, MVP consistently delivers state-of-the-aa accuracy by optimizing path variance, producing iata-adaptive trajectories tailored to each problem's 's structure.
Table 6: Full MSE results for MI estimation on five geometrically pathological datasets. The top row of each sub-table indicates the varying correlation coefficient, p. Our MVP path demonstrates consistently superior or competitive performance across the wide range of challenging data geometries, highlighting the benefits of a learnable path.
(a) MSE results for the
Gaussian dataset.
Par
Cieatrabel
469
4.1
866
0.4
44
41
4.1
168
16.5
0.4
EEP
11.45
0,301
LA35
0,0803
1J882
0.363
8.8G
01011
1U5.33
0U9E2
21236
0.382
1190
(1:802
88113
109H
0181:
0186:
1990
8107:
008
19811
018183
019881
01887
1986
1196
(UK0)
MVP
(1803
0330
10014
(080)
1088:
1811
B163
10E31
01803
8863
1120
EXE15
0.341
1940
1196
JUKES
MYV
0.000
0.9991
0.860
0.301
0.0000
00031
0.001
0.000
01911
C4031
01001
0.000
1.01
110C
6/00)
(b) MSE results for the Half-Cube Map dataset.
Fub
Creatrake
-.9
-1.8
407
-66
4.5
-14
433
42
4.1
0.9
01
0.2
0.5
04
0.5
8.4
83
086
0P
03.00
8 1.006
1096
01011
01081
133
1936
0-3955
0/0
01865
1.965
11315
00093
0.309
1011
01013
0/0-3
01.445
3.309
11823
(U6E)
10801
0.363
1.633
1UE
0U942
8.00
1.322
LAE
0.363
1191
0.065
0.805
3.300
168.
0463:
0.301
1893
0U965
8.4
1.300
0.383
1941
(1945
0346
b/9993
00865
01492
0/993
00000
U1922
1134
0.006
01440
0/988
WEEESS
(URC
(c) MSE results for the Asinh Mapping dataset.
Fash
Clestrike
4.9
4.4
4.1
666
0.4
-.4
0.1
03
0.1
0.9
0.1
0.3
6.1
065
0.4
0.4
81
655
119
1.113
108055
1.80
61001
1UE
0.001
0143
31:34
110.0:
0.365
1085
1188
1J612
1.311
1.08
01855
0.303
1UE
01862
3.330
0.9003
0.365
1940
8 1104
UUED
(UEC4
1914
1945
01.85
1028
01667
008:5
180m
01877
8863
1131
0.311
1989
1186
(083)
MVP (aan))
0.002
6.033
0.003
04677
0000
0.367
3.91
0.0000
0.000
0.805
0.999
0.8681
00007
0.365
6.003
0.062
0/005
(d) MSE results for the Additive Noise dataset.
Path
Constraint
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Linear
affine
0.3053
0.05-11
0.0233
0.0465
0.0367
0.0714
0.0339
0.0307
0.0268
Cosine
spherical
0.4803
0.1085
0.0788
0.0969
0.0719
0.0867
0.1015
0.1204
0.1304
Föllmer
spherical
0.4731
0.1048
0.0510
0.0864
0.1024
0,1226
0.0944
0.1115
Trigonometric
spherical
0.3615
0.0565
0.0508
0.065-1
0.0596
0.0517
0.0302
0.0148
0,0603
VP
spherical
0.3455
0.0290
0.0303
0.0346
0.0210
0.0313
0.0426
0.0390
0.0123
MVP (ours)
affine
0.23660
0.0199
0.0081
0.0167
0.0161
0.0216
0.0304
0.0155
0.0010
MVP (ours)
spherical
0.1016
0.0134
0.0078
0.0009
0.0138
0.0117
0.0196
0.0026
0.0029
(e) MSE results the Haama--xxooentiaa dataset.
Path
Constraint
1.0
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
Linear
affine
2.8166
0.7064
0.1086
0.1368
0.0513
0.0420
0.0051
0.0147
0.00G6
Cosine
spherical
7.2466
1.0785
0.4875
0,2582
0.2802
0.3063
0,2936
0.1864
0.3438
Hüllmer
spherical
0.4006
0.4275
0.6304
0.3741
0.2822
0.46662
0.2473
0.3838
0.3105
Trigonometric
spherical
10.4420
1.3841
0.2705
0.3854
0.0184
0.0681
0.1109
0.0103
0.0668
VP
spherical
2. 7,800
1.3572
0.1116
0.1224
0.1320
0.1817
0.1407
0.0919
0.2507
MVP (ours)
affine
0.4785
1.5466
0.1137
0.1274
0.0915
0.0063
0.0032
0.0294
0.0069
MVP (ours)
spherical
0.2721
0.2304
0.0917
0.0434
0.0075
0.0187
0.0091
0.0030
0.0004
High- figh-ddimensionll & High-discrepancy Distributions. To rigorously test model performance under extreme conditions, we designed an experiment focused on MI estimation between two high- dimensional Gaussian distributions with large discrepancy. This setup is intended specifically invoke the ddensity-chasm" problem, known failure mode for DRE-basee methods where the path between the two distributions traverses a region of vanishingly low density (Rhodes et al., 2020). The goal is to evaluate whether our path optimization strategy can effectively navigate this chasm where fixed paths may fail. We define the two distributions as (o(x) = (0, Ia) and p1 (x) = (00 E). The covariance matrix Σ is constructed to be block-diagonal, where each 2 x 2 block along the diagonal is given by A (). with 0.5. This structure creates strong pairwise correlations within each block while ensuring that the blocks themselves are independent. This results in a highly ill-conditioned and low-rank covariance matrix, known challenge for based DRE methods (Choi et al., 2022). The experiments are conducted across several dimensions, d E {{ {40, 80, 120, 160). As the dimension increases, does the true MI between the distributions, and thus the discrepancy,
33

---

Published as a conference paper at ICLR 2026
with corresponding MI values of approximately {10, 20, 20,30,0000000 40) nats. This allows systematically evaluate model robustness as the density-chasm problem becomes progressively more severe.
B.5 EXPERIMENTAL SETTINGS AND RESULTS FOR DENSITY ESTIMATION
Structured and Multi-modal Distributions. We suite of six 2D synthetic datasets eval- uate the model's ability to learn complex and structured distributions. Five of these are standard benchmarks used in Chen al. (2025c), while the tree dataset follows the setup from Karras et al. (2024). These datasets are specifically chosen to probe different failure modes of density models.
Disconnected Topologies. The circles and rings datasets test the model's ability to capture distributions with multiple, disconnected components and assign zero density to the regions between them.
Intricate Structures. The 2spirals and pinwheel datasets feature highly structured, non- linear manifolds that require the model to learn complex, curving paths.
Discontinuous and Branching Densities. The checkerboard dataset presents a particularly difficult challenge with its discontinuous, grid-like density. The tree dataset is designed to assess a model's ' capacity to generate sharp, branching topological structures, a known challenge for many generative models.
Together, these datasets form a comprehensive testbed for evaluating the flexibility and robustness of density estimation method.
Cosine
Trigenometric
MinPP (ours)
circles
pinwheel
checkerboard
rings
2epirals
Figure 4: Density estimation on six structured and modial datasets. MVP successfully learns data-adaptive path tailored to the specific manifold of each dataset.
Real-world Tabular Distributions. We evaluate five tabular datasets widely used in density estimation (Grathwohl et al., 2018), covering domains from particle physics to image patches. These datasets pose challenging, non-Gaussian structures with unknown generative processes and complex correlations, making them suitable for testing model expressiveness. We follow the preprocessing and splits of Grathwohl al. (2018) for fair comparison.
34

---

Published as a conference paper at ICLR 2026
B.6 ILLUSTRATION OF OPTIMIZED PATH SCHEDULES We visualize the path schedules learned under the MVP principle across different tasks and datasets. Here, "prior" and "posterior" denote the initialized and optimized path schedules.
Comcanioo W11
Compatien Mrit
2.5
0.6
0.5
2.6
14
0.4
3.4
0.3
2011
(a) affine.
(b) 9 affine.
(c) spherical.
(d) [3 spherical.
Figure 5: Optimized KMM-parameteeiizze path schedule for geometrically pathological distributions.
1.0
24
0.4
3.4
0.4
0.2
0.0
52
0.4
0.
0.5
1.0
0.0
0.2
0.4
0.5
0.35
2.5
Yras
(a) o; affine.
(b) 3; affine.
(c) a; spherical.
(d) 6; spherical.
Figure Optimized KMM parameettttte path schedule for high-dimensional distributions.
2.0
3.0
3.0
24
22
0.2
6.3
0.4
2.0
(a) at affine.
(b) 3: affine.
(c) a spherical.
(d) 3: spherical.
Figure 7: Optimized KMM-parameteriized path schedule for structured and multi-modal datasets.
2.0
3.0
3.2
2.0
5.8
0.6
0.2
3.8
3.6
0.
3.0
8.0
03
50
50
6.2
0.4
3.5
3.
1.0
0.0
04
066
68
1.4
0.0
02
0.6
0.6
0.4
3.5
5.0
6.2
0.4
06
3R
1.0
(a) ar affine.
(b) 3; affine.
a spherical.
(d) 3; spherical.
Figure 8: Optimized KMM-parrmeeterized path schedule for real-world tabular datasets.
35


Reviews:
[{"review_id": "g0cEm8Shww", "reviewer": "Reviewer_HpCs", "summary": "The paper studies density ratio estimation problems that integrate time-scores along some path. Even though any smooth path is in principle okay, the practical performance of such methods depends on the actual path when approximations of the time score are used. The authors quantify the approximation error trough variance of the time-score along the path and show that choosing a minimum-variance path is optimal, both theoretically and empirically.", "questions": "No specific questions.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is in overall a strong contribution for the DRE literature. The idea of analysing the variance of the time-score is intuitive but to my best knowledge novel, and the authors deliver a well-executed study that makes the connection between the variance and the estimation quality clear. Theorem 4.2 is a worthy objective in itself, and the closed-form expressions provided in Proposition 4.3 are interesting. The practical method is well explained and the empirical results provided in broad set of experiments are strong, and already the explanation for (one) reason for the path-dependency of practical performance would be valuable as such even if not demonstrating improved accuracy in practice.", "weaknesses": "The empirical gain is clear, but not so major that this would be a transformative contribution that would qualitatively change the state-of-the-art.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mJVwHfyOrO", "reviewer": "Reviewer_N7gR", "summary": "The goal is to estimate the ratio of two densities $p_0$ and $p_1$ using their samples. The authors do so using the framework of time score matching, where a design choice if the path of distributions $(p_t)_{t \\in [0, 1]}$ that is chosen by the user to link $p_0$ to $p_1$. \n\nThe authors upper-bound the estimation error of the density-ratio in terms of the path choice, and minimize the upper-bound. They obtain closed-form results (optimal time-reparameterization aka \"schedule\" of a given path) and empirical results (numerically optimizing over a family of paths).", "questions": "This is an interesting paper and I am glad the authors are tackling how the estimation error of the density-ratio depends on the choice of path: this is an open question in the community. \n\nCould the authors address my questions detailed in the weaknesses section? In there, my main concerns are my \"Issue #2\" and \"Evaluation\" sections. I would be happy to raise my score if these are addressed.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The authors identify an interesting question that has not been solved in the literature of density ratio estimation using the time score identity. Their analysis is rather novel, and is reminiscent of similar results in the flow matching literature (where the estimation error is upper bounded by the training loss).", "weaknesses": "## **Minor**\n\n- **Terminology.** \n\nLine 159: the authors say the time score loss in (Choi et al., 2022) is known as the *sliced* time score matching objective. I have read (Choi et al., 2022) and have not come across the terminology of \"sliced time score matching\" in the main text. As far I know, (Choi et al., 2022)  just call their cost function \"time score matching\". Is \"sliced\" a terminology that you are adding? If so, it would be fair to explicitly say so. \n\n---\n\n## **Math Questions**\n\n- **Reweighting.**  \n In the time-score matching loss (Eq 6), do you use a reweighting function as (Choi et al., 2022) do?\n\n- **On Section 4.1.**  \n  Is $\\theta^\\*$ the minimizer of the *empirical* loss, the *population* loss, or neither? Is it simply a fixed parameter value of the time-score model? If it corresponds to the *true data parameter*, then shouldn’t $L_{\\text{TSM}}(\\theta^\\*) = 0$, assuming the neural network can perfectly approximate the true time score?\n\n- **On Section 4.2.**  \n  You parameterize general paths between $p_0$ and $p_1$, but it is unclear how you enforce the boundary conditions. Specifically, how do you ensure that at $t = 1$, we have $p_t = p_1$?\n\n- **Potential inconsistency in Section 4.1.**  \n  My understanding is that:\n  $$\n  \\text{estimation error} \\le e^L \\times L_{\\text{TSM}}(\\theta^\\*) \n  \\le e^L \\times (L_{\\text{STSM}}(\\theta^\\*) + \\text{slack}),\n  $$\n  where $L$ is a Lipschitz constant of the path, and the slack term corresponds to the kinetic energy of the path.\n\n  *Issue #1*. The paper states that “to minimize the final estimation error, one must minimize the ideal score-matching loss.”  \n  I believe this is a **sufficient** but not **necessary** condition, since the ideal score-matching loss only provides an upper bound.  \n  If that upper bound is loose, it may be possible to reduce the estimation error without reducing the ideal score-matching loss.\n\n  *Issue #2*. You aim to minimize the estimation error with respect to the choice of path $p_t$ and therefore propose minimizing its upper bound over paths. However, your analysis focuses only on minimizing the *slack term* (the kinetic energy) while seemingly ignoring the term $L_{\\text{STSM}}(\\theta^\\*)$, which also depends on the choice of path. It is possible, for instance, that a path minimizing the kinetic term actually increases $L_{\\text{STSM}}(\\theta^\\*)$, in which case the upper bound (and the estimation error) may not improve.  \n\n---\n\n## **Missing Reference**\n\nThe authors claim that the literature lacks a theoretical analysis of how the choice of path $p_t$ impacts the estimation error of the density ratio.  \nWhile this remains an open and interesting problem, it would be relevant to cite prior work that provides partial theoretical results in this direction.  \n\nIn particular, **Theorem 4 of [1]** derives an upper bound on the estimation error of the density ratio as a function of the path’s Lipschitz constant—closely related to your Lemma 4.1, with a similar proof structure (see Appendix D.3 of that work).  \nThis reference would be highly appropriate to discuss, as it represents the closest known theoretical result to your contribution.  \n\nAnother reference that is less directly related but might be interesting to the authors is [2]. It also investigates the optimal path for reducing the estimation error of a density ratio using the time score identity, in the specific case when all the densities $p_t$ are parameterized only by their normalizing constant $Z_t$. \n\n[1] Yu et al. *Density Ratio Estimation with Conditional Probability Paths.* ICML 2025.  \n\n[2] Chehab et al. Provable benefits of annealing for estimating normalizing constants... NeurIPS 2023. \n\n\n---\n\n## **Evaluation**\n\n- **On Figure 2.**  \n  Figure 2 does not seem to show a substantial performance difference between optimal and suboptimal paths $p_t$.  \n  Could the authors comment on this observation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5fqtgGuUQv", "reviewer": "Reviewer_8HVQ", "summary": "Score-based approaches for Density Ratio Estimation (DRE) have gained interests in recent literature. While previous works typically employ a fixed probability path beforehand, the authors observe that the choice of the path can have implications on the downstream performances. In order to resolve the issue, the authors propose to explicitly learn the path by minimizing the resulting variance. Specifically, a Kumaraswamy Mixture Model (KMM) is employed to model the schedule. Empirical experiments are provided, which demonstrate that the proposed method helps to improve the performances of the DRE method.", "questions": "1. In the main paper the STSM objective is highlighted, e.g. the equality in Equation 9 relies on STSM objective. However, in Appendix B.2.1. it was noted \"We Use CTSM\". Which objective is employed in the experiments?\n2. The empirical performances of the methods naturally depend on many factors, e.g. the weighting function \\lambda(t) as employed in the objective. Have the authors made attempts to isolate the other factors to investigate whether the empirical improvements are indeed due to the choice of path?\n3. In terms of broader literature, the variance of the time score appears in information geometry context as the time score is the Fisher score with respect to t. Furthermore, learning the paths have been investigated in diffusion literature, e.g. [1].\n\n[1] Neural Diffusion Models, Bartoh et al. ICML 2024", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea of learning the interpolation path in general is well-motivated.\n2. The proposed solution relying on closed-form path variance is interesting.", "weaknesses": "1. As the authors themselves noted in the Appendix, the bound in Lemma 4.1 is with a specific choice of n and m, without strong justification of these choices.\n2. Relatively minor, the experiments are of smaller scale problems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)