Diffusion Bridge Variational Inference for Deep Gaussian Processes
Authors: JIAN XU
Delu Zeng
Qibin Zhao
John Paisley
Venue: ICLR 2026 Poster (2026)
Keywords: Deep Gaussian Processes
Diffusion Bridge
Variational Inference
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Abstract: Deep Gaussian processes (DGPs) enable expressive hierarchical Bayesian modeling but pose substantial challenges for posterior inference, especially over inducing variables. Denoising diffusion variational inference (DDVI) addresses this by modeling the posterior as a time-reversed diffusion from a simple Gaussian prior. However, DDVI’s fixed unconditional starting distribution remains far from the complex true posterior, resulting in inefficient inference trajectories and slow convergence. In this work, we propose Diffusion Bridge Variational Inference (DBVI), a principled extension of DDVI that initiates the reverse diffusion from a learnable, data-dependent initial distribution. This initialization is parameterized via an amortized neural network and progressively adapted using gradients from the ELBO objective, reducing the posterior gap and improving sample efficiency. To enable scalable amortization, we design the network to operate on the inducing inputs $\mathbf{Z}^{(l)}$, which serve as structured, low-dimensional summaries of the dataset and naturally align with the inducing variables' shape. DBVI retains the mathematical elegance of DDVI—including Girsanov-based ELBOs and reverse-time SDEs—while reinterpreting the prior via a Doob-bridged diffusion process. We derive a tractable training objective under this formulation and implement DBVI for scalable inference in large-scale DGPs. Across regression, classification, and image reconstruction tasks, DBVI consistently outperforms DDVI and other variational baselines in predictive accuracy, convergence speed, and posterior quality.

Published as a conference paper at ICLR 2026
DIFFUSION BRIDGE VARIATIONAL INFERENCE FOR DEEP GAUSSIAN PROCESSES
Jian Xu
Delu Zeng'
School of Mathematics
School Electronics and Information Engineering South China University of Technology & Department of Electrical and Computer Engineering University of Waterloo dl Zlzenggscuue edu.
South China University of Technology & RIKEN AIP
xian.xu@rian
Qibin Zhao
John Paisley
RIKEN AIP
Columbia University
qibin. saannnhhh@rrkeeb..
jwp2128 @ccoll umbia a edu
ABSTRACT
Deep Gaussian processes (DGPs) enable expressive hierarchical Bayesian mod- eling but pose substantial challenges for posterior inference, especially over in- ducing variables. Denoising diffusion variational inference (DDVI) addresses this by modeling the posterior as a time-reversed diffusion from simple Gaussian prior. However, DDVT's fixed unconditional starting distribution remains far from the complex true posterior, resulting in inefficient inference trajectories and slow convergence. this work, we propose Diffusion Bridge Variational Inference (DBVI), principled extension of DDVI that initiates the reverse diffusion from a learnable, data-dependent initial distribution. This initialization parameterized via amortized neural network and progressively adapted using gradients from the ELBO objective, reducing the posterior gap and improving sample efficiency. To enable scalable amortization, we design the network to operate the inducing inputs (1), which serve structured, low-dimensional summaries of the dataset and naturally align with the inducing variables' shape. DBVI retains the math- ematical elegance of DDVI- DDVI-including Girsanov-based ELBOs and reverse-timg Shis-while reinterpreting the prior via a Doob-bridged diffusion process. We derive tractable training objective under this formulation and implement DBVI for scalable inference in large-scale DGPs. Across regression, classification, and image reconstruction tasks, DBVI consistently outperforms DDVI and other vari- ational basclines in predictive accuracy, convergence speed, and posterior quality.
INTRODUCTION
Deep Gaussian processes (DGPs) (Damianou & Lawrence, 2013) extend the representational capac- ity of Gaussian processes (GPs) Seeger 2004) by composing multiple layers of latent functions, enabling flexible hierarchical Bayesian modeling. However, posterior inference in DGPs is noto- riously challenging due to the non-conjugate likelihoods, strong inter-layer dependencies, and the large number of inducing variables required for scalability. Stochastic variational inference (SVI) with inducing points (Hensman et al. [2013] Salimbeni & Deisenroth, 2017; Xu & Zeng (2024) has become the standard approach, but designing accurate and efficient variational posteriors remains central bottleneck.
Recently, denoising diffusion variational inference (DDVI) (u al. 2024c) has been proposed to approximate the inducing point posterior via the time-reversal of diffusion stochastic differential equation (SDE) (Song et al. 2020) starting from a simple Gaussian prior. By parameterizing the reverse drift with neural network, DDVI can flexibly capture complex posteriors while retaining
"Corresponding author.

---

Published as a conference paper at ICLR 2026
DDVI
DBVI
Gaussian prior U,
Input-conditioned
Reverse diffusion SDE: (f- g" sal,
Observation Doservation-comditoonedd Bridge SDE: dU, =f-=(r-0 (f-9'' Sccull, (.)l) dt dwW,
Posterior over inducing vars
Posterior over inducing vars
Figure I: Comparison between DDVI and DBVI. (Left) DDVI starts from an unconditional Gaus- sian prior and reverse diffusion SDE towards the posterior. (Right) DBVI starts from input- conditioned initial distribution and uses an observation-condiiionn diffusion bridge SDE, leading shorter and more efficient inference trajectories.
scalable SVI (Hoffman et al. 2013 Xu et al. 2024b; Chen et al., 2026b) training. However, a key limitation of DDVI lies reliance on fixed, unconditional Gaussian distribution as the start of the reverse diffusion. Since the true posterior over inducing variables is typically far from this initial distribution, the reverse-time SDE must traverse long and complex path to reach the target, resulting in inefficient inference and slow convergence.
To address this, in this work, we propose Diffusion Bridge Variational Inference (DBVD, which replaces the unconditional reverse diffusion in DDVI with a learnable, input-condittoned distri- bution that adapts over training. By parameterizing the start point of the diffusion using a neural network, gradients from the ELBO naturally push the initial distribution closer to the posterior. This effectively narrows the inference gap and alleviates the burden on the reverse SDE, yielding more observa-
stable and sample-efficient inference. Moreover, conditioning the initial distribution tions introduces a natural connection to amortized variational inference: the generative structure of DBVI allows each posterior sample to be generated by single forward pass of drift network, without requiring per-dataset optimization. Unlike naive amortization that conditions directly on raw
high-dimensional mismatches and overfitting- our design uses the inducing inputs ( each layer as input proxies the amortizer. This enables batch-wise inference while preserving global dataset structure and matching the dimensionality of inducing variables.
Importantly, the theoretical elegance of DaVi-particularl its of time-reversed SDEs, Gir- sanov's theorem (Li et al. 2020). and ELBO construction-remanne fully preserved in DBVI. We build upon the same diffusion framework, but reinterpret the prior Doob-bridged process whose start is parameterized by amortized network. This allows DBVI to inherit the key benefits of DDVI while significantly improving its flexibility and inference efficiency. We derive an evidence lower bound (ELBO) objective for training DBVI within the SVI framework and provide scalable implementation for large-scale DGPs. Empirical results on regression, classification, and image re- construction benchmarks demonstrate that DBVI achieves more accurate posterior approximations, faster convergence, and improved predictive performance compared to DDVI and other sfate-of-the- art DGP inference methods. Our contributions are as follows:
We propose Diffusion Bridge Variational Inference (DBVI), novel extension of DDVI that replaces the unconditional start of the reverse diffusion with a learnable, input-conditioned initial distribution, effectively reducing the inference gap and improving posterior approx- imation.
We introduce bridge-based reinterpretation of the DDVI framework by integrating Doob's h-transform into the variational formulation, while preserving the core machinery of reverse-time SDEs, Girsanov-basee ELBO, and SVI scalability.
We develop structured amortization strategy that leverages inducing locations ( as the input to the drift network, enabling batch-wise amortized inference without requiring access to the full dataset or high-dimensional conditioning on raw inputs.
We provide a scalable implementation of DBVI for deep Gaussian processes and validate its performance regression, classification, and image reconstruction tasks, showing sistent improvements over DDVI and other ofate-of-the-ar DGP inference methods in terms of accuracy, convergence speed, and sample efficiency.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Deep Gaussian Process Inference. Scalable DGP inference initially extended sparse varia- tional and inducing-poime GP methods to multilayer settings (Hensman et al. 2013 Damianou & Lawrence, 2013; Salimbeni & Deisenroth, 2017 Xu et al. c
b). with DSVI improv- ing practicality (Salimbeni & Deisenroth 2017) but still seeming struggle
capture complex
inducing-variable posteriors, particularly in deeper models. IPVI Yu et al. 2019) addresses expres- siveness by using a neural network represent the anducing-poin posterior and training via an adversarial, GAN-like objective (Goodiellow et al., (014);; however, this formulation appears hard to optimize in practice and can unstable, which may lead biased posterior estimates.
Diffusion-based Variational Inference. Score-based generative modeling and diffusion proba- bilistic models (Song et al. 2020; Ho et al., 2020, Li et al. 2023b; Chen et al., 2024 Li et al. a Chen et al., 2026a) have inspired new VI methods (Xu et al. (202cc) that represent pos- teriors as solutions to reverse- time SDEs. DDVI (Xu et al.) 2024c) applies this to DGP inference, parameterizing the reverse drift via score network. However, its unconditional start distribution can be far from the posterior, requiring long diffusion paths and increasing variance. Our DBVI can be viewed strict extension of DDVI: same theoretical framework, but with bridge correction and amortized initialization, yielding both theoretical guarantees and empirical improvements.
Diffusion Bridge Models. Diffusion bridges (Zhou al. 2023 Li al., 2023a Lin et al.,
2025 Chen et al. 2025) constrain dynamics between fixed endpoints or distributions, enabling more direct and sample-efficiet transitions. inservation-condiiionee bridges have been explored in Schrödinger bridge formulations (Shi et al., (023) and consistency diffusion models (He et al.)
2024). Our DBVI adapts this idea to variational inference in DGPs, integrating an amortized param-
eterization (Kim et al. 2018 Agrawal & Domke 202 Margossian & Blei 2023 Ganguly et al.
2023 to map inputs to iitiill states, reducing the KL gap and improving efficiency.
METHOD
3.1 DEEP GAUSSIAN PROCESSES
Deep Gaussian Processes (DGPs) (Damianou & Lawrence, 2013) generalize standard Gaussian Pro- cesses (GPs) by hierarchically composing multiple GP layers, enabling deep non non-linear probabilistic mappings. Let E Rd be input and e R the corresponding output. A DGP with L layers defines latent variables (f() 1 recursively through:
p((() (()- GP (0, k (P(-1), (((-))))(11))) r(-1)))
(1)
where 1((1) is the kernel function layer | with hyperparameters (1) For scalability, each layer introduces Mj inducing variables (() located at inducing inputs Z ((), with GP prior:
Pprior (1) (0.Kk)
Assuming conditional independence across layers given inducing variables, the full joint distribution over outputs y, latent variables (, and inducing variables {u')} factorizes as:
ply,F,U F. U)
[(( (()1)) =n)) p(u ply p(1)),
(3)
where the inter-layer conditional distribution p(P() p(l-1), (()) given by the sparse GP condi- tional distribution
(((()) 1(1-1) ())
(4)
N CN(KpAM (KrInnZa Z ()), Keu-nga-n K Kanpt Kzoppu-))
Here, we denote k. as the kernel matrix at layer evaluated between sets and b. The input to the first layer is defined as p())
3

---

Published as a conference paper at ICLR 2026
3.2 DENOISING DIFFUSION VARIATIONAL INFERENCE (DDVI)
Denoising Diffusson Variational Inference (DDVI) et et al. 2024c) is recently proposed method for performing posterior inference over inducing variables Deep Gaussian Processes (DGPs). draws inspiration from the success of score-based generative modeling and diffusion probabilistic models (Ho et al. 2020; Song al. 2020), and adapts these ideas to variational inference by model- ing the variational posterior as the solution to reverse-time stochastic differential equation (SDE). Traditional variational inference in DGPs typically relies on simple, factorized Gaussian approxi- mations over the inducing variables U {u 122 which are often too restrictive to capture the complex, multimodal posterior arising from deep hierarchical GPs. DDVI seeks to construct more expressive variational distributions by modeling them as the terminal distribution of a reverse-lime diffusion process, effectively defining flexible transformation from known fixed initial distribu- tion the posterior.
Variational posterior via reverse diffusion. DDVI defines the variational distribution as the marginal at time t = of a reverse-time SDE (U,):
du, [S(U,,)) V (. di dW) LE [0.
(5)
where Ug ~ (0, I) is the fixed initial distribution, f and are drift and diffusion coefficients, and Pe pe(U,, (U,) is the marginal law at time under the reverse process. The reverse SDE is not simulated directly; instead, the score function Vu, log Pe (U,) in the drift is parameterized by neural network (U,,)) t) to approximate the optimal reverse dynamies.
Diffusion-based ELBO. Rather than relying score estimation (e.g.. denoising score match- ing), DDVI employs a variational diffusion framework to derive a tractable evidence lower bound (ELBO). Specifically, the reverse-time process U1e is treated variational diffusion process, and the ELBO is expressed using the Girsanov formula for likelihood ratios between stochastic processes:
1 2 /
CDDVI = BUaINQ
[-
1 | 12 20
logo y (())
s(t))) + st U dt
+ Pprior (U,) KI, D k 1))
(6)
where Qa denotes the pathwise density of the variational reverse-time SDE, U, is the terminal state of the reverse diffusion SDE using (o(,t), t), f()) denotes the forward inference of the DGP at U,.
Training and implementation. In practice, DDVI jointly trains the variational drift network 85 and DGP hyperparameters y by maximizing LDDVI using stochastic gradient descent. Sampling from (U,) is achieved by solving the reverse SDE from Un ~ o²),, which allows for reparameterized gradients through the sampled trajectories.
Limitations. Although DDVI offers flexible and theoretically grounded approach for inference in deep GPs,, it still seems to face two key drawbacks: first, it uses an unconditional Gaussian initial- ization Ua o²) for the reverse diffusion, which typically appears far from the true posterior over inducing variables, so the reverse-time SDE must follow a long, complex trajectory to reach the target distribution, resulting in inefficient inference, higher variance, and slower convergence; sec- ond, this initialization is not conditioned on observations, so sampling remains input-agnostic rather than amortized and scalable. These limitations motivate our Diffusion Bridge Variational Inference (DBV VI), which introduces observation observation-condiitonnee conditioned diffusion bridges and amortized initializations to enable more efficient and accurate posterior inference.
3.3 VI: OISERVATIONNCONDIIONNE DIFFUSION BRIDGE
We begin by introducing the data-dependent initialization of DBVI. Instead of starting from fixed Gaussian prior as in DDVI, we amortize the mean of the initial distribution using neural network:
T x) -VU0; no oT),
4

---

Published as a conference paper at ICLR 2026
where only the mean [x( depends the data, while the variance x² is kept fixed. This initializa- tion provides a closer match to the posterior and shortens the diffusion trajectory. To formalize the resulting dynamics,
turn diffusion bridge representation.
Proposition 1 (Forward & Reverse SDE under Doob's h-transform). Let the initial constraint be encoded by the Doob f--transform with
,,,, Uo) = Vu, logp(U) U,),
(8)
Then the forward bridge has drift
f(U,,,, = f(U,,t) + g(t)² 2,,, Uo),
(9)
with the same diffusion coefficient g(t). Moreover, the reverse-time bridge SDE is
au, [[( g(t)² soond U2, t, Uod dt + d()
(10)
Equivalently, the conditional score equals Scond U,, Up) = t Uo) + h(U,, t, Uo).
Proposition states that, by introducing Doob's h-transform, we can reinterpret the dynamics as bridge process, which essentially bends the diffusion toward the posterior endpoint. The forward SDE incorporates an additional drift term that nudges the path toward the target, while the reverse- time bridge SDE involves conditional score function Scond. This result provides the mathematical foundation for DBVI, showing how conditioning on the initialization modifies both forward and reverse dynamics.
Building on this, we next leverage the bridge process trick introduced DDVI Xu et al. (002cc) characterize the marginal distribution of the bridge process under linear drift. This formulation yields tractable Gaussian form for the bridge marginal, which will be crucial for deriving the DBVI training objective.
Proposition 2 (Marginal of Doob-augmented bridge process). with Doob bridge correction
Consider the linear forward SDE
dU, [- X(t) U, + h(U,, Un) + g(t) dB.)
(11)
where h(U,, Uo) Vu, logp( Un U²) is the Doob h--rrassfrm. Assume isotropic initialization
( x) (No(x)), I²I).
(12)
Then for each & E [0, 1}. the marginal law remains Gaussian,
Pu x) = NU m. Ki I), where the mean my and variance 64 satisfy the coupled ODE system
(13)
d
= - (A(t) + c(t)) m, + a(t) (x(x): mo =x)(x),
(14)
dt
dt Kr = 2(((1) + c(t) K4 K4 =(()) ((1))
Ko = 0²,
(15)
with
e(1)= A(t) S' (s)ds,
and correction coefficient
0² a(t) (a q(t) q(t)'
a(1) a(1)² [
g(r)²
c(l) = =
dr.
a(r)²
In the special e(t) this reduces to the bridge process trick in DDVI.
5

---

Published as a conference paper at ICLR 2026
Proposition 2 provides closed-fome expression for the bridge marginal, with mean my and vari- ance K2 determined by the amortized initialization [x)) This Gaussian form plays a crucial role in deriving the training objective, as it enables explicit comparison between the amortized start dis- tribution and the bridge marginal. In particular, it provides the analytical structure needed to express the KI. divergence in a tractable score-matching form, which then incorporated into the DBVI loss. Additional derivations and detailed proofs are provided in the Appendix. Finally, we combine the above results to obtain the DBVI training loss. In particular, we express the KL divergence in score merthma form, yielding tractable ELBO that can be optimized efficiently.
Proposition 3 (DBVI loss with amortized mean). Let pg(u x) = [N (x)((x), I be the data- dependent start, and let (m, K4) be the mean/variance of the reference bridge marginal at time t [0, 1] (given by Proposition via ODEs in the Doob- moob-augmented case). Then the pathwise KL between the variational reverse bridge Qe and the reference bridge admits a score-matching representation. Consequently, tractable per-mini-batch ELBO is
(DVII 0.7) = EUa.1QQ
log log (U,) + logp(y: (
1
Ri (U, (U)) + Scoud (t. Uc, Uo) || dt
2 J
1 g(t)
(16)
+ log Pprioe (U1) KL (N(uo(x), 021) || N(mi, Ki 1))
where is the batch size, N the dataset size, Scond = 86 + h is the conditional score used by the reverse bridge SDE, U1 is its terminal state, ((L) denotes the DGP forward mapping at U1, and collects DGP hyperpurameters. When p*(x) = 0 (and thus m, 0), the objective recovers the original DDVI loss.
Proposition 3 shows that DBVI departs from DDVI in two essential ways: (i) the initialization is amortized ng (x), which induces time-dependent reference mean m, in the bridge marginal, and (ii) the loss involves the conditional score Scond = sy + h, explicitly accounting for the bridge correction. When the amortized mean collapses to zero (so that m, 0). the objective reduces
exactly to the original DDVI loss, recovering DDVI as special case. We summarize the full training procedure DBVI for deep Gaussian processes in Algorithm 1
DEVI Amortized Initialization using Inducing Inputs
Amort bad network ug(c gfw 2/w
Induring iyuee
Mean
shee: D5 1p) Degresetoiieeeeee
Shape: D1, fual
Fived vatianos
Stxrt ritrrstion p. 911/125 (,tc)),
Hoveras briqge SEIE Susa 5 Paw obee Pentherton-notte conditiond
tn RGP by Barihoossg Dapset over layers in
Inducing varlables V 2-4 i?)
Figure 2: DBVI amortized initialization using inducing inputs Z()
3.4 STRUCTURE OF THE AMORTIZED NETWORK μt9
In DBVI, the amortized network μ4 provides the parameters of the initial distribution P UU x). Ideally, it would take the full dataset x input and output variational parameters for the inducing variables U. In practice, this is infeasible: full-dataset amortization is prohibitively expensive in both memory and computation, while mini-batch amortization observes only small subset of the

---

Published as a conference paper at ICLR 2026
Algorithm Diffusion Bridge Variational Inference (DBVI) for DGPs
Input:
Training data y; mini-batch size B; forward drift f; diffusion scale
Parameters: DGP hyperparameters y bridge score network søi amortizer va(+) with fixed variance o² parameter
Precompute reference bridge marginals (m, R4) for t E [0, 1]: numerically integrate the ODEs for (m, K4) in Prop. 2 with att) e A(1)
repeat
Sample mini-batch indices / C (1,.... N) with 11| B Amortized start: draw Up (. X1) = N (No 0²1) Initialize the integral accumulator Lo for k = 0 to K - do
Set < K 1.+1 < * draw G, I(O,T)
Compute conditional score: Scond (ts, U,,, Uo) t se (te: U1,,X1,y11 + h(te, Un.: Uo) Reverse bridge SDE step:
Ut U. - f(U,, (.) At + g(t,)² Scond (ts, U. Uo) At + g(t.)v A.
Bridge marginal update: Obtain m Ktat1 and accumulate score- motch-na term:
U M
L Lt, g(ta+z)?
+ Fcond(ta+)) UL44 Un) At
K1.41
end for
Set {u 17
1/=1 + U1
for 8 = 1 to do
Draw e ~ (O, and compute
p(() = K FZ (K 22
(0) + (K(9 (iee-upee-)) - ((0)
(K (8) -1 'k(9
(2
D2
2
end for
Mini-batch ELBO:
(DBVI(0, log po [u1 X1) + log Pprior (us) + N
logply: F(L))
B
1 2
Li (L (((mm,m11)))
Gradient update of (BV until 0, Φ converge
data and can therefore yield biased or unstable parameter estimates. Moreover, there is a fundamen- tal dimensional mismatch: mini-batch inputs have shape [B, dul, whereas the inducing variables layer / lie in [Mr dou)]. Directly mapping x (!!) would require flattening high-dimensional tensors, which breaks efficient batching and scales poorly with model depth.
To avoid these issues, we use the inducing points ( as inputs to the amortizer. This follows sparse GP intuition: Z can be viewed representative features of the dataset. At layer I, the inducing inputs (1) E Md are already aligned with the size of (!) We therefore define layer-wise network
11) Rd R 14
which maps each inducing input ( to a deur-iimession representation. Applying it to all induc- ing points gives M4 ( (z) E I8 xddnn, This choice leverages information encoded in ( which updated during training, and yields an amortization scheme whose output dimension naturally matches the inducing variables, without requiring access to the full dataset.
4 EXPERIMENTS
We empirically evaluate DBVI against recent state-of-the-an inference methods for Deep Gaus- sian Processes (DGPs). Our evaluation covers regression on UCI benchmarks, image classifica- tion standard vision datasets, large-scale physics datasets, and unsupervised reconstruction
7

---

Published as a conference paper at ICLR 2026
21e33 pwibt
sewgy Borks Doa)
Aso (le330 Deet
Oeerdi (862230 Celt
Jahe 180203755 OWN
06127
09033
090099
001371
1b-13200
050031
BM107
0000F
Figure 3: Test mean NLL (with one standard deviation error bars) of deep Gaussian processes with different inference methods (DDVI, IPVI, SGHMC, DSVI, and our proposed DBVI) across 10 benchmark datasets.
Table Mean test accuracy (%) and training details achieved by DSVI, SGHMC, IPVI, DDVI, and proposed DBVI three image classification datasets. Results shown for 3 and 4 layers as indicated, and runtime is given per iteration.
Data Set
Model
Time3
Iter3
Acc3
Time4
Iterd
Ace4
DSVI
0.34s
20K
97.17
0.54s
20K
97.41
MNIST
IPVI
0.49s
20K
97.58
0.62s
20K
97.80
SGHMC
1.14s
20K
97.25
1.22s
20K
97.55
DDVI
0.38s
20K
98.84
0.50s
20K
99.01
DBVI (ours)
0,41s
20K
99.02
0.55s
20K
99.10
DSVI
0.34s
20K
87.45
0.50s
20K
87.99
Fashion
IPVI
0,48s
20K
88.23
0.61s
20K
88.90
SGHMC
1.21s
20K
86.88
1,25s
20K
87.08
DDVI
0.40s
20K
90.36
0.55s
20K
90.85
DBVI (ours)
0.43s
20K
90.53
0.60s
20K
91.07
DSVI
0.43s
20K
91.47
0.66s
20K
91.79
CIFAR-10
IPVI
0.62s
20K
92.79
0.78s
20K
93.52
SGHMC
8.04s
20K
92.62
8.61s
20K
92.94
DDVI
0.45s
20K
95.23
0.69s
20K
95.56
DBVI (ours)
0.49s
20K
95.42
0.74s
20K
95.68
task. Across these diverse settings, we assess both predictive performance and posterior quality, with particular attention to convergence behavior and scalability. The goal of our experiments is to demonstrate that DBVI consistently improves predictive accuracy and uncertainty estimation while remaining computationally efficient.
8

---

Published as a conference paper at ICLR 2026
Table 2: Test AUC values for large-scale classification datasets. Uses random 90% / 10% training and test splits.
SUSY
HIGGS
N D
5,500,000
11,000,000
18
28
L = 2
0.876
0.830
DSVI M 128
L 3
0.877
0.837
L =
0.878
0.841
L 5
0.878
0.846
L 2
0.879
0.843
IPVI M 128
L = 3
0.882
0.847
I.
0.883
0.850
I. 5
0.883
0.852
L 2
0.879
0.842
SGHMC M = 128
I = 3
0.881
0.846
L 4
0.883
0.850
0.884 L = 5
0.853
L 2
0.883
0.849
DDVI 128
L = 3
0.885
0.852
I 4
0.887
0.856
0.886 L = 5
0.857
I, = 2
0,885
0.851
DBVI (ours)
L 3
0.887
0.854
M = 128
L = 4
0.889
0.858
L 0.889
0.859
4.1 BASELINES AND SETUP
We compare DBVI with the following baselines: DSVI (Salimbeni & Deisenroth 2017). the stan- dard mean-field Gaussian variational approximation; IPVI (Yu et 2019),, which parameterizes the posterior with neural network trained adversarially; SGHMC (Havasi et al. 2018), a sampling- based inference approach; and DDVI (Xu et al. 2024c), the diffusion- basedd inference method upon which DBVI builds.
All models use RBF kernels and 128 inducing points per layer unless otherwise specified. For fairness, we adopt identical initialization and hyperparameter ranges across methods, and optimize using Adam with learning rate 0.01.
4.2 REGRESSION ON UCI BENCHMARKS
We evaluate our method on 10 widely used UCI regression datasets with sample sizes N ranging from a few hundred to over two million, using an 80/20 train/test split. We report root mean squared enror (RMSE) and negative log- likelihoodd (NLL)) on the held-out data, as summarized in Figure and Figure 4 Consistent with prior work, we consider deep Gaussian processes with 2-5 layers. The results demonstrate that DBVI consistently outperforms all baseline methods, with particularly pronounced gains large-scale datasets such YearMSD and Airline, where unconditional DDVI suffers from slow convergence. By leveraging amortized bridge initialization, DBVI effectively shortens the diffusion path length, resulting in both improved posterior approximation and lower predictive error. Figures [] and show the test RMSE 2-layer DGP trained with DDVI and DBVI during the early stage of optimization.

---

Published as a conference paper at ICLR 2026
Table Mean RMSE and NLL achieved by DSVI. SGHMC, IPVI, DDVI, and our proposed DBVI
on the GPLVM data recovery task (Frey Faces). Standard deviation is shown in parentheses. Run- time is given per iteration.
Data Set
Model
Time
Iter
RMSE
NLL
DSVI
0.32s
20K
8.32 (0.20)
1.49 (0.02)
Frey Faces
IPVI
0,42s
20K
7.91 (0.40)
1.33 (0.02)
SGHMC
1.13s
20K
(0.30)
1.36 (0.03)
DDVI
0.36s
20K
(0.20)
1.17 (0.01)
DBVI (ours)
0.40s
20K
7.52 (0.18)
1.12 (0.01)
4.3 IMAGE CLASSIFICATION
We next evaluate DBVI on MNIST, Fashion-MNIST, and CIFAR-10. For CIFAR-10 10, we adopt ResNet-20 convolutional features as inputs to the DGP classifier. We report both test accuracy and per-iteration runtime in Table Across all three datasets, DBVI consistently surpasses DDVI and other baselines. In particular, on CIFAR-10 with 4-layer DGPs, DBVI achieves an accuracy of 95. 15.68% slightly higher than DDVI's 95.56%. These findings underscore the advantage of amortized conditioning in handling complex high-dimensional posteriors.
4.4 LARGE-SCALE CLASSIFICATION
We further evaluate DBVI two large-scale physics datasets, SUSY (5.5M examples) and HIGGS (11M examples). We report AUC scores with 2-5 layer DGPs under random 90/10 train-test splits. As shown in Table 2 DBVI consistently outperforms DDVI across all depths, yielding the best overall AUC values. Compared to SGHMC, DBVI attains comparable or higher performance while being substantially more computationally efficient, highlighting its scalability and effectiveness in modeling complex, high-dimensional posteriors at scale.
4.5 UNSUPERVISED RECONSTRUCTION
Finally, we evaluate posterior quality on the Frey Faces dataset using a missing-data reconstruction task. Following prior work, randomly mask 75% of pixels for subset of training images and task the models with recovering the originals. Table 3 reports reconstruction RMSE and test log- likelihood. DBVI achieves the lowest RMSE and highest likelihood, surpassing DDVI well as variational and sampling-bane -aase baselines. In qualitative comparisons, reconstructions generated by DBVI are visually sharper and demonstrate better calibrated uncertainty than those competing methods.
5 CONCLUSION
We introduced Diffusion Bridge Variational Inference (DBVI), a principled extension of DDVI that initiates the reverse diffusion from a learnable, input-conditioned distribution. By incorporating Doob's 's h-transform into the variational formulation, DBVI preserves the theoretical elegance of cestusion-basee inference while substantially reducing the inference gap. Our structured amorti- zation strategy, which conditions on inducing inputs, further enables scalable and data-efficient posterior approximation in deep Gaussian processes. Empirical results across regression, classi- fication, and image reconstruction tasks confirm that DBVI consistently improves over DDVI and other state-of-the-art variational baselines in predictive accuracy, convergence speed, and sample efficiency. These findings highlight the benefits of bridging- learning. Future directions include extending DB to broader probabilistie models, integrating with advanced inducing point strategies for large-scale applications, and exploring theoretical guarantees of diffusion bridges in variational inference.
inference for scalable Bayesian
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported in part by grants from National Natural Science Foundation of China (52539005). the fundamental research program of Guangdong, China (2023A151501 1281), the China Scholarship Council (202306150167), Guangdong Basic and Applied Basic Research Foun- dation
Foshan Science and Technology Research Project(222000111 8608).
REFERENCES
Abhinav Agrawal and Justin Domke. Amortized variational inference for simple hierarchical mod- cls. Advances Neural Information Processing Systems, 34:21388-21333 2021.
Wei Chen, Shigui Li, Jiacheng Li, Junmei Yang, John Paisley, and Delu Zeng. Dequantified diffusion schrödinger bridge for density ratio estimation. arXiv preprint arXivv2500..05004, 2025.
Wei Chen, Jiacheng Li, Shigui Li, Zhiqi Lin, Junmei Yang. John Paisley, and Delu Zeng. Don't forget its variance! the minimum path variance principle for accurate and stable score-based density ratio estimation. arXiv preprint arXiv:2602.000 00834, 2026a.
Zhichao Chen, Haoxuan Li, Fangyikang Wang, Odin Zhang, Hu Xu, Xiaoyu Jiang, Zhihuan Song, and Hao Wang. Rethinking the diffusion models for missing data imputation: A gradient flow perspective. Proc. Adv. Neural Inf. Process. Syst., volume 37, pp. 112000-112103, 2024.
Zhichao Chen, Yulong Zhang. Odin Zhang, Fangyikang Wang, Le Yao, Hao Wang, and Zhihuan Song. Blending data and knowledge for process industrial modeling under riemannian pre- conditioned bayesian framework. IEEE Trans. Knowl. Data Eng., 38(1):82 95, 2026b. doi: 10.1109/TKDE..225..222
Andreas Damianou and Neil Lawrence. Deep Gaussian processes. In Conference Artificial Intelligence and Statistics, pp. 207-215, 2013.
Ankush Ganguly, Sanjana Jain, and Ukrit Watchareeruetai. Amortized variational inference: A systematic review. Journal of Artificial Intelligence Research, 78:167-215, 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Marton Havasi, José Miguel Hernández-Lobato, and Juan José Murillo-Fuentes. Inference in deep Gaussian processes using stochastic gradient Hamiltonian Monte Carlo. In Conference on Neural Information Processing Systems, pp. 7517-7527, 2018.
Guande He, Kaiwen Zheng, Jianfei Chen, Fan Bao, and Jun Zhu. Consistency diffusion bridge models. Advances in Neural Information Processing Systems, 37:23516-23548, 2024.
James Hensman, Nicolo Fusi, and Neil D Lawrence. Gaussian processes for big data. arXiv preprint arXiv:1309.66 6835, 2013.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley, Stochastic variational inference. Journal of Machine Learning Research, 140)):33331337, 2013.
Yoon Kim, Sam Wiseman, Andrew Miller, David Sontag, and Alexander Rush. Semi-amortized variational autoencoders. In International Conference Machine Learning, pp. PMLR. 2018.
Bo Li, Kaitao Xue, Bin Liu, and Yu-Kun Lai. Bbdm: Image-to-image translation with brownian bridge diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern Recognition, pp. 1952-1961, 2023a.

---

Published as a conference paper at ICLR 2026
Jiacheng Li, Wei Chen, Yican Liu, Junmei Yang. Zhiheng Zhou, and Delu Zeng. Generative self- supervised time series forecasting leveraging wavelet diffusion. IEEE Transactions on Instrumen- tation and Measurement, 2025a.
Shigui Li, Wei Chen, and Delu Zeng. Scire-solver: Accelerating diffusion models sampling by score-integrand solver with recursive difference. arXiv preprint Sktv::330,,088900 2023b.
Shigui Li, Wei Chen, and Delu Zeng. Evodiff: Entropy-aware variance optimized diffusion infer- ence. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b.
Xuechen Li, Ting-Kam Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pp. 3870-3882. PMLR, 2020.
Zhiqi Lin, Wei Chen, Jian Xu, Delu Zeng, and Min Chen. Reciprocalla-liee Low-light image enhancement with luminance-awee reciprocal diffusion process. Neurocomputing, pp. 131438, 2025.
Charles C Margossian and David M Blei. Amortized variational inference: When and why? arXiv preprint arXiv:2307.1 1018, 2023.
Hugh Salimbeni and Mare Deisenroth. Doubly stochastic variational inference for deep Gaussian processes. In Conference Neural Information Processing Systems, pp. 4588 4599, 2017.
Matthias Seeger. Gaussian processes for machine learning. International journal of neural systems, 14(02))69-106, 2004.
Yuyang Shi, Valentin De Bortoli, Andrew Camphell, and Arnaud Doucet. Diffusion schrödinger bridge matching. Advances in Neural Information Processing Systems, 36:62183-62223, 2023.
Yang Song. Jascha Sohl-Dickstein, Diederík P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:201 1. 13456, 2020.
Jian Xu and Delu Zeng. Sparse variational student-t processes. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 38. pp. 16156-16163, 2024.
Jian Xu, Shian Du Junmei Yang, Qianli Ma, and Delu Zeng. Neural operator variational inference based on regularized stein discrepancy for deep gaussian processes. IEEE Trunsactions on Neural Networks and Learning Systems, 2024a.
Jian Xu, Shian Du, Junmei Yang, Qianli Ma, and Delu Zeng. Variational learning of gaussian process latent variable models through stochastic gradient annealed importance sampling. arXiv preprint arXiv:2408.066 060000,061,,.. 10, 2024b.
Jian Xu, Delu Zeng, and John Paisley. Sparse inducing points in deep gaussian processes: Enhancing modeling with denoising diffusion variational inference. In International Conference on Machine Learning. pp.
PMLR, 2024c.
Jian Xu, Shian Du Junmei Yang. Xinghao Ding, Delu Zeng, and John Paisley, Bayesian gaussian process odes via double normalizing flows. In The 28th International Conference on Artificial Intelligence and Statistics, 2025a.
Jian Xu, Zhiqi Lin, Min Chen, Junmei Yang, Delu Zeng, and John Paisley. Fully bayesian differential gaussian processes through stochastic differential equations. Knowledge-Bard -Based Systems, 314: 113187, 2025b.
Haibin Yu, Yizhou Chen, Bryan Kian Hsiang Low, Patrick Jaillet, and Zhongxiang Dai. Implicit posterior variational inference for deep gaussian processes. Advances in neural information pro- cessing systems, 32, 2019.
Linqi Zhou, Aaron Lou, Samar Khanna, and Stefano Ermon. Denoising diffusion bridge models. arXiv preprint 239.:/309//694 2023.
12

---

Published as a conference paper at ICLR 2026
A PROOF OF PROPOSITIONS
Proposition A.1 (Forward & reverse SDE under Doob's h--rransform). Let the initial constraint be encoded by the Doob h--transform with
,, = Vu, log U,),
(17)
Then the forward bridge has drift
f(U,,,, Uo) = f(U,.t) + g(t)² t,,, Uo),
(18)
with the same diffusion coefficient g(t). Moreover, the reverse-time bridge SDE is
au,- t() g(t)² scamd Ut, t, [o) dt g(t) (W) Equivalently, the conditional score equals coodd U Uo) s(U,, Uo) + h (U,, Uo).
(19)
Proof. Setup. Consider the forward diffusion on [0, 1].
au,- = f(U,, dt g dB,
(20)
where B, is standard Brownian motion, and g(() > 0 is scalar. Assume standard regularity (existence/uniqueness, non-degeneracy, smooth strictly positive densities). Let pt denote the time-l marginal density of equation 20 The (backward) Kolmogorov operator is
(L) = f'u, () V.p(u)
+ Ap(u).
g(t)²
(21)
2
(i) Doob h-transform with an initial-point constraint. Define the space time positive function
Htt, u) == P(U, U, = =)).
(22)
By the Markov property and Chapman-Kolmogoroy H is space- time harmonic for a + Lt, i.e.
(1H[(,, u) + (C, u) 0.
(23)
The Doob H-transform equation 20 is the time-inhomogeneous Markov process with generator
(c (u) = (t, u) (L))
(24)
Expanding (HH))) (see remarks below) and using equation 23 to cancel the 0,H term yields, for smooth i
cl (u) = f(u, t). Vv(()= + 5(t)² 2 Ap(u) + g())² Vlog Ht,, u) Viplu)
(25)
Therefore, the transformed process again Itô diffusion with the same diffusion coefficient g(t) and drift
f(u,,, Un) = g(t)² Vu ((t f(u,t) g(t) V. log (U u).
(26)
With the proposition's 's definition
,U,,t, Up) = Vu, logp( U.). l,),
(27)
we obtain the forward bridge SDE
du, [[(,,,) A(U,,, Up) dt g(() dB,
(28)
ie. f(U,,,, U.) = S(U,, + ((()))( Un) with unchanged diffusion g(().
(ii) Reverse-time SDE of the initial-point bridge. p: p(U, u Uo) denote the time-t conditional density of the bridge. By the Haussmann Pardooux time-reversal formula for diffusions with isotropic diffusion matrix a()) I (so a = 0), the rive-seversed bridge T U1- solves
aU.- - Uo) - g(1 - 1) log ( di + g(1 - 1) aw..
(U.)
(29)
13

---

Published as a conference paper at ICLR 2026
Re-indexing t > t and renaming the Brownian motion gives the reverse time SDE in forward
orientation:
dU, [(U,, Uo) (o + + 2(1) dw,.
(30)
Substituting f(U,, t, Up) -
Uo) yields
du, [t(,,,t) + ggt)) g Vog
log (U,) dt g(t) dW,
(31)
Define the conditional score
Scond (U, Ub) == Vu, log (U,) Vu, log Un),
(32)
to obtain the claimed reverse-time bridge SDE:
du,- f(U,,t) (t)) scond (U, Uo) at d.,
(33)
(iii) Conditional score identity (for the initial-point constraint). By Bayes' rule,
log U₁ Un) log Un U²) (U,) log p(u))
(34)
Taking Vu, gives
000077
09119)
0003:
0009
89119
07019
000027
06003P
(4-330 0-0)
(5w10330 2w2)
Krhr (90032 DeXt
0001331
(W133)
WIEFF
(WIEP)
Figure 4: Test MSE (with one standard deviation error bars) of deep Gaussian processes with differ- ent inference methods (DDVI, IPVI, SGHMC, DSVI, and our proposed DBVI) across 10 benchmark datasets.
S Scond t,,, t, Un) Vu, (U,,)(UU) + A(U,, Un).
(35)
If we write the unconditional score as s (U,, 1) = Vu, log (U)), this is
Scond + h.
(36)
14

---

Published as a conference paper at ICLR 2026
Remarks:
explicit derivation for Doob H-transform generator. Fix 0 < S I 1. The transformed semigroup acting on a test function yp is
H(s,u) B (.) [U U, =u]
(37)
The (backward) generator C is defined by
a, (T))))()) (TH f (u))
with T ld.
(38)
Applying Itô the product H1,, Uz) (PU,) under the forward SDE 20 and taking conditional pectations gives
E HCL (U.) U, = =E(==]((0 (H) U, (U=w) u)
(39)
di
By the space-time harmonicity of H,
(8, + (t) 0,
(40)
we can expand (0, + (Hp)) and cancel the 0,H terms cleanly. Since the backward generator is (w) ((b()))V + a(t)² Avp(u),
(41)
the product rules for gradient and Laplacian give
V(H₂) H V VH,
A(H+)= +
(42)
Hence
((H1))= f- (V)()) 2012 A(H+))
(43)
= H (J o0 Ap) + O (MVV
2002 AH) + +
(44)
=Le
=c,8
H Lup + HLH + g())
(45)
Therefore,
(0. (H(p) (0.H) HOp + H + + VH
VH
(46)
=11 + L.)y + ((0, + (.) & + g(t)² VH-Ve
(47)
by equation
30
(0, + L.) + g(1) V..VP.
(48)
Dividing by Ht, u) (i.e., multiplying by the scalar (t,, u) !) gives the pointwise identity
H (a: + (.)()))) = (0. + L.) + g(t) Vlog H Vvp.
(49)
Comparing with the definition of the transformed semigroup TH (which removes the explicit 0 on the left via the semigroup relation), we identify the buckward generator of the H-transform at time l as
(C (u) (u) (() Vlog H(t, Vy(u).
(50)
Since H(t, u) PUO U. u), writing h(U,,t Un) Vu, log U.) yields the drift correction
f Uo) f(U,,t) + g(t)² hU,, Uo). with the same diffusion coefficient gt)).
(51)
Proposition A.2 (Marginal of Doob-augmenter bridge process). Consider the linear forward SDE with Doob bridge correction
au, - X(1) U, + (U,,, Un)) di + g(() dB.,
15

---

Published as a conference paper at ICLR 2026
where h(U,,t,U,) Vu, log (U Ut) is the Doob h-transform. Assume isotropic initialization
p (o x) = -N d²1).
Then for each E [0, 11, the marginal law remains Gaussian,
pr (T x) -N mi, Kt I),
where the mean m, and variance Kig satisfy the coupled ODE system
dt m, = (2(t) + c(t)) m. + +(t) a(t) px(x); ma = (<((x)
(52)
dt Kr = + - Kr + g(t)² +
NO = o²,
(53)
with
A(t) = S( 2(s)
= a(t)=e-Mo)
and correction coefficient
d² a(t))
(t) = a(t) S g(r)
ctt) = g(t)² (alt)202 a()) q(1)'
a(r)² dr.
In the special case c(t) = 0, this reduces to the bridge process trick DDVI..
Proof. We work in the isotropic, SDE is
setting adopted in the main text: the base forward
au, = -tt) U, dt + g(t) dB, with =e())=-AA0) and A(t) fo X(s) ds. In this OU setting,
Uo ~ ~()((x) o²T),
(54)
E[U,] a(t) (x(x,, Cov(U.) a(t)20²1 + I(t)
(55)
where
q()) = a(t) L-' g(r)2
a(r)² dr.
(56)
Moreover, the joint law of (Uo. U.) is Gaussian with
Cov U.) o² at)
(57)
These identities are standard and match the bridge process used in the DDVI paper.
Step 1: Conditional p(U. U.). By joint-Gaussian conditioning,
PUU U.) = N( moje: Sotz
(58)
with isotropic (scalar) parameters
maje (x) + k(t)) (U, (xt))
Sojo
I(t)
(59)
where
d2a(t) K(t) a(t)2o² q(t)'
0a((1)²
d²q(1)
S(t) 0²
(60)
a(t)20² + q(t)
(1))20++9(1
Step 2: Doob h-transform (bridge) term. Define the Doob correction
h(U,, 6, Un) = Vu, pog U.).
(61)
Since PUU, U,) (Mmoj S()))) and maje is affine in U, with coefficient K(III, have
Vu, lU, U.) (Vu. T s(t) (Uo mojt) = K(I)
s(t) (U, mmjz).
(62)
Hence the aoob-augmented forward SDE reads
dU. = | A(t) U, + g(t)² K(t) S(t)
(U, - molc) + g(t) dB,
(+)
16

---

Published as a conference paper at ICLR 2026
Step 3: Mean dynamies. Let := E[U,]. Taking expectations in (+) and using B[u [o(x
and maje px(()) (U)) a())uo(x)) gives
E
Uo mo po(x) (uo(x) + (m) a(t)uo(x))) {(t)(a(t)uo(x)- m.). (63)
Therefore,
d d
m, -(t) m. + g(t)² K(t)
K(t) (a(t)uo(x) m.).
(64)
5(1)
Introduce the scalar
c(l) - a(1) K(t)²
o²a(t)²
g(1)²
(65)
S(1)
(a(t)202 + (()
which yields
d m, (A(1) + +c(() m. c(()a(() px(x);
mo- (x)())
(66)
dt
Step 4: Variance dynamics. Let denote the (isotropic) variance so that Cov(U,) = KzI. From Itō's lemma for U.U. under (+) and isotropy.
d dt Ki -21() Ke. + s(1)² 2g(1)² K(t) 5(1)
Cov (U,, Uo -mojj)
(67)
Using Cov (U, Uo) a(l)a²I and moje - [x( + (())(U( a(l))o(x))) we get
Cov (U, moj:) k(t) COv(U,, U,) = K(t)me I
(68)
hence
Cov(Ur, Uo molt) = K(I)xeL
(69)
Plugging in and using c(l) = -)))))(()///(())
dt Ki 22(() me g(t)2 + c(t)
d
K(t) (alt)o² -K(()))(()==== =- -(()())+((t))<z + +(t)² =c(())
(70)
with Kig 0².
Step 5: Gaussian form. The drift in (x) is affine in (U,, Uo), and the driving noise is Gaussian. Therefore (U,, Uo) remains jointly Gaussian, and the marginal pr(U (UBa x) is Gaussian with mean m, and variance KgI. This proves the claimed form and the coupled ODE system. In the special case c(t) = 0 (no Doob correction), the system reduces to the closed form bridge process in DDVI g()()) paper, i.e., m = (x)) and K2 = o² 2 + a()) S arr)
Proposition A.3 (DBVI loss with amortized mean). Let p x) = N o'I) be the data- dependent start. and let (m,, K2) be the mean/variance of the reference bridge marginal at time t E [0, 1] (given by Proposition 2 via ODEs in the Doob- noob-augmented case). Then the pathwise KI. between the variational reverse bridge Qo and the reference bridge admits a score-matching representation. Consequently, tractable per per-mini-bane
ELBO is
(D 2, = EUa.r~a [
log log ()) + logdyy: ((())
N
B
I. 2
1
U. m.
+ scone(t, (, U., Un) | dt
9(1)2
Ktt
o2I) || Nmii, Ko 1)) + log Pprior (U1) (L(())o()))
(71)
where B is the batch size, N is the dataset size, Scond = 56 + h is the conditional score used by the reverse bridge SDE, U, is its terminal state, (()) denotes the DGP forward mapping at U1. and y collects DGP hyperparameters. When px(x)) = 0 (and thus m, = 0), the objective recovers the original DDVI loss.
17

---

Published as a conference paper at ICLR 2026
Proof. Setup. Let Qo be the path law of the reverse-time time bridge SDE
du, = [5(U,.)) g(1) seona U., Uo) dt g())
Scond := 866 + h,
with ,,, Un) Vu, log P(U U.). Let P be the forward reference diffusion and PBri be the auxiliary forward process that shares the same drift/diffusion as P but starts from pē(n x) (x), I)). Assume Novikov's condition that Girsanov applies.
(A) Path term = score-matching quadratic. By reverse-time Girsanov (see lemma D. the process KL splits into path term plus a boundary term; the path term is
Qu log dQ& dPBri
= [. ( (t, U,, Uo) sta U,) 113 dt,
(72)
where SBri (t, U,) Vu, p. (U,) is the (marginal) score of the reference bridge time t. By the marginal is Gaussian pi (U,) = (Mm,, N,T), hence
Proposition 2
stait U,) U m,
(73)
Substituting this into equation 72 yields the integrand
Scond
8Bri
12
U,
+ Scond (1. U, Uo)
(74)
Kit
which is exactly the wote-intching form stated in the proposition.
(B) Boundary term. Since P and pEri share dynamics and only differ at the start, the remaining term in the KL decomposition collapses to boundary difference (as in DDVI):
dp
Po (U)) = - (LV W| pp (U))
EQ.
log
EQ.
log
(75)
dP
where Bri (m1,, I) follows from Proposition
2
(C) Collecting the ELBO terms. Adding the model likelihood and prior contributions (as in DDVI; mini-batch scaled by N/B) B) yields, up constants independent of (0, o. 7)
N log (OEP((U) + log (())
(DVII (,7) EUaI
%' g(0)2 2
1
U. m,
+ soond t. U, Un)
| dt
14.
2
(L(NNHo(X)) N(m1, I)) + log Pprior (U1)
(76)
which matches the statement.
(D) Reduction to DDVL If (x() = 0, then m, = and the loss reduces to the original DDVI objective (same path integral with Scond s + h and the usual boundary terms).
Lemma (Reverse-time Girsanov: path term as score-matching) Fix T = 1. Consider two reverse- time SDEs dRd with the same diffusion scale g(t) 0:
(2) du, (f(U,, ()(()) scond (t, U. Un) dWW)
(77)
(pBr) du, (f(U,,t) g(t)) st U.) dt + g(t) (P!
(78)
where Scond = So + is the conditional (bridge-correctt score, and spri (t, u) = V, log (u) is the marginal score of the reference bridge (with marginal (u NMm,,, I)). Assume standard
18

---

Published as a conference paper at ICLR 2026
integrabilityy (e.g. Novikov) that Girsanov applies and both path laws Qo and pBri are mutually absolutely continuous on Fr. Then the pathwise Kellback-Leibbee divergence decomposes as
KL(Q& PBa) 2 / s(1)² Ee [| (1, U., Un) - s (4, U.)112 di + Eq log path
qr (U))
pr (UT)
boundary
(79)
where qr and pr are the terminal densities of the two reverse processes (equivalently: the initial densities of the corresponding forward processes). In particular, the path term equals
1 J,
2 g(t))
Eq Scoud (t, U. Uo) - -See (,) 11 dt.
(80)
which is the score-mare matching quadratic used in equation 72
Proof. We write both SDEs common probability space up equivalent change of measure. Let the reference path law be pBri (drift bp == f 9 sBa). Under plari, define the progressively measurable process
Oe = (Q, 1) (U,( = - g(t) (scona U., Uo) - sBri U.)),
(81)
s(1)
where bq == f g2 Scond. By Novikov's condition, the Dôleans-Dade exponential
Ze expl S 0_ dw! (P) - /// 110,12dss) ds
(82)
a true pBri-martingale. Define new measure Qo on Fr by apsn Fr Zr
dQa
pr i.c. we
(Ur)
also correct the endpoint density (boundary term) so that the terminal marginal under Q is gr- Girsanov's theorem yields that under Qp.
w(!)
W(P)
S 0, ds
(83)
a Brownian motion and the drift becomes bq f g² Scond, i.e. the reverse SDE for Qs-
Now compute the log Madon-Nikodie derivative and take ex-execctation:
dQ PBri
KL(Q& pBri)- = Eq.
log
(84)
1' T
1 /. 110.1/ + Ee log (r((U)
= Eq
0f dw
(85)
2
(r(UT)
Use =d + 8, dt to rewrite the stochastic integral:
J 0f aw! - [[' dw!
S. 110.112 dt.
(86)
Taking Q -epepeectaiion annihilates the martingale term, hence
KL(Q& pBrl) 1 Eqa L." 110.112 dt + Eqo log Pr (Ur)
2T (Ur)
(87)
Finally substitute 9,
--1(() (Scona (1, U., Uo) - snri(l, U.)) get
2 5 g(t)² Eq, [iaeollssoe ||sconaltt (t, U. Uo) sta U dt
1 T
(88)
the path term, plus the boundary correction.
19

---

Published as a conference paper at ICLR 2026
Test RMSE on Energy
1.0
DDVI
DBVI
0.8
lest RMSE
0.6
0.4
200
400
600
800
1000
Iteration
Figure 5: Comparison of DDVI and DBVI on the ENERGY dataset (Test RMSE).
Test RMSE on Concrete
DDVI
DBVI
0.45
lest RMSE
0.40
0.35
0.30
250
500
750
1000
1250
1500
1750
Iteration
Figure 6: Comparison of DDVI and DBVI on the CONCRETE dataset (Test RMSE).
B ABLATION STUDY
B.I CONVERGENCE SPEED UNDER MATCHED COMPUTE
We directly compare DDVI and DBVI under matched compute (same T,, optimizer, and hardware)
and track Test RMSE during early optimization. As shown in Figures 5 6 and 7 DBVI reduces Test RMSE more rapidly and consistently achieves lower error than DDVL. This confirms that ditioning the reverse diffusion on a data-dependent initialization shortens the inference trajectory and improves convergence speed on the ENERGY, CONCRETE and POWER datasets.
20

---

Published as a conference paper at ICLR 2026
Test RMSE on Power
DDVI
DBVI
0.27
0.26
Test lest RMMEEE
0.25
0.23
200
400
600
800
1000
1200
1400
1600
Iteration
Figure 7: Comparison of DDVI and DBVI on the POWER dataset (Test RMSE).
B.2 TRAJECTORY SHORTENING DIAGNOSTIC VIA PATH LENGTH
key motivation of DBVI is that the reverse diffusion should traverse shorter and less complex trajectory, since the amortized initialization is closer to the true posterior. To quantitatively diagnose this effect, we report discrete path-length proxy for the reverse diffusion trajectory,
K Σ
L =
U
Ur, 112
where the expectation is taken over the stochasticity of the reverse SDE (and mini-batch sampling) and (tx {< denotes the discretization grid of the reverse-time SDE with K steps. Intuitively, L measures how far the trajectory moves in latent space during sampling, and smaller values indicate shorter reverse diffusion path.
Empirically, DBVI consistently yields substantially smaller path length than DDVL On the Con- crete dataset, DBVI achieves average path length of 0.368, compared to 12. for DDVI. On the Energy dataset, DBVI similarly yields 0.367 versus 11.953 for DDVI. These results provide direct quantitative evidence that the proposed bridge initialization significantly shortens the reverse diffusion trajectory.
C STATEMENT ON THE USE OF LARGE LANGUAGE MODELS
Large language models (LLMs) were used solely for polishing and editing the text of this manuscript.
21


Reviews:
[{"review_id": "I5kajxseqB", "reviewer": "Reviewer_9e3v", "summary": "This paper proposes Diffusion Bridge Variational Inference (DBVI), an extension of the recently introduced Denoising Diffusion Variational Inference (DDVI) method for Deep Gaussian Processes. While DDVI models the variational posterior as the terminal distribution of a reverse-time diffusion SDE parameterized by a neural score network, it suffers from an unconditional Gaussian initialization that is typically far from the true posterior, resulting in long, inefficient inference trajectories. DBVI addresses this limitation by making the diffusion start data-dependent...using an amortized initialization. Using Doob’s h-transform, they reinterpret the reverse SDE as a bridge process that explicitly “bends” the diffusion between a start and an end distribution. Empirical results across regression, classification, and image-reconstruction benchmarks show that DBVI consistently improves predictive accuracy, posterior quality, and convergence speed over DDVI, and other benchmarks.", "questions": "- The model fixes the covariance and learns only the mean. How sensitive is DBVI to this choice? Do performance or convergence degrade meaningfully if the variance is misspecified, and can a learned covariance or low-rank structure further improve inference? \n- The Doob bridge construction relies on the fact that the forward reference diffusion has closed form Gaussian marginals at all t, does learning the covariance disrupt this?\n- More generally, what are the limitations of tying amortization exclusively to the inducing input grid?\n- Could there be a diagnostic metric which can quantify how much the data conditioning shortens the reverse path?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper identifies the unconditional start distribution in DDVI as the source of slow convergence and inaccurate posteriors, and replaces it with a data-conditioned start via a Doob bridge.\n- The use of a linear reference drift ensures that even after introducing endpoint conditioning, the bridge process has closed-form Gaussian marginals at all intermediate times.\n- Like DDVI, the variational posterior is defined implicitly by a reverse diffusion, which is significantly more flexible than mean-field or low-rank Gaussian approximations typically used in DGP inference.\n- Improvements are observed not only on small UCI datasets but also on large datasets (like HIGGS/SUSY), image classification (more subtle), and the frey faces task.", "weaknesses": "- The paper explains the Doob correction, but the experimental section gives limited direct visualization of how the bridge shortens the reverse diffusion trajectory (e.g path length, KL rate, or score norm decay). This makes it harder to see the effect that drives the performance gains.\n- The method (in my understanding) depends crucially on using a linear reference diffusion so that intermediate marginals remain Gaussian and h is tractable. If the model or dataset induces posteriors that are far from those implied by linear reference dynamics, the reference score may become a poor baseline and learning may slow or plateau.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sE9C2zjFQu", "reviewer": "Reviewer_uAKB", "summary": "This paper generalizes denoising diffusion variational inference (DDVI) for deep Gaussian processes (DGP) by replacing the unconditional starting distribution with a learnable, data-dependent initial distribution and reinterpreting the DDVI framework with incorporation of Doob's h-transformation as a diffusion bridge model (DBVI). The proposed method can reduce the gap between the prior and the posterior in the diffusion process and hence speed up the training. There are a few benchmarks included to demonstrate the improvements in accuracy, convergence speeds and posterior quality.", "questions": "1. There is only Figure 4 illustrating arguably after convergence on the small dataset. It appears DDVI reduces the error faster initially before 50 iterations and catches up around 200 iterations. How about after 200 iterations? Can you include more of such plots to demonstrate \"faster convergence\"?\n\n2. Claiming better posterior quality, the authors only show reconstruction RMSE and test log-likelihood, which may not well evaluate quality of the learned posterior. How about empirical KL if you can test in some simulation? Or the rate of credible intervals covering true values?\n\n3. Is the computational cost proportional to the layers of deep GP? How does the diffusion time step K interact with the number of layers L interact?\n\n4. Minor suggestion, are $d_{in}$ and $d_{out}$ layer specific? Should then carry layer index $l$?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It elegantly extends DDVI with Doob-bridge modification. The theory behind such extension is sound and neat. The derived loss clearly connects to that of DDVI and it is straightforward to spot the innovation.", "weaknesses": "However, the numerical experiments could be improved. The benchmark results mainly focus on estimation/prediction accuracy, not much on the posterior analysis. See more comments in the questions below.\n\nI would rate 5 if allowed and will raise my score if the numerical evidence could be strengthened.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JFwjcNaDPf", "reviewer": "Reviewer_VWQi", "summary": "This paper proposes Diffusion Bridge Variational Inference (DBVI), a novel method for posterior inference in Deep Gaussian Processes (DGPs). It aims to solve a key limitation of its predecessor, Denoising Diffusion Variational Inference (DDVI), namely the inefficiency of starting the reverse diffusion from a fixed, unconditional Gaussian prior (a \"cold start\").The core idea of DBVI is to replace this fixed start with a learnable, data-dependent \"warm start\" distribution, $p_0^\\theta(U_0|x) = \\mathcal{N}(U_0; \\mu_\\theta(x), \\sigma^2 I)$. The paper introduces two key technical innovations to make this work: It formally re-interprets the diffusion process as a Doob-bridged diffusion, which is grounded in Doob's h-transform. This allows for the derivation of a new, tractable ELBO objective.To make the initial distribution $p_0^\\theta$ scalable and avoid conditioning on the full dataset, it proposes a structured amortization network $\\mu_\\theta$ that cleverly conditions on the layer-wise inducing inputs $Z^{(l)}$ as proxies for the data.", "questions": "Baselines: Could the authors provide results for two critical missing baselines?a. A competing SOTA expressive VI method, such as a Normalizing Flow-based DGP.b. The \"backbone-only\" baseline for the CIFAR-10 experiment (i.e., the accuracy of the ResNet-20 feature extractor).\n\nGiven that the strongest empirical result is the convergence speedup (Fig. 4), while the final accuracy gains are marginal, would the authors agree that the primary contribution of DBVI is in inference efficiency rather than predictive accuracy? If so, I would recommend reframing the paper's narrative to emphasize this.\n\n Could the authors provide a more detailed breakdown of the additional computational cost of DBVI over DDVI? Specifically, what is the wall-clock cost and scaling behavior of the $\\mu_\\theta$ forward pass and the $(m_t, \\kappa_t)$ ODE solver?\n\nThe $Z^{(l)}$-amortization is a clever feedback loop. How sensitive is this mechanism to the initialization of the inducing inputs $Z^{(l)}$? Does a poor $Z^{(l)}$ initialization lead to a poor $U_0$ start, which in turn hinders the model's ability to find good $Z^{(l)}$ locations?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper's core strength is its theoretical rigor. It correctly identifies a clear weakness in DDVI (the \"cold start\") and proposes a non-trivial, principled solution. Grounding the \"warm start\" in the mathematics of Doob's h-transform allows for a clean derivation of the ELBO and proves that DBVI is a strict generalization of DDVI.\n\nThe most innovative practical contribution is the structured amortization design. Naively conditioning $\\mu_\\theta$ on the raw data $x$ would be infeasible. The proposal to use the learnable inducing inputs $Z^{(l)}$ as a data-dependent, low-dimensional proxy is an elegant and effective solution that neatly sidesteps issues of dimensionality and data dependency.\n\nThe paper's central hypothesis a \"warm start\" improves inference efficiency, is directly and convincingly validated by the case study in Figure 4. This plot clearly shows DBVI converging significantly faster and to a better final RMSE than DDVI, confirming the mechanism works as intended.", "weaknesses": "The experimental evaluation is incomplete for a paper claiming state-of-the-art posterior approximation. A major, competing line of work for expressive DGP posteriors, Normalizing Flows (NFs), is entirely absent from the related work and experimental comparisons. Without benchmarking against NF-based VI methods, the claims of superior posterior quality and accuracy are unsubstantiated.\n\nMissing Practical Baseline: The paper fails to establish a practical \"sanity-check\" baseline. For the image classification tasks, the DGP is applied to features from a ResNet-20. The performance of this feature extractor backbone alone must be reported. If the final, highly complex 4-layer DBVI model (which achieves 95.68% accuracy on CIFAR-10) does not substantially outperform the ResNet-20, it implies the entire DGP/DBVI machinery adds significant complexity for little to no practical gain.\n\nUnfavorable Complexity-Performance Trade-off. This is the most significant weakness. The paper advocates for a method that is substantially more complex than its predecessor. It requires an SDE solver for $s_\\phi$, a new NN for $\\mu_\\theta$, and an ODE solver for $(m_t, \\kappa_t)$. The justification for this complexity rests on predictive gains that are empirically marginal (e.g., 95.68% vs 95.56% on CIFAR-10; 0.859 vs 0.857 AUC on HIGGS). This trade-off makes the practical utility of DBVI highly questionable.\n\nWhile Table 1 provides per-iteration timings, the paper lacks a formal analysis of the additional computational overhead. It should provide a breakdown of the cost of the $\\mu_\\theta$ forward pass and the $(m_t, \\kappa_t)$ ODE solver, and discuss how these new costs scale with the number of inducing points ($M$) and layers ($L$).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8hAMzNMbA4", "reviewer": "Reviewer_vk1c", "summary": "This paper proposes Diffusion Bridge Variational Inference (DBVI), a new approach for performing inference in Deep Gaussian Processes (DGPs). DBVI builds on Denoising Diffusion Variational Inference (DDVI) but improves it by learning how to start the reverse diffusion process from a more informed, data-dependent initialization instead of a random prior. This helps the model start closer to the true posterior and improves both accuracy and efficiency. The authors interpret this modification through Doob’s h-transform, giving a bridge-based view of the diffusion process that keeps the method theoretically consistent while making it more flexible. They also describe a practical inference scheme based on inducing points to make training scalable.\nIn experiments on regression, classification, and unsupervised learning tasks, DBVI shows consistent improvements over DDVI and other standard inference methods for DGPs such as DSVI, IPVI, and SGHMC.", "questions": "1: Could the authors comment on the computational cost of the additional amortization network used to initialize the diffusion bridge?\nIn particular, does this learned initialization introduce noticeable overhead compared to the standard DDVI setup?\n\n\n2: In Figure 3, the improvements over DDVI are consistent but often modest. Could the authors comment on whether larger benefits might appear in other settings and why? \n\n\n\n\n3: Do the authors plan to release code and trained models for reproducibility and to facilitate adoption by the community?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality:\nThe paper proposes the novel idea of reinterpreting DDVI as a kind of diffusion bridge using Doob’s h-transform. This leads to a principled way of conditioning the diffusion process on input data. The use of an input-dependent initialization for diffusion-based inference is conceptually elegant and gives a new perspective on how diffusion models can be adapted for Bayesian inference.\n\n\nQuality:\nThe technical development is solid and well thought out. The authors clearly connect their bridge formulation to the underlying variational objective. The experiments are thorough, covering regression, classification, and unsupervised learning, and DBVI consistently outperforms DDVI and other inference methods like DSVI, IPVI, and SGHMC. \n\n\nClarity:\nOverall, the paper is well written and easy to follow. The authors do a good job of explaining the motivation behind their changes to DDVI. The main ideas are presented in a logical order, and the appendix includes helpful details for implementation. \n\n\nSignificance:\nThis work makes a meaningful contribution to improving inference in deep Gaussian process models. By addressing a key limitation of DDVI and showing consistent gains across several tasks, the paper offers a practical improvement that should be useful to researchers.", "weaknesses": "Primary Weakness: \n\nMagnitude of improvements: While DBVI consistently outperforms DDVI and other methods, the numerical improvements are sometimes modest (e.g., some overlapping error bars in Figure 3). A discussion of whether these gains translate to meaningful practical differences would strengthen the empirical section.\n\n\nMinor Weaknesses: \n\nFigure readability: The font size in Figures 1, 2, and 3 is quite small, making some labels difficult to read without zooming in a lot. This is a minor visual issue that can easily be fixed for the camera-ready version.\n\n\nFormatting issue: The arrow in Figure 2 partially obscures the word “likelihood.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)