An Optimal Diffusion Approach to Quadratic Rate-Distortion Problems: New Solution and Approximation Methods
Authors: Dror Freirich
Nir Weinberger
Venue: ICLR 2026 Poster (2026)
Keywords: information theory
rate-distortion
diffusion processes
stochastic control
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We establish a connection between rate-distortion (RD) and optimal control, and estimate RD functions using diffusion processes.

Abstract: When compressing continuous data, some loss of information is inevitable, and this incurred a distortion upon reconstruction. The Rate–Distortion (RD) function characterizes the minimum achievable rate for a code whose decoding permits a specified amount of distortion. We exploit the connection between rate-distortion theory and entropic optimal transport to propose a novel stochastic-control formulation for the former, and use a classic result dating back to Schrodinger to show that the tradeoff between rate and mean squared error distortion is equivalent to a tradeoff between control energy and the differential entropy of the terminal state, whose probability law defines the reconstruction distribution. For a special class of sources, we show that the optimal control law and the corresponding trajectory in the space of probability measures are obtained by solving a backward heat equation. In more general settings, our approach yields a numerical method that estimates the RD function using diffusion processes with a constant diffusion coefficient. We demonstrate the effectiveness of our method through several examples.

Published as a conference paper at ICLR 2026
AN OPTIMAL DIFFUSION APPROACH TO QUADRATIC RATE-DISTORTION PROBLEMS: NEW SOLUTION AND APPROXIMATION METHODS
Dror Freirich Nir Weinberger"
Viterbi Faculty of Electrical and Computer Engineering Technion Israel Institute of Technology
drorfrc@gma 1 com, nirwel nê nê wewtteecccnnnnee
ABSTRACT
When compressing continuous data, some loss of information inevitable, and this incurred distortion upon reconstruction. The Rate-Distortion (RD) function characterizes the minimum achievable rate for a code whose decoding permits a specified amount of distortion. We exploit the connection between rate-distortion theory and entropic optimal transport to propose a novel stochastic-contro for- classic result dating back to Schrödinger
mulation for the former, and show that the tradeoff between rate and mean squared error distortion is equivalent tradeoff between control energy and the differential entropy of the terminal state, whose probability law defines the reconstruction distribution. For a special class of sources, we show that the optimal control law and the corresponding tra- jectory in the space of probability measures are obtained by solving backward heat equation. In more general settings, our approach yields a numerical method that estimates the RD function using diffusion processes with a constant diffu- sion coefficient. We demonstrate the effectiveness of our method through several examples.
INTRODUCTION
As is well-known from information theory (Shannon) 1948 Cover and Thomas 2012). when com- pressing any probabilistic data source at an encoding rate lower than its entropy, it is inevitable to suffer some loss of information, which results in distortion at reconstruction. This is particu- larly true for continuous sources, where distortion may arise, e.g., due to quantization. Rate distor- tion (RD) theory (Shannon 1959. Berger 2003) Cover and Thomas 2012) addresses the trade-off between the encoding rate and the decoder's reconstruction: Given a distortion measure between pairs of data samples, the reconstruction loss is the total accumulated distortion between the source samples and reconstructed samples produced by the decoder from the compressed bits sent by the encoder. The operational RD function thus characterizes the minimal encoding rate required for given average distortion level D, in the limit of large number of samples.
For independent and identically distributed (i.i.d) samples, the celebrated lossy source-coding the- orem of information theory (Shannon 1959 Berger 2003, Cover and Thomas 2012) establishes that the operational RD function equals the informational RD function R(D). where the latter is expressed as single-letter expression (though a single sample may itself be vector). Specifically, the RD function is the minimum the mutual information (MI) between a random variable X. representing a single sample from the source, and a random variable X, representing a single re- construction sample. This minimization is over the test channel the conditional distribution of X given X under the constraint that the average distortion between X and X below (see $
The optimization problem involved in the computation of R(D) should therefore be solved for the given distortion measure and distortion level. However, although this problem has been extensively studied for almost 7-decades (Shnnnon 1959), closed-form solutions of this problem only known
'The authors wish thank Lena 7ugman and the Technion ECE library staff for their support.

---

Published as a conference paper at ICLR 2026
for few canonical examples, such as binary sources under Hamming distortion Gaussian source under the mean squared error (MSE) Cover and Thomas (2012). Efficient algorithms for computing the RD function therefore necessary. When the data source alphabet is discrete and finite, the classical Blahut-Arimoto (BA) algorithm (Arimoto 1972 Blahut 1972) provides such a method and is especially effective for small alphabets. However, the typical setup for lossy compression in- volves a continuous data source, and in this setting efficient computation of the RD function remains long-standing challenge. Recently, [ei et al. (2023); Yang al. (2024) identified an interesting connection between the BA algorithm and cearopy-regulaizzer Optimal Transport (OT), which al- lows approximation of the RD function in cases where BA is intractable (Yang et al., (024). OT (Ambrosio et al. 2008) is widely used formulation, which underlies the training of deep genera- tive models, such as generative adversarial networks (GANs) (Arjovsky et al. 2017] and variational auto-encoders EVA (Tolstikhin et al., 2017), and is defined as follows: Given pair of distribu- tions, the objective of OT is to find an optimal plan between them, namely, a joint distribution with the given marginals (called coupling) that minimizes a prescribed metric. cntrropy-reullaiiee optimal transport (a.ka. weak OT, entropic OT, or EOT) was introduced by Cuturi (2013) as an ap- proximation to OT, for which efficient solution methods exist, e.g., Sinkhorn's algorithm (Altschuler et al. 2017). More recently, Gushchin et al. (2022) suggested finding the optimal plan of an EOT problem using diffusion processes. Their method is based on well-known connection (Léonard, 2013 Chen et al. 20221) between EOT and a stochastic control problem known as the Schrödinger
bridge (SB) (Schrödinger 1932 Chetrite al. 2021 Chen et al., 2021
Diffusion processes are popular for generative modeling (Ho et al. 2020 Song et al. 2020), where sample from a data distribution gradually drifted and becomes noisier as it approaches a com- pletely noisy sample, typically Gaussian. The celebrated paper of [Song et al. (2021) suggested finding the drift term of the model by learning the score function and plugging it back into the re- verse stochastic differential equation (SDE) (Anderson) 1982). As we state later, a solution to SB can be written as a finite-energy diffusion process. then becomes natural to investigate diffusion processes in the context of RD theory, generative models casting the source probability to the iittortion-optmmll reconstruction distribution. As we show in this work, this fresh point of view reveals surprising analytical results as well novel estimation methods.
In this paper, we focus on the computation of the RD function for continuous data sources under the MSE distortion. This leads to OT problem with quadratic cost. More specifically, we exploit the connection between RD and EOT to propose a novel stochastic control formulation to RD, where the classical result of Schrödingerr (9322) implies that the tradeoff between rate and MSE distortion is equivalent to a tradeoff between the control energy and the differential entropy of the terminal state, whose probability law yields the reconstruction distribution. For a special class of sources, we show that the optimal control law and trajectory in the space of probability measures are given by solving Backward Heat Equation (BHE). the more general case, our approach gives rise to a numerical solution method in which the RD function is estimated using diffusion processes, with a constant diffusion coefficient.
Our contributions:
1. We establish novel connection between RD and optimal control by presenting Terminal-Entropy Control (TEC), a stochastic-control formulation regularized by terminal uncertainty, and showing that this formulation is equivalent to the RD problem.
2. We characterize the optimal solution to TEC under certain regularity conditions.
3. Using this characterization, we provide a closed-form solution for the reconstruction distri- bution of a Gaussian mixture source. We also demonstrate our approach on mixture distri- butions with non- Gaussian components via Fourier analysis. To the best of our knowledge, such results were previously unknown, which emphasizes the theoretical contribution of our work.
mixtsie
4. Based on our approach, we propose R2D2, a novel neural method for estimating the RD function and the reconstruction distributions using simple diffusion model.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES
2.1 THEORETICAL BACKGROUND
Cate-Distortion theory Let px = P(R") for d 2 denote a single sample from the source, where P(RR) is the set of probability measures on IR, and where the different source sam- ples are i.i.d. Let X E Rd denote a reconstruction sample, where the pair XX, follows the joint distribution (X, X) Pxx E P(R" RR"). In addition, let Px denote the induced conditional dis- tribution, which is also called the reconstruction law. Let :): 12 x F + R+ denote distortion measure between E IR and à E R,, and let the average distortion be D(X, X) EId(X, X) which an implicit function of Px For a given pair of probability measures Po, PP P(R let DKL (PP)PP)) A Ex-P, Ilog dFu (X) denote the Kullback-Leibler divergence, and let the mutual dP:
information be X) = DKL (p & |px p The lossy compression theorem (Shannon 1959
Berger 2003 Cover and Thomas 2012 states that the operational RD function is equivalent the
informational RD function
R(D)
min
I(X; X);
PXIX: D(X,X)
on which we will focus. The BA algorithm (Cover and Thomas 2012 computes R(D) by optimiz-
ing the Lagrangian (with a Lagrange multiplier c 0)
LRA(PXIx Po,e) D(X, X) + <I(;
(2)
w.r.t Px Alternatively, if we let P1 P& be the marginal distribution of the reconstruction, then an equivalent formulation is Yang et al. 2024)
min )
min LBA Po.c- PxIx
min PIEP(R*) (x.X (x.e)==el(...P))))
(3)
where IT(Po, PPi) is the set of all couplings, that is, the of joint distributions Pxx E P( Rd)) whose X-marginal (resp. X --marginal) is Py (resp. Pi). In this work, we focus on the quadratic cost
d(á x) = 1 f 21( The average distortion then given by D(X. X) [| - X 1 and the
minimization problem Eq. (3) reads
f [IX {|p] + (TL (4)
min LBA(PXIx: Po. = min x
min P. (E.
Entropic optimal transport For probability measure IP E P(RR) with density p(x), (P) &
- -xx,r log(p(X)) denotes its differential entropy. Now, considering probability measures Po, P1 P(RR), the HOT problem (Cuturi 2013) is given by
x 111 dr 2) + eDKL(T|IPo & PP) 2
inf xeIl(Po.P:)
U
(5, BOT)
As one may readily recognize, for values of e where the optimal reconstruction has density, the inner minimization of Eq. 44) coincides with Eq. (5, EOT (Lei et al. 2023 Yang et al. 2024)
The Schrödinger Bridge inf E [ [I((
The SB problem (chhöödinger 1932 with parameter c formulated as
s.t. Xn ~ Po. X1 Pr dxx; = u(X,, t)dt + Vedw,
(6, SB)
2
where Po. P1 are absolutely continuous probability measures w.r.f. the Lebesgue measure on Rd. and W, a standard Wiener process, independent Xo. The drift u Rdd x [0, |] + Rd can be seen as controller designed to steer Xo Po into X, ~ IP, with minimal average energy, acting against 'natural' force Wt. Recently, Gushchin et al. (2022) have utilized the well-established equivalence Po. P1, e) between SB and EOT (Léonard) 2013 Chen
(up to an additive constant depending ct al., (021). showing that the latter can be optimized via a game theerettc formulation. The optimal joint probability æ in Eq. (5, BOT) is then given by the joint probability law of (Xo, X1).
3

---

Published as a conference paper at ICLR 2026
2.2 RELATED WORK
Neural estimation of information-theorete quantities Following ongoing research on neural models, variety of methods have emerged in recent years for estimating information measures, as well as for the design of compression methods aimed at achieving these fundamental limits (see the survey by Yang et al. (.23)). Belghazi et al. (2018), for example, used the Konsker-Varadhar identity to estimate MI. Kholkin et al. (2025) used samples from Brownian bridges to estimate both the MI between datasets and differential entropies. [ei et al. (2022) suggested approximating the RD function using a deep neural network model and proposed an operational coding scheme. Tsur et al. 2024) suggested approximating RD by modeling unknown input distributions, both continuous and discrete. Recently, Lei et al. (2023): Yang et al. (024) pointed out an intriguing connection between the BA algorithm and EOT. This connection was further exploited in Yang et al. (2024) estimate the RD function using a discrete approximation of the reconstruction probability law. Finally, (Zuu et al. (2025) used the connection between EOT and the Schrödinger problem to characterize this density. In this paper, we target the reconstruction distribution in RD problems with MSE loss, i.e., a quadratic cost, by optimizing a continuous diffusion model. Compared to the methods mentioned above, this novel approach allows us not only to offer new analytic solutions to this classical problem but also better approximate RD functions, especially for high rates, as we show in $
Diffusion models for lossy compression Lossy compressors that hinge diffusion processes have become popular in recent years. In Elata et al. (2025) pre-trained diffusion models were sam- pled for zero-shot image compression. Recently, Ohayon al. (2025) replaced the noise at every timestep of the reverse diffusion with samples from a sequence of predefined codebooks, achiev- ing high perceptual reconstruction quality (Blau and Michaeli 2019 Freirich al., 2021 2024).
In Theis et al. (0022) a noisy version of the source data was compressed and then used in a re- verse diffusion model for reconstruction. Here, we use a forward model obtain the reconstruction distribution and compute both its rate and distortion.
Schrödinger Bridge and EOT Numerical solution methods to SB include iterative fitting Shi
et al. 2023), adjoint-state matching (Liu et al. 2025 Domingo-Enrich et al.) (024) and sampling
the potentials of the system (Puchkin et al.] 2025a b Rapakoulias et al., 2024; Gushchin et al.
2024]. Dai Pra (199T) further expressed the optimal control and the objective Eq. 6. SB) in terms of these potentials. This differs from our approach, where we directly optimize the control function u. avoiding the need evaluate or sample potentials. The closest method ours is given in Gushchin eet al. (2022). which used the equivalence between SB and EOT (Chen et al., 2021: Léonard (013) to offer a game formulation for the former. which can be solved by optimizing diffusion model. In 3 we propose a modified problem in which the target probability is free. As such, the terminal constraint is replaced by a penalty on the final state for being uncertain. We show (Thm. 3.1 that this formulation is equivalent to the RD problem under the MSE distortion.
Entropy-regularizee stochastic control In addition to presenting a novel approach to the RD problem, our results can also be considered contribution control theory. As we stated, Eq. 66. SB) is formulated as a stochastic-contro (' ('dynamic)) problem. Both this form and its 'static' counterpart given in Chen et al. (2021). Entropy regularization is common in stochastic control (Lambert let al. 2025) and reinforcement learning (Haarnoja et al. 2018 Ziebart et al.] (008). However, most studies aim to maximize the control-policy entropy, encouraging diverse actions. Alternatively, Fridman and Shaked (2000) proposed minimizing the steady-state entropy of closed-loop linear systems Ho control problems with infinite horizons. Here, we focus on penalizing the terminal- state uncertainty, leading to a novel tradeoff between energy and entropy. After submitting this manuscript for publication, we became aware of the work of (198)), which, within the context of physical systems, derived variational form similar to Eq. (15, vEr- However, there, the controlled state and observation have pointwise initial conditions and control is reversed in time. To the best of our knowledge, information theory, which may open the door to additional applications in the broader fields of statistics and stochastic control, connecting these major disciplines.
work is the first to present such formalism in the context of
4

---

Published as a conference paper at ICLR 2026
[ - -R())
0.04
Low-
R2D2 (ouns)
rec. density
NERD
0.02
WGD
0.2 MSE
0.3
0.1
Figure 1: Evolution of reconstruction distri- butions w.r.. parameter Typically, as- sume the reconstruction density to be close to the continuous source in the low distortion regime (small e's), while eventually becoming singular for low enough rate (large e's).
Figure 2:
Estimation error on a 1-D Gaus-
sian source. Xo A (0, 1) and applied R2D2 (Alg. [ to [0.05, 0.95]. We compare our algorithm to NERD and WGD. where we ob- serve the improved accuracy of R2D2 over exist- ing methods.
3. RATE-DISTORTION FUNCTIONS AND TERMINAL-ENTROPY STOCHASTIC CONTROL
3.1 PROBLEM STATEMENT
Let the source Xo Po. where Po E P2 (2R() is absolutely continuous w.r.l. the Lebesgue measure, and where P2 (2( finite. As in the BA algorithm, we aim to compute the RD Lagrangian
(IR") is the subset of ((R for which E. |||| We further assume that H HPP))
| x dx, - H(T) + H(PI)) (Po)), (7, RD) =El((f.PP1) 2c Jd'
(ID <) min min
which follows from standard decomposition of MI to entropy terms (Cover and Thomas 2012 (TIlP & F1) H + (T). Here, > is tuning parameter, and H (Fo) is not subject optimization.
Assumption A1: Eq. (7. RD) admits solution in which the optimal reconstruction distribution is absolutely continuous and satisfies P1 (R with finite differential entropy H (PP))
Although A1 is necessarily violated for low rates (high values of e). in many cases it is still expected be valid in the low-distortion regime, where IP is close Po. illustrated Fig. [ This holds in variety of settings, as we demonstrate in $ 3.3
Targeting the oowdissortionn regime is of special interest since, in modern communication, high bandwidth channels are often available (Chafii al.) 2023), and thus sources are compressed at high bitrates, allowing low distortion.
Let the process dW, = VedW,, with initial state W Po. be the (scaled) Brownian motion starting at Ioo, and denote nt = Pw: wr. the joint law of its start and end points. It is known (Gushchin et al. 2022] Chen al. 202T) that
1
d x:|:dT(x, H(T) H(P)) log(2Te)
(xl(F|(T')
(8)
2e xdd
for every π with marginal distribution Po. Let F(Po, be the class of random trajectories T; Rd I. E [o, 1] with joint distribution कग = PT..T (TP, Pt). For every T E F(Po, Pi),
DKL DKL(T][W9) = DKL(TTIITT
JRAXR (Ta (xr,
(9)
where there exists process Tž minimizing DxL WW) over F(Po, P.) with DKL(T la, (n.))
for all x, f. Rd Léonard 2013 Prop. 4.1,2.3). The process
TF, is known to take the form Tüp of an Itô diffusion (ushchin et al. 2022
Tv: dX, ul (x, VelW,
(10)
5

---

Published as a conference paper at ICLR 2026
with drift up,, where E have Paavn and Wakolbinger 1991),
Jo uu, (X,, d
< co. Furthermore, for such a finite-energy process
I lp (X,, dt
DKL. (Tr, |W*)
E
(11)
2E
Considering Eq. (7. RD -R.. 11
we suggest the following surrogate loss
d log(2
(Po, c) min min Pi T.EF(PP.PP))
[|u D1 + (((P))
(12)
where T. is a finite-energy diffusion given by Eq. (10) with drift u 2 {u(z,t) ((P.) > -oo}, that is P1 has a finite differential entropy. The above discussion leads to the following equiv- alence between Eq. (7. RD) and Eq. [2
Theorem 3.1. Given Ito and > O, under Al we have CRD = LRD. Furthermore, let u² wa( (x,
minimize the surrogate objective
{ 1 2E
[%. (lu(X E)ll H(X))
u" E arg min
(13)
well
under the law in Eq. 10 Then, the distribution Pxç of Xi associated with u² through Eq. 70) is
the minimizer in Eq. 422, where P Px; is the optimal reconstruction distribution Eq. RD)
and x² = xxxx,x;; =x=xxx== x;
is the optimal plan.
The proof is given App. C where we also establish the opposite direction; whenever (P1 T) minimizes Eq. (7, RD (Xp. n.
there exists a drift term u(x, t) minimizing Eq. (13) under Eq. (10). where
3.2 CONNECTION TO STOCHASTIC CONTROL: THE ENERGY-ENTROPY TRADEOFF
Motivated by Thm. 3.1 present the problem of Ceeminnal-Entrope regularized stochastic Control
()) + =H((x)) s.t. (o Po, Px, is free J
inf
(14, TEC)
weM
[dX, ul (1 t)dt + VedW,
where the admissible control set is again 4 (u(xr,t) (T():) -x). In light of Eq. (6. SB here u can be viewed as control law for reducing the terminal-state uncertainty while spending minimal energy. As consequence of Thm. 3.1 in order estimate the RD function for the source Xu ~ Po. one should solve Eq. (14, TEC) with range of values. Given the drift term u, X1 can be efficiently sampled (e.g., using the Euler- Maruyaaa algorithm, see App. Taking Assumption A1 into account, Eq. (14, TEC) takes the following variational form
inf {1 [2/e= dxc L. dillu(e, (x))
well (2 RRd
dapi (1) (1)
(15, var-
JRd
where, for the diffusion process in Eq. (10). the density state pr()) is governed by the fokker-Plande equation (Oksendal, (013)
0
0r P((x)) = (p)(z)((I,)) t)) + =<Dem po(x) = Po(x),
(16, FPE)
with V. Σ 2 ( being the divergence operation, and Are = Σ
8²
being the Laplace op-
ae,
dz?
erator. Interestingly, under suitable regularity conditions, the solution to Eq. 14, TEC can be characterized by a simple equation, as we show next (proof given in App.
C
Theorem 3.2. Let pi (x) E ( 1] x R2 satisfy the backward heat equation (BHE)
i
1 Amari (x), pō(x)) Po:
pl
(17, BHE)
such that logpī(2) E ((0, 1] Rdd) and 1/ (a:) log ff (x:) as |||||| - Da for all e [o, 1). Let n² = e log (I), where V log pr(x)) ()) is the Stein score function. Then, (u'.pt) is an optimal
pair in Eq. [5, var-TECI-E [16, FPE and the optimal solution to Eq. (14, TEC admits the SDE
dx; e log Pi (X,)dt + VedW.
(18)
(c,,2 1] x Rd) is the set of functions that continuously differentiahle w.t.. f, and twice continuously differentiable

---

Published as a conference paper at ICLR 2026
Fourier analysis of Eq. (17, BHE higher dimensions. Let the source Xo ~ Po in R have density Po and characteristic function p(w), namely po(x)) 2n -_o e (w)... It is easy verify that solution to Eq. (7., BHE is
Let assume d = 1 for simplicity; similar arguments hold in
1
pa(x)
clwu+ $ fow? p(w))
(19)
2 -00
whenever the integral converges for all t E [0, 1] and V log (x) is defined for all x, t E R x [0, 1].
3.3 SPECIAL CASES
Backward heat conductance problems are generally ill-posed and unstable (Miranker 1961 Fu al, 2007]. However, Thm. 3.2 yields an exact solution for certain special cases, as we show next.
Gaussian source We begin with canonical example of scalar Gaussian source Po N (0,og), and show how our formulation recovers its known RD function. For < og solution to
2100 -0 The optimal controller is therefore given by u(z, t) = <V log (x = o3-et Evidently, under u(æ, t), Xo and X1 are jointly Gaussian
Eq. (17, BHE is given by pe(a)) 2 at
where D = [ - x fe, and R = I(Xoi X1) = - log -/4 and we recover the
known closed-form result
RGause (D) log
1
T
2
0 2D o
(20)
2D
Note that the factor of arises because our distortion definition can be easily generalized to Gaussian vector sources and e values smaller than the eigenvalues of the covariance matrix E. In this case, the solution Eq. (17, BHE is pr(2)) N' -
half of the MSE. This result
Gaussian-mixture source The Gaussian example can be easily extended to the case of a Gaussian mixture, for which no closed-form solution for the RD function known. In this case,
N
P
(o(x)=
(21)
2TO
Because Eq. 17, BHE is a linear equation, the solution is given by the superposition
(a (
(x
Σ
Pi 2 et)
20
(0, mino?)
(22)
The optimal controller u(x; t) = v log (r) is derived accordingly. Knowing Pt (r) and u(x; f). possible to obtain rate and distortion values through Monte Carlo simulation of Eq. (14, TEC), or via neural estimation as we suggest in $
A non-Gaussian-mixtee result to a mixture of sinc functions. Being band-limited, for settings where (z) > 0, such source satisfies the conditions of the theorem and allows the desirable trequency-domant analysis of $ (3.2 Although this is a toy problem, we emphasize that to the best of our knowledge, no other approach is known to tackle this case.
source To show the wide applicability of Thm. 3.2 we apply its
We consider the source Xo drawn from the mixture po(x) = D.C.P.C sinc ( where sinc(z)
A sin(z) E (R), and C, = 2 m4 are appropriate normalization factors. In this case, the characteristic function is (w(w)
P.C. (.p.Cmmppp(mm) where pu(w) 27 (m(1 ww)+ (11 - } [w])]] and is the convolution operation. Now, for non-vanishing mixture distributions and sufficiently small values of c, the Fourier analysis of Eq. (19) implies that can write the solution to Eq. (17, BHE
/"
4
2 N
Σ P.C. Ep.CT'm, mi S' (o(()a))) cos(wa)) y ow (23)
(x( = 2T
e jinz+ jew?t (o(w)d 2T
7

---

Published as a conference paper at ICLR 2026
1.0
132
-2
50
10
=2 2.06o-02
50
15
= 7 7.50e-03
16
Pr(a))
66
62
22
Figure 3: Reconstruction distribution pi (x) of non-Gaussian mix source. In the (left) pane, we approximate reconstruction distribution Eq. (23) by numerical integration. closer look different points is provided the (middle) and (right) panes.
Fig. demonstrates this result for N = m, [1, 22, π. c] and p; = where numerically integrated Eq. (23) to approximate the reconstruction distribution pi (1 for different values of Parameters were chosen such that pt (z) > 0 everywhere on R.
4 R2D2: NEURAL ESTIMATION OF RATE-DISTORTION FUNCTIONS
Although elegant, as may be pointed out, the assumptions of Thm. 3.2 may be restrictive: They require (x) be non-vanishing and twice differentiable everywhere in Rd even under these requirements, Eq. (17. BHE) may be ill-posed, depending on the initial condition Po: Finally, the explicit source distribution rarely known in practice, and is instead accessible only from samples. Therefore, in this section, we propose R2D2 (Alg. [ a sample- based method for solving Eq. (14. TEC and approximating the RD function in the general case (under Assumption A1).
Algorithm 1 Revealing RD functions with Diffusion (R2D2)
1: Input: source Xo ~ Po P((R)), initial controller us, batch size M, timestep Ar, Emins 6max
0, learning rate a.
2: while Training do
Training
3: Choose E Uniform Ciin Emax]]
4: Sample batch (X0e) M
m=1 PP.
5: Sample trajectory ( ( his c), XMMM Jm=1 + EuMa(us, (XT" (X (, A,). 1 Σ E,, lluo (Xm t,, 2
6:
Estimate energy LG + 2M
7: Estimate terminal entropy (X,) (see App. B
8: RD loss L5 <- I. (X.).
9: Step 0 +- VoLs.
10: end while
12: Sample batch {XT)) M
m== ~ Po-
b Evaluate specific E [<mins Crax
Sample trajectory (ug (Xmm t;, e), X XmnM
Jmm EuMa(us, {Xp M Jmm,, At).
14: Obtain RD loss CS (lines 6-8).
Estimate distortion: D
2nd Σ M
Em=1 |XTT -T-11
16: Estimate rate: R Ch-
(2TEe.
17: Output: (R, D).
Our method R2D2 (summarized in Algorithm is based modeling the controller function u0(x, t, c) using a DNN with parameters 6. The flexibility and generalizability offered by DNNs allow capture multiple positions the RD-curve (different values) using single controller model (cf. Yang et al.] (2024)). To train our model, we access the data source X, to draw a batch of M samples. Using the Haler-Maryyama method (FuMa, Alg. in App. A), sample discretized
8

---

Published as a conference paper at ICLR 2026
R(D) [nats
3
R2D2 (ours)
2
p1 Pi
0.5
py (mmpproo))
SLB
0.0000
0.0025
0.0050
0.0075
0.0
1 MSE
2
Figure 4: The R(D) function for a mixture of Gaussians. Xo ~ Po is a mixture of Gaussians. (lett) We apply R2D2 (Alg. D [4 10 1.64 10 -] and compare the result with SLB. Green markers indicate higher precision. (right) For = 1.56 x 10-2, we plot the reconstruction distribution Pr. The empirical distribution matches the analytical result (bold line).
random trajectories X,, from Eq. 10 The minimization objective in Eq. is approximated (up to an additive factor of 2 log (TTE) Cg LG (Xf, where the estimated controller energy is
to 4, c)l The terminal entropy A (X,) is estimated through the approximated negative entropy through kernel method (see App. B for details). To evaluate
T. ~ TM Em= Ex, (
R(D), after training, we recalculate Ls. and use Eq. (12) to compute the empirical values
1 2M Σ XT 13 A(E) = I(X" xT") C6 D(c)
d log (2TC).
(ee)
(24)
2
Remark 4.1. Both methods of Yang al.) (2024) and Lei al. (022) assume an upper bound on the rate (i.e., R < log M, where M is the batch size or the support size of the atomic probability model). Our method does not suffer from this substantial limitation. This makes Alg. D suitable for estimating the RD function at the low-distortion regime ( ('high-resolutionn high rates).
NUMERICAL RESULTS
In this section, apply results to both toy and real-world problems. For full details and more numerical results, we refer the reader to App. For full simulation details, we refer to App.
5.1 GAUSSTAN DATA
In Fig. 2 we demonstrate the efficiency of Alg. on the 1-D Gaussian casé of 3.3 We compare method with NERD (Lei al., 2022) and WGD (Yang al., 2024 64 independent periments (seeds) and plot median absolute error with interquartile ranges, We observe that R2D2 clearly has lower estimation error then exisiting methods, both the high-rate and low-rate regimes.
In Fig. {-.4.0, .). of = (4, 5, 6) x 10-2, p; = We apply Alg. to E [4 10-4, 4, 1.64 1.64. x 10 and compare the estimated RD function with the approximate Shannon's lower bound (SLB) Cover and Thomas 2012 Berger 2003), given here by H(PP)) 5 log (T) For
show results for Gaussian mixture Eq. source, where N
14
1.56 x T
we further plot the reconstruction distribution Pi, the law of the diffusion process's 's outcome. We observe that the empirical distribution obtained by Alg. A matches the closed-form in Eq. (22).
5.2
REAL DATA
5.2.1
CIFAR10 DATASET
We demonstrate the efficiency of Alg. oo realistic source. More specifically, as input R2D2, sample 4 x 4-pixel grayscale image patches from the 'CIFARIO' dataset (Krizhevsky and Hinton
All codes are publicly available http://jtttnhaccomm/lmgaagapppzz

---

Published as a conference paper at ICLR 2026
CIFARIO
Speech
20
10.0
R2D2 (ours)
R2D2 (ours)
R(D) [nat]
NERD Lea 2022
R(D) [nat]
15
NERD [Las en a22022
7.5
WGD Noog
WGD
16
2.5
0.01
0.02
0.03
0.01
0.05
0.06
0.07
1.5
2.0
2.5
3.0
3.5
10
MSE
MSE
Figure 5:
Estimating R(D) functions on real data. (left) CIFARIO images. Xo is a random
4 x 4 patch from grayscale image. The RD function, as estimated by R2D2 (Alg. T and baselines. (right) Speech dataset. We observe the efficiency of R2D2 at high-rates (> 20 [ntss), ) while existing methods are practically upper bounded by 13 [nats].
source
1.77e
5.05e
03
1.16 02
2.47
Rate:
10.73
6.65
4.46
3.12
Distortion
1.02e 02
2.32e
02
4.15e
02
7.09c
02
Figure 6:
Reconstruction distribution for the CIFAR10 dataset. Patches drawn from the recon- struction distribution X, obtained for different values.
2009). Fig. [[left) demonstrates the efficiency of our method in solving this problem, where we present the RD function, estimated by R2D2 (Alg. [T. In Fig. 6 we present images, drawn from the reconstruction distribution P1 for different values of e.
5.2.2 SPEECH DATASET
We further test our method on the high-dimensional Free Spoken Digit dataset of Jackson et al. (2018) (see also Yang al. (2023) 2024)). Here, the 33-dimensional samples consist of spoken- digit recordings. The data are obtained and preprocessed as in Yang et al. (2023) A.5.4), and each feature is then whitened. We present the estimated RD function, compared to the NERD and WGD estimations, with special attention given to approximating the low-distortion regime (see Remark 44.1.. We set the latent dimension size in NERD to 1024, the batch size M
10 and the number of particles in WGD to n 2 105, as in Yang et al. (2024). As we can see in Fig. right), our method is capable approximating (theoretically) unbounded rate values (> 20 [nats]) whereas previous methods practically bounded by approximately 13 [nass] or less.
CONCLUSION
We considered the computation of the RD function and optimal reconstruction distribution for con- tinuous data sources under the MSE distortion. We exploited the connection between RD and EOT to estimate the RD function using diffusion processes through novel control formulation in which the RD tradeoff is equivalent to a tradeoff between energy and entropy. Under regularity conditions, the optimal control is given by BHE. We demonstrated ing closed-form solutions, and in a real-world setting using numerical method. This work paves the way for solving RD in settings beyond the MSE loss and continuous distributions.
results in certain special cases, obtain-
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The work of NW was supported by the Israel Science Foundation (ISF), grant no. 1782/22.
REFERENCES
Jason Altschuler, Jonathan Niles- Weed, and Philippe Rigollet. linear time approximation al- gorithms for optimal transport via sinkhorn iteration. Advances in neural information processing systems, 30, 2017.
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient flows: In metric spaces and in the space of probability measures. Springer Science & Business Media, 2008.
Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3)331-3326, 1982.
Suguru Arimoto. An algorithm for computing the capacity arbitrary discrete memoryless chan- nels. JEEE Transactions Information Theory, 18(1):14-20, 1972.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference muchine learning, pages 214-223. PMLR, 2017.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R. Devon Hjelm. MINE: Mutual information neural estimation. arXiv preprint arXiv:1801.044 1801.00011 2018.
Toby Berger. Rate-distortion theory. Wiley Encyclopedia of Telecommmmiicationss 2003.
Richard Blahut. Computation of channel capacity and rate-distortion functions. IEEE transactions on Information Theory, 18(4):460 473. 1972.
Yochai Blau and Tomer Michaeli. Rethinking lossy compression: The rate-distortion-perrepioo tradeoff. In International Conference on Machine Learning, pages 675-685. PMLR, 2019.
Marwa Chafii, Lina Bariah, Sami Muhaidat, and Merouane Debbah. Twelve scientific challenges for 6g: Rethinking the foundations of communications theory. IEEE Communications Surveys & Tutorials, 25(2):868 904, 2023.
Yongxin Chen, Tryphon T. Georgiou, and Michele Pavon. Stochastic control liaisons: Richard Sinkhorn meets Gaspard Monge Schrödinger bridge. Siam Review, 63(2):249-313, 2021.
Raphaël Chetrite, Paolo Muratore-Ginamesshii and Kay Schwieger. E. schrödinger's 1931 pa- per "on the reversal of the laws of nature" ["Über die umkehrung der naturgesetze", sitzungs- berichte der preussischen akademie der wissenschaften, physikalicch-mathhatiiee n9 144-153]. The European Physical Journal H, 46(1), November 2021. ISSN 2102-6467. doi: 10. 110140002pphh1133222220000003330
klasse,
URL http: /xx doi. 111..oo////...1//pp.......
s 213129-021-00033
Thomas M. Cover and Joy A. Thomas. Elements of information theory. John Wiley & Sons, 2012.
Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information processing systems, 26, 2013.
Paolo Dai Pra. A stochastic control approach reciprocal diffusion processes. Applied mathematics and Optimization, 23(1):313-329, 1991.
Carles Domingo-Eninn Enrich, Michal Drozdzal, Brian Karrer, and Ricky TQ Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. arXiv preprint irXiv:2409.0886), 2024.
Noam Elata, Tomer Michaeli, and Michael Elad. Psc: Posterior sampling- based compression, 2025.
URL htt.hhtpp///aatttt httprr//bs/24000 09896

---

Published as a conference paper at ICLR 2026
Giulio Franzese, Mustapha Bounoua, and Pietro Michiardi. MINDE: Mutual information neural diffusion estimation. arXiv preprint arXiv:311000031,, 2023.
Dror Freirich, Tomer Michaeli, and Ron Meir. A theory of the distortion-perception tradeoff in
wasserstein space. Advances in Neural Information Processing Systems, 34:25661-25672, 2021.
Dror Freirich, Nir Weinberger, and Ron Meir. Characterization of the distortion tradeoff for finite channels with arbitrary metrics. In 2024 IEEE International Symposium on Information Theory (ISIT), pages 238-243 243. IEEE, 2024.
Emilia Fridman and Uri Shaked. Robust hoc minimum entropy static output-feedhack control of singularly perturbed systems. Automatica, 36(8):1181-1188) 2000.
Chu-Li Fu, Xiang-Tuan Xiong, and Zhi Qian. Fourier regularization for backward heat equation.
Journal of Mathematical Analysis and Applications, 331(1):472-48, 2007.
Nikita Gushchin, Alexander Kolesov, Alexander Korotin, Dmitry Vetrov, and Evgeny Burnaev. En- tropic neural optimal transport via diffusion processes. arXiv preprint arXiv:2211.01 2022.
Nikita Gushchin, Sergei Kholkin, Evgeny Burnaev, and Alexander Korotin. Light and optimal schrödinger bridge matching. In Forty-first International Conference on Machine Learning, 2024.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor. In International confer- ence on machine learning. pages 1861-1870. Pmlr, 2018.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Zohar Jackson, César Souza, Jason Flaks, and Hereman Nicolas. Jakobossk/free--ppken-iiig dataset v1. 0.6. Zenodo, 2018.
Sergei Kholkin, Ivan Butakov, Evgeny Burnaev, Nikita Gushchin, and Alexander Korotin. Info- Bridge: Mutual information estimation via bridge matching. arXiv preprint arXiv:2502.01383, 2025.
Diederik Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Mare Lambert, Francis Bach, and Silvère Bonnabel. Entropy regularized variational dynamic pro- gramming for stochastic optimal control. 2025.
Eric Lei, Hamed Hassani, and Shirin Sacedi Bidokhti. Neural estimation of the rate-distortion function with applications to operational source coding. IEEE Journal Selected Areas in In- formation Theory, 3(4):674-686, 2022.
Eric Lei, Hamed Hassani, and Shirin Saeedi Bidokhti. On a relation between the rate-distortion function and optimal transport, 2023. URL /ecttp:://axxiw.or///bbat
00246
Christian Léonard. A survey of the Schrödinger problem and some of its connections with optimal transport. arXiv preprint arXiv:1308.02111 2013.
Guan-Homg Liu, Jaemoo Choi, Yongxin Chen, Benjamin Kurt Miller, and Ricky TQ Chen. Adjoint schri(" odinger bridge sampler. arXiv preprint arXiv:2506.22555 2025.
Willard Miranker. A well posed problem for the backward heat equation. Proceedings of the American Mathematical Society, 12(2):243-247, 1961.
Guy Ohayon, Hila Manor, Tomer Michaeli, and Michael Elad. Compressed image generation with denoising diffusion codebook models. arXiv preprint arXiv:2502.01188, 2025.
Erikki Oja and A. Hyvarinen. Independent component analysis: Algorithms and applications. Neu- ral networks, 13(4-5):41 1-430, 2000.
12

---

Published as a conference paper at ICLR 2026
Bernt Oksendal. Stochastic differential equations: An introduction with applications. Springer Science & Business Media, 2013.
A Paszke. Pytorch: An imperative style, tegh-performance deep learning library. arXiv preprint arXiv: 703, 2019.
Michele Pavon. Stochastic control and nonequilibrium thermodynamical systems. Applied Mathe- matics and Optimization, 19(1::17-202, 1989.
Michele Pavon and Anton Wakolbinger. On free energy, stochastic control, and schrödinger pro- cesses. In Modeling, Estimation and Control of Systems with Uncertainty: Proceedings of Conference held in Sopron, Hungary, September 1990, pages 334-348 Springer, 1991.
Georg Pichler, Pierre Jean A Colombo, Malik Boudiaf, Günther Koliander, and Pablo Piantanida. A differential entropy estimator for training neural networks. International Conference on Machine Learning, pages 17691-17715. PMLR, 2022.
Nikita Puchkin, Iurii Pustovalov, Yuri Sapronov, Denis Suchkov, Alexey Naumov, and De- nis Belomestny. Sample complexity of schr\" odinger potential estimation. arXiv preprint arXiv:25006.03043 2025a.
Nikita Puchkin, Denis Suchkov, Alexey Naumov, and Denis Belomestny. Tight bounds for schr' odinger potential estimation in unpaired data translation. arXiv preprint arXiv:2508.07392 2025b.
George Rapakoulias, Ali Reza Pedram, Fengjiao Liu, Lingjiong Zhu, and Panagiotis Tsiotras. Go with the flow: Fast diffusion for gaussian mixture models. arXiv preprint arXiv:24/2.0905, 2024.
Erwin Schrödinger. Sur la théorie relativiste de tique. In Annales de I' l'institut Henri Poincaré, volume 2. pages 269-310, 1932.
d' l'électron et I' I'interprétation de la mécanique quan-
Claude E. Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379-42 1948.
Claude E. Shannon. Coding theorems for discrete source with fidelity criterion. IRE Nat. Conv. Rec, 4(142-163):1, 1959.
Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Amaud Doucet. Diffusion schrödinger bridge matching. Advances in Neural Information Processing Systems, 36:62183-66223, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020.
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben
Poole. Score-based generative modeling through stochastic differential equations, 2021.
Lucas Theis, Tim Salimans, Matthew D. Hoffman, and Fabian Mentzer. Lossy compression with Gaussian diffusion. arXiv preprint arXiv:2206.08888 2022.
Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly, and Bernhard Schoelkopf. Wasserstein auto- encoders. arXiv preprint arXiv: 2017.
Dor Tsur, Bashar Huleihel, and Haim H. Permuter. On rate distortion via constrained optimization of estimated mutual information. JEEE Access, 2024.
Yibo Yang, Stephan Mandt, and Lucas Theis. An introduction neural data compression. Founda- tions and Trends@ in Computer Graphics and Vision, 15(2):113-22) 2023.
Yibo Yang, Stephan Eckstein, Marcel Nutz, and Stephan Mandt. Estimating the rate-distottonn function by Wasserstein gradient descent. Advances in Neural Information Processing Systems, 36, 2024.
Brian D. Ziehart, Andrew I.. Maas, J. Andrew Bagnell, and Anind K. Dey. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pages 1433-1438. Chicago, II., USA, 2008.
Jiayang Zou, Luyao Fan, Jiayang Gao, and Jia Wang. A revisit to rate-distortion problems via optimal weak transport theory. arXiv preprint arXiv:2501.0936) 2025.
13

---

Published as a conference paper at ICLR 2026
A SULER-MARUYAMA SAMPLING
Here we review the Euler-Maruyama (EuMa) Algorithm, which we used in our simulations to sample from the diffusion process Eq. (10). The sampling procedure given Alg.
Algorithm Murur-ma (EuMa)
Input: > drift u(æ, t, e), initial batch {Xo JM A and m 1,.... M do 3: Sample 2,t N (0,
Po. timestep A..
2: for f = 0, A.....,
4: X xx,, + u(X]" tis e) Ved,2,".
5: end for
6: Return {u(XE, ti, E), X1" M
B ENTROPY ESTIMATION
For the sake of completeness of Alg. here we present the techniques used in the paper for estimat- ing entropy (line in the Algorithm). We emphasize, though, that the entropy estimator black-box, where any method could be plugged-in, in, orthogonally to our main ideas.
In our experiments on low-dimensional settings used the approximated negative entropy method, for its simplicity and ease of compute. This could be hardly scaled to higher dimensions since requires the computation of large covariance matrices. For real-world settings used the scalable kernel method of Pichler et al. (2022)
B. NEGENTROPY
Negative entropy, negentropy OOja and Hyvarinen, 2000) of random variable X1 E I2 is the difference in entropy from the Gaussian distribution with the same second-order statistics. Explicitly
d negentropy(X:) & H(Z) - H()) - H(x) 2 log A log log(2
(25)
2
where A are the eigenvalues of the covariance matrix Ex, We have the following connection between negentropy and KI. divergence
Lemma B.1. (Kholkin et al. 2025 Corvllary A.3) Let 2 ~ N (Mi Ei) where pi, En are the mean
and covariance of X1, respectively. Then,
H(x)) = H(Z) DKL (P P2)
(26)
Using Eq. 26
we approximate the negentropy through the Donsker-Varadhan identity (Belghazi
al. 2018
negentrroyy(X)) DKI. xx (Px) ssup Ez [() - llog Ez-P e (())
(27)
which be estimated from samples (ef. Kholkin al. 2025 A.2) and Franzese et al. (0033
$ 3.2)) )
In simulations, model the argument in Eq. 27 we approximate the negentropy from M samples of X as
parametric model (z) where now
negentropy(X1,0 (X1, 1 M
Σ Ze (XT)
log
1 M
Σ (u(("")e)
(28)
z are i.i.d. Gaussian samples with the empirical mean and covariance of X,. The entropy H(X,
be estimated through Eq. 25
14

---

Published as a conference paper at ICLR 2026
B.2 KNIFE
The KNIFE estimator (Pichler et al. entropies. For x E I the empirical distribution is approximated using
2022) is a Gaussian-mixture plug-in estimator for differential
PKNIFE 0) >g- Ux (I; Hix A&),
(29)
where (:ar(: Mb; A&) are Gaussian kernels with mean and variance μkx As respectively, and ug. are weights, 2g 1.
The plug-in estimation then given by
HXx;;0) EEz~, log [ÉKNIFF (2; 0)) 2 H(X)). For tight estimation, H ()) is minimized over == {(uR, pcx )=)
(30)
15

---

Published as a conference paper at ICLR 2026
C PROOFS OF THINGS
C.I PROOF OF THM. 3.
Theorem C.1. (Thm. 3.1 in the Main text) LND LND. Furthermore, let u² (x, minimize the
surrogate objective
//' () + I(Xx)). 26
1
u" arg min
E
(31)
under
dx; uX,, f)dt + VedW,, No ~ Po-
(32)
Then, the distribution Px Px; of X2 associated with u² through Eq. 132 is the minimizer in Eq. 72)
where PT xx; the optimal reconstruction distribution Eq. 7, RD and n Pxg.x; the optimal plan.
Here, we also prove the opposite direction; whenever (P1, R.) minimizes Eq. 7. RD there exists drift term u(x, I) minimizing Eq. (3 under Eq. (32) where Xo, X1 π.
Proof. Let m) be solution Eq.
7, RD
where т E PI.. Then, we choose
T(w)=
W (w)=(
(33)
Z,
as probability law in IP))., It can be easily deduced that
(T..(la,. [| (2.aa) = O,
(34)
hence, from Eq. 18 -Eq. we have
|X 11
d H(T) log(2
DKL (TlI=>) = Ex
(35)
2e
which is equal Eq. 77, RD up
additive constant, depends only Po, P1 and c. Since minimizes Eq. 7. RD) for P1. it minimizes DKL (xI[==] over I1(Po, Pi). thus T minimizes
(KL(T]][')) over F( o, P1) (see Chen et al. (2021) Problem 4.2, Eqn. (4.8)); mutatis mutandis).
As solution to FiP.r) (T][W"][')) takes the form Eq. 10) with some drift function
[Loonard] 2013 Prop. 4.1).
Using Eq.
we get
1 E
[ [lu(x,,))]3dd)
d log(2 log(2
LRD
H((P))
(36)
2c
d
DKL + H(P,)
loog(2
(37)
2
DKL(T[[TT)) + H(P,)
log(2ITe)
(38)
2
[|X- {3]
2c En
H(T)) + H(Po) + (Pp))
(39)
LRD.
(40)
implying that CRD 2 ERD-
On the other hand, let u² as in Eq. 31 and let IP; = Px; Clearly, Px; u² minimize Eq. 12 Furthermore, the law 7 induced by
T dx; u" (X,, Ddl VedWo, X Po.
(41)
minimizes DT'(T"I[W)) LE lo (xt, over F(P,, hence it is so-
lution minreF(a.P;) (
Thus, according Léonard 2013 Prop. 2.3),
16

---

Published as a conference paper at ICLR 2026
(2,
O, Vx, Let x¹ Pxj.xx E [I, We have from Eq. (9)
||x x| - -X11
DKL(T"|[T") = Ex-
H(()))) ) + ((P)) + log(2T))
(42)
2x
DKL DTu((T>WW3)
(43)
E
(44)
2c
Thus,
| 1 - x11
LRD S Ey--
-H(=)) - H(()) + H(P))
(45)
2e
1
!/- ||u" (X7,
d
E
+ H(1)) 2 log(2
(46)
2E
ÊND,
(47)
yielding LRD LRD, which completes the proof. Note that arguments similar to Eq. 45
Eq.
47
yield that under A1, LRD -X..
C.2 PROOF OF THM. 3.2
Theorem C.2. (Thm. 3.2 in the Main text) Let pī
(1r) such that log (a) E C2.2 1] IR)) and
||V (z)] (x) + 0 as || + do for all t E [0, 1), satisfying the BHE
1
Pc = - Po(I) ~ Po.
(48, BHE)
ot
and let 1² -V log pt (I: where V log Pr(r)) is the Stein score function. Then, (u", .P.) is optimal pair in Eg. (15, var-TEC Eq.
[6, FPE) and the solution to Eq. (14, TEC) admits the SDE
dX, = eV lo VedW.
(49)
Proof. Let define the Lagrangian
L(u.p.H,A) / dæ S' dtllu(z, dttl (t)
Rd
1 { dx ('' dip(z,)) (((x)) + V. (up(z,() z 2<AenPe(())
Rd
f dxpı(x) logpr()) / (r)())(oo(o) -Po(x))
(50)
where dP denote the density function of Fo-
We now apply the following integration by parts property of the divergence: For g Rd R and f: Rdd + Rd and bounded domain D C Rd with boundary OD,
I o())V (())dx f(z)d== / Vg(x) f(z)dr
o f g(x)f(x) rdd
(51)
where V, V. are the gradient and divergence operators, respectively. If [[]g()))())) decays as ||>||| + DO, we can integrate over domains with large enough diameters, thus ignoring the boundary term and practically integrate over Rdd, In our setting, P, are scalar functions, and u is a field.
Integrating by parts, we have
[ dtp(x, (x) = m(x, 1)p (x:) 1(x, 0)po(a) L'' dtjl (p((())
(52)
17

---

Published as a conference paper at ICLR 2026
and
J dxpla, 1)V (up(æ, 1)) -
-fee dxpu . Vula, 6) + & upu Tdd
(53)
as well as
drplx, 1) 1)Axxp(x,))
Rd
1 drp(e,t) VVlzzz) Vp(z,t)
(54)
dxVm(e, 1) Vp(o, + o ((V((x,1)) iida
(55)
d=Vplz, t) Vp(z,t) + f pvu Ada & Iu (u(Vplz,t) - -VVV) 7da
(56)
JRd
dxV.m (x)) & pvu rda + & (VVplz,)) ida.
(57)
JRd
Provided that all boundary terms vanish as ||ز|| + co, and putting everything back together, we obtain
1 dx déllu(a, (x+))
L(u,P. n4
2 dt dæ ji(z;t) + u((,t) Vp(x, 1 (esmmp(o pe(x)
Rd
do(u(x, 1) elogpi (x))p(())
nd
[ dr(p(x, 0) -xxx))po(z) f (x)())dPo(x))
(58)
Taking the first variation to zero,
dL Du
= 0 yields
u (a. (p(x,)) 1).
(59)
From
Dp 8L = 0 we obtain (Hamilton-Jaconl equation),
ú(xxt) 1 Vula t)]? + == 0, p(x, e(1 + logpi(x)).
1 2
(60)
We also know that (Fokker-Planck equation)
(x) + V (p(x,) ()) - (xpp(()) = px) = dlPo
(61)
We now substitute solution of the form u(x,() <(1 + (-lggt())) (x)) into Eq. (60). and verify it satisfies our equations:
pt/p: + = Dpll/pp/ptl 2 - (Vpt/pe)
= =p:/Pr + デ Ppt ppl + 2
Σ (Pe,a./Pi - (Po./Pt)
(62)
- ( <)IlVpt/pel { - =0
(63)
where the last equality stems from Eq. 48, BHE)
18

---

Published as a conference paper at ICLR 2026
D EXTENDED RESULTS AND TECHNICAL DETAILS
For the sake of completeness and in-depth reading, in this section we extend $ S with full tech- nical details and additional results. We also present detailed and full-sized figures for improved accessibility.
D.1 GAUSSIAN SOURCES
Let P. = N (0, oī) A solution to Eq. 17, BHE is given by
1
(x(x)=
210g .0
(64)
v /2 ct)
The optimal controller is hence given by
u(x, 1) V leg = 0 ct
(65)
Let us denote a, o8-e and Gaussian where
It is easy to see that under u(x, t), Xo and X, are jointly-
o
dx, = a,dx + VedW,
(66)
U7 X, = Xu + ve / Up'dWs,
(67)
with
Ug = - S6 aodd =1
(68)
That is,
X1 (1 Xo + N1
(69)
0 = <(1 r)² / U. 2 = = - r)²
of = <(1 - r).
(70)
d
where N₁ is a Gaussian noise, independent of Xo. We can now compute the distortion
E [[{] - Xo) = -o0 + c(1 - r) = c - 2002 + +200 = c,
(71)
and the correlation
p 2 E| = (1 r) = = -E
(72)
and then also compute the MI by plugging
of = (1-))00 c(1 r) o -2--2 2e + 200 2 +
2002 -2
0
(73)
into
I(Xo: X1) = log
1
(74)
2
(1 - 2)
log
(75)
2
(o) c)o
(1 - r)2
log
1
(76)
2
1
log
(77)
To summarize, we obtained
D
R. (Xo
=
R = I(Xo:
-
log e
(78)
2
19

---

Published as a conference paper at ICLR 2026
R2D2 (ours)
R(D) [nat]
NERD [Lei et al 22
WGD [Yang et al 23.
R(D) (groundtruth)
0
0.0
0.1
0.2
0.3
0.4
0.5
2
MSE
IR() R(D)
0.04
R2D2 (ours)
NERD
0.02
WGD
0.1
0.2 2 MSE
0.3
0.4
Figure 7: The R(D) function of a 1-D Gaussian source. (top) Here, X, ~ N (0, 1) and we applied Alg. I to c [0.05, 0.95]. Green markers indicate higher precision. We also plot the analytical result (black line). (bottom) We compare our algorithm to NERD and WGD, where we observe that our method is more accurate.
and recovered the well-known result
1
2
RGause (D)
log
0 2D < o
(79)
2
2D
Note that the factor of 2 due to our convention for Gaussian vector sources and e's smaller than the eigenvalues of the covariance matrix Σ. In this
2 .MSE. This result can be easily generalized
case, the solution to Eq. [7, BHE) is
(x(x) -
(80)
In Fig. 7 we demonstrate the efficiency of Alg. [ on the 1-D case. We compare our method with NERD (Lei et al.) 2022) WGD (Yang et al. 2024), where we observe that R2D2 is clearly superior the existing methods in terms of estimation error, in both the high-rate and low-rate regimes.
D.2 MIXTURE OF GAUSSIANS
The latter Example can be easily extended to the of Gaussian mixture, where
Pa
Po(x)
20
(81)
2TO
As Eq. [7, BHE is a linear equation, the solution is now given by the superposition
N
(r(x) =
Ps
e
Rio
CE
0, min
(82)
2n (o?
Et)
20

---

Published as a conference paper at ICLR 2026
(D) [nats]
3
R2D2 (ours)
2
SLB
F0.000
0.0025
0.0050
0.0075
MSE 2
1
1.75
R2D2. (ours)
R(D) [nats
SLB
1.50
1.25
0.002
0.004
0.006
0.008
MSE
Pxo
Panaallyyiial) Px. (empirical)
pi
0.5
0.0
0
1
C
Figure 8: The R(D) function for mixture of Gaussians. Here, Xo Po is mixture Gaus- sians where ,4 = -.4, 0, o²² = 4 x 1,-2,5 5 x 10-2,6 x 10-2 2 = 1.. (top) We apply Alg. to E [4 10-4, 10.4,1.64 1.64 x 10-2] and compare the result with SLB. Green markers indicate higher precision. (middle) A closer look on error bars (inter-quartile range) over 8 evaluations. (bottom) For e = 1.56 x 10-2, we plot the reconstruction distribution P which is the distribution of the dif- fusion process' outcome, X₁. Observe that the empirical distribution matches the analytical result (bold line).
21

---

Published as a conference paper at ICLR 2026
1.0
6 =2. 1=2.72F-02 (r(rff.)
0.5
(analytic)
0.0
-1
0
analytic u
uo ( t)
1:0,000
-=0.50
1-99
1-0.4
0
R(D) [nats]
R2D2. 1.D2..ours) (ours)
1.6
SLB
1.4
0.006
0.008
0.010
0.012
0.014
2 MSE
Figure 9: The R(D) function for mixture of Gaussians (second example). Here, X, is mixture of Gaussians where 14 88 .8, of 4 10-2, 5 10-2 x 10-2, p: 3-
(top) The empirical reconstruction distribution for 2.72 x 10-2 matches the analytical result. (middle) Trained controller model ug (for 2.8 x 10 00-), compared to analytical result at times
t 0, 0.5, 0.99. (bottom) RD function estimated over 16 evaluation steps (medians and inter- quartile ranges).
22

---

Published as a conference paper at ICLR 2026
The optimal controller u(æ, <V log Pe (x) is derived accordingly. We illustrate this result in
Fig. 8 where N and (4, 0,.4, of = 4 x 1002. 10 66x 10 P: = 3' We apply
Alg. to E [4 10-1, 1.64 x 10-2] and compare the estimated RD function with Shannon's 's lower Berger 2003)
bound (SLB) (eLB)(CCon) Cover and Thomas 2012
(IP)) 2 log(Arel)
(83)
approximated here from M = =211 i.i.d. samples X Po by
H(PP)
M Σ lx")o(XX""..
(84)
In practice, we estimated Eq. 84 for 8 independent trials, and used the median value for our ap- proximation. For e 1.56
10 We further plot the reconstruction distribution Fi, which the probability law of the diffusion process' outcome. We observe that empirical distribution obtained by Alg. matches the closed-form result Eq. (82
In the setting of Gaussian mixtures, we conducted an additional experiment in which μ4; -.8,00 0, .8. The results are given in Fig. 9 where we also plot the outcome of the deep controller model we train, compared to the desired product.
D.3 NON-GAUSSIAN MIXTURE
Consider the source X, drawn from the mixture Po(z) I sinci (M²,) where sinc(x) & 2x m, are appropriate normalization factors. Recall that the characteris- 3
sim(x) E C (R), and C₁ =
tic function in this case is
po(w) = f po(t)e in dt Σ P.O, mcpolmaw).
(85)
where (with being the convolution operation) Pa(w)-
(86)
x(1 [wwl)+ 7(1 - ww)+
(87)
2T
((w - 4) sign (w 4) 4(w 2) sign (w - 2)
(88)
96
+Gw sign 4(2 w) sign (2 + w) (4 w)3 sign (4 + w))
(89)
which vanishes outside (lwl < 4]. Now, for non von-vanig mix distributions and small enough e's, by Eq. (19p we can write the solution to Eq. [7, BHE) as
1 2n
2 Σ
P.O, mi 1/" (o((mw)) cos(war)e aw ddw. (90)
(x(x)=
2T
We numerically approximate
N
K-1
po cos( 4k
4k K
(x) C.C,
m;
Σ
(91)
K
K
km
Figure 10 demonstrates this result for N = 4, m, = [1,v2, (m,'s were chosen such that
(o(x) 0 everywhere I) and p; 1. where numerically integrated Eq. (91) with K 4 x 32 x 10 in order to approximate the reconstruction distribution pı (z()) for different values of We emphasize that although this is a toy problem, to the best our knowledge, no other technique known to tackle this case.
23

---

Published as a conference paper at ICLR 2026
x10
X10
10
2.94-
144
2.5
2,92
2.0
2.00
E = 2.066-02
-
2.88
0.8
E =7.50e-03
2.86-
po(x)
1.0
2.84-
8.5
2.82
0.2
0.0
00 25
0.00
0.25
10
12
Figure 10: Reconstruction distribution pl (x) of non-Gaussian mix source. In the (left) pane, approximate reconstruction distribution Eq. (90) by numerical integration. closer look different points is provided in the (middle) and (right) panes.
D.4 EXAMPLE: CI DATASET
We now demonstrate the efficiency of Alg. T on realistic high-dimensional source. More specifi- cally, as input to the Algorithm, we sample 4 x 4 grayscale image patches from the 'Cifar10' dataset (Kiihevsky and Hinton 2009). Pixel values ficiency of our method in solving this problem. In the upper pane, we present the RD function, as estimated by R2D2 (Alg. T.. In the lower pane, present images, drawn from the reconstruction distribution P1 for different e's.
normalized to [0, 1]. Fig. demonstrates the ef-
24

---

Published as a conference paper at ICLR 2026
10.0
R2D2 (ours)
R(D) hatt
NERD [Lei et. al.2022]
7.5
WGD [Yang al.2024]
O
5.0
2.5
0.01
0.02
0.03
0.04
0.05
0.06
0.07
MSE
source
6 1.77c
03
c= 5.05e
03
1.16c
02
= 2.47c
02
Rate
10.73
6.65
4.46
3.12
Distortion:
1.02e 02
2.32c 02
4.15e 02
7.09c - 02
Figure 11: The R(D) function of Cifar10 images (best viewed on screen). Here, Xo is a random x patch from a grayscale image. (top) The RD function, estimated by R2D2 (Alg []. (bottom) Patches drawn from the reconstruction distribution X, obtained for different c's.
25

---

Codes for the WGD baseline (Yang al. (2024) are pro- Our codes are publicly available at Published as a conference paper at ICLR 2026
E IMPLEMENTATION NOTES
E. GENERAL DETAILS
For all experiments, used 2 fully-connected DNN models:
The controller ug taking tx) R [0, [emins Cmax] as input and whose output is in IR..
1-D experiments we used Zu network for estimating the negentropy (in 1-D experi- ments), taking (z,c) R (miins Emax] and returning scalar value (see App. B for details).
For real dataset experiments, alternatively used network for representing KNIFE pa- rameters.
Despite having different input and output layers, both models are of the same depth and hidden- layer sizes. We used LeakyReLU activation following each hidden layer. The two models were trained in 3-ratio of update steps, using the ADAM optimizer (Kingma 2014) with parameters (.9,. .999) and learning rate a.
At each evaluation step, draw batch of samples and evaluate R(D) according to Alg. When- ever there are more than one evaluation step or independent seeds, the presented (R, -vavulues are the medians over all steps, while error bars indicate the inter-quartile (25%-75%) range.
Codes for the NERD baseline (Lei et al. 2022) are provided by the authors at httss//sthubccoom/eeeee/ttRRRRR vided by the authors at https//ggihbbbboom/MMguuuuu
All experiments were implemented in PyTorch Passke 2019) environment, and performed using
NVIDIA RTX A6000 GPU,
E.2 SIMULATION PARAMETERS
Gaussian sources (Fig.)
Cmin Emax 0.025,0.975
DNN hidden layers:
Hidden layer size: 128
Step size: A, 100
Train steps: 25,000
Batch size (train): M = 512
Experiments: 64
Batch size (evaluation): 1024
Learning rate: a = le-3
Mixture of Gaussians (Fig.
Emin <max =4e-4,1.666
DNN hidden layers: 4
Hidden layer size: 128
Step size: A.
Train steps: 3.7M
Batch size (train): M 256
Evaluation steps: 8 (32 at high-precision)
Batch size (evaluation): 1024
Learning rate: Se-4
26

---

Published as a conference paper at ICLR 2026
Mixture of Gaussians #2 (Fig. 9
Emin Cmax 1.2e-2.2.8e-2
DNN hidden layers:
Hidden layer size: 100
Step size: A 100
Train steps: 855,000
Batch size (train): M = 256
Evaluation steps: 16 (64 at high-precision)
Batch size (evaluation): 1024
Learning rate: a le-3
Cifar10 dataset (Fig.
Cmin, Cmax 0.0005,0.026
DNN hidden layers:
Hidden layer size: 1024
Step size: A; 128
Train steps: 292,968
Batch size (train): M = 256
Evaluation steps: 128
Batch size (evaluation): 2048
Learning rate: a 2.5e-4
KNIFE parameter: 512
27


Reviews:
[{"review_id": "rF8QfNdYIj", "reviewer": "Reviewer_ZRmZ", "summary": "This paper introduces Terminal-Entropy Control (TEC), a novel approach to stochastic control that establishes a link between rate–distortion (RD) and entropic optimal transport (EOT) for continuous data sources under mean squared error (MSE) distortion. The authors demonstrate that the classical RD tradeoff between rate and distortion is equivalent to a tradeoff between control energy and the terminal-state differential entropy, and they characterize the optimal solution to TEC under certain regularity conditions. The authors also propose R2D2, a novel neural method for estimating the RD function and the reconstruction distributions using a simple diffusion model. Theory yields closed-form reconstruction distributions for mixtures; experiments validate on Gaussian, Gaussian mixtures, and small CIFAR-10 patches.", "questions": "1. Relation to prior work (WGD / NERD):\nCould the authors clarify why comparisons to WGD and NERD are only held for the 1D Gaussian case?\nIs there a theoretical or computational reason these methods cannot handle Gaussian mixtures or CIFAR-10 patches?\n2. Stability:\nThe backward heat equation underlying TEC is known to be ill-posed and potentially unstable. Could the authors elaborate on the numerical stability of training, especially for empirical datasets? In particular, how sensitive is R2D2 to the choice of entropy estimator (negentropy vs. KNIFE) in high-dimensional settings, and were any regularizations used to maintain stability? How strong or restrictive are the regularity constraints in practice—for instance, when $p_0$ represents real, high-dimensional data such as CIFAR-10 patches rather than smooth analytical densities?\n3. Experimental setup:\nWhy are experiments performed on grayscale CIFAR-10 patches instead of full RGB images?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Originality:  The paper introduces a novel dynamic formulation of the rate–distortion problem through the Terminal Entropy Control framework, which reinterprets RD optimization as a stochastic control process.\n- Theoretical depth: The work provides substantial mathematical analysis and formal proofs, including clear optimality conditions via the Backward Heat Equation (BHE) and score-based drift characterization.\n- Practical method (R2D2): Straightforward training; works from samples; empirically outperforms NERD/WGD on 1D Gaussian and matches analytic mixture results; scales to small image patches.", "weaknesses": "- Limited comparison: Experiments benchmark primarily against NERD and WGD only on the 1D Gaussian case. There is no comparison on gaussian mixtures or CIFAR-10, nor a discussion of why prior estimators cannot be adapted to these settings. Including why R2D2 generalizes better would enhance empirical credibility. \n- Clarity and positioning: \n    - Although the theoretical connection between RD, OT, and Schrödinger bridges is elegant, the manuscript could more explicitly contrast TEC with prior entropic OT formulations (e.g. WGD) and clarify where its generality or computational benefits concretely exceed them.\n    - While the paper claims a “closed-form solution” for Gaussian mixtures, the actual rate–distortion values R(D) are still obtained via Monte Carlo or neural estimation. Clarifying this distinction would strengthen the paper’s theoretical claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7gfmrRywT3", "reviewer": "Reviewer_GzUT", "summary": "The authors consider the problem of probabilistic data compression while keeping the distortion rate at an acceptable level. They look at this problem through the lens of entropy-regularized stochastic optimal control and suggest the Terminal-Entropy Control (TEC) method aiming at minimizing the loss function with respect to the admissible control $u(x, t)$ and the terminal distribution $\\mathbb P_1$. The authors elaborate on connections of the TEC formulation with the approach based on entropic optimal transport (Theorem 3.1). They also derive an explicit expression for the optimal control function $u^\\star(x, t)$ (Theorem 3.2). Finally, the authors illustrate the performance of their procedure with numerical experiments on artificial and real-world data.", "questions": "1. Can you simplify the proof of Theorem 3.2 in the present submission using Theorem 3.2 from [Dai Pra, 1991]?\n\n2. Can you simplify the loss (12) using Theorem 3.2 from [Dai Pra, 1991]? Is it possible to exclude $u$ from the optimization problem?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "According to the numerical experiments, the method outperforms its competitors.", "weaknesses": "1. I do not see a reason why numerical experiments on one-dimensional Gaussian data and CIFAR-10 were not attached as a supplement. This raises concerns about reproducibility of the results reported in Section 5.\n\n2. It seems that the authors missed a very relevant paper [Dai Pra, 1991], where the author studied the problem of stochastic optimal control. His findings were recently applied to generative modelling (see, for instance, [Rapakoulias et al., 2023] and [Puchkin et al., 2025]). I suppose that the proof of Theorem 3.2 in the present submission can be simplified if the authors took into account Theorem 3.2 from [Dai Pra, 1991], where the author derived an explicit expression of the optimal control function $u^\\star(x, t)$ for a fixed $\\mathbb P_1$ through Schrodinger potentials (see, e.g., [Korotin et al., 2024] for the definition). I would be grateful if the authors could elaborate on this point.\n\n3. In the suggested algorithm, one has to optimize both the terminal distribution $\\mathbb P_1$ and the control $u(x, t)$. However, for any fixed $\\mathbb P_1$ the form of the optimal control $u^\\star(x, t)$ is known (see Theorem 3.2 in [Dai Pra, 1991]). Moreover, Dai Pra [1991] derives the corresponding value of $(2\\varepsilon)^{-1} \\mathbb E \\int_0^1 \\|u^*(x, t)\\|^2 \\, dt$. Hence, the loss can be substantially simplified.\n\nIn view of the weaknesses 2 and 3, I would not recommend the paper for acceptance in its present form. I think that it will benefit from a revision, if the authors take into account the results of Dai Pra.\n\n\n**Minor remark**\n\nOn page 4 the authors write: ``Recent developments (Gushchin et al., 2022) has drawn an equivalency (up to an additive constant, depends on $\\mathbb P_0$ , $\\mathbb P_1$ , $\\varepsilon$) between SB and EOT, where the latter can be optimized via a game-theoretic formulation.'' The connection between SB and EOT was known far before (Gushchin et al., 2022). In particular, it was mentioned in the survey of Leonard (2013).\n\n**References**\n\n[Dai Pra, 1991] Paolo Dai Pra. A stochastic control approach to reciprocal diffusion processes. Applied Mathematics\nand Optimization, 23(1):313–329, 1991.\n\n[Korotin et al., 2024] A. Korotin, N. Gushchin, and E. Burnaev. Light Schrodinger bridge. In The Twelfth\nInternational Conference on Learning Representations, 2024.\n\n[Puchkin et al., 2025] N. Puchkin, I. Pustovalov, Y. Sapronov, D. Suchkov, A. Naumov, and D. Belomestny. Sample complexity of Schrodinger potential estimation. Preprint. ArXiv:2506.03043, 2025.\n\n[Rapakoulias et al., 2024] G. Rapakoulias, A. R. Pedram, and P. Tsiotras. Go With the Flow: Fast Diffusion for Gaussian Mixture Models. Preprint. ArXiv:2412.09059v3, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FBVZns0yko", "reviewer": "Reviewer_GSoo", "summary": "This paper exploits the connection between rate-distortion (RD) and entropic optimal transport to propose a novel stochastic control formulation, and use a classic result dating back to Schrodinger to show that the tradeoff between rate and mean squared error distortion is equivalent to a tradeoff between control energy and the differential entropy of the terminal state, whose probability law yields the reconstruction distribution. For a special class of sources, they show that the optimal control law and trajectory in the space of probability measures are given by solving a backward heat equation. In the more general case, their approach gives rise to a numerical solution method, estimating the RD function using diffusion processes with a constant diffusion coefficient.", "questions": "Section 3.3 seems to focus on mixtures. Will there be any non-mixture examples?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "They present a novel stochastic control formulation that is regularized by terminal uncertainty, and show that this formulation is equivalent to the RD problem. \nThey characterize the optimal solution under some regularity conditions.\nFound a closed-form solution for the reconstruction distribution of a Gaussian-mixture source.\nProposed a novel neural method for estimating the RD function and the reconstruction distributions, using a simple diffusion model.", "weaknesses": "In some figures, the font size look weird. This is in the Appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GxSNmRs1qf", "reviewer": "Reviewer_ndvv", "summary": "This paper introduces a method to compute the _rate-distortion function_ $R(D)$, which is the smallest information rate needed to achieve a given level of distortion $D$. The approach focuses on the case where the data source is continuous, and the average distortion is quadratic, $D(X, \\hat{X})=\\tfrac{1}{2} \\mathbb{E}[\\| X - \\hat{X} \\|^2]$. Then, the problem reduces to an entropic optimal transport problem, which, in the low distortion regime, can be mapped to a Schrödinger bridge problem. Through this chain of reasoning the task of determining $R(D)$ is converted to a free energy minimization problem, which the authors refer to as _terminal-entropy stochastic control_, wherein the optimal rate corresponds to a balance between control energy and terminal state uncertainty. This optimum can be determined by parameterizing the control with a neural network and scanning through the free-energy landscape, which R2D2 algorithm presented in the paper. Experiments with some mixture distributions and the CIFAR-10 dataset demonstrate the algorithm at work.", "questions": "1. The core arguments of the paper bear a strong resemblance to the _adjoint-matching_ paper from last year [2]. Briefly, given samples of a distribution $p_{\\rm base}$ that paper addresses the problem of sampling from the tilted distribution $e^{r(x)} p_{\\rm base}$. It seems to me that this is also a form of distortion. While $r(x)$ is fixed in their case, making it free may allow their argument to be extended to some class of distortions besides the one addressed in your paper. Could this be an interesting direction for follow-up work?\n\n2. The terminal entropy stochastic control picture in Sec. 5 looks a lot like max-entropy RL, which you mention in the introduction. Both problems extremize an expected cost (or distortion) while regularizing by an entropy. Does this speak to a deeper connection between compression and learning? Is it correct to say that _the optimal policy is the most efficient compressor of the admissible solutions at a fixed cost_? Sharpening such connections could make the paper much more compelling.\n\n3. On the difficulty of determining $H(X_1)$ in high-dimensions, if you are simulating the trajectories for different $u_\\theta$ in Algorithm 1, is it possible to use the samples at $t=1$ to compute that entropy, say with the approach from [3] you cited?\n\n4. Around line 207, what does it mean when you write $W_0^\\epsilon \\sim \\mathbb{P}_0$? If $\\mathbb{P}_0$ denotes the source distribution, is $W_0^\\epsilon$ the initial state under a small diffusion of it?\n\nOverall I think this is a good paper that can be further improved with some minor changes. I would also appreciate if the authors can address my comments/questions.\n\n[1] Pavon, M., “Stochastic control and nonequilibrium thermodynamical systems,” Applied Mathematics & Optimization, vol. 19, pp. 187-202, Jan. 1989.\n\n[2] Domingo-Enrich, C., Drozdzal, M., Karrer, B. & Chen, R. T. Q. \"Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control,\" ICLR 2025.\n\n[3] Franzese et al., “MINDE: Mutual Information Neural Diffusion Estimation,” ICLR 2024\n\n[4] Chetrite, Raphaël; Muratore-Ginanneschi, Paolo; Schwieger, Kay. “E. Schrödinger’s 1931 paper ‘On the Reversal of the Laws of Nature’ [‘Über die Umkehrung der Naturgesetze’, Sitzungsberichte der preussischen Akademie der Wissenschaften, physikalisch mathematische Klasse, 8 Nº 9 144-153].” arXiv preprint arXiv:2105.12617 (2021).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "1. The paper leverages both established literature and new developments in stochastic control theory and entropic optimal transport to address their central problem.\n2. The proposed solution is elegant, and the argument is generally well-presented.", "weaknesses": "1. The authors state that their approach is restricted to quadratic distortion, and the low-distortion regime. Why is this corner interesting from a practical standpoint? Or was it chosen because it makes the problem tractable? The paper would benefit from motivating this choice early on.\n\n2. In its present state, the paper reads like a stochastic control paper which shows that the rate-distortion function is the minimum of a free energy functional. The numerical solution to the latter involves a neural network. Beyond this, the paper makes no wider connection to machine learning in general. See my second question below.\n\n3. Per my understanding, the method does not scale well to higher dimensions. The main bottleneck is the estimation of $H(X_1)$, as explained in appendix B of the paper. If this is an actual limitation it should be highlighted earlier in the paper. See my third question below.\n\n4. Not really a weakness, but the derivations in Sec. 3.2 appears to be mostly a special case of the one presented in [1]. Include this reference.\n\nMinor comment about formatting: The captions for the subfigures in Figure 1 could be spaced apart more for readability.\nMinor comment about references: I encourage you to cite [4], which is an English translation of Schrodinger's paper 'On the reversal of the laws of nature.'", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)