InfoBridge: Mutual Information estimation via Bridge Matching
Authors: Sergei Kholkin
Ivan Butakov
Evgeny Burnaev
Nikita Gushchin
Alexander Korotin
Venue: ICLR 2026 Poster (2026)
Keywords: Mutual Information
Diffusion Bridge Models
Bridge Matching
Primary area: generative models

Abstract: Diffusion bridge models have recently become a powerful tool in the field of generative modeling. In this work, we leverage their power to address another important problem in machine learning and information theory, the estimation of the mutual information (MI) between two random variables. Neatly framing MI estimation as a domain transfer problem, we construct an unbiased estimator for data posing difficulties for conventional MI estimators. We showcase the performance of our estimator on three standard MI estimation benchmarks, i.e., low-dimensional, image-based and high MI, and on real-world data, i.e., protein language model embeddings.

Published as a conference paper at ICLR 2026
InfoBRIDGE: MUTUAL INFORMATION ESTIMATION VIA BRIDGE MATCHING
Sergei Kholkin
Ivan Butakov
Applied Al Institute, Russia kholk k indd@gmai 1. com
Applied AI Institute, Russia,
MIRAI, Russia Institute of Numerical Mathematics (RAS),
Russia
ivan, waa.bbttakko@pppleeeea....
Ergeny Burnaev
Nikita Gushchin
Applied Al Institute, Russia, AXXX, Russia
Applied AT Institute, Russia,
AXXX, Russia
i . www.sguuhhhhhhhiii...... com
Alexander Korotin
Applied AT Institute, Russia,
AXXX, Russia iamalexkorrtiieemaall.cc com
ABSTRACT
Diffusion bridge models have recently become powerful tool in the field of generative modeling. In this work, we leverage them to address another important problem in machine learning and information theory, the estimation of the mutual information (MI) between two random variables. Neatly framing MI estimation domain transfer problem, construct an unbiased estimator for data posing difficulties for conventional MI estimators. We showcase the performance of estimator three standard MI estimation benchmarks, i.e., low-dimensional, image-based and high MI, and on real-world data, i.c.. protein language model embeddings. The code for our estimator can be found
INTRODUCTION
Information theory offers an extensive set of tools for quantifying probabilistic relations between random variables. It is widely used in machine learning for advanced statistical analysis (Berrett & Samworth 2017| [Sen et al., 2017 Duong & Nguyen 2023b) [Bonnoua et al.) 2024a)) assessment of
deep neural networks' performance and generalization capabilities (Tishby & Zaslavsky [2015] Xu &
Raginsky| 2017 Goldfeld et al. 2019 Steinke & Zakynthinou 2020 Amjad et al. [022 Butakov
self-supervised and sumi-super learning (Linsker 1988 Bell & Sejnowski 1995
Hjelm et al,, 2019; Stratos 2019; Bachman et al. 2019 Veličković et al. 2019] van den Oord et al.
2019
Tschannen et al. 2020) and regularization or alignment generative modeling (Chen et al.
2016
Belghazi et al. 2018: Ardizzone et al. 2020 Wang et al. 2024).
The majority of the aforementioned applications revolve around one of the central information- theoretic quantities- - mutual information (MI). Due to several outstanding properties, MI is widely used invariant measure of non-linear dependence between random variables. Unfortunately, recent studies suggest that the curse of dimensionality is highly pronounced when estimating MI (Gold feld al. 2020, McAllester & Stratos 2020). Additionally, it is argued that long tails. high values of MI and some other particular features of complex probability distributions can make mutual information estimation even more challenging (Czyž et al. 2023). On the other hand, recent develop- ments in neural estimation methods demonstrate that sophisticated parametric estimators can achieve notable practical success in situations where traditional mutual information estimation techniques struggle (Belghazi et al. 2018; van den Oord et al., 2019; Song & Ermon] 2020; Rhodes al.) 2020
Ao & Li 2022: Butakov et al. 2024a). Among neural estimators, generative approaches
"osscow Independent Research Institute of Artificial Intelligence

---

Published as a conference paper at ICLR 2026
of particular interest, they have proven to be effective in handling complex data (Duong & Nguyen 2023at Franzese et al. 2024 Butakov et al. 2024a). Since MI estimation is closely tied can argue that leveraging state-of-the-ar
to approximation of joint probability distribution, generative models, e.g., diffusion models, may result additional performance gains.
Diffusion Bridge Matching. Diffusion models are a powerful type of generative models that show
an impressive quality of image generation (Ho et al., 2020: Rombach et al.) (022). However, they
have some disadvantages, such as the inability to perform data-to-data translation via diffusion. To
tackle this problem, novel promising approach based on Reciprocal Processes (Léonard et al. (014) and Schrödinger Bridges (Schrödinger 1932, Léonard (0144)) has emerged. This approach is called the diffusion bridge matching and is used for learning generative diffusion processes for data-to-data
translation. This type of model has shown itself as a powerful approach for numerous applications in
biology (Tong al. 2024 Bunne et al.] 2023) , chemistry (Somnath et al. 2023) [gashov et al. (0024)
computer vision (Liu et al. [2023] Shi et al. [023 Zhou ct al. 2024), speech processing (Chen et al.
2023), unpaired learning (Shi et 2023 Gushchin et al.) 2024b 2023b Ksenofontov & Korotin
2025 Kholkin et al. 2026 Carrasco et al. 2026) and beyond (He et al. 2024; Gushchin et al. 2025)
Contributions.
In this work, we employ the Diffusion Bridge Matching for the MI estimation.
1. Theory. We propose an unbiased mutual information estimator based on reciprocal processes. their diffusion representations and the Girsanov theorem ($4.1).
2. Practice. Building on the proposed theoretical framework and the powerful generative methodol- ogy of diffusion bridges, we develop practical algorithm for MI estimation, named InfoBridge (4.22 We demonstrate that our method achieves performance comparable to existing approaches on dineesenniional benchmarks and superior performance on image data benchmarks, protein embeddings data benchmark and high MI benchmark ($5).
Notations. We work in IRD, which is the D-dimensional Euclidean space equipped with the Eu- clidean norm Il. We use PIR to denote the absolutely continuous Borel probability distributions whose variance and differential entropy are finite. To denote the density of P(I at a point r E TDD, we use ((x)). We write KT. (-I1-) to denote the Kullback-Leibler divergence between two dis- tributions. We denote the Dirac delta function as 8. We use Ω to denote the space of trajectories, i.e., continuous -valued functions of t [0, 1). We write ((2) to denote the probability distributions on the trajectories S2 whose marginals at f = 0 and f belong to P(I this is the set of stochastic processes. We use dW, to denote the differential of the standard Wiener process W E P(2). We use denote the distribution of stochastic process Q conditioned on Q's values zo and
Qizo and Q = To. 11 at times / = 0 and /. = 0, 1, respectively. For process Q P()),, we denote its marginal distribution at time by q(xx2) E P) and if the process is conditioned on its value F, at time s, we denote the marginal distribution of such process time by q(<a(x,)) E P(B SDE states for Stochastic Differential Equation (Øksendal 2003 55).
2 BACKGROUND
Mutual information. Information theory is well-established framework for analyzing and quanti- fying interactions between random vectors. this framework, mutual information (MI) serves as a fundamental and invariant measure of the non-linear dependence between two -valuue random vectors Xo, X1. It is defined as follows:
T T(Xo; X1) del KL. (T [xx πx,),
where Xx,.x, and 1xx. Ix, are the joint and marginal distributions of a pair of random vectors (Xo. If the corresponding PDF πT T(zg. 21) exists, the following also holds:
T<z. æ1)
I(Xo: X1) = En( log
π (
Mutual information is symmetric, non-negative and equals zero if and only if X, and X, are independent. MI also invariant to bijective mappings: I(X,;XX = ([a(X));; if exists exissss and g. g are measurable (Cover& & Thomas 2006; Polyanskiy & Wul 2024).
Brownian Bridge. Let W be the Wiener process with a constant volatility e > 0, i.c., it is described by the SDE dW VedW,, where W, is the standard Wiener process. Let W denote the (0o.0)
2

---

Published as a conference paper at ICLR 2026
process W conditioned on its values To, 21 times t 0, 1, respectively. This process W [20,21 is
called the Brownian Bridge (lbe 2013 Chapter 9).
Reciprocal processes. Reciprocal processes are a class of stochastic processes that have recently gained attention of research community the contexts of stochastic optimal control (éoonard et al. 2014), Schrödinger Bridges (Schrödinger 1932. Léonard 2014a) and diffusion generative modeling (Liu et al. 2023 Gushchin 20.442 our paper, we consider a particular case of reciprocal processes which are induced by the Brownian Bridge W (ro.21'
Consider a joint distribution t (200 21) E P(TRXx2) and define the process Qr E P(Q2) as a mixture of Brownian bridges W (1o.. with weights π (20..1)) 21):
Qr def /Wi.... (roo.,dT(app) [Wino.nd (10,
This implies that to get trajectories of Qr one has to first sample the start and end points, 2o and 21. times t 0 and t from it (20, .1) and then simulate the Brownian Bridge W |z0.2, Notice that process Qx depends on e, but next in the paper do not write and just stick Qn notation. Due to the non-causal nature of trajectory formation, such a process is, in general, non markovian. The set of mixtures Brownian Bridges can be described as:
{Q P(L)) s.t. 3m P(IDXx2). :Q Qx)
and is called the set of reciprocal processes (for W³).
Reciprocal processes conditioned on the point. Consider reciprocal process Qn conditioned
on some start point Zo. Let the resulting process be denoted as Q which remains reciprocal.
Then process Qriza is known as the Schrödinger Föllmer process (Vargas al. 2023). While QT.
in general, not markovian, Qu is markovian. Furthermore, if Rp 2
x
< 00, is
diffusion process governed by the following SDE, i.e., see Proposition
A.3
Qiio dx, Uzo (2,, t)dt + VedW,, To 6(zg),
3 It
Uno (It, 1) = Egg. (211
Representations of reciprocal processes. The process Qn can be naturally represented as a mixture of processes Q7jo conditioned on their starting points Tu:
Qx = Q - (T
Therefore, one may also express Q. via an SDE but with non-markovian drift (conditioned on 20):
Qx dæ, v( zy)dt VedW, Ng (2x));
Zt
v( t, zo) Uni (xe, t) Egg, (xz (m)))))
Conditional Bridge Matching. Although the drift Uzo (Zz, t) of Qx in (3) admits closed form, it usually cannot be computed estimated directly due to the unavailability of way easily sample from (2) [e,, .o). However, can be recovered by solving the following regression problem (Zhou
et al. 2024)
I1 - 2t u(
Uxo arg min Bagin (e
(5)
1-1
where (t) denotes the uniform distribution over t E [0, 1], and the optimization is performed over drift functions DD x [0, 1] IR The same holds for the Qx and its drift v(ze, xo) through the addition of expectation w.r.t. ((x)):
F1 - De u(x (00,,,,00) m00 1-t
x = arg min Ey (end
(6)

---

Published as a conference paper at ICLR 2026
I1 Ut u( L 30l
arg min y. 20).
where u TR. (0, RDD RDD Problem
usually solved with standard deep learning techniques. Namely, one parametrizes with neural network Ug, and minimizes 7 using stochastic gradient descent and samples drawn from qu (2o, 2t, x1). The latter sampling is easy if one can sample from x (2o, (10, 21). Indeed, In (To, 21) = 4r (Iy (2g, 21 z1) (To, T1), and one can sample first from π Tg. F1) and then from Qx (Irg (2o 21), which is the time slice of the Brownian Bridge (Korotin et al. 2024 Eq 14), i.e., Wi (-1t).
Such a procedure of learning drift with neural network is popular in generative modeling to solve a problem of sampling from conditional distribution { and is frequently applied in
the image-to-image transfer (Liu et al. 2023). The procedure of learning drift v(2;, 3o (7) is
usually called the conditional (or augmented) bridge matching (De Bortoli et al. 2023 Zhou et al.
2024). In addition, such procedure can also be derived through the well-celebrated Doob h-transform
(De Bortoli et al. 2023 Palmowski & Rolski 2002) or reversing diffusion (Zhou et al. 2024.
3 RELATED WORK
Mutual information estimators. Mutual information estimators fall into two main categories: non-parametric and parametric. Parametric estimators are also subdivided into discriminative and generative (Song& & Ermon, 2020 Federici al., 2023). Beyond this natural categorization, also distinguish diffusion-based approaches to better position our method w.r.t. prior work.
Non-parametric estimators. Classical MI estimation methods use non-parametric density esti- mators, such as kernel density estimation [Weglarczyk] 2018 Goldfeld et al.) 2019) and k-nearest
neighbors (Kozachenko & Leonenko 1987 Kraskov al. 20044) Berrett al. 2019). The estimated densities are plugged into (2. and MI is computed via Monte Carlo integration. While appealing in
low-dimensional settings, these approaches have been shown to fail on nigh-dimensional complex
data (Goldfeld et al., 2019 $5.3: Czyž et al. 2023 $6.2; Butakov et al., 2024a Table 1).
Non-diffusion-baseed generative estimators. Parametric density models such normalizing flows and variational autoencoders can be used to estimate MI via density estimation (Song & Ermon 2020; McAllester & Stratos 2020 Ao & Li 2022; Duong & Nguyen| (023a).. However, results in (Song & Ermon 2020) Figures 1,2) show that direct PDF estimation may introduce significant bias.
To address this, recent works (Duong & Nguyen 2023a Butakov et al. 2024a Dahlke & Pacheco
2025) avoid PDF estimation and instead estimate the density ratio in (2) leveraging the invariance
property of MI. While these methods perform better on synthetic benchmarks, they may still suffer from inductive biases introduced by simplified closed-form approximations.
Discriminative estimators. Another class of MI estimators relies on training a classifier to distinguish
between samples from the joint distribution π (Eg.22) ) and the product of marginals π (2 (x: ) ) as in MINE (Belghazi et al., 2018), InfoNCE (van den Oord et al. 2019), fDIME (Letizia et al., 2024) and SMILE (Sog & Ermon 2020). These approaches use variational bounds KL. divergence and offer parametric estimators suitable for nigh-dimensional and complex data. Despite their scalability, they suffer from well-known theoretical limitations, such as high variance in MINE and large batch size requirements in InfoNCE (Song & Ermon 2020). Recent high-MI and complex distributions based benchmarks further indicate that discriminative methods may underperform compared to generative approaches (Franzese et al. 2024 Butakov et al. 2022a)).
Neural Diffusion Estimator for MI (MINDE). One of the most recent generative methods for MI Estimation diffusion-basee based (Song et al.. (022)) MINDE (Franzese al.) 2024). To estimate KL (m [I+ () the authors learn two standard backward diffusion models to generate data from distributions -d and π. c.g., for A:
QA dz = I- t) + g(t)² (2,, t)]dt + g(t) ~ (xT),
(8)
backward diffusion
where f and g are the drift and volatility coefficients, respectively, of the forward diffusion (Song et al., 2021). dW, is the Wiener process when time flows backwards, and q is the distribution of the noised data at time & (Franzese et al. 2024 52, 3). The similar expressions hold for zB and
4

---

Published as a conference paper at ICLR 2026
Qa. Then, the authors formulate KL divergence estimator through the difference of diffusion score functions:
KL (T (1======= KL LLLLLLIAA (Q^110B)= [ ("IE (()))
g()) 2 ||s (Ec, s t)12 dt KL (of
47)
(9)
Here, KL (9 of) is the bias term, which vanishes only when diffusion has infinitely many steps, i... T' - 00. When the diffusion score functions s and s (8) are properly learned, one can draw samples from the forward diffusion (xx) = q (xz [o)q( and compute the estimate of KL divergence (9). In this way, the authors transform the problem of training the KL divergence estimator into the problem of learning the backward diffusions (8) that generate data from noise.
To estimate mutual information, the authors propose a total of four equivalent methods, all based on the estimation of up to three KI. divergences (9) or their expectations.
InFoBRIDGE MUTUAL INFORMATION ESTIMATOR
In (44.). we propose our novel MI estimator which is based on difference of diffusion drifts of conditional reciprocal processes. We explain the practical learning procedure in ($4.2) and suggest some straightforward generalizations of method in ($4.3).
4.1 COMPUTING MI THROUGH RECIPROCAL PROCESSES
Consider the problem of MI estimation for random variables Xo and X1 with joint distribution π (Fo, 21). To tackle this problem, we employ reciprocal processes:
Q+ def
W /WEOO. ( 21), Qind daf /
W W da (no))d
(10)
We show that the KI. between the distributions л (2o, 21) and (zz)) ( is equal to the KL. between the reciprocal processes 2 and Qind and decompose the latter into the difference of drifts.
First, let us introduce a classical stochastic calculus result that allows us to compute the KL between diffusion processes, e.g., Qx and Qiri SDE representations.
KL divergence between diffusion processes. Consider two diffusion processes with the same volatility coefficient /e that start at the same distribution To:
dx: (x2, dt VedW,, 3o (o
E (A,
(11)
By the application of the disintegration theorem Léonard 2014b $1) and the Girsanov theorem (Léonard 2012) one can derive the KL divergence between these diffusions:
2c / (a)())) (at, f (z,, 1)] dt,
KL (Q^1l
(12)
where (xx) is the marginal distribution of QAA time
This allows one to estimate the KL divergence between two diffusions with the same volatility coefficient and the same initial distributions, knowing only their drifts and marginal samples x;
(2x). This fact is widely used in Bridge Matching (Shi et al., 2023 | Peluchetti, (002)), Diffusion (Franzese et al. 2024) and Schrödinger Bridge Models (Vargas al. 2021 Gushchin al. 2023a).
Theorem 4.1 (Mutual Information decomposition). Consider random variables Xo. X, and their joint distribution π (20,, 21), such that I(Xo; X1) < co and J&p ||>ı|| di ) co. Consider reciprocal processes Qs. Qind induced by distributions (1)) and (21))(Z1)) respectively, as in (1O.. Then the MI between the random variables Xo and X, can be expressed as:
1 (')
/
I(Xo; = 2e
Egg (xL..0)) [Ujoine (Iet, L xi) - W1n 1 20) 20)
(13)
where
Ø F2
Vjonear (2 t, 2o) = Eg
(14)
1 -
Ø1 Lt.
Vind (2n, (e, 1, (.0)) = (eseee)))))
(15)
1
Ujoiin and Unnd the drifts of the SDE representations (4 of the reciprocal processes Qn and ind
5

---

Published as a conference paper at ICLR 2026
Algorithm 1: InfoBridge. Training the model.
Input DDistribution [ 21) accessible by samples, initial neural network parametrization Up of drift functions
Output:learnen neural network uo approximating drifts Ujoin and Uind
repeat
Sample batch of pairs {=0 x1 NN
21 I ~ (I0,
Sample random permutation { 11m=0 = Permute ((T))
Sample batch {t" In=1
(t);
Sample batch {xz Jn=1 W 12
(.1(Em) Jn= ))
I/ Sample trajectory point from QT.
Sample batch {2] h=
W
120.2 (. (:[{t Jn=1l
// Sample trajectory point from Q
C
I[ve(=7 t",
2f =""
11:
C
Σ n= |1 t", 2b 0)
27 -"-""
1-p"
Update using until converged:
ae + 00
Algorithm 2: InfoBridge. MI estimator.
Input :Distribution π (200 v1) accessible by samples, neural network parametrization of drift functions approximating optimal drifts Vjoint and find- number of samples
Output: Potput:Mutiee Mutual information estimation MI
Sample batch of pairs {xā #f )= ~ (10g 21);
Sample batch {t"),N_ In=1 (t);
Sample batch {x/ Jami
W 10a (-I(!") I)=
MI
2cN Ex (12 1", x0, 1) us (at x0 0)
Proof sketch:
Starting from the quantity KL (Qx Qird). we first apply the disintegration the- @01 Q ) and apply it more time to express it as KL (T an (2o)<<<<)))) Finally, we decompose the KI. between the conditional diffusion using Girsanov's theorem. Full proof is provided in Appendix A
orem to express it as Ex(xa) [KL
processes KL T Qimd
Once the drifts Vjoint and Vind are known, our Theorem provides a straightforward way to estimate the mutual information between the random variables Xo and X, by evaluating the difference between the drifts 'juin (Z,, f, 2o) (14) and Uin (I,, (o)) (15) at points æp, 3y sampled from the distribution of the reciprocal process Q. at times 0, f. Regularity assumptions (Shi et al.) 2023 Appendix C) are relatively mild and common for diffusion bridges generative modeling, i.e., they include restrictions
such finite first moment.
Remark. Our approach is different from MINDE, because we frame MI estimation as a domain transfer task, while MINDE frames it as generative modeling task. Our formulation allows unbiased
M estimation (13) while MINDE has non zero bias term (9). Furthermore, the domain transfer perspective yields easier to learn trajectories and reduced variance in MI estimation Tables and More details on the comparison with MINDE are presented in the Appendix C
4.2 InFOBRIDGE. PRACTICAL OPTIMIZATION PROCEDURE
The drifts Ujoint and Uind of reciprocal processes Qn and Qind be recovered by the conditional Bridge Matching procedure, see 7 We have to solve optimization problem 7 by parametrizing Ujoint and Vind with neural networks Ujoint, & and Uind, respectively, and applying Stochastic Gradient Descent on Monte Carlo approximation of (7). The sampling from the distribution qu (Iz, Øo) of reciprocal process Q, at times 0, easy because:
On (zt, mn) (x (xx) Ign (Ie, 20 21) (x ( ( 200)
In our case, by the word unbiased we mean that under full access to distributions and under the assumption that learn drift functions ideally, the estimation of the objective is unhiased.

---

Published as a conference paper at ICLR 2026
Table 1: Mean MI estimates over 10 seeds using 10k test samples against ground truth (GT), adopted from Franzese al. 2024). The closer the estimate is to the ground truth, the better. Color indicates relative negative (red) and positive bias (blue).
Gr
Methed Type 020403044
1004 9906
0.4 040334 170304
070307
NVF
IVF
Flow
02040330 100
01000ER2 014 104400040444 101110 1.00
10 1.044
1004
1.066 1604 L. 101
100 00 10090 88 0.001
0.5 0.50
ID
0.4022
0.016 044
1.5 1,5020
MINDE MINDE
Diflision
020403304
020400304
1.004
1000
100.10000000
1.0.0001000 0.3 0.302 040304 1,7034
0.2
1.7034
MINE InfoNCE
020202244
0.4 1,0
1,003
1.0A
1,006
0.4 OUS 0 IDS www.G0BB........... C. 020204170306
0.40.4
L/
1,003 1.0
1,00
1,006
1.0 110. 088 1,0 1.00203 0.200. 040304170000
Dall (Cousstan)
0.205 0:388
LU10
1. 08 CO6.BHBTT
08787777 1875 4.7
DoE (Logstic)
Clasical
011 0.4 0.4 Du4
0.4
8903
1,003,0.222
1.0669
0A6S
LI 1.0.5 0.6 0.6 0.5 0.5.50004 G.R L505 16011
KSG
0.2 0.4 B22
Du4 0.4 0.40..
1,00300
1.0--07
0.0
0.17700 0.7 RLGER 0.P 02022 OLI 0.1 0. 0.204 1.7304
(1-21r) EX is O uaa aoo 1855 di upse
as 0 get Puiuit
EX LPPO penooee upp cny 25 250
Mr a tine as cayyere whe i A and cereeinn
TH
2 2 (ipaii
Mr
Mr 25 25 C
25 2225x22
No. X 22 pain)
Mt Sic
5 5 Copair
Mr. S
Ms 20x 30 (less)
Ne CD) & Uimatt Me x
Aix
No CDF ar Mr xx (2 part
Sy & Aft 25 25 (2gain
Sy e Mil xx5 part
by i CDF Mn 25 ss 25 papp
Sy 8 NoCDDE CDE o Mr.5 5 Niv CDF Mn (2qai
55 1.x. 22 23
(0mma)
"o 1e-2i (e cosee 1
(lmo))
Mr (2- pain
3. pai))
your
Therefore, to sample from qr (2t,, vn) it suffices to sample T0, ~ ((00, and sample from In (2///11 (11, which is again just Brownian Bridge.
Vector field parametrization. In practice, we replace two separate neural networks that approxi- mate the drifts Ujoine (,,, 25) and (,,, (o) with single neural network that incorporates an additional binary input. Specifically, we introduce a binary input E {0, 1} to unify the drift approxi- mations in the following way: v=(:;1) 1) (-)int(:) and u- 0) s3 Uind (-). Such binary conditioning is widely used for the conditioning of diffusion (Ho & Salimans [021 and bridge matching (Bortoli
al. 2024) models. In Appendix C.3 we show that it provides a more accurate estimation.
We call our practical MI estimation algorithm InfoBridge. The drifts vo training procedure is and and the MI estimation procedure is described in Algorithm One can see
described in [
that both model training and MI estimation procedures are simulation-
Several strategies exist for sampling data used in computing the independent drift loss 23 in Algo- rithm 1 While regular permutation works well in practice, it may introduce correlations (Letizia al. 2024). Alternatives include "derangement" permutations (Letizia et al., 2024) or resampling from the marginal. See Appendix C.4 for further discussion.
The computational complexity our algorithm as Diffusion Bridge model is greater than that of discriminative methods and is similar to that of MINDE.
4.3 GENERALIZATIONS AND IMPLICATIONS
Generalizations. Our method admits several straightforward extensions. For completeness, we present a method for the unbiased estimation of the general KL divergence in Appendix [.I Accord- ing to Theorem B.1 the KL divergence between any two distributions can be decomposed into the difference of diffusion drifts in a similar way to [3
Furthermore, our method can be extended to estimation of MI between random variables of different dimensionality (Appendix B.5) and naturally generalizes to the estimation of mutual infor- mation involving more than two random variables, known interaction information (Appendix B.4.
Practical procedures for these generalizations can also be derived in a similar way to (54.2).
Generative byproduct. Note that the learned drifts ca 1) and (a( 0) define the distributions To.j [æ1 (20) x [21 (2)) and and (x (mn) x (xx1) as solutions to the corresponding SDEs (4). One can simulate these SDEs by utilizing diffusion based generative modeling methods. Samples Figure This is unnecessary for the MI estimation, but can be considered as an additional feature.
from To Joij (z) (an) and no ( (2) (2o) for the image based benchmark (Section 5.2) can be seen at

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
We test our method on a diverse set of benchmarks with known ground truth values of mutual information. To cover low-dimensional cases and some basic cases of data lying on a manifold, we employ the tests by (Czy2 et al., 2023). Benchmarks from (Butakov et al., 2024b a) are used to assess the method on manifolds represented as images. To evaluate our method on real-workd data, we use protein language model embeddings and adopt the benchmark construction procedure proposed in (T.ee & Rhee, 2024). Finally, we evaluate our method on high mutual information tasks from (Czy2 et al. 2023). In addition, we present an ablation study in Appendix C.2 analyzing the impact of key iyperparaaeters, including the volatility coefficient e, neural network architectures.
We compare our method with a diverse collection of other MI estimation methods: diffusion based
MINDE (Franzese et al. 2024). flow-based NVF, JVF (Dahlke & Pacheco 2025), MIENF (Butakov
et al. 2024a), classical MINE (Belghazi et al. 2).8). InfoNCE (van den Oord et al. 2019). fDIME
(Letizia al. 2024 NWJ (Nguyen et al. 2010 KSG (Kraskov et al.] (0004).
Across all four experimental setups, our method demonstrates consistently strong perfor- mance. Its main advantage emerges in challenging settings involving high-dimensional data and large mutual information values. In these regimes, our approach consistently outperforms both the diffusion-basse based MINDE estimator and classical MI estimation baselines.
5.1
LOW-DIMENSIONAL BENCHMARK
The tests from (Czy2 et al. 2023) focus on low-dimensional distributions with tractable mutual information. Various mappings are also applied to make the distributions light- or heavy-tailed, or to non-linearly embed the data into an ambient space of higher dimensionality.
InfoBridge is tested with e = 1 and a multi-layer dense neural network is used to approximate the drifts. Our computational complexity is comparable to MINDE (Franzese et al.) (024). For more details, please. refer to Appendix DD. In each test, we perform 10 independent runs with 100k train set samples and 10k test set samples, where we train InfoBridge neural networks for 100k grad steps.
We report the mean MI estimation in Tables 1 and 21
Overall, the performance of our estimator is similar to that of MINDE (3 and superior to the classical and flow based, methods, with the Cauchy distribution being the only notable exception. Unfortunately, the Cauchy distribution lacks the first moment, which poses theoretical limitations for Bridge Matching (Shi et al.] 2203] Appendix C). However, overcome this limitation using the tul-shortening asinh transform, which enables our method to achieve ncar-accurate MI estimates, see first two Asinh columns in Tables [and21 21
5.2
IMAGE DATA BENCHMARK
In (Butakov et al. 2024b), it was proposed to map low-dimensional distributions with tractable MI into manifolds admitting image-like structure, thus producing synthetic images (in particular, images of 2D Gaussians and Rectangles, see Figures and [b). By using smooth injective mappings, one ensures that MI is not altered by the transform (Butakov et al. 2024a Theorem 2). In the original works, is argued that such benchmarks are closer to real data, and therefore closer realistic setups.
Each neural algorithm is trained with 100k train set samples and validated using 10k samples. In addition we explore experimental setups with less train set samples in Appendix C.3 InfoBridge is tested with = 1, we use a neural network with U-net architecture to approximate the drift and train the model for 100k grad steps with batch size of 64. Other experimental details are reported in Appendix D.3 and the ablation study neural network architecture and volatility coefficient is presented in Appendix
and
Table 2: Mean Absolute Error (MAE) averaged for all setups and random seeds, and standard deviations averaged over setups for Image based benchmark experiment. The best result is bolded.
Mothod
InfoBridge
InfoNCE
KSG
MINE
MIENF
NWJ
MINDE-C
MINDE-J
MAF 4
0.38
1.44
1.15
0.92
0.45
1.24
0.56
1.66
Average std. L
0.04
0.13
0.08
0.43
We present our results for 16x 16 and 32x32 resolution images with both Gaussian and Rectangle structure in Figure [ and report aggregated MAE and std in Table [ while the samples from the
learned conditional bridge matching models can be viewed in Figures 5c 5f
8

---

Published as a conference paper at ICLR 2026
MINIECC
VIRDEC
MINDE-
NWO MINE
SINL SINTE
BENCE
LEESCE
(a) 16 16 images (Gaussians)
(b) 32 x 32 images (Gaussians)
10
NINDB-C
MIXDEU
SINE
KSG
MINNT
(c) 16 images (Rectangles)
(d) 32 images (Rectangles)
Figure 1: Comparison of the MI estimators. Along x axes is (Xoo: X1), along y axes is MI estimate [(oo X1). We plot 99% confidence intervals acquired from different seed runs.
Our estimator is competitive, being consistently more precise and more stable than both MINDE-C and MINDE-J, and overall slightly better than the previous best-performing method: MIENF Butakov ct al. 2024a). In addition, method delivers less variance in MI estimation Table 2
5.3
PROTEIN EMBEDDINGS DATA
We evaluate InfoBridge on real-world data, i.e., conduct experiments protein lan- guage model embeddings. Following the benchmark construction method from (Lee)
0.5
InfoNCE
MI
Estimated MI
KSG
InfoBridge
& Rhee 2024), we generate paired datasets with known MI by ensuring that only class labels are shared between variables X and Y. making the MI analytically tractable (Lee & Rhee 2024 84.5). We use se- quence embeddings from the ProtTrans5 model (Elnaggar al. 202T), based on proteins from A. thaliana and H. sapiens, create datasets with varying ground truth MI. The final number of training pairs is 20641 and the dimensionality of embed- dings is 1024. In addition, the ablation study on the volatility coefficient is presented in
0.1
0.3
Ground Truth
Figure 2: Comparison of the selected estimators on the ProtTrans5 data. Along axes is ground truth T T(Xu, X1), along y axes MI estimate T(Xo, X1).
InfoBridge with e = 1 is trained for 100 epochs with batch size of 128 and neural network along the other hyperparameters We compare against MINE, InfoNCE, KSG, and MINDE. All the technical details can be found in Appendix D.4 We present the results plot in Figure 2 and MAE in Table One can that InfoBridge is the only method to consistently estimate MI accurately; MINDE-C and MINDE-I are omitted they drastically overestimate MI (e.g.. MAE: MINDE- C 9.29 vs. InfoBridge 0.04).
similar to the low-dimensional benchmark ($5.1) hyperparameters.
5.4 HIGH MUTUAL INFORMATION
"To evaluate the performance of our method in high mutual information regimes, conduct exper- iments on high-dimensional tasks introduced in (Czyž et al., 2023), including correlated uniform, smoothed uniform, correlated gaussian its half eube transformed variant. We systematically vary

---

Published as a conference paper at ICLR 2026
Intdridge (ours) IYbACCE IDME
eébirege COUIC PRSACE ICNC
Praloge ICAIN
inkdrage (cand) IMRRICE EIPE
MAOCC
Estinated HI
Extimateed Mi
Extimatedd MI
PÉRVCE EIPE
Estimated
Dim
(a) Gaussian
(b) Half Cube Gaussian
(c) Correlated Uniform
(d) Smoothed Uniform
Figure 3: Comparison of MI estimates across dimensions and MI for high mutual information.
the dimensionality and mutual information of the random variables, setting E {20, 40, 80, 160) and MI E {10, 20, 40, 80}. Each experiment uses training set of 100k samples and test set of 10k samples. We evaluate our method with = 0.01 and show the results in Figure B The InfoBridge is trained for 200 epochs with batch size of 128, and the neural network is the for low-dimensional benchmark & 1 To keep the narrative concise, we provide additional baselines and experimental details, refer Appendix D.5
As shown, our method yields the most accurate estimates compared to the ground truth, whereas MINE, InfoNCE, and fDIME (GAN, fail capture high mutual information values, and MINDE-C consistently underperforms relative to our approach.
6 DISCUSSION
Potential Impact. Our contributions include the development of novel unbiased estimator for the MI grounded in diffusion bridge matching theory. The proposed algorithm, InfoBridge, demon- strates superior performance compared to commonly used MI estimators on both synthetic and real-world challenging nigh-dimensional benchmarks, Also, our approach can be used to estimate the KL divergence and differential entropy Appendices B.1 and B.2
We believe that work paves the way for new directions in the estimation of MI in high dimensions. This has potential real-world applications such as text-to-image alignment (Wang et al. 2024), self- supervised learning (Bachman al. 2019), deep neural network analysis (Butakov al. 2024b). and other use cases in high-dimension settings.
Moreover, our approach offers opportunities for extension by exploring alternative types of bridges within reciprocal processes, for instance, variance-presery SDE bridges (Zhou et al. (202)). or by incorporating advanced diffusion bridges techniques like time reweighting (Kim al. 2025).
LIMITATIONS
Bridge Matching as a general generative modeling paradigm is limited w.r.t. distributions without the first moment. To well defined Bridge Matching procedure requires some assumptions be satisfied, which are described in Appendix A.I If one of the I (20) or π (Ix;) distributions doesn't t have a first moment, then these assumptions are not satisfied, and our method not guaranteed to work. One such case can be seen in Table one-degree-on (dof=1), also known as the Cauchy distribution, that has no first moment.
freefredomm offeree----freedo Student-t distribution, i.e., Sr
Acknowledgements. We thank Kirill Sokolov for the careful feedback and support in refining the proofs. The work was supported by the grant for research centers in the field of Al provided by the Ministry of Economic Development of the Russian Federation in accordance with the agreement 100000C313225P4F0000 and the agreement No 139-10-2025-0033
LLM Usage. Large Language Models (LLMs) were used only assist with rephrasing sentences and improving the clarity of the text.
Reproducibility statement. All the technical details that are required to reproduce the work are stated either in main experimental section ($5) Appendix Additionally, provide code for the reproduction of experiments in supplementary. All the proofs for provided theoretical results are provided either in Appendix or Appendix
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Rana Ali Amjad, Kairen Liu, and Bernhard C. Geiger. Understanding neural networks and individual neuron importance via information-ordereed cumulative ablation. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7842-7855, 2022. doi: N0.1109/TNNLS32221308888
Ziqiao Ao and Jinglai Li. Entropy estimation via normalizing flow. Proceedings of the AAAI
Conference on Artificial Intelligence, 3691::)),,333 Jun. 2022. doi: 10.1609/aaai URL htwphttts/ssooo aaal www.oog/ngcooo wwt.:aaw..........e...pp.../e//e http:///aaa..pp//AAA/////aaaaaa1
Lynton Ardizzone, Radek Mackowiak, Carsten Rother, and Ullrich Köthe. Training normal- izing flows with the information bottleneck for competitive generative classification. In Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 7828-7840 Curran Associates, Inc.. neurips.....e cc/paperrppppee files/ /aLLe//pppee2220/ 38e6 49d22 d3e (0ee777cd0-ppppr.e
H. Larochelle,
2020. URL https: btttps://proceediine 7
Philip Bachman, R Devon Hjelm, and William Buchwalter. Learning representations by maximizing mutual information across views. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, Cur- ran Associates, Inc., 2019. URL
neuripgs....uu.pppppe... wwtpurrips.ccppapppppa ccppaper files/
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Rexearch, pp. 531-540. PMLR, 07 2018. URL https v/proceedings.uo mml. wrw.speseppp....
html
Bell and T Sejnowski. An information-mmxximzzzioon approach to blind separation and blind deconvolution. Neural Comput., 7(6): (29::11291119,, November 1995.
Thomas Berrett and Richard Samworth. Nonparametric independence testing via mutual information. Biometrika, 106, 11 2017. doi: 10. (0.1093//iommetass022.
Thomas B. Berrett, Richard J. Samworth, and Ming Yuan. Efficient multivariate entropy estimation via k-nearest neighbour distances. Ann. Statist, 47(1):288 318, 02 2019. doi: 10. 1881688 URL
1214/
688
Valentin De Bortoli, Iryna Korshunova, Andriy Mnih, and Arnaud Doucet. Schrodinger bridge flow for unpaired data translation. In The eighth Annual Conference on Neural Information Processing Systems, 2024. URL htllitttps///pperreeee httppppp///ppoeerreeeeeeeeeoaoaa oot.:/eeeeettoooooooooo--CCECCC
Mustapha Bounoua, Giulio Franzese, and Pietro Michiardi. Swi: Score-based O-INFORMATION estimation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Confer- ence Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 4444 4471, PMLR, 21-27 Jul 2024a. URL httpttttt////oceeeeete www.ingg.......s///222... bounoua24a. h
Mustapha Bounoua, Giulio Franzese, and Pietro Michiardi. Swi: Score-based o-information estima- tion. In ICML 2024, 41 International Conference Machine Learning, 2024b.
Charlotte Bunne, Ya-Ping Hsich, Marco Cuturi, and Andreas Krause. The schrödinger bridge between gaussian measures has closed form. In International Conference on Artificial Intelligence and Statistics, pp. 5802-5833. PMLR, 2023.
Ivan Butakov, Alexander Tolmachev, Sofia Malanchuk, Anna Neopryatnaya, and Alexey Frolov. Mutual information estimation via normalizing flows. In The eiirty Annual Conference Neural Information Processing Systems, 2024a. URL https::/pt/ttt http::///ppezzevieee wwt/forumf
id=JiQXa
11

---

Published as a conference paper at ICLR 2026
Ivan Butakov, Alexander Tolmachev, Sofia Malanchuk, Anna Neopryatnaya, Alexey Frolov, and Kirill Andreev. Information bottleneck analysis of deep neural networks via lossy compression. In The Twelfth International Conference Learning Representations, 2024b. URL https: //openreviews wwwewviiwww.... net forum? dinnnidchuuGGZddPP
Xavier Aramayo Carrasco, Grigoriy Ksenofontov, Aleksei Leonov, Iaroslav Sergeevich Koshelev, and Alexander Korotin. Entering the era of discrete diffusion models: A benchmark for schrödinger bridges and entropic optimal transport. In The Fourteenth International Conference on Learning Representations, 2026. htttsss:::// http:////ppeeevveeeeeee / net. wtt.://orummmiio
wttp://wwwweeeeeeeorrriiiiiieie dxcpd 5Gd
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2172-2180 2016. URL BtL/sttpss/pproeeeddngg -eerriiiiinnnnnp.s.......... 7c9d0b1f9 hta.aa/eeeeaadtooaacaaaaaeeeo.a Sebeeedaaaaabbbbb Abbb- neurips httpaapp.ccppppppppeeem cctppppee/.1./////// 6/hashh// html
Zehua Chen, Guande He, Kaiwen Zheng, Xu Tan, and Jun Zhu. Schrodinger bridges beat diffusion models on text-to-speech synthesis. arXiv preprint arXiv:23/2.03491, 2023.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommu- nications and Signal Processing). Viley-Intersccence, USA, 2006.
Pawel Czyž, Frederic Grabowski, Julia E Vogt, Niko Beerenwinkel, and Alexander Marx. Beyond normal: On the evaluation of mutual information estimators. In Thirty-seventh Conference Neural Information Processing Systems, 2023. URL httpss://ppeereeeeeeee wwt.:.eee/forrro
1d==25vRtG56YH
Caleb Dahlke and Jason Pacheco. Flow-based variational mutual information: approximations. In The Thirteenth International Conference on Learning Representations, 2025.
Fast and flexible
Valentin De Bortoli, Guan- Horng Liu, Tianrong Chen, Evangelos A Theodorou, and Weilie Nie.
Augmented bridge matching. arXiv preprint arXiv:2311.06978, 2023.
J. L. Doob. Conditional brownian motion and the boundary limits of harmonic functions. Bulletin de la Société Mathématique de France, 85:431 1588,14581111 1957.
Giovanni Dosi and Andrea Roventini. More is different... and complex! the case for agent- macroeconomics. Journal of Evolutionary Economics, 19.1-37, 2019.
Bao Duong and Thin Nguyen. Diffccomorphic information neural estimation. Proceedings of the AAAI Conference on Artificial Intelligence, 376):7468-77755 Jun. 2023a. doi: 10. http://aaai.i....25553 URL httts://s:///
Bao Duong and Thin Nguyen. Normalizing flows for conditional independence testing. Knowledge and Information Systems, 66, 08 2023b. doi:
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. Prottrans: towards cracking the language of life's code through superieeee learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11-7112-7127. 2021.
Ammar Fayad and Majd Ibrahim. On slicing optimality for mutual information. In Thinth-seentth Conference on Neural Information Processing Systems, 2023. URL htttps://openreet
nett/forun d== 2x2
Marco Federici, David Ruhe, and Patrick Forré. On the effeettiveeeesss of hybrid mutual information estimation, 2023. URL htt.tpss//arrii..rr//aa/////////
60608
Giulio Franzese, Mustapha BOUNOUA, and Pietro Michiardi. MINDE: Mutual information neural diffusion estimation. In The Tweifth International Conference on Learning Representations, 2024.
UtR.hhttss:///penee http://rreeeeeeeeoooooooooooooos
12

---

Published as a conference paper at ICLR 2026
Z. Goldfeld, K. Greenewald, J. Niles- Weed, and Y. Polyanskiy. Convergence of smoothed empirical measures with applications to entropy estimation. IEEE Transactions on Information Theory, 66 (7):4368-433)) 2020. doi: 10.1 1t..109//TT.2222299554444
Ziv Goldfeld and Kristjan Greenewald. Sliced mutual information: A scalable measure of statistical dependence. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL https: http::://openrevieeeee / hwwp:ewwwweee.fooeoeeeeeeee
id-Svryl-FDep
Ziv Goldfeld, Ewout van den Berg, Kristjan H. Greenewald, Igor V. Melnyk, Nam H. Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In ICML, 2019.
Ziv Goldfeld, Kristjan Greenewald, Theshani Nuradha, and Galen Reeves. $kS-sliced mutual information: A quantitative study of scalability with dimension. Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022.
Kristjan H. Greenewald, Brian Kingsbury, and Yuancheng Yu. High- fiihh-dimensional smoothed entropy estimation via dimensionality reduction. In IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, June 25-30, 2023, pp. 2613 2618. IEEE, 2023. doi: 10. 109/SSI1777.... 2023 10206641
4023.1020660 URL /
Nikita Gushchin, Alexander Kolesov, Alexander Korotin, Dmitry Vetrov, and Evgeny Burnaev. Entropic neural optimal transport via diffusion processes. In Advances in Neural Information Processing Systems, 2023a.
Nikita Gushchin, Alexander Kolesov, Petr Mokrov, Polina Karpikova, Andrey Spiridonov, Evgeny Burnaev, and Alexander Korotin. Building the bridge of schr odinger: A continuous entropic optimal transport benchmark. In Thirty-sevnnth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023b.
Nikita Gushchin, Sergei Kholkin, Evgeny Burnaev, and Alexander Korotin. Light and optimal schrödinger bridge matching. In Forty-first International Conference on Machine Learning. 2024a.
Nikita Gushchin, Daniil Selikhanovych, Sergei Kholkin, Evgeny Burnaev, and Alexander Korotin. Adversarial schrödinger bridge matching. Advances in Neural Information Processing Systems, 37: 89612-89651, 2024b.
Nikita Gushchin, David Li, Daniil Selikhanovych, Evgeny Burnaev, Dmitry Baranchuk, and Alexan- der Korotin. Inverse bridge matching distillation. In International Conference on Machine Learning, pp. 21471-21496. PMLR, 2025.
Guande He, Kaiwen Zheng, Jianfei Chen, Fan Bao, and Jun Zhu. Consistency diffusion bridge models. Advances in Neural Information Processing Systems, 37:23516-23544 2024.
Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildonn Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. International Conference on Learning Representations, 2019. URL https //openreview oot.://orrommmddd11111111111
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-665 6851, 2020.
Oliver lbe. Markov processes for stochastic modeling. Newnes, 2013.
Ilia Igashov, Ame Schneuing, Marwin Segler, Michael M Bronstein, and Bruno Correia. Retrobridge: Modeling retrosynthesis with markov bridges. In The Twelfth International Conference on Learning Representations, 2024.
Bert Kappen. Lecture notes in eds machine learning, September 2024.
13

---

Published as a conference paper at ICLR 2026
Sergei Kholkin, Grigoriy Ksenofontov, David Li, Nikita Kornilov, Nikita Gushchin, Alexandra Su- vorikova, Alexey Kroshnin, Evgeny Burnaev, and Alexander Korotin. Diffusion &amp; adversarial schrödinger bridges via iterative proportional markovian fitting. In The Fourteenth International Conference on Learning Representations, 2026. URL htl.https:/ htttps://openrvileet net./ for rum?
(d=38FGGChhFF5
Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Marco Cuturi, Jong Chul Ye, Bahjat Kawar, and James Thortoo.. Simple reflow: Improved techniques for fast flow models. In The Thirteenth International Conference on Learning Representations, 2025. URL
net/ htte//orrmmmieeee fPVgSSSXXYY
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
Alexander Korotin, Nikita Gushchin, and Evgeny Burnaev. Light schrödinger bridge. In The Twelfth International Conference on Learning Representations, 2024.
L. F. Kozachenko and N. N. Leonenko. Sample estimate of the entropy of a random vector. Problems Inform. Transmission, 23:95-100 1015, 1987.
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Phys. Rev. E, 69:0669:366 138, Jun 2004. doi: 10. 10.11033PhsRReeE.6..66611 URL httts:://iinkkk aps org/doi/100 100001111.. 1103/ http://///...11//////.eereeee/
69. 066138
Grigoriy Ksenofontov and Alexander Korotin. Categorical schrödinger bridge matching. In Interna- tional Conference on Machine Learning, pp. 31727-31751. PMLR, 2025.
Kyungeun Lee and Wonjong Rhee. A benchmark suite for evaluating neural mutual information estimators on unstructured datasets. Advances in Neural Information Processing Systems, 37: 06319-463388 2024.
Christian Léonard. Girsanov theory under finite entropy condition. In Catherine Donati-Marte Antoine Lejay, and Alain Rouault (eds.), Séminaire de Probabilités XLIV, volume 2046 of Lecture Notes in Mathematics, pp. 429-465. Springer, Berlin, Heidelberg, 2012.
Christian Léonard. Stochastic derivatives and generalized h-transforms of markov processes, 2013.
Christian Léonard. A survey of the Schrödinger problem and some of its connections with optimal transport. Discrete and Continuous Dynamicul Systems - Seriie A, 34(4): 2014a.
Christian Léonard. Some properties of path measures. Séminaire de Probabilités XLVI, pp. 207-230, 2014b.
Christian Léonard, Sylvie Rælly, and Jean-Claude Zambrini. Reciprocal processes. a measure- theoretical point of view. Probability Surveys, 1 11:237-269, 2014.
Nunzio Alexandro Letizia, Nicola Novello, and Andrea M Tonello. Mutual information estimation via --divergencce and data derangements. Advances in Neural Information Processing Systems, 37: 105114-1051500 2024.
R. Linsker. Self-organization in perceptual network. Computer, 21(3):105-117, 1988. doi: 10.1109/2.36.
Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A Theodorou, Weili Nie, and Anima Anandkumar. sb Image-to-image schr\." odinger bridge. arXiv preprint arXiv.2302.05872. 2023.
David McAllester and Karl Stratos. Formal limitations on the measurement of mutual information. In Silvia Chiappa and Roberto Calandra (eds.). Proceedings of the Twenty Third International Conference Artificial Intelligence and Staristics, volume 108 of Proceedings of Machine Learning Research, pp. 875-884 884. PMLR, 08 2020. URL ht https: https://prroeeeiings mlr Treas/v108/ne 08/ lestcerllsseeee html
14

---

Published as a conference paper at ICLR 2026
XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11))5847-5861, 2010.
Bernt Øksendal. Stochastic differential equations. Springer, 2003.
Zbigniew Palmowski and Tomasz Rolski. A technique for exponential change of measure for markov processes. Bernoulli, pp. 767-785, 2002.
Stefano Peluchetti. Diffusion bridge mixture transports, schrödinger bridge problems and generative modeling. Joumal of Machine Learning Research, 24(374):1-51, 2023.
Y. Polyanskiy and Y. Wu. Information Theory: From Coding to Learning. Cambridge Univer- sity Press, 2024. ISBN 9781 108832908. URL https::/ wtt.sooo.l.oeeeeeeooeeeeeeeeeee.
CYSоOAEACAAJ
Benjamin Rhodes, Kai Xu, and Michael U. Gutmann. Telescoping density-ratio estima- tion. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Ad- vances in Neural Information Processing Systems, volume 33, pp. 4905 4916. Curran Asso- ciates, Inc., 2020. URL httpp:: http::///ppo.e.......... neuriss caapprr files/paper/ 2020/File/ / 57ddc08 0ade00011/odd0000aa12aa2aaaaaa 909 Paperr
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted Intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part IIl 18, pp. 234-241. Springer, 2015.
Fernando E Rosas, Pedro AM Mediano, Michael Gastpar, and Henrik J Jensen. Quantifying high- order interdependencies via multivariate extensions of the mutual information. Physical Review E. 100(3):032305, 2019.
Jakob Runge, Sebastian Bathiany, Erik Bollt, Gustau Camps- Valls, Dim Coumou, Ethan Deyle, Clark Glymour, Marlene Kretschmer, Miguel D Mahecha, Jordi Muñoz-Mari, et al. Inferring causation from time series in earth system sciences. Nature communications, 10(1): (0:15,553,, 2019.
Erwin Schrödinger. Sur la théorie relativiste de l'électron et l'interprétation de la mécanique quantique. In Annales de I' l'institut Henri Poincaré, volume 2, pp. 269-310 1932.
Rajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros G Dimakis, and Sanjay Shakkottai. Model-powered conditional independence test. In I. Guyon, U. Von Luxburg, Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL httpt:ttt... http:p/pp///poooecngg.....ss/sss File/ /ttp:///SSSC11000000060530002220
Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. Diffusson schrödinger bridge matching. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Tarree///orrriii forum? wwwp:ammddddo0o0o000010
Vignesh Ram Somnath, Matteo Pariset, Ya-Ping Hsich, Maria Rodriguez Martinez, Andreas Krause, and Charlotte Bunne, Aligned diffusion schrödinger bridges. In Uncertainty Artificial Intelli- gence, pp. 1985-1995. PMLR, 2023.
Jiaming Song and Stefano Ermon. Understanding the limitations of variational mutual information estimators. In International Conference on Learning Representations, 2020. URL https //openreview n www.:netet..ntmetoee
Yang Song, Jascha Sohl- Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2021.
15

---

Published as a conference paper at ICLR 2026
Thomas Steinke and Lydia Zakynthinou. Reasoning About Generalization via Conditional Mu- tual Information. In Jacob Abernethy and Shivani Agarwal (eds.), Proceedings of Thirty Third Conference Learning Theory, volume 125 of Proceedings of Machine Learning Research, pp.
3437-3452. PMIR, 09-12 Jul 2020. URL htt.:httt:://pooo.etc steinke20a ht.ml
ngs. www.gmm.............mmm....... www..mmr..... www.ppresssvvv
Vincent Stimper, David Liu, Andrew Campbell, Vincent Berenz, Lukas Ryll, Bernhard Schölkopf. and José Miguel
Lordindez--obbat normflows: A pytorch package for normalizing flows.
Journal of Open Source Software, 8(86);3361, 2023. doi: 10.21 10211005/oss.00366 URI. https
V/doi. org/ 0. 21105/ josoos///..//// joss 05361
Karl Stratos. Mutual information maximization for simple and accurate part-of-speech induction. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers); pp. 1095-1 1104, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10. 10.18653/11/119-1113 URL https
V/aclanthology org/N19 1113/
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. 2015 IEEE Information Theory Workshop (ITW), pp. 1-5, 2015.
Alexander Y Tong, Nikolay Malkin, Kilian Fatras, Lazar Atanackovic, Yanlei Zhang, Guillaume Huguet, Guy Wolf, and Yoshua Bengio. Simulation-fris schrödinger bridges via score and flow matching. In International Conference on Artificial Intelligence and Statistics, pp. 1279-1287. PMLR, 2024.
Michael Tschannen, Josip Djolonga, Paul K. Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. In International Conference on Learning Representations, 2020. URL [RLLppps://ppeerreeii
net .et.:e/orrummdddxxxo20222222222.
Dor Tsur, Ziv Goldfeld, and Kristjan Greenewald. Max-sliced mutual information. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://ppenrreiier feet/foruumiddceeeBBee
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, 2019. URL httphhttt///aattee wrw/abs./////.ooooooo. htrg/abb/1880. 03748
Francisco Vargas, Pierre Thodoroff, Austen Lamacraft, and Neil Lawrence. Solving schrödinger bridges via maximum likelihood. Entropy, 23(9):1 134, 2021.
Francisco Vargas, Andrius Ovsianas, David Fernandes, Mark Girolami. Neil D Lawrence, and Nikolas Nüsken. Bayesian learning via neural schrödinger-föllmee flows. Statistics and Computing, 33(1): 3, 2023.
Petar Veličković, William Fedus, William I.. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.
URL htttt:::://ppeeeeeeeeee
net wtte:/ooommiidddrllll0000
Chao Wang, Giulio Franzese, Alessandro Finamore, Massimo Gallo, and Pietro Michiardi, Infor- mation theoretic text-to-image alignment, 2024. URL hwt.s:tppsp// http::ps///xxx...rrggoo/
20759
Stanislaw Weglarezyk. Kernel density estimation and its application. ITM Web of Conferences, 23: 00037. 01 2018. doi: 10. //tmccon/201182300007..
Aolin Xu and Maxim Raginsky. Information-theorede analysis of generalization capability of learning algorithms. Advances in neural information processing systems, 30, 2017.
Linqi Zhou, Aaron Lou, Samar Khanna, and Stefano Ermon. Denoising diffusion bridge models. In The Twelfth International Conference on Learning Representations, 2024.
16

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
Proofs
17
A.1 Auxiliary results
17
2 Proof of Theorem
20
Extensions of our methodology
B.1
KL divergence estimator
21
B.2
Differential entropy estimator
22
Entropy estimation experiments
22
B44
Interaction information
23
B,5
Mutual Information between random variables of different dimensionality
23
Additional discussion on method
24
Conceptual comparison with MINDE
24
Ablation study
24
Train sample size study
26
Strategies for drawing train samples
26
One neural network approach versus two neural networks approach
27
D Experimental supplementary
27
D.I Metries
27
D.2
Low-dimensional benchmark
28
D.3 Image data benchmark
28
D.4
Protein embeddings data
31
D.5
High mutual information experiments
33
A
PROOFS
A. AUXILIARY RESULTS
The following subsection mostly contains results very similar to the ones presented in (Doob 1957) and the comprehensive derivations presented for the reader convenience.
Setup Let Ω C(O0, IR), Xx(w) w(t) F, o(x, i s t). Let a E PRR x PD with marginals To, TT Let pzo be Brownian motion with volatility V started at zg. Its transition density
|/ - x0
(f y) = (2TEt) D/2 exp
2el
We fix regular version of the Brownian bridge kernel W
which is measurable.
Qx / W (I(,
17

---

Published as a conference paper at ICLR 2026
Let 1 := =T((/o)). For t (0, define the h--unctions:
16-1(1,8) (y).
h (t, = JRD pi(xo,1)
Intuitively, 1 x) is the likelihood factor that reweights the reference Brownian motion started at Zo so that, at final time 1, it has terminal law 100
Assumption A.1.
dzilldd1(a) (21) < co.
RDD
Lemma A.2. Fix 8 E (0,1/2) and zo E RD. Then:
1. hg (1, (0, o0) for all (1, E 10,1 8) RR:;
2. hg ((6, 6] P);;;
3. on each compact R.D, inl 161-6 1: >
and V, log 1: Lipschitz x on
[6, 8] K.
Proof. Set
(1-1(1.))
Ke (v)
pi pi
direct computation shows that, for each fixed (0, 1),
Kr Kr, (y) (C(t)exp) 2= - |ul blt, y 2(t,2,
where C(t) (1 - D/2 and b, are affine/quadratic expressions in 2; (their exact form is not important). Crucially, the coefficient in front of ||y||² is negative and bounded away from when t (ō, 1 - 8). Therefore, for such f the function y + Ki,,,ao (y) is strictly positive and achieves finite maximum, hence is bounded.
(1) Positivity and finiteness. Since Ke.z.zo (y) > 0 and bounded in for each fixed (t, with t 66, 1 6). integrating against the probability measure n gives heo (f,x) oc.
(2) C1.2 regularity. For |a| S 2, the derivatives 0% Kz (v) are polynomials y times the same exponential factor, hence they also bounded in for t 8, 1-5). Thus, any set [8, 1 5] {x} we can diffcrentiate under the integral sign by dominated convergence, obtaining that A is C in
(t)
(3) Positive lower bound and Lipschitzness of V log h compacts. Fix a compact K C RD. Continuity of he on the compact set [ō, 1 - 8] x K implies it attains its minimum there; by (1) this minimum is positive, so m inf 1 8]x h O.
Moreover, by (2) the derivatives Vh and 2 are continuous, hence bounded on [8, 6] x K. Finally, using V log h = (Vh)/h and the identity
v2 logh 4
Vh & Vh
h
that 2 log h is bounded (8, Lipschitz in x on this set.
8) K. A bounded Hessian implies that V log h is
Proposition A.3. For each vo and each 1, the conditional laws
Q!a0 (- xo),
satisfy, for any A E Fi.
Q 1o (A) pao [1 1: (t, X,)
Consequently, on every interval [5, 1 - 8) the process under Q dX, =2 X.) dt + VE dW, and u is locally Lipschitz in there.
1 solves the SDE (weakly)
1 (t, = V log h (t,
18

---

Published as a conference paper at ICLR 2026
Proof. Let A E Fe. By the Markov property of Brownian motion, Iat (A, X1 E dy) = Epro (X dy,
and also P (X1 E dy) pi (zo. y) dy. Therefore, by Bayes' rule (i.e. by the definition of regular conditional distribution),
Po.y (A) (A] yy) Epe [1 P
W
Pi (2 v)
Now integrate the previous identity in against 1 (dy). Since A E Fr. we can pull 1 outside the g-integral:
Q. (A) = (r (1 (A) (dy)
E 1 /
pf pi-e(Xe.V)
10 (dy) E [I ( (1, Xt)]
pf (x y)
In other words, on F, the measure Q 00 is absolutely continuous w.r.t. pro with Radon-Noi -Nikoyy derivative 1.50 (t, Xx) (Dooh 1957 Léonard 2014a).
On [8. 1 8] Lemma A.2 gives that heo strictly positive and C Moreover, ++ 2 y) solves the heat equation, hence (1,x)) + l:go (1, solves the backward heat equation on / < 1.
Let L. = 5A be generator under pro. Since (0. + h on < 1, Ité's formula gives, for
t t 5,
dh (t, Xi) (t, VedW, = h (t, X1) Ve V log (t, Xt) (.)x(a)l dW..
Hence M, == 1 X2) is a positive -mr-mantingale and dM, = M, 02 dW, with 02 =
V log (t, Xt) Define Q on F, by dQ]=, = M₁ d |r,i then by Girsanov,
Wº = We / 0,dd ds
a Brownian motion under Q, and therefore
dX,, = VedW, = =V log 2 (t, Xz) dt + Vs dd
Consequently the generator under Q is f = f + + h Vf, i.c. the claimed SDE with drift o > log he (Léonard 2013 2014a).
Local Lipschitzness on compacts follows from Lemma A.2
Lemma A.4. Assume
A.l Then, for each fixed xn and each E (0, 1),
X1
o (1,
E Bl
X = T
1 -
Proof. Under Q! first draw X1 u and then, conditional on X1 y. the Brownian bridge from so to Hence, for fixed 1 < and X, = x, the conditional distribution of X, is proportional
pi f v)
du
d (v).
pi (x
Assumption A.1 liplies f ||y|| d (y) < oo, and therefore X1| || < oo under Ti. For n, note that E, ||X 1| = E,, 12 < co, hence | |||| < oo a.s. so u has finite second moment for mo-a.e. and the conditional mean finite.
Differentiate
/
pi (x,))
1600 (1, x:) =
d (v)
(o y)
19

---

Published as a conference paper at ICLR 2026
with respect to (justified for any fixed t exactly in Lemma A.2
Using the Gaussian identity
<V EV y
1 px_)
we obtain
/ リー 1 dp (z).
<V hh
1-t p v)
Now divide both sides by ( (f) Since the conditional law of X1 given X = has density proportional to the same integrand, the right-hand side is exactly the conditional expectation of (Xi x)/(1 t). Therefore,
X1 x
o = eV log 22 (1, = 0
X, =
1 1-t
A.2 PROOF OF THEOREM 4.1
Proof. Using the disintegration theorem and additive property of relative entropy Theorems 1.6, 2.4) at time / 0, we get:
(Léonard 2014b
KL (Q+IIQ=T) = KL (T(20))1=(ZO)) + En(mo) KL (Qx120 || @
Note that since Qx Qid share the same marginals at time = 0, first KL term vanishes. In addition, KL ( Q
oo for -.e. Ip, since the I(Xo; X1) < co. Similarly, by using the disintegration theorem again for both times / = 0, 1, we get:
KL (Q+ (Q=1(Qim)) = KL (I (mo, a1) (1))l=(())) (m( [L (Qatzo.za 1001zo.a///0mm.. 1 )
Recap that Qx and Qind are both mixtures of Brownian Bridges. Therefore, Qr/Z0.=1
Qind F1
W |zo.21 and KL (
Qind ( 0. Then the following holds:
KT. (Q= (Q= = KT. (1) (xx) KT.
(Qu lmo @
)
(16)
Moreover, processes 2r and Qind 4|2o are diffusion processes ($))
Then we can apply the Girsanov
Theorem from (Léonard 2012):
KL @m1oo (mmp.)
olo (t, Xt) vinn (1, X1) dt
(17)
28 10
Now let J. O. The integrand is nonnegative, so the integrals increase with | 0, and by monotone convergence obtain the same identity with fo and combine it with (6)
KL (I x1 ([(())))) = Ex(en) [KL (mm Qime
(18)
2e f Eyg (m) [llijoial (2,, to) Vina (are t, ao) 12] I(Xo; Xz).
where drifts Ujiin and Uind defined in and (15 respectively.
20

---

Published as a conference paper at ICLR 2026
B
EXTENSIONS OF OUR METHODOLOGY
In this appendix section we present:
Appendix
B.
General KL divergence estimation method with proof and discussion.
Appendix
B.2
Differential entropy estimation result with two possible algorithms and discussion.
Appendix
B.
Experimental illustrations the entropy estimation algorithm.
Appendix
B
Interaction information estimation method.
B.1 KL DIVERGENCE ESTIMATOR
In this section, we present a general result for the unbiased estimation of KL divergence between any two distributions mi (1r), (2)) ((Rd) through the difference of drifts in the SDE formulation (4) of reciprocal processes induced by these distributions, i.e.:
Qm def f Wi dr (x1)dp((0o)
(19)
Qma def
f Wi dr d
(20)
Theorem B.1 (KL divergence decomposition). Consider distributions Tı (I(1),, #2(21) (, p(wo)
P(R"), that satisfy some regularity assumptions (Shi et al. processes Qn Qx, induced by distributions (I1), (2(11) (1)) 20) Then the KL divergence between distributions Ti (x1) and T2 (21) can be represented the following way:
2023 Appendix C), and reciprocal
KL. (I((())))))( 2c {( Eg,, (an)zo) [Il0* (Iz, t,o)
=²2 " 2,0,
(21)
where
I1 - It 1
"x (oe, L, 2.0) = Bgg (21 200
(22)
" (3r, 10) = Byg (21(
I1 It
(23)
t
are the drifts of reciprocal processes Qx and Qmz respectively. Holds for any distribution p(æro)-
Our theorem allows
estimate KT. (T (2()) knowing only the drifts nnt (xx, wo) and
" (Ia, f, mn), which can be recovered using conditional bridge matching ($2) Note that the expres- sion (1) is very similar to the expression (13) in (84.1). Distribution p(ro), can be considered as
part of the design space and optimised for each particular problem.
The KL divergence is fundamental quantity, and its estimator can have many applications, such as mutual information estimation or entropy estimation using results described in B.2
Proof. Consider
KL (m (an (m))))) KL (p(ao))lll(())) + +Bp(s [KL ((a)) =
Ep(zo) [KL (T)) KL (I((a))))
Next to get
1 - { Ey,, (ur) [lle (er, 20) - (z,,
KL (mm
2c
repeat all the steps that taken show:
21

---

Published as a conference paper at ICLR 2026
1
/
KL ( (1)1)(00))(()(
g (m) Voli t, 2,0)) (an t, xx,) xr)oll]dc
2e
in the proof of Theorem 4.1
B.2 DIFFERENTIAL ENTROPY ESTIMATOR
general result the information projections and maximum-entrope distributions suggests way of calculating differential entropy through the KL divergence estimation.
Theorem B.2 (Theorem 6.7 in (Kappen 2024)). Let Φ: R TR be any measurable function,
absolutely continuous probability distribution p(x) P(RR) and define def Ep 0(z). Now, for any 0 E R consider absolutely continuous probability distribution qo E *(RR)) with density:
qe (z) = exp((0, <(z)) - A())), A(0) = log Eg (x) e(0.0(x)).
there exists l such that p is absolutely continuous w.r.t. 90- and Egg (x) Φ(x) a, then
(p) = H KL (pll40-),
Corollary B.3. Let X be d-dimensional absolutely continuous random vector with probability density function p, mean m and covariance matrix E. Then
(p)) H (N(m, E)) KL (pll Σ)), H (W(m, Σ)) = log ((2Te) det Σ)
where N (nn, E) is Gaussian distribution of mean and covariance matrix E. Corollary B.4. Let X be an absolutely continuous random vector with probability density function and supp X S, where S has finite and non-zero Lebesgue measure u(S). Then
H(p) (U(S)) KL (pI(U(S)),
H (U(S)) log ((S),
where U(S) is uniform distribution S.
Similar results can also be obtained for other members of the exponential family. The first result (B.3) can considered as a general recipe, while the second one (B.4) can be useful when we have prior knowledge about the support of X being restricted. Approach described in B.2 is very flexible and can be considered as a generalization of the method used in (Franzese et al. 2024).
In practice, to estimate the entropy of some probability distribution, it is sufficient to follow one of the described in B.4 3.44 or B.3 For example, if one uses Corollary [B33] 1) estimate mean m and covariance matrix Σ using set of data samples, 2) calculate entropy H (M,, E)) via the closed form expression, 3) calculate the KL divergence KL (p]lWWmm Σ)) via learning two conditional diffusion bridges models and utilizing our estimator
21 from B.
B.3 ENTROPY ESTIMATION EXPERIMENTS
We conduct entropy estimation experiments using our method proposed in Appendix B.2
We estimate the entropy of high-dimensional (D 10) exponential and uniform distributions with varying distribution parameters. Corollary B.3 (Gaussian) method is used for the both distributions and Corollary B.4 (Uniform) method for the uniform distribution, since is has finite support. For each of the setups we prepare the dataset of 8000 training and 2000 test samples. Additionally we explore different reference distributions p(rro), see Theorem B. for the KT. (-IlA Σ)) or KT. (S) estimation. We explored four reference distributions p(zo low (e 0. medium (e = 1), and high = 10) variance Gaussians (all with zero mean), and a "data Gaussian" with mean and standard deviation derived from the training data.
One can observe that InfoBridge capable of entropy estimation on practice. Versions with low/medium variance gaussians generally show superior performance to high variance gaussian and data gaussian. the case of uniform distribution (Corollary [B4] method performs a lot better, suggesting that it is preferable in the cases when it can be applied (restricted support distributions).
22

---

Published as a conference paper at ICLR 2026
Ooeed Tull
Settonn Mons Vaker Ceaa ligh
Saute Met Valince Gravlle
Caubs inc righ tarane
Unilmm Lon sorance Couaran
Unftm Verlanc Guantio
Crrmay
Grund
Figure 4: Comparison of our InfoBridge combined with different methods Corollary and C.roollayB.44Aniiifee
B.3 (Gaussian)
B. (Uniform) and with different reference distributions p(ro) the entropy estimation task. Along axes is the ground truth entropy H (X), along y axes is the entropy estimate 11 (X). Each configuration is ran once.
B.4 INTERACTION INFORMATION
Here we propose the generalization of InfoBridge for the interaction information estimation, which is the generalization of mutual information for more than two random variables (Rosas al. 2019).
Interaction information for random variables Xu, X1, X2 is defined by:
[(Xo; X2) def - 1(xo; KL xX,,, [x, Ix,,) KL (Ixo.x,Ixx, [x.1xa Ix.xx.)
(24)
This definition can be generalized for more random variables similar way. Both MI information terms be estimated using Theorem 4.1 and our practical algorithm InfoBridge. Applications of interaction information include neuroscience (Bounoua al. 2024b), climate models (Runge et al. 2019). econometrics (Dosi & Roventini 2019).
B.5 MUTUAL INFORMATION BETWEEN RANDOM VARIABLES OF DIFFERENT DIMENSIONALITY
Here we outline the ways to estimate the mutual information between variables different dimension. In turn this allows to calculate the mutual information between random variables from different modalities, c.g., text and images.
Zero or noise padding. One can embed the lower dimensional variable into the higher dimensional space by padding the spare dimensions with zeros or noise. Such padding does not affect the mutual information, (1) the padding can be decoupled from the original random variable, and (2) it remains uncorrelated with the other random variables.
Employ the general KL estimation result. One can employ our general KL estimation result, i.e., Theorem B.I There, we propose to add a third distribution, e.g., point, normal distribution, distribution of images, to start our two diffusion bridge matching models from. One can set Ti (x) = ((11, x2) and Ty (x) = 4u (111)g2(z)))) where #1 and 22 random variables of different dimension, i.c., 21 E dd and 22 d2
Autoencoder-basee latent alignment. Another approach is to embed both random variables into common latent space via autoencoders. Since properly trained autoencoder preserves mutual information (Butakov et al., 2024b), MI estimation can then be carried out in this shared latent space.
23

---

Published as a conference paper at ICLR 2026
C ADDITIONAL DISCUSSION ON METHOD
C.I CONCEPTUAL COMPARISON WITH MINDE
While MINDE also utilizes diffusion models for the MI estimation task, our method is built with totally different diffusion processes in mind. Utilization of regular diffusion generative modeling processes in MINDE has several disadvantages that our method particularly addresses.
First, by leveraging finite-time bridge processes instead of infinite-time diffusion (which in practice is approximated with finite-tine time horizon), we provide an unbiased estimator. In particular, one can see that the MINDE MI estimator (9) has an intractable bias term it, i.c. (L (f).. while our MI estimator
113 is exact.
Second, MINDE treats MI estimation as a generative modeling task, learning drifts transform (0,, I) into II,, or [x No- In contrast, our method frames it it as domain transfer task, directly transforming input marginal Xx,, distributions xx, XXo or X1x, This lead two consequences:
In domain transfer the diffusion bridge process easier to learn, since it both starts and ends at data. This is contrast to complex generation diffusion process that starts noise and ends at data. The diffusion bridge approaches for domain transfer with diffusion process starting at data are known be superior to conditional diffusion models (Liu al., 2023 Zhou et al. 2024).
our method each of the diffusion process starting at some particular point zo is a Schrödinger Föllmer process (Vargas et al. 2023), which is special instance of Schrodinger Bridge (Bortoli
et al. 2024; Vargas et al. 2021). Trajectories of Schrodinger Bridge processes with small volatility coefficient require less energy for translation and are relatively "straight". As result, these trajectories are easy to follow, are expected to deliver less numerical error when dealt with and are easier to learn.
To empirically support the mentioned superiorities of our domain translation trajectories over MINDE's data generation trajectories, we compute the variance of MI estimation, which is based on trajectory Monte Carlo integration (Eq 133 for our method, Eq 19 (Franzese et al. (2024) for MINDE). Specifically, evaluate MINDE-C C and our method the image-based benchmark ($ 5.2 using
16x16 rectangular images. We vary the number of sample tuples {>r) at, xy N for MINDE-C and
N {ab. 2/, i 2m1 for our method and run 10 MI estimates per setting, and report the mean standard deviation in Table 3
Method \ N samples
128
1024
4096
16384
65536
131072
InfoBridge (ours) MINDE
9.79 = 0.72
9.84 = 0.34
9.9 + 0.1 10.62
9.84 + 0.06
9.86 + 0.028
9.84 = 0.028
12.20
3.32
11.17
10.67
Table Comparison of MI estimation accuracy across different sample sizes.
Our method achieves 1-2 orders of magnitude lower standard deviation in MI estimates compared to MINDE. Even with 1000x more samples, MINDE only approaches our accuracy. This due to its reliance on complex noise-to-data trajectories, which make its estimates highly volatile and unreliable, unlike our more stable data-to-data paths. As shown in Figure [] MINDE consistently exhibits the largest and most unstable confidence intervals, undermining the reliability and precision of its MI estimates. Furthermore, every sample requires inference of neural network and hence the reliable estimation of MI by MINDE requires several orders of magnitude more computational resources.
C.2 ABLATION STUDY
Here vary different important hyperparameters of our InfoBridge method.
Volatility coefficient c. We evaluate different volatility coefficients on the image benchmark ($5.2 and protein embeddings data ($5.3). We run the image benchmark experiment on Gaussian 16x16 setting with ground truth MI=10 and the full protein embeddings data experiment with one random seed and report mean MAE (Mean Absolute Error) error for the last 5 measurements during training,
24

---

Published as a conference paper at ICLR 2026
in both of setups. Results presented in Table 4 and Table One can that MAE relatively stable for all the setups, while the optimal e values is different for the both experiments.
One can notice that 1 works relatively well on both of the setups. Furthermore, well on all the experiments we have done, we use it for the three out four experiments in the main part of the paper and show its competitiveness light we call c = a robust choice suitable for all the data.
performs
the high MI experiment in Appendix D.5 In that
We consider hyperparameter corresponds to the reduction of variance and growth of bias, i.e., the learning procedure is very stable but slow, MI is mostly underestimated, and the method is stable w.r.t. other hyperparameters. While a smaller hyperparameter corresponds to the growth of variance and reduction in bias, i.e., the learning procedure has more variance but converges faster, and the model is less stable w.r.t other hyperparameters. The general advice is to take as low as possible to keep the learning procedure stable. In most cases, 1 is enough to get decent results and keep the model stable.
tool for bias/variance trade-off. The bigger hyperparameter
Table 4: Mean Absolute Error (MAE) values on Image data benchmark (Gaussian 16x16) with ground truth MI=10 experiment for the different e values. Each configuration (expect for 1) is ran one time and 5 last MI measurements during training are averaged produce the shown result. The best configuration is bolded and configuration used in the main part of the paper is underlined.
0.001
0.003
0.01
0.03
0.1
0.3
1
3
10
MAE 4
2.09
1.79
1.53
1.53
1.72
1.79
2.46
2.82
6.45
Table 5: Mean Absolute Error (MAE) values on ProrTran5 experiment, averaged over all the ground truth MI values, for the different values. Each configuration (expect for 1) is ran one time and 5 last MI measurements during training are averaged to produce the shown result. The best configuration is bolded and configuration used in the main part of the paper is underlined.
0.3
10
MAE 0.775
0.232
0.04
0.089
0.129
Number of parameters in a neural network. The width of the Unet (Ronneberger et al. 2015 neural network architecture used in InfoBridge Image benchmark experiments is varied to yield different number of parameters of a neural network. Numbers of base filters, 256, 64, 32, 16, 8, yield number of parameters, 27M, 1. 1.7M, 428K, 108k, 28k. Each configuration, except for 256 number of filters and 27M parameters is run once, and the final five mutual information (MI) measurements during training are averaged obtain the reported results. Outcomes are presented in Table As one can see our InfoBridge is robust w.r.t. neural network architecture.
Table 6: Comparison of the InfoBridge with different neural network architectures on the Image data benchmark (16 16 Gaussians setup) with ground truth MI=7.5. Each configuration ran one time and 5 last MI measurements during training are averaged to produce the shown result. The configuration used in the main experimental part underlined.
N Params
GT MI=7.5 (mean + std)
28k
7.56 0.3
108k
6.67 + 0.15
428k
6.67 0.13
1.7M
6.52 + 0.07
27M
6.59 + 0.17
Training dynamics. Here show the training dynamics of our InfoBridge the High MI experiment within the Smoothed Uniform distributions setup from Section Originally we train our model for 200 epochs and the Table shows the MI estimation the different stages of training:
25

---

Published as a conference paper at ICLR 2026
10, 20, 30, 60, 120, 200 epochs. The results presented in Table 7 One that our method is robust w.r.t number training iterations or number of training epochs used.
N epochs GT MI
10
20
40
80
10
8.14
16.09
30.87
47.70
20
7.79
15.51
30.26
48.61
30
7.70
15.50
30.47
50.16
60
7.51
15.02
29.67
50.20
120
7.29
14.63
29.55
50.01
200
7.01
14.32
28.89
50.82
Table 7: The MI estimation of ours InfoBridge method is shown for different training stages. The experimental setup is High MI test with Smoothed Uniform distributions. The same 10, 20, 40, 80 MI distributions, as in the main text are used.
C.3 TRAIN SAMPLE SIZE STUDY
Here we study the stability of our InfoBridge method w.r.t. the size of train dataset. While in main text we study datasets that do consist of 100k train samples in all the cases except for protein data experiment Section 5.3 in this section we test lower train samples sizes of and 10k.
In particular, we take the Rectangle 16 x 16 experiments setup from Section 5.2 where we do vary the number of training samples: 100k, 10k and 5k. The number test samples is the same 10k. The results from 100k are taken directly from the original experiment in Section 5.2 For MINDE and our InfoBridge methods we do make neural networks smaller to prevent overfitting.
Method
N train samples
N params
MAE
0.1
2.5
5
7.5
10
InfoBridge (ours)
5k
106k
0.199
0.008
0.79
2.33
4.83
7.20
9.75
MINDE
5k
106k
2.397
0.47
0.36
0.44
0.38
4.90
5.91
MINE
5k
50k
1.618
0.60
2.00
3.74
4.77
5.28
InfoBridge (ours)
10k
428k
0.142
0.006
1.02
2.65
5.24
7.65
9.80
MINDE
10k
428k
1.663
0.46
0.44
0.48
5.93
6.93
4.46
MINE
10k
50k
1.513
0.00
0.78
2.10
3.90
4.85
5.39
InfoBridge (ours)
100k
27M
0.228
0.00
1.16
2.79
5.47
7.67
9.82
MINDE
100k
27M
0.815
0.41
0.91
2.45
4.22
5.63
8.21
MINE
100k
50k
0.978
0.47
1.15
2.43
4.41
5.86
6.95
Table Image benchmark experiment with Rectangle x 16 setup with variable train samples size. MAE stands for Mean Absolute Error. The best method along each N train samples setup is bolded.
The results are presented in Table One can see that our method's performance stays almost the same, while MINDE and MINE performance degrade. This shows the robustness of our method w.r.t. the size of train dataset.
C.4 STRATEGIES FOR DRAWING TRAIN SAMPLES
Here we explore several design choices for sampling the data utilized for computation of the indepen- dent drift function loss C3 in the Algorithm In (ettizia al. 2024 authors argue that independent permutation used in the Algorithm introduces additional correlation. In practice, the simplest one as regular permutation works well, but can be replaced by either "derangement" permutation (Letizia al. 2024
marginal draws.
We test these strategies on the High MI experiment with the Smoothed Uniform distribution. All the ours Inforrdge method hyperparameters (except for the training data sampling method) match the ones used in the main text. The results are presented in Table Table/aand show a slight performance improvement for both derangement and fresh marginal draws strategies across all the setups.
26

---

Published as a conference paper at ICLR 2026
Samples draw strategy
MAB
MI=10
MI=20
MI=40
MI=80
Independent Permutation*
12.24
7.01
14.32
28.89
50.82
Derangement
10.51
7.24
14.74
31.21
54.78
Fresh sample draw
10.62
7.31
14.76
30.06
55.38
Table 9: Study on methods for drawing independent samples the computation L3 in Algorithm The experiment setup is High MI (see Section 5.4) with Smoothed Uniform distributions. for method used in the main paper experiments. In the top MI states for ground truth MI and the closer results to the ground truth the better. The best MI estimation is bolded for each MI setup.
states
C.5 ONE NEURAL NETWORK APPROACH VERSUS TWO NEURAL NETWORKS APPROACH
In this section, we empirically justify the one neural network choice described in Section In particular, we take Image Data experiment Rectangle 16 16 setup from Section 5.2 and try two drift functions parameterizations: 1) one neural network approach described in Section 4.2 and 2) two neural networks approach where each Ujoim and Cind parametrized by different neural networks Both approaches have hyperparameters that do match ones used for the experiment in the Section 5.2 fully. The results are presented in Table I0 One can see that the two neural networks approach consistently overestimates the MI and generally provides more error. That is why stick to the one neural network approach.
Model
GT MI=0.1
GT MI=1
GT MI=2.5
GT MI=5
GT MI=7,5
GT MI=10
One Neural Network
0.00
1.16
2.79
5.47
7.67
9.82
Two Neural Networks
0.83
1.86
3.49
5.93
8.12
10.23
Table 10: The comparison between the parametrizations for drift functions Ujoint and Vinds i.e., one neural network approach and neural networks approach. The comparison is done Image Data benchmark Rectangle 16 x 16 setup. The table shows MI estimation and GT MI in the top row stands for ground truth MI. The best result along each setup bolded.
D EXPERIMENTAL SUPPLEMENTARY
In this section, additional experimental results and experimental details are described.
D.1 METRICS
MAE. Mean Absolute Error (MAE) is computed as follows:
MAE(x, v) yl.
We measure the MAE between MI estimates and ground truth values. Image Data benchmark analysis Table 2 the MAE is shown averaged over all of the experiments including the different seed runs. In Appendix C.2 Ablation study Table the MAE averaged over last MI estimations during training and in Table the MAE averaged over all the experimental setups and 5 last MI estimations during training.
Std. Standard deviation (std) is computed over samples as follows:
1
std ad(((=))
c T)
where Eiv Zz.
27

---

Published as a conference paper at ICLR 2026
Table 11: Neural networks hyperparametes for low-dimensional (czy2 al. 2023) benchmark. "Dim"
dimensionality of a MI estimation problem, "Filters" number of filters in MLP, "Time Embed" number filters in time embedding, "Parameters" number of overall neural network parameters.
Dim
Filters
Time Embed
Parameters
<5
64
64
43K
25
128
128
176K
50
256
256
699K
D.2 LOW-DIMENSIONAL BENCHMARK
Additional results. In Table 21 we present the results of low-dimensional benchmark CZyz et al.
2023) with precision of 0.01 nats.
Experimental details.
The benchmark implementation is taken from the official repository:
InfoBridge. Neural networks are taken of almost the same architecture as in (Franzese et al. 2024). which is MLP with residual connections and time embedding. Additional input described in ($1.2) and is processed the same way time input. Number of parameters is taken depending dimensionality of the problem, see Table 11 Exponential Moving Average as a widely recognized training stabilization method
used with decay parameter of 0.999. For all the problems neural networks are trained during 100k iterations with = 1, batch size 512, Ir 0.0003. Mutual Information estimated by Algorithm 2 with N pairs of samples {ar xnj= 13 J==1 where N equal to the number of test samples times 10, I.c., 100k. The time is sampled uniformly in E [0, 1 10-9) segment. The approximate runtime was under the 20 minutes for each of the experimental setup on NVIDIA A100 GPU.
NVF and JVF. The implementation of NVF and JVF is taken from the official repository:
and used with all the provided by authors default hyperparameters for the benchmark. The methods CPU and execution takes less then two hours for each benchmark problem run. The size of a neural network is from 10k to 400k parameters depending of the task dimensionality and the submethod. The approximate runtime was under the 30 minutes under the 30 minutes for NVF and under the 1 hours for JVF for each of the experimental setup on NVIDIA A100 GPU.
D.3 IMAGE DATA BENCHMARK
Random seeds. We InfoBridge, MIENE, and MINDE with 3 random seeds, and all other methods with 5 seeds per experimental setup. In Figure Table we report the mean performance across seeds, along with standard deviations and confidence intervals computed from the seed-wiss variability.
Experimental details for InfoBridge. The implementation of image data benchmark (Butakoy
al. 2024a was taken from the official repository:
ht.p::://gtthub..mmm/aaeessssss
Following authors of the benchmark, Gaussian images were generated with all the default settings, Rectangle images were also generated with all the default settings, but minimum size of rectangle is 0.2 to avoid singularities. All the covariance matrices for the distributions defining the mutual information in the benchmark were generated without randomization of component-wist mutual information, but with randomization of the off-diagonal blocks of the covariance matrix.
28

---

Published as a conference paper at ICLR 2026
(a) Gaussiams daia samples zo x(zo]
(b) Reciangles data samples zo rv ((
Gaussians samples generated from (aro) (in (uno)
(d) Rectangles samples generated from (or [za) x(ar (no)
(c) Gaussians samples generated from si (01 (an) ze m(()))
(f) Rectangles samples generated from ni (2r (20) ss x<(1))
Figure 5: Examples of synthetic images from the (Butakov et al. 2024a) benchmark can be seen Figures 5a and [bb] Note that images are high-dimensional, but admit latent structure, which is similar to real datasets. Samples generated by our InfoBridge from the learned distributions TT ss и (I1 (zo) and Tian (2o) = (211) defined as solutions to SDEs 4 with approximated drifts (:; 0) and uo( 1), respectively, can be seen in Figures 5c to 5f resolution.
All the images have x2x32
To approximate the drift coefficient of diffusion U-Net (Ronneberger et al. 2013) with time, con- ditional neural networks were used, special input s was processed time input. For all the tests, neural networks were the same and had 2 residual layers per U-net block with 256 base channels, positional timestep encoding, upscale and downscale blocks consisting of two resnet blocks, one with attention and one without attention. The number of parameters is ~27M. During the training, 100k gradient steps were made with batch size of and learning rate 0.0001. Exponential moving average was used with decay rate 0.999. Mutual Information was estimated by Algorithm 2 with N
pairs of samples {x?" #0 / N where N is equal to the number of test samples, i.e., 10k. Nvidia A100 was used for the InfoBridge training. Each run (one seed) took around and 18 GPU-hours for the 16 16 and 32 x 32 image resolution setups, respectively. The time is sampled uniformly in t [0, 1 10 segment.
Visualization of learned distribution. Figure presents samples generated from the learned distributions alongside ground truth samples from the dataset. It is evident that the samples from both To ( (2o) and lnd (21 ((o)) closely resemble those drawn from x (xp), indicating that the marginal distribution of Tig well approximated by the learned models.
Experimental details for other methods. In this part, we provide additional experimental details regarding other methods featured in Figure We report the NN architectures used for neural estimators in Table 12
Table 12: The NN architectures used to conduct the tests with images in
NN
Architecture
GLOW,
(5) splits, GLOW blocks between splits,
x1: 16 hidden channels in block, leaky constant 0.01
x2 is panales
16 x 16 (32 x 32)
images
x1:
Orthogonal projection linear layer inparatiel
x1:
[Conv2d(1, 16, ks=3), MaxPool2d(2), LeakyRelJU0.010)
Critic NN, 16 x 16 (32 x 32)
x x2()):
[Conv2d(16) 16, 16, ks=3), MaxPool2d(2), LeakyRel.U00.01)] x paraliel
x1:
Dense(256, 128), LeakyRel.U(0.01)
images
x1:
Dense(128, 128), LeakyReLU(0.01)
x1:
Dense(128,
MINDE. We the official implementation of MINDE:
29

---

Published as a conference paper at ICLR 2026
As neural network, adapt same U-net architecture for our InfoBridge in image data experiments with approximately the same number of parameters i.e., 27M. Training procedure hyperparameters do match the InfoBridge, i.c., 100k gradient steps with batch size 64 and learning rate 0.0001, exponential moving average with 0.999 decay rate. Mutual Information was estimated using 10 samples of {t', 21) per each pair {ar zf} in the test dataset. Nvidia A was used to train the models. Each run (one seed) took around 2 and 5 GPU-hours for the 16x 16 and x2x32 image resolution setups.
MIENE. Under MIENF we consider gaussian-based N-MIENE This method is based on bi- gaussianization of the input data via a Cartesian product of learnable diffeomorphisms (Butakov et al. 2024a). Such approach allows for closed-form expression to be employed to estimate the MI.
With only minor hability-increasing changes introduced, we adopt the Glow (Kingma & Dhariwal 2018) flow network architecture from (Butakov et al. (0024)), which is also reported in 12 ("GLOW").
We used the normfl package (Stimper et al., (023) to implement the model. Adam (Kingma used to train the network on 10 images with a batch size 512, and the learning rate decreasing from 5 10 10- to 10 geometrically. For averaging, we used 3 different seeds. Nvidia A100 was used to train the flow models. Each run (one seed) took longer than four GPU-hours to be completed. The size of normalizing flow neural network is around 200k parameters.
& Ba 2017) optimizer
4
KSG. Kraskov-Stöggbuuer www--Stötbuurr--rra Sttigbbaurr-Grasheeger (Kraskov et al.) 2004) mutual information estimator is a well- known N-NN pon-paramitrrcc method, which is very similar to unweighted Kozachenko-Leonenle estimator (Kozachenko & Leonenko 1987). This method employs distances k-th nearest neighbors to approximate the pointwise mutual information, which is then averaged.
We used (one nearest neighbour) for all the tests. The number of samples was 10 for Gaussian images and 10 for images of rectangles (we had to lower the sampling size due to degenerated performance of the metric tree-based k-NN search in this particular setup). A single core of AMD EPYC 7543 CPU was used for nearest neighbors search and MI calculation. Each run (one seed) took no longer than one CPU-hour to be completed.
MINE, NWJ, InfoNCE. These discriminative approaches are fundamentally alike: each method estimates mutual information by maximizing the associated KI.-divergence bound:
T(Xo: X1) = KT. (1)
sup F[T (1))
(25)
T: B XB
where T is measurable, and F is method-specific functional. In practice, 7 is approximated via a neural network, with the right-hand-side in (25) being used as the loss function.
Motivated by this similarity, we use a nearly identical experimental framework to assess each approach within this category. To approximate T in experiments with synthetic images, we adopt the critic NN
architecture from (Butakov al. 2024a), which we also report in Table T2 ("Critic NN")
The networks were trained via Adam optimizer on 10 images with a learning rate 10-3 batch size 512 (with InfoNCE being the only exception, for which we used batch size 256 for training and 512 for evaluation due to memory constraints), and MAE loss for 10 steps. For averaging, we used 5 different seeds. Nvidia A100 was used to train the models. In any setup, each run (one seed) took no longer than two GPU-hours to be completed. The size of discriminator neural network is around 50k parameters.
AE-WKL Baseline. Here we include additional baseline Auto Encoder + Weighted Kozachenko- Leonenko (AE-WKL.) for the image based benchmark, which we exclude from the main part of the paper due to instability w.r.t. hyperparameters and huge inductive bias.
The idea of leveraging lossy compression to tackle the curse of dimensionality and provide better MI estimates is well-explored in the literature (Goldfeld & Greenewald 2021 Goldfeld et al.) 2022 Tsur et al. 2023 Fayad & Ibrahim 2023 Greenewald et al. [003]] Butakov et al. 2024b). In our work, we adopt the non- non-linear compression setup from (Butakov et al. 2024b), which employs autoencoders for data compression and weighted Bozachenko-Leonene method (Berrett et al. 2019) for MI estimation in the latent space.
30

---

Published as a conference paper at ICLR 2026
However in this approach non-parametric MI estimator (WKL) approach heavily relies the autoencoder latent space.
First, prior works suggest that non- -a--aaaaraeec MI estimators are highly prone to the curse of dimen- sionality compared to NN-based approaches (Goldfeld et al. 2020 Czyž et al, 2023 Butakov et al.) (2024) and hence WKL performance seriously deteriorates with growth of latent space dimensional- ity. In particular, Figures 2 and 3 in (Goldfeld et al. 2020) and Table in (Butakov et al. (024a) indicate that weighted Kozachenko-Leonenke estimator fails to yield reasonable estimates at all if the dimensionality reaches certain threshold.
Second, the performance of estimator depends the amount of information preserved in the latent space by the autoencoder. If the dimension of latent space is less than dimension of the data manifold, the performance of the method should deteriorate as well.
We validate the AE-WKL approach with different dimensionality of autoencoder latent space in the Gaussian images setup. We increase the latent space dimensionality from d (which is the intrinsic dimensionality of the dataset in question) to d E {4, 5,8}. We report our results in Figure 6 One can see while with
latent space (which the true latent dimensionality of gaussian images) AE-WKL approach achieves remarkable results, but with growth of dimension to d E (4, 5, 8) the performance starts to deteriorate with
even leading to exploded estimates.
That leaves autoencoder approaches combined with non parametric MI estimators highly dependent on the successful latent space choice. This choice trivial on the image based benchmark, where the dimensionality of data manifold is known by construction, however such information is usually unknown in real world scenario, which makes approaches like AE-WI WKL highly unreliable.
The autoencoders
trained using Adam optimizer 10th images with batch size 512, learning rate 10 3 and MAE loss for 10t steps. For averaging, we used different seeds. Nvidia 100 was used train the autoencoder model. Each (one seed) took no longer than GPU-hour to be completed.
Runtime comparisons methods in Table
One see the approximate runtime comparisons between the featured
Method
Runtime
Hardware
InfoBridge
hours
GPU
MINDE
hours
GPU
MIENF
hours
GPU
InfoNCE
1 hour
GPU
MINE
1 hour
GPU
NWJ
1 hour
GPU
KSG
1 hour
CPU
Table 13: Comparison of methods approximate runtime on the Image benchmark 16 x 16. GPU states for NVIDIA A100.
D.4 PROTEIN EMBEDDINGS DATA
Table 14: The best result is bolded and underlined and second best result is bolded.
Method
InfoBridge
MINE
InfoNCE
KSG
MINDE-C
MINDE-J
MAE I MAE Std
0.04 0.01
0.223 0.017
0.237
0.116
9.29
1342
0.007
Data. We use the UniProt database proteins, their embeddings from prottrans_t5 xl_50 Elnaggar et al., 2021) model and download them from [https://githur
com/ / All proteins longer than 12 10 residues are excluded. These embeddings are then unit variance normalized. The final number of each protein embeddings (samples) used for training is 20641 and the dimensionality is 1024.
Latent-mutual welllliffooaatiioo
31

---

Published as a conference paper at ICLR 2026
10
ground truth
ground truth
MINE
N-MIENF
InfoBridge (ours)
AE+WKL 5:NN (d =
AE+WKL 5-NN (d =
2
16 16 images (Gaussians)
10
ground truth
ground truth
MINE
N-MINN
InfoBridge
AB+WKL 5-NN
AE+WKL N-NN (d
AB+WKL 5-NN
4
5 6
10
(b) 32 32 images (Gaussians)
Figure 6: Comparison of the MI estimators. Along æ axes is [(x; X1), along y axes is MI estimate
i(X;; X1). We plot 99% confidence intervals acquired from different seed runs.
Methods implementation.
InfoBridge. Hyperparameters are the same as for diwdemenniooal benchmark Appendix D.2 MLP has ~ 900k parameters, weight decay is 0.001. dropout is 0.2 and the model is trained for 100 epochs. Results for InfoBridge averaged over last MI estimations during training. The size of neural network is around 900k parameters. The time / is sampled uniformly in / E [0, 10-3) segment.
MINDE. We leverage the official implementation of MINDE https: / titbb.coom Must MuutaphaBBunoual nndeeell in the configuration used for the now-dimensional benchmark, with ~ 900k parameters MLP, le batch size of 256, ema decay 0.999 and train model for 100 epochs.
MINE, InfoNCE.
These estimators are taken from the bmi library cbg-ethz/ bmi with all the default parameters. The size of discriminator neural network is around 1k parameters.
com/
KSG. Implementation
is taken from https://githut latent-mutual inaallifformationn and used with all the default parameters on 20000 training samples.
All the neural methods were trained on Nvidia A100 and took less then one GPU-hour to complete. KSG estimation algorithm took approximately hour complete.
32

---

Published as a conference paper at ICLR 2026
Runtime comparisons. methods in Table
One the approximate runtime comparisons between the featured
Method
Runtime
Hardware
InfoBridge
20 minutes
GPU
MINDE
3 minutes
GPU
MINE
minute
CPU
InfoNCE
minute
CPU
KSG
20 minutes
CPU
Table 15: Comparison of methods approximate runtime on the Protein Data experiments. GPU states
for NVIDIA A100.
D.5 HIGH MUTUAL INFORMATION EXPERIMENTS
Half Cube transform. We adopt the half-cube transform for Gaussian random variables intro- duced in (Czyż et al. 2023):
[(x) - xv/=]
This transform lengthens the tails of Gaussian random variables, while preserving the MI due to f(x) transform being reversible.
Additional experimental results. In addition to the high MI experiment in the main text (5 5.4 here we present extended comparisons with other baselines: see Table [6] for Gaussian random variables, Table 17 for Gaussian random variables with half cube transform, Table [8le[[8] for Correlated Uniform random variables, Table 19 for Smoothed Uniform random variables. Overall our method still is the most reliable and accurate of the bunch.
As shown, our method performs well with both c and 0.01, with the latter providing slightly better results. For this reason, we adopt 0.01 in the main text comparisons.
One can see that MINDE-J offers the closest to the ground truth it drastically overestimates the MI in the d=160, MI=80 setup across all the distribution types. Such behaviour is typical for MINDE-J on complex datasets also, see ($ 5.2) and ($ (.33) that is why we leave it out in the main text comparisons in (5 5.4).
many setups, but we note that
Method
d=20, MI=10
d=40, MI=20
d=80, MI=40
d=160, MI=80
InfoBridge (ours, 1)
9.85
19,99
39.31
78.86
InfoBridge (ours, = 0.01)
10.35
21.79
41.92
79.76
MINDE-C
9,59
16.96
35.51
56.27
MINDE-J
10.23
20.02
42.11
450
MINE
6,51
8.43
7.76
8.08
InfoNCEt
5.14
5,40
4,76
3,80
ADIME (KL,
5,51
2.49
0.0
IDIME (HD, J)
5.24
3,66
1.14
0.0
fDIME (GAN, J)
6,88
6,71
3,54
0.0
fDIME (KL, D)
3.71
4.06
0.22
0.0
IDIME (HD,
4.01
4.24
0.23
0.0
IDIME (GAN, D)
4.86
0.32
0.0
Table 16: Mutual Information estimates across increasing dimensions and mutual information levels for the Gaussian random variables. Best MI estimation bolded and second best underlined. stands for method shown in the high MI experiment in the main text. Empty cell stands for method explosion.
Experimental Details. We use 100k train samples dataset and 10k test dataset for all the methods. Datasets are generated using mutinfo library github repository:
33

---

Published as a conference paper at ICLR 2026
Method
d=20, MI=10
d=40, MI=20
d=80, MI=40
d=160, MT=S0
InfoBridge (ours, 1)
9.07
18.23
34.35
61.63
InfoBridge (ours, 0.01)*
9.92
20.69
39.6
84.24
MINDE-C
8.31
14.69
33.64
40.63
MINDE-J
9.28
16,49
43.3
308
MINE
3.18
3.0
3.28
3.74
InfoNCE!
4.11
4.7
3.84
2.74
IDIME (KL., J)
7.52
6.84
11.3
8.18
IDMME (HD, J)
6.56
11.59
10.77
IDIME (GAN, J
9.7
14.7
20.84
13.91
IDIME (KL.,
5.41
6.82
6.67
6.76
IDIME (HD, D)
5.71
7.66
7.99
8.74
IDIME (GAN, D)
7.83
10.4
11.49
10.86
Table 17: Mutual Information estimates across increasing dimensions and mutual information levels for the Half Cube transformed Gaussian random variables. Best MI estimation is bolded and second best underlined. stands for method shown the high MI experiment in the main text.
Method
d=20, MI=10
d=40, MI=20
d=80, MI=40
d=160, MI=80
InfoBridge (ours, 1)
7.95
15.15
28.46
41.9
InfoBridge' (ours, = 0.01)
8.65
16.93
33.69
57.73
MINDE-C
8.03
15.14
30.78
55.91
MINDE-J
17.56
34.59
539
MINE
6.12
7.9
7.92
7.11
InfoNCE'
4.48
3.71
3.42
DIME (KI,, J)
7.6
6.1
0.99
IIMM (HD,
8.28
8.84
4.8
0.19
IDME (GAN, J)
10.33
8.75
1.56
IDIME (KL,, D)
5.62
5.1
2.71
0.1
IDMME (HD, D)
6.6
7.02
2.14
0.41
IDIME (GAN, D)
6.49
7.88
3.79
0.0
Table 18: Mutual Information estimates across increasing dimensions and mutual information levels for the Correlated Uniform random variables. Best MI estimation is bolded and second best is underlined. stands for method shown in the high MI experiment in the main text.
https:
InfoBridge. Implementation was taken with exactly the same hyperparameters as for low-dimensional benchmark experiment, Appendix D.2
MINDE. We leverage the official implementation of MINDE https://github..o com YbusaaphBBunoual htt//minnde mindel in the configuration used for the low-dimensional benchmark, with ~ 900A parameters MLP, le 4, batch size of 256, ema decay of 0.999 and train model for 100 epochs.
These estimators are taken from the bmi library https: com/ cbg- ethh/ hhbmmmmm with all the default parameters. The size of discriminator neural network is around Ik parameters.
MINE, InfoNCE.
DIME. implementation taken from the official github repository:
ht httpss//gsttabb.co//toeeeeeeeeet EDMME
with all the default hyperparameters. KI, HD, GAN stand for different divergences and J, stand for Joint and Derranged architectures of fDIME.
Runtime comparisons. methods in Table 20
One can see the approximate runtime comparisons between the featured
34

---

Published as a conference paper at ICLR 2026
Method
d=20, MI=10
d=40, MI=20
d=80, MI=40
d=160, MI=80
InfoBridge (ours, 1)
6.4
12.83
24.21
29.83
InfoBridge (ours. 0.01)
14.63
29.25
50.28
MINDE-C
6.53
13.23
24.98
44.2
MINDE-J
7.63
14.39
31.15
511
MINE
5.56
7.45
7.91
7.17
InfoNCE!
3.84
3.67
3.34
3.14
ADIME (KI., J)
6.15
IDIME (HID, J)
5.9
8.12
3.95
0.0
IIME (GAN, Jyt
7.75
8.49
4.75
0.0
ADIME (KI., D)
4.91
4.17
0.32
IDIME (HID, D)
4.59
4.82
0.37
0.0
IDIME (GAN. D)
5.65
5.65
0.33
0.0
Table 19: Mutual Information estimates across increasing dimensions and mutual information levels for the Smoothed Uniform random variables. Best MI estimation is bolded and second best is underlined. stands for method shown the high MI experiment in the main text.
Method
Runtime
Hardware
InfoBridge
30 minutes
GPU
MINDE
10 minutes
GPU
f-DIME
minute
GPU
MINE
minute
CPU
InfoNCE
minute
CPU
Table 20: Comparison of methods approximate runtime the High MI experiments in the D 160.
MI=80 case. GPU states for NVIDIA A100.
35

---

Published as a conference paper at ICLR 2026
02 022
1S 0. 0-4
16 051 0
06 0.51 0.4
29 1.66 0372 0-41
6. 0.32 0
0-41
1.99 0.52 (1-4
1.68 0.32 0
0,42
1.63 0.19 2290000
0.42
0.900.3
0.38
Wigsly Bivariate Nox xx]
1. 0533
1.69 0.3
0.88
0.08 0:35
Uniform x (addirive noisew.75) 75)
Table 21: Mean estimate over 10 seeds using 10 10º test samples compared each against the ground-truth Size of train datasel for every neural method is 100k. All
the methods for comparison. except for NVF and JVF Dahlke & Pacheco (2025) with InfoBridge were taken from Franzese el al. 2024 List of abbreviations (Mr.
16 S
|
Uniform x (additive noise=,1) I)
04 15 0 0e
22 0. LA
000 0. 06
0.00
0.23
LS
1.58
Swiss roll 2 x
00
V0 . 000 000 21D 0112 T01 1160022
0.2
X 5
0.000
1.48 0.50
5t 5 (dof=3)
11 00771
0 NIN 200 LIu DL DI
0 0. O0 OO oo o o0 24 D
co Sel
000 0.31
St 5 x 5 (dof=2)
1.00 U a Us0 01900 ( 1100 & < & 0 0.0000
00
&8 U. 3.30 U. 0.2 211
a0 0.3 A0 a Ou
0.50
St 3 x 3 (dof=3)
13 2666 100 000
1.72 OO 98 0.% 999 & 060 6% 0500000 0.90 0 0.005 000000 0.00
03
0.13
2.54
0,82
St x 3 (dof=2)
1.62 1.620-11 1100 1o0 1.00 in 10 on cor on u 2220 11000 10
momo Moooa 120
100
0.11
St x 2 (dof=2)
2 (a 02 155 1. tn POD
0.53 200000
0.33
7.98
St x 2 (dof=1)
6.7
St x (dof=l)
1o00 i de ai DI
deo 0 t00D 08 000 1o0 n
1,65 0.41 1.22 102 100 033 0. 0-4 0.00 0.89 0.99
000 1.73 0,4, 1:7 6 99 Se Su 03 101 D3 0.0
34 0.90
0. 0:77
0.55 0.67 0, 79 0.8
Sp @ Nm CDF @ Mic 5 x 5 (2- (2-pair)
1 O 88 U.S 9 OR O 7 SS 8 09 a.
@ Nin CDF 3 x (2-pair)
DI 0. 93 U
to as DIR O 0000 U
0.25 0.39 0.99
0. 044000000 0.9 Too 4000000 0o00 00 000000000000
0.99 0.93 0-45 0.58 0.560.57
Sp @ Nin CDF @ Mr 25 x 25 (2-pair)
1.62 0.20000 0. 099 .90 22 DO D
USS
5100 2.3
Sp Mr 5 x (2-pair)
0.4| 0.78 1.08 1.05 0.477.000000 0.60
Sp @ Mr 3 3 (2-pair)
0.33 1,00 0.97 0.96 86
0.95
Sp @ Mr x 25 (2-pair)
1.60 0.000.0 u 98 den 8Or
1.81 0nu0 o2 U 19
69 199
0,40 0. Pnd 22
1.28 0.42 2 092200
7:2
Nm CDF @ Mn 5 (2-pair)
Nm CDF @ Mn 3 (2-pair)
1.52 0.40 1
1.61 O oe S0 i
0.70
Nm CDF @ Mn 25 25 (2-pair)
1.68 0.60
0 1.39
0.37
Nm CDF @ Bivariate Nm 1
1.75
Mrr 50 50 (dease)
16.
16.
0:41 1.02 1.02 1.02 102 0.29 1.02 1.29 1.02 0.11 0.20.59
40 0. 0.00 0.95 0.96 0.98 0.29 1.01 1.28 0.09 0.11 0.999.0
1.27 1:01 041 1.02058
0.41 0.41 0:41 1, 1O 1.00 1:1 0.29099 1. 1.00 042 1.00
0.41 0.96 1OD00 99 1OI 0.29 ..00 1.26 101 0.11 1.01 1.01 1.01559
1OI 1.0
0.41 0.96 0.99 9. LOI 0.30 1.28 1.01 0.1 1.00
0.22 0.4 0.27 0.4 0.41 0.41 0,98 1,I LOI 102 0.29 0.99 99 1.28 IOI 044 1.020 055
1.020.59
1.02
0.07 0.95 0.4 0. 0.57
0.02 0.60
27
Mr x 5 (dense)
0.41 0.41 1.02 1.02 0. 102 0.29 1.03 1.29 1.02 0.41 1.02
94
0.430.4 0.35 0.36 0.62 0.92 0.95 0.340.69 6.669 83 0.95 0.63 0.93
Mic 5 pair))
1.28 101 0.41
0.41
0.95 0.35 0.67 0.677.88 7.89 0.05 0.64
Mu x 3 (dense)
0.02 0,29 1,06 1.33 102
Min 3 (2 puir)
0.42 1.19 102 00 102 0.29 0.99 1.31
Mn 25 x 25 (dense)
0.41 0.96 1o1 00 102 0299099
0.41 1,00 TOI 1OI LOI 0.29 1.00
0.22 0.27 0.4 41 0.41 0:41 0.08 1.01 1.01 1.22 0.29
0.29
1.03 0
Mic 25 25 (2-pair)
1 0.65
Mu x 2 (dense)
Multinormal, St: Student-t, Nm: Normal, Hc: Half-cube, Sp: Spiral)
Min x 2 (2 (2-pair)
1.01
0787 0.66
0.95 0.96
0.97 0.90 96
Hr. @ Mn 5 x 5 (2-pair)
68
Hc Mn x (2-pair)
0,41 0.38 0,99
0.57 0.330. 0.39 0.67
Hc Mn 25 x 25 (2-pair)
0.41
0.42 0:42 0.42
0.42 0.42
Hr. Bivariate No x
0-41
0.41
0.41
Bivariate Nm x
0.15 0:41
Bridge Matching 0.26 0.600888 48 0.31 0.45 0.44
0.55 0.41
0.0 0/ 0. 0 0.41
0.41
0.23 0.24 0.36
0.22 0.27 0.41
0.41
0.34
Bimodal 1x
0.21 0:42 0.27 4.4
0-15
0.40
0.21 0.42 0.28 0.42
0.22 0.38 0.19 0.24
6.68
0.00
Asinh @ St 5 x (dof=2)
0.22 0.29
30
0.21 0 40 26
0.22 0.42 0.
0.00 0.000000000 0.00
D. 0.160.48 0.27
0.3 0.21
Asinh @ St x (dof=2)
0.1 0.13 0.37
Asinh @ St 2 x (dof=l)
0.18
0.250.39
Asinh St (dof=1)
Method Type
Dificsion
Classical
Flow
MINDE-J (or = 1)
MINDE c (ar = 1)
DeEiGuusiin
InfoBrädge
MINDE-]
MINDE-
DeEiLoggste
MINE
GT
NVF
IVF
D.
NWJ
KSG
LNN
CCA
dist
36


Reviews:
[{"review_id": "bFnA5NieE2", "reviewer": "Reviewer_zM99", "summary": "This paper propose an unbiased estimator of mutual information constructed on bridge matching.\nThe evaluation is solely conducted by measuring the discrepancy from a ground truth mutual information.", "questions": "The typical condition for the application of Girsanov's theorem is that of Novikov. How do you suggest relaxing it to use InfoBridge with long tailed distributions as suggested in section 6?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "+ The main contribution is much sought after: An unbiased, low variance estimate of mutual information has impact across fields. \n\n+ The paper's literature review is apt and does a great job motivating the problem.\n\n+ The paper is full of illuminating insights tying information theory, diffusion models and stochastic calculus.", "weaknesses": "+ The evaluation is solely conducted by measuring the discrepancy from a ground truth mutual information. This is sound an necessary but also limited by itself. The paper would be more convincing if the proposed estimate of the mutual information were \nused as an objective, or its learned representations evaluated on downstream tasks.\n\n+ The work could be made much more accessible. The background section runs two full pages, and the authors only introduce their method in page 5. We suggest the more direct route of introducing a simple example right after the introduction, and introduce the stochastic calculus concepts as needed.\n\n+ The paper differentiate itself from MINDE[1] by 1) being unbiased while the latter is only asymptotically unbiased, 2) By presenting itself as a domain transfer task. Unfortunately, this shift in conceptual framing while thought provoking is only  indirectly evaluated through a reduction of the estimators in variance. This is a bit unfortunate as eliminating bias and reducing variance is enough to differentiate well enough from MINDE.\n\n[1] Franzese, Giulio, Mustapha Bounoua, and Pietro Michiardi. \"MINDE: Mutual information neural diffusion estimation.\" arXiv preprint arXiv:2310.09031 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OoNOQRBeY7", "reviewer": "Reviewer_at8D", "summary": "The authors propose InfoBRIDGE, a mutual information (MI) estimator that reframes MI estimation as a domain transfer problem between a joint distribution and the product of marginals using reciprocal processes built from mixtures of Brownian bridges. The required drifts are learned via conditional bridge matching with a neural net; training uses samples from the joint and a permuted batch to emulate independence, and evaluation computes the integral by Monte Carlo over Brownian-bridge samples. The authors argue this yields an unbiased MI estimator (given perfect drift learning) and report results on low-dimensional synthetic, image-based, high-MI benchmarks, and protein-embedding data.", "questions": "* Is MI provably independent of the choice of ϵ? in Theorem 4.1? If yes, please add a formal statement and a numeric ablation sweeping ϵ.\n* What is the estimator variance as a function of t-grid resolution and batch size? Any control variates that reduce variance when t → 1 (where the (1-t)^{-1} factor amplifies noise)?  \n* Does training two drifts (joint vs. independent) materially increase sample complexity vs. one-model approaches? Could a shared backbone with binary conditioning stabilize learning?  \n* How would you share a NN between variables of different dimensionality?\n* How does InfoBRIDGE behave on long-tailed or heavy-tailed marginals, beyond the cited remarks that classical estimators struggle there? Please add such stress tests. \n* Table 1: What are the reported metrics? Please add to caption description of the metric presented in the table.\n* Figure 1: why MI estimation undershoots for Gaussians (a,b) but not for rectangles (c,d)?\n* Discussion of basic implementation, training, and hyper-parameters details should be included in the main body. Include discussion of the computational cost.\n\nComment:\n\n* Limitations should be discussed in the main body, not only in the appendix.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Conceptual novelty: Clear, principled decomposition of MI into a time-integral of drift differences between two bridge processes; different viewpoint than score-based MINDE.\n* Methodological soundness: Builds on established conditional bridge matching; training and estimation procedures are straightforward (Alg. 1–2). \n* Practicality: Uses only joint samples and Brownian-bridge simulation (no explicit density estimation), which is attractive for complex data like images/embeddings.", "weaknesses": "* Unclear dependence on ϵ: The estimator formula scales with 1/(2ϵ). Please analyze or empirically demonstrate ϵ-invariance of the resulting MI and give guidance for choosing ϵ.  \n* Bias/variance in practice: “Unbiased” is only in the idealized limit. Did you check the empirical calibration with confidence intervals vs. ground-truth MI? Did you check sensitivity to network size, training steps, and t-discretization?\n* Independence construction: The independent process uses permuted pairs (x_0,\\hat{x}_1). What is possible bias when datasets contain duplicated or highly correlated samples? Will a two-sample protocol (fresh marginal draws) materially changes results?\n* Scope of experiments: The authors mention low-dimensional, image, high-MI, and protein settings; more detail is needed on metrics, baselines, and runtime/sample efficiency, especially versus strong discriminative bounds (e.g., InfoNCE variants).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DGgoLcwYXE", "reviewer": "Reviewer_uyVy", "summary": "The paper presents a novel unbiased MI estimator based on diffusion bridge models. The proposed estimator is tested on different low-dimensional and high-dimensional datasets.", "questions": "- What is $\\delta()$ in equation (3)? It is not defined.\n- In line 148 the authors refer to Problem (2). Does it refer to the second equation? That would be the MI definition. In the line below the authors write “minimizes (2)”, but they are not referring to the minimization of MI, I guess they are referring to the minimization of the long equation in line 145.\n- In the pseudocode of algorithm 1 and in the code provided in the supplementary material, the authors perform a random permutation to obtain the samples from the product of marginal densities: \ny_samples_permuted = y_samples[torch.randperm(y_samples.shape[0])].\nHowever, using such a function leads a certain number of elements of the shuffled vector to be positioned in the same initial position (before the permutation). This strategy was highlighted in f-DIME’s paper as problematic and in that specific case the authors showed that using a specific  discriminative architecture that mistake would lead to an upper bound in the MI estimate. For InfoBridge I do not see any upper bound. However, performing such a random permutation is logically incorrect and the authors should perform the experiments after correcting such a mistake. How does the performance of the proposed algorithm change after this correction? If it does not change, why? \n- Where are the experiments justifying the sentence in lines 314-315 about the additional binary input?\n- Can the authors show a comparison of the computational complexity between the different algorithms used?\n- Why the reference Lee and Rhee has no year?\n- The discriminative models used during comparisons have the same size (number of parameters) of the generative estimators?\n- Isn’t 100k train samples a very large number? How do the algorithms perform with a lower number of training samples?\n- In Figure 2 the authors specify which version of MINDE they are using, but do not specify which f-DIME version. Can the authors specify it?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The proposed estimator relies on diffusion bridge models, thus leveraging a new technique for MI estimation\n- The proposed generative MI estimator is unbiased, differently from MINDE which is biased\n- The experimental results show the effectiveness of the proposed estimator", "weaknesses": "- Although the proposed estimator leverages diffusion bridge models, which differ from diffusion models, the proposed framework appears as an extension of MINDE\n- The paper is not well-written. The clarity could definitively be improved. For instance, SDE is not defined (stochastic differential equation). The equation in line 145 should be written in two lines, as it exceeds the borders of the paper. There are many typos: line 96 (conditioned its on values), line 264 (as a domain transfer task), multiple wrong references to equation (2) while I assume the authors wanted to refer to other equations. The same wrong reference to (2) happens also in line 300. Table 2 reports Mean Average Error, but I assume the authors used the Mean Absolute Error.\n- In the experiments  the authors compare the proposed method with f-DIME. However, there is no reference of f-DIME in the related work section, so it should be inserted in the Discriminative estimators paragraph.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vvbRuxMJ5b", "reviewer": "Reviewer_UxJn", "summary": "The paper introduces a new formulation of mutual information (MI) estimation as a domain transfer problem. In particular, it estimates MI by quantifying the difference between the drift functions of diffusion bridge processes associated with the joint distribution and the product of marginal (independent) distributions of the variables. The idea is interesting and improves previously studied diffusion-based estimators, to which the authors compare by running extensive benchmarks.", "questions": "- For the experiment in Table 1, some methods do not require training. Do you only provide them with 10k test samples, or do they get access to 110k samples, i.e., the full train and test split. How would the method perform with only 10k samples for training and testing jointly?\n- Do you have an intuition as of why higher MI values seem to lead to worst estimation in the image gaussian 16x16 case, for all estimators?\n- “First, by leveraging finite-time bridge processes instead of infinite-time diffusion (which in practice is approximated with a finite-time horizon), we provide an unbiased estimator.” Why does discretization on the bridges process not induce a bias while it does for noise-to-data diffusion?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The authors theoretically validate the correctness of their new approach to use Bridge Matching for MI estimation.\n- The difference with prior work on diffusion-based estimator MINDE is well-expressed and theoretically grounded.\n- A rich set of standard benchmarks and ablation studies has been conducted.", "weaknesses": "- The main weakness in the experimental evaluation is that the authors only consider very large sample sizes; 100k training datapoints, while classical approaches have also been tested for 5k or less. Although the main competitor, MINDE, also used 100k training points, it would be interesting to see if the method can also perform well with significantly fewer samples, e.g., 10k.\n- While the MINDE estimator requires only the learned score functions to estimate mutual information, InfoBridge relies on full sampling of two diffusion bridge trajectories to approximate the drifts. It is not clear if this has implications for computational cost and scalability. It would be relevant for the authors to address and discuss the computational cost difference between the two methods.\n- The paper refers several times to regularity conditions, such as the existence of finite first moments, as prerequisites for the main theorem (Theorem 4.1). However, these assumptions are not stated explicitly in the main text nor appendix. Since these are key hypotheses for the validity of the theoretical results, it would be important to summarize them clearly.\n- The volatility coefficient $\\epsilon$ is studied in the 16×16 Gaussian and protein embedding experiments, leading to a rule-of-thumb setting $\\epsilon = 1$. However, the paper does not provide a detailed analysis of how $\\epsilon$ influences the estimator’s bias and variance, nor a principled procedure for selecting it.\n- Methods for MI estimation between random variables of different dimension are mentioned but not considered for experiments.\n- As mentioned by the authors, heavy-tailed distributions cannot be modelled well.\n\nMinor remarks:\n\n- Some formulas and tables violate the space constraints (see e.g., page 3).\n- Some references in the appendix are not rendered correctly.\n- Reference: lee & Rhee missing year.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)