A Bayesian Bootstrap Framework for Mutual Information Neural Estimation: Bridging Classical Mutual Information Learning and Bayesian Nonparametric Learning
Authors: Forough Fazeli-Asl
Michael Minyi Zhang
Linglong Kong
Bei Jiang
Venue: Accepted by TMLR ()

Abstract: In this work, we introduce a Bayesian bootstrap resampling framework for estimating mutual information (MI) via ``mutual information neural estimation'' (MINE), making MINE directly applicable in a Bayesian nonparametric learning (BNPL) framework. The resulting estimator shows low variability across batch sizes and high-dimensional settings, as demonstrated through extensive numerical studies. In particular, our proposed bootstrap version yields tighter and lower-variance estimates than the original MINE formulation, both theoretically and empirically. We further demonstrate its practical value in a downstream task by improving VAE-GAN training within BNPL, leading to higher-quality outputs. Beyond enabling MI-based BNPL, the proposed bootstrap estimator also performs competitively against leading frequentist state-of-the-art benchmarks. Overall, our findings establish the first principled framework for Bayesian bootstrap-based MI estimation and highlight its effectiveness as a reliable tool for future BNPL studies.

Published in Transactions on Machine Learning Research (02/2026)
A Bayesian Bootstrap Framework for Mutual Information Neural Estimation: Bridging Classical Mutual Information Learning and Bayesian Nonparametric Learning
Forough Fazeliasl Department of Mathematical and Statistical Sciences University of Alberta, AB, Canada
Juzelias@uulberta.com
Michael Minyi Zhang
1rahamg1@@haan
Department of Statistics and Actuarial Science Universily of Hong Kong Pok Fu Lam, Hong Kong
Linglong Kong
Wong@hralberta.co
Department of Mathematical and Stafistical Sciences University of Alberta, AB, Canada
Bei Jiang
bei wwll@uaabbrra.c. ca
Department of Mathematical and Statistical Sciences University of Alberta, AB, Canada
Reviewed on OpenReview: https: // openreview. net/ forum? id= id=mqG2GKXFFi
Abstract
In this work, we introduce a Bayesian bootstrap resampling framework for estimating mi- lual information (MT) via "mutual information neural estimation" (MINE), making MINE directly applicable in Bayesian nonparametric learning (BNPL) framework. The resulting estimator shows low variability across batch sizes and high-dimensional settings, as demon- strated through extensive numerical studies. In particular, our proposed bootstrap version yields tighter and lower-variance estimates than the original MINE formulation, both theo- retically and empirically. We further demonstrate its practical value in a downstream task by improving VAE-GAN training within BNPL, lcading to higher-quality outputs. Beyond enabling MI-based BNPL, the proposed bootstrap estimator also performs competitively against leading frequentist state-of-the-art benchmarks. Overall, our findings establish the first principled framework for Bayesian bootstrap-based MI estimation and highlight its effectiveness as reliable tool for future BNPL studies.
1 Introduction
Estimating and optimizing mutual information (MI) is important in many machine-learinng problems, from representation learning to generative modeling. The MI is defined through Kullback-Leibeer (KL) di- vergence between the joint and marginal distributions to mcasure the dependency between two variables, making its direct optimization intractable in practice. For this reason, most practical methods use a varia- tional lower bound (VLB) together with an auxiliary function, usually discriminator that compares joint and marginal samples (Belghazi al., 2018; Hjelm et al., 2018). Many studies have improved these VLBs make estimations tighter and more stable, and using these bounds now a standard approach MI-based learning (Nguyen et al., 2010; Oord et al., 2018: Dorent al., 2025).

---

Published in Transactions on Machine Learning Research (02/2026)
However, these developments have been designed mainly for standard learning settings. Bayesian nonpara- metric learning (BNPT.) is now growing as a flexible way to model complex data (Fong et al., 2019; Lyddon et al., 2018; 2019; Dellaporta et al., 2022; Fazeli-Asl et al., 2024), with the potential to provide posterior inference under model misspecification; yet none of the existing VLB-MI estimators can be used directly in MI-based BNPL, procedures. BNPL. relies on posterior sampling induced by randomized objective functions together with posterior bootstrap algorithms, whereas current MI estimators do not account for the under- lying distribution being treated as a random probability measure. This gap leaves MI-based BNPL without practical or compatible lower bound estimator.
In this paper, we fill this gap by introducing a Bayesian bootstrap framework to estimate the VLB of MI. The core idea is to apply Bayesian bootstrap resampling for Mutual Information Neural Estimation (MINE) (Belghazi et al., 2018), replacing uniform sample weights with Sirichlet-resampled weights. Beyond providing this structure that allows VLB optimization to be used in MI-based BNPL, which is the main contribution of this work, we also show that this Bayesian bootstrap produces even a tighter VLB compared with the original MINE. We further compare our estimator numerically with several stale-of-the-art VLBSs, including NWJ (Nguyen et al., 2010), CPC/InfoNCE (Oord et al., 2018), and JSD-LB (Dorent et al., 2025), and show that it is a strong competitor that offers improved stability and lower variance. To illustrate the practical value of our approach, we incorporate the proposed estimator into learning problem already formulated within the BNPL. framework, namely a VAF-GAN architecture from Fazeli-Asl & Zhang (2023). This provides a natural context to assess how our estimator performs in MI-based optimization and to demonstrate its effectiveness on real data. These results may encourage researchers to consider their learning problems within a Bayesian context.
The rest of the paper is organized as follows. Section 2 reviews standard VLB methods for MI estimation. Section 3 presents our main contribution, the Bayesian bootstrap framework DPMINE, and outlines its basic theoretical properties. Section 4 reports experiments showing that DPMINE provides stable, low-variance MI estimates and can also improve training in a BNP VAE-GAN model when applicd within BNPL for data-synthesis purposes. Section 5 concludes with a brief discussion of future work. Additional proofs and experimental details appear in the Appendix.
2
Related Work
In this section, we briefly review standard VLBBs for estimating MT. We then give a short note on their common use in standard lata-generating tasks.
Mutual Information Estimation. Estimating MI in machine learning optimization problems is challeng- ing, especially with high dimensional data small mini-batches. Let (U, V) be pair of random variables with a joint cumulative distribution function (CDF) Fuv and marginal CDFs Fu and Fv, defined over the product space U @ V. The MI between U and V is given by:
MI(U, V) = DFU Fu & = H())=(()- ((UU)-((VV))
(1)
where (s denotes the KL divergence, and H()) and H(--.) represent marginal and conditional Shannon entropies, respectively.
Standard approaches to MI estimation either depend on parametric assumptions on the data distribution that rarely hold or rely on nonparametric mothods such as k-ncarest neighbors (Al-Labadi et al., 2022). Such nonparametris approaches become unstable as the dimension increases due to the curse of dimensionality (Sugiyama, 2012). These limitations have led to neural VLBs estimate MI, which scale better but can still struggle when the true MI is large when training relies small mini-batch sizes (Dorent al., 2025).
Standard Neural VLBs. The KL-MINE introduced by Belghazi et al. (2018) is one of the most widely used neural VLBs for MI estimation. For instance, Belghazi et al. (2018) introduced KL-MINE, which uses the Lonsker-Varadligg (DV) representation (Donsker & Varadhan, 1983) of the KL divergence to form a
lower bound of equation 1 as:
(U, V) = -Erov (U, V)] - In E En (VT4(
(2)
2

---

Published in Transactions on Machine Learning Research (02/2026)
where {Ta her be set of continuous functions parameterized by neural network compact domain F that maps U @ V to R and In(.) denotes the natural logarithm. In this framework, the auxiliary statistic T serves as the key component that learns to discriminate between samples from Fuv and those from Fu 0 lv.
The KL-MINE is then obtained by maximizing VLB given in equation 2 over 7y E T. In practice, equation 2 is estimated using the empirical cumulative distribution functions (FCDDs), FU,.. = E du, and Fv,.. = 1
n +==1vv ōv,:
" 1 (U.,,))))))) n 1 TU, V.) In Σ Σ t=1
TU VIn.
n
8=1 n
where {m(11 : n)) denotes random permutation of {1: n), standard technique to empirically approximate E (-) in equation 2 (Belghazi et al., 2018; Hjelm et al., 2018). However, MINE is prone to high variances (Liao et al., 2020).
Another variant of MINE is based on a variational lower bound for the Jensen-Shannon (JS) divergence (Nowozin et al., 2016) between Fuv and Fu @ Fv (Hjelm et al., 2018; Jones et al., 2023). It is defined as
L (U,( =
V) - SEsusvis((T,0 V))],
(4)
where <(-) = stable results, its connection to the KT-based definition of MI remains unclear (Dorent et al., 2025).
(+)) denotes the softplus function. Although JS-MINE yields lower variance and more
Dorent et al. (2025) addresses this concern by providing a new VLB on the KL divergence expressed as a function of the JS divergence, clarifying why maximizing the JS divergence is equivalent to maximizing lower bound on MI. Their VLB also overcomes well-known drawbacks of NWJ (Nguyen et al., 2010), which oflen has high variance, and CPC/InfoNCE (Oord et al., 2018), whose estimates are restricted by the batch size.
Alternative Neural Bounds. There are additional neural variational bounds for MI estimation, including VNMC and VPCE (Ivanova al., 2024), which provide upper and lower bounds, respectively. These methods rely on additional modeling assumptions, including the availability of at least one distribution in closed form, and estimate MI using nested Monte Carlo procedures based on Bayes' rule. Consequently, they are commonly used in modeling settings that differ from distribution-free neural VL. estimators such as MINE, NWJ, CPC, or JS- based bounds, which operate directly on samples without imposing analytic distributional forms. We therefore view VNMC and VPCE as complementary approaches that target MI estimation under different assumptions, rather than methods intended for the same training setting considered in this paper.
Two-Stage Estimators. Several other neural MI estimators in the literature use a two-stage design, where a separate model is trained to estimate MI before it is used in an optimization task, see for example, Letizia et al. (2024); Shalev et al. (2022); Liao al. (2020); Tsai et al. (2020); Song & Ermon (2019). These approaches can provide stable estimates, but they must be retrained whenever the joint distribution changes, which makes them impractical for learning settings where the data features change during training. These methods operate differently from single-stage estimators and cannot be used directly during training.
Applying Standard VLBs to Improve GANs. All the estimators discussed are often used within learning problems where MI guides the optimization. One example to examine their behavior within the GAN family, which provides a natural setting for incorporating MI estimation into a downstream task. Models in this family, including GANs and VAE-GANs (Goodfellow et al., 2014; Larsen et al., 2016), use generator to create samples, a discriminator to judge their quality, and in VAE-GANs data to latent codes. MI has been introduced into these models to better align latent variables with generated outputs. InfoGAN (Chen el al., 2016) encouraged part of the latent code to remain informative about the output by adding an MI term to the objective. Building on this idea, Belghazi et al. (2018) used MINE to increase the MI between data, and their encoded representations a BiGAN-stylc model (Donahue et al., 2016: Dumoulin et al., 2016), improving data coverage and reducing mode collapse. These examples show that MI objectives can help GAN variants learn richer representations and produce more diverse samples.
encoder that maps
3

---

Published in Transactions on Machine Learning Research (02/2026)
3 Methodology
In this section, we present the main contribution of this work, a Bayesian bootstrap resampling for MINE. We begin with a brief background on BNPL and then present the resulting bootstrap algorithms. Finally, we illustrate how this framework can be applied in a downstream task.
3.1 Dirichlet Process for the Modeling Data Distribution
The DP is an infinite generalization of the Dirichlet distribution that is considered on the sample space denoted as I, which possesses a o-algebra A comprising subsets E' (Ferguson, 1973). F follows DP with parameters (a, H) with the notation F (a, H), if for any measurable partition AI,..., Au of with & #22 2, the joint distribution of the vector (F(A)):... F(A))) follows Dirichlet distribution characterized by parameters (A1), probability one. The base measure H captures the prior knowledge regarding the data distribution, while sigmifies the strength or intensity of this knowledge.
Moreover, it is assumed that (A)) implies F(Ag) with
As a conjugate prior, the Dirichlet process yields a posterior that is also a DP. Specifically, for n indepen- dent and identically distributed draws (Xim E Rd) from the random probability measure F, the posterior distribution of
Pos = (F (1.) DP(a + n, H,).
(5)
The updated base measure is
H n
H; +n a+n
Fx
(6)
where Fx,.. denotes the empirical cumulative distribution function of the sample X
It can be shown that a Dirichlet process has an infinite series representation and is therefore a discrete probability measure. Although the blick-breaking representation is a commonly employed series represen- tation for DP inference (Sethuraman, 1994), lacks the necessary normalization terms convert into a probability measure (Zarepour Al-Labadi, 2012). Additionally, simulating from an infinite series is only feasible through using random truncation approach handle the terms within the series. To address these limitations, Ishwaran & Zarepour (2002) introduced an approximation of the DP in the form of a finite scries, which allows for convenient simulation. In the context of posterior inference, this approximation is given by
N
FPos
Pos 8xpOs
where (JEx) Dirichlet((a+n)/NN., (a+n)/NN, (xxx lid Ha and 6xp is the Dirac delta measure. In this study, the variables J. and X!*-= represent the DP's weight and location, respectively. The sequence (FPoR) N221 converges in distribution to PPos, where FP and FPss are random values in M1 (Rd), the space of probability measures on Rd endowed with the topology of weak convergence (Ishwaran & Zarepour, 2002).
Since the approximation in equation 7 is finite by construction, the remaining practical question concerns the choice of the truncation level N. Rather than fixing in advance, an adaptive truncation rule is employed to determine the number of retained atoms so that additional terms contribute negligibly to the total mass.
Following Zarepour & Al-Labadi (2012), let V, Gamma level as
atn 1 for i I,....J, and define the truncation
V Σ
N = inf j JoO
<6
V
for a prescribed threshold E (0, 1). This adaptive selection of N ensures that the finite approximation FPos captures the dominant posterior mass while remaining computationally tractable, and it is used throughout the remainder of the paper.
4

---

Published in Transactions on Machine Learning Research (02/2026)
3.2 BNPL: Posterior Bootstrap Framework
Let (X1:n) be IID random vectors in I with common distribution F E P(R4), where P(R4) denotes the space of Borel probability measures on R Suppose we are interested in parameter ab v C R2 indexing a collection of parametric models
F = [ 2/ E }.
(9)
for example, the distribution induced by neural network family parameterized by 1b.
Letting F' $ Fv, which allows for the possibility of model misspecification, the nonparametric learning (NPL) framework defines the parameter of interest as a functional 2/ P(RR) - v of the data distribution, given by
(FF arg min / L(x, 1b) dF)
(10)
where T(x,, v/) is user-chosen loss that determines the target statistic (Fong et al., 2019; Lyddon et al., 2018; 2019).
Since F unknown, it is modeled as a random probability measure with DP prior F DP(a, H). The core idea of BNPL that the posterior random measure PPos induces a posterior distribution on the parameter of interest through the functional & P(Rd) V given by
min |in / L(x, 2b) dx)
(Ps)) mig
(11)
Because the DP is discrete and we use the finite approximation in equation 7, the expression in equation 11 admits the practical approximation
Pos) xo arg min Σ J. Pos L(x, Pos ,b),
(12)
which corresponds to the Bayesian bootstrap procedure described Fong et al. (2019, Algorithm 2), with the difference that Fong et al. (2019) employs a truncation of the stick- boeckbbeaakig series representation to obtain this finite approximation.
Dellaporta et al. (2022) and Fazeli-Asl al. (2024) also independently introduced posterior bootstrap meth- ods based on two different DP approximations for learning the data distribution. In both works, learning is carried out by minimizing the maximun mean discrepancy between the posterior distribution pPos and the distribution induced by generator modeled as parameterized neural network.
Overall, these posterior bootstrap methods operate within a robust learning framework and are designed to handle model misspecification.
3.3 DPMINE: Posterior Bootstrap Framework for MINE
We now present our main contribution by exlending the Bayesian bootstrap idea of Fong et al. (2019) to the MINE. Although our setting differs from Fong al. (2019), the connection direct: the posterior on the MINE parameters is induced by placing DP posterior on the unknown data distribution. This construction makes the original MINE applicable for MI-based BNPL. We first develop this approach for the KL-MINE objective and then show that the same steps apply to the JS-MINE. case.
KL-based representation: Let (Xim) be n IID random variables with X E Rd F. For any continuous mappings fe: Rdd + Rdd i 1,2, we are interested in estiniging the MT between (x(X and f2(X) using the KL-MINE objective from the BNPL perspective. In this sctting, we define the parameter of interest as the functional t P(Rd) r, given by
(F) arg min (- (())))
(13)
7Er
5

---

Published in Transactions on Machine Learning Research (02/2026)
Algorithm 1: MINE Posterior Bootstrap
Input: training data X1m, mappings fi. f2, concentration parameter a, base measure H, truncation
level N, number of bootstrap samples B, critic network Ty
for i i= to B do
Select the i-th mini-batch { of size Rmbi
ubb Rmn
a Define H Bub a+himb
+
F
a+Nimb
4 Draw posterior samples Pos
üdd
I:N
Draw Dirichlet weights (JiN) Compute ((,-) using equation 15 (DPKL) equation 17 (DPJS) with the critic Ty, posterior
+ n)/N);
samples X and Dirichlet weights J
Update the critic parameter
800) = arg min
{ 7Er L. (fi(X fe fx(X
Compute the MI cstimate
MTDP(1) I (fi L:N): f2 f2(x).)))
return Posterior bootstrap sample MIP((1
Given posterior draw Pos DP(a n, HH); the posterior for +(F) is obtained through the functional y P(R)) + r, defined by
(PPPoN = arg min
L DPKL ( ((x))
Pos).
(14)
TET
where
N
L LOKKL (fi (XPP), f2 (:N))
1 PoST (fi(X (,(xX In Σ 71100x000000000 (15)
Finally, equation 16 represents the posterior belief about MIKL
MIDP (F f2(XPon)) = DPKL (f (fi(X fz( Px(
(16)
Algorithm 1 provides clear illustration of these steps.
JS-based representation: All steps used in the Bayesian bootstrap construction for KL-MINE extend simply to the JS-MINE objective. The DP-based analogue of the JS variational bound is given by
L LDPJS (f Pos f2
-
-(I,((((X))))) f2
(17)
The following theorem states that the proposed Bayesian VLB estimator asymptotically no smaller, on average, than the standard VLB used in MINE. This results tighter VLB on the true MI and allows for more accurate estimation of MT by maximizing the tighter bound
compact domain r.
Theorem (Limiting expectation). Considering DP posterior representations defined in equation 15 and equation 17. Let both Ty and eTh are M-bounded (i.e., [Ty]. TeT S M). Given the DP posterior approzi- mation in equation 7, for any N > 0, we have,
lim Epg
LOPKL (P ((X LKL
n 000

---

Published in Transactions on Machine Learning Research (02/2026)
it. lim Epg 10 100000
(L LPPJS (f
Pot f2(X (xD)) 2 L (f1())
In Theorem 1, the expectation is taken with respect to the nonparametric posterior the data-generating process. As result, the randomness in 9, which arises from the posterior bootstrap, is automatically averaged out. Equivalently, f is treated as fixed conditional on posterior draw, and the theorem charac- terises the resulting average behaviour. This treatment is standard in Bayesian nonparametric learning and postcrior bootstrap analyses (Dellaporta et al., 2022).
3.4 Application: Integrating DPMINE into VAE-GAN Models within BNPL
The BNPWMMD-GAN framework proposed by Fazeli-Asl & Zhang (2023) leverages representation learning within BNPL by placing a DP prior F DP H) to generate synthetic samples.
The model follows standard VAE-GAN architecture, consisting of an encoder En R + R2, 7 E H. that maps data into latent representation, generator (dccoder) G R - Rd and generates samples from latent codes, code generator CGw R + RP, w E St, with g < p that injects additional latent noise to improve coverage of underrepresented regions of the latent space, and a discriminator De R + R. 0 E θ that distinguishes real from generated samples. These components are trained jointly using Wasserstein and MMD-based losses evaluated on cesterior-weightee samples generated by DP approximation equation 7. Further details are provided Appendix C.
St that reconstructs
Training is implemented by estimating the network parameters as functionals of PPss Let
E = S H O x S
denote the joint parameter space. The baseline BNPWMMD-GAN objective is defined as
(a, n. 0, (LF
arg
min (wm.0.W
[L(G,, En, F + L(Da, F + L(CGW, }
(18)
To improve information extraction during encoding and preserve meaningful structure during decoding, this objective refined by two regularization terms estimated via the proposed DPMINE framework, yielding
(ws O..n 33) (FPP)) = arg min (E)
L(Gw: E,, FR + L(Do. FRee) + L(CGw',
Poe E,, (x - -LDI (xPoe
(19)
L (En Pos Gw (X,(()))))) }.
(20)
The regularization terms in equation 19 and equation 20, computed using the corresponding crities Top and Tap: estimate MI(X, (X)) and (X)E,,(X). Gw (E), respectively. The associated critic parameters are estimated as posterior functionals,
(F arg min (-I mer,
DP
Pos Pos I:N: E,, (X }
X
(21)
52 (1 arg min L-L (E,, (XPo)) G (E,((()))) meT
DP
(22)
Overall, this formulation integrates DPMINE terms into a VAE-GAN training procedure, while maintaining the BNPL interpretation.
Using the notation of Section 3.3, the term in equation 21 corresponds to
fi X Pae
Pos X
f2 (xPx)) E,, (x
(23)
Similarly, the term in equation 22 corresponds to
h (x = F,, [x (1x))
(x (IN)
Gw (F., (X
(24)
when Gu acts as the decoder.

---

Published in Transactions on Machine Learning Research (02/2026)
4 Experimental Results
In this section, the procedure is evaluated on both simulated and real datasets. All experiments are conducted using mini-balches, with the batch size chosen appropriately for each setting. To place a DP prior on the data distribution F, its hyperparameters must first be specified. Unless an experiment is designed to study their specific influence, the base measure taken to be a multivariate normal distribution estimated from each mini-batele For a mini-batch x(1)
mb. i {l...., B), the sample mean and covariance are
B
Xmb
x
( mb
Sxmb
x _x mb
Xmb) xmb))
(25)
B
1=1
1=1
and the base measure defined as
= N(=b(Xbb> Sxmb).
The concentration paramcter is then selected as the marimum posteriori (MAP) estimate of obtained by maximizing the log-likelioood of /Pas with respect to a. Full details of this computation are provided in Fazeli-Asl & Zhang (2023, Section 4.2). Additionally, the value N in the DP approximation of equation 7 is chosen as prescribed in Section 3.1, using = 10-4
We start by presenting simulation studies that evaluate the KT-based and JS-based versions of our Bayesian bootstrap framework for the MI estimation. We then show how the better- peeter-performing estimator can be used to refine the training of VAE-GANs within the BNPL framework.
4.1 MI Estimation using Bayesian Bootstrap Resampling
We follow the staircase protocol of Letizia et al. (2024), adapted experimental sctting, to construct the loy examples. In our design, we let.
X = x=UU, V) E 2k
U VER
and consider estimating MI(U, V) for k E {1,2,3, 4,5. 5}. Increasing produces the successive staircase levels: the first stage uses = 1, the sccond uses k: = 2, and this continues until the fifth stage uses k = This yields five distinct increases in the true MI, giving clear benchmark for comparing estimator performance more dimensions added.
Binsry-Gaussian Example. For each value of kt we generate binary-Gaussian example by defining
U = sign(()) (Z),
Z Io), U=U+ 6~ N(0, 0.2
In the notation of Section 3.3, where the MI was writlen as (x) this setting corresponds to the choice fi fi(X U and f2(X) V.
Finally, cach stairstep provides 4000 training steps, where step means gradient update of the neural isttittion tasks. At every step, we draw mini-batch from the distribution associated with the current value of k. Additional toy examples following the same protocol are provided in the appendix.
4.1.1 DPMINE versus MINE
As the main contribution of this paper is to apply Bayesian bootstrap resampling to the mini-batches used in MINE, we begin by examining whether Bayesian version outperform the original estimator. Figure I shows our results, where we use mini-batch size of 16. The solid curves display the running averages of each estimator; for BNP-MINE, this running average corresponds to the posterior expectation induced by the Bayesian bootstrap and represents the expected behavior of the MI posterior, which converges to the true MI in the KT.-based setting. The shaded regions show the raw mini-batch variability. In the KL sotting, our BNP-MINE estimator gives tighter and more stable lower bound compared to FNP-MINE, which numerically supports the theoretical results in Theorem 1(i). The FNP curves vary widely and rise
8

---

Published in Transactions on Machine Learning Research (02/2026)
above the true MI, reflecting the instability often seen in MINE training. In contrast, the BNP-KL curve has less variance and remains close to the true MT, leading to more consistent. estimate.
The BNP-JS also shows reduced variance and avoids the large jumps observed in the FNP-JS, even though it not as tight as the KL version. Overall, the BNP framework leads to more stable and reliable MI estimates, with the KT. formulation giving the strongest improvement.
BNP-MINE VS FNP-MINE (KL)
BNP-MINE FNP-MINE (JS)
BNP-MINE Variants (KL JS)
3
21
2
l
10000
15000
10000
20000
Steps
Steps
Steps
True MI
MINE (KL)
MINE (JS)
Ours (KL)
Ours (JS)
Figure 1:
Staircase scheme for the binary-Gaussin setting with X = V) ~ F. BNP-MINE and FNP- MINE (KL and JS) are compared using mini-batch size 16, with F ~ DP(AMAP, H) and = N (Xub, Sxm.). across dimensions k .....
4.1.2 Sensitivity to different mini-batch sizes
To examine how different mini-bate bati- sizes affect the estimator, Figure 2 shows BNP-MINE under several choices of batch size. Across all settings, the estimates remain close to the true MI, and larger batches tend to produce smoother and slightly more accurate curves. Overall, the method remains stable, and the effect of the batch size is modest.
Mini-batch size 16
Mini-batch size = 32
Mini-batch size 64
3
1-
0
5000
10000
15000
20000
5000
10000
15000
20000
5000
10000
15000
20000
Steps
Steps
Steps
True MI
BNP-MINE (KL)
BNP-MINE (JS)
Figure 2: Staircase scheme for the sinary-Gaussial sctting with X= (U, ~ F. BNP-MINE evaluated
across different mini-batch sizes, with F DP(AMAP, H) and H = N(Xmb; Sxma); across dimensions
k 1. 5. 5.
4.1.3 Sensitivity to different choices of DP hyperparameters
To examine how the DP hyperparameters affect our procedure, we study a range of concentration values E [0.005, 250, 2500} as well as CMAP, under two choices of the centering measure H. In both sets of experiments, dMAP produces stable estimates in the sense that the MI remains close to the true value.
9

---

Published in Transactions on Machine Learning Research (02/2026)
Figure 3(a) shows that when H is taken to be standard normal distribution, increasing forces the prior to dominate the data, and the estimator gradually converges to the MI of a standard normal model, which is zero, In contrast, Figure 3(b) shows that when H is defined using the sample mean and sample covariance of cach mini-batch, the prior remains aligned with the data, and the estimator stays accurate even for large a. These results, which follow directly from the posterior base measure in equation 6, show that reasonable choices of keep the method reliable even for large a. Although strongly misspecified choices of H can push the estimates toward the wrong target, the method remains robust the choice of H when is sufficiently small.
3MAP
a= 0.005
a 25
=5250
a =5200
U003
IERE
144
2050
653
IEKE
140:0
2003
1993
IBKE
1590
2033
5003
JAKE
35030
10
2330
15030
25093
True
BNP-MINE (KL)
BNP-MINE (JS)
a : T2x)
0.005
25
250
2500
Steps
Stept
Stete
Stace
Stepe
True MI
BNP-MINE (KL)
BNP-MINE (JS)
(b) H N (Xmb: Sx..))
Staircase schome for the binary-Gaussian Gaussian setting with (U, BNP-MINE evaluated using a mini-batch size of 64, with F Da H) under different choices of the concentration parameter a and the centering measure H, across dimensions k = 1....,5.
Figure 3:
4.1.4 Comparison with further differentiable MI-VLB estimators
In addition to the traditional MINE estimator (Belghazi et al., 2018), compare method against several influential differentiable MI-VLB approaches. NWJ (Nguyen et al., 2010) is unbiased but lypically yields loose and high-variance lower bound. CPC/InfoNCE (Oord et al., 2018) offers stable optimization but is fundamentally constrained by In(rmb), restricting its ability to capture high MI. We also include JSD- LB (Dorent et al., 2025), the most recent state-of-the-arr estimator providing a tight and tractable lower bound connecting JS divergence reliable lower- bound estimate than NWJ and JSD-based methods, and although CPC appears smoother, its structural upper bound prevents consistent performance across settings. Additional results in the appendix demonstrate that CPC becomes non- non-tight in several benchmarks, while our estimator consistently preserves the staircase structure with substantially reduced variance.
MT. As our experiments show, BNP-MINE delivers a more stable and
Sensitivity to high dimensionality. To examine how well our method handles high-dimensional data, we test it with = 10, 100, 1000 as shown in Figure 5. all cases, DPMINE produces smooth and stable curves with much lower noise compared the bascline estimators. As the dimension increases, MINE and NWJ become noticeably more unstable, while DPMINE continues to give a steady estimate of the MI lower bound. These results show that our estimator remains reliable even in very high dimensional settings.
10

---

Published in Transactions on Machine Learning Research (02/2026)
BNP-MINE vs NW)
BNP-MINE Vs JSD-LB
BNP-MINE VS CPC/InFONCE
3
M1
2-
1
0
5000
10000
15000
20000
5000
10000
15000
20000
5000
10000
15000
20000
Steps
Steps
Steps
True MI
NWJ
JSD-LB
CPC/InfONCE
Ours (KL)
Figure 4: Staircase scheme for the binary-Gaussian Gaussian setting with X (U, V) F. BNP-MINE is compared against different baselines using a fixed mini-batch size of 64, with F DP(AMAr, and H [(mmh, Sxm.), across dimensions = 5.
Table 1: Runtime of neural variational lower-bound estimators across data dimensions and training epochs.
Dimension (d 2 x k)
Estimator
Epoch
2
20
200
2000
MINE, NWJ,
500
5 sec
5 sec
5 sec
5 sec
CPC/InfONCE, JSD-LB
1500
12 sec
12 sec
12 sec
12 sec
500
42 see
42 sec
42 sec
42 sec
DPMINE
1500
min 25 sec
min 25 sec
min 25 sec
min 25 sec
4.2 Improving VAE-GAN training using DPMINE within BNPL
To implement the BNPWMMD+DDMING we consider the Gaussian kernel function, defined as exp(=IX-XII"
kc, Y) exp(= 2 with bandwidth parameter o, in the MMD distance given in Appendix B. We scarch for the appropriate bandwidth parameter a over fixed grid of values, We then compute the mixture of Gaussian kernels, denoted as k(.,,) = E. kG, (..). This selection of kernel function and bandwidth has been shown to yield satisfactory performance in training MMD-bbaed GANs, as mentioned Li et al. (2015); Fazeli- Asl et al. (2024); Fazeli-Asl Zhang (2023). Then, we consider next examples to investigate the proposed approach.
{2, 5, 10, 20, 40, 80).
4.2.1 Toy Example
Coil Dataset: We first look at toy example that showcases the effectiveness of the BN- PWMMD+DPMINE in mitigating mode collapse. In this example, we simulate 5000 samples in 3D space, denoted as (X(t), ((t), Z(t)), where X(t) = 6cost, t, Y(t) = 6sin t, and Z(t) t, with t ranging from -2 to 4r. We then normalize all datasets to a range between -1 and 1. This normalization ensures compatibility with the hyperbolic tangent activation function, which used in the generator's last layer all compared models. We also used latent dimension of 100 with sub-latent dimension of 10 in this example. Addition- ally, provide the results of the BNPWMMD to display the basic model's performance in covering data space in the absence of DPMINE.
Figure 6 illustrates the elfect of incorporating DPMINE into the BNPWMMD-GAN framework. Compared to BNPWMMD alone, geometric structure of the real data, with reduced dispersion and improved alignment along the underlying manifold. This improvement is particularly evident in the random generation results, where DP-MINE helps preserve global structure and continuity.
produces generated samples that more closely follow the
11

---

Published in Transactions on Machine Learning Research (02/2026)
DPMINE Vs MINE
DPMINE VS NW]
DPMINE vs JSD-LB
DPMINE VS CPC/InfONCE
MI
1000
2000
3000
4020
1,00
2003 Steps
30000
6000
1000
2000
3:00
40.00
1000
2005 Steps
3020
4030
Steps
Steps
NWJ
JSD-LB
MINE (KL)
CPC/InfoNCE
Ours (KL)
(a) = (d 20)
DPMINE MINE
DPMINE vs NWJ
DPMINE JSD-LB
DPMINE CPC/InfONCE
30
MI
0-
Steps
NWJ
JSD-LB
MINE (KL)
CPC/InfoNCE
Ours (KL)
(b) k 100 (d 200)
DPMINE VS MINE
DPMINE vs NWJ
DPMINE vs JSD-LB
DPMINE vs CPC/InfONCE
MI
1000
2000
3203
4020
1000
2003
3000
4000
1000
2000 Steps
3:000
4020
10:00
2200
3000
4002
Steps
Stops
Stops
NWJ
JSD-LB
MINE (KL)
CPC/InfoNCE
Ours (KL)
(c) 1000 (d 2000)
Figure Binary-Gan Gaussian setting with X (U. ~ F. BNP-MINE uses mini-batch size 64, with F. (P(MMAP, H) and H = N(Xmb, Sx...). Experiments are run for 10, 100, 1000, using 4000 steps
per dimension.
In the reconstruction task, the inclusion of DPMINE leads to more faithful reconstructions with smoother trajectories and fewer distortions, indicating improved information retention between the data and latent spaces. Overall, these results show that DP MINE enhances representation quality, stabilizes training, and reduces mode collapse.
4.2.2 Real Example
This dataset comprises of 3D chest CT images of 1000 subjects diagnosed with lung infections after testing positive for COVID-191 All images in this dataset are stored in the DICOM format and have resolution of 16 bits per pixel, with dimensions of 512 512 pixels in grayscale. We randomly selocted 200 paticnts, resulting in total of 91,960 images. As part of the preproccssing step, we first stored each patient's data in Neuroimaging Informatics Technology Initiative (NIFTI) format file. Then, we
COVID-19 Datasct:
TTe dataset is freely available online https://doi.org/10..r htt../rrrrg..o...ooo//////a//aa (license: CCO 1.0).
12

---

Published in Transactions on Machine Learning Research (02/2026)
BNPWMMD +DPMINE
BNPWMMD
Random
Generated Samples
Real Dataset
BNPWMMD +DPMINE
BNPWMMD
Reconstruction
Figure 1000 randomly generated and reconstructed samples using SWWWWWMDD--PMMIINN compared
with BNPWMMD, after 5000 epochs for the coil example.
converted each NIFTI file into 3D image with a dimension of 64 64 x 64. Each dimension represents the axial, sagittal, and coronal views of the lungs. After normalizing the dataset, we used a mini-batch size of 16 and trained all compared models for 7500 epochs. We also used latent dimension of p 1000 suggested in Kwon et al. (2019) for high-dimensional cases, with sub-latent dimension of 100 as used in Fazeli-Asl & Zhang (2023).
To illustrate the image generation performance ENPWMMD-DPMINE in comparison with BNPWMMD alone, Figure presents a 3D visualization of real and randomly generated chest CT samples. The results indicate that ENPWMMD+DDMINE produces sharper and more coherent images. For a more compre- hensive view of the generated samples, included additional slices from each dimension representation in Appendix E.2.2. We also compared the reconstruction capability of different models by displaying slices of a reconstructed sample from each model in the Appendix. The ENPWMMD-DPMINE yields reconstructions that are visually closer to the training data under BNPWAMD+DPMINE.
Axial
Axial
Axial
Sagittal
Coronal
Sagittal
Coronal
Sagittal
Coronal
(a) Real Dataset
(b) BNPWMMD DPMINE
(c) BNPWMMD
Figure 7: A 3D visualization of a real sample and a randomly generated sample using BN- PWMMD+DPMINE after 7500 epochs, compared with BNPWMMD, in the COVID-19 example.
13

---

Published in Transactions on Machine Learning Research (02/2026)
However, additional quantitative tools are essential for assessing the similarity between real and generated samples. Dimensionality reduction provides a useful way to visualize how well generated samples align with the real data distribution.
In Figure 8(a), the t-SNE embedding (Van der Maaten & Hinton, 2008) of the real data and 200 randomly generated samples shows that BNPWWMD++DPMING achieves broader and more consistent coverage of the real data manifold compared to ENPWMMD alone. In contrast, samples generated by BNP WMMD appear more concentrated and exhibit gaps relative to the real data distribution.
Figure 8(b) provides a complementary view using features obtained from a custom two-layer linear encoder. This representation supports the same conclusion: samples from BNPWMMD+DPMINE overlap more closely with real samples and better capture the underlying structure, indicating improved representation quality and reduced mode collapse.
To quantify dissimilarity between real and generated samples in the two-dimensional feature space, we consider the real features f, == 1 Feat :re2 and the generated features fg == (Featurel 2.000 We report the mean Fréchet Inception Distance (FID) and Kernel In- ception Distance (KID) over 100 replications, together with variability mcasured as + 20, using both the t-SNE embedding and a custom encoder representation.
In addition, Table 2 reports empirical MMD values computed in the same two-dimensional feature space, using the same kernel and bandwidth parameters as those employed in the BNPWMMD training objective. We intentionally avoid reporting MMD in the original data space, since BNPWMMD -aaeededddddddd models explicitly minimize data-space MMD during training, which would render such comparisons unfair.
Across all feature represenlations and metrics, BNPWMMD+DD MINE consistently attains lower mean FID, KID, and MMD values than ENPWMMD alone. While the absolute differences are moderate and the to intervals indicate non-negligible variability, the improvements are systematic across both t-SNE and custom-encoder embeddings. These trends are consistent with the visualizations in Figure 8 and suggest improved alignment between generated and real feature distributions, logether with reduced mode collapse.
We further assess perceptual image quality using the Multi-Scale Structural Similarity Index (MS-SSIM) (Wang et al., 2003). MS-SSIM is reported as mcan + 20 over 100 replications, with higher values indicating better structural fidelity. Although the compared models are unconditional generators, the use of MS- SSIM is reasonable due to the strong anatomical alignment of axial, sagittal, and coronal lung CT slices across paticnts. The higher MS-SSIM score achieved by BNPWMMD+DP-MIIE MINE indicates improved visual consistency relative to BNPWMMD.
Additional implementation details are provided in Appendix E.3. Appendix E.2. further extends the evalu- ation to brain MRI dataset, demonstrating that the observed improvements generalize across datasets and imaging modalities.
Table 2: Comparison of statistical scores (mean + 2o) for the COVID-19 example over 100 replications.
Evaluator
BNPWMMD + DPMINE
ENPWMMD 0.00084 = 0.00535
FID
Custom Encoder
0.00063 + 0.00468
t-SNE
9.48447 + 7.05185
11.10431 10.8224
KID
Custom Encoder
0.00069 1 0.00073
0.00361 = 0.00102
t-SNE
0.99858 + 4. 4.16996
1.011342 + 6.66391
MMD
Custom Encoder
0.00902 1 0.00236
0.01172 0.00703
t-SNE
0.00074 + 0.00130
0.00358 = 0.00782
MS-SSIM
0.43465 + 0.02189
0.41781 = 0.06755
5 Concluding Remarks
In this work, Bayesian bootstrap resampling framework was introduced for mutual information estima- tion via MINE (DPMINE), enabling variational lower-bound optimization within Bayesian nonparametric
14

---

Published in Transactions on Machine Learning Research (02/2026)
30-
Real BNPWMMD+DDMINE BNPWMMD
0.05
Real
#NPWMMD+DPMINE BNPWMMD
20
0.00
10-
0.05
2
0-
-0.10
Feature
-0.15
10
0.20
-00
0025
-30-
-0.30-
-0.35
-30
-20
-10
0
10
20
0.4
0.3
0.2
0.1
0.0
Feature
Feature
(a) t-SNE
(b) Custom encoder
Figure Scatter plots of two-dimennsonall feature representations the COVID-19 example, comparing samples generated using BNPWMMD+DDMMNG with those generated using BNPWMMD. Panel (a) shows the t-SNE embedding of the features, while panel (b) displays the features obtained from custom encoder.
learning (BNPL). By replacing uniform weights with Dirichlet-resampled weights, the proposed approach naturally accounts for uncertainty in the underlying data distribution, a core feature of BNPL.
The resulting estimator, DPMINE, yields tighter and lower-variance MI estimates than standard MINE and performs competitively with existing variational bounds. Empirical results show stable behavior across batch sizes and high-dimensional settings. When incorporated into BNP VAF-GAN model, DPMINE improves training stability and generative quality, demonstrating its practical value.
Overall, this work bridges an important gap between MI-based learning and BNPL and enables broader use of information-theoretie objectives in Bayesian nonparametric frameworks. While the focus here is on generative modeling, the proposed framework is applicable to other learning problems, including representation learning and Bayesian decision-making.
Extending DPMINE to non-IID sottings and distributed or privacy-sensitive learning sccnarios remains an important direction for future work. In particular, incorporating DPMINE into federated learning frame- works could strengthen privacy-preserving training by providing more robust and stable control of infor- mation flow across decentralized and heterogeneous data sources. However, the current formulation relies on an IID assumption underlying the Bayesian bootstrap. Adapting DPMINE to fodcrated settings will therefore require new methodological developments to handle client-level heterogeneity, non-IID sampling, and communication constraints while preserving theoretical guarantees.
Acknowledgments
Bei Jiang and Linglong Kong were partially supported by grants from the Canada CIFAR AI Chairs program, the Alberta Machine Intelligence Institute (AMII), and Natural Sciences and Engineering Council of Canada (NSERC), and Linglong Kong was also partially supported by grants from the Canada Research Chair program from NSERC. Michael Zhang was supported by the University of Hong Kong Seed Fund for PT Research #2402101367. The authors would also like to thank the anonymous reviewers for their constructive comments that improved the quality of this article.
15

---

Published in Transactions on Machine Learning Research (02/2026)
References
Luai Al-Labadi, Forough Fazeli Asl, and Zahra Saberi. A test for independence via Bayesian nonparametric estimation of mutual information. Canadian Journal of Statistics, 50(3):1047 1070, 2022.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information ncural estimation. In International Conference on Machine Learn- ing, pp. 531 540. PMLR, 2018.
Mikolaj Binkkowkii Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Iya Sutskever, and Picter Abbeel. InfoGAN: In- terpretable representation learning by information maximixing generative adversarial nets. Advances in Newral Information Processing Systems, 29, 2016.
Charita Dellaporta, Jeremias Knoblauch, Theodoros Damoulas, and François-Xavil Xavier Briol. Robust Bayesian inference for simulator-based models via the MMD posterior bootstrap. In International Conference on Artificial Intelligence and Slatistics, pp. 943 970. PMLR, 2022.
Jeff Donahue, Philipp Krähenbühh, and Trevor Darrell. arXiv:160.009782, 2016.
Adversarial feature learning. ar preprint
Monroc D Donsker and SR Srinivasa Varadhan. Asymptotic evaluation of cortain Markov process expecta- tions for large time. IV. Communications Pure and Applied Mathemalics, 36(2):183 212, 1983.
Reuben Dorent, Polina Golland, and William M Wells lii. Connecting jensen-shannon and kellback-leibler divergences: A new bound for representation learning. In NeurIPS 2025, 2025.
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, and Aaron Courville. Adversarially learned inference. arXiv preprinl arXiv:1606. 00704,070444 2016.
Forough Fazeli-Asl and Michael Minyi Zhang. A Bayesian nonpaaramctric approach to generative models: Integrating variational autoencoder and generative adversarial networks using Wasserstein and maximum mean discrepancy. arXiv preprint ar Xiv:2030 14048, 2023.
Forough Fazeli-Asl, Michael Minyi Zhang, and Lizhen Lin. A semi-bayesian nonparametric estimator of the maximum mean discrepancy measure: Applications in goodness-of-fit testing and generative adversarial networks. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreviers het.:eenttt/tooorrmddddllllaaa
Thomas Ferguson. A Bayesian analysis of some nonparametric problems. Annals of Statistics, 1(2): 209 209-230,11 1973.
Edwin Fong. Simon Lyddon, and Chris Holmes. Scalable nonparametric sampling from multimodal posteriors with the posterior bootstrap. In Internalional Conference on Machine Learning, pp. 1952- 1962. PMLR, 2019.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 27:2672 2680, 2014.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. Journal of Machine Learning Research, 13(1):723 -73,7 2012.
Ishaan Gulrajani, Faruk Ahmod, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of Wasserstein GANs. Advances in Newral Information Processing Systems, 30, 2017.
R Devon Hjelm, Alex Fedorov, Samuel Lavoic-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arviv preprint or Xie: 1808. 068.06670, 2018.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Hemant Ishwaran and Mahmoud Zarepour. Exact and approximate sum representations for the Dirichlet process. Canadian Journal of Statistics, 30(2):269-28) 2002.
Desi R Ivanova, Marvin Schmitt, and Stefan T Radev. Data-efficient variational mutual information esti- mation via bayesian self-consistency. In Proc. Workshop on Bayesian Decision-Making and Uncertainty. URl: https://openrevieet net/forum, 2024.
Daniel C Jones, Patrick Danaher, Youngmi Kim, Joseph M Beechem, Raphael Gottardo, and Evan W Newell. An information thcoretic approach to detecting spatially varying genes. Cell Reports Methods, 3(6), 2023.
Gihyun Kwon, Chihye Han, and Dac-Shik Kim. Generation of 3D brain MRI using auto-cncoding generative adversarial networks. In Medical Image Compuling and Computer Assisted Intervention MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part III 22, pp.
118 128.126.2000 Springer, 2019.
Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using learned similarity metric. In International Conference on Machine Learning, pp- 1558 1566. PMLR, 2016.
Nunzio Alexandro Letizia, Nicola Novello, and Andrea M Tonello. Mutual information estimation via f-divergence and data derangements. Advonces in Newrol Information Processing Systems, 37:105114 105150, 2024.
Yujia Li, Kevin Swersky, and Rich Zemel. Generative moment matching networks. In International Confer-
ence on Machine Learning, pp. 1718-1721 PMTR, 2015.
Ruizhi Liao, Daniel Moyer, Polina Golland, and William M Wells. Demi: Discriminative estimator of mutual information. arXiv preprint arXiv:201..00 766, 2020.
Simon Lyddon, Stephen Walker, and Chris C Holmes. Nonparametric learning from Bayesian models with
randomized objective functions. Advances in Neural Information Processing Systems, 31, 2018.
Simon P Lyddon, CC Holmes, and SG Walker. General Bayesian updating and the foss-likelihood bootstrap. Biometrika, 106(2):465-44) 2019.
XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11): 5847 5861, 2010.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Processing Systems, 29, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 2Xxt:1807.03745 2018.
Jayaram Sethuraman. A constructive definition of Dirichlet priors. Statistica Sinica, pp. 639-650, 1994.
Yuval Shalev, Amichai Painsky, and Irad Ben-Gal. Neural joint entropy cstimation. JEEE Transactions on Neural Networks and Learning Systems, 3544)55488 5500, 2022.
Jiaming Song and Stefano Ermon. Understanding the limitations of variational mutual information estima- tors. arXiv preprinl arXiv:191.002222 2019.
Masashi Sugiyama. Machine learning with squared-loss mutual information. Entropy, 15(1):80 112, 2012.
Yao-Hung Hubert Tsai, Han Zhao, Makoto Yamada, Louis-Philippe Morency, and Russ R Salakhutdinov. Neural methods for point-wise dependency estimation. Advances in Neural Information Processing Sys- tems, 33:62 -2,277,221 2020.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(11), 2008.
17

---

Published in Transactions on Machine Learning Research (02/2026)
C Villani. Optimal transport, old and new. Grundlehren der mathematischen Wissenschaften, 3, 2008.
Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality assess- ment. The Chrity-Seventh Asilomar Conference on Signals, Systems Computers, 2003, volume 2, pp. 1398 1402. leee, 2003.
Mahmoud Zarepour and Luai Al-Labadi. On rapid simulation of the Dirichlet process. Statistics 8 Probabilily Lellers, (2(5))916 924, 2012.
Appendix
A Notations
F: Real data distribution.
(Xim): A sample of n independent and identically distributed random variables generated from F.
DP + n, H:): The Dirichlet process (DP) posterior with concentration parameter a a n, a > and base measure Нa
H A mixture probability measure given by a H + atn Fx
H: The base measure of the DP prior H),
Fx The empirical probability measure based on the sample data defined by 1 n 8x. n
FNoo The probability measure of the DP posterior approximation defined by Σ i=l 8xpo with (XPx) Hą and (JPx) ~ Dirichlet n)/N,.... ...,, (a + n)/N) Ishwaran Zarepour (2002).
f(X): A continuous function of random variable X ~ F for i 1.2.
(Tr)eer: A set of contimuous functions parameterized by neural network on a compact domain r.
Ta: Identity matrix of size d x d.
"as..: Standing for "almost surely" and indicates that the statements hold with probability 1.
B Definition of MMD and Wasserstein Distances: DP and Empirical Representations
Selecting appropriate statistical distance is crucial for effective generative model training. Here, we focus on the DP representation of two popular distances used in BNP deep lcarning and we will briefly mention their frequentist counterparts.
B.1 Maximum Mean Discrepancy Distance Treature-Matchine Comparison)
The MMD distance was initially introduced in Gretton et al. (2012) for frequentist two-sample comparisons. Recently, a DP-based version of this distance has been proposed for BNP hypothesis testing Fazeli-Asl el al. (2024). Consider a set functions [Gw parameterized by a neural network that can generate n IID random variables I.)); where the likelihood function intractable and not accessible. Let k(.,)) be a continuous kernel function with feature space corresponding to universal reproducing kernel Hilbert space Gretton al. (2012) defined on compact sample space X Gretton al. (2012). Given sample
18

---

Published in Transactions on Machine Learning Research (02/2026)
(X1m) lid F, the MMD distance between Poss and FY... is approximated as:
M (FF
Fy...) Σ
1f 1: ((X , X.
6,1=1
"
Pos Y.)
Σ t,1=1
k(Y Y.). (26)
n
n
In the frequentist version of the MMD distance, as defined in Gretton et al. (2012), the empirical distribution Fx.. is considered. This is denoted as MMM2((Fx.... FY..,, which is obtained by replacing N, JEN and X EN with n, 1/n, and Xi:h, respectively, in equation 26.
B.2 Wasserstein Distance (Overall Distribution Comparison)
The frequentist version of the Wasserstein distance completely discussed in (Villani, 2008, Part 16). Fazeli- Asl et al. Fazeli-Ast & Zhang (2023) proposed BNP version of this distance through its Kantorovich- Rubinstein dual representation. Let {Do Joee be parametrized family of continuous functions that all are 1- I-Lipschitz. Then the Wasserstein distance between pPos and Fy... is approximated as:
N Σ
J. J! Do PUr) Do(Y.))
WS(F FYI..) = max 9
(27)
By utilizing modifications in equation 27 similar to those described for the MMD distance, the empirical representation of the Wasserstein distance can also be obtained. In this section, propose two novel representations for the MINE using the DP. These representations are based on the KL and JS divergences will be used in our BNP learning framework to maximize information during the training process.
C BNPWMMD-GAN
The BNPWMMD-GAN architecture introduced by Fazeli-Asl & Zhang (2023) consists of four networks: encoder {En a generator [Gwlwen. code generator {CGw hweerr, and a discriminator {De)ece.
Given the codes (CI:N == E,, (xPx) 1:N and generated codes (Ōı:N == CG (E'r.)) along with the latent noise (ELNN E IP) ~ Fe and sub-latent noise (Ei.N E R2) ~ Fe', where g < p, the BNPWMMD-GAN trained by updating the worksrks parameters according to the following hybrid objective function?:
(wo (O = arg min A. 7
+ Do(Gw Do + MMD(F Pos FFa(enn)) D
N
MMD( Fasleux)) + MMD(FF "N Fou(eu)))
MMD ( Fean))
(28a)
I, I.(w)))
Iz(wim)
N
0 P(F = min
Do))u(e:)) + Do(Gu(ci)) + (o(Gu((:)) -3J, " (x(Po)))
+\ LGr.D
N
o
J2(
(.)0)
(28b)
às PF = arg min MMD(Fa, Fa Fe..).
(28c)
N
Fach indexed by sample in equation indicates corresponding empirical distribution.
19

---

Published in Transactions on Machine Learning Research (02/2026)
Simultancous updates of parameters w and 7 facilitated by the essential role played by the generator function, which serves decoder during VAE training. Here, the generator is fed with (ELN, C1.N, Ci:N. and terms Zí (w, 7) and Ti(0) refer to minimizing the combined distance given by:
dwMMD(FFor FG.) WS(FP FG.. + MMD(FPop F
(29)
Since the MMD measure is an L--norm distance, the MMD-based terms Zι (w, 77) serves as the posterior reconstruction errors, while [2 17) serves as the regularization error for approximating the variational distribution FE, (CL:N (XP.) Furthermore, the gradient penalty J2(0) with positive cocfficient À was used to ensure training stability by forcing the 1-Lipschitz constraint on the discriminator (Gulrajani et al., 2017). Specifically, this penalty is given by
1 N LGP-D
Σ
Do X P0s x 1)?,
(30)
where XPos= uXPox + (1- Pas and X Pos represents any posterior fake sample generated by the generator.
D Theoretical Proofs
Theorem 1. Consider the DP posterior representations of MINE, presented in the main paper. Assume that both T, and e are M-bounded for all 7 E I (i.c., [Tyl, 1e s M). Given the DP posterior approximation in equation 7, for any N > 0, we have,
1. lim Epgn (T.PPKL (2(x))) 2 KKL (X,, Xz),
i. lim Eppi
(LDPIS (xx L X²).
+00
Proof.
Recall that, for all y T,
N
N
T.DKKL ( =
Σ
J. POST (((P (((Pom))) In Σ
J
pos p T ((100)))))00
(31)
1=1
1=1
where (JPP) D(richlet((a+ (n)nna+++nn.)..... m)/N,..., n)/N) and (X
idd
H
L:N
Applying the law of total expectation yields:
Eprox DPKL
EHg (Er (I. (IDPKL (Xx(X (((X
X Pos
(32)
where Fjrm is the CDF corresponding to JPos (JPcz
20

---

Published in Transactions on Machine Learning Research (02/2026)
Taking the expectation in equation 32 with respect to the Dirichlet weights gives:
FF (LDPKL (fi(X (NN) (Xx))
X Poc
(JP
Ix( f2 (x(()))
N In
Ey
Σ J
Pox
(33a)
t=1
(Linearity of Expectation)
N
Ery (J. ( (2((((())))
8=1
N
In
EF groe (J
(33b)
(Jensen's Inequality)
N
N (.
N
ln
(33c)
(Dirichlet Weights).
Taking the expectation in equation 33 with respect to the posterior atoms gives
Eh;; (Er (IDPK) (f1 (X (.))) fa (Xx)))
X
x
Es:
N
Em
1 (x
In
(34a)
N
(Lincarity of Expectation)
N
EHE (fx(FiXXPo)
Pos) f2(((()))
t=1
Eh; (TT(fi())))
In
(34b)
(Jensen's Inequality)
000) EH: (Tx()(
- Im (x(11(08")))axXX00
(34c)
(Identical Atoms).
On the other hand, as n approaches infinity, the Glivenko -Cateelii thcorem implics that Fx... converges F' and subsequently H; converges to F. This convergence implies that
P 4 Xx.
(35)
where 4.'h denotes convergence in distribution and X is random variable distributed according to F, for 1, 1,...,N. N. By applying the continuous mapping theorem, we have
$ (x()) e 2.000 4, (T(C)(0))))
(36)
21

---

Published in Transactions on Machine Learning Research (02/2026)
Since T and e"T are M-bounded, it implics
- Er (7((I((())
(37)
Ehд ( + EF (27401180cha(011111)
(38)
Finally, taking the limit in equation 34 as n + oo and applying the continuous mapping theorem implies
lim Ehz (Er (47 LPKL (fi Pay fx())x) n0000
X Pox
2 EF (T(((((X))
- EInEF (21161))))))
(39)
BFxxx (Ty X2))
In Bx &Fx; ( (22((X)
(40)
Lk (X', Xx2)
(41)
with X fi(e)) and X² = fx(X)). Consequently, for all y E r,
lim Eppn [DPKLL (X1 Pos f2 f2 Px))
(42)
Moreover, since
max I. DPKL (J((X I:N J2
((PPN)) 2 I I.PRIL (Fx(((
(43)
yET
for any 7 E F, taking expectations and limits yields
lim
Eppcs
max DPKI, (X
Pos I: fa (XPx))))) 2 1x
(44)
120000
7ET
Since the above inequality holds for all y T, it follows that
lim Epr max I DPKL ( PN (NN ((())))))))))))))))))
2
max 7 (X), X²),
(45)
200
LET
TET
which completes the proof of part (i).
A similar method is used to prove (ii) and it is then omitted.
E Additional Results and Implementing Details
E.1 DPMINE Evaluation on Complex Examples
In this section, we evaluate DPMINE against several benchmark methods on a set of complex examples adapted from Letizia et al. (2024), which are commonly used for assessing mutual information estimators.
E.1.1 Complex Gaussian
Gaussian Example. Following the same protocol used in the main experimental section, a Gaussian dimension, staircase experiment is considered to evaluate mutual information estimators as the dimensionality of dependence increases. Let
x = (U
U, V.R R
with & E (1,2,3, (,5,), where each value of is defines one staircase level.
22

---

Published in Transactions on Machine Learning Research (02/2026)
For fixed correlation parameter p 0.95, the variables generated according to the lincar Gaussian relationship
v= pU+ v1 E²E,
E E~ N0,
with U ~ (0, I&) independent of e.
The true MI at each staircase level is given by
MI(U, V) = 2 p²)),
and is used as reference for comparing estimator performance in this example.
The cubic, half-cube, and asinh transformations introduced below induce heavy-tailed marginals while pre- serving the MT of this Gaussian example, since each is an invertible transformation applied componentwise.
Cubic Example. Let
U (0)
and is generated through a nonlincar transformation of a correlated Gaussian,
V ( p1= e~ No,,
where the correlation parameter is fixed to p 0.95.
Half-Cube Example. Let U NO..I.).. The variable V first generated through a linear Gaussian relationship
= -=p V1-p²s E~ N(O,
with p = 0.95, and is then transformed elementwise using half-cube (lail-stretching) nonlinearity,
V = sign ()) |Z 3/2
Asinh Example. Let U E && be sampled from standard multivariate Gaussian distribution, U (O, The variable V is generated by first forming a Gaussian vector
z = U -1-126 E~ NOO,
and then applying the elementwise inverse hyperbolic sine (asinh) transformation,
V = asinh(Z),
where asinh()) denotes the inverse hyperbolic sine function and p 0.95 is fixed across all staircase levels.
E.1.2 Non-Gaussian Distributions
Student Example. To examine estimator behavior under heavy-tailed dependence, Student staircase example is considered. For cach value of k the joint vector (U, V) is sampled from centered multivariate Student-t distribution with the correlation structure
(U, V) ~ to (0. (0 pI.
Is
pI L
).
where the degrees of freedom are fixed to v = 2 and the correlation parameter is set to p = 0.95.
The true MI at each staircase level is approximated via Monte Carlo integration of the corresponding Student- densities and serves reference throughout training.
23

---

Published in Transactions on Machine Learning Research (02/2026)
DPMINE vs MINE
DPMINE vs NW]
DPMINE vs JSD-LB
DPMINE vs CPC
5200
12000 Stops
25000
20003
5020
10300 Steps
25,00
20000
5202
13000 Steps
15000
20300
5000
10000 Steps
15000
20002
True MI
MINE (KL)
NW]
JSD-LB
CPC InfoNCE
In(nmo)
Ours (KL)
a Gaussian Example
DPMINE vs MINE
DPMINE VS NWJ
DPMINE vs JSD-LB
vs CPC
5000
19000 Stops
35000
20100
5000
10300
15,000
20000
5000
10000
15000
20000
5000
10000
15000
20:90
Steps
Steps
Steps
True MI
MINE (KL)
NWJ
JSD-LB
CPC/ InfoNCCE
In(pmo)
Ours (KL)
(b) Cubic Example
DPMINE MINE
DPMINE NWJ
DPMINE JSD-LB
DPMINE CPC
-2
Stops
Steps
Steps
True
MINE (KL)
NWJ
JSD-LB
CPC/ InfoNCE
In(0mo)
Ours (KL)
(c) Half-Cube Example
DPMINE vs MINE
DPMINE vs NWJ
DPMINE vs JSD-LB
DPMINE vs CPC
5-
-2
-2
5002
35000
20000
5000
10200
15000
5000
20090
Steps
MINE (KL)
CPC InfoNCE
In(mmo)
(d) Asinh Example
Figure Complex Gaussian setting with X (U,V) ~ F. BNP-MINE uses mini-batch size 64, with F (P(MMAP, H) and H N( Sx..) Experiments for k 1,.... 5, using 4000 steps per dimension.
24

---

Published in Transactions on Machine Learning Research (02/2026)
Uniform Example. A uniform-additive staircase example is used to study estimator behavior under bounded, non-Gaussian noise. At staircase level k, we consider pair (U, V) with variable U is sampled uniformly on [0, 1 and V is generated by adding independent uniform noise,
V --U+E ~~
with 0.05 fixed across all staircase levels. Since the dependence is additive and independent across di- mensions, the MI scales linearly with the dimension. The one-dimensional MT is approximated via numerical integration and scaled by k to obtain the true MI at each staircase level.
DPMINE MINE
DPMINE NWJ
DPMINE JSD-LB
DPMINE CPC
5200
5000
Steps
Sceps
Steps
(KL)
NWJ
JSD-LB
CPC/
In(0ma)
(KL)
(a) Student Example
DPMINE vs MINE
DPMINE vs NW]
DPMINE vs JSD-LB
DPMINE vs CPC
10
10
2
-2
$200
12000
25000
20000
5000
10300
15000
20000
5002
13000
15000
20000
5000
10000
15000
20002
Steps
True
MINE (KL)
NWJ
JSD-LB
CPC/ InfoNCE
In(7ma)
Ours (KL.)
(b) Uniform Example
Figure 10: with F ~ DP(AMAP; H) and H (Xmmb, Sx..). Experiments are run for k 1,..., 5, using 4000 steps per dimension.
Complex non- non-Gaussian setting with X VU ~ F. BNP-MINE uses mini-batch size 64,
Figures 9 and 10 compare DPMINE with several varial ional MI estimators across Gaussian and non- Con-Gaussian settings. Across all examples, DPMINE tracks the true MI more closely while exhibiting substantially reduced variance compared to MINE, NWJ, JSD-LB, and CPC/InfONE. Competing estimators either suffer from high variability or exhibit systematic underestimation and saturation effects, particularly MT increases. In contrast, DPMINE shows stable and monotonic convergence toward the true MT across iterations, demonstrating improved robustness and reliability in both low- and high-MI regimes.
E.2
Application DPMINE (KL) in VAE-GAN
E.2.1
Toy Example
Stanford Bunny Dataset: We use the Stanford Bunny datasct, available at https://graphic. stanford.edul acdu/data//Dscanreee to investigate the impact of DPMINE on an additional toy example. The dataset consists point clouds representing the Stanford Bunny, a renowned 3D model provided by the Stanford University Computer Graphics Laboratory. These point clouds, captured with the Cyberware 3030 MS scanner and stored in PLY files (Polygon File Format) developed Stanford, represent spatial locations the object's surface. Point clouds provide 3D spatial representation of the object, enabling
25

---

Published in Transactions on Machine Learning Research (02/2026)
80
88
0.75
0.75
0.00
0.0
BNPWMMD
9.2
0.25
50
0.50
Real Sample
Reconstruction
Random
BNPWMMD ENPWMMD+DDMINN +
100
6.7
50
25
00
0.00
0.25
25
0.8
0.8
x
Figure 11: 2500 samples of the Stanford Bunny generated using ENPWMMD+DPMING after 5000 epochs, compared with BNPWMMD alone.
detailed visualization and analysis. For our study, we use three point clouds from this datasct and apply filtering with the pyoints Python library to extract points 3D space.
We randomly sampled 5000 points from the available 43,188 points as the training dataset and implemented our model on them by feoding the model with noise inputs to generate 2500 random samples and with encoded real inputs to obtain reconstruction samples. Figure 11 illustrates the significant impact of DPMINE on the performance of the BNP VAE-GAN in data generation.
E.2.2 Real Examples
The red border in Figure 12 indicates the corresponding slices depicted in Figure of the experimental findings discussed in the main paper. The results indicate that BNPWMMD-DPMINE yields improved sharpness and diversity in the generated 3D slices relative to the baseline methods. Fig- ure 13 further shows reconstructed samples that exhibit increased similarity to the training data under the BNPWMMD-DPMING model.
Covid-19 dataset:
BraTS 2018 Dataset: The BRATS 2018 dataset, a benchmark resource medical imaging available at html, employed for training models in the genera- tion of brain tumor MRI scans. For the experiments, data from 210 subjects labeled as "HGG' (High-Grade Glioma) are used, focusing on patients with aggressive brain tumors. Each subject's MRI data includes four distinct imaging modalities: T1-weighted (T1), Tl-weighted with contrast enhancement (TIce), T2- weighted (T2), and Fluid Attenuated Inversion Recovery (FLAIR). Notably, the FLAIR modality, which is particularly effective for highlighting edema and tumor boundaries, is used for the experiments, providing essential insights for synthetic MRI generation.
https:// //www//wwww/woo www. upenn. upwnn.ddu/bb............ wdt.sbbaabbattt/1/////////
The results presented Figures 14-17 and Table 3 reinforce our previous findings on the lung datasot, demonstrating similarly strong performance when applying the methodology the BraTS dataset. This cross-domain evaluation further validates the model's robustness and adaptability to diverse data types.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Axial
Sagittal
Coronal
Real Dataset
BNPWAIMD CEAAMD+DDER DPMINE
BNPWWMD
Random Samples
Figure 12: 27 slices along each axis of a 3D sample randomly generated using BNPWMMD+DPMINE after 7500 epochs, compared with BNPWMMD, for the COVID-19 example.
Axial
Sagittal
Coronal
Aepp:/aa H
BNPWWSD
Reconstructed Samples
Figure 13: 27 slices along each axis of a 3D sample reconstructed using ENPWMMD++ after 7500 epochs, compared with BNPWMMD, for the COVID-19 example.
E.3
Implementing Details
E.3.1
Calculation of Evaluation Scores
Considering the real and generated features, f. = [Featurel]:200. Featuure2 and fg (Featurel..000.g. Feature21.200.g) the FID and KID metrics are calculated using the following Eqs., respec- tively Binkowski et al. (2018):
FID(Sr, fz) = -1lgg 11,11 + Tr TT(22 Eg. + +E, -2.
(46)
KID(Fr, fa) = =MMD?(f.. fa),
(47)
27

---

Published in Transactions on Machine Learning Research (02/2026)
Axial
Axial
Axial
Sagitta!
Coronal
Sagittal
Coronal
Sagittal
Coronal
(a) Real Dataset
(b)
(c) BNPWMMD
Figure A visualization of a real sample and a randomly generated sample using BN- PWMMD+DPMINE after 7500 epochs, compared with BNPWMMD, in the BraTS18 example.
Real BNPWMMD+DPMINE BNPWMMD
150-
100
-0.1
Feature 2
A
Feature
50-
0.2
-0.4
Real
-50-
BNPWMMD+DPMINE BNPWMMD
00.5
100
-0
100
150
0.9
0.8
-07
0.66 0.5 -0.4
0.3
0.2 -0.1
Feature 1
Feature
t-SNE
(b) Custom encoder
Figure 15: Scatter plots of two-dimensional feature representations in the BraTS18 example, comparing samples generated using ENPWMMD+DPMINE with those generated using ENPWMMD. Panel (a) shows the (-SNE embedding of the features, while panel (b) displays the features obtained from custom encoder.
where ps,. and ff. and Σ,, and Eg. are the mean vector and covariance matrix of f. and fy, respectively. The empirical MMD metric in Gretton et al. (2012) is used to compute MMD² with polynomial kernel k(fr, f,) (f.5f. + +)" and degree The "Tr" denotes matrix trace, "T" denotes matrix transpose, and "I|-|I" denotes Euclidean norm. We use the provided code at https: i seatthedocssio/e/// v0.8. 2//imag//keree//// http://c.....gg.....e../////aaae
readthedocs.
code at htaaa////eemee_nnettiiioo.. distance. html to compute KID with v = 3, and also use the provided wwwp/rgtt//g///////toott.......t generated/ignitee... www.mmttetii.... FID .html to calculate FID.
Additionally, we calculate MS-SSIM using the available codes at https://torcmmetricssreeeee hew.heehrrrrtrrrrethcooooooo.o.. 1o/ en/v0. 8. 2/ http//iaaggmmmll// scaleessaeee_eeeeeeeeeeeeeeeeeee attpallesstuuuuralllommmmm sattlllssmmlllaarree .tml.
E.3.2 Network Setting
This section summarizes the network architectures used in all experiments. The encoder, generator, and discriminator architectures listed in Tables 4, 6, and follow standard designs commonly adopted in 3D generative modeling and are consistent with those used in the a-WGAN setting (Kwon et al., 2019). The BNPWMMD model Fazeli-Asl & Zhang (2023) builds these components and additionally includes code
28

---

Published in Transactions on Machine Learning Research (02/2026)
Axial
Sagittal
Coronal
Real Dataset
13% COPPIRS CORAAAANN DPPMIN
BNPWWSD
Random Samples
Figure 16: 27 slices along each axis of a 3D sample randomly generated using BNPWMMD+DPMINE after
7500 epochs, compared with BNPWMMD, for the BraIS18 example.
Axial
Sagittal
Coronal
ANT -WIKaa---NNNAAEA CIPEE
Reconstructed Samples
Figure 17: 27 slices along each axis of a 3D sample reconstructed using ENPWMMD+ after 7500 epochs, compared with BNPWMMD, for the BraTS18 example.
generator, detailed in Table 5. Figure 18 illustrates the overall workflow of the BNPWMMD+DDMMINE framework.
Additionally, Table 8 presents a simple architecture used for updating the paramcters (Ty)yer in the DP- MINE and MINE calculatiionsBelgaadd et al. (2018).
All architectures have been implemented in PyTorch using the Adam optimizer and a learning rate of 0.0002 on an NVIDIA Tesla V100-SXM2 with 4 GPUs, each with 32GB of RAM. Our code requires 48-72 hours provide results for the COVID-19 and BraTS18 examples. For the coil experiments, our code takes about 4-6 hours.
29

---

Published in Transactions on Machine Learning Research (02/2026)
Table Comparison of statistical scores (mean + 22) for the BraTS18 example ver 100 replications.
Evaluator
BNPWMMD DPMINE
BNPWMMD
FID
Custom Encoder
0.00197 = 0.00015
0.002644 0.00021
I-SNE
25.93170 14.68720
34.13418 + 16.00405
KID
Custom Encoder
0,00038 0.00004
0.00067 0.00007
t-SNE
8.81000 = 7.18880
9.63544 = 8,95555
MMD
Custom Encoder
0.00075 0.00078
0.00092 = 0.00087
I-SNE
0.19887 0.06760
0.27312 + 0.07383
MS-SSII SSSM
0.75007 0. 1.702
0.74998
0.12031
DPMINE
Mooii Poss
ao c)
Discriminator
Generation
Original Input fake?
PPss (E). (o) (c)
P8 xPos
Code Generator
č (F))
Encoder
(x
P
DP MINE
c) fra DOOO Meaall
(XPos
Figure 18: general flowchart to illustrate the actual operation process of ENPWMMD-DDMMNN..
F Limitations
The BNP framework proposed in this paper assumes that the training data follows the IID assumption. However, this assumption restricts the model's applicability to certain privacy-preservigg techniques, such as federated learning, which often rely on non-lID assumptions at the local device level. While this limitation is acknowledged, it is beyond the scope of the current work to address it. In future research, we plan to extend the proposed BNP framework to incorporate non-lID assumptions and apply it to federated learning. This extension aims to mitigate issues such as slow convergence and unfair predictions that may arise when applying global model to unseen data.
G Broad Impacts and Safeguards
One potential positive impact of this paper is the advancement of artificial intelligence (Al) techniques in healthcare. The proposed model offers a solution to the challenge of accessing high-quality, diverse medical datasets for clinical decision-making. This can lead to improved accuracy and effectiveness in diagnosing and treating discascs, ultimately benefiting patients and healtheare providers. Additionally, the incorporation of BNP procedure helps uncover underlying structures in the data and reduce overfitting, enhancing the reliability of the generated samples.
However, the use of AI in healthcare also raises concerns about data privacy and sccurity. It is important to prioritize patient privacy and data protection by implementing strict regulations and protocols to ensure that patient data is anonymized and protected. Additionally, there is a risk of bias in the generated samples, which could lead disparitics healthcare outcomes if the models are not properly validated and tested on diverse populations. Therefore, transparency and accountability in the development and deployment of these AT models are necessary. Rigorous testing and validation on diverse datasets should be conducted to mitigate
30

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: Encoder: BNPWMMD-GAN.
A 3D Convolutional Network Architecture for the COVID-19 Dataset used in
Input dimension
Output dimension
Kernel Size
Activation Function
Layer
Stride Padding
x 64 64
Convolution
64 32 x 32 32 X 4 x
Leaky ReLU (negative slope=0.2)
(Data dimension)
Leaky ReLU
Convolution
64 32
32
128
164 X
(negative slope=0.2)
BatchNorm
128 x 16 x 16 x 16
128 x 16 x 16 x 16
Leaky ReLU (negative slope= 0.2)
Convolution
128
16
16
256 8 8
1x4x x
BatchNorm
256 x 8 x 8 x
256 x 8 x 8
Leaky ReLU
Convolution
256 8 x x
x
4 4
2
(negative slope=0.2) 0.2)
Narc
512
5 4
Convolution
512 X
1000 x1011x111
1x4x4
Table 5: Code- Gede-Gencrttor: A 3D Convolutional Network Architocture for the COVID-19 Datasct used in BNPWMMD-GAN Layer
Input dimension
Output dimension
Kernel Size
Activation Function
Stride Padding
100
Convolution
(Sub-latent
16
2
5
103 x
1
ReLU
dimension)
BatchNorm2
16
x
1016
x 10
Max-pooling
16
16x2x
10
16 16x2xx5x2
1
X 2
x 2
Convolution
16
x
32 x
2
x3xx
1
ReLU
BatchNorm2
x
2
Max-pooling
x
x2
x112 2
x 11x2
Convolution
x
x X
3 3
ReLU
BatchNorm3
64 x
64
Max-pooling
64 x 2
64 x
2 5
x 1 2
1 X2
Flatten
64 x 2
640
Fully-connected
640
1000
bias and ensure fairness. Collaboration between healthcare professionals, Al researchers, and policymakers is essential to establish guidelines and regulations that prioritize patient privacy, data protection, and equitable heartheare outcomes.
31

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Generator: BNPWMMD-GAN.
3D Convolutional Network
Architecture for the COVID-19 Datasct used
Input dimension
Output dimension
Kernel Size
Activation
Layer
Stride Padding Function
1000
Transposed
512 4 4
1x4xa
ReLUU
convolution
(Latent dimension)
BatchNorm
512 x x 1
4 X 4
Upscale
512
512 4
Convolution
512 4 4
256 X x 8 8
3x3x x
1
ReLU
BatchNorm
256 x
256 x 8 x x $
Upscale
x
256
Convolution
256 8
128 x 16 16 x 16.3 x x
ReLU
BatchNorm
128 16 x 16 x 16
128 x 16 16 x 16
Upscale
16
16
128
16
Convolution
128 16 x 16 16
64 x 32 32 x 32 x x
1
ReLU
BatchNorm
64 x 32 x 32 x 32
64 x 32 x 32 x 32
Upscale
32 x 32
64 x 32
Convolution
64 32 x 32 x 32
64 x 64 64 3x3x3
I
Tanh
Table Discriminator: A 3D Convolutional Network Architocture for the COVID-19 Dataset used
BNPWMMI-GAA.
Input dimension
Output dimension
Kernel Size
Activation Function
Layer
Stride Padding
Convolution
x
64 32 x 32 32 4 4x
Leaky ReLU (negative slope=0.2)
(Data dimension)
Leaky ReLU
Convolution
x 32 x 32
16 164 X
(negative slope=0.2)
Batch.Nrrm
128
16 x 16 x 16
128 x 16 16 16
Leaky ReLU
Convolution
128
16
16
256 8 x
4x4x x
(negative slope=0.2)
Barch
256 x 8 x x 8
256 x 8 8
Leaky ReLU
Convolution
256
8
4
4x4xA
(negative slope= 0.2)
NarchNorm
512 x x 4
512 612x4x4x4
Convolution
512 x 4xx44
Ixlx1x1
4x4x
Sigmoid
Table 8: Architecture of the shared Ty network employed by DPMINE and all benchmark mutual information estimators.
Activation
Layer
Input Dimension
Output Dimension
Function
1000 + 64
concatenation
263144
(Latent dimension Data dimension)
Linear
263144
400
ReLU
Linear
400
400
ReLU
Linear
400
400
ReLU
Linear
400
1
ReLU
32


Reviews:
[{"review_id": "GxOACTbWme", "reviewer": "Reviewer_wpM7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The core idea of the paper is to apply Bayesian bootstrap resampling to minibatches used in MINE, replacing uniform sample weights with Dirichlet-resampled weights. This is a well-known technique (Rubin, 1981) but is here repackaged as a “Dirichlet-process MINE” and applied in the context of VAE-GANs. Concretely, the method is inserted into MMD-GAN training using Gaussian kernels, along with an encoder-decoder structure closely resembling MMD-GAN (Li et al., 2017). The contribution appears to be a bit incremental: MMD-GAN + bootstrap-weighted MINE. This would not have been a core issue if the contribution was supported by convincing quantitative results.", "claims_and_evidence_yn": "No", "claims_explanation": "- The paper suffers from a bit of an identity mismatch: it presents itself as proposing a new mutual information estimator, but it evaluates only against MINE, a rather outdated and unstable baseline. Comparisons to modern MI estimators such as Nested Monte Carlo (NMC, Anastasiou et al., McAllester, Rainforth, etc.), Variational Prior Contrastive Estimators (VPCE), InfoNCE/CPC, NWJ, and other $f$-divergence–based methods that directly address variance, bias, and stability issues, would be mandatory in this case. As a result, it is unclear whether the contribution is meant to be a new MI estimator, a robustness trick for MINE, or simply a Bayesian bootstrap trick to improve MMD-GAN. Clarifying this intended role and including comparisons to the appropriate state-of-the-art MI estimators would substantially strengthen the paper, help position the work properly within the literature, and let readers to judge whether the DP-based weighting provides notable gains.\n\n-  The writing needs improvement. The **Abstract** and **Introduction** can be tightened a lot to get to the gist of the method faster. Currently, it reads like a lot of jargon-heavy AI-generated text that heavily dilutes the contributions (which I suggest should be clearly stated at the end of Introduction, instead of the redundant enumeration of the following sections).\n\n- On a related note, the paper really needs to focus its contribution. The subsections in Section 2 are not well connected, contain lots of generic statements and surface summaries of references. Is the goal of the paper to validate a new MI estimator or to improve VAE-GANs? I suggest the authors streamline the readability of Section 2 using paragraphs containing a *single idea that flows into the next one*, and create a dedicated **Related Work** section which discusses direct contender methods (those that are later compared against the proposed metod).\n\n- Overall, I found the notation unnecessarily laborious. Defining everything in terms of CDFs instead of PDFs makes sense for general RVs, but the authors never consider discrete variables or measure-theoretic notation, so I think it can be simplified in terms of PDFs. That way, the notation can also connect to the related work on GANs. I did not see any benefit of having triple subscripts in **Section 3** and the math exposition is at times inconsistent. Furthermore, Theorem numbering in the Appendix does not correspond to Theorem numbering in the main text (e.g., Theorem 3 is Theorem 1, and so on).\n\n- Moreover, I am afraid that the proof of Theorem 2 is circular, as it assumes the function approximator class is rich enough to contain or uniformly approximate the DV minimizer and the network training can reach the optimum. The authors are basically assuming that because the DV bound converges and max is continuous, everything works . Because of this, I find the formalism of the “Theorems” (together with the \"Remarks\") to be, if anything, more discombobulating than enriching.\n\n- Finally, the evaluation is lacking in many respects, as Table 2 contains the only quantitative benchmark that clearly shows the benefit of the method over naive MINE, but these are > 7 year old contenders. It is also missing consistent decimal formatting and error bars.", "audience_yn": "Yes", "audience_explanation": "Mutual information estimation is a hard and important problem in ML and statistics, and Bayesian bootstrap is an interesting addition to standard MMD-GAN losses.", "requested_changes": "- The contributions need to be focused, the positioning of the paper clarified, the contenders of the method clearly stated and compared against, the writing needs to be streamlined.\n\n- I would suggest that all equations should be numbered and use consistent symbols (e.g., L vs calligraphic L). The remarks before the theorems are unnecessary and pretentiously formal. The theorems are semi-formal and rather argumentative, questioning the need for having them or emphasizing them too much. The entire mathematical exposition can be greatly simplified. Please ensure all typos and inconsistent equation references are fixed (e.g., already the first equation is referenced as \"Eq. equation 1\", and this applies to all further equations).\n\n- The proposed method should be evaluated with proper quantitative benchmarks and the presentation of the results needs to be considerably improved; the current quality of the figures (squished, sheered, and accompanied by uninformative captions) is unacceptable by any standard.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "RrL45MmZyp", "reviewer": "Reviewer_77Se", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes DPMINE, a Bayesian nonparametric method for the estimation of mutual information (MI). This is done through placing a Dirichlet Process (DP) prior over the unknown joint data-generating distribution and propagating the posterior uncertainty via a neural MI estimator. The method draws approximated DP posterior samples, through a finite approximation of the DP, generates synthetic data from each sample, and uses a MINE-style variational objective (DV or JS) to estimate the MI for each posterior draw.  A further contribution, described in Section 3.3, is the proposed integration of DPMINE into deep generative models, where the MI posterior or its summary statistics, such as posterior mean, is used as a regularizer or objective in representation learning.", "claims_and_evidence_yn": "No", "claims_explanation": "I think the proposed estimator is appealing, the idea is novel, and the method shows strong empirical performance. However, I believe that in its current form, the manuscript lacks adequate justification of some methodological and theoretical aspects. Moreover, I think a clearer and more structured presentation of the background and methodology would substantially improve the accessibility of the work and better motivate the contributions. I outline these concerns in detail below:\n\n- The method appears to rely on a form of bootstrapping that is closely related to the BNP techniques discussed in the paper. However, the presentation does not clearly explain how it connects to this existing body of work, nor does it fully clarify the methodology itself. In particular, the proposed procedure seems to involve:\n    - Sampling (approximately) probability measures from the DP posterior.\n    - For each sample, defining an estimator of the mutual information by maximising a VLB that is constructed using that posterior draw.\n    - Treating the resulting collection of mutual information estimators as posterior samples of the MI functional.\n\n     This is effectively the posterior bootstrap, which is closely related to the Bayesian bootstrap, as developed by Lyddon (2018), Fong (2019), with the main distinction being that the functional of interest here is the MI estimator rather than a parameter minimising a loss. Although these papers are cited, the relationship between the proposed approach and these bootstrap methodologies is not discussed in sufficient depth.\n\n- The discussion of the Dirichlet Process (DP) as a nonparametric Bayesian model is, at times, conflated with the broader BNP perspective on belief updating. For example, Section 3.1 is devoted to the DP and its approximation, yet much of the accompanying discussion, such as Remark 2, which addresses prior elicitation, relies on arguments drawn from papers situated within the Bayesian Nonparametric Learning (BNPL) framework. Because the BNPL framework has not been introduced at that point in the paper, it becomes difficult for the reader to follow the motivation and reasoning behind these arguments. A similar issue appears in Remark 1, where the authors claim that using a prior centering measure in a DP “makes the model less sensitive to fluctuations […] for optimization processes in models that rely on loss functions.” It is unclear which model or loss function this statement refers to, leaving the reader without sufficient context. Furthermore, the work by Bariletto (2024) cited here is grounded in a Distributionally Robust Optimization framework, in which the role and interpretation of the centering measure differ substantially from its role in standard Bayesian nonparametric methods.\n\n- Some claims in the paper, while potentially correct, are not sufficiently justified or explained. For instance, in multiple places (e.g. the beginning of Section 3 and in Section 1), the authors state that the proposed method is “more robust” than existing approaches. However, it is not clear what form of robustness this refers to until quite a bit later in the paper. This clarification is crucial, especially because the proposed approach differs in some ways from standard BNP methodologies, which, as the authors themselves note, typically address robustness to model misspecification in parametric inference. Hence, a clear explanation and position of the method within the literature would add to the overall motivation of the method.\n\n- I think Theorems 1 and 2 are insightful. However, a few theoretical issues are not discussed:\n  -  Theorem 2 states that, conditionally on a posterior sample, the neural MI estimator converges to the true MI of that sampled distribution as the amount of synthetic data grows. However, it is not entirely clear to me if this would imply that the posterior distribution of MI converges to the true MI. This concern is amplified by the fact that the functions $f_i$, which produce joint and product-of-marginals samples, are not properly defined, creating ambiguity in the interpretation of the theorem\n- The approximated DP posterior draws are random discrete probability measures, whereas the true MI is (presumably) defined for an underlying continuous distribution. Even if DPMINE computes the DV bound exactly for a discrete DP draw, would this necessarily imply that it provides a good approximation to the MI of the continuous data-generating distribution? This distinction is not adequately discussed in the paper.\n- How does the choice of truncation of the DP introduce any bias or error in the MI estimation\n\n- The presentation of the methodology and the results are unclear in places. For example, theorem 1 mentions equation 15, which has not yet been introduced. The functions $f_i$ are not properly defined or explained in the context of the method. \n\n- I found that discussions around potential limitations of the method and empirical results about how hyperparameters like $\\alpha$ or $N$ could affect the method are not present, although they could enhance the clarity of the paper.", "audience_yn": "Yes", "audience_explanation": "Mutual information estimation is central to many problems in machine learning, including representation learning, generative modelling, and causal inference. Providing a Bayesian quantification of uncertainty over MI is appealing, and the attempt to integrate such an estimator into deep generative models is timely. The conceptual combination of DP-based posterior sampling and neural MI estimators is novel, and the ambition of producing a posterior over MI is likely to attract interest. However, I believe a clearer presentation of the methodology and results , as well as a thorough discussion of the issues raised above would lead to a stronger theoretical grounding and a clearer characterisation of limitations, making the work more accessible to a larger audience.", "requested_changes": "I believe this paper is a potentially valuable contribution to TMLR. Strengthening the clarity of the exposition and addressing some methodological and theoretical gaps would substantially improve the manuscript and make its contributions more accessible and compelling. Hence, I propose the following 'critical' changes:\n- A detailed step-by-step description of the proposed algorithm, together with an explicit discussion about any connection to existing BNP algorithms such as the posterior bootstrap and Bayesian bootstrap frameworks, would help situate the work and clarify its novelty. \n- In the Background, could the authors clarify and separate the discussion around the DP, as a nonparametric model for the data-generating process, and the general nonparametric learning framework of Lyddon, Fong as a bootstrap sampling algorithm. \n- The paper should clearly define and analyse the sampling operators $f_1$ and $f_2$, explaining how synthetic joint and product-of-marginals samples are produced, and discuss how these are or aren't incorporated into the theoretical results.  \n- Could the authors elaborate more on the type of robustness induced by the methodology earlier on in the manuscript, providing some intuition about this robustness?\n- The authors should address the mismatch, if any, between discrete DP posterior draws and the continuous MI functional (more details in the answer above)\n- Could the authors discuss whether the BNP MINE is supposed to yield (non-standard) posterior beliefs about the MI estimator, similarly to previous Bayesian nonparametric learning works? If so, could they clarify the expected behaviour of this posterior, beyond the consistency result provided for a single posterior draw? Is this induced posterior expected to converge to the true MI? If a formal theoretical statement is not possible, a discussion around why that is the case and some intuition behind this behaviour would still be very valuable. \n- [non-critical] Could the authors discuss empirically or otherwise the effect of a 'sub-optimal' choice of a prior centering measure $H$, the selection of the hyper-parameter $\\alpha$ ? It seems to me that all of these could potentially affect performance, for example, through a large value of $\\alpha$ when a bad choice of $H$ is made. This is to be expected in any nonparametric learning method and doesn't necessarily undermine the method's validity, but a discussion (beyond that of Remark 2) or toy example showing for example, how DPMINE’s performance varies with $\\alpha$ in the generative-model setting, would strengthen the paper.", "broader_impact_concerns": "No broader impact concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OgDd4gXHN6", "reviewer": "Reviewer_sopd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper makes several contributions. First, it introduces a Bayesian nonparametric estimator of mutual information that replaces empirical-distribution–based losses with a loss constructed from a finite Dirichlet process posterior representation. Second, it shows that this regularization reduces variance and mitigates sensitivity to sampling fluctuations and outliers, providing more stable gradients during training and improving convergence of neural MI estimators. Third, it establishes theoretical guarantees for. Fourth, it demonstrates the practical utility of the estimator by applying it to mutual-information maximization in variational autoencoders, yielding improved convergence and representation quality on synthetic data and 3D CT image generation.\n\nThe key strengths of the paper are (i) its creative use of BNP ideas, and in particular of prior regularization, to achieve improved and stabilized performance in a DL task such as MINE, and (ii) its experimental validation supporting the claims. Some major weaknesses are the relatively poor organization of the presentation, the lack of clarity in the theoretical proofs, and an insufficient provision of insights about experiments.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The above “yes” is with caveats. The experiments are quite extensive and convincing, although I do believe that their interpretation is a bit hindered by the unclear presentation of the part on GAN (see more later). The theoretical proofs, instead, are extremely unclear, though (as I explain later) I don’t believe the theorems add much to the treatment, so they could either be amended or eliminated altogether.", "audience_yn": "Yes", "audience_explanation": "The contribution fits within a recent interest in the incorporation of Bayesian ideas into modern learning frameworks (e.g., optimization), so a community of readers (though not extremely large) would exist.", "requested_changes": "Major adjustments:\n\n-\tThe proofs of the theoretical results are quite messy and, as written, seem to contain flaws. For instance, the proof of Theorem 1 takes an expectation with respect to the posterior law of the sample. However, somehow it treats the $X_\\ell$ as constants with respect to the law (as if randomness was only because of the weights J). But the atoms are part of the random measure, and I don’t see a quick way to solve this flaw. I would also encourage the authors to be precise about which quantity ($n$ or $N$) is sent to infinity first, and with respect to which probability measure the a.s. statements should be understood. The proof of Theorem 2 is equally hard to follow. It may be that I am misinterpreting the results and/or the proofs, but I do have some experience with these calculations. This implies that, whether I am misinterpreting or not, the authors should consider improving their treatment of the theoretical results significantly, if they plan to keep those results at all. This is because if someone with some experience is finding the reading of the results hard, I can only imagine how hard it may be for less experienced readers. To be clear, this being a predominantly methodological contribution, I don’t see any obstacle in removing the results altogether, especially if the authors are not able to fix the proofs and/or provide a much clearer presentation. In fact, the results don’t add much to the contribution of the paper, and may only end up creating confusion (at least in their current form).\n\n-\tThe paper makes many somewhat vague statements, sometimes to simply tie its contribution to existing literature. Moreover, especially in the later part of it, a lot of math is introduced with not enough context. The combination of these two factors makes it somewhat hard to discern the true contributions of the paper from smaller implementation details or weak ties to existing literature. I know this is a bit of a generic request, but I really encourage the authors to think hard about what the essential points of their article are, and to center their writing around those, with a lot focus and parsimony in the presentation. For instance, the transition between the part on MI estimation and the GAN architecture is quite sharp and it is hard to understand the connection until later, and even then that connection is buried behind a wall of equations that are hard to grasp all at once. Please consider revising in a major way the text to take these considerations into account.\n\nMinor adjustments/clarifications:\n\n-\tThe authors should be careful with typesetting (e.g., there are multiple instances of Eq. equation X, some idiosyncratic indentations), and in general a thorough review of English writing is necessary.\n\n-\tCan the authors elaborate further on the MAP strategy put out in Remark 2? My intuition is that one could try to find the optimal $\\alpha$ by maximizing the EPPF associated to the DP, but that would lead to choosing an arbitrarily large value in the presence of distinct data points. I am interested in knowing more about this procedure.\n\n-\tIn the simulated data experiments (e.g., figures 2 and 3), how does hyperparameter (say, DP concentration and centering measure) choice affect the results? What I mean is: is the DP very concentrated on a good guess for the data-generating process? If so, the result is expected due to prior regularization. This is not bad in itself, but I guess the choice of hyperparameters concentrating the prior on a good guess need to be justified by actual prior (e.g. scientific) beliefs about the data-generating process, not just (as it may be in this case) by the knowledge of the authors of the generative process. Summarizing, the main thing I’m asking is to add intuition as to what features of the proposed methodology lead to better performance (in this case, lower variability), and a discussion as to how the choices leading to these features are justifiable in practice.", "broader_impact_concerns": "I do not see any ethical concerns with this work.", "additional_comments": "None.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)