A Semi-Bayesian Nonparametric Estimator of the Maximum Mean Discrepancy Measure: Applications in Goodness-of-Fit Testing and Generative Adversarial Networks
Authors: Forough Fazeli-Asl
Michael Minyi Zhang
Lizhen Lin
Venue: Accepted by TMLR ()

Abstract: A classic inferential problem in statistics is the goodness-of-fit (GOF) test. Performing such tests can be challenging when the hypothesized parametric model has an intractable likelihood and its distributional form is not available. Bayesian methods for GOF testing can be appealing due to their ability to incorporate expert knowledge through prior distributions. However, standard Bayesian methods for this test often require strong distributional assumptions on the data and their relevant parameters. To address this issue, we propose a semi-Bayesian nonparametric (semi-BNP) procedure based on the maximum mean discrepancy (MMD) measure that can be applied to the GOF test. We introduce a novel Bayesian estimator for the MMD, which enables the development of a measure-based hypothesis test for intractable models. Through extensive experiments, we demonstrate that our proposed test outperforms frequentist MMD-based methods by achieving a lower false rejection and acceptance rate of the null hypothesis. Furthermore, we showcase the versatility of our approach by embedding the proposed estimator within a generative adversarial network (GAN) framework. It facilitates a robust BNP learning approach as another significant application of our method. With our BNP procedure, this new GAN approach can enhance sample diversity and improve inferential accuracy compared to traditional techniques.

Published in Transactions on Machine Learning Research (09/2024)
A Semi-Bayesian Nonparametric Estimator of the Maximum Mean Discrepancy Measure: Applications in Goodness-of- Fit Testing and Generative Adversarial Networks
Forough Fazeli-Asl
foroughffhbaa.k
Department of Statistics and Actuarial Science University of Hong Kong
Michael Minyi Zhang
mahang 18hhun hk
Department of Statistics and Actuarial Science Universily of Hong Kong
Lizhen Lin
lizhen00 1 &hurnd edu
Department of Mathematics The University of Maryland College Park, MD, USA
Reviewed on OpenReview: https: pptpr/veeerr..e...mmmee net/ formmf id= L0n UHS1FTT
Abstract
A classic inferential problem in statistics is the goodness-of-iit (GOF) test. Performing such tests can be challenging when the hypothesized parametric model has an intractable like- lihood and distributional form not available. Bayesian methods for GOF testing can be appealing due their ability to incorporate expert knowledge through prior distribu- lions. However, standard Bayesian methods for this lest often require strong distributional assumptions the data and their relevant parameters. To address this issue, propose semi-Baycsian nonparametric (scmi-BNP) procedure based on the maximum mean discrep- ancy (MMD) measure that can be applied to the GOF test. We introduce a novel Bayesian estimator for the MMD, which enables the development of a measure-basee hypothesis test for intractable models. Through extensive experiments, we demonstrate that our proposed test outperforms frequentist MMDD-baed methods by achieving a lower false rejection and acceptance rate of the null hypothesis. Furthermore, showcase the versatility of our ap- proach by embedding the proposed estimator within generative adversarial network (GAN) framework. It facilitates a robust BNP lcarning approach as another significant application of our method. With our ENP procedure, this new GAN approach can enhance sample diversity and improve inferential accuracy compared to traditional techniques.
1 Introduction
Goodness-of-itt (GOF) tests are commonly used evaluate an empirical data set against hypothesized parametric model. However, there are cases when the likelihood of the parametric model is intractable and the explicit form of the model distribution is unavailable, making it challenging to directly assess the model's fit. One such example is the case of deep generative models, where independent samples can be generated, but the required likelihood function needed for traditional GOF tests is intractable. In such situations, a potential solution to use the the maximum mean discrepancy (MMD) measure as an alternative approach for conducting GOF tests (Gretton al., 2012a; Key et al., 2021) in addition to some existing kernelized methods (Liu et al., 2016). The MMD is metric on the space of probability distributions and is commonly used in hypothesis testing to quantify the difference between the distribution of the data and the hypothesized model.

---

Published in Transactions on Machine Learning Research (09/2024)
It can be conveniently estimated using available samples generated from desired distributions. The MMD estimator has proven to be effective in various applications, including analyzing large-scale datasets with higl-dimensional features and implementing generative models, especially generative adversarial networks (GANs).
Bayesian nonparametric methods, while powerful, have received comparatively little attention, especially regarding their application in estimating the MMD. One of the primary benefits of the Bayesian approach is that expert knowledge can be incorporated into the prior distributions in a diagnostic setting. Moreover, BNP learning procedure can provide certain level of regularization to the training process. This is partially a result of placing uncertainty on the sampling distribution of the data, via a Dirichlet process (DP). Therefore, the lack of such methods in MMD estimation proves to be a hindrance for the statistician who wishes to be Bayesian without overly strong assumptions. This paper seeks to fill this crucial gap.
In this paper, we propose a BNP estimator that accurately estimates the MMD dernel-bascd mcasure between an intractable parametric model and an unknown distribution. To develop the procedure, we place the DP prior solely on the unknown distribution. Therefore, refer to this procedure a semi- Bayesian nonparametric (semi-BNP) estimator. Having established our MMD estimator, we demonstrate that can generalize the bootstrap procedure given in Dellaporta et al. (2022) beyond posterior parameter inference. First, we apply our estimator in a variely of hypothesis testing problems. Next, we introduce a robust Bayesian nonparametric learning (BNPL) approach for training GANs based on simulating from the posterior distribution on the parameter space of the generator. Our approach utilizes the aforementioned estimator as a robust discriminator between the generator's distribution and a DP posterior on the empirical data distribution. Specifically, our framework unifies concepts of the MMD) measurement and the BNP inference to leverage their respective benefits into single discriminator. Furthermore, we will investigate the ability of our discriminator to reduce mode collapse and increase the ability of the generator to fool the discriminator more effectively than the frequentist counterpart for GAN training.
The paper is organized as follows: In Section 2, we review previous works and methods related to our proposed technique. We then introduce our novel semi-BNP estimator for the MMD measure bet ween an unknown and intractable parametric distribution in Section 3, and provide theoretical properties of our proposed estimator. In Scction 4, we utilize our semi-BNT estimator of the MMD measure to ereate powerful GOF test based on the rvlative belief (RB) ratio, which serves as the Bayesian evidence to judge the null hypothesis. Moreover, Section 5 outlines the incorporation of the semi-BNP estimator as the discriminator in the GAN architecture. This results in a robust BNPL procedure that accurately estimates the generator's parameters for generating realistic samples. The section also discusses the theoretical properties of the proposed discriminator, such as robustness and consistency. We evaluate the novel semi-BNP procedures for hypothesis testing and GAN training through numerical experiments in Section Lastly, we conclude the paper in Section and discuss potential future directions. All proofs, algorithms, notations, and additional experiments are given in the Appendix.
2
Previous Work
Our proposed method consists of two fundamental components: the MMD measure and the DP prior. First, we will review these two concepts.
2.1 Maximum Mean Discrepancy Measure
For a given data space X, consider the random variables X and Y, drawn from distributions Fj and Fz respectively. Here, Fi and F2 belong to B(T), which represents the set of Borel probability distributions on X. We consider the discrepancy d B(x) BXx) + oo) through the integral proballiyybaillty metric (TPM) (Mäller, 1997), defined as shown in (1). The class of functions F is designed to be rich enough to distinguish between F1 and F2, and restrictive enough to provide accurate ostimates based finite sample.
dira F1, F2) =upp Fr, (h(X)) - Fr,, (M (Y)))) hef
(1)
2

---

Published in Transactions on Machine Learning Research (09/2024)
The MMD is then defined by considering I = {h Heel I 1}, which represents unit ball in reproducing kernel Hilbert space (RKHS) Hy with associated kernel &: X x X - IR. In this context, |1-1 denotes the norm function in the RKHS. The function A(-,-) is positive definite, such that for any function h E HG and any X E 3, h(X)= = (h, k(x, ))ok.. where (5,)),, represents the inner product in Hg. Consider function (.)
EF, [k(X,-) }] E Hgg which is defined as the kernel mean embedding of the distribution
oo for all Fi in Gretton et al. (2012a). Then, for given X, X i.d. FY,YY Y didd F2, EF EF(vk(X,
F B33), the MMD is given by
MMD (FI, (2) ||FF - uFal LF (XX, 2E (K Y) BP (Y(,, Y')I.
(2)
Note that MMD F2) = 0 if and only if Fi F2, when Hee is a universal RKHS defined on a compact metric space E' and k(---) is continuous (Gretton et al., 2012a, Theorem 5). In practice, distributions Fi and F2 are not accessible, and then the biased, empirical estimator of (2) (V-statistic) is calculated using empirical distributions Fi,n and F2,m as
MMD2(F1..o) F2,m)
1 n2
Σ
K(X,
2
k kX,, Y,)
m2 Σ K(Y,,
(3)
where XI...., Xx, is a sample from Fi and Y,,...; Y,,. is a sample generated from F2.
Recently, Key et al. (2021) proposed GOF test using the MMD measure when the hypothesized model belongs to a parametric family of intractable models. It was proposed to be employed in training generative models such as togyle-switch models and GANs. There are also numerous generative models closely linked to the implementation of MMD in GANs, which can be found in Briol et al. (2019), Niu et al. (2023), Oates (2022), and Bharti et al. (2023). These models offer distinct MMD estimators that are specifically designed to further improve the MMD's capability in estimating the generator's parameters.
2.2 Bayesian Methods: Approximate Bayesian Computation, the Dirichlet Process and Bayesian Nonparametric Learning
Previous work simulation-based inference has largely focused on applying discrepancy measures from a frequentist nonparametric (FNP) perspective. A Bayesian perspective on simulation-based inference involves similar methodology, using approximate Bayesian computation (ABC) to estimate the model parameters via simulation (Beaumont et al., 2002). In ABC, we first sample a proposal by sampling from a prior distribution placed on the paramcter space of the generative model (Step 1). Rather than inferring parameters directly from the posterior distribution, we compare the summary statistics of the simulated data, given the current state of the parameters, with those of the observed data using a discrepancy measure (Step 2). The simulated parameter values corresponding to the accepted summary statistics are retained if the distance falls within predetermined threshold (Step 3).
Identifying informative summary statistics in ABC is challenging task and it depends the specific application of the data being analyzed, such as the mean effective hetcrozygosity or the mean of variance in repeat numbers in genetic populations (Blum & François, 2010; Csilléry et al., 2012). An inappropriate choice may result in poor posterior inference from the data (Robert et al., 2011; Aeschbacher et al., 2012). One solution proposed by Park et al. (2016) is to use the MMD metric between simulated and real data distributions to avoid manually selecting the summary statistics. However, as the threshold approaches zero, ABC tends to the standard Bayesian posterior, which is susceptible to model misspecification and lacks robustness (Dellaporta et al., " To address these two issues, generalized Bayesian inference (GBI) proposes allernative method by replacing the likelihood in the posterior distribution with the exponential of robust loss function.
For a detailed exploration, Jewson et al. (2018) offers varions examples, including the use of exponential of the Hellinger divergence within the GBL. Within the GBT framework, there are two prominent procedures that use
A detailed discussion is given in (Dellaporta al. 2022, Sections 2); however, we have provided some details in the A Appendix for easier access.
3

---

Published in Transactions on Machine Learning Research (09/2024)
the MMD loss. Chérief-Abdellatif & Alquier (2020) propose pseudo-likclihod based the MMD metric and approximate the posterior using variational inference. Pacchiardi & Dutta (2021) extend this method to a more general Bayesian likelihood-free model using stochastic gradient Markov chain Monte Carlo (MCMC) to perform posterior inference Stein variational gradient descent (Liu & Wang, 2016) is another variational inference method that uses gradient-based updales derived from Stein's method and kernel functions. It leverages Bayesian principles to iteratively move particles to match the posterior distribution by minimizing Kullback-Leibler divergence, particularly when exact inference is computationally intractable (Feng et al., 2017).
However, Dellaporta et al. (2022) noted that the performance of GBI is very sensitive to the choice of a learning rate and that there is no general heuristic for selecting this hyperparameter. Additionally, these calculations often require MCMC sampling methods, which can impose a significant computational burden. To address these issues, Dellaporta et al. (2022) developed an MMD posterior bootstrap procodure following the BNPL strategy developed in Lyddon al. (2018; 2019); Fong al. (2019). In this BNPL strategy, a BNP prior is defined on F, leading to BNP posterior on F, denoted by Fros. The key idea is that any posterior on the generator's parameter space W can be derived by mapping Fros through the push-forward mcasure
w (ppoon) == arg min 8(F .Faw), WEW
which visually depicted in Dellaporta et al. (2022, Figure 1). In particular, Dellaporta et al. (2022) considered pros as the DP posterior and 8 as the MMD measure.
The DP, introduced by Ferguson (1973), is commonly used prior in Bayesian nonparametric methods. It can be viewed as infiniteddimensionaal generalization of the Dirichlet distribution constructed around H (the base measure), fixed probability measure, whose variation is controlled by (the concentration parameter), a positive real number. To formally define the DP, consider a space E with a o-algebra A of subsets of X. For a base measure G on (2, and a > 0, a random probability measure F = (F(A) : A E A} is called a DP on A), denoted by peri := (F ~ Da H)), for every measurable partition AI...., A of X with k 2 2, the joint distribution of the vector (F(1)).. ..... F(A))) has the Dirichlet distribution with parameters (AH))).... aH (At)). It is assumed that (AA) 0 implics F(A,) O with probability one.
One of the most important properties of the DP is the conjugacy property-wher the sample X (X,...., X,) is drawn from F ~ Da H), the posterior distribution of F' given Xim, denoted by pres, is also a DP with concentration paramcter a + n and base mcasure
H" a(a - - + n(a + -
where Fn denotes the empirical cumulative distribution function (FCDF) of the sample Xim- Note that, Ht is convex combination of the base measure I and Fn. A guideline for choosing the hyperparameters and H for the test of equality distributions will be covered in Section 4.
In previous work, there are several BNP GOF tests (Al-Tabadi & Evans, 2018; Al-Labadi et al., 2021a;b), as well as two-sample tests (Al-Labadi & Zarepour, 2017; Al-Labadi, 2021) and a multi-sample test (Al-Labadi et al., 2022a), that are closely connected to the posterior- distance estimation employed in the BNPL procedure of Dellaporta et al. (2022). These methods are developed using different discrepancy measures to compare the distance between DP posteriors, placod on unknown distributions, with the corresponding one between DP priors. However, unlike our proposed method, none of them employ the MMD measure.
Sethuraman (1994) proposed an infinite series representation as alternative definition for DP. The con- struction of Sethuraman (1994) is known as the stick-breaking representation and is popularly used method in DP inference. Particularly, for a sequence of identically distributed (i.i.d.) random variables {8:)z2 from Beta(1, a), let w1 = 31, and w; = B, (1 83), for i 2 2. Then, the stick-breaking representation is given by FSB = Eie w;dy,, where (1)) is sequence of i.i.d. random variables from H. However, Zare- pour & Al-Labadi (2012) addressed some difficulties in using these representations. Meanwhile, Ishwaran &
2 A comprehensive list of other GBT procedures for addressing this issue can found Dellaporta al. (2022).
4

---

Published in Transactions on Machine Learning Research (09/2024)
Zarepour (2002) proposed finite representation to facilitate the simulation of the DP. Let
N J
P
~ Dirichlet(a/NN..... a/N), and Y, f.d. H. Ishwaran & Zarepour (2002) showed that N= converges distribution to F, where FN and F are random values in the space M1 (R) of probability measures on endowed with the topology of weak convergence. Thus, to generate {Ji,N put JN,N = / E Ti,N, where (i is sequence of ii.d Gamma(a/N, 1) random variables independent of [Yi]-=r- This form of approximation leads to some results in subsequent sections.
where
To determine the number of DP approximation terms, we apply a random stopping rule, inspired by the method described in Zarepour & Al-Labadi (2012). This rule, given a specific e (0, 1), is defined as:
<E <e)
inf j: ( Eis Tis
(4)
3 A Semi-BNP MMD Estimator
This section introduces our semi-BNP estimator for approximating the MMD measure. We consider a sce- nario where Fi represents completely unknown distribution, while F2 represents an intractable parametric distribution with complex generating process. For a given sample Yı,..., Y from F2 and by assuming FTT = (Fi ~ (a H)) for non-negative value and a fixed probability measure H, we propose the prior-based MMD estimator
2 m
MMD BNP (P F2.m)
E JuNJeNk(VE V.)
Je Y.) m Σ kYY,, Y.),
(5)
where (JI,N, .... JN,N) is sampled from Dirichlet(a/NN...... Vi,..., VN fid. H, and N is the number of terms in the DP approximation Σi Je,vov, proposed by Ishwaran & Zarepour (2002). Since we only impose the DP prior on the distribution of the real data, we refer to the approach as a semi-BNP procedure.
Theorem For a non-negative real value a and fized probability distribution H, let F = (Fi DP(a, H)), HN be the ECDF correspording H, and k(.,)) be any continuous kernel function with feature space corresponding to a universal RKHS defined on a compact metric space X. Assume that (b2zz')| < K, for any 2, z' Rdd. Then,
i. MMD BNP I F2,m) - MMD? (H F2.m), F2,
(FPri
DO,
2 it. E(MMD BNP (FPri N. F) + MMD? (H, (H. as + co, N + co, and m + 0o, iti. E(MMD BNP L,N, F.FF,N,F.... F2.m)) < (H, + 3K, for any N, m and a E Rt, where un.ai, " denotes the almost surely convergence, N denotes the natural numbers and R+ denotes the posilive real numbers.
After observing samples XI,...,XX Xn from Fi and considering Vi...., Vy Ltd. H", and (J.,....., Dirichlet (a ..... ) we update the prior-based MMD estimator (5) the posterior one as
N Σ f.tml
J. Vi) 2 J. YJ
MMD BNP (F F2,m)=
m² Σ k(Y Y.). (6)
where, H = a/(a + n)H + na + n. Fi,n; Fi,n denotes the empirical distribution of observed data, and Fr.00 refers to the approximation of F, X ~ DP(a + n, H²). The following Theorem presents asymptotic properties of MMD2 (Poos F2.m).
5

---

Published in Transactions on Machine Learning Research (09/2024)
Theorem For non-negative real value and fized probability distribution H, let F = (Fi Da H)) and :) be any continuous kernel function with feature space corresponding to universal RKHS defined on compact metric space X. Assume that 1k(zzz)) < K, for any z, E Rd. Then, for given sample XI...., from distribution Fi,
as a + xc (informative prior),
MMD I,N: 22 0.S., MMD2 F2,m), b. E (FT F2,m)) - MMD3 (H, N 00, and 30,
ii. (consistency).
a. MMD BNP (PT,N: F2.m) 0. (Fi,,,
E (PY,N F2.m)) - (F, as N - 00, n n 00, and m + 00.
We conclude this section by presenting a corollary that plays a significant role in the two following sections.
Corollary 3 Under the assumption of Theorem 2, a8 a xc, N 00, m + 0G, then,
a. E(MMD BNN (Fpri F2.m)) + 0, if and only if H = F2,
E(MMD (F F2,m.) + O, if and only if H = F22,
ii. for any choice of a and H, F(MMD ( (FI,N F2,m)) + O, if and only if Fi F2, as N - co, and - xo, and m 00.
Besides the theoretical result presented in this section, provided the density function of the posterior- based estimator (6) compared to the baseline (3) in Figure 10 in the Appendix. This comparison, made sample sizes approach infinity under the null hypothesis, helps to examine and understand the asymptotic distribution. The results indicated faster density convergence around zero for our proposed estimator.
4 Constructing a GOF Test with RB Ratio
In this section we introduce our novel semi-BNP test, utilizing the proposed estimator discussed in the previous section, to evaluate the hypothesis Ho: F1 = F2. Let the RKHS be universal and the sample space compact, we put forward an equivalent formulation to test the hypothesis
Ho (MD)(FF, O,
using the RB3 ratio, introduced by Evans (2015), as the Bayesian evidence.
By relating our problem to RB inference, with v = MMD? F2) and vo = 0, the RB ratio measures the change in belief regarding the true value of tho from a priori to a posteriori, given an observed sample XI...., Xn from Fj. It can be expressed by
(B FM (0|x1..)))
(8)
(MMD*(F,,r))())
where, (FI (-) and (FM denote the probability density functions (PDFs) of the estimators given by (6) and (5), respectively.
3 A detailed discussion the ratio provided in the Appendix.
4 Note that the subscript. (Fi, F2) may omitted whenever it. clear in the context.

---

Published in Transactions on Machine Learning Research (09/2024)
The density in the denominator of (8) must support Hg in order to reflect how well the data can support the null hypothesis based on the comparison between the prior and the posterior, utilizing the fundamental concepts of the RB ratio. Here, supporting Ho by MMDD= (-) means to place most of the prior mass on zero. To enforce this term on TMMDD it is enough to set H = F2 in DP(a, H), which is deduced from the Theorem 1, part (iii), to make the denominator of (8) concentrate most of its mass around zero; otherwise, any other choice for H contradicts the RB ratio rule in GOFs. In this case, when Hg is not true, for a fixed a and K (the upper bound of the kernel k), the range of ANP F2.m) should, on average, vary within a smaller range than its correspouding posterior version. Specifically, this range should be (0, 3K),, compared to (0, MMMD2(H* F2) + 3K) which can be similarly obtained for the posterior-based MMD estimator. This indicates that Ko should be rejected, as it is desirable. On the other hand, when Kg is true, although the prior- and posterior-based MMD estimators have approximately the same range of variation (0, 3K), Corollary 3(ii) implies that increasing the sample size leads the posterior to provide stronger evidence in favor of the mull hypothesis compared to the prior, resulting in the acceptance of Hg. However, the above discussion provides a general understanding rather than a tight inequality for comparison around zero. For accuracy, we use a critical value dig/M and the interval [0, d /M) ) to approximate the RB, addressed in (10).
With regards to choosing the concentration parameter a in our proposed test, we note that a controls the variation of per around H, which in turn controls the strength of belief in the truth of Kg- It is recommended to choose a n/2 based on the definition of H in pross (Al-Labadi & Zarepour, 2017). The idea behind using such a value of a is to avoid the excessive effect of the prior H on the test results by considering the chance of sampling from the observed data to be at least twice the chance of generating samples from H. Corollary 3(1) also elearly point to this issue in the informative prior case, as both expectations of MM (FT,,, F22m) and MMD (PT,NN F2.m) tend to 0 as a + oc, N + oo, and m + 0o, iff = F2. Hence, both prior and posterior densities in (8) should be heavily massed and coincide with each other at zero. It causes the value of (8) to become very close to 1, based on which no decision can be made about Hy.
For the proposed test, we will empirically choose to be less than n/2 and then compute (8). However, some computational methods in the literature have been proposed to elicit a that one may be interested in using (see Al-Labadi et al., 2022b; Al-Labadi, 2021). Generally, for a given a, Corollary 3(i) implics that MMDEN (FTNN F2.m) should be more dense than MMDDEN (FT,, F2.m) at 0 if and only if Kg is true. Hence, the value of (8) presents evidence for or against Ho, RBMMMD (0) x1=) > 1 or RBMMDA (0 x1m)) < 1,
respectively. Following Evans (2015); for & E Rt, , the calibration of (8) is defined as:
(9) where, - (.) is the posterior probability measure corresponding the density TMMMD<-(xx>>) When (7) is false, small value of (9) provides strong evidence against e/o, whereas large value suggests weak evidence against tho Conversely, when (7) is true, small value of (9) indicates weak evidence in favor of vo, while a large value suggests strong evidence in favor of vo. Particular attention should be paid here to the computation of (8) and (9). The densities used in (8) do not have explicit forms. Thus, we use their corresponding ECDF based on i' sample sizes to estimate (8) and (9), respectively, as
StraMD= xIm) = MMMD> (RBMMD= (t x1s)) S RB (0) |
RB RB (10, dis/Me) ) x(m) = fMMMD= (din//M/ X1:a)
(10)
AMMDD' (dho/ar)
StruMDe (10, drg/ar) xI.n) = Σ (a (xm)) xxm) flMMDs (de/Arl X1.n)),
(11)
where, D= (io sis M RB ( deets// x1s.))= S RBMMDA (10, du, /2x) (x1)))) }: in which M is a positive number, d.m is the estimate of dayM: the (i/M)-th prior quantile of (5),
RBMMDA (DMMMDe(IddJM.. (IdyjMs des1)/s)) (xim)) flmmne x1m) - fIMMDE (d X1:n)
(12)
f.MMD' (d - f.MMDD (d+)
Thiis interval is the simplified form of (0, MMD? Fb) + 3K), where MMD2 F2) is zero based on setting H F.
7

---

Published in Transactions on Machine Learning Research (09/2024)
and to in (10) is chosen that io/M is not too small (typically io/M = 0.05). Here di/M is the threshold value that the prior-based estimator (the semi-BNP statistic under the mull hypothesis) and the posterior-based estimator is compared against. Further details are available in Algorithm 1 and Figure 6 in the Appendix. For fixed M, as & + oc, then dry converges almost surely to di/M and (10) and (12) converge almost surely to RBMMD (1O, dio/ar)]Xxnnl and [BMM )x respectively. These are obtained from the left Riemann sum approximation and the relationship between CDFs and PDFs with considering
de/ar + e d(341)/M8 for any small 0, as follows:
d #
Sa
MMDD (f) dt
CTMMD> (d M (1.m)
RB (ld ;. di (x1nn)
~ RBMMD (di/M
d+t1 Ja TMMD=(t) #
CTMMD: (d
Consequently, (11) converges almost surly to StrMMD=> di ( xiin)x))== ED Sdups fdetay (t)x dt, where D' is defined similarly to D by replacing RB and days with RB and dayas in D. The following result from Al-Labadi & Evans (2018, Proposition 6) gives the consistency of the proposed test. II Kg is true, then -e 0 for in S i M - 1, as n - 0C, which implies that StrMMD= ((O, dio,/r) ) X1m) converges to almost surely. Otherwise, both RBMMDE /2./M) x1m)) and Str (10, / Xiss) converge to
(MM dlo/M))xx (=))) a.E., M/io(> 1) and RBMMDD (BMMDD((dd/M d((+))/)))
The proposed test is suggested to overcome several limitations present in its frequentist. counterparts. In frequentist test, for given permissible type I error rate denoted by a, the test rejects Hy if the value of MMD2 F1, F2) is greater than some threshold ca- The corresponding p-value for this test can also be computed by (F(MMDD(FF,, F2) 2 Ca Hoo) which leads the test to reject Hg if it is less than a. However, Li et al. (2017) noted that if (F, (Fi, F2) is not significantly larger than C, for some finite samples when Hg is not true, the null hypothesis Hg is not rejected. Furthermore, there is a trade-off between the permissible type I error rate a and the probability of failing to reject a false null hypothesis (type II crror), denoted by 3, as + < 1. Decreasing one error rale inevitably leads to an increase in the other, indicating that cannot arbitrarily drive to type I error rate to zero. Moreover, the p-values are uniformly distributed between 0 and 1 under the null hypothesis. In fact, it does not allow evidence for the null, which is one of their weaknesses compared to Bayesian criteria in hypothesis testing problems.
5 Embedding the Semi-BNP Estimator in GAN Learning
In this section, we propose BNPL. procedure that leverages posterior-based MMD estimator train GANs. It is inspired by the idea presented in Dellaporta et al. (2022) to approximate the posterior the generator's parameters.
5.1 Generative Adversarial Networks
The GAN (Goodfellow al., 2014) is a machine learning technique used to generate realistic-looking artificial samples. In this context, the discriminator D can be viewed as a black box that uses discrepancy measure 8 to differentiate between the real and fake data. Meanwhile, the generator Go is trained by optimizing simpler objective function, given by
arg min FG.), wEW
where FG.. represents the distribution of the generator. In fact, D attempts to continuously train Go by computing distance 8 between F and Fc., until this distance is negligible, making their difference indistin- guishable. This technique leads to omitting the neural network from D, whose optimization may lead to a vanishing gradient. An effective measure of discrepancy for 8 is the MMD, which is a kernel-based measure that offers several desirable properties such as consistency and robustness in generating samples (Gretton et al., 2012 2a; Chérief-Addellatit & Alquier, 2022).
8

---

Published in Transactions on Machine Learning Research (09/2024)
Numerous frequentist GANs applying the MMD measure to estimate the generator's parameters can be found in the literature. (Dziugaite et al., 2015; Binkowski et al., 2018; Li et al., 2015). These models are devised by comparing the generated fake samples with real samples. In addition to the MMD, several other discrepancy measures are commonly used for GANs, including the f-divergence measure (Nowozin et al., 2016), the Wasserstein distance (Arjovsky al., 2017), and the total variation distance (Lin et al., 2018). Nevertheless, the MMD kernel-based mcasure is remarkably robust against outliers and has the exceptional ability to capture complex dependencies in the data (Sejdinovie al., 2013; Chérief-Abdeellatif & Alquier 2022). This makes it highly effective in handling model misspecification and detecting subtle differences between distributions. This property is particularly useful for modeling complicated datasets such as images, which are a common application for GANs. Moreover, Al-Labadi et al. (2022a) used the energy distance to expand their procedure, which is member of the larger class of MMD kernel-based measures (Sejdinovic et al., 2013). From here, is obvious that choosing among a larger class can lead to designing more scnsitive discrepancy mcasures to detect differences.
Although particular case of the test in Al-Labadi et. al. (2022a) be used to compare two distributions, it cannot be easily used a discriminator in the minimum distance estimation technique to train GANs. In GANs, the objective is to update the parameter of the deterministic generative neural network Gw- Therefore, treating FG.. as an unknown distribution on which we place a BNP prior is nonsensical. Con- sequently, a more suitable distance criterion is required to compare an intractable parametric distribution with an unknown distribution.
5.2 Architecture
Various GAN architectures can be found in the literature to model complex high-dimensional distributions. However, we consider the original archilecture of the generalor network in Goodfellow et al. (2014). Specif- ically, we follow the neural network architecture in Goodfellow al. by setting the generator, Gw, to be a multi-layer neural network with parameters w, rectified linear units activation function for each hid- den layer, and sigmoid function for the last layer (output layer). The generator receives a noise vector U = (U,,..., -U, same distribution Fv.
Up) as its input nodes, where < d, and each element of U is independently drawn from the
Our discriminator follows the work of Dziugaite et al. (2015); Li et al. (2015) by optimizing the objective function:
arg min MMD WEW
BNP (FN (F FF..,))
but we now estimate the MMD using our semi-BNP method. In fact, our BNPL procedure implicitly approximates samples from the posterior distribution on the parameter w by minimizing the posterior- MMD estimator. For any differentiable kernel function k(,,-), this optimization is performed by computing the following gradient based on samples from F|X ~ DP(a + n, H*) as
AM ..G.m) Ow;
m ΣΣ
m
J(V Y.)
OY,
m
0Y, dw
Σ f
k(Y,, Yv)
Nm
where, Y;= = (U,), U, = (Un...., and Us's are generated from a distribution Fu, for = 1...., m. and i= .p. Then, the backpropagation method is applied for calculating partial derivatives aY. to update the parameters of Gw-
However, Li ct al. (2015, Equation 8) remarked that considering the square root of the MMD measure given by (2) in the cost function of frequentist GANs is more efficient than using (2) to train network Gw. They mentioned that since the gradient of | (Fx, with respect to w is the product of
71 2 2/MDD Fo and 72 AFM
then Yi forces the value of the gradient to be relatively
Ow
9

---

Published in Transactions on Machine Learning Research (09/2024)
large, even if both MMD"(FN (F..F and 72 small. This can prevent the vanishing gradient, which improves the learning of the parameters of Gw in the early layers of this network. We consider this point in order to improve our semi-BNP objective function:
arg min MMD (PN (FN .F..... WEW
(13)
Algorithm 2 in the Appendix provides steps for implementing the training.
Let w" be the optimized parameter of Gw that minimizes MMDBNN (FN. FG...). Since MMDDNN(FF mm can be viewed as semi-BNP estimation of (2), it becomes imperative to assess the accuracy of this estimation, specifically in terms of how effectively the proposed GAN generate realistic samples that faithfully represent the true data distribution (generalization error). Furthermore, it is crucial to take into consideration the generator's performance in dealing with outliers which includes a small proportion of observations that deviate from the clean data distribution Fo (robustness). The next lemma addresses these two concerns.
Lemma 4 Let W be the parameter space for Go and w E W be the value that optimizes the objective function (13) and w' be the true value that minimizes MMD(F, FG.). Assume that F DP(a, H) and let k(-,-) be any continuous kernel function with feature space corresponding to a universal RKHS defined on a compact metric space & such that 16(2,2') < K, for any z, E Rd. For a given sample XI....,X, from distribution F:
Generalization error:
2K 4aK E (MMD(F, Fc.-)) MMD(F, Fo..) + Vn a+1 n
(a+n+N)K + +2 (a+n+ 1)N
it. Robustness: Suppose there erist outliers in the sample data, which arise from noise distribution Q. Consider the Hüber's contamination model (Huber, 1992; Chérief-Abdellan & Alquier, 2022), given by (1-e)Fo+ Fo) FE, where e E (0, }) is the contamination rate, and the latent variables Zi...., Un Bernoulli(e) i.d.
are such that X; i.d.d Fo if Z4 = 0; otherwise, X; s.d.d. Q. Then,
2K
4aK
(a+n N)K (a+n+ I)N
E (MMMD(Fo FG..)) min Fa. + 4e
Vn a ann +
Lemma 4(ii) demonstrates that despite encountering outlier data, FG.- and Fo are negligibly different for sufficiently large sample size. This feature results in the majority of the posterior on the parameter space W being distributed on value w", which is a desirable outcome of the proposed method.
Although the precoding statements investigate propertics of the estimated paramcters by providing upper bounds for the expectation of the MMD estimalor, the next lemma presents stochastic bounds for the estimation error in order to assess the posterior consistency.
Lemma 5 Building upon the general assumptions stoted in Lemma 4, for a given sample XI...., X,, from distributton in the probabilily space (I, (C,A,Pr) Pr) and any é >> O, i. Pr (IMMD(FN) FG....m MMD(F, Fa 1 2 h(n, m. K,c) + | + A2)) S 2 exp 2k(n
-2 2m
i. Pr (MMD(F,, FF. >>) MMD(F, + 2K + a +2
4aK
aa N
Vn
(a+ 1)N
where, h(n m. K) 22K(K vm)/ A LMDDFFRO .FG.)) MMD(FM F,,, Fo and Ag MMD(F, Fo..) MMD(FF, ).
A direct consequence of Lemma 5(ii) is that for a fixed value a, Pr(MMD(F, and N + co, for any > O, when MM = 0 (well-specifedd case). This implics FG.. converges in probability to the data distribution F as the sample size increases in well-specifted cases.
xc
10

---

Published in Transactions on Machine Learning Research (09/2024)
The choice of a in the test proposed in Section 4 plays a crucial role in determining the degree of support for the null hypothesis against the alternative. In the context of approxinating the posterior on the parameter space, the prior choice for F' and determining the strength of belief becomes challenging. We consider a small value for a as a non-informative prior, following the suggestion by Dellaporta et al. (2022), thanks to its broad ability to characterize uncertainty (Terenin & Draper, 2017). However, it's important to note that setting a = 0 as done by Dellaporta et al. (2022) is not always well-defined mathematically, as the DP is only defined for a 0. Therefore, we opt for 10-6.
The main distinction between our ENPL method and the proposed by Dellaporta et al. (2022) lies in the fact that we generalize their BNPL procodure beyond estimating parameters and explicitly consider the terms of the DP posterior approximation and their corresponding weights. Dellaporta et al. used the following DP approximation:
F+N Σ Jen +
Σ JuNdv.
where (Jim, JI.N.N) Dirichlet(1,.... a N..... N), (X1m) lid. F, and (VI:N) iid. H. In contrast, we Our approach offers an advantage over the approximation used in Dellaporta et al. (2022) due to its reduced number of terms, significantly reducing both compulational and theoretical complexity, where our method scales O(N) and Dellaporta et al. scales O(N + n)) Additionally, a further difference is that Dellaporta's bootstrap procedure needs to query the loss function B times to simulate B posterior paramcters, whercas our procodure does not require bootstrap algorithm and we only need to simulate a single parameter. Although their bootstrap procedure is embarrassingly parallelizable, the number of bootstrap samples generally should be fairly large number and the typical statistical practitioner does not have access to B cores to truly parallelize the additional cost of bootstrap sampling.
employ FR = Ei=1 J' with (Ji.N.)) ~ Dirichlet atn ...., N.).,,) =t").
5.3 Kernel Settings
In our method, we choose to use the standard radial basis function (RBF) kernel as its feature space corresponds to a universal RKHS. For a comprehensive understanding of RBF functions, refer to Section D in the Appendix. Dziugaite et al. (2015); Li et al. (2015) and Li et al. (2017) used the Gaussian kernel in training MMD-GANs because of its simplicity and good performance. Dziugaite et al. (2015) also evaluated some other RBF kernels such as the Laplacian and rational quadratic kernels to compare the results of the MMD-GANs with those obtained based on using Gaussian kernels. They found the best performance by applying the Gaussian kernel in the MMD cost function.
Hence, we consider the Gaussian kernel function in our proposed procedure. To choose the bandwidth parameter we follow the idea of considering a set of fixed values of a's such as orT), then compute the mixture of Gaussian kernels k(.,-) = Σ kG., (+-)), to consider in (6). For each o (t)), 0 S kc.,(.)) S 1: hence, 0 & k(::)) < T, which satisfies the theoretical results presented in the paper. As it is mentioned in Li et al. (2015), this choice reflects good performance in training MMD-GANs.
6 Experimental Investigation
In this section, we empirically investigate our proposed methods through comprehensive numerical studies in the following two subsections, which demonstrate the superior performance of our proposed semi-BNP test as a standalone test as well as an embedded discriminator for the semi-BNP GAN 6
6.1 The Semi-BNP Test
To comprehensively study test performance evaluation, we consider some major representative examples in two-sample comparison problems. For this, let Yn, be a sample generated from F3 = (O,, and
oTe relevant codes for the Bemi-BNP procedure available
11

---

Published in Transactions on Machine Learning Research (09/2024)
X1 ................................ ..,,,............ be an observed sample generated from each below distributions: F1 = NO... Ia) (No differences), F1 = 5d. Ta) (Mean shift), Fj (Oa, Ba) (Skewness), Fi = 2 1 Ta) + 2 (1a,, Ia) (Mixture),
F = (O,, 21a) (Variance shift), F1 = f3 Oz. Ia) (Heavy tail), and Fi = LG (Kurtosis).
To implement the test, we set l = 1000, M = 20, and <= 10-3 to be used in Algorithm 1 in the Appendix. We first considered the mixture of six Gaussian kernels corresponding to the suggested bandwidth parameters 2. 5, 10, 20, 40, and 80 by Li et al. (2015). We found that although this choice can provide good results in training GANs, it does not provide satisfactory results in hypothesis testing problems.
Instead of using mixture of several Gaussian kernels, we propose choosing specific value for the bandwidth parameter that maximizes the area under the receiver operating characteristic curve (AUC) empirically.
In a binary classifier, which can also be thought of as a two-sample test assessing whether two samples are distinguishable or not, the receiver operating characteristic (ROC) curve is a plot of true positive rates (scnsitivity) against the false positive rates (1-spocificity) based on different choices of threshold to display the performance of the test. The positive term refers to rejecting Ho in (7), while, the negalive term refers to failing to reject Kg. The false positive and false negative rates are equivalent to type I and type II errors, respectively. Hence, a higher AUC indicates a better diagnostic ability of a binary test?.
The ROC curves and AUC values of the synthetic examples are provided in Figure I for the sample size n = 55, d = 60, a = 25, and various values of the bandwidth parameter, including the median heuristic оMH. The red diagonal line represents the random classifier. An ROC curve located higher than the diagonal line indicates better test performance and vice versa. It is obvious from Figure 1 that the best test performance (AUC 1) is first achieved for the bandwidth parameter 80. For this o, the AUC jumps from zero to 1 as either Ho is strongly rejected by too small values of RB (near to O, the minimum RB) in alternative experiments or Hg is strongly accepted by large values of RB (near to 20, the maximum RB) in null experiments. As noted in the Appendix, if a is too small or too large, the MMD approaches zero, resulting in poor performance, as shown in Figure 1. In the context of this paper's semi-BNP test, both the prior- and posterior-based MMD estimators converge lo zero under these conditions, causing their correspouding density functions to coincide at zero, resulting in RB and rendering the test unable to evaluate Hg- Another test of interest is to assess the effect of different hyperparameter settings for a and H through simulation studies to follow our proposed theoretical convergence results. To do this, we generate 100 60- dimensional samples of sizes n = 50 from both F1 = (0 Aoo) and F = NOso, Loo) and represent the result of the semi-BNN test by Figure 2 for two choices of the base measure H (H = F2 and H = LG(O0oo Io0)) and various values of a (a I,.., 1000). In this figure, the solid line represents the average of the RB and the filled area around the line indicates a 95% confidence interval of the RB over the 100 samples. Figure 2-a clearly shows that by choosing H + F2, the test wrongly accepts the null hypothesis. It is because the prior does not support the null hypothesis mentioned earlier when presenting the RB ratio in Section 4. On the other hand, when H = F2, Figure 2-b shows good performance for the test at a = n/2. Failing to reject Ho for small values of a is due to the lack of sufficient support from the null hypothesis by the prior. We remark that the value of a determines the concentration of the prior FPri around H, thus it is obvious that for small values of a, the test does not perform well. It should also be noted that for any choices of H in Figure 2, the ability of the test evaluate the null hypothesis is reduced by letting a go infinity, which can be concluded by Corollary 3(i).
Now, to conduct more comprehensive investigation, we present the average of RB and its relevant strength over the 100 samples in Table 1 for n 30, 50. Furthermore, we present the results of the BNP- BNP-energy test by Al-Labadi et al. (2022a) in Table 1, which demonstrate its weak performance in certain scenarios. Additional results in the power comparison can be found in Section F,1 of the Appendix.
Since we cousider fo/M- 0.05 estimate the RB ratio, the values of RB can vary between 0 and 20. Therefore, in computing the AUC for the BNP test, the threshold should vary from 20. More details for plotting the ROC and computing the AUC are provided by Algorithm in the Appendix.
12

---

Published in Transactions on Machine Learning Research (09/2024)
Table 1: The average of RB, the average of its strength (Str ), and the relevant AUC out of 100 replications based on using = 25, f = 1000, M =200+ 10-3 in (4), and bandwidth parameter = 80 in RBF kernel for two sample of data with n = 30, 50.
BNP
FNP
MMD
Energy
MMD
Energy
Example
RB(Stt)
AUC
(Stx)t)
AUC
Panduue
AUC
Pallue
AUC
30
50
30
50
30
50
30
50
30
50
30
50
30
50
30
54
No dillerences
(.U6((0.6)) (Z)
2.41(0.67)
(.76)
1.910.55)
0.50
0.46
0.30
0.49
5
4.06(0.77)
6.91(0.76)
(.99)).73) (.)6()
0.18
0.50
0.51
0.52
(0.778
10.74(0.79)
10.42(0.76) 6.92(0.67)
0.50
0.51
0.54
0.47
20
9.62(0.80)
(6.02(0.83)
11.76(0.78) 8.24(0.73)
0.16
0.50
0.51
0.50
13.07(0.885)
(5(097))
(1.56(0.75)
17.39(0.84)
0.61
0.49
0.53
0..4
(4.09(0.57)
19.71(1)
(8.38(0.81)
18.51(0.95)
0.52
0.16
0.50
0.18
16.2(0.89)
19.57(1)
14.16(0.87)
19: 18:10(1)
0.46
0:47
0.53
0.56
100
15,83(0.91)
19.74(1)
11.81(0.92)
19:31(1)
0.11
0.16
0.19
0.55
Moan shift
0.76(0.24)
0.40(0.09)
0.82
0.56
0.87(0021))
0.45(0.11) 0.87
0.50
0.15
0.05
0.56
0.91
00.19
0.12
0.79
0.36
0.21 (0.03) 0.07(0)
0.99
0.99
0.00(0.01) (.28(00.0)
0.98
0.01
0.002
0.98
0.02
0.004
0.97
0:97
1D
0.10(0.D1)
0.05(0)
0.17(0.05)
0.02((0) D.98
0.001
0.001
0.006
0.004
0.98
0.09(0.01)
0(0)
0.09(0.01)
0(0)
0.001
0.001
0.004
0.004
40
0.1l6(0)
O(D)
0.06(0
0(0)
0.001
0.001
0.004
0.004
0.09(0.03)
0(0)
(.07(00004)
D(0)
0.001
0.001
0.004 0.004
80
0.106(0.02)
0(0)
(.05())035)
D(0)
0.001
0.001
0.004
0.004
100
0.04/0.01)
0(0)
(.03(0)
0.001
0.001
0.004
0.004
Skewness
I
0.01 (0)
0(0)
0.99
(.07(0)
D(0)
0.99
0.009
0.001 0.98
1
0.007
0.00L 0.94
0(D)
0(0)
0(0)
D(0)
0.001 0.001
0.001
0.00L
10
0(00]
0(0)
0(0)
0(0)
0.001 0.001
0.004
0.00L
1
0(0)
0(0)
0(0)
D(0)
0.001 0.001
0.001 0.00L
40
0(00
0(0)
0(0)
0(0)
0.001 0.001
0.001 0.004
0(D)
0(0)
((0)
D(0)
0.001 0.001
0.004
0.004
80 0(0)
0(0)
1
0(0)
D(0)
0.001
0.001
0.004
0.004
100 Q(D]
0(0)
010]
(F0)
0.001
0.001
0.004
0.004
Mixture
1
(.06(()
0(0)
0.5KJ
0.97
(011
(.04(0)
0.97
0.43
0.38
0.58
0.57
01221
0.17
0.00
D.ST
5 0(0)
((0)
1
0(0)
D(O)
0.15
0.09
0.84
0.91
0.06
0.01
0.95
1D 0(0)
0(0)
0(0)
D(0)
0.03
0.007
0.95
0.38
0.022
0.007
0.96
20 0(0)
0(0)
0(0)
D(0)
0.002
0.001
0.96
1
0.01
0.006
1
40
o(0)
0(0)
0(0)
D007
0.001
0.001
0.01
0.006
[(0)
((0)
0(0)
D(0)
0.001
0.001
0.006
0.009
0(0)
((0)
0(0)
D(0)
0.001
0.001
0.008
0.006
100 0(0)
((D)
0(0)
((0)
0.001
0.001
0.001
0.006
1
1
Variance shift
0.87(0.29)
(.85(0.19)
0.71
0.83
1.10(0.36))
(.0800333)
0.53
0.46
0.54
0.77
5
0.56(0. 0.55(0.12)
15)
0.99 0.99
(.0((0.35)
(.99()
0.80 0.98
0.34
0.20
0.65
0.80
0.20
0.07
0.82 0.90
(.44(0.11)
0.27(0005)
0.99
(.87(0)22)
(.20(0
0.97
20
0.34(0.07)
0.08(0)
0.65(0.17)
0.60(0.13)
0.90
0.01
0.001
0.95
1
0.03
0.006
0.95
0.13(0.01)
0.02(0)
(.67
0.08(0.14)
0.001
0.001
0.004
60
0.12(0.01)
0.01(0)
0.47(0.10)
0.45(0.11)
0.001
0.001
0.008
0.004
1
(.17(0.01)
0(0)
0.54(0.12)
0.47(0.11)
0.001
0.001
0.005
0.004
100
0.14(0.01)
0(0)
1
1
(.40(0..8) 0.45(0.10)
0.001
0.001
1
0.004
0.001
1
1
Haavy cail
0.98(0.28)
0.66(0.20)
0.89
0.92
1.19(0.41)
1.10(0.38)
0.70
0.78
0.43
0.57
0.56
0.59
0.02
5
(.32(0.06)
0.37(00.8)
0.99 0.99
(.77(0.24)
0.78(0.23)
0:53
0.99
0.20
0.11
0.79
0.80
0.00
0.006
0.97 0.90
0.35(0.06)
0.13(0.02)
0.99
(0616)
0.68(0.19)
0.96
0.007
0.92
0.96
0.90 0.97
20
0.15(0.02)
0(0)
0.48
0.46(0.12)
0.002
0.001
0.96
1
0.02
0.005
0.96
0.07(0.01)
0(0)
0.18(0004) 0.25(0.04)
0.001
0.001
0.005
0.001
60
0.02(0))
0(0)
0.14((.01) 0.22().03)
0.001
0.001
0.004
0.004
1
(.01(0)
0(0)
0.13(0.01)
0.15(0:02)
0.001
0.001
0.001
0.00L
100
0.04(0)
0(0)
0.14(0.01)
(.09().01)
0.001
0.001
0.004
0.004
1
Kurtasis
(0.17)
0.1910.04)
0,80
0.388
1.00(0.37)
0.28(0.28)
0.77
0.00
01.28
0.23
0.74
0.72
0.18
0.11
0.70 D.88
5
0.16(0.03)
0.06(0.01)
1
1
0.63(0.18)
(.40(0.00)
0.96
0.99
0.04
0.01
0.94
0.98
0.033
0.008
0.97
0.96
0.02(0)
a(0)
1
0.35(0.08)
(.06)
0.97
0.001
0.001
1
01007
0.001
0.96
1
20
O(0)
0(0)
1
0,20(0.03)
(.18(
0.001
0.001
1
0.004
0.004
1
0(0)
0(0)
1
0.06(1.01)
(.0G())
0.001
0.001
1
0.001
0.001
1
1
60 0(0)
0(0)
1
0,05(0)
(.)
0.001
0.001
0.004
0.004
80
0(0)
0(0)
1
0.05(0)
0.03(0)
0.001
0.001
1
1
0.001
0.001
1
1
100 0(0)
0(0)
1
1
(.02(0)
BK0
0.001
0.001
1
0.004 0.004
1
13

---

Published in Transactions on Machine Learning Research (09/2024)
0.045
0,65
AUC.
DJ645
T7A Lenee MhP
0.2
AUton 0.324
AUCa, 00.0016
31
G4154
AUCA
AUCes
0.4
Allos 1.099
AUCo
AUCa
-0 0..008
1344
AUC 0.0676
00
82
04
03
2.0
60
6.
6.
19
33
02
85
300
(a) y1 Yv ~ (0.5500 Teo)
(b) y1.... Yn (060, 2T60)
(c) y1.....y
Yn ~ to 10 T80)
RGC Curoe (Hieur))
(Sonanent
ROC Curve Burtec))
AUG,, 0.016 FAUC, D.0JM
0.6156
0.4
D.0132
0.5924
AUC
AuCa #1
AuCo
AuCo
AUC
AUCa d 0.892
0.2
AUC
1176
AUCa
32
AU 3455
42
64 (pgccce)
6.4 Spseediiyyyy
L2
33
HM (teechet)
(d) y1...r Yn (1.o Iso)
0.5N( 16o, Iso)
(c) Yls Yn (O Boo)
(f) yl Yn LC(Ooo. Iua)
Figure 1: The ROC curves and AUC values of the BNP-MMD test for X1,..., Xn N ((66 Iso), using range of bandwidth parameters including 2, 5, 10, 20, 40, 80, 102, 10 10 as well as the median heuristic MMH.
20.0
175
17.
150
125
100
75
7.5
50
2.5
0.0
00
400
600
800
1000
200
000
1002
(a) H = LC(Oan. Iso)
(b) F2
Figure 2: The solid line represents the average of the RB and the pink area represents a 95% confidence interval of the RB over the 100 samples with various choices of H and a for the heavy tail example. The lower and upper bounds are the 2.5% and 97.5% quantiles of the RB, respectively. The red dotted line represents RB- = I.
To compare the BNP and FNP tests, the p-values of the frequentists counterparts corresponding to each Bayesian test are presented in Table using the R packages energy and maotai AUC values of all tests are also given to facilitate comparison between tests. Generally, the proposed test reflects better performances than its frequentist counterparts in lower dimensions. For instance, in the variance shift example, when = 5 and n = 30, the average of the RB and its strength for the Bemi-BNP-MMD test are 0.55 and 0. 12, respectively, which shows strong evidence reject the null. While the average of the p-value corresponding to the MMD frequentist test is 0.34, which shows failure to reject the mull hypothesis. The AUC value of the semi-BNP test is also 0.99 which indicates a better ability than its frequentist counterpart with an AUC
$nttpe://CRAN R-- org/pacckge-eeerrg Battps://cRAN.R-preee te-ppoject.org/pcceeee.... wtt.:pccaagemaatt.......oo
14

---

Published in Transactions on Machine Learning Research (09/2024)
of 0.65. To examine the large sample property, additional results for n 500, 1000 are presented in Section F.1 of the Appendix, revealing the relatively poor performance of the BNP-Energy test in comparison to other tests.
6.2 The Semi-BNP GAN
According to the results reported in the previous subsection, the semi-BNP estimator suggests a test that outperforms other competing tests in many scenarios. Therefore, we expect that embedding this estimator in GANs as the discriminator will accurately distinguish real and fake data. We use the database of handwritten digits with 10 modes, bone marrow biopsy histopathology, human faces, and brain MRI images to analyze the model performance. Following the design choices of Li et al. (2015), we use the Gaussian neural network for the generator with four hidden layers each having rectified linear units activation function and a sigmoid function for the output layer. For fitting a deep neural network, there are numerous methods to choose network parameters. Furthermore, we select the number of nodes in hidden layers and tuning parameters of the network using Bayesian optimization (Snoek et al., 2012). We also set mini-batch sizes to be Rymb 1, 1,000 and use mixture of six Gaussian kernels corresponding to the bandwidth parameters 2,5, 10, 20, 40, and 80 to train networks discussed in this section.
6.2.1 MNIST Dataset (LeCun, 1998):
The MNIST dataset includes 60,000 handwritten digits of 10 numbers from 0 to 9 cach having 784 (28 x 28) dimensions. This dataset split into 50000 training and 10000 lesting images and is a good example to demonstrate the performance of the method in dealing with the mode collapse problem. We use the training set to train the network. A sample from the training MNIST dataset is shown in Figure 3-a. Following I'm = 40, 000 iterations, we generate samples from the trained semi-BNP GAN using Algorithm 2 from the Appendix, as depicted in Figure 3-b. The results of Li et al. (2015) are also presented by Figure 3-c as the frequentist counterpart of our SeM procedure 10, Based on these preliminary results, we can see that our generated images can, at least, replicate the results of Li et al. (2015) and in some cases produce sharper images. This result can also be deduced from the presented values of certain score functions in Section F.2 of the Appendix. On the other hand, unlike the scmi-BNP test, our experimental results demonstrate that the
2
2
8
5
4
1
/
6
2
F
2
}
5
7
O
4
/
9
3
9
6
0
8
0
8
0
8
4
5
6
7
8
9
7
3
6
7
9
9
0
1
5
4 7 3 7 7 3
0
5
5
2
4
3
2
3
5
0 8
4
8
9
7
6
6
3
0
4
3
9
8
6
2 4 6 4 / 8 0
96
7
0
7
8
1
6
6
4
9
/
2
9
5
8
3
4
7
7
9
03
6
4
/
4
4
3
4
/
$
2
5
7
2
7
4
9
3
0
4
6
5
4/
632
8
8
6
6
3
5
9
7
3
4
6
6
5
9
/
2
8
4
7
2
6
8
1
4
2
3
8
9
9
7
4
9
5
0
7
9
9
3
/
4
2
7 7
3
9
4:
3
5
2
4
1
0
2
4
4 7
6 7
0
/
/
3
8
7
5
3
6
2
2
6
6
/
8
9
3
8
0
8
1
9
4
8
6
9
8
1
3
5
9
0
4
9
2
5
8
8
9
3
5
2
9
9
5
6
2
3
4
i
0
5
3
(a) Training data
(b) Bemi-BNP-MMD GAN
(c) FNP-MMD GAN
Figure 3: Generated samples of sizes (10 x 10) from Scmi-BNP-MMI and MMD-FNP GAN for the MNIST dataset using a mixture of Gaussian kernels in 40,000 iterations.
semi-BNP GAN, using a mixture of Gaussian kernels, outperforms the approach that considers only a single Gaussian kernel. To investigate this matter further, we present several samples of the trained generator using a Gaussian kernel with different values of a, as well as the median heuristic оMH, in Figure 4. Note
10 code for the GAN proposed by Li et al. (2015) is available Generative- -eeeettive-ooee---aaeeeae Konentt-attchiin--eeeoo Netoonn--tttoorr--tttteee master.zipffllle subpath--/PaRAADN....
15

---

Published in Transactions on Machine Learning Research (09/2024)
that the value of оMH is updated in each iteration, and therefore, specific value is reported for it in this figure. While increasing the value of enhances the diversity of the generated images, it is evident that the resolution of the images in Figure 4 does not reach the image quality achieved by the mixture kernel.
In contrast to using MMD kernel-based measures, it may also be interesting to consider the energy distance in learning GANs from a BNP perspective. To address this concern, we embed the two-sample BNP-energy test of Al-Labadi et al. (2022a) in training GANs a discriminator and show the generated samples in Figure 5-a. This image clearly shows the inefficiency of the two-sample BNP test of Al-Labadi et al. (2022a) in training the generator. The main issue in this test procedure is treating FG.. as unknown distribution to place a DP prior on it which is contrary to update paramcter w in the paramcterized generative neural network G
One may also be interested in considering the semi-BNP-enn energy procedure learning GANs in which it may be more sensible to compare the ScMI-BNP-MMD results. To do this, we use the energy distance instcad of the MMD in Algorithm in the Appendix. The results are presented in Figure 5-b and show blurry and unclear images with no variety, which reflect the inefficiency of using the energy distance compared to the MMD kernel-based measure. More experiments are given in Section F.2 of the Appendix.
7
7
8
1
/
3
8
7
3 2
6
3
3
3
8
6
6
3
44
9
5
9
(a) 2=
(b)
0=
(c)
10
(d)
20
5
9
7
8
-
8
6
7
0
3
5
9
/ 2 0
4 /
2
7
2
4
1
2 Q
0
4
C
5
O
6
8
6
2
8
2
.2
3
0
9
4
2
6
7
2
9
1
S
7
4
4
8
3
3
7
/
4
7
(c)
a
40
(f) a 80
(g)
OMH
Figure 4: Generated samples from Semi-BNP-MMIT for the MNIST datasct using a single Gaussian kernel with various values of bandwidth parameter in 40,000 iterations.
(a)
(b)
Figure 5: Generated samples from BNP-Energy GAN (a) and semi-BNN- BNN- BNP--nergy GAN (b) for the MNIST dataset in 40,000 iterations.
7 Conclusion
Our semi-BNP approach effectively estimates the MMD measure between an unknown distribution and an intractable parametric distribution. It outperforms frequentist counterparts and even surpasses a recent BNP competitor in certain scenarios (Al-Labadi et al., 2022a). This approach shows great potential in training GANs, where the proposed estimator serves as a discriminator, inducing a posterior distribution on the generator's parameter space. Stick-breaking representation lacks normalization terms and exhibits stochastic
16

---

Published in Transactions on Machine Learning Research (09/2024)
decrease, making it inefficient for simulations (Zarepour & Al-Labadi, 2012). Thus, exploring alternative DP approximations for MMD) estimation presents an intriguing avenue for future research. Future work will focus on generating 3D medical images to further enhance results.
References
Simon Aeschbacher, Mark A. Beaumont, and Andreas Futschik. A novel approach for choosing summary statistics in approximate Bayesian computation. Genetics, 192(3):1027-1047, 2012.
Luai Al-Labadi. The two-sample problem via relative belief ratio. Computational Statistics, 36(3):1791-1808, 2021.
Luai Al-Labadi and Michael Evans. Prior-based model checking, Canadian Journal of Statistics, 46(3): 380-398, 2018.
Luai Al-Labadi and Mahmoud Zarepour. Two-sample Kolmogorov-Sminan test using Bayesian nonpara- metric approach. Malhematical Methods of Stolislics, 26(3):212 225, 2017.
Luai Al-Labadi, Forough Fazeli Asl, and Zahra Saberi. A Bayesian semiparametric Gaussian copula. approach to multivariate normality test. Journal of Statistical Computation and Simulation, 91 91(3):5444 :543 563, 2021a.
Luai ALLabadi, Forough Fazeli Asl, and Zahra Saberi. A necessary Bayesian nonparametric test for assessing multivariate normality. Mathematical Methods of Stalistics, 30(3-4):64 81, 2021b.
Luai Al-Labadi, Forough Fazeli Asl, and Zahra Saberi. Bayesian nonparametric multi-sample test any dimension. ASIA Advances in Slalislical Analysis, 106(2):217 242, 2022a.
Luai Al-Labadi, Forough Fazeli Asl, and Zahra Saberi. A test for independence via Bayesian nonparametric estimation of mutual information. Canadian Journal of Statistics, 50(3):1047-1070, 2022b.
Luai Al-Labadi, Ayman Alzaatreh, and Michael Evans. How to measure evidence: Bayes factors or relative belief ratios? arxiv preprint ar Xiv:2300 008999, 2023.
Varghese Alex, Mohammed Safwan K.P., Sai Saketh Chennamselly, and Ganapathy Krishnamurthi. Gener- ative adversarial networks for brain lesion detection. In Medical Imaging 2017: Image Processing, volume 10133, pp. 113-121. SPIE, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. Tn International Conference on Machine Learning, pp. 214 223. PMLR, 2017.
Mark A. Beaumont. Approximate Bayesian computation. Annual Review of Statistics and its Application, 6(1):379-400) 2019.
Mark A. Beaumont, Wenyang Zhang, and David J Balding. Approximate Bayesian computation in popula- tion genetics. Genetics, 162( 2035, 2002.
Ayush Bharti, Masha Naslidnyk, Oscar Key, Samuel Kaski, and François-Xavaa Xavier Briol. Optimally- weighted estimators of the maximum mean discrepancy for ikclihood-free inference. arXiv preprint arXiv:2301.11674, 2023.
Mikolaj Binikowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018.
Michael G.B. Blum and Olivier François. Non-linear regression models for approximate Bayesian computa- tion. Statislics and Compuling, 20:63 0663-73,4 2010.
Karsten M. Borgwardt and Zoubin Ghahramani. 4rXiv:0906.4032.1 2009.
Bayesian two-sample tests.
arXie preprint
Francois-Xavier Xavier Briol, Alessandro Barp, Andrew B Duncan, and Mark Girolami. Statistical inference for generative models with maximum mean discrepancy. arXio preprint arXi:: 006.05944. 2019.
17

---

Published in Transactions on Machine Learning Research (09/2024)
Badr-Eddine Chéricf-Abdellatit and Pierre Alquier. MMD-Bayer Robust Bayesian estimation via maximum mean discrepancy. In Symposium on Advances in Approsimate Bugesion Inference, pp- 1-21. PMLR, 2020.
Badd--Ediiee Auérief-Abdellatte and Pierre Alquier. Finite sample properties of parametric MMD estimation: Robustness to misspecification and dependence. Bernoulli, 28(1):181-211 2022.
Katalin Csilléry, Olivier François, and Michael G.B. Blum. abc: R package for approximate Bayesian computation (ABC). Methods in Ecology and Evolution, 3(3):475 479, 2012.
Charita Dellaporta, Jeremias Knoblauch, Thoodoros Damoulas, and François-Xavin Xavier Briol. Robust Bayesian inference for simulator- models via the MMD posterior bootstrap. In International Conference on Artificial Intelligence and Statisties, pp. 943-970. PMLR, 2022.
Gintare Karolina Dziugaite, Daniel M. Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mcan discrepancy optimization. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, pp. 258-267, 2015.
Michael Evans. Measuring stalistical evidence using relative belief. CRC Press, Boca Raton, FL, 2015.
Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized Stein variational gradient descent. Xir preprint arXiv: 707.06626, 2017.
Thomas S. Ferguson. A Bayesian analysis some nonparametric problems. Annals of Statistics, 1(2): 209 230, 1973.
Edwin Fong, Simon P. Lyddon, and Chris C. Holmes. Scalable nonparametric sampling from multimodal posteriors with the posterior bootstrap. In International Conference on Machine Learning, pp. 1952-1962. PMLR, 2019.
Gonzalo García-Donate and Ming-Hui Chen. Calibrating Bayes factor under prior predictive distributions. Statistica Sinica, 15(2):359-33)) 3800, 2005.
Marc G. Genton. Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research, 2(Dec):299 312, 2001.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farloy, Sherjil Ozair, Aaron
Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Newral Information Processing Systems, 27:2672-2688 2014.
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. A kernel two-samplc test. Journal of Machine Learning Research, 13(1):723-773, 2012a.
Arthur Gretton, Dino Sejdinovic, Hciko Strathmann, Sivaraman Balakrishnan, Massimiliano Pontil, Kenji Fukumizu, and Bharath K Sriperunbudur. Optimal kernel choice for large-scale two-sample tests. Ad- vances in Neural Information Processing Systems, 25, 2012b.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to a local Nash equilibrium. Advances in Neural Infor- mation Processing Systems, 30, 2017.
Chris C. Holmes, Franccois Caron, Jim E. Griffin, and David A. Stephens. Two-sample Bayesian nonpara- metric hypothesis testing. Bayesion Analysis, 10:297 320, 2015.
Gary B. Huang Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In Workshop on Faces in 'Real- Life' Images: Detection, Alignment, and Recognition, 2008
Peter Huber. Robust estimation of location parameter. Breakthroughs in Statistics: Methodology and Distribution, pp. Springer, 1992.
18

---

Published in Transactions on Machine Learning Research (09/2024)
Hemant Ishwaran and Mahmoud Zarepour. Exact and approximate sum representations for the Dirichlet process. Canadian Journal of Statistics, 30(2):269 283, 2002.
Harold Jeffreys. Theory of Probability. Clarendon Press, Oxford, third edition, 1961.
Jack Jewson, Jim Q Smith, and Chris C. Holmes. Principles of Bayesian inference using general divergence criteria. Entropy, 20(6):442, 2018.
Wittawat Jitkrittum, Zoltán Szabó, Kacper P. Chwialkowski, and Arthur Gretton. Interpretable distribution features with maximum testing power. Advances in Neural Information Processing Systems, 29, 2016.
Robert. E. Kass and Adrian E. Raftery. Bayes factors. Journal of the American Statistical Association, 90 (430))773-79hh 1995.
Oscar Key, Tamara Fernandez, Arthur Gretton, and François-Xavier Briol. Composite goodness-of-fit tests with kernels. arXiv preprint arXiv:2111.1027, 2021.
Yann LeCun. The MNTST database of handwritten digits, 1998. URL http:://yamm.eecu..om/// mnist.
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMD-GAN: Towards deeper understanding moment matching network. Advances in Neural Information Processing Systems, 30, 2017.
Yujia Li, Kevin Swersky, and Richard Zemel. Generative moment matching networks. In International Conference on Machine Learning, pp. 1718-1727. PMIRR 2015.
Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. PacGAN: The power of two samples generative adversarial networks. Advances in Neural Information Processing Systems, 31, 2018.
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose Bayesian inference algo- rithm. Advances in Newral Information Processing Systems, 29, 2016.
Qiang Liu, Jason Lee, and Michael I. Jordan. A kernelized Stein discrepancy for goodness-of of-fit tests. PMTRR 2016.
International Conference on Machine Learning, pp.
Simon P. Lyddon, Stephen G. Walker, and Chris C. Holmes. Nonparametric learning from Bayesian models with randomized objective functions. Advances in Neural Information Processing Systems, 31, 2018.
Simon Lyddon, Chris C. Holmes, and Stephen G. Walker. General Bayesian updating and the loss-
likelihood bootstrap. Biometrika, 106(2):465-478, 2019.
Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in Applied Probabilily, 29(2):429-44 -49,4499944, 1997.
Masoud Nickparvar. Brain tumor MRI dataset, 2021. URL https://www.kagg// http://www..agle.e.ccc////////// com/dav/2645886
Ziang Niu, Johanna Meier, and François- Xavier Briol. Discrepancy- inference for intractable generative models using quasi-Monte Carlo. Electronic Journal of Statistics, 17(1):1411-141) 2023.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Processing Systems, 29, 2016.
Chris Oates. Minimum kernel discrepancy estimators. arXiv preprint arXio:2210.16355 2022.
Lorenzo Pacchiardi and Ritabrata Dutta, Generalized Bayesian likelihood-free inference using scoring rules
estimalors. arxio preprinl urXiw:2104.0355 2021.
Mijung Park, Wittawat Jitkrittum, and Dino Sejdinovic. K2-ABC: Approximate Bayesian computation with kernel embeddings. In Artificial Intelligence and Statistics, pp. 398-407. PMIRR 2016.
19

---

Published in Transactions on Machine Learning Research (09/2024)
Christian P. Robert, Jean-Marie Cormuet, Jean- Marin, and Natesh S. Pillai. Lack of confidence in approximate Bayesian computation model choice. Proceedings of the National Academy of Sciences, 108 (37):15112 15117, 2011.
Bernhard Schölkopf, Alexander J. Smola, Francis Bach, et al. Learning with Kernels: Support vector Ma- chines, Regularization, Optimization, and Beyond. MIT Press, 2002.
Antonin Schrab, Tmmun Kim, Mélisande Albert, Béatrice Laurent, Benjamin Guedj, and Arthur Gretton. MMD aggregated two-sample test. arXiv preprint arXiv:2110. 150.7739, 2021.
Antonin Schrab, Ilmun Kim, Benjamin Guedj, and Arthur Gretton. Elficient aggregated kernel tests using incomplete U-statistics. Advances in Neural Information Processing Systems, 35:18793-18807, 2022.
Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance- based and RKHS- statistics in hypothesis testing. Anmals of Statistics, pp.
2013.
Jayaram Sethuraman. A constructive definition of Dirichlet priors. Slatislica Sinica, pp. 639 650, 1994.
Jasper Snock, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machinc learning algorithms. Advances in Neurol Information Processing Systems, 25, 2012.
Danica J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alex J. Smola, and Arthur Gretton. Generative models and model criticism via optimized maximum mean discrepancy. arXiv preprint 1ix::611.004488 2016.
Alexander Terenin and David Draper. A noninformative prior space of distribution functions. Entropy, 19(8):391, 2017.
Jakub M. Tomezak and Max Welling. Improving variational anto-cncoders using houscholder flow. arXiv preprint arXio:1611.096: 2016.
Jelmer M. Wolterink, Anna M. Dinkla, Mark H.F. Savenije, Peter R. Soevinck, Cornelis A.T. van den Berg, and Ivana l5gum. Deep MR to CT synthesis using unpaired data. In International Workshop on Simullation and Synthesis in Medical Imaging, pp. 14 23. Springer, 2017.
Xin Yi, Ekta Walia, and Paul Babyn. Generative adversarial network in medical imaging: A review. Medical Image Analysis, 58:101552, 2019.
Mahmoud Zarepour and Luai Al-Labadi. On a rapid simulation of the Dirichlet process. Statistics & Probability Letters, (2(5):116-924, 2012.
Kaifeng Zhang. On mode collapse in generative adversarial networks. In Artificial Neural Networks and Machine Learning ICANN 2021, pp. 563-574, Cham, 2021. Springer International Publishing. ISBN 978-3-030-86310-11
Feng Zhao, Chenhui Lei, Qingkun Zhao, Huiya Yang, Guoping Ling, Jiabin Liu, Haofei Zhou, and Hongtao Wang. Predicting the property contour contour-map and optimum composition of Cu-Co-Si alloys via machine learning. Materials Today Communications, 30:103138, 2022.
Appendix
A Technical Proofs
A.1 Theoretical Properties of the DP Approximation given by Ishwaran Zarepour (2002)
Proposition For a non-negative real value and fixed probability distribution H, let Frri = F (P H) and (J.. Dirichlet ..... a) be the weights in the approzimation of Frri given by Ishwaran & Zarepour (2002). Then, 00,
20

---

Published in Transactions on Machine Learning Research (09/2024)
i. Je,N -
No for any E {l,...,N),
it. JI.N e,,, N2. for any l, E (l,..., NY, where + t.
Proof. Recall
N Joy
PN
(14)
Since Eppri (Je,N) = t, for any l E NI, and > 0, Chebyshev's inequality implies
Var(Je,N)
Pr(JJN 1/N c
2
N-1 where, Var pame N2(a+1)
Assuming = R²c for E N and fixed positive number gives
Pr - 1/N1 2 e S R2CC?
The convergence of series Ex= K implies Pr
Pr
1/N1 2 c) s0.
co. By letting - oo, the first
Borel Cantelli lemma concludes JJ,N 1/N| م 0 and the result of (i) follows. To prove (ii), is enough to show Pr (li No
i O. To prove this for the probability space
let
1 N2 J. B
1 N
A w D::: lim (Je,N (L.N(w)Je.N(())) +
w Q lim (JEN(W)) #
10000
C = WER: lim (J)N(W)) + N
where, Pr(B) and Pr(C) are zero by (i). Since C BUC, then,
- Pr PrrWEEE a-
WE:: lim (w))
N Pr(A) & Pr(B) + Pr(C) = 0,
which concludes the result.
A.2 Proof of Theorem 1
Proof. For samples (V
AN and { (Y respectively, from H and F2, the triangle inequality implies
MM (PTTs F2,m. MMD F2,) F2m)) SK K
Σ
JENJIN
N2
1 N
J
m
By Proposition 6, which provides some theorotical properties of the DP approximation given in (14), the right-hand side of the above inequality converges almost surely to 0 as + oo for fixed N. This conver- gence immediately concludes the proof (i). To prove (ii), since (JI.N...., JN.N) ~ Dirichlet N E Feprr(( (Je,N) N and
4
a 11N1)
if(
Egg (Je,Jee,)
a+ N
if
(a + 1)N2
21

---

Published in Transactions on Machine Learning Research (09/2024)
Applying these properties in definition of MMD BNP (FI.N. F2,m) results in
a V.)
N N
(a NV V,)
E pppr (MMD BNP (FT,N m2 =
(a 1)22
(a +a+ 1)N2
8= tp
N
2
KVV, Y.) +
Σ MYY,,Y,) t1=1
(15)
Nm
m
Now, it is sufficient to compute the following conditional expectation,
E(MMDENP (1 F2,m)) = (2 (epp (P P2. (i))))
(16)
Since sets {Vi) and {r include i..d. random variables, separately, replacing equation 15 in expecta- tion (16) implies:
(16) = a(N- - 1)
a a N
En (V 11, V2)l E Y2) + m
(a 1)N EHla(Vi, vi)l - - 2k(,,, YD)
(a 1)NV
+ m
EF Y1, 1 Y.)l.
(17)
The proof of (ii) is concluded by letting a - xc, - xc, and m - oo in the above equation. Lastly, since a-N (a+1)N < 2, then, for any N.m E and a a R
1 < 1, m-1 1, a(N-1)
< 1, and
(a+1)N
(17) < E 1V V2) FF (V Y,) + Ep, k (Y,, Y2) 33,
which concludes the proof of (iii).
A.3 Proof of Theorem 2
Proof. Applying triangular incquality implics
Ji JiNJi Vi) 1 &V
MMDDNN(FPP. F2,m) -MMD m2
<
Σ
{,1=
N
Jink(V2 Y.) I kV. Y.). Y.)
(18)
where, samples (Vi) and (Y,) are generated from H² and F2, respectively. Similar to Proposition 6, it can be shown that Jē,N + I/N and Ji, I/N2, the other hand, since H' + И as + co, the chance of sampling from H and Fi, tends, respectively, to one and zero, which implies V." + V,, where V2 ~ H, for i = 2. Applying the continuous mapping theorem implies k(Vi. V2) - k(V,, V.) and k(Vi, Y,) + k(V,, Y.), which completes the proof of (i)(a) To prove (b), follows from the proof of Theorem
20, using conjugacy property of DP. On
E(MMD (TY,0 F2.m)) = (a, n, EH. (VVi, Va)l + h2(a, n, N)E [M(i, Vi)
2E F. (V(Vi Y,)
m Ek,,k(Y Y2)
1 + EP 1K(,, Y.)l,
(19)
m
where n, (N+1)(N-1)
(at=+1)N and h2(a,n,N)
atutN [ainti)N. Since k(;-) is bounded above by K,
the dominated convergence theorem implics Eg- (V-,, V;)] - En(h(V, V2)] and EHr,F, Y,)] + FH, Y1)]. Since h1(a, n, N) + 1 and hz(a, n, N) + 0 as oc, N oo; and, m/(m 1) 1 and 1/m- 0 , as m oc, the results follow.
22

---

Published in Transactions on Machine Learning Research (09/2024)
To prove (a) and (i)) Fi, - F1, and then H' + F1 indicates that the probability of sampling from H and Fi,n tends, respectively, to zero and one. Therefore, Vi + X, as n + 0o, where X; ~ Fi, for i = 1,2. The proof of (ai is completed with the same strategy as the proof of (a) by letting n + oo in (18). The proof of (ii) is also concluded with a similar argument that in (i)(b), when + xc in (19).
by the Cevenko-Cantelll theorem. It
A.4 Proof of Corollary 3
Proof. The proofs are immediately followed by Theorem and Theorem 2.
A.5 Proof of Lemma
Proof. The proof of Lemma 4(1) relies on the proof given in Dellaporta et al. (2022, Theorem 9) which is expanded for infinite stick-breaking representation while we consider the finite DP approximation given (14). By employing a similar technique in the previously mentioned theorem, have
E (MMD Fc. )) Ep (Fipoo Fo.. 1m))
min MMD(F, FG.. + 2E (MMD(F,, F)) + 2 (MMD(FN" H))
WEW
+ 2Fp (Eи- (Er-(MMD H (1m))
Building on the results Dellaporta et al. (2022, Lemma 7), we can establish that
N
Epen WiN k(V2, V < (a (a+n) n N)K (a+ 1)N
Epses (MMD?(FF) pos _11))) S
> t=1
where the right-hand side of the above inequality follows from the fact that k(,,) s and Ep [J a Now, the Jensen's inequality implies
(ata+1)N7
a N)K
Eppos (MM H²))
(a+
On the other hand, Chérief-Abcdellatff & Alquier (2022, Lemia 7.1) and Dellaporta et al. (2022, Lemma 8), respectively, imply that
K EF (MMD(F,, F)) < T EF (En- (MMD(F,, H H'I|X < 20K
which concludes the proof of (i). To establish (ii), we adopt the approach used in the proof of Dellaporta et al. (2022, Corollary 5). Initially, we employ Chérief-Abdellane & Alquier (2022, Lemma 3.3) to bound (M. Fo..) by 2e + MMD(F, FG..), resulting in:
E (MMD((F, Fo.. )) 2e (MMD(F, (MM.(F,FF..)).
Applying the result (i) to the right-hand side of the above inequality implies:
2K 4aK
a + (a+n+NN) (a+n- I)N
E' (M, FG..)) < 2E + min MMD(F, FG..) + + Vn +
+ +2
WEW
Finally, we employ Chérief-Abdellatif & Alquier (2022, Lemma 3.3) once again, but this time bound MMD(F, Fa..) by 2e MMD(Fo, Fo, Fo..) for any E W, thereby completing the proof of (ii).
23

---

Published in Transactions on Machine Learning Research (09/2024)
A.6 Proof of Lemma 5
Proof.
Let (w) MM Faw), [,,((w) MMD(Fn, Fow.m,, and L(w) = MMD(F. Fa.):
Then, for wt E W, Gretton et al. (2012a, Theorem 7) implies
-m
Pr (mmmm))(")) [(ww)\] h(n m, < 2xxp 2K (n m)
(20)
Hence, with probability at least 1 - 2exp
-e2nm
2k(x+m)
(n,m(w") L(ww)) m, K,C).
(21)
On the other hand, the triangle inequality implies
[BNN(W") - L(ww)) & [m,,(("")) - [(w")| + (BBN(W"") =nmm(w") + (k(w") - [(w').
(22)
Finally, the proof of (i) is concluded by considering incquality 21 in equation 22. To prove (ii), Markov's inequality implies
Pr (MMD(F, FG.)) E (MMD(F.. Fa..))
The result follows by substituting the bounds from Lemma 4(i) into the right-hand side of the above in- equality.
B
Computational Algorithms
B.1
Implementing the Semi-BNN GOF Kernel-based Test
A Algorithm Pseudocode of semi-BNP two-sample MMD kernel Lest
1: Initialize a, É, M, du, and in equation 4 to determine N. Consider an observed sample X1.1 from Fi.
H - F2 STEP 1: Computing the BNP MMD
3: for 0 to E do
Generate an approximate sample of Fi DP(a, by using E JeN&v, where (JiNN Dirichlet (,........ ...., ;), and ( H.
5: Generate an approximate sample of Filxi:n DP(a + H*) by using Σ Jin where JJiN Dirichlet atn ,..,, *,",, and (V)J,, ~ HI. IN
Use samples generated in steps and to compute MMDD I F2,m) and MMD BNP FIN F2,m), respectively.
end for
8: return (MMD (FT F2.m) and (M, (PP.0 F2 STEP 2: Estimating RB and Str
9: UMMDA (x1m)) < (CDF(MMDEEND. (FT, f2m)))=
D The ECDF of posterior-based MMD The ECDF of arior-based MMD
10: [MMMD* ()) DCDF(MMMD) BNP, (F.ri N. P2.m )5
11: da//M < quantited (MMMI (FY. . m2 1 iMMM
D The estimation of the i quantile of
MMD2 ANN (FPri (FPri
F2.m)
12:
R. (0 (0x)).))
a (dig/ar)
13:
Str +-
14: for i 0 M - 1
15: de/s* <- quanbble
I.N (/M
16:
2/tD/MM + quamtilel (MMMDD BNP, (FT F2.m)).= 1)/M)
Ma (/ N Wayss 11))))
17:
Momp (d/4)//s)) 0 (a/se))
24

---

Published in Transactions on Machine Learning Research (09/2024)
18:
if RBMMD (x/m//am)) I MMDD (0/x1nn) then
19: (tr(U|x Str [ (141)//M/11)) RMMDD (deyss x
20: end if
21:
end for
22: return MBD2, Str
a n/22
>> n
X1:1 F1
H Fz (ML, F2) 0
# Fz MMD* (H. F2) =
For any choice of
denstyyyyyyy Tuart Pas denety Paa
Pridersity Rusptt
18.
Pos-density: Reseit/M...)
Hy true F1 = F2
02
Denotty
can reject sccept.Kg)
Actual Condition
Accepn.95 Kg
Accept
2000
iv
X54
CCe
Density
hr-dararly: lmsrl!
Rh Pce-dersity: thsart a
deniitty ty: mTmm/(+1
Ho is not true
PL' Risilersity harrd RR...
Pos-density: Musesll (X:
F1 # F2
4 Densky
Decision
Aucupa
Dectsion
Rlject
Accept/N1
0.028 00000
08012
0001
notta
babss a 60020 Ditti
MMD
Manun (-) supports 2i
prrijs constructed arvund base measure H. whose variation is controlled by MMD* (pert () constructed around MMD* F2)
Theoren & Corollary 3)
RB is calculated as the ratio of the area under MyMD? [X:=)) to the area under NMMD> () on the interval [0, dly/m)- false hypothesis.
Choosing ff leads Msrmm (Xta) not supporting 2G, z> The fails reject Hf, when
Choosing an significantly larger than results in "TMDD" ( (N,,)) and "TeND" cainciding z>
The test unable to evaluate Hi
Figure 6: General diagram of the BNP-GOF lest. The prior density of MMD versus the posterior densily after observing Xin from distribution Fi under all possible conditions for both true and false null hypothesis. Here, we consider Fi = Fg N (0,o,, Toa) as the true mull hypothesis, while F, to ta Too) and F2 (O Iso) for the false null hypothesis to plot densities.
B.2 Training the Semi-BNP GAN
A Algorithm 2 Pseudocode of training a GAN using the semi-BNP approach
Set 10-6 to employ non-informative prior leading DP posterior DP(n, F.).
2: Initialize é in equation 4 to determine N using conjugacy property DP.
3: Tmn < Number of training iteration, nmmb <- Mini-batch size
Wo An initial parameter for generator Gw, {xx] real dataset
5: for +++0 0 to Tmb do
Generate random sample (xm=b Bomb Ponb from real dataset {xe]²_ 7: Generate sample of noise vector {ut )ime from uniform distribution U-11
8: Gencrate a sample from FG.,, distribution of Gu., as fyz Gw, (u:)) Timb
Generate a sample of size N from Fros F1xxm>) n using N Jr, by replacing Fi by F, and (xmm))/mb Rom by x in step (4) of Algorithm 1. 10: Use generated samples in steps 9 and 10 to compute AMDDNPPFE No Compute the gradient:
Rin
11:
OMMD (pops Fow, m))
1
8MMD (P Paw.
Owh
2/ MMDA BNP ppos Faw.)
Ow
25

---

Published in Transactions on Machine Learning Research (09/2024)
8Gu, (ue) Bw.
12: Use backpropagation for calculating partial derivatives
the previous step to update pa-
rameter w,.
13: end for
14:
return w
> An optimized parameter for Ga that minimizes the cost function.
B.3
Hypothesis Testing Evaluation
Algorithm 3 Pseudocode of plotting ROC and computing AUC in semi-BNP test
1: Initialize o, N, f, and M.
2: Set the number of repeating experiments: r + 100
3: RBi[HG + Compute RB for r samples of sizes n generated under the null hypothesis. RB[9[1 +- Compute RB for samples of sizes generated under the alternative hypothesis.
T +- A sequence of numbers between 0 to 20 with length L. Thee discrimination threshold for the scmi-BNP test.
6: TP + A vector whose each component represents the number of components of the vector RB [G which less than each component of T.
7: FN +A vector whose each component represents the number of components of the vector RB[J which is greater than each component of 7.
FP + vector whose each component represents the number of components of the vector RB HG which is less than each component of T'. TN <- A vector whose each component represents the number of components of the vector RB Ho which greater than each component of T.
10: Compute the confusion matrix as:
TNR TN+FP (1-Type I error)
FNR = FN+TP (Type IT etror)
FP FPR := FP4TN
TP TPR = TP+FN (1-Type 11 error)
(Type I error)
11: ROC + Drawing a linear plot of TPR against FPR.
12: AUC < Computing the area under the ROC.
13: return ROC and AUC.
1 should be changed to the p-value in the FNP test.
: should be changed to in the FNP test.
C Relative Belief Ratio: A Bayesian Measure of Evidence
The RB ratio (Evans, 2015) is a form of Bayesian evidence in hypothesis testing problems and has shown excellent performance in many statistical hypothesis testing procodures (Al-Labadi et al., 2022a; 2021a; 2022b). The RB ratio is defined by the ratio of the posterior density to the prior density parameter of interest in the population distribution whose correctness is under investigation. Precisely, for statistical model (I,3) with F = (Jo 0 E θ}, let be prior on the parameter space O and =(0 xim) be the posterior distribution of 0 after observing the data Ximn = (XI....,x,), Consider parameter of interest, 1 = (0) such that Ti satisfies regularity conditions so that the prior density Ty and the posterior density Fy (- x1m1) of s exist with respect to some support measure on the range space for V. When Ty and (- x1..) are continuous at vh, the RB ratio for a value vi is given by
particular
RBy xI:)) = (bb xim) / (tb).
Otherwise for sequence No(1)), ), the neighborhoods of se that converge nicely to th 8 - 0, the RB ratio is defined by RBy (v) x1m) lims- Ig ( //ll (Na( )), where II and II (:) x1:)) are the marginal prior and the marginal posterior probability measures, respectively.
Note that RB (v) xim) measures the change in the belief of v being the true value a priori to a posteriori. Therefore, is a measure of evidence. RBy (v x1m) > 1, then the probability of & being the true value from a priori to a posteriori increased, consoquently there evidence based on the data that & is the true value. If R (o) X1.) < 1, then the probability of & being the true value from priori to a posteriori is decreased. Accordingly, there is evidence against based on the data that being the true value. For the
26

---

Published in Transactions on Machine Learning Research (09/2024)
case R. (1b X1:=) = 1 there is no evidence in either direction. For the mull hypothesis Hy (0) = vo, it is obvious RB (wo) X1.)) measures the evidence in favor of or against Ho- In this scenario where evidence for the null hypothesis is plausible, the frequentist notion of controlling the probability of falsely rejecting Ho (type I crror) does not apply.
The possibility of calibrating RB ratios is a desirable feature that makes it attractive in hypothesis testing problems. After computing the RB ratio, it is very critical to know whether the obtained value represents strong or weak evidence for against Kg. A typical calibration of RB (th x1:)) is given by the strength of evidence
Stru ( x1:.) HT [RBo(t) [xi=)) S RBo(vo X1=) X1:]).
(23)
The value of equation 23 indicates that the posterior probability that the true value of e has a RB ratio no greater than that of the hypothesized value thp. When ( (xms) < 1, there evidence against tho then a small value of (23) indicates strong evidence against th because the posterior probability of the true value having RB ratio bigger is large. On the other hand, a large value for (23) indicates weak evidence against vo. Similarly, when RBy (vol x1:=) > 1, there is evidence in favor of tho then a small value of (23) indicates weak evidence in favor of vo. while a large value of (23) indicates strong evidence in favor of v/u-
The RB can be considered as a strong alternative to the Bayes factor (BF) criteria. The BF is defined as the ratio of the marginal likelihood of data under the null hypothesis to the alternative hypothesis in Bayesian hypothesis testing problems. However, computing the BF often involves intractable calculations of marginal likelihoods, which typically require computationally burdensome methods such as MCMC. The tests proposed by Holmes et al. (2015) and Borgwardt & Ghahramani (2009) are two examples of BNP tests that utilize marginal likelihood computation, and their practical usage in high-dimensional statistics is low due to this computational issue.
On the other hand, the construction of tests using the BF relies on assigning prior mo to the null hypothesis Ho, a prior Ti to the alternative hypothesis Hi, and a discrete probability mass po for Hg- However, practitioners often face challenges in eliciting these prior components within the overall prior π = Po + (1 1-) Another concern of using BFs is their calibration to indicate whether weak strong evidence is attained. For example, Jeffreys (1961) and Kass & Raftery (1995) proposed similar rules to calibrate BFs but Carcía-Donato & Chen (2005) pointed out that such rules are inappropriate to calibrate BT's as they ignore the randomness of the data and, again, lead to improper inference"
D Radial Basis Function Kernels Family
The construction of MMM--based procedures is proposed based on considering a kernel function with feature space corresponding to a universal RKHS. The radial basis function (RBF) kernel is the most well-known kernel family satisfying the above situation. For two vectors X, Y E Rd, the RBF kernel is represented by
A(X, Y) = M(|X - YI1/o),
where, h is a function from the positive real numbers R+ to Rt, ||- represents the L LL-norm, and a is the bandwidth parameter that indicates the kernel size. There are many functions assigned to h, for example, the Gaussian, exponential, rational quadratic kernels, and Matern, represented by
22 =xp((=)) h1(x)= = exp (- h2(x)= = exp ha(x) = (1 2a
1
_
11(x)) = (1+ V2uzje Veur
rospectively; where, a in hy is positivo-valued scalc-mixture parameter, and the in hs is paramcter that controls the smoothness of the kernel results (Zhao el al., 2022; Genton, 2001).
One of the simplest kernel functions above is the Gaussian kernel, which is mostly used in machine learning problems and only depends on bandwidth parameter a. The Gaussian kernel tends to 0 and 1 when o + 0 and の + oc, respectively. Both situations lead to MMD being zero. Hence, the choice of the parameter
Il A comprehensive study that explains why the RB ratio more appropriate measure evidence than the BF can also be found in Al-Labadi et al. (2023).
27

---

Published in Transactions on Machine Learning Research (09/2024)
has a crucial effect the performance of this kernel. Numerous methods proposed to choose the value of o, however, there is no definitive optimization method for this problem. The median heuristic is one of the first methods used in choosing empirically and will be denoted in our experimental results by омH. More preciscly, for two samples (X; and {Y;)== the омн is considered as the median of -|X;-
1 S i n, S S m), which is mostly used in kernel-based tests (Schölkopf et al., 2002). Selecting a based on maximizing the power of two-sample problems another strategy considerod by Jitkrittum et al. (2016). The selection of the MMD bandwidth on held-out data to maximize power first proposed by Gretton et al. (2012b) for linear-time estimates and by Sutherland et al. (2016) for quadratic-time estimates. Recently, bandwidth selection without data splitting has been proposed for quadratic (Schrab et al., 2021) and linear (Schrab et al., 2022) MMD estimates. Regarding the choice of a in kernel-based GANs, a common idea is assigning several fixed values to and then considering the mixture of their corresponding Gaussian kernel. This strategy has received much attention and shown an acceptable performance training GANs
E
Training Evaluation
E.1 Traditional Approaches
Evaluating the quality of samples generated by GANs is considered to assess the mode collapse problem (Zhang, 2021).
E.1.1 Fréchet Inception Distance (FID)
The FID is widely used metric to assess the similarity between the distribution of generated images and real images (Heusel et al., 2017). It is based on the concept of comparing the statistics of feature representations of these images. Specifically, it computes the Fréchet distance between two multivariate Gaussian distributions fitted to the feature representations of the inception network for real and generated images.
the feature representations for the real images {x,) in the inception network, if P and (x) are the sample mean vector and covariance matrix of (Xi))( and Heme (Y) and (Y) are the corresponding statistics for the generated images (Y,(w*)) [/= then the FID defined as:
Let (X,n
FID (X) Porrae (x) M Tr(E
Expra 21 (X) (p
where "Tr" denotes matrix trace.
A lower FID indicates that the generated images are more similar to the real images.
E.1.2 Kernel Inception Distance (KID)
The KID is another metric used to evaluate the quality of generated images. Unlike FID, which assumes the feature representations follow a Gaussian distribution, KID is based on the MMD between the real and generated images' feature representations (Bińkowski et al., 2018) as
KID MMD Prn (X), F Fome(()).)) (Y),))
where (X),n denotes the empirical distribution corresponding the sample (X:)).
The KID uses the polynomial kernel function in calculating MMD, which is defined as:
k(Crne (X+), Pine
(a (Ys(="))) ((grno ((x)))(X.)) @ine (n (Y,(w")) 1)",
where v is the degree of the polynomial and "T" denotes matrix transpose.
Similarly to FID, a lower KID indicates higher similarity between the real and generated images.
Foror further details, see Lá al. (2015) and Li al. (2017).
28

---

Published in Transactions on Machine Learning Research (09/2024)
E.2 An MMD Matching Score Function
To develop a stronger method for evaluating the differences between real and generated data manifolds, we propose using the MMD dissimilarity measure as follows: For = T'bs let {X,, J== and {Y₁, (w)))) be two samples drawn, respectively, from the real dataset X,,..., Xn, and the generated dataset Yi(w"), ).... (n"))) with the same sample size nmm < n. Then, we define the MMD-based matching score as
MMDS = max MMD2 (Fnm..))) (1), Fo....... (1))
(24)
mm.....mmo)
where, MMD3 (F (i), Fe mo (i)) is the MMD approximation given by Equation (2, main paper) using samples (X,, J= and (Y, (wo") }} (mini-batch samples). Our proposed matching score returns the max- imum value of the MMD approximation between subset of the real and a subset of the generated dataset with the same size Amb (mini-bater sample size) over T'm resamplings (mini-batch iteration). According to Equation (2, main paper), all components of mini-batch samples are compared together in the MMD mea- sure, which provides a comprehensive assessment between subsets of the data in each iteration. Eventually, it is obvious smaller values of MM DS indicate better quality and more diversity of the generated samples.
F
Additional Experiments
F.1 The Semi-BNP Test
To further illustrate the difference in performance between the BNP and ENP tests, we conducted tests on two alternative distributions: Fi ((, o²) for o² [1,4] and F1 0.5N + 1) 0.5. v, 1) for u [0, 1]. The corresponding results are reported in Figure and 8 for univariate cases with n 50. Figure 7(a) specifically shows that the proposed test exhibits higher growth rate of the AUC when o² is increased compared to the other tests. Additionally, Figure 7(b) indicates that our test starts to detect differences carlier than other tests (o² 2 1.67). Similar results can be found in Figure 8 for mixture distribution with various means.
Tree ofth BAP tese Eeargy
MNO
Type of thetest DVP Ersrp DNP RPP-MID--. s NVP TPP-Eegg TNP WWD
(a)
(b)
Figure 7: (a) AUC values in lesting alternative F = N(0,0²) for o² E (1,4) in variance shift example. (b)-Top: Test critical values against different values of o², (b)-Bottom: The lighter density corresponds to larger value of o².
Figure 9 provides a more focused comparison between the semi-BNP test and its Bayesian competitor, the BNP energy test. This figure illustrates the proportion of rejecting Hg over the 100 samples for both Bayesian tests mentioned, across different data dimensions. The first row of Figure 9 represents the type error, while the remaining rows represent. the test power. The figure demonstrates the effectiveness of the
29

---

Published in Transactions on Machine Learning Research (09/2024)
CA
Reengy
MMQ
34
test EMP Eregy UNP RNP. RPP-Eregy FP
19
(b)
Figure (a) AUC values in testing alternative Fi 0.5N u, 1) + 0.5N - vv, for v (0, 1) in mixture example. (b)-Top: Test critical values against different values of o². (b)-Bottom: The lighter density corresponds to smaller value of v.
semii-BNE kernel- based test in detecting differences, especially in scenarios involving variance shift, heavy tail, and kurtosis examples, where the ENP-energy test does not perform optimally in high sample sizes.
Moreover, conduct a comprehensive analysis of the large sample properly of all the tests in comparison, present Table 2 for sample sizes 500, 1000. This table clearly demonstrates the weak performance of the BNP-Energy test in particular scenarios that are currently being mentioned.
Additionally, to display the asymptotic behavior of the posterior-basee estimator compared to the baseline (3) (FNP-MMD), we provide the density plot of both under the null hypothesis in Figure 10 demonstrating the superior performance of the semi-BNP estimator terms of faster convergence to zero.
F.2 The Semi-BNP GAN
Now, we examine the performance of the proposed GAN through additional datasets, the details of which are given below. The generated samples are shown in Figures 11. Generally, the generated images using semi-BNP GAN show better resolution than the FNP GAN. The MMD scores presented Table are also evidence to demonstrate this claim. To further assess the performance of MMD-based GANs, we report the commonly used Fréchct inception distance (FID) and the Kernel inception distance (KID) metrics (Bińkowski et al., 2018). These metrics well-suited for evaluating the performance of GANs. The corresponding 13 are reported in Table 3. Similar to our MMD scores, the smaller values of FID and KID show better performance of the GAN.
F.2.1 Bone Marrow Biopsy Dataset (Tomczak & Welling, 2016):
The bone marrow biopsy (BMB) dataset is a collection histopathology of BMB images corresponding 16 patients with some types of blood cancer and anemia: 10 patients for training, 3 for testing, and 3 for validation. This dataset contains 10,800 images in the size of 28 x 28 pixels, 6,800 of which are considered sets of equal size for testing and
for the training set. The rest of the images have been divided into com homm///tooczak/veeeeeeooo householder flow///lww////
validation. The whole dataset can be found at https://githee The results based on 6800 training images are presented in
Figure 11-(a-c).
13eThe codes compute KID and FID available http://///i.ii..oo..aaaaaaaaaaaa
compute_scores,
30

---

Published in Transactions on Machine Learning Research (09/2024)
d
d=
d
d' 20
40
d 60
d 80
d 100
No diferences
0.1
0.0
1.00
Mean shift
0.75
0.50
1.0
Skewness
0.9
0.8
1.00
Mixture
0.75
0.50
Variance shift
1.0
0.5
0.0
1.0
Heavy tail
0.5
0.0
1.0
Kurtosis
0.0
0
10000
10000
10000
10000
10000
10000
10000
1000
Semi-BNP-MMD test
BNP-Energy test
Figure 9: The proportion of rejecting H6g out of 100 replications against sample of sizes n = 1.. 1000 based on using a = 25, = 1000, = 10-3 in equation 4, M = 20 for the semi-BNP-MMDD (blue line) and BNP-energy (red dotted) tests.
31

---

Published in Transactions on Machine Learning Research (09/2024)
Table 2: The average of RB, the average of its strength (Str) ), and the relevant AUC out of 100 replications based on using a 25, L 1000, c 10-3 in equation 4, M 20, and bandwidth parameter o 80 in RBF kernel for two sample of data with n 500, 1000.
Example
MMD
Energy
MMD
Eaergy
RB(SS)
AUC
RB(Su)
AUC
P.valuee
AUC
P.talue
AUC
500
1000
500
1000
500
1000
500
1000
500
1000
500
1000)
500
1000
500
1000
No difererress
78)
6.5300880)
3.75(0.60)
(.30(0.00)
0.52
0.50
0.48
0.49
18. (00.5))
19. (0.95)
18.7 (0.58)
(9.58(0.76)
0.50
0.51
0.51
0.14
10
(9.99(
20(1)
20(1)
20(1)
0.51
0.50
0.53
0.48
20
20(1)
20(1)
20(1)
20(1)
0.53
0.51
0.51
01.44
10
20(1)
2)
20(1]
20(1)
0.45
0.52
0.51
0.50
60
20(1)
20(1)
20(11
20(1)
0.51
0.50
0.50
0.53
80
20(1)
20111
20(11
20(1)
0.401
0.48
0.54
01.49
100 20;1)
1)
20(1)
20(1)
0.49
0.48
0.51
0.50
Mean abift
[0(00
(20)
D[U]
010]
0.58 0.98
11:011
0.0001
0.000L 0.104
0(0)
020)
O(0)
010]
0.001
0.001
0.004 0.004
10 ((D)
020)
0(0]
0101
1
0.001
0.001
0.004 0.004
20 (0(0)
0:0)
0(0)
0101
0.001
0.001
0.001 0.504
40 0(0)
(20)
((0)
010)
0.001 0.001
0.004
60 0(0)
020)
0(0)
0(0)
0.00 0.001
0.004 0.004
80 ((0))
0:0)
0(0)
0(0)
0.001 0.001
0.001 0.004
0(0)
0:0)
0(0)
010)
0.001 0.001
0.004 0.004
Steeweess
(((0)
0,0)
010
1.001 (.001
0.001 0.004
0(0)
0(0)
0(0)
0.001 0.001
0.004 0.004
10 0(0)
(20)
0(0)
0100
0.001 0.001
0.004 0.004
20 0(0))
0:0)
0(0]
0:0)
0.001 0.001
0.001 0.001
40 ((0)
0:0)
0/0)
0(0)
0.001 0.001
0.001 0.004
60 0(0)
0:0)
0(0)
010:
0.001 0.004
80 0(0)
0:0)
0(0)
0(0)
0.001 0.001
0.004 0.004
100
0(0)
0(0)
0(0)
0101
D.001 0.001
0.004 0.004
Mixtare
0(0)
(20)
((0)
010)
0.06 0.01
0.83 0.98
0.004 0.004
0(0)
0:0)
0(0)
010)
1
0.001 0.001
0.004 0.004
10 [(0))
0:0)
0(0)
0/0)
0.001 0.001
0.001 0.004
20 0(0)
020)
0(0)
010:
0.001 0.001
0.004 0.004
40 0(0)
0.0)
0(0)
0100
B.001 0.001
0.004 0.004
60 ((0))
0:0)
0(0)
0(0)
0.001 0.001
0.001 0.004
80
0(0)
020)
((0)
010)
0.001
0.004
100
0(0)
020)
0(0)
0100
D.001
0.001
0.004 0.004
Variance shift
(.01(0))
1.73(0.59)
2.10((.59)
0.53
0.077
0.01
0.53 0.98
0.006 0.004 0.30
0.42 (7)
0:4000.00 (8)
0.98
4,42 72)
7.30
0.73 0.64
0.001
0.001
0.001 0.004
10 0.39(0.06)
0.22(0.06)
0.69)
13.1 13.12(0.73)
0.55
0.40
0.001
0.001
0.004 0.004
20
(((0)
(:0)
13. 13 78)
18.1210.69
0.35 0.07
0.001
0.001
0.001 0.004
0(0)
0:0)
(0.66
19. 19.8210.68)
0.001
60
0(0)
0.0)
19.19(0.55)
19.9%
0.02
0.001
0.001
0.004 0.004
80
0(0)
0(0)
(7777
20(1)
0:001
0.001
0.004 0.004
((0)
0:0)
(9.52(0.64)
20(1)
0,001
Henvy tui
0.00((0]
(:0)
(65(0.54)
0.56
0.09
B.0KS
0.00%
U.90 0.99
0.01
0.005 0.98
0.90
0.04(0))
0.02(0)
2.89(0.71)
4.54
0.91
0.76
0.001
0.001 0.004
10 0(0)
020)
4.4
(0.7))
0.78
0.64
0.001
0.001
0.004
20 0(0)
0(0)
5 (.76)
11 (3775
0.77
0.42
0.001
0.001
0.004 0.004
10 ((0)
0:0)
9. 9.
10. 1100788
0.54
0.20
0.001
0.001
0.001 0.004
60 0(0)
0:0)
L..0 74)
18.06 821 0.52
0.001
0.001
0.004
80 0(1)
0(0)
18.51 12.63(0777
(0.)))
0:41 0:09
B.00
0.001
0.004 0.004
100 ((0))
13.17
19.07(0.97)
0.06 0.50
0.001
0.001
0.001 0.004
Kartosis
0(0)
(20)
L55(0.52) 32)
0.96
0.00
0.004
0(0)
0.0)
1. 1.75().59)
(..4
0.96 0.88
0.001
0.001
0.004 0.004
10 ((0)
0.0)
2.81(0.66)
6.11 (0.76)
0.91 0.75
0.001
0.001
0.001 0.004
20 0(0)
0(0)
L.330.71)
0.78)
0.001
0.001
40 0(0)
0.0)
5.70(0.73)
13.4 (30.7()777)
0.74 0.28
0.001
0.001
0.004 0.004
60 0(0)
000
7.06(0.75)
16. 18.38U0.8 81) 0.72 0.23
0:001
0.001
0.001 0.004
0(0)
0.0)
17.50(..5)) 6.11(0.79)
0.001 0.004
100
0(0)
0.0)
8.83(0.78)
18.5 18.52
89)
0.35
0.09
0.001
0.001
0.004 0.004
32

---

Published in Transactions on Machine Learning Research (09/2024)
Density Plots of the Empirical Estimator of MMD2 (V-Statistic)
Sample Sizes
3000
n=5
n=10
n=
2500
n=40
n=60
n=80
n=100
n=140
Density
n 160
1500
180
n=200
n=240
n=260
1000
n=280
n-300
n=400
n=500
n= 1=1000
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Value
Density Plots of the Posterior-based Estimator of MMD (BNP counterpart of V-Statistic)
Sample Sizes
n=5
1000
n=10
n=20
800
n-60
n-80
n=100
n=140
Density
600
n=160
n=180
n=200
n=240
n=260
n=280
n=300
200
n=400
5=500
1000
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
Value
Figure 10: Density plots of the semi-BNP and FNP estimators of the MMD for various sample sizes.
F.2.2 Labeled Faces in the Wild Dataset (Huang et al., 2008):
The labeled faces in the wild dataset (LFD) include 13,000 facial image samples with 1,024 (32 32) dimensions. The dataset available https://conradee http:://conradanderroon
au/lfwerop/.
F.2.3 Brain Tumor MRI Dataset (Nickparvar, 2021):
In the last experiment, we consider a more challenging medical dataset including brain MRI images available at https://wwx kaggle. com/dsv/2645886. This datasct has two groups including training and testing sets. Both are classified into four classes: glioma, meningioma, no tumor, and pituitary. To train the networks, we consider all 5,712 training images. The images vary in size and have extra margins. We pre-processing
33

---

Published in Transactions on Machine Learning Research (09/2024)
code to remove margins and then resize images to 50 x 50 pixels. We also scale the pixel value of prepared
images to range 0-1 to make the range of distribution of feature values equal and prevent any errors in the backpropagation computation.
(a) Training data
(b) Semi-BNP-MMD GAN
(c) MMD-PNP GAN
(d) Training data
(e) Semi-BNP-MMD GAN
(N) MMD-FNP GAN
(g) Training data
(h) BOni-BNP-MMD GAN
(1) MMD-FNP GAN
Figure 11: Generated samples of sizes (6 x 6) from semi-BNP- and MMD-FNP GAN for the BMB and LFW datasets using a mixture Gaussian kernels in 40,000 iterations.
Table 3: The values of MMD, KID, and FID scores for four groups of datasets considering n.d = 1000 and
Tmb = 1000 in equation
Dataset
Scores
MNIST
BMB
LFW
MRI
Semi-BNP
FNP
Semi-BNP
FNP
Semi-BNP
FNP
Semi-BNP
FNP
MMD
0.0384
0.0404
0.0285
0.0315
0.0281
0.0302
0,2059
0.2231
KTD
0.0034
0.0046
0.0030
0.0036
0.0019
0.0026
0.0260
0.0264
FID
35.560
37.934
17.006
17.264
14.010
14.473
87.975
87.831
34

---

Published in Transactions on Machine Learning Research (09/2024)
F.2.4 Learning Rate Comparison: Semi-BNP-MMO GAN versus FNP Counterpart
To assess whether the proposed discriminator used in the Semi-BNP-MMD GAN leads to faster or better convergence of the generated samples compared to the baseline proposed by Li et al. (2015), we consider the synthetic distribution (- Ia) (1d, Ia) as the true distribution and provide the corresponding MMD values for both models over 20,000 iterations in the data generation process, shown in Figure 12. Our proposed GAN clearly displays higher speed of convergence for the corresponding cost function to zero, and thus better performance compared to the baseline.
BNP-Learning
0.25
FNP-Learning
0.20
MND
0.15
0.05
7500
10000
Iteration
Figure 12: Learning rate: Values of the cost function in the proposed GAN and its frequentist counterpart (Li et al., 2015) over 20, 000 iterations.
G More Discussion on the Potential Research
GANs increasingly used in medical imaging applications which effective tools for tasks such as medical imaging reconstructions. The synthetic images generated have often been proven to be valuable especially when the original image is noisy or expensive to obtain. GANs have also been used for generating images in croas-modality synthesis problems, where we observe magnetic resonance imaging (MRI) for a given patient but want to generate computed tomography (CT) images for that same patient (Wolterink et al., 2017). This type of generative method for medical imaging can drastically reduce the time and cost of obtaining data if the quality of the synthetic examples is sufficiently high. GANs have also been used a diagnostic capacity for example, in detecting brain lesions in images (Alex et al., 2017).
Here, the GAN is trained by distinguishing between labeled data of brain images that contain and do not contain lesions. Then, the discriminator of the GAN is used to detect brain lesions on new images. However, GANs are far less commonly used for tasks like diagnosis. According to a survey on medical imaging research in GANs, less than 10% of the top papers surveyed were dedicated towards making diagnoses, whereas the vast majority of papers were dedicated towards generating realistic synthetic examples of medical images for further analysis (Yi al., 2019). We believe this because where the cost of making errors diagnosis is immediately consequential people, unlike other AI applications where GANs largely used.
We plan to extend the current work by mapping the data lower dimensional space using variational auto-encoder (VAE), dimensionality reduction model helps to reduce the noise in data and tries to optimize the cost function between the real data and fake data in the code space. There is a significant potential to combine the elements Stein variational gradient descent (SVGD) and GANs. For instance, SVGD can used for the inference of latent variables approximate the variational distribution (the distribution of the latent variable given the observed dataset) in VAE-GAN. Then, we will propose 3D semi-BNP GAN in the code space to improve the ability of the GAN to generate medical datasets. The VAE method should
35

---

Published in Transactions on Machine Learning Research (09/2024)
further reduce the chance of mode collapse and the 3D semi-BNP GAN will reduce the blurriness of the generated samples that may be caused by using the VAF. In future work, our model will be able to generate 3D images and, hence, increase the resolution of images, especially for MRI images. We hope that our future work will make an impact in the field of medical imaging.
Notations
Notation
Definition
N(.))
Normal distribution
LN(.))
Lognormal distribution
ta(;))
E-distribution with 3 degrees of freedom Logistic distribution
LG(.))
Ba
d x matrix with 0.25 the main diagonal and 0.2 off the diagonal d-dimensional column vector of c's
Ia
d x d identical matrix
all distribution notations, the first component represents mean vector and the second component represents the covariance matrix.
A Review the Approximate Bayesian Computation
For given data space X and the set of Borel probability distributions B(3), consider the observations Xi:n drawn from Fruee E B(X). In the standard Bayesian perspective, given parametric model Ba = (Fo
0 E } C. B(x), prior π is placed on the parameter space 0. After observing data Xim, the prior is updated to obtain a posterior distribution given by:
L(x (0)
T (0(x = Je L(x1:n (0)
where L(x 0) II (x, is the likelihood function and So is the density function corresponding to the distribution Fg.
For the standard Bayesian inference to be considered well-spocified, there must cxist a parameter Gg such that the distribution Fe, matches the true data distribution Firuer ie., Fe, Fruee When the parametric model Fe does not contain Freee (i.e., there is 0g O such that Fo, Frree), the model is said to be misspecified. This can lead to problems in the standard Bayesian inference, the posterior distribution will be based on an incorrect model assumption.
ABC addresses some of these challenges, particularly in scenarios where the likelihood function is intractable or difficult to compute. ABC circumvents the need for explicit likelihood evaluations by using simulations, summary statistics, and comparison mechanism between sinulated and observed data. Below shows how ABC works within this framework:
Simulation: Generate synthetic datasets by sampling parameters 0 from the prior distribution æ and simulating data yı. from Fg. Here F' corresponds to the distribution of a generative model with implicitly defined likelihood functions.
Summary Statistics: Reduce both the observed data Xı. and the simulated data Yım to lower- dimensional summary statistics S(x1.)) and S(y:.)) These summary statisties should capture the essential features of the data relevant to the parameters of interest.
Comparison: Compute a distance metric between S(x1..) and S(y).))
Acceptance Criterion: Accept the parameter values 0 if the distance between S(x and S(y less than a predefined tolerance level e. This results in an approximate posterior distribution based on the accepted parameters.
36

---

Published in Transactions on Machine Learning Research (09/2024)
5. Posterior Approximation: The ABC posterior distribution is then given by:
(=(0((())m)) x Je-
f [II= - e) dyi ... dyn
n
where I an indicator function that equals if the distance is within the tolerance e, and otherwise (Beaumont, 2019).
As a result, the ABC posterior distribution converges to the standard posterior approaches 0 (Beaumont, 2019; Dellaporta et al., 2022). This convergence follows from the fact that as decreases, the ABC posterior places increasing weight on 0 values that generate simulated data close to the observed data, effectively approximating the likelihood function. However, as already mentioned, the standard Bayesian posterior can be sensitive to model misspecification. generating process, the Bayesian posterior (and thus the ABC posterior when + 0) may give misleading inferences. This lack of robustness to model misspecification can lead to poor performance in practice.
the assumed model is not a good representation of the true data-
37


Reviews:
[{"review_id": "gGGwsJUwPZ", "reviewer": "Reviewer_ZKRi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposed estimator for MMD is theoretically sound and its performance is promising. In particular, the theoretical results on the semi-BNP GAN in lemmas 4 and 5 are interesting.\nHowever, there are some unclear discussions, especially in the experimental investigation section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a semi-parametric Bayesian nonparametric (semi-BNP) estimator for approximating minimum mean discrepancy (MMD) between two probability distributions.\nThe asymptotic properties of the proposed estimator are presented in theorems and corollary. \nThe authors develop a goodness of fit test with the relative belief (RB) factor between the densities of the proposed posterior- and the prior-based estimators.\nThe authors apply the proposed estimator also to generative adversarial network (GAN) learning. The generator’s properties, generalization error and robustness, are shown in lemmas.\nThe semi-BNP test and the semi-BNP GAN are investigated with synthetic datasets and the MNIST dataset, respectively.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Some adjustments on the following three points are critical.\n\np.6: The discussion right after Eq.(7) says that the proposed test is based on the property of Theorem 1 (iii). However, this property is given in the form of an inequality, which is not so tight as it is derived for example with $1/m < 1$ (right after Eq.(15) of Appendix). I wonder how an accurate test can be constructed based on the inequality without assessing its tightness.  \n\np.11, the third paragraph of Section 6.1: Although it is said that the best performance, AUC=1, is achieved for the bandwidth parameter 80, I wonder if AUC=1 is possible.\nDoesn’t it mean that the type I error is not controlled appropriately? Since 80 is the largest bandwidth parameter tested and reported in the experiment, if it performed best, I wonder why the bandwidth parameters greater than 80 are not considered.\n\np.26: Although Appendix E discusses quality measures of generated samples by GANs, they are not used and reported in the experimental investigation section.\n\nminor comments:\np.2, the last line: continues -> continuous\np.3, l.14 from the bottom: ``Monte Carlo Markov Chain’’ -> ``Markov Chain Monte Carlo’’\np.6, l.14 from the bottom: Corollaries -> Corollary\np.7, l.12: The phrase ``In the sense that’’ is confusing.\np.9, Lemma 5, the definition of $h(N,m,K,\\epsilon)$: $N$ should be $n$. Also, in the assertion i, the function $h$ lacks the third argument, $K$.\np.20, l.1: Chebyshev’s inequality\np.25, l.3 from the bottom: As $\\sigma \\rightarrow $\\infty$, does the Gaussian kernel tend to the linear kernel (not only it tends to 1)? \np.27, l.3: Some & (and) symbols are garbled.", "broader_impact_concerns": "I have no concern on the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QeqtNSRG4N", "reviewer": "Reviewer_Rb2H", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed approach can be applied to hypothesis testing with intractable statistical models where the theoretical analysis and its empirical improvements over baselines make it a suitable alternative\n2. The use of the estimator to design a novel discriminator for GANs along with the accompanying empirical evaluation shows that the idea is also applicable to high dimensional practical settings where goodness-of-fit estimates are required.\n\nWeaknesses:\n\n1. The exposition of the background in Section 2.2. is a bit rushed and may overwhelm readers unfamiliar with the literature (see requested changes below)\n2. The results of the GAN experiments in Fig 3 appear inconclusive but that is a common issue with GAN evaluation, so I do not see it as a major concern.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. The paper introduces a semi-Bayesian nonparametric (semi-BNP) estimator for the Maximum Mean Discrepancy (MMD) measure and demonstrates that it can be used to create an improved goodness-of-fit test compared to frequentist and other BNP methods.\n\n2. The proposed semi-BNP estimator is applied as a discriminator to Generative Adversarial Networks (GANs) where it shows comparable performance with other MMD-based approaches.\n\n3. The paper provides theoretical properties of the proposed estimator and discusses its robustness and consistency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In Section 2.2. can you provide an example of summary statistics and how they are used by ABC to estimate the model parameters.\n\n2. Can you clarify why the ABC tends to approximate the standard Bayesian posterior as the threshold approaches zero and why that would be a problem?\n\n3. Can you also give an example of the exponential of robust loss functions used in Generalized Bayesian Inference?\n\n4. How is this approach connected to sample based variational inference approaches like SVGD [1-3]?\n\n5. Why is the best test performance in Fig 1 always achieved by setting the bandwidth parameter to 80 despite the range of distributions considered?\n\n6. Can the GAN experiments be augmented by considering data from a synthetic/simulated distribution and showing that the proposed discriminator leads to faster/better convergence of the generator samples compared to baselines?\n\n[1] Liu, Qiang, Jason Lee, and Michael Jordan. \"A kernelized Stein discrepancy for goodness-of-fit tests.\" International conference on machine learning. PMLR, 2016.\n\n[2] Liu, Qiang, and Dilin Wang. \"Stein variational gradient descent: A general purpose bayesian inference algorithm.\" Advances in neural information processing systems 29 (2016).\n\n[3] Feng, Yihao, Dilin Wang, and Qiang Liu. \"Learning to draw samples with amortized stein variational gradient descent.\" arXiv preprint arXiv:1707.06626 (2017).", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "j8DL5brNnF", "reviewer": "Reviewer_46Pm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths:\n* As a Bayesian non-parametric people the high level idea is easy to follow.\n\n### Weaknesses:\n* The notation is in a chaos. For example, $H$ is used to denote both base measure and the Gamma random variables. Some terms are used without definition, e.g. $H_N$ and $mmd$ in Eq (8). I suggest the authors make a proofreading and fix the issues like this. Furthermore, I also suggest to use the same font style to define the samples to make the presentation more clear. Now $x$ is also used to denote the samples and make me confused at the first glance.\n* The words in Section 4 are hard to understand. Why not using a flow chart?\n* Several claim in Section 4 is confusing. How can we choose $H=F_2$? If not, the reasoning in the middle of Page 6 is nonsense.\n* For the discussion on how to set $a$, the claims on two statistics tend to $0$ is only correct when the hypothesis holds, then why we cannot make any decision on the hypothesis? Definitely when $a$ is extremely large then prior and posterior has so little difference which may make the testing statistics non-informative, but this does not make the reasoning here sound.\n* The authors mentioned there should be a consistency result for the estimator (9) and (10), but I don’t find it. Also (7) should not converge to $M/i_0$ as it does not depend on these terms. So the claim here is not so rigorous.\n* It is not clear why the proposed new test will overcome the frequentist counterpart. At least the authors should show the (asymptotic) distribution of the testing statistics and compare with the frequentist counterpart. (Note that for MMD we have lots of the tools from U-/V- statistics to characterize it).\n* For the goodness-of-fit testing, how do you set the criterion for the proposed test?\n* For the MNIST experiments, I would like to see some quantitative evaluation like e.g. FID scores to compare.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript proposed a semi-nonparametric estimator for maximum mean discrepancy that can be applied to model with intractable likelihood and unknown distributional form. The authors further show how to perform goodness-of-fit test and generative modeling based on the proposed estimator. Empirical results are provided to demonstrate the effectiveness of the proposed method.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please completely revise Section 4 as the weakness part suggests.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: I am happy to recommend acceptance of the paper.  The authors may consider addressing some of the lingering issues that reviewers had.  In particular, one of the reviewers felt the notation was still hard to follow ("the notation is complicated and it can be hard for the unfamiliar reader to follow") and, as mentioned above, there are some concerns about the amount of empirical evaluation in the paper.  However, I think the paper is solid enough that I will not insist on a revision to be further reviewed.
Decision: Accept (poster)