Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space
Authors: Sandesh Ghimire
Aria Masoomi
Jennifer Dy
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: RKHS
KL Divergence
Sample Complexity
Statistical Learning Theory
Kernel Methods
Bayesian inference
Mutual Information
Reliable estimation

Abstract: Estimating Kullback–Leibler (KL) divergence from samples of two distributions is essential in many machine learning problems. Variational methods using neural network discriminator have been proposed to achieve this task in a scalable manner. However, we noticed that most of these methods using neural network discriminators suffer from high fluctuations (variance) in estimates and instability in training. In this paper, we look at this issue from statistical learning theory and function space complexity perspective to understand why this happens and how to solve it. We argue that the cause of these pathologies is lack of control over the complexity of the neural network discriminator function and could be mitigated by controlling it. To achieve this objective, we 1) present a novel construction of the discriminator in the Reproducing Kernel Hilbert Space (RKHS), 2) theoretically relate the error probability bound of the KL estimates to the complexity of the discriminator in the RKHS space, 3) present a scalable way to control the complexity (RKHS norm) of the discriminator for a reliable estimation of KL divergence, and 4) prove the consistency of the proposed estimator. In three different applications of KL divergence -- estimation of KL, estimation of mutual information and Variational Bayes -- we show that by controlling the complexity as developed in the theory, we are able to reduce the variance of KL estimates and stabilize the training.

Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space
Sandesh Ghimire, , Aria Masoomi, Jennifer Dy
Department of Electrical and Computer Engineering Northeastern University
sandesh@ece.c neu. edu, masoomi.. adnoorchaastter..edd, jdyβece. neu. edu
Abstract
Estimating Kullback-Leibler (KL) divergence from samples of two distributions is essential in many machine learning problems. Variational methods using neural network discriminator have been proposed achieve this task in a scalable manner. However, we noted that most of these methods using neural network discriminators suffer from high fluctuations (variance) in estimates and instability in training. In this paper, we look at this issue from statistical learning theory and function space complexity perspective
understand why this happens and how to solve it. We argue that the cause of these pathologies is lack of control over the complexity of the neural network discriminator function space and could be mitigated by controlling it. To achieve this objective, we 1) present novel construction of the discriminator in the Reproducing Kernel Hilbert Space (RKHS), 2) theoretically relate the error probability bound of the KL estimates to the complexity of the discriminator in the RKHS space, 3) present scalable way to control the complexity (RKHS norm) of the discriminator for reliable estimation of KL divergence, and 4) prove the consistency the proposed estimator. In three different applications of KL divergence estimation of KL, estimation of mutual information and Variational Bayes we show that by controlling the complexity as developed in the theory, we are able reduce the variance of KI. estimates and stabilize the training.
Introduction
Estimating Kullback-Leible (KL) divergence from data samples is an essential component in many machine learning problems including Bayesian inference, calculation of mutual information or methods using information theoretic objectives. Variational formulation of Bayesian Inference requires KI. divergence computation, which could be challenging when we only have finite samples from distributions. Similarly, computation of information theoretic objectives like mutual information requires computation of KI. divergence between the joint and the product of marginals.
KL divergence estimation from samples studied thoroughly by Nguyen al. [1] using variational technique, convex optimization and RKHS norm regularization, while also providing theoretical guarantees and insights. However, their technique requires handling the whole dataset at once and is not scalable. Many modern models need to use KL divergence with large scale data, and often with neural networks, for example total correlation variational autoencoder (TC-VAE) [2], adversarial variational Bayes (AVB) [3], information maximizing GAN (InfoGAN) [4], and amortized MAP [5] all need compute KI. divergence in a deep learning setup. These large scale models have imposed new requirements on KI. divergence estimation like scalability (able to handle large amount of data samples) and minibatch compatibility (compatible with minibatch-hased optimization).
Methods like Nguyen et al. [I] are not suitable in the large scale setup. These modern needs were later met by modern neural network based methods such variational divergence minimization
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

(VDM) [6], mutual information neural estimation (MINE) [7], and discriminator based KL estimation with GAN-type objective [8. 5]. A key attribute of these methods is that they are based on updating a neural-net based discriminator to estimate KI. divergence from a subset of samples making them scalable and minibatch compatible. We, however, noticed that even in simple examples, these methods exhibited pathologies like unreliability (high fluctuation of estimates) or instability during training (KL estimates blowing up). Similar observations of instability of VDM and MINE have also been reported in the literature [8, 9].
Why are these techniques unreliable? In this paper, we attempt to understand the core problem in the KL estimation using discriminator network. We look at it from the perspective of statistical learning theory and discriminator function space complexity and draw insights. Based on these insights, we propose that these fluctuations are consequence of not controlling the smoothness and the complexity of the discriminator function space. Measuring and controlling the complexity of function space itself becomes a difficult problem when the discriminator is a deep neural network. Note that naive approaches to bound complexity by the number of parameters would neither be guaranteed to yield meaningful bound [10], nor be easy to implement.
Therefore, we present the following contributions resolve these challenges. First, we propose a novel construction of the discriminator function using deep network such that it lies in a smooth function space, the Reproducing Kernel Hilbert Space (RKHS). By utilizing the learning theory and the complexity analysis of the RKHS space, we bound the probability of the error of KL- divergence estimates in terms of the radius of RKHS ball and kernel complexity. Using this bound, we propose scalable way to control the complexity by penalizing the RKHS norm. This additional regularization of the complexity is still linear, (0(m) in time complexity with the number of data samples. Then, we prove consistency of the proposed KI. estimator using ideas from empirical process theory. Experimentally, we demonstrate that the proposed way of controlling complexity significantly improves KL divergence estimation and significantly reduce the variance. In mutual information estimation, our method is competitive with the state-of-the-art method and in Variational Bayesian application, our method stabilizes training of MNIST dataset leading to sharp reconstruction.
2 Related Work
Nguyen et al. [1] used variational method to estimate KL divergence from samples of two distribution using convex risk minimization (CRM). They used the RKHS norm as a way to both measure and penalize the complexity of the variational function. However, their work required handling all data at once and solving a convex optimization problem which has time complexity in the order of O(m²) and space complexity in the order of O(m²). Ahuja [11] used similar convex formulation in RKHS space and found difficult to scale. VDM reformulated the f-Divergence objective using Fenchel duality and used neural network to represent the variational function [6]. Although close in concept to [1], it is scalable since it uses a separate discriminator network and adversarial optimization. It, however, did not control the complexity of the neural-net function, and faced issues with stability.
One area of modern application of KL-divergence estimation is in computing mutual information, which is useful in applications such as stabilizing GANs [7]. MINE [7] also optimized a lower bound to KL divergence (Donsker- Varadhan representation). Similar to VDM, MINE used a neural network as the dual variational function: it is thus scalable, but without complexity control and is unstable. Another use of KL divergence is scalable variational inference (VI) as shown in AVB [8]. VI requires KL divergence estimation between the posterior and the prior, which becomes nontrivial when sample based scalable estimation is required. AVB solved using GAN-rype adversarial formulation and neural network discriminator. Similarly, [5] used GAN-type adversarial formulation to obtain KL divergence in amortized inference.
Chen et al. [2] proposed TC-VAE to improve disentanglement by penalizing the KL divergence between the marginal latent distribution and the product of marginals in each dimension. The KL divergence was computed by minibatch-basse sampling strategy that gives biased estimate. Our work is close to Song et al. [9] who investigated the high variance existing mutual information estimators and found that clipping the discriminator output is helpful in reducing variance. In our work, we take a principled way to connect variance to the complexity of discriminator function space and constrain it by penalizing its RKHS norm instead. None of the existing works considered looking at the discriminator function space, connecting its complexity to the unreliable -l--erereee estimation, or mitigating the problem by controlling the complexity.

---

3
Reproducing Kernel Hilbert Space
Let H be Hilbert space of functions f Kernel Hilbert Space (RKHS) if vx E X' the evaluation functional, 6g H + IR, 5g f (x)), is linear and continuous at every f. Every RKHS, Hk. is associated with a unique positive definite kernel, K X' x X + IR, called the reproducing kernel [12], such that it satisfies:
IR defined non-empty space X. It Reproducing
Vx X, (., E Hk. 2. Va X, Vʃ E Hk: (.,
f(
RKHS is studied using a specific integral operator. Let (2(dp) be a space of functions f f + IR that are square integrable with respect Borel probability measure dp X, we denote integral operator Lx= (2(d)) + (d(dp) [13, 14]: (x) = Jx f K(z y)dp(y). This operator will be important in constructing function in RKHS and in computing sample complexity.
4 Problem Formulation and Contribution
GAN-type Objective for KL Estimation: Let p(x) and g(æ) two probability density functions in space X' and we want to estimate their KI. divergence using finite samples from each distribution in scalable and minibatch compatible manner. As shown in [8, 5], this can be achieved by using a discriminator function. First, discriminator f X IR is trained with the objective:
argmax (Fp() Pa(a log a (f(x)) + Fa(a)) log(1 (f(()))
(1)
where is the Sigmoid function given by o(x) = Then it can be shown [8, 5] that the KL divergence K (=)()))(((z)) is given by: KL(p(z)
Ef
Sources of Error: Eq. (1) ambiguous the sense that it silent about the discriminator function space over which the optimization is carried out. Typically, neural network is used as the discriminator. This implies that we are considering the space of functions represented by the neural network of given architecture the hypothesis space, over which the maximization occurs in eq. (1).
Hence, we must rewrite eq. (1) as
f% = argmax log (f(x)) + Eq(r)) log(1 off(())( feh
(2)
where h is the discriminator function space. Furthermore, also approximate integrals in eq. (2) with the Monte Carlo estimate using finite number samples, say m, from the distribution p and q.
fT
argmax feh
Σ lo am-pp(a))
Σ log(1-o([t (((x))))
m
zyma(mz)
Similarly, write KL estimate obtained from, respectively, infinite and finite samples as:
KL()) = (x))) KLm(J)
Σ [s(z)]
(4)
Each these steps introduce some error in our estimate. We can now start our analysis by first decomposing the total estimation error
KTm (fTt KL(*)) = K Kmm T. (fT
KI(SK) + KLUF)- - -I(R)) + KL(F)) - KT(S
Deviation- fram--frmmmm-
Discriminator incuced
Bins
(5)
This equation decomposes total estimation error into three terms: error in KL estimate by the discriminator due to using finite samples in optimization eq. (3), and 3) bias when the considered function space does not contain the optimal function. Here, we concentrate on quantifying the probability of deviation-fromm from-mean error which directly related to observed variance of the KL estimate.
1) deviation from the mean error, 2)
Summary Technical Contributions: Since the deviation the difference between random variable and its mean, we can bound the probability of this error using concentration inequality and complexity the function space of ST function out of neural networks such that it lies on RKHS (Section 5). Then, we bound the probability of
use smooth function space, we propose to construct a
error through the covering number of the RKHS space (Section 6. 6)1,, then control complexity (Section 6.2) and prove consistency of the proposed estimator (Section 7).

---

5 Constructing f in RKHS
The following theorem due to Bach [15] paves way for to construct neural function in RKHS. Theorem 1. [[15] Appendix A] A function f Cy(dp) is in Reproducing Kernel Hilbert Space, Hg. if and only if it can be expressed as
Vx X, (()) - -In (2,
(6)
for certain function g : W + R such that (lal(es(ar) < cc. W is compact space and functions w w+> 1p(x, w) are measurable for all x. The RKHS norm of f satisfies Ilfl S =1 and the kernel K is given by
K(x,t) I,, (2x w) w)dr(w)
Theorem I not only gives condition when a square integrable function is guaranteed lie in RKHS, it also provides us with a recipe to construct a function in RKHS. We construct I using this theorem where v and y are realized with the neural networks and dr is probability measure with Gaussian distribution. We sample w ~ NN, 71) and pass it through two neural networks, & and where &a takes x and as two arguments and g takes only w an argument. More precisely, we consider (z, w) = (1)))" w. where dgg is a neural network with parameters, O. The kernel K, as defined in eq. (7), can be obtained as:
Ko(a", ,-),,,))=- = [,, (ho(a))" www (o())(()) = (x"""")"
(o(())
(8)
where Fw [ww"]] = 71. We sometimes denote the kernel K by Ko to emphasize that it is a function of neural network parameters, 9. Furthermore, representation of in Theorem I provides
us an important upper bound on the RKHS norm of f as Il I llgl 12 (2(r) which we will use later to bound the complexity of the discriminator function space.
Traditionally, kernel K remains fixed and the norm of the function f determines the complexity of the function space. In our formulation, both the RKHS kernel and the norm of f with respect to the kernel changes during training since the kernel depends on neural network parameters, 0. Therefore, the challenge is to tease out how neural parameters, 0 affect the deviation-fromt certatioo--ro--mmen mean error in eq. (5).
Error Analysis and Control
Before starting our analysis, we list assumptions upon which our theory is based. A1. The input domains X and W are compact.
Assumptions:
A2. The functions dh and g are Lipschitz continuous with Lipschitz constants I and I.g respectively.
A3. Higher order derivatives Do (x, t) up to some high order v/2 of kernel K exist.
Assumptions AI is satisfied in our experiments since we consider a bounded set in R. and RD as our domains. Similarly, A2 satisfied since we enforce Lipschitz continuity of Φ and g by using spectral normalization [16]. Assumption A3 is a bit subtle. By the definition of K in (8), higher order derivative of K exists if and only if higher order derivative of de exists. This is readily satisfied by deep networks with smooth activation functions, and is true everywhere except at origin for ReLU activation. Using the boundedness of the input domain and Lipschitz continuity, we show the following proposition which will be useful later in the error bounds.
Proposition 1. Under the assumptions AI, A2, we have sup Ko(æ, < and |19| (2))))) x²t
6.1 Bounding the Error Probability of KL Estimates
Bounding the probability of deviation-fromm from-mean error (eq. (5)) is tricky since, in our case, the kernel is not fixed and are also optimizing over them. We bound it in two steps: 1) we derive bound for a fixed kernel, 2) we take supremum of this bound over all the kernels parameterized by 8.
For a fixed kernel, we first bound the probability of deviation-froms number in Lemma 1. We then use an estimate of the covering number of RKHS due to [14] to relate the bound to kernel Ke in Theorem 2. identifying the role of neural networks in this error bound.
emor in terms of the covering

---

Lemma 1. Let JH be the optimal discriminator function in an RKHS H. which is bounded by M with respect to |1-1|so Let K [f -. fRg (x.) and K KL(fHx ) UR (a) be the estimate of KL divergence from m samples and that by using the true distribution p(z) respectively.
Then the probability of error at some accuracy level, 6, is oower-bounded as:
mc²
Prob. (KLLmfR KLURK 2) 4VSK
exp(
4M2
where NHH q) denotes the covering number of an RKHS space Hg with disks of radius 1,, and SK- = sup K (z,t) E) which we refer as kernel complexity.
2,1
Proof Sketch. We cover RKHS with discs of radius 7) 4/SK" Within this radius, the deviation bound deviation probability the center of disc and apply
does not change much. So, union bound over all the discs. To bound deviation probability at the center, we apply Hoeffding s inequality and applying union bound simply leads to counting number of discs which exactly the covering number. See supplementary materials for the full proof.
Lemma bounds the probability error in terms of the covering number of the RKHS space. Note that the radius of the disc is inversely related to SK which indicates how complex the RKHS space defined by the kernel Kę is. Here Kę depends on the neural network parameters 0. Therefore, we denote Sy function of 0 Sk (0) and term it kernel complexity. Next, we use Lemma 2 due to [14] to obtain an error bound in estimating KL. divergence with finite samples in Theorem 2.
Lemma ((14), Let K : X x X - R be T Mercer kernel and the inclusion IK "HK + C(X) be the compact embedding defined by K to the Banach space C(X). Let Bn be the ball of radius R RKHS Hx. Then Vn 0, R > 0, n, we have
2m
RC 7
(I S
where N gives the covering number of the space K (Bx) with discs of radius 1,, and n represents the dimension of the input space X. Cv is given by C. C. V1.L where L, is a linear embedding [1 denotes operator norm and from square integrable space L2(dp) to the Sobolev space /2 C2 is constant.
To prove Lemma [14], the RKHS space is embedded in the Sobolev Space H using L, and then the covering number of the Sobolev space is used. Thus the norm of L, and the degree of Sobolev space, v/2, appears in the covering number of a ball in Hr. In Theorem 2, we use Lemma and 2 to bound the estimation error of KI. divergence.
Theorem 2. Let ( and K Lm (f be the estimates of KL divergence obtained by using true distribution p(x) and m samples respectively as described in Lemma 1, then the probability of error in the estimation at the error level is given by:
2n
ARCC Sp
L
mc²
i.e. Sp, Lp correspond to a kernel for which the Prob. (oob.((KK [. (1 KI(JAM 5()
-eepp
4.2²
where Cp V L
Ko
bound maximum.
Proof. We prove this in two steps: First we obtain error bound for fixed kernel space and apply supremum over all For any RKHS HK., with fixed kernel Kg, we have
Sx L. KL )1 2| e) < exxp [( [ARC, (K
2a
[ARC.
me²
Prob.[[KLp) (
(10)
4M2
We prove this error bound as follows. Lemma gives the covering number of an RKHS ball of radius R. which we apply to Lemma 1, We fix the radius of discs to 4/SK in Lemma I and substitute
C, C. [2()) to obtain c10).
5

---

Since we are continuously changing & during training, the kernel also changes. Hence, to find the upper bound over all possible kernels, we take the supremum over all kernels.
Prob.( Lm (fR) K ((RR
2 E)
S sup Prob.(KK Lm (fix,) Kg
KL(SKK, )1 E)
(11)
2pl
me
2exp
(12)
c
4M²
where Sp = -Kx(0g and Lp = (0)) i.e. So and Lp correspond to kernel complexity and Sobolev operator norm corresponding to optimal kernel Ko, that extremizes eq. (11). Theorem statement readily follows from eq. (12)
Theorem shows that the error increases exponentially with the radius of the RKHS space, R. complexity of the kernel Sx())), and the norm of the Sobolev space embedding opera- tor 1|. The Sobolev embedding operator, Lp. is a mapping from (2 to the Sobolev space H1/2 It can be shown [14] that the operator norm can be bounded as || S p(x) Ejalsv/2 sup (D² Ko, 1))?, where p the measure of the input space X. Therefore, z,tEX
the norm Lpll directly measures smoothness of Kg, while Sk only depends on the supremum value of Ko Cp is constant not depending Sp. Lp or R. and it is always finite (see [14, 17] for more details). Sp and Lp are related to á, y through the kernel Ko Sp is defined as Sp sup 2,(
K K(x, t) = sup ,20, (x) (f), so it is directly related to the network de,, We show that this is xz!
finite due to Lipschitz constraint on Φ8, as described in Assumption A2 (see Supplementary material for proof). Lp is upper bounded follows: || S 2 su/2 sup (D; Ko, (z, t))²
=.Ex
p(x) Ejolsu/2 sup (02 (z))o, where D a differentiation operator. Therefore, Lp is
Z,EE finite if the higher order derivatives of Φo, exist and are finite.
6.2
Complexity Control
From Theorem 2, we see that the error probability could be decreased by decreasing R. ||-E, and Sk (0p). Using argument similar to the proof of Proposition 1, can show that the Lipschitz constraint on dy also affects Sx and may affect || In our experiments, however, we fix the Lipschitz constraints during optimization and do not change Sy and || dynamically. Here, focus on the norm, R from Theorem 2. From Theorem we know that the RKHS norm is upper bounded by the norm of function g as || S 119112 We have access finite approximation of [[ (191(2(cr)) by construction and hence, we can use it to penalize the complexity during optimization. To obtain the optimal discriminator S*" we optimize the following objective with an extra penalization of the upper bound, i.e. [|gllc.(dr) on the RKHS norm of ʃ:
SH" argmax- feh
1
Σ log logo(((z;))
Σ alma((ay)
log(1 ((((x)))) Ao (19li. (13)
m
m
m
The regularization term prevents the radius of RKHS ball from growing, maintaining low error probability. Optimization of eq. (13) w.r.t. neural network parameters 0 allows dynamic control of the complexity of the discriminator function on the fly in a scalable and efficient way. Note that, computation of [ا (dr) requires randomly sampling w ~ NO, II) and passing through neural network g independent of the data Zz. Therefore, the computational complexity of optimization is O(m), it will remain the same after incorporating this additional term, i.e. regularization does not increase asymptotic time complexity which linear with the number of samples, m.
7 Variance and Consistency of the Estimate
7.1 Variance Analysis
Theorem 2 gives an upper bound on the probability of error. Intuitively, the variance and probability of error behave similarly for many distributions, i.e. higher variance might indicate higher probability of error. Below we quantify this intuition for Gaussian distributed estimate:

---

3.0
NN discr
NN diszr
NN discr
2.75
RX3H5 discr
RKHS diser
RXHS diser
[d]
2.2
Scatter
2.00
42-
100
20-
125
blas varlance
blas variance
0.6-
[b]
var ance
Bios Variance
Plat 0.4
(RIHSS dier)
20-
200
0.0
10
10:2
10"1
10
10at
10
10"2
109
À in log scale
Figure 1: a) Top scatter plot compares KT. divergence estimates hetween a method using Neural network discriminator without complexity control (red) and that using RKHS discriminator with compelxity control (blue); b) In the bottom, show the effect of varying the regularization parameter bias and variance while using the RKHS discriminator with complexity control as in eq.(13).
Theorem 3. X (f%) be the estimated KL divergence using samples described in Theorem 2. Assuming that X follows Gaussian distribution ~ N(u, <²), we can obtain an upper bound on the standard deviation of the estimate as follows:
erf 4exp
[
ARCCPV E
mi 4M
+
where erf is the Gauss error function and is a monotonic function.
Theorem suggests that by decreasing R, the radius of the RKHS ball, the variance of the estimate could be decreased. Experimentally, we observe that the variance decreases as we penalize the RKHS norm more, consistent with the spirit of Theorem 3.
Note that Theorem 3 makes strong assumption that the estimate distributed as Gaussian distribution. While it gives good intuition about the decay of variance the complexity increases, is natural to inquire about the validity of this type of relation in a more general sense without assumption on the probability distribution of the estimate. To make general statement, the key idea is to understand how the cumulative distribution function (CDF) is related to the variance. To clarify this point further, let's look the (33) in the proof of Theorem in supplementary material:
2n
1- 1- + c) < 2exp
ARC. 5x 1x
me²
This equation connects the CDF to the
1M
variables like Sx, R, L, of the discriminator function space without assuming anything about the shape of the distribution. For Gaussian distribution, we plug in CDF of Gaussian distribution,
(+) = 1 + erf( ($-7) and obtain the result of Theorem 3. For any other distribution, we can carry out similar analysis, key factor that determines the behavior between the variance and the discriminator complexity is how variance appears in the CDF expression. For example, in both Gaussian distribution and in exponential distribution,
know that the relation between CDF function and variance is inversely related. We believe that as long as this inverse type of relation between CDF and variance holds, we can obtain statements like Theorem for other distributions as well. This leads to key insight: similar to the Gaussian case, the decaying behuvior of the variance with decreasing complexity holds as long as the estimate is distributed such that its CDF function has inverse relation with the variance.
7

---

Estimated
Ground Truth
CPC
Nej
SMILE (T- 1.50
SMILE (1- 5.00
SMILE
(T T---
Mi(nntt)
Am
2007
2008
6000
MAY
DPV
AMP
WPh
2VAR
4000
Ours(tamodo 0)
Ourstlambda Se-05)
Oursilambdo
Ours/ambds
Oursilambca
2450
Figure 2:
Comparing method with CPC [19], convex risk minimizationnNWW.) [1] and SMILE [9] regarding
mutual information estimation between two variables.
7.2 Consistency of Estimates
regularized objective to obtain an optimal discriminator instead of unregular- ized objective as in e3.( It is important to show that the estimator of KI. divergence obtained by using this regularized objective is consistent and approaches the true estimate in the limiting case as the number of data samples grows to infinity. In the following theorem, we show this.
In (13),
Theorem 4. Let f* and f be optimal discriminators as described in eq. (1) and eq. (13) respectively, and the KL estimate is given by LL() (f))), K Km (f) = m E (xxx(e)((x))) Then, in the limiting case as m + oo, JKLm(f) KL(f)) + 0 in probability.
Proof Sketch. The difference between the true KI. divergence and the estimated KL. divergence can be divided into three terms as shown in eq. (5). We assume that our function space is rich enough to contain the true solution, driving bias to zero. From Theorem 2, we see that in the limiting case of m - the that the discriminator induced error (second term in (5) also goes to 0 as m + 00.
error goes O. Therefore, the key step that remains be shown is
It can be shown if we can prove that the optimal discriminator in eq. (13) approaches the optimal discriminator in eq. (2). To prove this, we show that the argument being maximized by f approaches the argument being maximized by f in the limiting case. To show this, we need to show that the function space, log f. is Glivenko Cantelli [18), which we prove in following steps:
1. We show that f is Lipschitz continuous by definition and due Lipschitz continuity of de. Then we show that log o f is Lipschitz continuous if f is Lipschitz continuous.
2. Then we show that for class of functions with Lipschitz constant L, the metric entropy, log N, can be obtained in terms of L and entropy number of the bounded input space, X.
3. Since the metric entropy does not grow with the number of samples m, we show that 1 log N m which lets us show that log of belongs to Glivenko Cantelli class of functions by using Theorem 2.4.3 from [18]. See supplementary material for the complete proof.
8 Experimental Results
We present results on three applications of KL. divergence estimation: 1. KL estimation between simple Gaussian distributions, 2. Mutual information estimation, Variational Bayes. In our experiments, the RKHS discriminator is constructed with v and networks as described in Section 5, where the network v is very close to a regular neural network. In two experiments, we compare our results with the models using regular neural net discriminator to ensure that the difference in performance between RKHS and regular neural network is not due to architectural difference. Our code is publicly available at https: :// /ggittu....... cww homm/sandesggh/Bee.. Reeehh/eelllbae wibelllllllee.. waaal-KKl -KL--KLttttte esstimaiion
8

---

Neural Net discriminator
5
8
8
0
RKHS discriminator
9
9
7
7
8
Epochs
100
300
500
700
900
(a)
Neural Net ciscriminator
RKHS discriminator
1000
300
Reconstruution KL div.
Raxconddruction
600
400
100
50
Poachs
820
1000
200
4001
Epochs
600
8BC
1000
(b)
(c)
Figure 3:
Comparison MNIST digit reconstruction using AVB autoencoder model [8]. Trace KL
divergence and reconstruction loss in AVB model with Neural network discriminator (b) and RKHS discriminator in (c).
KL Estimation between Two Gaussians We assume that have finite sets of samples from two distributions. We further assume that we are required to apply minibatch based optimization. We consider estimating KL divergence between two Gaussian distributions in 2D, where we know the analytical KL divergence between the two distributions as the ground truth. We consider three different pairs of distributions corresponding true KL divergence values of 1.3, 13.8 and 38. 38.29, respectively and use m = 5000 samples from each distribution to estimate KL in the finite case. We repeat the estimation experiments with random initialization 30 times and report the mean, standard deviation, scatter and box plots.
Fig. top row compares the estimation of KL divergence with regular neural and RKHS discrimi- nator with complexity control based on eq. (13). With our proposed RKHS discriminator, the KL estimates are significantly more reliable and accurate: error reduced from 0.5 0.04, 5.8 1.07 and 60.6 to 9.7 and variance reduced from 0.2 to 0.002, 223 to 4.4 and 3521 to 33 for true KL 1.3, 13.8 and 38.29 respectively. In Fig. bottom row, we investigate our complexity control method on the effect of varying the regularization parameter À >o/m. mm.. As expected, increasing regularization parameter penalizes more on the RKHS norm and therefore reduces variance. This is consistent with our theory. Regarding bias, however, as we increase the X, the bias decreases and then starts to increase. Hence, one needs strike a balance between bias and variance while choosing
Mutual Information Estimation Computation of mutual information is a direct use case of KL divergence computation. replicate the experimental setup of [20, 9] to estimate mutual information between (ar, y) drawn from 20-d Gaussian distributions, where the mutual information is increased by step size of 2 from 2 to 10. We compare the performance of our method with traditional KL divergence computation methods like contrastive predictive coding (CPC) [19], convex risk minimization (NWJ) [1] and SMILE [9]. In Fig.2, our method with RKHS discriminator (with X = 1e-5) performs better than CPC [19] and NWJ [1], and is competitive with the state-of-the-arr SMILE [9]. row, also show the effect of regularization parameter À in method. Similar to the previous experiment, increasing the regularization parameter decreases the variance and increases the bias. consistent with our theoretical insights about the effect of reducing RKHS norm variance.
bottom
Adversarial Variational Bayes Variational Bayes requires KL divergence estimation. When we do not have access to analytical form of the posterior/prior distributions, but only have access to samples, we need to estimate KI. divergence from samples. Adversarial Variational Bayes (AVB) [8] presents way achieve this using discriminator network. We adopt this setup and demonstrate that the training becomes unstable if we do not constrain the complexity of the discriminator. First,

---

Epoch
100
200
300
400
500
600
700
800
900
NNet
49.64
43.25
47.52
44.29
51.31
52.45
148.9
157.5
261.9
RKHS
37.58
33.18
31.46
31.39
30.36
29.37
28.17 17
28.18
28.17
Table 1: FID score (smaller the better) at different epochs of training between the reconstruction
(using respective discriminator in VAE) and the ground truth.
train AVB on MNIST dataset with a simple neural network discriminator architecture. As the training progresses, the KL divergence blows up after about 500 epochs (Fig. 3()) and the reconstruction starts to get worse (Fig. 3())). We modify the same architecture according to our construction such that the discriminator lies in the RKHS and then penalize the RKHS norm in eq. (13). It stabilizes the training for a large number of epochs as shown in Fig. 3(c) and the reconstruction does not deteriorate as the training progresses, resulting into sharp reconstruction (Fig. 3(a)). To make this comparison more precise and quantitative, we compute FID score between reconstruction and the ground truth after each epoch as tabulated in Table In the earlier epochs. the FID score ( and the reconstruction in Fig.3(a)) is okay even for the neural net discriminator, However, this score is worse than using the RKHS discriminator, where the scores are the range of 30-40 upto epoch 400. For the neural network discriminator, the score increases (worsens) in the mid epochs and becomes completely unstable with FID score shooting up to 262 at epoch 900. On the other hand, for the RKHS discriminator, the score steadily and smoothly decreases the epoch increases reaching the best 28. 17 at epoch 900. This experiment demonstrates that the proposed RKHS discriminator with norm regularization is both reliable and effective in terms of standard metric like FID.
We want clarify that this instability in training neural net discriminator present we use basic discriminator architecture. It does not mean that there exists no other method to design a stable neural net discriminator. In fact, AVB [8] presents a discriminator that adds additional inner product structure to stabilize the discriminator training. Our point here is that we can stabilize the training by ensuring that the discriminator lies in a well behaved function space (the RKHS) and controlling its complexity, consistent with our theory.
9 Limitations, Discussion and Conclusion
The proposed construction of neural function in RKHS exhibits good properties of both the deep learning and kernel methods. However, it requires constructing two separate deep networks, y and g. It makes our model a bit bulky and also requires more parameter due to additional g. Moreover, currently our RKHS discriminator's output is scalar; generalizing this function to a multivariate output could make our model bulkier and increase parameters even more. Second limitation is the requirement of higher order derivative of kernel K in assumption A3. While this requirement is satisfied if smooth activation function is used in ④g. for activations like ReLU or LeakyReLU, the derivatives exist everywhere except at the origin. In these cases, we need to carefully investigate if we can use subgradients to define operator norm 111
Limitations:
We have shown that using regular neural network discriminator in estimating KL divergence results in unreliable estimation if the complexity of the function space is not controlled. We then showed a solution by constructing discriminator function in RKHS space using neural networks and penalizing its complexity in a scalable way. Although the idea to use RKHS norm to penalize complexity is not new (see for example [1]), it is not clear how to use this idea directly on the function f. In traditional kernel methods, algorithms often do not work with RKHS function f directly, but rather work with kernel matrix, K by using, for example, the Representer Theorem [21]] In the case of big data, working with the big kernel matrix is computationally expensive although some methods have been proposed to speed up the computation, like Random Fourier Feature [22]. We propose different view by directly constructing a function in RKHS space, which led us scalable algorithm while incorporating the advantages of neural networks. Moreover, our representation could also be seen as an improvement over RFF by using neural basis, v instead of Fourier basis. The idea of constructing neural-net function RKHS and complexity control could also be useful in stabilizing GANs in general. Currently, the most successful way to stabilize GANs to enforce smoothness by gradient penalization [23, 24, 25J. On the light of the present analysis, gradient penalty could also be thought as way to control the complexity of the discriminator.
Discussion and Conclusion:
10

---

10 Acknowledgements and Disclosure of Funding
We would like to express our deep gratitude to Prof. Linwei Wang and Dr. Prashnna Kumar Gyawali for the helpful discussion at the initial stage of this work. Their discussion was especially important in the inception of this project, in identifying the need for KL estimation from samples and limitation of existing KL estimation approaches. We would like to thank Prof. Dana H. Brooks for the helpful discussions and giving exposure to this work in the early stage. We are grateful to Prof. Octavia Camps for her support, encouragements and resources. We are very thankful to Zulqarnain Khan for his feedback on the early draft of the paper.
Funding disclosure: We are thankful to the support from NIH/NCI R01CA240771 and NIH/NHLBI U01 HL88888855....
References
[1] X. Nguyen, M. Wainwright, and M. I. Jordan, "Estimating divergence functionals and the likelihood ratio by convex risk minimization," IEEE Transactions on Information Theory, vol. 56, no. 11, pp. 5847-5861, 2010.
[2] Q. Chen, Li, R. Grosse, and D. K. Duvenaud, "Isolating sources of disentanglement in variational autoencoders," in Advances in Neural Information Processing Systems, pp. 2610- 2620, 2018.
[3] L. Mescheder, A. Geiger, and S. Nowozin, "Which training methods for gans do actually converge?," in International Conference on Machine Learning, pp. 3481-3490, 2018.
[4] X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel, "Infogan: Inter- pretable representation learning by information maximizing generative adversarial nets," in Advances in neural information processing systems, pp- 2172-2188 2016.
[5] C. K. Sønderby, J. Caballero, L. Theis, W. Shi, and F. Huszár, "Amortised map inference for image super-resolution, ICLR, 2017.
[6] S. Nowozin, B. Cseke, and R. Tomioka, "f-gan: Training generative neural samplers using variational divergence minimization," in Advances in neural information processing systems.
pp. 271-279, 2016.
[7] M. I. Belghazi, A. Baratin, S. Rajeshwar, S. Ozair, Y. Bengio, A. Courville, and D. Hjelm, "Mutual information neural estimation," in International Conference on Machine Learning. pp. 531-540 2018.
[8] L. Mescheder, S. Nowozin, and A. Geiger, "Adversarial variational hayes: Unifying variational autoencoders and generative adversarial networks," International Conference on Machine Learning (ICML). 2017.
[9] J. Song and S. Ermon, "Understanding the limitations of variational mutual information estima- tors," in International Conference on Learning Representations, 2020.
[10] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, "Understanding deep learning requires rethinking generalization," arXiv preprint arXiv:161 1.05530, 2016.
[11] K. Ahuja, "Estimating kullback-leibler divergence using kernel machines," in 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pp. 690-696, IEEE, 2019.
[12] A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer US, 2011.
[13] F. Bach, "On the equivalence between kemnel quadrature rules and random feature expansions." The Journal of Machine Learning Research, vol. 18, no. pp. 714-751, 2017.
[14] Cucker and S. Smale, "On the mathematical foundations of learning." " Bulletin of the American mathematical society, vol. 39, no. 1, pp. 1-49, 2002.
[15] F. Bach, "Breaking the curse of dimensionality with convex neural networks," " The Journal of Machine Learning Research, vol. 18, no. 1, pp. 629-681, 2017.
[16] Miyato, Kataoka, M. Koyama, and Y. Yoshida, "Spectral normalization for generative adversarial networks," in International Conference on Learning Representations, 2018

---

[17] D. E. Edmunds and H. Triebel, Function Spaces, Entropy Numbers, Differential Operators. Cambridge Tracts Mathematics, Cambridge University Press. 1996.
[18] A. W. Van Der Vaart and J. A. Wellner, "Weak convergence," in Weak convergence and empirical processes, Springer, 1996.
[19] A. v. d. Oord, Y. Li, and O. Vinyals, "Representation learning with contrastive predictive coding," " arXiv preprint arXiv: 507.03748, 2018.
[20] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker, "On variational bounds of mutual information," " in International Conference Machine Learning, pp. 5171-5180, PMLR, 2019.
[21] B. Schölkopf, A. J. Smola, F. Bach, et al., Learning with kernels: support vector machines, regularization. optimization, and beyond. MIT press, 2002.
[22] A. Rahimi, B. Recht, et al., "Random features for scale kernel machines.," " in Neural Information Processing Systems, vol. p. 5, Citeseer, 2007.
[23] M. Arjovsky, S. Chintala, and L. Bottou, "Wasserstein generative adversarial networks," in International Conference on Machine Learning, pp. 114-2,,,222 2017.
[24] Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville, "Improved training of wasserstein gans," in Advances in Neural Information Processing Systems 30 (I. Guyon, U. V. Luxburg, S. Bengio, Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.),
pp. 5767-5777, Curran Associates, Inc., 2017.
[25] M. Binkowski, D. J. Sutherland, M. Arbel, and A. Gretton, "Demystifying MMD GANs," in International Conference on Learning Representations, 2018
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] We have dedicated separate
paragraph describing limitations in Section 9.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] We have discussed potential societal impacts in our supplementary material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] We devoted paragraph to describe our assumptions in Section 6. Whenever possible, we have mentioned our assumptions throughout the paper.
(b) Did you include complete proofs of all theoretical results? [Yes] Proof sketch is provided for most of the main results in the paper. Full proof of all the theoretical results is included in the supplementary material.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] We describe our code/architectures in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We describe some in the main paper and the rest in the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We include these information in the supplementary material.
4. If you using existing assets (c.g., code, data, models) or curating/releasnng assets...
12

---

(a) If your work uses existing assets, did you cite the creators? [Yes] The code repository we used has been cited in the Supplementary material and the paper has been cited in the main text.
(b) Did you mention the license of the assets? [N/A] The code/data used freely available.
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "V8CGbQ_zAdd", "reviewer": "Reviewer_kqvB", "summary": "This papers studies the problem of estimating KL divergence from samples via variational methods with neural network discriminators. They propose a novel way to construct the discriminator in some RKHS and prove the consistency of the resulting estimator. By penalizing the RKHS norm of the discriminator, they reduce the variance of their estimator and stabilize the training.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\n\nAlthough their idea of constructing a function in the RKHS is not new, the usage of this construction in GAN-type objective for KL estimation seems to be novel. Related work is adequately cited and it is clear how their method differs from previous ones.\n\n**Quality**\n\nThe submission is technically sound and most of the claims are well supported. Theoretical results are solid and informative. Experimental results are reasonable and show the improvements of the proposed method over previous ones. Below are several comments/questions I have:\n\n1. In Abstract and line 40, they claim that methods with a neural-net based discriminator suffers from unreliability or instability. However, they do not include the two methods, VDM and MINE, cited in the paper in the experiments. I would like to see results that can support the claim.\n2. In Theorem 2, the quantities $C_p$, $S_p$, $\\mathscr{L}_p$ are not discussed. How are they related to the architecture of $\\phi_\\theta$ and $\\gamma$? Are they always finite?\n3. The meaning of $\\text{sup}_{K_\\theta}$ is unclear (e.g., in Theorem 2). Does that include all possible architectures of $\\phi_\\theta$ and all values of $\\gamma$? If you maximize over $\\gamma$, then the finiteness in the above comment may be an issue.\n4. Since what you are advertising is the regularized version of the estimator, it would be better to include theoretical analysis for the consistency of this regularized estimator.\n5. Spectral normalization is used in the proposed discriminator but not in the baseline. This seems to be unfair. Do you have an explanation about it?\n\n**Clarity**\n\nThe paper is clearly written and easy to follow.\n\n**Significance**\n\nThe results are likely interesting and useful to the community. The idea of constructing a function in RKHS and controlling its complexity could be used in GANs in general.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See comments above for suggestions.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "OsShS1lsmgH", "reviewer": "Reviewer_2N7B", "summary": "The authors consider the problem of KL divergence estimation. Modern machine learning methods for KL divergence estimation make use of neural networks as function approximators. The authors assert that many of the challenges with using such methods is the uncontrolled complexity of the function space. They propose to resolve this by constructing their neural network functions to be RKHS functions, and control the RKHS norm. They prove learning-theoretic results about their method, showing that it is consistent and exhibits controllable fluctuations around its mean (deviation-from-mean error).", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nThe paper is nicely written and introduces the problem well. I haven't read the proofs in detail but I believe the authors make a solid technical contribution. They are honest about the limitations of their method (discussion of assumptions and paragraph at end) and do not overstate their results, which is refreshing. They provide sketches of proofs in the main text to provide intuition / high level understanding of the results. \n\nI can't comment on the novelty of the idea; I am unaware of work that tries to use RKHS norms to control neural network function complexity in this way, but I am not an expert in this field and am not up to date with the latest literature. If this is a novel idea then I find it quite exciting and is worthy of merit in its own right (regardless of the rest of the paper). If it is not novel, appropriate citations should be added.\n\nI'm generally happy with it but have a few points I'd like to be addressed. I've given \"weak accept\" but would be willing to raise this rating if the authors can address them: \n\n- the results are all cast in terms of the L2(\\tau) norm of g. How does this relate to the RKHS norm of f? At the beginning of the paper, I thought you would control the RKHS norm of f -- I think this is implicitly being done, but would be great if you could make this more explicit / comment on this.\n- variance analysis: I understand that theorem 3 only applies under the strong assumption of the deviation-from-mean being Gaussian. Is it not possible to make some statements about how the variance decays under some other weaker assumptions? If not, are there (perhaps pathological) cases where the variance does not decay?\n- at the beginning you claim that the existing methods suffer due to having uncontrolled function complexity. Do you have any evidence to support this claim? Do these other methods not use regularisation in practice?\n\n\nNon-technical issues:\n- Figures: I think you're converting your figures to PNG or JPG before embedding them into the document. Much better is to save them as PDFs -- that way they are not blurry when you zoom in on a computer.\n- I think you are maybe just writing KL in math mode, but if you use use add this latex command: \\newcommand{\\KL}{\\mathrm{KL}} and then use \\KL it may look better depending on your taste (feel free to ignore this if you don't like it though).\n\n\n \nOther detailed comments:\n- line 54: gap before (RKHS)\n- line 138: \"w ~ N(O, \\gamma I)\" -> is this saying that \\tau is the normal distribution? Could you state that explicitly if so?\n- line 140: could you maybe make it explicit here that \\phi (and I guess also g?) will be implemented as neural networks?\n- line 144: \"both the RKHS kernel and its norm\" -> \"both the RKHS kernel and the norm of f\"\n- You don't really discuss the RKHS norm of f directly. Could you at least briefly mention it here, in particular how it relates to \\phi and g? \n- line 151: both \\tau and h have already been used, but I don't think the occurrences in this line are referring to either of them?\n- line 166: what does M-bounded mean?\n- line 194: approximately what kind of order of magnitude do you expect the positive part inside the exp to be? (The practical usefulness of the results does depend on this.)\n- equation (14): I guess the \\sigma on the left hand side is supposed to be the standard deviation of  the estimator? I\n- line 264: typo \"Guaussian\"\n\n\n[Edit: raised score by 1 point after authors addressed comments in review]", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "-KjG5aJ_kB", "reviewer": "Reviewer_mM61", "summary": "An improved method for estimating the KL divergence between two distributions based on samples. To estimate the log density ratio, the authors proposed to train a discriminator while constraining the family of the discriminator function. It constructs the function using the defined in F. Bach (2017), which is a convolution between two network functions. The authors did a careful (though no very thorough) performance analysis of this method, and demonstrated the performance in simple datasets for three different tasks involving the KL divergence. ", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of constructing a constrained RHKS function for this density ratio estimation appears to be new. It provides a different formulation of the deep kernels used in supervised and unsupervised settings. The analysis method uses standard theoretical methods, such as concentration inequalities and covering numbers. Though some analyses involved much deeper derivations/proofs than others, and the experiments were not yet strong enough, I think the method and theoretical contributions are clear.\n\nThe written quality needs some improvement. For example, in eqn (5), $\\psi$ is only defined later, and $\\tau$ is never defined. \n\nHere are some detailed comments:\nI cannot fully follow the logic in the proof of Lemma 1, and there appear to be some (minor) inconsistencies. It also appears that what would be useful is the contrapositive of Eqn (17), i.e. $\\ell_z(f_j)\\le \\epsilon \\Longrightarrow \\dots le 2\\epsilon$, and then we can proceed by writing $\\text{Prob}(\\dots \\le 2\\epsilon)$ rather than a  $\\ge$ statement. Also, is $\\eta$ set to $\\frac{\\epsilon}{2\\sqrt{S_K}}$ (in the proof) or $\\frac{\\epsilon}{4\\sqrt{S_K}}$ (in the main text)?\n\nLine 219. How's the norm of g related to R in Theorem 2?\n\nThere are a few questions regarding the experimental results.\n1. Figure 1: top row shows a small bias for the blue clusters. Does this never go away? The bias in the bottom row also doesn't go away. \n2. Figure 3: the neural network decoding seems bad at the end of training, but before the loss blows up, there seems to be a relatively stable region. How does the model at that stage perform in terms of decoding? Also the y axis are not equal, making the results a little hard to compare. Also, although the authors reminded the readers that the point is on stability, but is the proposed method reliability and effective, in terms of standard metrics like the log-likelihood or sample quality (FID, KID)?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "While the theory might be clear, the experiments were less than sufficient. They were mostly simple Gaussian distributions, and MNIST only appears to demonstrate stability. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Converging reviews regarding the acceptance of the paper after a fruitful discussion between reviewers and authors, where the latter took time to clarify all the questions raised.
The recommendation for this paper is to accept it, counting on the authors to take into account the outcomes of the discussions with all the reviewers in the final version of the work (preferably in the main text rather than in the supplementary material).


Decision: Accept (Spotlight)