Scaling Densities For Improved Density Ratio Estimation
Authors: Akash Srivastava
Seungwook Han
Benjamin Rhodes
Kai Xu
Michael U. Gutmann
Venue: ICLR 2022 Submitted (2022)
Keywords: density ratio estimation
scaled bregman divergence
mutual information estimation
representation learning

Abstract: Estimating the discrepancy between two densities ($p$ and $q$) is central to machine learning. Most frequently used methods for the quantification of this discrepancy capture it as a function of the ratio of the densities $p/q$. In practice, closed-form expressions for these densities or their ratio are rarely available. As such, estimating density ratios accurately using only samples from $p$ and $q$ is of high significance and has led to a flurry of recent work in this direction. Among these, binary classification based density ratio estimators have shown great promise and have been extremely successful in specialized domains. However, estimating the density ratio using a binary classifier, when the samples from the densities are well separated, remains challenging. In this work, we first show that the state-of-the-art solutions for such well-separated cases have limited applicability, may suffer from theoretical inconsistencies or lack formal guarantees and therefore perform poorly in the general case. We then present an alternative framework for density ratio estimation that is motivated by the scaled-Bregman divergence. Our proposal is to scale the densities $p$ and $q$ by another density $m$ and estimate $\log p/q$ as $\log p/m - \log q/m$. We show that if the scaling measures are constructed such that they overlap with $p$ and $q$, then a single multi-class logistic regression can be trained to accurately recover $p/m$ and $q/m$ on samples from $p, q$ and $m$. We formally justify our method with the scaled-Bregman theorem and show that it does not suffer from the issues that plague the existing solutions.

Under review as a conference paper at ICLR 2021
SCALING DENSITIES FOR IMPROVED DENSITY RATIO ESTIMATION
Anonymous authors Paper under double-blind review
ABSTRACT
Estimating the discrepancy between two densities (p and q) is central to machine learning. Most frequently used methods for the quantification of this discrepaney captuee it as function of the ratio of the densities p/q. In practice, closed-form expressions for these densities or their ratio are rarely available. As such, esti- mating density ratios accurately using only samples from and q is of high sig- nificance and has led to a flurry of recent work this direction. Among these, binary classification based density ratio estimators have shown great promise and have been extremely successful in specialized domains. However, estimating the density ratio using a binary classifier, when the samples from the densities are well separated, remains challenging. In this work, we first show that the state-of-the-ee solutions for such well separated cases have limited applicability, may suffer from theoretical inconsistencies or lack formal guarantees and therefore perform poorly in the general case. We then present alternative framework for density ratio es- timation that is motivated by the -caled-Bregman divergence. Our proposal is to scale the densities p and q by another density m and estimate log p/q as log p/ mm log q/ g/m. We show that if the scaling measures are constructed such that they overlap with p and g, then a single multi-class logistic regression can trained accurately recover p/ m and m on samples from p. and m. We formally jus- tify our method with the from the issues that plague the existing solutions. We provide a large battery of empirical evaluations of our method with both synthetic and real datasets on the tasks of density ratio estimation, mutual information estimation, and representa- tion learning. Finally, we demonstrate that our method can be applied to improve non-classification based model-free density ratio estimators well.
-Beemmnn theorem and show that it does not suffer
INTRODUCTION
Quantification of discrepancy between two distributions underpins the foundation of a large number of machine learning techniques. Of es- pecial prominence are distribution discrepancy measures known f- divergences (Csiszár, 1964), which are defined as expectations of the functions of the ratio of the two densities. As such, density ratio estima- tion is often a central task in deep generative modeling, mutual informa- tion and divergence estimation, representation learning, etc. (Sugiyama et al., 2012; Gutmann & Hyvärinen, 2010; Goodfellow et al., 2014 Nowozin et al., 2016; Srivastava et al., 2017; Belghazi et al., 2018; Oord et al., 2018; Srivastava et al., 2020). However, estimating density ra- tio by modeling each of the densities is challenging for most problems of interest as modeling high dimensional densities is harder problem than estimating their ratios Sugiyama et al. (2012). Therefore, in prac- tice, estimators are employed that only require samples from the pair of densities in order to estimate their ratio without explicitly modeling the individual densities. One of the most commonly used density ratio esti- mators (DRE) a binary classifier based DRE (BC-DRE) which can be formally shown to estimate the ground truth density ratio when trained correctly discriminate be-
40t Only
130 nOD
Figure 1: First Order Discrepancy vs Higher Order Discrepancy.

---

Under review as a conference paper at ICLR 2021
tween the samples from the two densities (e.g. Gutmann & Hyvärinen, 2010; Gutmann & Hirayama, 2011; Sugiyama et al., 2012; Menon & Ong, 2016).
BC-DREs have been tremendously successful in problems involving minimization of the discrep- ancy between the data and the model distributions even in high-dimensional settings (Nowozin et al., 2016; Radford et al., 2015). However, they do not fare well when applied to the task of estimating the discrepancy between two distributions that are far apart or easily separable otherwise. This issue has been characterized recently as the density-chssm problem by Rhodes et al. (2020). The discrep- ancy between two distributions can manifest itself in many different ways. Here, we roughly group these ways into two classes, First Order Discrepancy (FOD) and Higher Order Discrepancy (HOD). Two distributions can easily be separated when the first order statistics, such as their means, are far apart from each other. We refer to this type of discrepancy between p and & as FOD. Similarly, if the discrepancy between p and q (with no FOD) originates due to the difference in higher order statistics we refer to this as HOD. For example, consider Figure where in all the three cases, the 5, 1), in (b) it originates from HOD since M(u = 01 = 1), N(uz = 0, = 0.097), and (c) originates from FOD and HOD since M(111 - 1.5, 01 0.5). (P22 01 0.25).
[L[pllg] 50, but in (a) it originates from FOD since N(w -5, 01 = 1), N
In a recent work, Choi et al. (2021) attempt to address the density-chasn problem. They considered distributions p and q that are clearly separated and proposed that in such cases density ratios can be better estimated if the mixture of the two densities is mapped onto a unit-sphere using a flow-based map (Rezende & Mohamed, 2015). Using bijective map ensures that the density ratio between the original distributions is maintained post mapping. While this method (F-DRE) may resolve the density-chasm problem due to FOD, we found that struggles when the densities are separated due to HOD.
On the other hand, Rhodes et al. (2020) proposed to estimate the density ratio between p and q as Pt x x ... x PK This telescoping is achieved by training K BC-DREs where the distributions P PP are designed to be closely-packed such that all of the K ratios can be easily estimated using BC-DREs without suffering from the density-chasm problem. They call this estimator, tele- scoping density ratio estimator or TRE. Empirically, we found that while TRE performs relatively better than BC-DRE and F-DRE in cases with density-chasm problem due to HOD, cannot deal with the density-chasm problem originating due to FOD. Intuitively, this happens because TRE trains K DREs, each of which is trained on K different pairs of distributions. This leads to the distribution shift issue test time, especially when p and q have high FOD. Further, care needs to be taken when designing intermediate distributions pı.....PP PK to ensure that each of the ratios are well defined. In fact, the construction schemes introduced in the original work may render TRE theoretically ill-defined if both and do not have full support (Appendix A).
q
We will show that neither of these kate-of-the-an methods are applicable when the density ratio estimation problem involves both FOD and HOD. Unfortunately, most problems in practice tend to have both types of discrepancy. This leaves a clear technical gap in the task of density ratio estimation. Hence. in this work, we present SCALED DENSITY RATIO ESTIMATOR (sDRE), a novel method for estimating density ratio that:
1. Theoretically well-defined and does not suffer from distribution shift.
2. Resolves both FOD and HOD types of density-chasm issues.
3. Does not require training additional FLOW-based models or BC-DREs and therefore is computationally more efficient.
NOTATION AND BACKGROUND
Bregman distance between two d-dimensional vectors a and is defined as Ba = d(a) ((b) - V (bb b). Here a Rdd + R is convex function. Following the notation in Stummer & Vajda (2012) the scaled-Bregmant Divergence (sBD) between probability measures and Q scaled by an arbitrary measure A can be defined as,
()( = m P -
9 (
f m
P
dx.
m
m
2

---

Under review as a conference paper at ICLR 2021
Here, f X IR is convex function, f' is its right derivative. Further, all the measures P, and M are defined on the measurable space (2, and are absolutely continuous with respect to o-fiinit measure X, therefore admitting the following densities,
dP
dQ
d.M m dx
(2)
dx
dx
It is easy to show that for f(t) = t log(t), sBD reduces to KL divergence and is independent of the scaling measure M and X,
Bo cl (P.Q
m
log
P
P
log
! m
dd KL(PII)).
(3)
x
m
m
m
We define
p(x)
g(x)
(p/a(x) p(z) g(z))
Tp/m (z)
m(x) (afm(()
(4)
m(x)
as the density ratios between p & m, q & m and p & respectively.
Given samples { and {x:))==1 from distributions p and respectively and assuming p << q. the goal of DRE Fp/9 is to estimate Tp/g-
2.1 BINARY CLASSIFICATION TO DRE USING BD
Proposition 3 (Menon & Ong, 2016) formally establishes the link between binary classification and density ratio estimation using sBD. We re-state it here.
Proposition 1 (Proposition 3 (Menon & Ong, 2016) rephrased) Given a class drobabbryyyyylity esti- mator f X' + [0, 1] such that the density ratio estimator, fp/g(z) = T-a differentiable [0, 1] + R
_ Then for any convex
Europpannnall [D(((())))(X)) = Eung [D (Fp/o
(5)
where D is Bregman Divergence (BD), ft (0) + R ft(x)= (1 + ( Dy is sBD and Pmarginat is the marginal distribution of X.
2.2 DENSITY CHASM
Kato & Teshima (2021) note that most empirical estimators of density ratio, especially those im- plemented using deep neural network tend to over-fit the loss function some way or the other. This phenomenon was recently characterized by Rhodes et al. (2020) for the BC-DRE as the density chasm problem. They found that the BC-DRE tends substantially underestimate the ground-truth KL-divergence (as a function of the estimated ratio) between p and when these distributions are significantly separated from each other. We conjecture that this underestimation occurs because in the case of finite training data, when the two sets of samples are easily separable, the estimated ratios do not need to be as large as they theoretically had to be to achieve perfect classification performance, In such case, many different decision boundaries achieve equally good performance.
3 RELATED WORK
Our work is most closely related to and improves upon the shortcomings of the recently proposed TRE method. Therefore, in order to better understand these shortcomings, we begin by analyzing TRE under Proposition 3 of Menon & Ong (2016) deseribed in equation 5 above.
TRE uses a two step, divide-and-conquer strategy to resolve the density chasm problem. In the first step, they construct K waymark distributions PI...., PK by gradually transporting samples from p towards samples from q. Then, they train K BC-DRE models, one for each consecutive pairs of distributions. This allows for estimating the ratio Fp/9 as the product of the K BC-DREs.
3

---

Under review as a conference paper at ICLR 2021
(a) TRE 3 waymarks.
(b) SDRE with Cauchy(O,,
Figure 2: TRE VS SDRE on ((-1,0.1) and = ((L,0.2)
2. X...X Px. They introduced two schemes for creating waymark distributions that make P1 sure that consecutive pairs of distributions are packed close-enough to ensure that none of the K BC-DREs suffers from the density-chasm issue. Hence, TRE attempts to resolve the density-chasm issue by replacing the ratio hetween and q with a bridge of K intermediate density ratios that by design (of the waymark distribution) do suffer from density-chasm problem.
Application of Proposition 3 to TRE shows that each of the K individual binary classifiers, upon training lead to K DREs that minimize the sBD their corresponding true density ratios on the samples from their corresponding denominator densities. This however, does not guarantee that product of such individual DREs minimizes the sBD to rp/g on samples from & (or all K interme- diate distributions for K > 1). This issue is acknowledged by the authors who note that TRE's performance suffers significantly from the distribution shift issue especially when the distributions p and y have high FOD. An example of this can be seen in Figure 2a where we train TRE to estimate the density ratio between p = (--1, ((-1,0.1) 0.1) and g = (1-1,0.2 using intermediate distributions P1,P2. P2, P3 constructed using the linear-mixing construction described in Section 4.2. In the first row we evaluate the ratios p/p P/P1. P1. 12/22,P5 Pz/Ps, and ps/g on samples from their respective denominator densities and in the second
evaluate log P samples from all the K + 2 distributions. As q per equation 5, individual DREs are in fact able to estimate their corresponding density ratios accu- rately on the samples from their corresponding denominator densities. But, their combination does lead to an accurate estimation of p/q for any of the K + distributions except P2, making TRE not useful for estimating discrepancy measures such as KL divergence as the model is not accurate on samples from p.
Another interesting related work comes from Choi et al. (2021) (F-DRE), who propose to train BC- DRE in a learned feature space where the two distributions p and q are mapped onto unit sphere. To this end, they employ FI..W-based model on the input and train it to map the mixture of the samples from the two distributions to a standard Gaussian. It is easy to show that any bijective map will preserve the original density ratio Tp/g in the feature space the Jacobian correction term simply cancels out, However, such a method cannot bring the distributions closer along both FOD and HOD since doing that will change the original density ratio. Due to this, in practice, we found that F-DRE tends to work only in fairly limited settings.
4 DENSITY RATIO ESTIMATION BY SCALING DENSITIES
We tackle the density chasm problem using the notion of scaling densities from the scaled-Bres Bregman divergence (equation sBD). sBD introduces a scaling measure m in the estimation of the discrep- ancy between two distributions and q. This allows for changing the sampling distribution in the estimation of, for instance, KI. divergence as described in equation from p to m by re-writing the original integrand log 2 as ς
p = log P m
log 9
log 9
(6)
We leverage this insight and propose the SCALED DENSITY RATIO ESTIMATOR (sDRE) method for density ratio estimation that is theoretically well-defined, computationally more efficient, does not suffer from distribution-shie issue and thereby, mitigates the density-chasni problem originating from both FOD and HOD discrepancies. To this end, in sDRE, we first scale the densities p and q
4

---

Under review as a conference paper at ICLR 2021
by another probability measure and estimate log as log - log If the scaling measure m is constructed such that p << m, q << m and its samples overlap with the samples from both p and g. then as shown in the next section, a single multi-class logistic regression trained to classify samples from p. q and into 3 classes, can accurately recover & and 2 on samples from any of the 3 distributions. Later, we show that the SDRE approach can also be applied to model-free DREs (Section 5.3).
yn m
4.1 MULTI-CLASS CLASSIFICATION TO MULTI-DISTRIBUTION DRE
Nock al. (2016) build upon Proposition 3 of Menon & Ong (2016) (equation 5) using their scaled- Bregmann theorem and state the following lemma.
Lemma Given a class cross-robabbiltyy estimator ó X + [0,1|--1, let the density ratio estimator Then for any convex differentiable f [0, +
F(x) (a)
M(x))
E Precgett [D;(((X]I[(X)) (1 TC) Bnwwe To (+(((())I(((
Here, - g(>) g(a)) and g(x) - -- + न x. me is the prior probability of class C, ne (2) is the class probability of x under the class C and PC is the distribution of X indexed by the class C. f a C - dimensional vector of density ratios between the C' 1 densities (P1.... PC-1) and the reference density Pc. Similarly, ff(r) is a C - 1-dimensional vector of class probabilities. Please note that the choice of the reference class is arbitrary. In simple terms, Lemma shows that a multi-class logistic regression (among other multi-class classifiers) leads to multi-class density ratio estimator that minimizes the sBD to the corresponding true density ratio on samples from the reference distribution.
We demonstrate the implication of Lemma 1 on SDRE in Figure 2b. Here, we train SDRE on the same exact setup as TRE in Figure 2a where p = (M11, 0.1) and q W1+1, 0.2). To this end, we propose to use Cauchy(O,, 1) as the scaling measure. This is because the samples from the heavy tailed distributions easily overlap with the highly concentrated samples of and q as shown in Figure 2b. Then we train a single 3-way logistic regression to classify samples from p. 9 and m. As clearly shown, sDRE is able to perfectly estimate logp/q samples from all the three distributions without suffering from any distribution shift issues. This empirically confirms that, unlike TRE, the sDRE estimator converges to the correct density ratio on the entire domain of m long as m is such that both p and q are absolutely continuous to m.
sDRE can also use more than one scaling measures {m+)k=1 Not only does Lemma I hold true for any number (K) of distributions, but also Ma & Collins (2018) show that, when the number of samples is fixed, increasing K leads to a more consistent estimator. Please note, increasing the number of scaling measure does not change the estimation of Tp/q- In multi-class logistic regression model trained on samples from p. q and mg, the ratio between p and q is estimated as the function of their logits h1, h2, h2 as follows. From Lemma 1, Tp/me = exp(h)) exp(hx) i.e. log Tp/m = log exp(hr) = h1 h4- Similarly, log q//m log exp(h2) = h22 - h4. This gives us the final ratio exp(hx)
exp(kx)
as, rogr = log pp/m= llgg gg/ms hn h2, that can be calculated similarly with respect any of the K scaling measure.
To demonstrate that the improved performance of sDRE over TRE in the example from Figure 2 does not only stem from the novel construction scheme for m, we also train sDRE using TRE's waymark distributions as scaling measures. This is illustrated in Figure 5e in Appendix B. Unlike TRE that does not estimate log e accurately on samples from p, q. P1 and Ps. SDRE is able to estimate log & on all 5 distributions significantly better even with sub-optimal scaling measures. q
4.2 CONSTRUCTING M
In this work, we use three ways of constructing the scaling measure M. Here we describe them briefly. Please see Appendix A for details. Overlapping Distribution: We introduce novel scaling measure construction scheme under which, M is defined as any distribution whose sam- ples overlap with both p and q and p << m. q << m. For example, it can be heavy-tailed
5

---

Under review as a conference paper at ICLR 2021
distributions, such as Cauchy and Student-t, simply normal and uniform distributions or their mixtures. Linear Mixing: In this construction scheme, distribution M is defined as the empir- ical distribution of the samples constructed by linearly combining samples X, ( and X2 = { / from distributions p and q respectively. That is, m the empirical distribution over the set Xm.. = {I Fm = (1 o) + ozr Tp E xx, E 1 This construction is related to the linear combination waymark of Rhodes et al. (2020). We clarify the difference in Appendix A. Dimension-wie wisee Mixing: In this construction scheme, that is borrowed from TRE as it is, M defined as the empirical distribution of the samples generated by combining different subsets of dimensions from samples from p and q.
4.3 SDRE VS TRE
SDRE differs from TRE in two major ways. First, TRE instructs to create K, closely-packed way- mark distributions P......PP. PK between and q such that none of the K+1 BC-DREs that it proposes to train, suffers from the density chasm problem. This approach is not only computationally more demanding as it requires training K + models (with some parameter sharing), but also leads to the aforementioned distribution shift issue. In contrast, sDRE instructs to introduce scaling densities {m²) such that their samples overlap with the samples from both p and g. It then estimates 7 as log P. log 1 by training a single multi-class logistic regression model across all K + 2 distributions. This approach is not only computationally more efficient, but is also theoretically guaranteed not suffer from distribution shift as shown in the previous section.
K
log _
my
Second, for K = 1, TRE and SDRE are similar as they both can be seen as using telescoping of ratios to estimate rp pp/q: However, they are not equivalent. TRE proposes the following telescoping: p/q = log p/ + log m/q and sDRE proposes to telescope in the following manner: log p/q logr/m- log g/m. As such, TRE and SDRE use two different sets dM dQ in the second term)
eQ
aM
of Radon-Nikodym Derivatives (RND) to re-define log p/g. The consequence of this is that TRE is only defined when << m << 9, whereas sDRE is defined for any m such that, p << and q << m. This implies that only under the following two conditions TRE and SDRE telescopings equivalent: (1) K = 1 and (2) m is such that p << m << q and q << m. This condition holds when p, 9 and m have full support. However this condition easily breaks if, for example, p and g are mixtures of finite support distributions except for the trivial case when support of exactly equal to the support of We demonstrate this in Figure 6 in Appendix B with a concrete example.
Another important and practical implication of this distinction is that, since m only appears in the denominator in SDRE, it can also be used to improve model-free approaches to density ratio es- timation, such as the fixed-design estimators based on the maximum mean discrepancy measure (Sugiyama et al., 2012). We show this in Section 5.3. is also worth noting that unlike TRE, sDRE realized as a single multi-class logistic regression that is trained using a softmax cross-entropy loss. This allows us to use the results from Ma & Collins (2018) who formally studied the distinc- tion in the convergence of softmax cross-entropy and binary cross cross-cntropp entropy based NCE methods (c.g. sDRE vs TRE/BC-DRE), They showed that canking-basee noise contrastive estimation (Gutmann & Hyvärinen, 2010) (multi-class objective) is consistent under a much weaker assumption compared to the binary version. Specifically, in contrast the binary version, the ranking based approach does not require the capacity to model the normalization constant of the distributions.
5 EXPERIMENTS
In this section, we provide a thorough empirical analysis on the accuracy and robustness of SDRE and demonstrate that SDRE outperforms all other baselines - BC-DRE, TRE (Rhodes et al., 2020), and F-DRE (Choi et al., 20.1). We start with a toy 1D Gaussian dataset and fully analyze the behav- ior of SDRE. Then, extend to nigh-dimensiooal complex tasks ranging from mutual information (MI) estimation to representation learning and show significant improvements on all accounts. Fi- nally, we also show how sDRE can be applied improve model-free DREs.

---

Under review as a conference paper at ICLR 2021
P
q
GT-KL
BC-DRE
TRE
F-DRE
SDRE
W(,, x 1)
1)
13.32
1.82
2.92c-5
4.57
13.32
W(-5,1)
50
69.56
34.51
15.96
53,97
W(-1,0008)
(2200.15)
200.27
19.2
142.64
15.84
201.32
22-200.08)
0.250.15
355.82
17.95
227.98
14.49
358.81
Table 1: 1D density ratio estimation task. GT-KI. stands for ground-truth KT. Divergence.
5.1 ID EXPERIMENT
In the following 1D experiments, we use two ID Gaussian distributions and consider the four se- tups described in Table 1. The first and second configurations (row-wise) are designed such that the distributions are only separated by HOD and FOD, respectively. The other two configurations represent the cases in which the distributions are separated by both FOD and HOD. In SDRE, we use the quadratic form p(z) = wr² + to model each of the densities and use Cauchy centered at 0 and linear mixing to construct m. We use KL-divergence estimation (expectation of the log ratios on samples from p) as the evaluation metric. We provide the exact setup for sDRE in Table 3 in Appendix C.
In all of the configurations, while the baseline methods, including TRE and F-DRE, fail to correctly estimate the KL divergences by a significant margin, SDRE reliably estimates them as shown in Table I. Figure 7 in Appendix C illustrates that sDRE accurately estimates the underlying density ratios on the entire support of m as well. These results demonstrate that is better to use a single scaling measure whose samples overlaps with those from both and q, instead of using chain of upto
28 closely-packed waymark distributions (see Table in Appendix for the exact TRE configurations). To provide further clarity into SDRE's density ratio estimation behavior, we analyze the reliability of its log ratio estimates using the Bayesian setup in Appendix D. The high accuracy of SDRE's KL divergence estimates can be attributed to the fact that the SDRE estimator most certain and precise around the support of p. and the KL divergence estimates are computed with samples from p. This is illustrated in Figure 8 of Appendix D.
5.2 HIGH DIMENSIONAL EXPERIMENT
Following from TRE, we use the MI estimation problem from Belghazi et al. (2018); Poole et al. (2019) to evaluate sDRE on the more challenging bigher-dimensionn setup. In this problem, we estimate the mutual information, between a standard normal distribution and a Gaussian random variable E RR with block-diagonal covariance matrix, where each block is 2 x 2 with 1 on the diagonal and p on the off-diagonal. p is computed given the number of dimensions and target mutual information with T = log(-p²s p²)) Because this method of distribution construction only creates separation in HOD, we further adapt it by moving the means the two distributions and create harder problems with both FOD and HOD. In sDRE,
model the densities of p. g, and the scaling measures m& with a quadratic form p(r) = Wr + go gx(x) + b, where Is is a linear layer. We use linear-mixing to construct my. where K = 3 or K 5. We provide the exact configurations for SDRE in Table 4 in Appendix E. We describe the setups and report the results averaged across 3 runs with different random seeds in Table 2. In all of the settings, SDRE outperforms all other baselines, confirming the generalization of the results from the toy ID experiments. These results corroborate our conjecture that TRE only solves problems with HOD. It worth noting that, using only upto 5 scaling measures that are constructed using the linear mixing scheme, SDRE beats TRE substantially on cases with both FOD and HOD even though, TRE uses upto waymarks that are also constructed using their linear mixing approach. This demonstrates, as per Lemma 1, our proposal of using the multi-class logistic regression does, in fact, prevent distribution shifts issues present in TRE when both FOD and HOD are present. F-DRE, in contrast, is only accurate in problems with FOD, but tends to underestimate the log ratios even more than BC-DRE on settings with MI greater than 20. Further details about this experiment such how we chose and K can be found in Appendix E, where we also provide plots of estimated log ratio vs the ground truth log ratio and training curves.
7

---

Under review as a conference paper at ICLR 2021
Dim
M2,/22
GT-MI
Single BC-DRE
TRE
F-DRE
SDRE
0, 0
20
10.90 + 0.04
14.52 + 2.07
14.87 = 0.33
18.81 0.15
40
-1,
100
29.03 + 0.09
33.95 + 0.14
13.86 + 0.26
119.96 0.94
0, 0
40
21.47 + 2.62
34.09 + 0.21
12.89 + 0.87
38.71 + 0.73
160
-0.5, 0.6
136
24.88 + 8.93
69.27 + 0.24
13.74 + 0.13
133.64 + 3.70
0, 0
80
23.47 + 9.64
72.85 + 3.93
9.17 + 0.60
87.76 0.77
320
-0.5, 0.5
240
24.86 + 4.07
100.18 + 0.29
10.53 0.03
217.14 + 6.02
Table 2: High-dimensional mutual information estimation task.
17.5
:DRE KMH ratics mixing
KMM ratics
15.0
Cauchy
35
-
SDRE W: KMN ratics & Cruchy in
ZDRE KMH ratics mixture
<DRE w/ KMM ratios & mixture
ratios
25
True ratics
ratos
10.9
7.5
10
2.5
0.0
0
1.0
2.0
1.00
1.25
1.50
1.75
2.00
2.25
2.50
2.75
3.00
(a) Varying d (bandwidth is 0.1)
(b) Varying (bandwidth is 0.02)
Figure 3: Improving KL estimation between two standard Normals via KMM-DRE using SDRE
Robustness and Generalization beyond Gaussian Distribution: We rigorously test the robust- ness of sDRE by evaluating it the following setups. (1) p and q ID truncated normal dis- tributions to evaluate it on another setup with distributions with finite support. (2) Randomized mean parameters to ensure that the sDRE model does not benefit from the symmetry of the dis- tributions around 0 in Table 2. (3) p and as Student-T distributions with randomized means to test the performance of sDRE under model mismatch. In all these settings, SDRE model is able reliably estimate the true KL-divergence demonstrating that it is highly robust. Please see Table 5 in Appendix G for results and additional details.
5.3 IMPROVING K.M-BAASDD DRE WITH SDRE
We demonstrate the applicability of sDRE in the fixed design setup by applying it to the Kernel Mean Matching-DRE method (KMM--RE; Huang et al., 2006; Sugiyama et al., 2012) for KL estimation. In fixed design setup, the DRE method directly outputs the densities for a set of samples without building model. We consider two KL estimation experiments with increasing difficulty: (a) We estimate KL between (-d, 1) and N(+d, 1) by varying d from to 3 and (b) We estimate KI. between NO, s²) and (O,, 1/5²) by varying from 1 to 3. For each experiment, following KI. estimates: (1) KI. MC using true ratios, (2) KI. by MC using KMM-DRE, estimates, and (3) KL by MC using estimates from KMM-DRE with SDRE for three types of construction: linear mixing, mixture, and Cauchy. We conduct the same experiments for different bandwidth ar, the parameter of the Gaussian kernel used by the KMM estimator. As it can be seen from Figure 14, SDRE consistently improves KMM-DRE in both experiments with different difficulty by improving the baseline KMM-DRE KL divergence estimates by up 11 nats in some cases. We defer a detailed discussion of the results to Appendix H.
report the
5.4 REPRESENTATION LEARNING IN SPATIALMULTIONIIGLLO
Following the setup from Ozair et al. (2019): Rhodes et al. (2020), apply sDRE to mutual in- formation estimation and representation learning in the Spatial/MuliOnnnglloo problem. The goal to estimate the mutual information between and v, where x n grid of Omniglot char-
8

---

Under review as a conference paper at ICLR 2021
Ground Iruhh
300
Mearn label accurncyy (teat)
Mutuol information
ADRZ
-20
Single Ratio
1DR2
Number characters
Number characters
(a) Mutual information estimation
(b) Representation learning accuracy
Figure 4: SpatialMulttOmmnigooo representation learning results
acters from different Omniglot alphabets and is X grid containing the next characters of the corresponding characters in u. After learning, we evaluate the representations from the encoder with a standard linear evaluation protocol (Oord et al., 2018). For the model, in TRE, we use a separable architecture commonly used in MI-based representation learning literature and model the densities with logp(u, v) g(u) 3 (w)), where I and ʃ are 14-layer convolutional ResNets (He et al., 2015). We construct the scaling distributions via dimension-we wise mixing. In this exper- iment, we only compare to the single ratio baseline and TRE because (Rhodes et al., 2020, Figure 4) already demonstrated that it significantly outperforms both Contrastive Predictive Coding (CPC) (Oord et al., 2018) and Wasserstein Predictive Coding (WPC) (Ozair et al., 2019) models of repre- sentation learning on the same exact task.
As can be seen in Figure 4a, SDRE outperforms both TRE and the single ratio baseline, exactly matching the ground truth MI. This improvement in MI estimation is reflected in the representa- tions. Figure 4b illustrates that sDRE's encoder learns representations that achieve ~ 100% Om- niglot character classification for both d n² 4, 9. On the other hand, the performances of the single ratio estimator and TRE (using the same exact dimension-win -wise mixing to construct waymark distributions) both degrade noticeably as the complexity of the task increases. As such, TRE only reach up to 91 % and 85% respectively for d = 4 and d = 9. All models were trained with the same encoder architecture to ensure fair comparison. Please refer to Appendix F for further details and two additional experiments where we study the effects of varying K and the encoder design.
CONCLUSION
In this work, we presented the alternative method of SDRE for density ratio estimation. We demon- strated that it has better theoretical grounding and improves the state-of-the-art in more challenging. high-dimensional density ratio estimation problems that have both FOD and HOD type of discrep- ancies. However, the presented framework has its limitations. For example, while sDRE estimates the ratio fairly well, (I) we do not provide any bounds on the estimation and (2) m is not learned but provided to the model. We hope to address these issues future work.
Moreover, our representation learning results suggest that improving MI estimation correlates with better representation learning. Therefore, another interesting direction for future work could be to explore if SDRE can improve MI-based representation learning methods, such as contrastive learning (e.g. Oord et al., 2018; Chen al., 2020).
ETHICAL IMPACT
While density ratio estimation does not directly have major ethical concerns, it can, however, be used towards improving other deep learning methods of representation learning, generative models, etc. These work tend to have significant ethical considerations. For example, improved generative models using sDRE could potentially be used to advance the deep-fake methods that may be used with malicious intent. However, we hope that other applications of our work, such as those of MI estimation in science domains, outweigh the drawback of its potential illicit usages.

---

Under review as a conference paper at ICLR 2021
8 REPRODUCIBILITY STATEMENT
We have taken efforts to clearly describe all the experimental setups in separate appendices, one for each of the experiments. For the baseline methods, have used models provided by the original authors. Where possible we have used existing evaluation tasks and metrics to report the results. Finally, in order to ensure reproducibility of our own results, we are releasing all the code at the time of submission. While most of our code is setup as Jupyter notebooks for easy accessibility, we will work to add documentation improve the readability further.
REFERENCES
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar. Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation. arXiv preprint 1xXiv:8801,44000 2018.
Ting Chen, Simon Korbllihh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, 2020.
Kristy Choi. Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. arXiv preprint arXiv:2107.02212, 2021.
Imre Csiszár. An cntemattiontthhoree inequality and its application to the evidence of the ergodic- ity of markoff's chains. Magyer Tud. Akad. Mat. Kutato Int. Koezl, 8:85-108, 1964.
lan J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Neural Information Processing Systems, 2014.
M. U. Gutmann and J. Hirayama. Bregman divergence as general framework estimate unnormal- ized statistical models. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2011.
Michael Gutmann and Aapo Hyvärinen. conta-coe estimation: new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings. 2010.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition, 2015.
Jiayuan Huang, Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex Smola. Correct- ing sample selection bias by unlabeled data. Advances in Neural Information Processing Systems, 19:601-608, 2006.
Masahiro Kato and Takeshi Teshima. Non-negative bregman divergence minimization for deep direct density ratio estimation. In International Conference on Machine Learning, pp. 5320- 5333. PMLR, 2021.
Zhuang Ma and Michael Collins. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. arXiv preprint arXiv:1809.01812, 2018.
Aditya Menon and Cheng Soon Ong. Linking losses for density ratio and class blasa-probayilityy estima-
tion. In International Conference on Machine Learning, pp. 304-313. PMLR, 2016.
Richard Nock, Aditya Menon, and Cheng Soon Ong. A scaled bregman theorem with applications. Advances in Neural Information Processing Systems, 29: 29:19-27, 2016.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 271-279, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint /rXiv: 037488 2018.
10

---

Under review as a conference paper at ICLR 2021
Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron van den Oord, Sergey Levine, and Pierre Ser- manet. Wasserstein dependency measure for representation learning, 2019.
Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, and George Tucker. On varia- tional bounds of mutual information, 2019.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks, arXiv preprint arXiv:1511.064: 2015.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. Interna-
fional conference machine learning, pp. 1530-1538. PMLR, 2015,
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. arXiv
preprint arXiv:2006. 12204, 2020.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, and Charles Sutton. Veegan: Reducing mode collapse in gans using implicit variational learning. In Proceedings of the 31st International Conference Neural Information Processing Systems, pp- 3310-3320, 2017.
Akash Srivastava, Kai Xu, Michael Gutmann, and Charles Sutton. Generative ratio matching net- works. In Eighth International Conference on Learning Represeniations, pp. 1-18, 2020.
Wolfgang Stummer and Igor Vajda. On Bregman distances and divergences of probability measures. IEEE Trans. Information Theory, 5873):1277-11888, 2012.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
A
APPENDIX
CONSTRUCTING M
We now elaborate on the three types of scaling measures that used in this work.
Overlapping Distribution: sDRE estimator, log 2 = log log m is defined when p << m and q << m. Therefore, m needs to such that its support contains the supports of and q. Any distribution with full support such as the normal distribution trivially satisfies this requirement. However, satisfying this requirement does not guarantee empirical overlap of the distributions p. q with m in finite sample setting. In order ensure overlap of samples between the two pairs of distributions we recommend the following:
g
Heavy-tailed Distributions: Distributions such as, Cauchy and Student-t are better choice for Af compared the normal distribution. This is because their heavier tails allow for easily connecting p and with higher sample overlap when they are far apart (especially in the case of FOD).
Mixtures: Another way to connect p and q using m such that they have their samples overlap, is to use the mixture distribution. Here, we first convolve p and q with a standard normal and then take equal mixtures of the two.
Truncated Normal: If p and q have finite support, one can also use truncated normal distribution or a uniform distribution that at least spans over the entire support of This is assuming that p << q.
Linear Mixing: In this construction scheme, distribution M is defined the empirical distribu- tion of the samples constructed by linearly combining samples X, = 1 and X, { ) from distributions
N
and q respectively. That is, is the empirical distribution over the set X,, = {x), |or = ar," + (1 a) Tp E X x E X, -- This construction is related to the linear combination waymark of Rhodes et al. (2020). In TRE the waymark distribution defined as the empirical distribution of the set X., ( 1o v1 a² axr, Tp Xp. X iN

---

Under review as a conference paper at ICLR 2021
This weighting scheme skews the samples from the waymark distribution towards Therefore, care needs to be taken when p and are finite support distributions so that the samples from the waymark distributions do not fall out of the support of q.
Using either of the weighting schemes, one can construct K different scaling measures. SDRE can either these K scaling measures separately using K+2-way classifier define single mixture distribution using them as component distributions and train a 3-way classifier. We refer to this construction Mixture of Linear Mixing.
Dimension-wise Mixing: In this construction scheme, that is borrowed from TRE as it is, M defined as the empirical distribution of the samples generated by combining different subsets of dimensions from samples from and We describe the exact construction scheme from TRE below for completeness:
Given d-length vector and that d is divisible by I, we can write down (x[1), [[]. where each z|i] has the length d/l. Then, a sample from the kth scaling measure is given by: zj. = (ari ".), +]; -.,,,,,,,, [4]), for = I,..,!), where oi ~ p and zi ~ q are randomly paired.
B APPENDIX
TRE VS SDRE
In Figure 2b in Section 3, showed that in comparison to TRE that uses 3 intermediate waymark distributions. sDRE is able to significantly better estimate the ground truth density ratio using just single Cauchy scaling measure that we introduced. To demonstrate that this improved performance does not only stem from the novel m. but also from the use of single multi-class logisitic regression (afforded by the exact form of sDRE telescoping), we train SDRE using TRE's waymark distribu- tions as scaling measures. This is illustrated in Figure 5c. Unlike TRE that does not estimate log
accurately samples from p, Q. P1 and P3: SDRE is able to estimate log 2 all 5 distributions significantly better even with sub-optimal scaling measures. The impact of using the sub-optimal scaling measure in sDRE on the quality of density ratio estimation can be teased out by comparing the results in Figures 5b where sDRE trained using Cauchy for m.
Finite Support Distribution Example for K = 1: For the case, K = 1, in Section 4.3 we general. We now demonstrate this with a specific example in Figure 6. Here we set p 0.5 x (W(11, 0.1,
theoretically showed that TRE and sDRE are not equivalent
-1.1, high
-0.9) x (1 (1,0..1.11 0.1. low = 0.9. high = 1.1) and g = 0.5 x TGx(N 0.2, low = = -1.2, high = 0.8) + 0.5 0.5 x T1 0.2, low 0.8, 0.8,high distribution TRE and the scaling measure for SDRE to m TN(0,, 1, low = -122, high 1.2) using the proposed overlapping distribution construction. As such, p << q << m and therefore. TRE is undefined for the second term as is not defined for samples from m that are outside the support of g. On the other hand, SDRE is well defined as 1 is finite and defined over the entire support of m. It can be clearly seen in Figure 6b that TRE estimator for m blows up to very high values on samples from m where does not have any support. This, however, is not the case with SDRE, which accurately estimates 2. as 0 on samples outside the support of q as shown in Figure 6c.
1), shown in Figure 6a. We set both, the waymark
m
Despite being undefined, when used with our proposed m. TRE may still estimate log TogTp/4 accu- rately on samples from p. We conjecture that this is because, the numerical estimation of the d.M /QQ finite over the support of p. As such, care needs to be taken when equating TRE to sDRE to ensure that the relevant RNDs well defined along with of the accuracy of the numerical estimation.
12

---

Under review as a conference paper at ICLR 2021
(a) TRE o = =V(-1,0.1) and (.2) with TRE's skewed linear waymarks
100
2-
00P2
-100
Yre alrd Bisary log pim]
Pue log mile2 Binary log mLlm2
Pue log máted
Foe log málg
-200
Binrry log m2/r3
Braary log m33g
0
leg
pig
-20
2
-2
p1
(b) sDRE on p - and (1, using m Cauchy(O, 1) as scaling measure.
3.
200
2.5
15
-100
10
0.
Tue log pla sDRE log
Tue log piq SDRE log
Tue log ph DRE log
-200
2
2
(c) SDRE using TRE waymarks as scaling measures.
m2
200
-100
-200
Tuelog pit
Toe log pia Mutt-caas log pi
Yae log pia Math log pit
Tue log pio Muts class pio
Tue márm2 clas-asss log
Math cass log pit
Figure 5: TRE vs sDRE on = and -V using intermediate distributions P1, P2.P2,77 constructed using the linear-combination construction.
(a) Setup
(b) TRE: dM/dQ
(c) SDRE: dQ/dM
80
Figure 6: sDRE vs TRE telescoping mixtures of distributions with finite support
13

---

Under review as a conference paper at ICLR 2021
Figure Label
p
TRE Pa
SDRE m
Linear Mixing with
Mixture Linear Mixing with
a
1e
1)
[6. 10e-05, 0.0078, 0.13]
a [6. 10e-05, 0.0078, 0.13]
Linear Mixing with [0.053, 0.11,
0.16, 0.21, 0.26, 0.31,
N(-100.08) (22.0.15)
C(O,
0.37, 0.42, 0.47, 0.53, 0.58, 0.63, 0.68, 0.74, 0.79, 0.84, 0.89, 0.95) Linear Mixing with = [0.03, 0.07,
0.1. 0.14, 0.17, 0.21,
0.24, 0.28, 0.31, 0.34,
N(-200008) (.2,0.15)
0.38, 0.41, 0.45, 0.48,
C(O,
0.52, 0.55, 0.59, 0.62,
0.66, 0.69, 0.72, 0.76.
0.79, 0.83, 0.86, 0.9, 0.93, 0.97] Linear Mixing with [0.11, 0.22, 0.33, 0.44, 0.55, 0.66,
d
5(-5,
15,
C(0,2)
0.77, 0.88]
Table 3: Experiment configurations for Figure 7
C APPENDIX
ID DENSITY RATIO ESTIMATION TASK
In Section 5.1 we used KL-diiergence as the evaluation metric to assess the accuracy of the density ratio estimation of each of the models, BC-DRE, TRE, F-DRE and SDRE. Since KL-divergence only evaluate the density ratio over samples from p, here we provide the plots the log density ratio for all the models over larger interval to better capture their behavior.
The configurations of the experiments are described in Table Column (Figure Labels) corre- sponds to the respective columns in Figure 7. The rows of Figure represent the four models in this order: BC-DRE, TRE, F-DRE, and SDRE. Except for the first configuration (Figure Label a), which was trained with 1,000 samples in total, all other configurations trained with 100,000 samples in total for all four models.
As evident from the figure, SDRE is the only model that not only accurately estimates the KL divergences, but also perfectly estimates the log density ratios on the entire support of m.
D APPENDIX
UNCERTAINTY QUANTIFICATION OF SDRE LOG-RATIO ESTIMATES WITH HAMILTONIAN MONTE CARLO
In the 1D experiments, sDRE consistently led to highly accurate KL. divergence estimates even in challenging settings where rate-of-the-art methods fail. To understand why SDRE gives such ac- curate KL estimates, we conduct an analysis on the reliability its log-ratio estimates by analyzing the distribution of the estimates in a Bayesian setup, and study how it impacts the KL divergence estimation. For this analysis, we use a classifier with the standard normal distribution as the prior on its parameters. The distribution of the log- fog-ratio estimates is simply the distribution of the estimates from the classifiers with different posterior parameters, which are sampled. We consider the setup where p (-100, 0.1) and q (1.00, 0.2) and draw samples from the posterior using an Hamil-
14

---

Under review as a conference paper at ICLR 2021
(a)
(b)
(c)
(d)
Mia 100
1330
110
1000
3209
1000
500
540
540
500
Log Tatto
-200
-200
-500
-10
-10
Figure 7: ID density ratio estimation task. The configurations of the ID Gaussians are the following in
wise order: (a) p = (0, 6).9 = (O. 1), KL= 13.32 (b) p = 0.08), 9 (22,0.155,
KI. = 200.27 (c) = ( - = (22 0.15), KI. 355.82 (d) = (- 1). = (5, 1), KL. 50.0. The models are the following row-wise: single ratio BC-DRE, TRE, F-DRE, and sDRE.
tonian Monte Carlo (HMC) sampler. We then compute set of samples of the log-ratio estimates from sDRE and estimate the mean and quantiles using these samples. Figure shows these results. We find that sDRE is accurate and manifests lowest uncertainty around the mean (-1.0) of p. The uncertainty increases as we move away from the modes of distributions p and q. Since KI. diver- gence is the expectation of the log-ratio on samples from and the high density region of exactly matches the high confidence region of SDRE, it is able to consistently estimate the KL divergence accurately even when p and y are far apart.
(a) log
2 4
(b) log P
(c) log 2 m
Tos Lo
Logath
Figure& 8: Uncertainty quantification for sDRE estimator. Here, = 1.110.0,.11. = NLLL 0.2) and
m = C(O, 1). We plot the 3x standard deviation around the mean in light blue.
15

---

Under review as a conference paper at ICLR 2021
Dim
MI
m using LM
20
[0.25,0.5,.75]
40
100
[0.35,0.5,.85]
40
[0.25,0.5,.75]
160
136
[0.1150.33.0055755995
[0.
80
[0.25,0.5..75]
320
240
[0.15.0.33.0.5.75.9955
Table 4: Configuration of sDRE for the high dimensional experiments. LMM stands for I Linear Mixing
GT vi CoB
GT KLD: 79.711716
Iist Loss Trin Loss
Accuracy
140
120
Log Publo CuB
Tbst LoGs
Accurryy
60
100
80
20
80
100
30
10 Epoch
15
m1
nd
Log Ratio
Distributions
Figure 9: Diagnostic plot for high dimensional experiment.
E APPENDIX
HIGH DIMENSIONAL EXPERIMENT
In Section 5.2, we showed that SDRE performs better than all baseline models when p and q are high dimensional Gaussian distributions. Prior work of Rhodes et al. (2020) has considered high dimensional cases with HOD only, whereas here we additionally consider cases with FOD and HOD to provide a more complete picture. Our results show that SDRE outperforms all other methods on estimation as the function of the estimated density ratio. worth noting that SDRE uses only upto scaling measures that constructed using the linear mixing scheme and beats TRE substantially on cases with both FOD and HOD, although TRE uses upto 15 waymarks also constructed using linear mixing approach. This demonstrates, using the multi-class logistic regression does, in fact, prevent distribution shifts issues TRE when both FOD and HOD present and help estimate the density ratio accurately.
the task
per Lemma our proposal of
We now describe the sDRE configuration and other setup related details.
Scaling Measures: For all the high dimensional experiments throughout this work, we construct m using the linear mixing scheme as described in Appendix A. Table provides the number K of scaling measures along with the exact mixing weights for each of the 6 settings.
As general principle, we chose these three sets of mixing weights so that their cumulative samples overlap with the samples of p and q similar how the heavy tailed distribution worked in the ID case. Please note, unlike the 1D experiment, the HOD in these experiments is significantly higher. Therefore, using a heavy tailed distribution does not work. Linear mixing p and q on the other hand, mixes first and higher order statistics and therefore, populates samples that overlap with both p and q. In some cases, we found that a mixture of linear mixing with K = 1 can also be used to estimate the density ratio. However, this requires using neural network- based classifier and requires much more tuning of the hyperparameters.
For choosing K, use grid search based approach. We monitor the classification accuracy across all the K + 2 distribution. If this accuracy is very high (> 95% for all classes), this implics that the classification task is easy and therefore the DRE may suffer from the density chasm issue. On
16

---

Under review as a conference paper at ICLR 2021
360
GT LR vs CoB
Accurcy
340
120
00
300
Seat Lens
60
12g
40
00
20
Bsst Loss
60
GT KLD: 110 42271
Trin Lows
80
100
120
140
5
10
10 Epoch
15
P
Id m2 nd m4 m5
Log Ratio
Distributions
Figure 10: Diagnostic plot for a high dimensional experiment with randomized means.
100
Mean label accuracy (test)
Mutual Mutual information
34
13
92
12
-88
1.00
1.25
1.5
2.00
2.25
2.50
2.75
3.00
Figure 11: Mutual information estimation and representation learning with varying K.
the other hand, if the classification accuracy is too low (< 50% for all classes), then again, the DRE does not estimate well. We found that targeting an accuracy curve as shown in Figure 9 (last panel) leads to optimal density ratio estimation. This curve plots the test accuracy across all the classes and, empirically when it stays between the low and the high bounds of (50%,95%) the DRE estimates the ratios fairly well. The first panel shows that SDRE estimates the ground truth ratio accurately across samples from all the K + 2 distributions, the second panel shows that KL estimates of sDRE close the ground truth KL and the third panel shows that both test and training losses have converged. Figure 10 shows another example for the case of randomized means. While sDRE also manages to get the ground truth KL correctly and most of the ratio estimates are also accurate, does, however, slightly overestimate the log ratio for some of the samples from p.
Additional Baseline Comparisons We also compare to Kato & Teshima (2021) in high- dimensional setting (row 2 of 2 with dimensionality of 40 and μl = -1, U2 1. In this setting in which the ground truth MI is 100, while sDRE meaningfully estimates the MI as 119.96, the best model from Kato Teshima (2021) only estimates it to 1.60.
F APPENDIX
SPATIALMUULTIONIIII.. EXPERIMENT
SpatialMultiOmnigloo is a dataset of paired images u and v, where u is a n X n grid of Omniglot characters from different Omniglot alphabets and is n X n grid containing the next characters of the corresponding characters in u. In this setup, we treat each grid of n X n as collection of n² categorical random variables, not the individual pixels. The mutual information I(u, v) can be
computed as: I(u, u) Σx 1 log 4, where I; is the alphabet size for the tt character in u. This problem allows us to easily control the complexity of the task since increasing n increases the mutual information.
17

---

Under review as a conference paper at ICLR 2021
300
Mearn label accuracyy (teat)
800
ADRZ
15.0
ZCRE with same enc
Number characters
Number characters
(a) Mutual information estimation
(b) Representation learning accuracy
Figure 12: SpatialMultiOnnniglot representation learning results with same encoder for fandg.
For the model, as in TRE, we use a separable architecture commonly used in MI-based represen- tation learning literature and model the densities with logp(u, v) g(u)"" f()), where g and are 14-layer convolutional ResNets (He et al., 2015). We construct the scaling distributions via dimension-win mixing exactly the way that TRE does.
To evaluate the representations after learning, we adopt standard linear evaluation protocol to train a linear classifier on the output the frozen encoder g(u) to predict the alphahetic index of each character in the grid u.
ADDITIONAL EXPERIMENTS
In addition to the experiments in the main text, we two experiments with SpatialMultiOm- niglot to test the effect of the size of K and the effect using the same encoder for g and f (i.e, modeling the densities with the form log p(u, v) g(u) ((u)"Wg(()) Wg))).
Varying K: We test the effect of changing K in the d experiment. For K = L, we aggregate all the fimension-wise mixed samples into class, whereas for K = 3, we separate them into their respective classes (corresponding to the number of dimensions mixed). We illustrate this effect in 11. In line with the finding of Ma & Collins (2018), increasing the number of K not only helps the model reach the ground truth MI, but also the quality of representations worsens from 100% to 86.7% test classification accuracy.
Single Encoder Design: Furthermore, we test the contribution of using two different encoders f and g instead of one. As seen in Figure 12, in both cases of d = 4, 9, the two models reach slightly different but similar MI estimates, but, interestingly, do not differ at all in the test classification also found that using one encoder helps the model converge much faster. Overall, this experiment demonstrates that using two different encoders does not necessarily work to our advantage.
accuracy. Empirically,
G APPENDIX
ROBUSTNESS AND MODEL MISMATCH ANALYSIS
In order to evaluate the robustness of SDRE method, we use the following setups.
Breaking Symmetry: In the experiment setup for the high dimensional experiments as described in Section 5.2, the means of the Gaussian distribution p and q were symmetrical in majority of the cases. order to ensure that this symmetry around zero did not provide an advantage to the SDRE method, we also evaluate it on Gaussian and with randomized means for the two settings shown in rows 2 and 3 of Table 5. Beside the mean parameters, we do not change any other configuration in the experiment the model. sDRE manages to estimate the ground truth KL divergence accurately in this case, therefore establishing that did not benefit unfairly from the symmetry of distributions around zero in the high dimensional experiments.
18

---

both Under review as a conference paper at ICLR 2021
Model Mismatch: In rows 3, 5 and 6 of Table 5, evaluate SDRE by replacing distributions p and q from Gaussian in the case above with Student-t distribution of the same scale with randomized means. We set the degrees of freedom to 10. These experiments test how sDRE performs when there a model mismatch, i.e. how SDRE performs using the same quadratic model that used when p and g to be very robust in these cases and is able to accurately estimate the ground truth KL.
to be Gaussian with lighter tails. As can be seen, SDRE seems
and with Finite Support: Finally, we test SDRE another case where and g are finite support distributions that have both FOD and HOD. For this, we set them truncated normal, as shown in row 1. This setting is similar to the ID example with Gaussian and y that we have used throughout the manuscript. We also set m to be truncated normal with scale set to 2 order to allow it to have overlap with both p and g. As expected, SDRE not only manages to correctly estimate the ground truth KL, but also correctly decays its log ratio estimates when the samples do not fall under the support of p as shown in Figure 13.
Dim
P
m
GT-KL
SDRE
Truncated Normal
Truncated Normal loc=1, scale=0.2 (-1.1,1.2)
Truncated Normal loc=- 1, scale=2
loc=- 1, scale=0.1
50.65
52.35
(-1.1,-0.9)
(-1.1,1.2)
Normal loc=R(-55,55),
Normal loc=R(--5.55, cov=]
160
LM
54.29
54. 10
cov=2 BD
Normal
Mixture of Normal loc=0.9, cov=/ and Normal loc=0.9, cov=/
160
loc=-1, cov=2 BD
LM
105.60
98.27
Student
Student T Ioc=R(-55,55, scale=1, df=10
loc=R(-5,,55,
320
LM
53.82
51.03
scale=2 x BD,
df=10
Normal loc=R(-1,1), cov=2 BD
Normal loc=R(-1,1),
320
LM
110.05
102.63
cov=/
Student 1oc=R(-1,1), scale=2 x 2 BD, df=10
Student T loc=R(-1,1),
320
LM
103.12
113.53
scale=/,
df=10
Student T
Normal loc=0,
loc=0,
320
LM
82.02
83.63
scale=/,
cov=2 BD
df=20
Table Robustness evaluation for sDRE, Here R(a.b) stands for randomized mean vector where each dimen- sion is sampled uniformly from the interval (a, b). LM stands for the linear mixing construction scheme.
H APPENDIX
IMPROVING KMM-BASED DRE WITH SDRE
Another advantage of SDRE is that it is applicable to DRE methods with fixed design setup, i.e. DRE method that directly outputs the densities for a set of samples without building a model; TRE is not applicable to DRE methods in this setup models the ratios directly. The flexibility of our model derives from the fact m is the denominator of both ratios. An example of DRE methods with fixed design setup is the kernel mean matching (KMM) based DRE method (KMM-DRE:
19

---

Under review as a conference paper at ICLR 2021
200
Co8
150
100
Log Ratio
50
0-
-50
-100
-2
2
Samples
sDRE is able match the ground truth log-ratio and correctly decays all the values towards - Do if they don't 't fall under the support of
Figure 13:
40
17.5
KMM ratios & mixing Cauchy
SDRE w/ KMM ratics mising in ratios Cauchy
35
15.0
<DRE KMM ratios & mixture
30
SDRE ratios
12.5
KNM ratios
KMM ratios
True ratles
10.0
1.5
10
0.0
1.00
1.50
2.25
3.00
(a) Varying d (bandwidth 0.1)
(b) Varying a (bandwidth 0.02)
Figure 14: Improving KL estimation between two standard Normals via KMM-DRE using SDRE
Huang al., 2006; Sugiyama al., 2012). The main advantage of this method is that the ratio estimators can be computed in closed form and therefore are computationally more efficient.
We demonstrate the applicability of sDRE in the fixed design setup by applying sDRE to the KMM-DRE method for KL estimation. In order to show the benefit of sDRE, we consider two KI estimation experiments with increasing difficulty: (a) We estimate KI. between N(-d, 1) and (++d, 1) by varying d from 0 to 3 and (b) We estimate KI. between N(0, 22) and N(0, 1/s2) by varying from to For each experiment, we report the following KI. estimates: (1) KI. by MC using true ratios, (2) KL by MC using KMM-DRE estimates, and (3) KL by MC using estimates from KMM-DRE with SDRE for three types of m construction: linear mixing, mixture, and Cauchy.' We conduct the same experiments for different bandwidth σ, the parameter of the Gaussian kernel used by the KMM estimator. Figure 14 summarizes the results. Overall, sDRE significantly improves the baseline KMM-DRE KL divergence estimates by up 11 nats.
As it can be seen from the figures, SDRE consistently improve KMM-DRE both experiments with different difficulty. In Figure 14a, it can be KL estimates closer to the true value even when d = 2.5. while KMM-DRE fails even for d = 1.0. In Figure 14b, we can see sDRE with mixing M helps calibrate the KL estimates from KMM-DRE: For < 1.75, KMM-DRE overestimates the KL and SDRE gives estimates closer to the true KL by giving a lower estimate; for s 2 1.75, KMM-DRE underestimates the KL and SDRE gives estimates closer to the true KL by giving higher estimate. It is also worth noting that different
SDRE with mixture and Cauchy as M gives
For KMM-DRF, we salve the constrained optimization problem using the solvers from JuMPjl, via Den- sityRatioFssimattio.... We the kernel bandwidth to 0.1 and unit sum tolerance
20

---

Under review as a conference paper at ICLR 2021
types of M helps in the two experiments as experiment (a) is case of FOD and (b) is case of HOD.
21


Reviews:
[{"review_id": "XNHYO5jqhGX", "reviewer": "Reviewer_Cfja", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n+ The method is clearly motivated, and is theoretically sound.\n\n+ The paper is written clearly.\n\n+ The proposed method shows significant advantages over other methods, especially when both FOD and HOD density-chasm issues are present.\n\nLimitations/questions:\n\n- The novelty of the proposed method seems incremental. Both Lemma 1 and the idea of telescoping are not new.\n\n- Assume that two densities have disjoint support, then wouldn’t the density ratio estimation be a trivial problem as the ratio is always 0 (or infinity) ? Also, even though two target densities overlap, if one is a mean-shifted version of the other (and they are far apart), intuitively logistic regression would give a better classification due to better separation. Would this contradict the challenge of FOD that you claim? I may have misunderstood some setting of the problem so I hope the authors can help clarify it.\n\n- Most experiments are on normal/Gaussian distributions. Very limited results are given on the robustness to other families of well-known distributions. I wonder how well the proposed method performs given more complicated distributions, e.g. mixture of Gaussian (as a single density) or other distributions of multiple modes.\n\n\nOther comments: \nOn page 5, in the definition of Xm, x_p should be x_p^i\n\n", "summary_of_the_paper": "This work estimates the ratio of two densities using intermediate densities that have sufficient overlap with the target densities. The proposed method is based on the scaled-Bregman Divergence and it can be equivalently formulated as a multi-class logistic regression problem. By combining the idea of telescoping, the proposed approach mitigate the issue of FOD/HOD density-chasm and show superior performance on a variety of datasets.", "summary_of_the_review": "In summary, I like the idea of the paper, but most examples and experiments are on Gaussian distributions which may limit the scenarios of real applications. I would encourage the authors to include more results on more “malicious” densities, or at least explain why such suggested experiments do not make sense.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "IrLdiRsGc-h", "reviewer": "Reviewer_u7f6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is easy to read and generally well-written. However, parts of the paper seem inaccurate or not very rigorous. \n\nAfter offering Lemma 1, the authors conclude that \"Lemma 1 shows that a multi-class logistic regression leads to a multi-class density ratio estimator that minimizes the sBD...\" However, Lemma 1 just shows the connection between regression and DRE, and does not really talk about \"logistic regression\". At least in the standard ML literature logistic regression is a linear classifier with logistic loss. Therefore I see no reason that logistic regression work, unless the distributions are nicely separated (like isotropic Gaussians with different means). \n\nAlso, in Lemma 1, $\\hat{\\eta}$ has not been defined properly. After the statement of Lemma 1, we read \"....the reference density C\". This is quite confusing, since C seems to be just the number of classes (and not a density). Moreover, $r(x)$ in the lemma is a number so how is $D(r(x)||\\hat{r}(x))$ defined?\n\nIt will help if the authors formally define the problem of density ratio estimation, and its measure of success. The discussion of the proposed method in section 4 are at times vague.  It will help to have a clear algorithm, pseudocode, etc. rather than just explaining things.\n\nI could not follow Section 4.3, in particular the argument that says replacing log(m/q) with -log(q/m) is useful. I also found the arguments like \"TRE is only defined when p<<m<<q...\" somewhat sloppy. Why is this true? What goes wrong if m=10q in some parts of the domain? Can you make the argument somewhat more formal? At the end of the same section, the authors make another argument about the benefits of using multiclass logistic regression. However, the connection between consistency of ranking-based noise constrastive estimation and the current paper is not really established. Therefore I don't find the argument well supported.\n", "summary_of_the_paper": "Given samples from two distirbutions, how can we estimate the KL divergence between them? More generally, how can we estimate the ratio of the two densities at given points (e.g, on the samples from one of the dists)? The authors propose a novel method for density ratio estimation (DRE). DRE is challenging when the two distributions have different supports or the density ratio is very high. The authors propose a solution that builds on the previous work, namely Rhodes et al..,  which instead of computing the ratio directly, it writes it as multiplication of a series of density ratios between some intermediate distributions (telescoping). \n\nThe authors improve over Rhodes et al. by accounting for the distribution shift in a systematic way and also slightly changing the way the intermediate distributions are constructed. The empirical results are promising and show the benefit of the approach compared to the baselines.\n", "summary_of_the_review": "The paper proposes a novel DRE method which addresses some of the issues that the previous methods (e.g., TRE) had. The paper is easy to read but some of the arguments are weak and not sufficiently justified. More formal arguments/statements will help with the accuracy and readability of the paper. The paper is not theoretically strong, but offers a useful and practical approach which can be impactful.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ElA4Bnq4u7b", "reviewer": "Reviewer_rhLq", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Main review:\n- This paper attempts to contribute to solving an important problem in the context of DRE.\n- The proposed method seems to be an extension of Rhodes et al. (2020), but has a novelty.\n- The authors propose a novel concepts: FOD and HOD, which are also interesting and meaningful in this context.\n\nQuestions:\n1. What is the data-generating process?\n2. What is the definition of $D_f$? $D_f(a||b) = B_f(a, b| M)$ for some $M$?\n3. What is $\\hat{\\eta}$ in the denominator of $\\hat{r}(x) = \\frac{\\hat{\\eta}(x)}{\\hat{\\eta}}$? \n4. Can the authors compare their proposed method with more methods, such as plain DRE methods and the method proposed by Kato and Teshima (2021)?\n\nMinor points:\n1. I could not understand the following sentence:\n> However, closed-form estimation of density ratios is impossible in most problems of interest, as it requires knowledge of the functional form of the underlying densities.\nWe can approximate the density ratio via a linear-in-parameter model with some kernel function. Combining with the linear-in-parameter model with certain methods yields an estimator with a closed form. For example, the uLISF has a closed form solution in the sense that we can compute the estimator analytically. In the other words, I could not catch the definition of \"closed form.\" \n", "summary_of_the_paper": "This paper addresses one of the essential tasks in machine learning, the problem of density ratio estimation (DRE). In this paper, the authors consider applying the binary classification method to DRE. When the samples are well separated, this problem becomes difficult. To tackle this problem, this paper proposes a framework called scaled Bregman divergence. Under this framework, the density ratio can be estimated successfully. The authors also present some applications in experiments.", "summary_of_the_review": "This paper attempts to contribute to solving an important problem in the context of DRE. While the attempt is novel and very interesting, there is insufficient comparison with existing methods and justification of the proposed method. Therefore, I vote for weak reject. However, depending on the rebuttal, there is a possibility to raise this rating.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Eb6EtC6Oste", "reviewer": "Reviewer_RSpk", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The descriptions and titles should be improved. \nThe paper does not seem to be written in a clear way. For instance, the title in 2.1 is logistic regression but the contents are general results. The title of 2.2 is density chasm but the authors have never explained what it is. \nAlso, while the authors kept mentioning 'logistic regression' in a couple of places (such as Sec 4.1), the results are not limited to logistic regression. \nThe logistic regression places a linear model of the log-odds (i.e., $\\log \\eta(x)$ in the paper), which is a restricted approach in some sense. \nAlso, there is a typo in Lemma 1 that $\\hat r(x)$ should be $\\frac{\\hat \\eta(x)}{1-\\hat \\eta(x)}$.\nMoreover, the quantity $\\pi_C$ in Lemma 1 was never defined.\nIn the first paragraph below Lemma 1, it stated that 'the reference density $C$'. But shouldn't $C$ be the index rather than a density?\n\n2. Misleading statement: \"This empirically confirms that, unlike TRE, the SDRE estimator is theoretically guaranteed ...\" (Page 5). \nThis statement does not make any sense. \nTheoretical guarantee is not from empirical analysis. It should be derived from symbolic analysis (e.g., mathematical derivation).\n\n\n3. Lack of novelty? \nIt seems to me that the only novel part was to use a 'single' intermediate density to estimate the ratio and replace the $\\log [m/q]$ by $-\\log [q/m]$. \nI don't think this is something that novel. \nsDRE has considered a more general version (a sequence) of this idea (See Sec. 4.3). And replacing the log by -log is an elementary approach that many people did.\nThe paper has to provide more analysis and justifications on its contribution. \n\n\n4. Choice of the measure M needs more analysis.\nThis part is the key to this paper since the paper is built on a single intermediate density. \nThis paper will be improved a lot if the authors can provide a detailed analysis and argument on how to choose a good measure $M$ and even prove some useful theorems (a better convergence rate, an elegant representation, an efficient algorithm).\nHowever, the authors only mentioned three ideas without much analysis on each of them. \n\n\n5. Estimation/Learning of $\\eta(x)$.\nIt was never explicitly explained how the authors estimate the class-probability $\\eta(x)$. \nFrom the contents, I guess the authors apply a logistic regression method. But again, $\\eta(x)$ can be estimated by other approach such as a tree classifier. Is the logistic regression offers any benefits on the later analysis? Or can we use other generative classifiers? Questions like this should be answered. \n\n\n6. A bit of redundant of FOD and HOD.\nIt was not clear to me why the authors introduce the concept of FOD and HOD. First, these concepts were not well-defined beyond Gaussian. Second, these concepts were not related to the proposed method and were only used in the experiments. Why not just call them 'mean-shift' Gaussian problems and `variance-shift' Gaussian problems?\n\n", "summary_of_the_paper": "This paper proposed to estimate the (log) density ratio, $\\log [p/q]$ by introducing an intermediate density $m$, and rewrite it as $\\log [p/q] = \\log [p/m] - \\log [q/m]$. The authors then estimate each ratio by a generative classifier (logistic regression). Some simulations were conducted to argue the benefit of the proposed method.", "summary_of_the_review": "This paper lacks several important analysis on the proposed methods and the writing was misleading and confusing. The paper requires a lot of work to improve. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a technique to improve density ratio estimation. This is an important problem and very relevant to the ICLR conference. The main idea is to consider density ratios with respect to intermediate distributions to “scale” the densities and make the ratios easier to estimate by training a suitable discriminative model (classifier). Reviewers found the idea interesting but there was a consensus the paper is not ready for publication.
Decision: Reject