Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression
Authors: Akash Srivastava
Seungwook Han
Kai Xu
Benjamin Rhodes
Michael U. Gutmann
Venue: Accepted by TMLR ()

Abstract: Functions of the ratio of the densities $p/q$ are widely used in machine learning to quantify the discrepancy between the two distributions $p$ and $q$. For high-dimensional distributions, binary classification-based density ratio estimators have shown great promise. However, when densities are well-separated, estimating the density ratio with a binary classifier is challenging. In this work, we show that the state-of-the-art density ratio estimators do perform poorly on well-separated cases and demonstrate that this is due to distribution shifts between training and evaluation time. We present an alternative method that leverages multi-class classification for density ratio estimation and does not suffer from distribution shift issues. The method uses a set of auxiliary densities $\{m_k\}_{k=1}^K$ and trains a multi-class logistic regression to classify the samples from $p, q$ and $\{m_k\}_{k=1}^K$ into $K+2$ classes. We show that if these auxiliary densities are constructed such that they overlap with $p$ and $q$, then a multi-class logistic regression 
allows for estimating $\log p/q$ on the domain of any of the $K+2$ distributions and resolves the distribution shift problems of the current state-of-the-art methods.
We compare our method to state-of-the-art density ratio estimators on both synthetic and real datasets and demonstrate its superior performance on the tasks of density ratio estimation, mutual information estimation, and representation learning.

Published in Transactions on Machine Learning Research (03/2023)
Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression
Akash Srivastava"
akashsri@mit.com edu
MIT-IBM Watson AT Lab and IBM Research
Seungwook Han"
swhan edu
MIT
Kai Xu Amazon
nuk@amazon.com
Benjamin Rhodes
ben, ben.rhodes Reroodesseed.ccom uk
School of Informatics, University of Edinburgh
Michael U. Gutmann'
michael eechael.gutmammmeed..oo
School of Informatics, University of Edinburgh
Reviewed on OpenReview: https: // openreview. net/ forum?? id- JM@nzUzBWWr
Abstract
Functions of the ratio of the densities p/g are widely used in machine learning to quantify the discrepancy between the two distributions p and q. For high-dimensional distributions, binary classification baase density ratio estimators have shown great promise. However, when densities are well seporated, estimating the density ratio with binary classifier is challenging. In this work, we show that the state-of-the-and density ratio estimators per- form poorly on well separated cases and demonstrate that this is due to distribution shifts between training and evaluation time. We present an alternative method that leverages multi-class classification for density ratio estimation and does not suffer from distribution shift issues. The method uses set of auxiliary densities {m+] 1 and trains multi-class logistic regression to classify the samples from p. and {m&] into K + 2 classes. We show that if these auxiliary densities are constructed such that they overlap with p and g, then multi-class logistic regression allows for estimating logp/q on the domain of any of the + 2 distributions and resolves the distribution shift, problerss of the current state-of- tho-art methods. We compare our mothod to state-of-the-art density ratio estimators on both synthetic and real datasets and demonstrate its superior performance on the tasks of density ratio estimation, mutual information estimation, and representation learning. Code: https: :/www///wwwwww blackswhan com/mdre/
1 Introduction
Quantification the discrepancy between two distributions underpins large number of machine learning techniques. For instance, distribution discrepancy measures known as f-divergences (Csiszár, 1964), which are defined expectations of convex functions of the ratio of two densitics, ubiquitous in many domains of supervised and unsupervised machine learning, Hence, densily ratio estimation is often central task in generative modeling, mutual information and divergence estimation, as well as representation learning (Sugiyama et al., 2012; Gutmann & Hyvärinen, 2010; Goodfellow et al., 2014; Nowozin et al., 2016; Srivastava et al., 2017; Belghazi et al., 2018; Oord et al., 2018; Srivastava et al., 2020). However, most problems
"equal contribution

---

Published in Transactions on Machine Learning Research (03/2023)
of interest, estimating the density ratio by modeling each of the densities separately is significantly more challenging than directly estimating their ratio for high dimensional densities (Sugiyama et al., 2012). Hence, direct density ratio estimators are often employed in practice.
One of the most commonly used density ratio estimators (DRE) utilizes binary classification via logistic regression (BDRF). Once trained to discriminate between the samples from the two densities, BDRFs have been shown to estimate the ground truth density ratio between the two densities (e.g. Gutmann & Hyväri- nen, 2010; Gutmann & Hirayama, 2011; Sugiyama et al., 2012; Menon & Ong, 2016). BDREs have been tremendously successful in problems involving the minimization of the density-ratio based estimators of dis- crepancy between the data and the model distributions even in high-dimensional settings (Nowozin et al., 2016; Radford et al., 2015). However, they do not fare as well when applied to the task of estimating the dis- crepancy between two distributions that are far apart or easily separable from each other. This issue has been characterized recently as the density-chasm problem by Rhodes et al. (2020). We demonstrate this in Figure where we employ a BDRE to estimate the density ratio between two 1-D distributions, p = W(-1,0.1) and q 9= (1,, 0.2) shown in panel (a). Since p and q are considerably far apart from each other, solving the classification problem is relatively simple as illustrated by the visualization of the decision boundary of the BDRE However, as shown in panel (b), even in this simple sctup, BDRE completely fails to estimate the ratio. Kato & Teshima (2021) have also confirmed that most DREs, especially those implemented with deep neural networks, tend to overfit to the training data in some way when faced with the density-chasm prob- lem. Since BDRE-based plug-in estimators are often used in many high-dimeusional tasks such as mutual information estimation, representation learning, energy- based modeling, co- ro-variate-shift resolution, and im- portance sampling (Rhodes et al., 2020; Choi et al., 2021b;a; Sugiyama et al., 2012), resolving density-chasm is an important problem of high practical relevance.
A recently introduced solution to the density-chasn problem, telescopic density-ratio estimation (TRE; Rhodes et al., 2020), tackles it by replacing the caslas-to- original logistic regression problem, by a set of harder-to-classiffy logistic regression problems. In short, TRE constructs a set of K auxiliary distributions ([mx)) 1K 1 to bridge the two larget distributions (p =: mi and q =: mk+1) of interest and then trains a set of K + BDREs on every pair of consecutive distributions (mx=1 and my for k = .,..., K), which are assumed to be close enough (i.c. not casily separable) for BDREs to work well. After that, an overall density ratio estimate is obtained by taking the cumulative (telescopic) product of all individual estimates.
In this work, we argue that the aforementioned solution to the density chasm problem has an inherent issue of distribution shift that can lead to significant inaccuracies in the final density ratio estimation. Notice that the i-th BDRE in the chain of BDREs that TRE constructs is only trained on the samples from distributions m, and m However, post-training, it typically evaluated on regions where the distributions from the original density ratio estimation problem (i.e. p and q) have non-negligible mass. If the high-probabilitty regions of p. 9 and the auxiliary distributions m; do not overlap, the training and evaluation distributions for the i-th BDRE are different. Because of this distribution shift between training and evaluation, the overall density ratio estimation can end up being inaccurate (sce Figure 2 and Section 2.1 for further details). We here provide another solution to the density-chasm problem that avoids this distribution shift.
We present MULTINOMIAL LOCISTIC REGRESSION BASED DENSITY RATIO ESTIMATOR (MDRE), a novel method for density ratio estimation that solves the density-chasm problem without suffering from distribution shift. This is done by using auxiliary distributions and malli-class classification. MDRE replaces the easy binary classification problem with a single harder multi-class classification problem. MDRE first constructs set of K auxiliary distributions km that overlap with p and q and then uses multi-class logistic regression on the K + 2 distributions to obtain density ratio estimator of log p/q. We will show that the multi-class classification formulation avoids the distribution shift issue of TRF.
The key contributions of this work are as follows:
We study the statc-of-tho-art solution to the density-chasm problem (TRE; Rhodes et al., 2020) and identify its limitations arising from distribution shift. We illustrate that this inherent issue can significantly degrade its density ratio estimation performance.
2

---

Published in Transactions on Machine Learning Research (03/2023)
Thep
Trun ps
log-tstic
224
400
409
-26 -85
0. 05
30
345
-10
04
10
460
(a) BDRE class probability (b) BDRE log density-ratio
(c) MDRE class probability (d) MDRE log density-ratio
Figure 1: BDRE vs proposed MDRE on estimation of log density ratio where p NN(-1,0.1 and M(1,0.21 For MDRE, the auxillary distribution m is Cauchy C(O, Plots (a) and (c) show the class probabilities P(Y[z) learned for BDRE and MDRE respectively overlayed on the plots of and m. Plots (b) and (d) show the estimated log density-ratio by BDRE and MDRE respectively, Using auxiliary distribution m allows MDRE to better estimate the log density-ratio.
2. We formally establish the link between multinomial logistic regression and density ratio estimation and propose a novel method (MDRE) that uses auxiliary distributions to train a multi-class clas- sifier for density ratio estimation. MDRE resolves the aforementioned distribution shift issue by construction and effectively tackles the density chasm problem.
3. We construct a comprehensive evaluation protocol that significantly extends on benchmarks used in prior works. We conduct systematic empirical evaluation of the proposed approach and demon- strate the superior performance of our method number of synthetic and real datasets. Our results show that MDRE is often markedly better than the current state-of-the-art of density ratio estimation on tasks such as f-divergence estimation, mutual information estimation, and represen- tation learning in high-dimensional settings.
2 Related Work
Telescopic density-ratio estimation (TRE, Rhodes et al., 2020) uses two step, divide-and-conquer strategy to tackle the density-chasm problem. In the first step, they construct K waymark distributions {ma J by gradually transporting samples from p towards samples from 9. Then, they train K BDREs, one for each consecutive pair of distributions. This allows for estimating the ratio Tp/g as the product of K + 1 BDREs, Tp/9 == P P x... ....X "x Rhodes et al. (2020) introduced two schemes for creating waymark my distributions that ensure that consecutive pairs of distributions are packed closely enough so that none of the K 1 BDREs suffer from the density-chasm issue, Hence, TRE addresses the density-chasm issue by replacing the ratio between p and g with product of K + intermediate density ratios that, by design of the waymark distribution, should not suffer from the density- (2021b) introduced DRF-co, a method that takes the number of waymark distributions in TRE to infinity and derives a limiting objective that leads to a more scalable version of TRE.
problem. In new work, Choi et al.
F-DRE is other interesting related work that comes from Choi et al. (2021a). F-DRE uses a FLOW- basod model (Rezende & Mohamed, 2015) which trained to project samples from a mixture of the two distributions onto standard Gaussian. They then train BDRE. is easy show that any bijective map will preserve the original density ratio Tp/q in the feature space the Jacobian correction term simply cancels out. However, due to the bijectivity of the FLOW map, such method cannot bring the projected distributions any closer than the discrepancy between the original distributions. At best, the method can shift the discrepancy between the original distributions along different moments after projection. Due to this issue, we found that F-DRE did not work well for the problems we considered (see experimental results in Scction 4). Recently, Liu al. (2021) introduced an optimization- solution to the density-chasm problem in exponential family distributions by using (a) normalized gradient descent and (b) replacing the logistic loss with exponential loss. Finally, while BDRE remains the dominant method of density ratio
3

---

Published in Transactions on Machine Learning Research (03/2023)
estimation in recent literature, prior works, such as Bickel et al. (2008) and Nock et al. (2016), have studied multi-class classifier-based density ratio estimation for estimating ratios between a set of densities against a common reference distribution and its applications in multi-task learning.
2.1 TRE's performance can degrade due to
distribution shifts
In supervised learning, distribution shift Couiñonero-Candeee et al., 2009) occurs when the training data (z, v) ~ Ptrain and the test data (x, ~ Ptest come from two different distributions, i.e. Perain # Prest- Common training methods, such as those used in BDRE, only guarantee that the model performs well on unseen data that comes from the same distribution as Pursin. Thus, in the case of distribution shift at test time, the model's performance degrades proportionately to the shift. We now show that a similar distribution shift can occur in TRE when distributions p and q are sufficiently different. Recall that in TRE, we use BDREs to estimate K ++1 density ratios p/ p/m1, m2/mm........................... mK m////q/ that are combined in a telescopic product to form the overall ratio p/g. Let us denote the estimates of the K + 1 ratios by f1, ......., ĀK+1
Given the theorctical properties of BDRE, for any E {1..... K ++,, f) estimates I'm 1/m; over the support of m: (Sugiyama et al., 2012; Gulmann & Hyvärinen, 2010; Menon & Ong, 2016). However, in TRE, when we evaluate the target ratio p/q on the supports of p and g, we evaluate the individual fr. on domains for which we lack guarantecs that they perform well. Since the overall estimator for p/q = it *.... * ft++1 combines multiple ratio estimators, it suffers from the distribution shift issue if any of the individual estimators' performance deteriorates. Thus, if the supports of {m P, and q are different, or when the samples from {m H/ P, and do not overlap well enough, the training and evaluation domains of the î are different and we expect the ratio estimate fi and, in turn, the overall estimator for p/q to be poor. We now illustrate this with a loy example.
We consider estimating the density ratio between p= = =N(11,0,,) and q (V11, Since, p and are well separated, we introduce three auxiliary distributions m., m2, my to bridge them, providing the waymarks that TRE needs. The auxiliary distributions m1,m2, my are constructed with the linear-miring strategy that will be described in Section 3.2. This setup is shown in the top-left panel of Figure 2. We train 4 BDREs în fi2 fr. f44 to estimate ratios p/ p/mI, mi /m2,, m2/mm and ma/g respectively. We begin by showing that each of the trained BDREs estimates their corresponding density ratio accurately on their corresponding training distributions. To show this, in panels 2-5 in the first row of Figure 2, we evaluate îu, îl, fa, îa on samples from their respective denominator densities m1, M2, Ma, 9 and plot them via a scatter plot where the x-axis is labeled with the distribution that we draw the samples from and the y-axis is the log-density ratio (red). We plot the true density ratio in blue for comparison. As evident, red and blue scatter plots overlap significantly, indicating the individual ratio estimators are accurate on their respective denominator (training) distributions.
Next, we evaluate the BDREs fn, fl, is, faa on samples from p and q instead of their corresponding training distributions as before. Distributions and are shown in panel 1 of the second row in Figure 2. In the rest of the panels (2-5) in the second row, estimators fn, fiz, fis, iu are compared to the ground-truth log-density ratios (bluc) p/m1,m, m1/m2, m2/m3 and m3/q that are also evaluated on samples from p and g. Unlike in row 1, the estimated log-density ratios do not match the ground-truth. This reflects the training-evaluation distribution-shift issues pointed out above. We show now that this deterioration in accuracy on the level of the individual BDREs results in deterioration of the overall performance of TRE. To this end, we first recover the TRE estimator by chaining the individually trained BDREs via telescoping product, i.c in * f/2 * fea * i4 and then evaluate it on samples from all the 5 distributions p, mı, M2, ma. q. The results are shown in panels 1-5 of the third row. The estimated log-density ratios (red) do not match the corresponding ground-truth log-density ratios (blue), which demonstrates that the distribution-shitt in the training and evaluation distributions of the individual BDREs significantly degrades the overall estimation accuracy of TRE. Additional issues occur when both p and q do not have full support, as discussed in Appendix G.
4

---

Published in Transactions on Machine Learning Research (03/2023)
Benry mndee
Rinry ochec
Barry
halognaiee Soary sina
Brary mine
Beary
Tue
Seary Aep
Enwykagh
Brary loga
Emary
In all scatter plots, x-axis denotes the sampling distribution and y-axis denotes the log-density-ratio. The density plot in the first row shows P. q and the 3 waymarks; the density plot in the second row shows and q only. The scatter plots in the first row show individual density ratio estimators evaluated on samples from their corresponding training data (denominator density), demonstrating accurate estimation the training set. The scatter plots in the second show individual density ratio estimators evaluated samples from p and q. The estimation accuracy has degraded notably due to the train-eval distribution shift. The last row shows the performance of the overall density ratio estimator on samples from p. m1, M,, ma, 9. We see that the overall ratio estimate is significantly affected by the deterioration of the individual ratio estimates, illustrating the sensitivity of TRE to distribution shift problems in case of well-soparated distributions.
Figure 2: TRE for p = - and q 0.2) from Figure 1.
3 Density Ratio Estimation using Multinominal Logistic Regression
We propose MULTINOMIAL LOGISTIC REGRESSION BASED DENSITY RATIO ESTIMATOR (MDRE) to tackle the density-chasm problem while avoiding the distribution shift issues of TRE. As in TRE, we introduce a K set of 2 auxiliary distributions (m But, in constrast. to TRF, we then formulate the problem
of estimating logp/q as a multi-class classification problem rather than sequence binary classification
problems. We show that this change leads to an estimator that is accurate on the domain of all K + 2 distributions and, therefore, does not suffer from distribution shift.
3.1 Loss function
We here establish formal link between density ratio estimation and multinomial logistic regression. Consider set of C distributions {pe]_= and let px(x) = Σ TePe(x) be their mixture distribution, with prior class probabilities म..² The multi-class classification problem then consists of predicting the correct class Y (I....,Cr C} from a sample from the mixture pz. For this purpose, we consider the model
P(Y c22;0)= Te cxp(h)(())
(1)
Σe k= TA exp (xp(h)(xr))
In our simulations, we will uniform prior over the classes.
5

---

Published in Transactions on Machine Learning Research (03/2023)
where the hz)x,, = I....,C are unnormalized log probabilities parametrized by 0. We estimate A by minimizing the negative multinomial log-likelihood (i.e. the softmax cross-entropy loss) L())
C
( = Σ
n E. Exp, llo P(Y = c|x;0]] =
_ n T.p..
-
-log
log Te - hg + log Σ Th exp(h
(2)
where, in practice, the expectations are replaced with sample average. We denote the optimal parameters by 6* = arg ming (()). To case the theoretical derivation, we consider the case where the hī(x) are parametrized in such a flexible way that we can consider the above loss function to be a functional of C? functions hi.... .......... Icc
C(hi,..., 1ic)
Σ Te TelEE
-log log ne. No + log Σ Th exp(((a (x))
The following propositions shows that minimizing C(hi,...., hc) allows us to estimate the log ratios between any pair of the C' distributtinss Pc-
Proposition 3.1. Lel fu....,hh he be the minimizzes of E(hi,..., .c)) in equation 3. Then the densily ralio between p+(a)) and py (x) for any i.j S is given by
(x(x) (5(x)
log
h2(2)) - -z)
(4)
for all x where p=(x) = =e mePe(x) > O.
Proof. We first note that the sum of expectations Eem1 TzEEpp, in equation 3 is equivalent to the expectation with respect to the mixture distribution Pa- Writing the expectation as an integral obtain
( C(h hc) Σ n T.Bpe l-gg hc(z)] + / px(x)]log Ty exp(hx(x)))ddz
(5)
The functional derivative of [(hi,..., ..li,...,hee hc) with respect to hi, i=1..., C, equals
8L mx)p()) + (x(
T. m<xp((;;xx))
(6)
8h;
E n exp(hx(x))
for all x where pxxx)= > 0. Setting the derivative to zero gives the necessary condition for an optimum
(x) T. exxx((;;(z))
i= 1,.., C, and for all x where p>(x) >
Pa (x) ki Th exp(hu(z))
The left-hand side of equation 7 equals the true conditional probability P*(Y i12) the critical point, hi,..., he are such that Pa (*(Y)X) is correctly estimated. From equation 7, follows that for two arbitrary and j, we have (mpp))/(z,p))) = (x; exp(hx))/(m) exp(h;)) i...
TxP:(x)
Hence,
P=(
(x(x) log
Rx(x)) - h21xx)
(8)
Pi(#')
for x where pz(x)= > 0, which concludes the proof.
Remark 3.2 (Identifiability). While we have unknowns hy...., he and C equations in equation 7, there is
redundancy in the equations bocause
TaPi(x) (x)
T. exp(h;(4))
De exp(h((())) =1 Ee Th exp(he(x))
=k=1 Th exp(he(xz)
This means that we cannot uniquely identify all h2 by minimising equation 3. However, the difference h hihhi, hyj for # j, can be identified and is equal to the desired log ratio between P: and P3 per equation 8.

---

Published in Transactions on Machine Learning Research (03/2023)
Remark 3.3 (Effect of parametrisation and finite sample size). In practice, we only have finite amount of training data and the parametrisation introduces constraints on the flexibility of the model. With additional assumptions, e.g. that the true density ratio po) - log Pj (2) can be modeled by the difference of (z) and (x), we show in Appendix A that our ratio estimator is consistent. We here do not dive further into the asymptotic properties of the estimator but focus on the practical applications of the key result in equation 8.
Importantly, equation 8 allows us to estimate rp/g by formulating our ratio estimation problem as a multi- nomial nonlinear regression problem as summarized in the following corollary.
Corollary 3.4. Let the distributions of the first two classes be p and q, respectively, i.c. pl = P. = 4 and the remaining K distributions be equal to the auriliary distributions mi, i.e. P3 = ....... PK+2 = mk. Then
log f p/)(())) = hz( - -2(x))
(9)
Remark 3.5 (Free from distribution shift issues). Since equation 8 holds for all x where the mixture pz(x) > 0, the estimator (x))(x) in equation 9 is valid for all x in the union of the domain of p, 9, m........ my. This means that MDRE does not suffer from the distribution shift problems that occur when solving a sequence
of binomial logistic regression problems as in TRE. We exemplify this in Section 3.3 after introducing three schemes to construct the auxiliary distributions my m M...
3.2 Constructing the auxiliary distributions
In MDRE, auxiliary distributions need to be constructed such that they have overlapping support with the empirical densities of and q. This allows the multi-class classification probabilities to be better calibrated and leads to an accurate density ratio estimation ion. We demonstrate this in panel (c) of Figure 1, where p= --1,,00) and q= W11, and the single auxiliary distribution m is set to be Cauchy C(O, 1) that clearly overlaps with the other two distributions. The classification probabilities are shown as the scatter plot that is overlayed on the empirical densities of these distributions. Compared to the BDRE case in panel (a), which has high confidence in regions without any data, the multi-class classifier assigns, for p and q. high class probabilities only over the support of the data and not where there are barely any data points from these two distributions. Moreover, the auxiliary distribution well covers the space where p and g have low density, which provides the necessary training data to inform the values of (z) and (2 in that area, which leads to an accurate estimate of the log-density ratio shown in pancl (d). This is contrast to BDRE in panel (a) where the classifier, while constrained enough to get the classification right, is not learned well enough to also get the density ratio right (panel b). This subtle, yet important. distinction between the usage of auxiliary distributions in MDRE compared to BDRE and TRE enables MDRE to generalize on out-of-domain samples, as we will demonstrate in Section 4.1.
Next, we briefly describe three schemes to construct auxiliary distributions for MDRE and leave the details to Appendix B: (1) Overlapping Distribution Unlike TRE, the formulation of MDRE does not require "gradually bridging" the two distributions p and 9, hence, we introduce a novel approach to constructing auxiliary distributions. We define my as any distribution whose samples overlap with both p and 4, and p <<< m, 9 << my- This includes heavy-tailed distributions (o.g. Cauchy, Student-t), normal distributions, uniform distributions, or their mixtures. We use this scheme in all low-dimensional simulations. (2) Linear Mixing In this scheme, m& is defined as the distribution of the samples generated by linearly combining samples X, {ar J and Xg = (= from distributions p and q, respectively. The generative process for a single sample zin from m, is given by I'n, = (1- ag)=", + ayæ, with Tp E Xp. E X,- This construction is similar to the linear combination scheme for auxiliary distributions introduced by Rhodes et al. (2020) with a few key differences that we expand upon in Appendix B. One difference is that aş is not limited o 0 < ag < 1, which allows for non-convex mixtures that completely surround both p and q. We use this construction scheme in higher-dimensional simulations. (3) Dimension-win wise Mixing This construction scheme was introduced in (Rhodes et al., 2020). Samples from the single auxiliary distribution m are obtained by combining different subsets of dimensions from samples from p and q. We use this scheme for experiments involving high-dimensional image data.
7

---

Published in Transactions on Machine Learning Research (03/2023)
2.5
100
2.0
1.-
100
-200
0.5
True lop pM KoRE kgg pto
Trac log pi) MoRE log pi
True log aq MoRE log pmu
0.0
300
-2
Figure 3: MDRE with CO, 1). The first density plots shows the target densities well as the auxiliary distribution (in green, hard to see due to heavy lail and the range of axes). The three scatter plots show the estimated (red) and true (blue) density ratio p/g evaluated samples from p, m, and MDRE accurately estimates the ratio across the input domain. Contrast with Figure 2.
150
True log piu Our log pig
Tue log pla Our log piq
100
50
Log
-50
-100
-150
The log Our pig
Tue log pia Our log pin
The pia Our log pia
-200
-2
-2
2
-2
Figure 4: MDRE using TRE's auxiliary distributions. Each scatter plot shows the overall log-density ratio estimates samples from the distribution on the x-axis (MDRE in red and true ratio in blue). MDRE is capable of accurately estimating ratios on all samples. Contrast with the bottom row of Figure 2.
3.3 Free from distribution-shiftt problems
We continue with the example task of estimating the density ratio between p N (-1, 0.1) and q (1, 0.2) and here illustrate Remark 3.5 that MDRE does not suffer from the distribution shift problem identified in Section 2.1. We test MDRF with two types auxiliary distributions. First using heavy-tailed distribution (m Cauchy(O, 1)), under the overlapping distributions scheme, and second, with waymark distributions m1, M2, my as used by TRE in Figure 2 using their linear-mixing construction scheme.
Figure 3 shows the result for the heavy tailed Cauchy (0, 1) auxiliary distribution (green, shown in the left most figure). We can see that the log-ratio learned by MDRE is accurate even beyond the empirical support of and q. This is because MDRE is trained samples from the mixture of p, and and hence, per Remark 3.5, does not encounter distribution-shift, over the support of the mixture distribution. Figure 4 shows the result when using the auxiliary distributions of TRE that we used in Figure 2. We see that the learned log-ratio well matches the true log-ratio on samples from and as well as the auxiliary distributions. This can be directly compared to third row Figure where TRE suffers from distribution shift problems and does not yield well-ostimated log-ratio. Note that we do not present results that correspond to the second row of since the estimation of the log-ratio in MDRE does not depend on any intermediate density ratios.

---

Published in Transactions on Machine Learning Research (03/2023)
p
True KT.
BDRE
TRE
F-DRE
MDRF (ours)
W(-1,0.08)
(2,, 0.15)
200.27
21.74 + 4.10
136.05 + 5.91
14.87 + 1.72
203.32 + 2.01
2(-2,0.08)
(2, 0.15)
355.82
20.22 + 3.64
208.11 + 18.31
14.22 + 5.30
360.35 + 1.37
Table 1: 1D density ratio estimation task for p and q with large first-order and higher-order differences. In all cases, MDRE outperforms all the baselines.
500
500
-500
-500
-1000
F-DRE
-1000
F-DRE
BORE
BDRE
15000
1530
Ground Truth
Ground Truth
Ours
-2000
-2000
-10
5
10
10
D
5
10
(a) p = q=
(b) p N( = (2
Figure 5: Log density-ratio estimates corresponding to the numbers reported in Table Note that the ground truth and MDRE curves are overlapping, while all the other estimators are significantly worse.
4
Experiments
We here provide an empirical analysis of MDRE on both synthetic and real data, showing that it performs better than previous BarhodssBDRA.e TRE, and F-DRE-on three different density ratio estimation tasks. We consider cases where numerator and denominator densities differ because their mean is different, i.e. p and g exhibit first-order discrepancies (FOD), and cases where the difference stems from different higher- order moments (higher-order discrepancies, HOD). Ratio estimation is closely related to KL divergence and mutual information estimation since the KL divergence is the expectation the log-ratios under p. and mutual information can be expressed as a KL divergence between joint and product of marginals. Being quantities of core interest in machine learning, we will use them to evaluate the ratio estimation methods.
4.1 1D Gaussian experiments with large KL divergence
In the following 1D experiments, we consider two scenarios, one where p 1==, and q = W/ 0.15), and one where the mean of p is shifted to -2 in order to increase the degree of separation between the two distributions. In both cases, MDRE's auxiliary distribution is Cauchy(O,1), so that have three-class classification problem (p, q, m) and three functions hi that parameterize the classifier of MDRE. The three functions are quadratic polynomials of the form wpx² + w2z bb. For all the methods we set the total number of samples to 100K.2 2 We provide the exact hyperparameter settings for MDRE and other baselines in Table 5 in Appendix C.
Table shows the results. We can see that MDRE yields more accurate estimates of the KL divergences than the baselines, which are off by a significant margin.
We note that KL estimation only requires evaluating the log-ratios samples from the numerator dis-
tribution P. In Figure 5, we thus show results for all methods where we evaluate the estimated log- ratios on a wide interval (-12, 12). The figure shows that none of the baseline methods can accurately estimate the ratio well on the whole interval while MDRE performs well overall. This is important be-
We found that MDRE's results are unchanged even when using smaller sample sixes IK 10K, see Table in Appendix

---

Published in Transactions on Machine Learning Research (03/2023)
Dim
[1, 122
True MT
BDRE
TRE
F-DRE
MDRE (ours)
0, 0
20
10.90 + 0.04
14.52 + 2.07
14.87 + 0.33
18.81 + 0.15
40
-1, 1
100
29.03 + 0.09
33.95 + 0.14
13.86 + 0.26
119.96 + 0.94
0, 0
40
21.47 = 2.62
34.09 + 0.21
12.89 = 0.87
38.71 - 0.73
160
-0.5, 0.6
136
24.88 + 8.93
69.27 + 0.24
13.74 + 0.13
133.64 + 3.70
0, 0
80
23.47 = 9.64
72.85 + 3.93
9.17 + 0.60
87.76 + 0.77
320
-0.5, 0.5
240
24.86 + 4.07
100.18 +: 0.29
10.53 + 0.03
217.14 + 6.02
Table 2: Higl-dimensional mutual information estimation task. MDRE is able to accurately estimate the MI often by very large margins.
cause it means that the ratio is well estimated in regions where p and q have little probability mass. These results demonstrate the effectiveness of MDRE with single auxiliary distribution whose samples overlaps with those from both p and q, in lieu of using chain of BDREs with up to K 28 closely- packed auxiliary distributions as used by TRE. Please see Appendix C for additional results and details.
200
To provide further clarity into MDRE's density ratio es- timation behavior, we analyze the uncertainty of its log ratio estimates using Bayesian analysis. We use a stan- dard normal prior the classifier parameters and obtain posterior samples with Hamiltonian Monte-Carlo. These posterior samples then yield samples of the density ra- tio estimates. Figure 6 shows that the high accuracy of MDRE's KL divergence estimates can be attributed to MDRE being confidently accurate around the union of the high density regions of both p and q. A more detailed analysis is provided in Appendix D.
true (mean)
100
ours (3o)
100
200
300
400
5000
-8
-6
-2
Figure 6: Bayesian analysis of MDRE for p
4.2 High dimensional experiments with large MI
Following Rhodes et al. (2020), we use the MI estimation benchmark from Belghazi et al. (2018); Poole et al. (2019) to evaluate MDRE on a more challenging, estimate the mutual information between a standard normal distribution and a Gaussian random variable x E 2d with a block-diagonal covariance matrix where each block is 2 x 2 with ones on the diagonal and P on the off-diagonal. The correlation coefficient. p is computed from the number of dimensions and the target mutual information d/2 log(1 1 p²). Since this problem construction only induces higher-order discrepancies (HOD), we added an additional challenge by moving the mcans of the two distributions, thus additionally inducing first-order discrepancies (FOD).
problem. In this task, the goal is to
For MDRE, we model the hi with quadratic functions of the form x" W1 W2 + b. We use linear-mixing to construct each m,, where K = 3 or K = 5. In Appendix E, we provide the exact configurations for MDRE in Table 6 and explain how to choose m and K in practice.
Table 2 shows the results for cach MI task averaged across runs with different random soeds. MDRE outperforms all baselines in the original MI task where the means of the distribution are the same. The difference between the performance of MDRE and the baselines is particularly stark when the means are allowed to be nonzero. Only MDRE estimates the MI reasonably well while all basclines dramatically underestimate it. We further note that MDRE only uses up to 5 auxiliary distributions, lowering its compute requirements compared to TRE, which is the next best performing method and uses up to 15 auxiliary distributions for its telescoping chain.
We found that the resolution proposed by Kato & Teshima (2021) to overcome the over- wver-fitting issue in Bregman Divergence minimizationn-asee DRFs, does not work well in practice. On the high-dimensional
10

---

Published in Transactions on Machine Learning Research (03/2023)
setup of row 2 in Table 2, while the ground truth MI is 100, and MDRE estimates it as 119 0, 94, the best model from Kato & Teshima (2021) ) yields 1.60, significantly underestimating the true value and being a factor of ten smaller than the classifier-basse DRE baselines. For further results, such as plots of estimated log ratio vs. ground-truth log ratio, training curves, and more, please see Appendix E.
Above, following prior work, we evaluated the methods on problems where p and are normal distributions. To enhance this analysis, we further evaluate MDRE on the three new experimental setups below. The results are summarized in Table 3.
Breaking Symmetry In our high-dimensional experiments reported in Table 2, the means of the Gaussian distributions p and were symmetric around zero in the majority of cases. In order to ensure that this symmelry did not provide an advantage to MDRE, we also evaluate it on Gaussians p and with randomized means. The results are shown in rows and 6 of Table 3. We see that MDRE continues to estimate the ground truth KI divergence accurately, demonstrating that it did not benefit unfairly from the symmetry of distributions around zero.
Model Mismatch In rows 4, 5, 7, and of Table 3, evaluate MDRE by replacing butions p and 9 with a Student-t distribution of the same scale with randomized means. For the Student-t distributions, we set the degrees of freedom as 5, 10 or 20. These experiments test how well MDRE performs when there is model mismatch, iee. how MDRE performs using the same quadratic model that was used when p and q were set to be Gaussian with lighter tails. We find that MDRE still able to accurately estimate the ground truth KL in these cases. We found the same be true for other test distributions such Mixture of Gaussians (shown in row 3 of Table 3).
both distri-
Finite Support and q Finally, we test MDRE another problem where and q are finite support distributions that have both FOD and HOD. This is done by setting and to be truncated normal distributions, as shown in row 1 of Table 3. We also set m to be truncated normal distribution with its scale set to to allow it to have overlap with both p and q. This setting is similar to the 1D Gaussian example illustrated Section 3.3 and MDRE manages lo estimate the ground-Iruth KL divergence accurately.
Dim
True KL
Est. KL
Truncated Normal loc=-1, scalc=0.1 support=(1.11.0.9)
Truncated Normal
Truncated Normal
loc=1, scale=0.2 (-111rt=(11111122))
loc=-1, scale=2
50.65
52.35
(1p1ort=(1111122)
Normal
Normal loc=R(-.5,.5), cov=1
160
Linear Mixing
54.29
54.10
loc=R(-55,55, cov=2 BD
Normal
MoG: 0.5* (0.9, I) + 0.5*Normal(111.)
160
Linear Mixing
105.60
98.27
loc=-1, cov=2 BD
Student T (loc=R(-5,55).
Student T
160 scale=2 2 BD, df=5
Linear Mixing
51.26
49.01
loc=R(-55,.5, scale=I, df=5
Student T loc=R((- ())=R(-.,,55),
Student T
320 scale=2 BD, df=10
Linear Mixing
53.82
51.03
loc=R(-55..5, scale=I, df=10
Normal
Normal loc=R(-1,1), cov=T
320
Linear Mixing
110.05
102.63
1-11R,-,,1), cov=2 x BD
Student T loc=R(-1,1),
Student T
320
Linear Mixing
103.12
113.53
scale=2 x 2 BD, d0-10
1oc=R(-1,1), scale=1, df=10
Normal
Student
320
Linear Mixing
82.02
83.63
loc=0, cov=2 BD
loc=0, scale= df=20
Table 3: Robustness evaluation for MDRE. Here R(a.b) stands for randomized mean vector where each dimension is sampled uniformly from the interval (a, b). MDRE is able to consistently estimate the ground-truth KL with high accuracy all of the cases.
11

---

Published in Transactions on Machine Learning Research (03/2023)
102
15
Mocn laboo scaraay Fool
-66
Putue intormat in
TRC Durs
24
-20
72
13-
62
-52
abd
92
Sivple
Our
125
1.50
2.00 2.35
2.50
2.5
3.00
Number charaatlers
Number characters
(a) MI estimation
(b) Classification accuracy
(c) Varying K (# of auxiliary dist.)
Figure SpatialMultiOmmiglott representation learning results. Plot (a) shows the MI estimated by the three methods, MDRE able to estimate the ground truth MI very accurately. Plot (b) shows the resulting classification accuracy and plot (c) the impact of varying the number of auxiliary distributions MI
estimation with MDRE.
4.3 Representation learning for SpatialMultiOmniiglot
In order to benchmark MDRE on large-scale real-world data, following the setup from Rhodes et al. (2020), we apply MDRE to the task of mutual information estimation and representation lcarning for the Spatial- MultiOmniglot problem (Ozair et al., 2019). The goal is to estimate the mutual information between w and v where u is a n X n grid of Omniglot characters from different Omniglot alphabets and v is a n x n grid containing (stochastic) realizations of the next characters of the corresponding characters u. After learning, we evaluate the representations from the encoder with a standard linear evaluation protocol (Oord et al., 2018). For MDRF, similarly to TRE, we utilize separable architecture commonly used in the MI- based representation learning literature and model the unnormalized log-scores hi with functions of the form g(u)TW W fu) where and f are 14-layer convolutional ResNets (He et al., 2015). While this model amounts to sharing of parameters across the hig we would like to emphasize that in all preceding examples, we did not share parameters among the hg. We construct the auxiliary distributions via dimension- wise mixing.
We here only compare MDRE the single ratio baseline and TRE because Rhodes et al. (2020, Figure 4) already demonstrated that TRE significantly outperforms both Contrastive Predictive Coding (CPC) (Oord et al., 2018) and Wasserstein Predictive Coding (WPC) (Ozair al., 2019) on exactly the same task. Please refer to Appendix F for the detailed experimental setup.
As can be seen in Figure 7a, MDRE performs better than TRE and the single ratio baseline, exactly matching the ground truth MI. This improvement in MI ostimation is reflected in the representations. Figure 7b illustrates that MDRE's encoder learns representations that achieve 1000% Omniglot character classification for both d n = 4,9. On the other hand, the performances of the single ratio estimator and TRE (using the same exact dimension-wise mixing to construct auxiliary distributions) both degrade as the complexity of the task increases, with TRE only reaching up to 91% and 85% for d = 4 and d = 9, respectively. All models were trained with the same encoder architecture to ensure fair comparison.
We further studied the effect of changing K in the d = 4 setup. For K = 1, we aggregate all the dimension- wise mixed samples into 1 class, whereas for K = 3, we separate them into their respective classes (corre- sponding to the number of dimensions mixed). We illustrate this effect in Figure 7c. In line with the finding of Ma & Collins (2018), increasing the number of not only helps MDRE to reach the ground truth M1, but also the quality of representations improves from 86.7% to 100% test classification accuracy.
5 Discussion
In this work, we presented the multinomial logistic regression based density ratio estimator (MDRE), a new method for density ratio estimation that had better finite sample (non-asymptotic) performance in our simulations than current state-of of-te--f-thh-rtt the-art methods. We showed that it addresses the sensitivity possible issues of the recent method by Rhodes et al. (2020). MDRE works by introducing auxiliary
12

---

Published in Transactions on Machine Learning Research (03/2023)
distributions that have overlapping support with the numerator and denominator distributions of the ratio. It then trains multinomial logistic regression model to estimate the density-ratio. We demonstrated that MDRE is both theoretically grounded and empirically strong, and that it sets a new state of the art for high-dimensional density ratio estimation problems.
However, there are some limitations. First, while the ratio was well estimated in our empirical studies, we do not provide any bounds on the estimation, meaning that estimated KL divergences or mutual information values may be over- or underestimated. Second, the choice of the auxiliary distribution, m, is impor- tant factor of consideration that significantly impacts the performance of MDRE. While in this work we demonstrate the efficacy of three schemes for constructing the auxiliary distribution, empirically, it is, by no means, an exhaustive study. We hope to address these issues in future work, including the development of dearning-basee approaches to auxiliary distribution construction.
References
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R. Devon Hjelm. Mine: mutual information neural estimation. arXiv preprint arXiv:1801. 040620 2018.
Steffen Bickel, Jasmina. Bogojeska, Thomas Lengauer, and Tobias Scheffer. Multi-task learning for hiv therapy screening. In Proceedings of the 25th international conference Machine learning, pp. 56-63, 2008.
Kristy Choi, Madeline Liao, and Stefano Ermon. Featurized density ratio estimation. ar preprinl arXiv:2107.02211 2021a.
Kristy Choi, Chenlin Meng. Yang Song, and Stefano Ermon. Density ratio estimation via infinitesimal classification. arXiv preprint arXi::2111.11011 2021b.
Imre Csiszár. An information-theoreiic inequality and its application to the evidence of the ergodicity of markoff's chains. Magyer Tud. Akad. Mat. Kutato Int. Koezl., 8:85 108, 1964.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. Neural Information Processing Systems, 2014.
M. U. Gutmann and J. Hirayama. Bregman divergence as general framework to estimate unnormalized statistical models. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAT), 2011.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnor- malized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and slatistics, pp. 297 304. JMLR Workshop and Conference Proceedings, 2010.
M.U. Gutmann and A. Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications natural image statistics. Journal of Machine Learning Research, 13:307-361 2012.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
Masahiro Kato and Takeshi Teshima. Non-negative bregman divergence minimization for deep direct density ratio estimation. In International Conference on Machine Learning, pp. 5320 5333. PMLR, 2021.
Bingbin Liu, Elan Rosenfeld, Pradeep Ravikumar, and Andrej Risteski. Analyzing and improving the optimization landscape of caiso-contrastive estimation. arXiv preprint arXiv:2110.11 117.11 2021.
Zhuang Ma and Michael Collins. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. arXiv preprint arXiv:1809.0188 01811, 2018.
Aditya Menon and Cheng Soon Ong. Linking losses for density ratio and class- probabbooyyylitt estimation. International Conference on Machine Learning, pp. 304 313..13. PMLR, 2016.
13

---

Published in Transactions on Machine Learning Research (03/2023)
Richard Nock, Aditya Menon, and Cheng Soon Ong. A scaled bregman theorem with applications. Advances in Neural Informalion Processing Systems, 29:19 27, 2016.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers us- ing variational divergence minimization. Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 271 279, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXi::107..0 03748, 2018.
Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron den Oord, Sergey Levine, and Pierre Sermanet. Wasserstein dependency measure for representation learning, 2019.
Ben Poole, Sherjil Ozair, Aaron van den Oord, Alexander A. Alemi, and George Tucker. On variational bounds of mutual information, 2019.
Joaquin Quidenero-Candela, Masashi Sugiyama, Neil D Lawrence, and Anton Schwaighofer. Dataset shift machine learning. Mit Press, 2009.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolu- tional generative adversarial networks. arXiv preprint arXiv:1511.0643/ 2015.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International con- ference on machine learning, pp. 1530- 1538. PMLR, 2015.
Benjamin Rhodes, Kai Xu, and Michael U Gutmann. Telescoping density-ratio estimation. arXiv preprint arXiv:2006. 1.XIV:2006.12044, 12204, 2020.
Akash Srivastava, Lazar Valkov, Chris Russell, Michael Gutmann, and Charles Sutton. Veegan: Reduc- ing mode collapse in gans using implicit variational learning. In Proceedings of the 31st International Conference Neural Information Processing Systems, pp. 3310-3320, 2017.
Akash Srivastava, Kai Xu, Michael Gutmann, and Charles Sutton. Generative ratio matching networks. In Eighth International Conference on Learning Representations, pp. 1-18, 2020.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Densily rulio eslimation in machine learning.
Cambridge University Press, 2012.
L. Wasserman. All of statistics. Springer, 2004.
A Consistency
In Section 3.1, we focused on properties of the loss function C(h,,..., .c)) in equation 3. The arguments of the loss functions were the functions hú, and the loss function was defined in terms of expectations over Pe- This simplified the analysis and provided important insights but does not correspond to practical settings. Herc, we relax the assumptions: We first consider the loss L(0) in equation 2 whore the functions he are parameterized by some parameters Then we consider the case where the expectations are replaced by a sample average based on n samples. The corresponding loss function will be denote by L.((). The main point of this section to derive conditions under which minimizing (0) leads to the results in Section 3.1, obtained by minimizing L(h1..... hhc)
Lemma A.1. Denoting the true conditional distribution by P*(Y(z)) we have
arg min L()) arg min E (w) KL (Y P(Y
(10)
where P(Y|2;; is defined in equation 1.
14

---

Published in Transactions on Machine Learning Research (03/2023)
Proof. We start with the definition of [(0) in equation 2:
((0) Σ B. Dog P(Y = c/2;; 0]]
(11)
The sum of weighted expectations Ees Berp. corresponds to joint expectation over p(Y,.). Decom- posing the joint as (X)t), we thus obtain
[( = - (p(x) log 2(Y
(12)
and
P* x))
[()) + Bp log (Y)))= = pptc) Ep log P(Y (x;
(13)
p. (PT P(Y|z;0)
(14)
The claim follows since the added term does not depend on 0.
If the true conditional P" P*(Yaz) is part of the parametric family {P(Y|a:; 0)},, 0 is thus such that P(Y|;;;) P* (Y(z)) for all where (=(x) > O. Hence the same arguments after equation 7 in the main text lead the parametric equivalent to equation 3.1, which we summarize in the following corollary.
Corollary A.2. If the true conditional P" (Y(z) is part of the parametric family {P(Y 0)3,, then
h1(x)) - 12 = =log P2(x))
(15)
Pj (x)
for all a where Px(x)= = Ee toPo(x) > 0.
We next derive conditions under which Ô the unique minimum, which is needed prove consistency. For that purpose, perform second-order Taylor expansion of L(0) around 0.
Lemma A.3.
C(0 + cΦ) = C(0) + 2 2
1
(16)
where 6 > O and 7= - ( [H(Y [H(Y,,) log-model, i.e. its (i.j)-th element is
The matric HY,x,) contains the second derivatives of the
0²
log P(Y [x; 8) H(Y,,))
(17)
00,00, where 0, and 0g are the i-th and j-th element of 0, respectively.
6=
Proof. A sedend-order Taylor expansion around L()) gives
C(O + eσ) = -Ep Σ PM = c|x) log P(Y = cco, + Eσ))
(18)
2 o0 H(Y = (xx)++0(6
=C(O) - VeL(o)
-ppa(x) PY = c|x)
(19)
A=b
2
-(( マ T
Epa(a)
P do) c.x) 6+0(E))
(20)
where we have used that the gradient of L(B) is zero a minimizer ô. Since Σ. P' (Y c Ep-(y H(, the result. follows.
15

---

& 0, and (tii) there is parameter value Published in Transactions on Machine Learning Research (03/2023)
Note that [(x) (Y) x)] is the conditional Fisher information matrix, and I (x) (x) [xx) is its expected value taken with respect to (z()).
Corollary A.4. If 1 is positive definite, then 0 the unique minimizer of (()).
Proof. If T is positive definite, then o whenever ø * 0.
Th To for all non-zero & and by Lemma A.3, L(O cω) > L())
We now consider the objective function (0) where the expectations in C()) are replaced by sample average over n samples. Lct d = arg ming (0).
Proposition A.5. If (1) I positive definite and (it) sups (0) [(0) 2 O, then 6, & 0.
Proof. By Corollary A.4, condition (i) ensures that 0 = arg min, C(0) is a unique minimizer, and hence that changing i by small amount will increase the cost function [()). Together with the technical condition (īi) on the uniform convergence of (0) to [()), this allows one prove that O, converges in probability to & as the sample size n increases, following exactly the same reasoning as e.g. proofs for consistency of maximum likelihood estimation (Wasserman, 2004, Section 9.13) noise-contrastive estimation (Gutmann & Hyvärinen, 2012, Appendix A.3.
Corollary A.6. If (1) I is positice definite, (ti) supø 0r such that Pa (Y(x) = pYY(; (x;)),; then 0n En l"
Proof. With Proposition A.5, condition (i) and (ii) ensure that On converges to 0 = arg ming L(0). With Lemma A.1, i is also minimizing Ex_ KL (P*(Y))) || P(Y],; Hence, if condition (ii) holds, 1 0" and the result follows.
]
Proposition A.7 (Consistency of the ratio estimator). (t) is positive definite, (ti sup, (0) 4. 0, (tit) Pa (Y p{{{2; 8*) for some parameter value 0", and (iv) the mapping from to h2 is continuous, then
h/, (x) l. (x) 2 log (x(x))
(21)
(z(x)
for all where Pa(x) -. TePo(x) 0.
Proof. By Proposition A.5, condition (i) and (ii) ensure that 0, converges to 0 = arg min, L(0). By Corollary
pi(a) for all where P=(x) = Σ. TePe(z) 0. Since (x) ((r)-Kij hg ) is continuous if condition continuous functions closed under addition, the mapping from 0 to hi
A.2, condition (iii) ensures that hz(()) (z) log py(x)
(iv) holds. We can then apply the continuous mapping theorem conclude that hi (x) h2 0.
(x) F
(x) hg(z)) log pa(a) which establishes the result. ()))
0
B Constructing M
We here elaborate on the three types of auxiliary distributions that we used this work.
Overlapping Distribution: The MDRE estimator, log = log _. - log is defined when << m and g <<< m. Therefore, m needs to be such that its support contains the supports of and q. Any distribution with full support such as the normal distribution trivially satisfics this requirement. However, satisfying this requirement does not guarantee empirical overlap of the distributions p, g with m in finite sample setting, In order to ensure overlap of samples between the two pairs of distributions recommend the following:
16

---

Published in Transactions on Machine Learning Research (03/2023)
Hcavy-tailed Distributions: Distributions such as, Cauchy and Student-t are better choice for M compared to the normal distribution. This is because their heavier tails allow for easily connecting and with higher sample overlap when they are far apart (especially in the case of FOD).
Mixtures: Another way to connect p and using such that they have their samples overlap, is to use the mixture distribution. Here, we first convolve p and with standard normal and then lake equal mixtures of the two.
Truncated Normal: If p and have finite support, one can also truncated normal distribution or uniform distribution that at least spans over the entire support of q. This is assuming that p <.
Linear Mixing: In this construction scheme, distribution M defined the empirical distribution of the samples constructed by lincarly combining samples X₂ { and X₂ [= from distributions p and 9 respectively. That is, m is the empirical distribution over the set Xm { |m + (1-a), E Xp.", E Xg Xg)) where a is not constrained to create a convex mixture. This construction is related to the linear combination auxiliary of Rhodes et al. (2020). In TRE, the auxiliary distribution is defined as the empirical distribution of the set Xm |< vI a²x; + azy Zp Xp. E Xg) where 0 S a S 1. This weighting scheme skews the samples from the auxiliary distribution towards p. Therefore, care needs to be taken when p and do not fall out of the support of q-
finite support distributions that the samples from the auxiliary distributions
Using either of the weighting schemes, one can construct K different auxiliary distributions. MDRE can either use these K auxiliary distributions separately using a K+2-way classifier or define a single mixture distribution using them as component distributions and train 3-way classifier. We refer this construction as Mixture of Linear Mixing.
Dimension-wise Mixing: In this construction scheme, that borrowed from TRE as is, M is defined the empirical distribution of the samples generated by combining different subsets of dimensions from samples from p and We describe the exact construction scheme from TRE below for completeness:
Given a d-length vector x and that is divisible by I, we can write down = (x[1),,..2[[[), where each x|i] has length d/l. Then, a sample from the kth auxiliary distribution is given by: #1. = (1) (0)l), Li), + 1), I,,, for j = I), where zi ~ p and xt are randomly paired.
C 1D density ratio estimation task
In Soction 4.1, we studied three cases in which the two distributions p and q are separated by both FOD and HOD. In all of these 1D experiments, all models trained with 100,000 samples, and all results are reported across 3 runs with different random seeds. Additionally, found that MDRE worked equally well for 1K and 10K samples. The experimental configurations, including the auxiliary distributions for MDRE, are detailed in Table 5.
p W-11, 0.08)
True KL
MDRE @ IK
MDRE 10K
MDRE 100K
(22, 0.15)
200.27
195.05 346.92
196.50 348.97
203.32
(-22, 0.08)
(22, 0.15)
355.82
360.35
Table 4: MDRE on 1D density ratio estimation for three settings of sample sizes. MDRE estimates the density ratio well for all the three settings.
17

---

Published in Transactions on Machine Learning Research (03/2023)
TRE pk
MDRE m
Linear Mixing with
= [0.053, 0.11,
0.16, 0.21, 0.26, 0.31, 0.37, 0.42, 0.47, 0.53, 0.58, 0.63, 0.68, 0.74,
0.-000.08) 02,10.
C(O,
0.79, 0.84, 0.89, 0.95)
Linear Mixing with a [0.03, 0.07, 0.1, 0.14, 0.17, 0.21, 0.24, 0.28, 0.31, 0.34, 0.38, 0.41, 0.45, 0.48, C(O, 0.52, 0.55, 0.59, 0.62,
(V-200088) (22.0.15)
0.66, 0.69, 0.72, 0.76, 0.79, 0.83, 0.86, 0.9, 0.93, 0.97] Linear Mixing with [0.11, 0.22, 0.33, 0.44, 0.55, 0.66, C(0,2)
10-10,
1)
0.77, 0.88)
Table Experiment configurations for Table 1 of main text and Table of Appendix C
BC-ORF
BC-DRE
400
F-ORF
TRE
TRE
F-DRE
200
Ground Truth
200
Ours
Log
0-
200
-200
-00
400
-600
-20
-15
-10
10
15
20
(b) Expanded domain: P = _V(-10,1) 1) and 8 =
(a) In-domain: p (0-10, and N(10,
1)
Figure 8: 1D density ratio estimation analysis. Figure 8a evaluates the log density ratios uniform samples inside the domain of the respective training distribution. Figure 8b evaluates the log density ratios on uniform samples from an expanded domain of the training distribution. The shading represents 1 standard deviation of the estimates.
D Uncertainty Quantification of MDRE Log-ratio Estimates with Hamiltonian Monte Carlo
In the 1D experiments, MDRE consistently led to highly accurate KL divergence estimates even in chal- lenging settings where state-of-the-art, methods fail. To understand why MDRE gives such accurate KL estimates, we conduct an analysis on the reliability of its log-ratio estimates by analyzing the distribution of the estimates
a Bayesian sctup, and study how it impacts the KI. divergence estimation. For this analysis, we use a classifier with the standard normal distribution as the prior on its parameters. The distri- bution of the log-ratio estimates is simply the distribution of the estimates from the classifiers with different
18

---

Published in Transactions on Machine Learning Research (03/2023)
400
true
100
ours (mean)
(Bol
300
-100
200
-200
100
0-
OUTS (mean)
ours (3o)
-500
(a) log for N (0000-01000..110000 N(1.0,
(b) log for p= N 10000,,00000.. =N(1000011
1750
600
1500
500
1250
veriante
400
1000
500
200
250
100
0-
-8
(c) Variance of log 2 function of distance from the modes of p and for N((-1000.1). ((1.0,0.2)
(d) Variance of log 2 as a function of distance from the modes of and q (.10,.0.1)
q
for p = N(-110,0.2), =
Figure 9: Uncertainty quantification for MDRE estimator. We plot the 3x standard deviation around the mean light blue. In plots (c) and (d) the bars show the means of p and
posterior parameters, which sampled. We consider two setups where first we set p == ==N(-110,0.) and = (1100,0.0 0.2) and then swap their scales, i.e. p V--100, 0.1) and g = (1100, 0.2). In both the cases, we draw samples from the posterior using an Hamiltonian Monte Carlo (HMC) sampler initialized by the maximum likelihood estimate of the classifier parameter. We then compute a set of samples of the log-ratio estimates from MDRE and estimate the mcan and standard deviation using these samples. Figure 9 (a) and (b) shows these results. We find that MDRE is accurate and manifests lowest uncertainty around the region between the means of p (-1.0) and q (+1.0). The uncertainty increases as we move away from the modes of distributions p and q- This is shown in plots (c) and (d), where we plot the variance of the estimates function of the location of the sample.
Since KL divergence is the expectation of the log-ratio on samples from and the high density region of p exactly matches the high confidence region MDRE, it is able to consistently estimate the KL divergence accurately even when p and q are far apart.
E
High Dimensional Experiment
In Section 4.2, we showed that MDRE performs better than all bascline models when and 9 are high dimensional Gaussian distributions. Prior work of Rhodes al. (2020) has considered high dimensional cases with HOD only, whereas we additionally consider cases with FOD and HOD to provide a more complete picture. Our results show that MDRE outperforms all other methods on the task of MI estimation the function of the estimated density ratio. It is worth noting that MDRE uses only upto 5 auxiliary distributions that are constructed using the linear mixing scheme and beats TRE substantially on cases with both FOD and HOD, although TRE uses upto 15 auxiliary distributions also constructed using linear mixing approach. This demonst rates that our proposal of using the multi-class logistic regression does, in fact, prevent distribution shifts issues of TRE when both FOD and HOD are present and help estimate the density ratio more accuratcly.
We now describe the MDRE configuration and other setup related details.
19

---

Published in Transactions on Machine Learning Research (03/2023)
GT LR vs Co8
120
GT KLD. 79.711716
Test Loss
Accuracy
140
100
Rist Loss
Accurscy
100
10.D
Log
80
100
15
m1
m2
m3
Log Ratio
Epoch
Distributions
Figure 10: Diagnostic plot for a high dimensional experiment.
180
160
GT LR vs Co8
Accuracy
10
160
140
Log Ratio Ca8
100
Accuracy
60
120
80
Test
20
Test Loss
80
100
100
10
5
q
m2 mi Distributions
Log Ratio
Epoch
Figure 11: Diagnostic plot for a high dimensional experiment with randomized means.
20

---

Published in Transactions on Machine Learning Research (03/2023)
Dim
MI
m using LM
20
[0.25,0.5,,
40
100
[0.35,0.5,.85)
40
[0.25,0.5,,775
160
136
0,115,0.35.0...775..
80
[0.25,0.5,,
320
240
(0.15,0.35,00,,,75..99
Table 6: Configuration of MDRE for the high dimensional experiments. LM stands for Linear Mixing
Auxiliary Distributions: For all the high dimensional experiments throughout this work, we construct m using the linear mixing scheme as described in Appendix B. Table 6 provides the mumber of auxiliary distributions along with the exact mixing weights for each of the 6 settings.
As a general principle, we chose these three sets of mixing weights so that their cumulative samples overlap with the samples of p and q similar to how the heavy tailed distribution worked in the 1D case. Please note that while heavy tailed distributions can effectively bridge p and when they have high FOD. However, they do not work as well if the discrepancy is primarily HOD. For example consider p = ~N(0, 1e 6) and q =N 1). In this case, setting m to a heavier tailed distribution centered as zero will not be of help. We need m that is concentrated at zero but also maintains a decent overlap with q. Linear mixing and q on the other hand, mixes first and higher order statistics (second or higher) and therefore, populates samples that overlap with both p and q. In some cases, we found that a mixture of linear mixing with K = 1 can also be used to estimate the density ratio. However, this requires using neural network- based classifier and requires much more tuning of the hyperparameters.
For choosing K, we use grid soarch based approach. We monitor the classification accuracy across all the K + distribution. If this accuracy is very high (> 95% for all classes), this implies that the classification task is easy and therefore the DRE may suffer from the density chasm issue. On the other hand, if the classification accuracy is too low (< 50% for all classes), then again, the DRE does not estimate well. We found that targeting an accuracy curve as shown in Figure 10 (last panel) empirically leads to accurate density ratio estimation. This curve plots the lest accuracy across all the classes and, empirically when it stays between the low and the high bounds of (50%,95%) the DRE estimates the ratios fairly well. The first panel shows that MDRE estimates the ground truth ratio accurately across samples from all the K +2 distributions, the second panel shows that KL estimates of MDRE is close to the ground truth KL and the third panel shows that both test and training losses have converged. Figure 11 shows another example for the case of randomized means. While MDRE also manages to get the ground truth KL correctly and most of the ratio estimates are also accurate, it does, however, slightly overestimate the log ratio for some of the samples from p-
F SpatialMultiOmmiglot Experiment
SpatialMultiOmniglol is a dataset of paired images u and v, where u is a X n grid of Omniglot characters from different Omniglot alphabets and v is a n x n grid containing the next characters of the corresponding characters in u. In this setup, we treat each grid of n X n as a collection of n² categorical random variables, not the individual pixels. The mutual information [u, v) can be computed as: [(u, v) = E log4, where 4 is the alphabet size for the tth character in u. This problem allows us to casily control the complexity of the task since increasing n increases the mutual information.
For the model, as in TRE, we use a separable architecture commonly used in MI-based representation learning literature and model the unnormalized log-scores with functions of the form g(u)T ['),,, where g and f are 14-layer convolutional ResNets (He et al., 2015). We construct the auxiliary distributions via dimension- wise mixing-exactly the way that TRE does.
21

---

Published in Transactions on Machine Learning Research (03/2023)
52.5
SORE with same enC
Mean label accuracy (teaz)
30.0
80
27.5
22.5
40
17.5
SDRE
SORE with same enc
15.0
characters
Number of characters
(a) Mutual information estimation
(b) Representation learning accuracy
Figure 12: SpatialMMuliOOmnngloot representation learning results with same encoder for f and g.
To evaluate the representations after learning, we adopt a standard linear evaluation protocol train a linear classifier on the output of the frozen encoder g(u) to predict the alphabetic index of each character in the grid
Additional Experiments
In addition the experiments in the main text, we run an additional experiment with SpatialMuuttiimnnghee to test the effect of using the same encoder for and f (i.e, modeling the unnormalized log-scores with the form g(u)"" W g(v)) instead of logp(u, v) w ((v).
Single Encoder Design: We test the contribution of using two different. encoders f and g instead of one. As seen in Figure 12, in both cases of d = 4,9, the two models reach slightly different but similar MI estimates, but, interestingly, do not differ at all in the test classification accuracy. Empirically, we also found that using one encoder helps the model converge to much faster. Overall, this experiment demonstrates that using two different encoders does not necessarily work to our advantage.
G TRE on Finite Support Distributions for K ==1
For K - 1, TRE proposes the following telescoping: logp/q = logp/m log l/g./q. As such, for TRE to be well defined, dQ i.e. the Radon-Nikodym Derivatives (RND) needs exist. The consequence of this is that TRE is only defined when p << << q. However this condition casily breaks if, for example, p and q are mixtures of finite support distributions except for the trivial case when support of m is exactly equal to the support of
We now demonstrate this with a specific example in Figure 13. Here we sot = 0.5 x TW(- low -1.1, -0.9) + 0.5 TW low = 0.9, high = 1.1) and g = 0.5 x TW 0.2, low = -1.2, high = +.5 TV 0.2, low 0.8, high = 1), as shown in Figure 13a where TA stands for Truncated Normal distribution. We set the auxiliary distribution m in TRE the proposed overlapping distribution construction. As such, p << 9 << and therefore, TRE is undefined for the second term as q is not defined for samples from m that are outside the support of q. It can be clearly seen in Figure 13b that m any support. Similar examples can be constructed for all the auxiliary distribution construction schemes proposed in Rhodes et al. (2020).
TV 1, low 1.2, high 1.2) using
a blows up to very high values on samples from m where does not have
Please note, despite dM/dQ being undefined, when used with the proposed m, TRE estimates log hggrp/ accurately on samples from p. We conjecture that this is because, the numerical estimation of the dM/dQ is finite over the support of p.
22

---

Published in Transactions on Machine Learning Research (03/2023)
(a) Setup
(b) dm/dQ
Tue mil
20
15
<ano
1000
05
0.0
1000
Samples
Figure 13: dM/dQ mixtures of distributions with finite support
23


Reviews:
[{"review_id": "Su8KbofivG", "reviewer": "Reviewer_jg6m", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n - The approach leverages some standard formulations and natural ideas and is easy to follow.\n - The experimental results showed significant differences between the performance of the new algorithm and existing approaches.\n - The proposed ratio estimator is consistent under some additional assumptions. \n\nWeaknesses:\n - The main contribution of the new approach is addressing the distribution-shift issue. But Section 3.3, titled \"Free from distribution-shift problems,\" mainly relies on some 1D examples. Hence there doesn't seem to be a concrete theoretical justification for the contribution. \n - The paper stated that the new method \"has better finite-sample (non-asymptotic) behavior than current state-of-the-art methods.\" Is this referring to empirical performance? The reviewer has not found much theoretical support for the method being better than SOTA. Similarly, another statement said that \"MDRE solves the aforementioned distribution shift issue.\" The word \"solve\" might be too strong here if the method only empirically addresses the issue in some examples. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of density ratio estimation and, in particular, tackles the density-chasm problem. \n\nThe proposed approach falls under telescopic ratio estimation methods that construct a sequence of auxiliary distributions to bridge the two targeted densities. A prior workstream called BDRE converts the original ratio estimation task into binary classification problems and recovers the desired ratio by the telescopic product of the neighboring distributions' ratio estimates. But as the authors showed experimentally, the BDRE methods can deviate significantly sometimes under train-eval distribution shifts. \n\nThe paper addresses the deviation issue by reforming the regressor as a multi-nomial logistic regression problem with a log-likelihood objective. The new approach imposes implicit regularization onto the final estimator and makes the estimator less sensitive to distribution shifts. The following experimental results and some specific examples further demonstrate the advantage of the new method in consistency and accuracy. ", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It would be great if the authors could address the abovementioned weaknesses. \n- A suggestion is to add more insights on why resolving the \"density-chasm problem\" is practically essential. ", "broader_impact_concerns": "The reviewer has not identified any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ofL0Y1FWOE", "reviewer": "Reviewer_58JZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n    - Very simple approach, very easy to reproduce\n    - Impressive results compared to state of the art\n    - Very important problem for many domains - density ratio estimation is at the heart of many research works in machine learning\n\nWeakness: \n    - Theoretical part not fully convincing (see next section)\n    - Reasons of benefits not clearly identified / discussed\n    - Positioning w.r.t. state of the art in calibration or density ratio estimation not sufficient from my point of view. \n    - More insights about the impact of the choice of auxiliary distribution would be useful", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper \"Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression\" proposes to deal with a well-known problem, called the chasm problem, for density ratio estimations, which arises when the two compared distributions have two strongly disjoint supports. The paper starts by pointing limitations of related work dealing with this problem, namely the distribution shift from which they suffer, and then demonstrates that the simple use of a cross-entropy loss for the discrimination of a set of auxiliary distributions (including source and target distributions) enable to obtain clearly better calibrated results than previous work.    ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "While I found the paper interesting and results promising for many applications, I am not convinced by the theoretical part. Since obviously $p_i(x)=p(x|y=i)=\\frac{p(x)p(y=i|x)}{\\pi_i}$, if we set an estimator $\\hat{p}(y=i|x) = \\frac{\\pi_i exp(h_i(x))}{\\sum_k \\pi_k exp(h_k(x))}$, we have $\\hat{p}_i(x)=\\frac{p(x) exp(h_i(x))}{\\sum_k \\pi_k exp(h_k(x))}$ and thus  $log \\frac{\\hat{p}_i(x)}{\\hat{p}_j(x)}=log \\frac{exp(h_i(x))}{exp(h_j(x))} = h_i(x) - h_j(x)$. Hence,  $h_i(x) - h_j(x)$ is an estimator of the log density-ratio. This justifies the use of this quantity to estimate the ratio in the approach. However, the derivation with optimizers  $\\hat{h}_i(x)$ to get (6) and (7) does not look correct to me, because this should not be taken for individual x samples but in expectation: optimization is performed on (5) with expectations leading to $\\frac{\\partial L}{\\partial h_i}= - \\pi_i + \\int p_x(x) \\frac{\\pi_i exp(h_i(x))}{\\sum \\pi_k exp(h_k(x))}$ which does not allow to conclude in (7) for all x when cancelling it. Could authors elaborate on this ? And also give more insights about why this derivation is needed (rather than simply giving obvious equations that I give at the beginning of this paragraph) ? \n\nMore importantly, while it could explain the absence of distribution shift, it gives no insights about why using auxiliary distributions in that case would allow to deal with the chasm problem. While this is quite intuitive for TRE from its construction based on a product of auxiliary ratios, for me the reason why the MDRE proposal obtains interesting results lies in the fact that all h share parameters which induces some regularization allowing better estimation. Please discuss further on the benefits of the approach over a classical BDRE approach. \n\nAnother point that I would be happy to see further discuss is the impact of the choice of the used auxiliary distributions on the results. Is it enough to only cover the full support of p and q? Is a single auxiliary enough or is it better to consider multiple distributions ? Does it have a correlation with the freedom of the network h ?  \n\nAs the paper is very related to the calibration field, It would be useful to further discuss the positioning w.r.t other approaches for this task, which go far beyond the use of auxiliary distributions.\n\nI also feel it would be useful to discuss perspectives of applications  of that work in domains considering complex data structures as sequences. For instance, density ratios are of crucial importance in imitation learning which induces sequential data. Would it be applicable in such domains ?       \n\nMinor remarks: \n   -Although it does not change anything in the following, (6) should be multiplied by (-1) to correspond to a derivative. \n   - Please indicate the axes of panels b and d of figure 1", "broader_impact_concerns": "No concern about ethical implications of the work to be mentioned. ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2jCgMOPfj3", "reviewer": "Reviewer_775Q", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- sound approach with proper theoretical underpinning\n- proper empirical evaluation\n\nWeakness:\n- it is unclear how severe the density-chasm problem is in practice\n- paper only hints at proving that the proposed estimator is consistent for finite samples", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper tackles the density-chasm problem in density ratio estimation via logistic regression. The paper first shows that the existing approach to this problem, i.e., telescopic density-ratio estimation that uses the telescopic product of logistic regressors trained on overlapping auxiliary distributions to bridge the gap between target distributions, can be inaccurate, since a single inaccurate estimator in the product deteriorates the entire accuracy. The paper instead proposes to train a single multinomial logistic regressor on auxiliary distributions. The paper shows that in the infinite data regime this estimator is accurate and empirically estimates the estimator on a set of synthetic datasets as well as for MI estimation on the SpatialMultiOmniglot problem.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- please show that the density-chasm problem has practical relevance, e.g., by showing how it can deteriorate representation earning.\n- I think the paper would benefit greatly from proving that the proposed estimator is consistent.", "broader_impact_concerns": "none", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3bGpRAPDSJ", "reviewer": "Reviewer_yAq2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper makes a compelling case for a flaw in an existing method and proposes an elegant approach to fix it, with good empirical results. I don't really have anything to complain about.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The proposed work addresses the problem of estimating density ratios, which is relevant for a variety of tasks in machine learning and statistics, most notably the estimation of f-divergences. \nA common approach is to use the output of a logistic regression classifier. However, these are known to perform poorly on well-separated distributions with extreme density ratios. In order to overcome this \"density chasm\", prior work has introduced (TRE) that uses intermediate distributions $m_{i}$ that allow an estimation as $\\frac{p}{q} = \\frac{p}{m_1}\\frac{m_1}{m_2} \\dots \\frac{m_{K}}{q}$. Here, the intermediate distributions $\\{m_i \\}_{1 \\leq i \\leq K}$ are constructed to serve as a bridge from $p$ to $q$, with consecutive distributions strongly overlapping. \n\nThe proposed work observes that TRE's performance suffers from the \"distribution shift\" due to the fact that the density estimation problem for $\\frac{m_{i}}{m_{i + 1}}$ is trained on data from $m_{i}$ and $m_{i + 1}$ but evaluated on data from only $p$ and $q$. To overcome this problem, the authors propose to obtain estimates for the density ratio from the $K+2$-class classification problem involving samples from all the $m_{i}$ and $p,q$.  ", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In my opinion, the paper could be published as is.", "broader_impact_concerns": "No concerns ", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors present a novel way to estimate the density ratio under distribution shifts. The key idea is to introduce a set of $K$ auxiliary densities, and leverage multi-class classification for density ratio estimation. Specifically, they show that if these auxiliary densities overlap with distributions $p$ and $q$, then a multi-class logistic regression can estimate $\log p/q$ on the domain of any of the $K + 2$ distributions. Finally, the authors demonstrate superior performance of their method in the experiments.

The idea of using multi-class logistic regression to estimate the density ratio is novel and interesting. The algorithm is very simple and works well in the experiments. The authors have made a significant contribution to density ratio estimation, and all the reviewers are positive with this paper. When preparing the camera-ready version, the authors should take all the discussions into consideration.

Decision: Accept (poster)