Scaling Densities For Improved Density Ratio Estimation

Estimating the discrepancy between two densities ( p and q ) is central to machine learning . Most frequently used methods for the quantification of this discrepancy capture it as a function of the ratio of the densities p/q . In practice , closed-form expressions for these densities or their ratio are rarely available . As such , estimating density ratios accurately using only samples from p and q is of high significance and has led to a flurry of recent work in this direction . Among these , binary classification based density ratio estimators have shown great promise and have been extremely successful in specialized domains . However , estimating the density ratio using a binary classifier , when the samples from the densities are well separated , remains challenging . In this work , we first show that the state-of-the-art solutions for such well separated cases have limited applicability , may suffer from theoretical inconsistencies or lack formal guarantees and therefore perform poorly in the general case . We then present an alternative framework for density ratio estimation that is motivated by the scaled-Bregman divergence . Our proposal is to scale the densities p and q by another densitym and estimate log p/q as log p/m− log q/m . We show that if the scaling measures are constructed such that they overlap with p and q , then a single multi-class logistic regression can be trained to accurately recover p/m and q/m on samples from p , q and m. We formally justify our method with the scaled-Bregman theorem and show that it does not suffer from the issues that plague the existing solutions . We provide a large battery of empirical evaluations of our method with both synthetic and real datasets on the tasks of density ratio estimation , mutual information estimation , and representation learning . Finally , we demonstrate that our method can be applied to improve non-classification based model-free density ratio estimators as well . 1 INTRODUCTION Quantification of discrepancy between two distributions underpins the foundation of a large number of machine learning techniques . Of especial prominence are distribution discrepancy measures known as f - divergences ( Csiszár , 1964 ) , which are defined as expectations of the functions of the ratio of the two densities . As such , density ratio estimation is often a central task in deep generative modeling , mutual information and divergence estimation , representation learning , etc . ( Sugiyama et al. , 2012 ; Gutmann & Hyvärinen , 2010 ; Goodfellow et al. , 2014 ; Nowozin et al. , 2016 ; Srivastava et al. , 2017 ; Belghazi et al. , 2018 ; Oord et al. , 2018 ; Srivastava et al. , 2020 ) . However , estimating density ratio by modeling each of the densities is challenging for most problems of interest as modeling high dimensional densities is a harder problem than estimating their ratios Sugiyama et al . ( 2012 ) . Therefore , in practice , estimators are employed that only require samples from the pair of densities in order to estimate their ratio without explicitly modeling the individual densities . One of the most commonly used density ratio estimators ( DRE ) is a binary classifier based DRE ( BC-DRE ) which can be formally shown to estimate the ground truth density ratio when trained to correctly discriminate be- tween the samples from the two densities ( e.g . Gutmann & Hyvärinen , 2010 ; Gutmann & Hirayama , 2011 ; Sugiyama et al. , 2012 ; Menon & Ong , 2016 ) . BC-DREs have been tremendously successful in problems involving minimization of the discrepancy between the data and the model distributions even in high-dimensional settings ( Nowozin et al. , 2016 ; Radford et al. , 2015 ) . However , they do not fare as well when applied to the task of estimating the discrepancy between two distributions that are far apart or easily separable otherwise . This issue has been characterized recently as the density-chasm problem by Rhodes et al . ( 2020 ) . The discrepancy between two distributions can manifest itself in many different ways . Here , we roughly group these ways into two classes , First Order Discrepancy ( FOD ) and Higher Order Discrepancy ( HOD ) . Two distributions can easily be separated when the first order statistics , such as their means , are far apart from each other . We refer to this type of discrepancy between p and q as FOD . Similarly , if the discrepancy between p and q ( with no FOD ) originates due to the difference in higher order statistics we refer to this as HOD . For example , consider Figure 1 where in all the three cases , the KL [ p‖q ] = 50 , but in ( a ) it originates from FOD since N ( µ1 = −5 , σ1 = 1 ) , N ( µ1 = 5 , σ1 = 1 ) , in ( b ) it originates from HOD since N ( µ1 = 0 , σ1 = 1 ) , N ( µ2 = 0 , σ1 = 0.097 ) , and in ( c ) it originates from FOD and HOD since N ( µ1 = −1.5 , σ1 = 0.5 ) , N ( µ2 = 1 , σ1 = 0.25 ) . In a recent work , Choi et al . ( 2021 ) attempt to address the density-chasm problem . They considered distributions p and q that are clearly separated and proposed that in such cases density ratios can be better estimated if the mixture of the two densities is mapped onto a unit-sphere using a flow-based map ( Rezende & Mohamed , 2015 ) . Using a bijective map ensures that the density ratio between the original distributions is maintained post mapping . While this method ( F-DRE ) may resolve the density-chasm problem due to FOD , we found that it struggles when the densities are separated due to HOD . On the other hand , Rhodes et al . ( 2020 ) proposed to estimate the density ratio between p and q as p p1 × p1p2 × · · · × pK q . This telescoping is achieved by training K BC-DREs where the distributions p1 , . . . , pk are designed to be closely-packed such that all of the K ratios can be easily estimated using BC-DREs without suffering from the density-chasm problem . They call this estimator , telescoping density ratio estimator or TRE . Empirically , we found that while TRE performs relatively better than BC-DRE and F-DRE in cases with density-chasm problem due to HOD , it can not deal with the density-chasm problem originating due to FOD . Intuitively , this happens because TRE trains K DREs , each of which is trained on K different pairs of distributions . This leads to the distribution shift issue at test time , especially when p and q have high FOD . Further , care needs to be taken when designing intermediate distributions p1 , . . . , pK to ensure that each of the ratios are well defined . In fact , the construction schemes introduced in the original work may render TRE theoretically ill-defined if both p and q do not have full support ( Appendix A ) . We will show that neither of these state-of-the-art methods are applicable when the density ratio estimation problem involves both FOD and HOD . Unfortunately , most problems in practice tend to have both types of discrepancy . This leaves a clear technical gap in the task of density ratio estimation . Hence , in this work , we present SCALED DENSITY RATIO ESTIMATOR ( SDRE ) , a novel method for estimating density ratio that : 1 . Theoretically well-defined and does not suffer from distribution shift . 2 . Resolves both FOD and HOD types of density-chasm issues . 3 . Does not require training additional FLOW-based models or BC-DREs and therefore is computationally more efficient . 2 NOTATION AND BACKGROUND . Bregman distance between two d-dimensional vectors a and b is defined as Bφ ( a , b ) = φ ( a ) − φ ( b ) −∇φ ( b ) ( a− b ) . Here φ : Rd 7→ R is a convex function . Following the notation in Stummer & Vajda ( 2012 ) the scaled-Bregman Divergence ( sBD ) between probability measures P and Q scaled by an arbitrary measure M can be defined as , Bf ( P , Q|M ) = ∫ X m [ f ( p m ) − f ( q m ) − f ′ ( q m ) ( p m − q m ) ] dλ . ( 1 ) Here , f : X → R is a convex function , f ′ is its right derivative . Further , all the measures P , Q and M are defined on the measurable space ( X , A ) and are absolutely continuous with respect to a σ-finite measure λ , therefore admitting the following densities , p = dP dλ , q = dQ dλ , m = dM dλ . ( 2 ) It is easy to show that for f ( t ) = t log ( t ) , sBD reduces to KL divergence and is independent of the scaling measure M and λ , Bt log t ( P , Q|M ) = ∫ X m [ p m log ( p m ) − p m log ( q m ) ] dλ = KL ( P‖Q ) . ( 3 ) We define rp/m ( x ) = p ( x ) m ( x ) , rq/m ( x ) = q ( x ) m ( x ) , rp/q ( x ) = p ( x ) q ( x ) ( 4 ) as the density ratios between p & m , q & m and p & q respectively . Given samples { xip } Ni=1 and { xjq } Mj=1 from distributions p and q respectively and assuming p < < q , the goal of DRE r̂p/q is to estimate rp/q . 2.1 BINARY CLASSIFICATION TO DRE USING BD . Proposition 3 ( Menon & Ong , 2016 ) formally establishes the link between binary classification and density ratio estimation using sBD . We re-state it here . Proposition 1 ( Proposition 3 ( Menon & Ong , 2016 ) rephrased ) Given a class-probability estimator η̂ : X 7→ [ 0 , 1 ] such that the density ratio estimator , r̂p/q ( x ) = η̂1−η̂ . Then for any convex differentiable f : [ 0 , 1 ] 7→ R , Ex∼pmarginal [ Df ( η ( X ) ‖η̂ ( X ) ] = 1 2 Ex∼q [ Df† ( rp/q ( X ) ‖r̂p/q ( X ) ) ] ( 5 ) where Df is Bregman Divergence ( BD ) , f† : [ 0 , ∞ ) 7→ R | f† ( x ) = ( 1 + x ) f ( x1+x ) , Df† is sBD and pmarginal is the marginal distribution of X . 2.2 DENSITY CHASM . Kato & Teshima ( 2021 ) note that most empirical estimators of density ratio , especially those implemented using a deep neural network tend to over-fit the loss function in some way or the other . This phenomenon was recently characterized by Rhodes et al . ( 2020 ) for the BC-DRE as the density chasm problem . They found that the BC-DRE tends to substantially underestimate the ground-truth KL-divergence ( as a function of the estimated ratio ) between p and q when these distributions are significantly separated from each other . We conjecture that this underestimation occurs because in the case of finite training data , when the two sets of samples are easily separable , the estimated ratios do not need to be as large as they theoretically had to be to achieve perfect classification performance . In such a case , many different decision boundaries achieve equally good performance .

Summary: Given samples from two distirbutions, how can we estimate the KL divergence between them? More generally, how can we estimate the ratio of the two densities at given points (e.g, on the samples from one of the dists)? The authors propose a novel method for density ratio estimation (DRE). DRE is challenging when the two distributions have different supports or the density ratio is very high. The authors propose a solution that builds on the previous work, namely Rhodes et al..,  which instead of computing the ratio directly, it writes it as multiplication of a series of density ratios between some intermediate distributions (telescoping).   The authors improve over Rhodes et al. by accounting for the distribution shift in a systematic way and also slightly changing the way the intermediate distributions are constructed. The empirical results are promising and show the benefit of the approach compared to the baselines.