Scaling Densities For Improved Density Ratio Estimation

Estimating the discrepancy between two densities ( p and q ) is central to machine learning . Most frequently used methods for the quantification of this discrepancy capture it as a function of the ratio of the densities p/q . In practice , closed-form expressions for these densities or their ratio are rarely available . As such , estimating density ratios accurately using only samples from p and q is of high significance and has led to a flurry of recent work in this direction . Among these , binary classification based density ratio estimators have shown great promise and have been extremely successful in specialized domains . However , estimating the density ratio using a binary classifier , when the samples from the densities are well separated , remains challenging . In this work , we first show that the state-of-the-art solutions for such well separated cases have limited applicability , may suffer from theoretical inconsistencies or lack formal guarantees and therefore perform poorly in the general case . We then present an alternative framework for density ratio estimation that is motivated by the scaled-Bregman divergence . Our proposal is to scale the densities p and q by another densitym and estimate log p/q as log p/m− log q/m . We show that if the scaling measures are constructed such that they overlap with p and q , then a single multi-class logistic regression can be trained to accurately recover p/m and q/m on samples from p , q and m. We formally justify our method with the scaled-Bregman theorem and show that it does not suffer from the issues that plague the existing solutions . We provide a large battery of empirical evaluations of our method with both synthetic and real datasets on the tasks of density ratio estimation , mutual information estimation , and representation learning . Finally , we demonstrate that our method can be applied to improve non-classification based model-free density ratio estimators as well . 1 INTRODUCTION Quantification of discrepancy between two distributions underpins the foundation of a large number of machine learning techniques . Of especial prominence are distribution discrepancy measures known as f - divergences ( Csiszár , 1964 ) , which are defined as expectations of the functions of the ratio of the two densities . As such , density ratio estimation is often a central task in deep generative modeling , mutual information and divergence estimation , representation learning , etc . ( Sugiyama et al. , 2012 ; Gutmann & Hyvärinen , 2010 ; Goodfellow et al. , 2014 ; Nowozin et al. , 2016 ; Srivastava et al. , 2017 ; Belghazi et al. , 2018 ; Oord et al. , 2018 ; Srivastava et al. , 2020 ) . However , estimating density ratio by modeling each of the densities is challenging for most problems of interest as modeling high dimensional densities is a harder problem than estimating their ratios Sugiyama et al . ( 2012 ) . Therefore , in practice , estimators are employed that only require samples from the pair of densities in order to estimate their ratio without explicitly modeling the individual densities . One of the most commonly used density ratio estimators ( DRE ) is a binary classifier based DRE ( BC-DRE ) which can be formally shown to estimate the ground truth density ratio when trained to correctly discriminate be- tween the samples from the two densities ( e.g . Gutmann & Hyvärinen , 2010 ; Gutmann & Hirayama , 2011 ; Sugiyama et al. , 2012 ; Menon & Ong , 2016 ) . BC-DREs have been tremendously successful in problems involving minimization of the discrepancy between the data and the model distributions even in high-dimensional settings ( Nowozin et al. , 2016 ; Radford et al. , 2015 ) . However , they do not fare as well when applied to the task of estimating the discrepancy between two distributions that are far apart or easily separable otherwise . This issue has been characterized recently as the density-chasm problem by Rhodes et al . ( 2020 ) . The discrepancy between two distributions can manifest itself in many different ways . Here , we roughly group these ways into two classes , First Order Discrepancy ( FOD ) and Higher Order Discrepancy ( HOD ) . Two distributions can easily be separated when the first order statistics , such as their means , are far apart from each other . We refer to this type of discrepancy between p and q as FOD . Similarly , if the discrepancy between p and q ( with no FOD ) originates due to the difference in higher order statistics we refer to this as HOD . For example , consider Figure 1 where in all the three cases , the KL [ p‖q ] = 50 , but in ( a ) it originates from FOD since N ( µ1 = −5 , σ1 = 1 ) , N ( µ1 = 5 , σ1 = 1 ) , in ( b ) it originates from HOD since N ( µ1 = 0 , σ1 = 1 ) , N ( µ2 = 0 , σ1 = 0.097 ) , and in ( c ) it originates from FOD and HOD since N ( µ1 = −1.5 , σ1 = 0.5 ) , N ( µ2 = 1 , σ1 = 0.25 ) . In a recent work , Choi et al . ( 2021 ) attempt to address the density-chasm problem . They considered distributions p and q that are clearly separated and proposed that in such cases density ratios can be better estimated if the mixture of the two densities is mapped onto a unit-sphere using a flow-based map ( Rezende & Mohamed , 2015 ) . Using a bijective map ensures that the density ratio between the original distributions is maintained post mapping . While this method ( F-DRE ) may resolve the density-chasm problem due to FOD , we found that it struggles when the densities are separated due to HOD . On the other hand , Rhodes et al . ( 2020 ) proposed to estimate the density ratio between p and q as p p1 × p1p2 × · · · × pK q . This telescoping is achieved by training K BC-DREs where the distributions p1 , . . . , pk are designed to be closely-packed such that all of the K ratios can be easily estimated using BC-DREs without suffering from the density-chasm problem . They call this estimator , telescoping density ratio estimator or TRE . Empirically , we found that while TRE performs relatively better than BC-DRE and F-DRE in cases with density-chasm problem due to HOD , it can not deal with the density-chasm problem originating due to FOD . Intuitively , this happens because TRE trains K DREs , each of which is trained on K different pairs of distributions . This leads to the distribution shift issue at test time , especially when p and q have high FOD . Further , care needs to be taken when designing intermediate distributions p1 , . . . , pK to ensure that each of the ratios are well defined . In fact , the construction schemes introduced in the original work may render TRE theoretically ill-defined if both p and q do not have full support ( Appendix A ) . We will show that neither of these state-of-the-art methods are applicable when the density ratio estimation problem involves both FOD and HOD . Unfortunately , most problems in practice tend to have both types of discrepancy . This leaves a clear technical gap in the task of density ratio estimation . Hence , in this work , we present SCALED DENSITY RATIO ESTIMATOR ( SDRE ) , a novel method for estimating density ratio that : 1 . Theoretically well-defined and does not suffer from distribution shift . 2 . Resolves both FOD and HOD types of density-chasm issues . 3 . Does not require training additional FLOW-based models or BC-DREs and therefore is computationally more efficient . 2 NOTATION AND BACKGROUND . Bregman distance between two d-dimensional vectors a and b is defined as Bφ ( a , b ) = φ ( a ) − φ ( b ) −∇φ ( b ) ( a− b ) . Here φ : Rd 7→ R is a convex function . Following the notation in Stummer & Vajda ( 2012 ) the scaled-Bregman Divergence ( sBD ) between probability measures P and Q scaled by an arbitrary measure M can be defined as , Bf ( P , Q|M ) = ∫ X m [ f ( p m ) − f ( q m ) − f ′ ( q m ) ( p m − q m ) ] dλ . ( 1 ) Here , f : X → R is a convex function , f ′ is its right derivative . Further , all the measures P , Q and M are defined on the measurable space ( X , A ) and are absolutely continuous with respect to a σ-finite measure λ , therefore admitting the following densities , p = dP dλ , q = dQ dλ , m = dM dλ . ( 2 ) It is easy to show that for f ( t ) = t log ( t ) , sBD reduces to KL divergence and is independent of the scaling measure M and λ , Bt log t ( P , Q|M ) = ∫ X m [ p m log ( p m ) − p m log ( q m ) ] dλ = KL ( P‖Q ) . ( 3 ) We define rp/m ( x ) = p ( x ) m ( x ) , rq/m ( x ) = q ( x ) m ( x ) , rp/q ( x ) = p ( x ) q ( x ) ( 4 ) as the density ratios between p & m , q & m and p & q respectively . Given samples { xip } Ni=1 and { xjq } Mj=1 from distributions p and q respectively and assuming p < < q , the goal of DRE r̂p/q is to estimate rp/q . 2.1 BINARY CLASSIFICATION TO DRE USING BD . Proposition 3 ( Menon & Ong , 2016 ) formally establishes the link between binary classification and density ratio estimation using sBD . We re-state it here . Proposition 1 ( Proposition 3 ( Menon & Ong , 2016 ) rephrased ) Given a class-probability estimator η̂ : X 7→ [ 0 , 1 ] such that the density ratio estimator , r̂p/q ( x ) = η̂1−η̂ . Then for any convex differentiable f : [ 0 , 1 ] 7→ R , Ex∼pmarginal [ Df ( η ( X ) ‖η̂ ( X ) ] = 1 2 Ex∼q [ Df† ( rp/q ( X ) ‖r̂p/q ( X ) ) ] ( 5 ) where Df is Bregman Divergence ( BD ) , f† : [ 0 , ∞ ) 7→ R | f† ( x ) = ( 1 + x ) f ( x1+x ) , Df† is sBD and pmarginal is the marginal distribution of X . 2.2 DENSITY CHASM . Kato & Teshima ( 2021 ) note that most empirical estimators of density ratio , especially those implemented using a deep neural network tend to over-fit the loss function in some way or the other . This phenomenon was recently characterized by Rhodes et al . ( 2020 ) for the BC-DRE as the density chasm problem . They found that the BC-DRE tends to substantially underestimate the ground-truth KL-divergence ( as a function of the estimated ratio ) between p and q when these distributions are significantly separated from each other . We conjecture that this underestimation occurs because in the case of finite training data , when the two sets of samples are easily separable , the estimated ratios do not need to be as large as they theoretically had to be to achieve perfect classification performance . In such a case , many different decision boundaries achieve equally good performance .

Summary: This work estimates the ratio of two densities using intermediate densities that have sufficient overlap with the target densities. The proposed method is based on the scaled-Bregman Divergence and it can be equivalently formulated as a multi-class logistic regression problem. By combining the idea of telescoping, the proposed approach mitigate the issue of FOD/HOD density-chasm and show superior performance on a variety of datasets.