Meta-Learning for Relative Density-Ratio Estimation

1 Introduction . The ratio of two probability densities , called a density-ratio , has been used in various applications such as outlier detection [ 14 , 1 ] , dataset comparison [ 49 ] , covariate shift adaptation [ 42 ] , change point detection [ 30 ] , positive and unlabeled ( PU ) learning [ 19 , 18 ] , density estimation [ 46 ] , and generative adversarial networks [ 47 ] . Thus , density-ratio estimation ( DRE ) is attracting a lot of attention . A naive approach to DRE is to estimate each density and then take the ratio . However , this approach does not work well since density estimation is a hard problem [ 48 ] . Therefore , direct DRE without going through density estimation has been extensively studied [ 44 , 17 , 35 , 13 ] . Although direct DRE is useful , its fundamental weakness is that the density-ratio is unbounded , i.e. , it can take infinity , which causes stability issues [ 29 ] . To cope with this problem , a relative density-ratio has been proposed , which is a smoothed and bounded extension of the density-ratio [ 49 ] . In the above applications , the density-ratio can be replaced with the relative density-ratio , and relative DRE has shown excellent performance [ 49 , 30 , 6 , 37 , 47 ] . Existing methods for ( relative ) DRE require many instances from both densities . However , sufficient instances are often unavailable for various reasons . For example , it is difficult to instantly collect many instances from new data sources such as new users or new systems . Collecting instances is 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . expensive in some applications such as clinical trials or crash tests , where DRE can be used for dataset comparison to investigate the effect of drugs/car conditions . In such cases , existing methods can not work well . αpA ( x ) + ( 1−α ) pB ( x ) , ( 0 ≤ α < 1 ) with tar- get datasets A and B that are generated from densities pA ( x ) and pB ( x ) , respectively . In this paper , we propose a meta-learning method for relative DRE . To estimate the relative density-ratio from a few instances in target datasets , the proposed method uses instances in different but related datasets , called source datasets . When these datasets are related , we can transfer useful knowledge from source datasets to target ones [ 31 ] . Figure 1 shows our problem formulation . We model the relative density-ratio by using neural networks that enable us to perform accurate DRE thanks to their high expressive capabilities . Since each dataset has a different property , incorporating it to the model is essential . To achieve this , given two datasets that consist of a few instances , called support instances , our model first calculates a latent vector representation of each dataset . This vector is calculated by using permutation-invariant neural networks that can take a set of instances as input [ 50 ] . Since the vector is obtained from a set of instances in the dataset , it contains information of the dataset . With the two latent vectors of datasets , each instance is non-linearly mapped to an embedding space that is suitable for relative DRE on the datasets . Using the embedded instances , we perform relative DRE , where the relative density-ratio is represented by a linear model on the embedded space . With the squared loss , the global optimal solution of the linear model can be obtained as a closed-form solution , which enables us to perform more stable and faster adaptation to support instances than numerical solutions . The neural networks of our model are trained by minimizing the expected test squared error of relative DRE after adapting to support instances that is calculated using instances in the source datasets . Since the closed-form solution of the linear model is differentiable , this training can be performed by gradient-based methods such as ADAM [ 20 ] . Since all parameters of our model are shared across all datasets , which enables knowledge to be shared between all datasets , the learned model can be applied to unseen target datasets . This training explicitly improves the relative DRE performance for test instances after estimating the relative density-ratio using support instances . Thus , the learned model can accurately estimate the relative density-ratio from a few instances . Our main contributions are as follows : ( 1 ) To the best of knowledge , our work is the first attempt at meta-learning for ( relative ) DRE . ( 2 ) We propose a model that performs accurate relative DRE from a few instances by effectively adapting both embeddings and linear model to the instances . ( 3 ) We empirically demonstrate the effectiveness of the proposed method with three popular problems : relative DRE , dataset comparison , and outlier detection . 2 Related Work . Many methods for direct DRE have been proposed such as classifier-based methods [ 4 , 33 ] , Kullback–Leibler importance estimation [ 43 ] , kernel mean matching [ 13 ] , and unconstrained least-squares importance fitting ( uLSIF ) [ 17 ] . Although these methods are useful , they can suffer from the unbounded nature of the density-ratio . That is , an instance that is in the low density region of the denominator density may have an extremely large value of ratio , and DRE can be dominated by such points , which causes robustness and stability issues [ 29 ] . This problem is particular serious when using flexible density-ratio models such as neural networks because they try to fit on extremely large density-ratio values [ 18 ] . To cope with the unboundedness , the relative uLSIF ( RuLSIF ) uses the relative density-ratio to uLSIF [ 49 ] . Since RuLSIF can obtain the optimal parameters as a closed-form solution , it is computationally efficient and stable and performs well in various applications when sufficient instances are available [ 49 , 30 , 6 , 37 , 47 ] . Since fast and effective adaptation to support in- stances is essential in meta-learning as explained in the end of this section , we incorporate RuLSIF in our framework . Although neural network-based DRE methods have been recently proposed [ 18 , 35 ] , they can not perform well when training instances are quite small due to overfitting . Although the proposed method uses neural networks for the instance embeddings , it can accurately perform relative DRE from a few instances by learning how to perform few-shot relative DRE with related datasets . DRE is used for transfer learning or covariate shift adaptation [ 40 , 42 , 45 , 22 , 7 ] . To transfer knowledge in a training dataset to a test dataset , these methods estimate the density-ratio between training and test densities that is used for weighting labeled training instances . These methods use only two datasets to estimate the density-ratio . In contrast , the proposed method uses multiple datasets to accumulate transferable knowledge and uses it for the relative DRE on two new datasets , which can be used in various applications including covariate shift adaptation as described in Section 1 . Meta-learning methods have been recently attracting a lot of attention [ 8 , 41 , 9 , 34 , 3 , 16 ] . These methods train a model such that it generalizes well after adapting to support instances using multiple datasets . In this framework , fast adaptation to support instances is essential since the result of the adaptation is required to train the model in each iteration of training [ 3 , 34 ] . Encoder-decoder methods such as neural processes [ 9 , 10 ] perform quick adaptation by forwarding support instances to neural networks . However , they have difficulty working well for any dataset since the adaptation is approximated by only the neural networks . Gradient-based methods such as model-agnostic metalearning ( MAML ) [ 8 ] adapt to support instances by using an iterative gradient descent method and are widely used . These methods require higher-order derivatives and to retain all optimization path of the iterative adaptation to backpropagate through the path , which imposes considerable computational and memory burdens [ 3 ] . Thus , they must keep the iteration number small and it prevents effective adaptation . In contrast , the proposed method quickly and effectively adapt to support instances by solving a convex optimization problem , where the global optimum solution can be quickly obtained as a closed-form solution . Although few methods adapt to support instances by solving convex optimization problems for fast and effective adaptation [ 3 , 26 ] , they consider classification tasks . To the best of our knowledge , no meta-learning methods have been designed for DRE , and thus , existing meta-learning methods can not be applied to our problems . 3 Preliminary . We briefly explain a relative density-ratio . Suppose that instances { xn } Nn=1 are drawn from a distribution with density pnu ( x ) and instances { x′n } N ′ n=1 are drawn from another distribution with density pde ( x ) . Density ratio r ( x ) is defined by r ( x ) : = pnu ( x ) pde ( x ) . Here , “ nu ” and “ de ” indicate the numerator and the denominator . The aim of DRE is to directly estimate r ( x ) from both instances { xn } Nn=1 and { x′n } N ′ n=1 . However , r ( x ) is unbounded and thus can take extremely large values when the denominator pde ( x ) takes a small value . This causes robustness and stability issues [ 29 ] . To deal with this problem , a relative density-ratio has been proposed [ 49 ] . For 0 ≤ α < 1 , relative densityratio rα ( x ) is define by rα ( x ) : = pnu ( x ) αpnu ( x ) + ( 1−α ) pde ( x ) . Relative density-ratio rα ( x ) is bounded since rα ( x ) ≤ 1α for any x. rα ( x ) is always smoother than r ( x ) . rα ( x ) can replace r ( x ) and is used for various applications [ 49 , 30 , 6 , 37 , 47 ] . When α = 0 , rα ( x ) is reduced to density-ratio r ( x ) . Thus , the relative density-ratio can be regarded as a smoothed and bounded extension of the density-ratio . 4 Proposed Method . 4.1 Problem Formulation . Let Xd = { xdn } Ndn=1 be a d-th dataset and xdn ∈ RM be the M -dimensional feature vector of n-th instance in the d-th dataset . Instances { xdn } Ndn=1 are drawn from a distribution with density pd . We assume that feature dimension M is the same across all datasets , but each distribution can differ . Suppose that D datasets X : = { Xd } Dd=1 are given at the training phase . Our goal is to estimate a relative density-ratio rα ( x ) from two target datasets that consist of a few instances , Sdnu = { xdnun } Ndnu n=1 and Sdde = { xdden } Ndde n=1 , where dnu and dde are not included in { 1. . . . , D } , that are given at the test phase .

Summary: The authors are proposing to use meta-learning to estimation the density ratio which is a very important problem in machine learning. Density ratio estimation is a very challenging problem as it involves matching an infinite number of statistical moments, which can cause stability issues. The authors have constrained the problem so that they are addressing so that they have limited data from the source domain to estimate the density. The density-ratio is expressed using a permutation-invariant neural network.  The authors derive the loss function for the density ratio that is represented by a neural network. Then show that the parameters of the neural network can be estimated by splitting the data set into support instances and test instances. The authors have demonstrated its capability for relative density ratio estimation with applications to inlier-based outlier detection. Their results have shown to be superior under all matrix.