Meta-Learning for Relative Density-Ratio Estimation
Year: 2021

Abstract:
The ratio of two probability densities, called a density-ratio, is a vital quantity in machine learning. In particular, a relative density-ratio, which is a bounded extension of the density-ratio, has received much attention due to its stability and has been used in various applications such as outlier detection and dataset comparison. Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. However, sufficient instances are often unavailable in practice. In this paper, we propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. Specifically, given two datasets that consist of a few instances, our model extracts the datasets’ information by using neural networks and uses it to obtain instance embeddings appropriate for the relative DRE. We model the relative density-ratio by a linear model on the embedded space, whose global optimum solution can be obtained as a closed-form solution. The closed-form solution enables fast and effective adaptation to a few instances, and its differentiability enables us to train our model such that the expected test error for relative DRE can be explicitly minimized after adapting to a few instances. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.

heading: 1 Introduction
text: The ratio of two probability densities, called a density-ratio, has been used in various applications such as outlier detection [14, 1], dataset comparison [49], covariate shift adaptation [42], change point detection [30], positive and unlabeled (PU) learning [19, 18], density estimation [46], and generative adversarial networks [47]. Thus, density-ratio estimation (DRE) is attracting a lot of attention. A naive approach to DRE is to estimate each density and then take the ratio. However, this approach does not work well since density estimation is a hard problem [48]. Therefore, direct DRE without going through density estimation has been extensively studied [44, 17, 35, 13].
Although direct DRE is useful, its fundamental weakness is that the density-ratio is unbounded, i.e., it can take infinity, which causes stability issues [29]. To cope with this problem, a relative density-ratio has been proposed, which is a smoothed and bounded extension of the density-ratio [49]. In the above applications, the density-ratio can be replaced with the relative density-ratio, and relative DRE has shown excellent performance [49, 30, 6, 37, 47].
Existing methods for (relative) DRE require many instances from both densities. However, sufficient instances are often unavailable for various reasons. For example, it is difficult to instantly collect many instances from new data sources such as new users or new systems. Collecting instances is
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
expensive in some applications such as clinical trials or crash tests, where DRE can be used for dataset comparison to investigate the effect of drugs/car conditions. In such cases, existing methods cannot work well.
αpA(x)+(1−α)pB(x) , (0 ≤ α < 1) with tar-
get datasets A and B that are generated from densities pA(x) and pB(x), respectively.
In this paper, we propose a meta-learning method for relative DRE. To estimate the relative density-ratio from a few instances in target datasets, the proposed method uses instances in different but related datasets, called source datasets. When these datasets are related, we can transfer useful knowledge from source datasets to target ones [31]. Figure 1 shows our problem formulation.
We model the relative density-ratio by using neural networks that enable us to perform accurate DRE thanks to their high expressive capabilities. Since each dataset has a different property, incorporating it to the model is essential. To achieve this, given two datasets that consist of a few instances, called support instances, our model first calculates a latent vector representation of each dataset. This vector is calculated by using permutation-invariant neural networks that can take a set of instances as input [50]. Since the vector is obtained from a set of instances in the dataset, it contains information of the dataset.
With the two latent vectors of datasets, each instance is non-linearly mapped to an embedding space that is suitable for relative DRE on the datasets. Using the embedded instances, we perform relative DRE, where the relative density-ratio is represented by a linear model on the embedded space. With the squared loss, the global optimal solution of the linear model can be obtained as a closed-form solution, which enables us to perform more stable and faster adaptation to support instances than numerical solutions.
The neural networks of our model are trained by minimizing the expected test squared error of relative DRE after adapting to support instances that is calculated using instances in the source datasets. Since the closed-form solution of the linear model is differentiable, this training can be performed by gradient-based methods such as ADAM [20]. Since all parameters of our model are shared across all datasets, which enables knowledge to be shared between all datasets, the learned model can be applied to unseen target datasets. This training explicitly improves the relative DRE performance for test instances after estimating the relative density-ratio using support instances. Thus, the learned model can accurately estimate the relative density-ratio from a few instances.
Our main contributions are as follows: (1) To the best of knowledge, our work is the first attempt at meta-learning for (relative) DRE. (2) We propose a model that performs accurate relative DRE from a few instances by effectively adapting both embeddings and linear model to the instances. (3) We empirically demonstrate the effectiveness of the proposed method with three popular problems: relative DRE, dataset comparison, and outlier detection.

heading: 2 Related Work
text: Many methods for direct DRE have been proposed such as classifier-based methods [4, 33], Kullback–Leibler importance estimation [43], kernel mean matching [13], and unconstrained least-squares importance fitting (uLSIF) [17]. Although these methods are useful, they can suffer from the unbounded nature of the density-ratio. That is, an instance that is in the low density region of the denominator density may have an extremely large value of ratio, and DRE can be dominated by such points, which causes robustness and stability issues [29]. This problem is particular serious when using flexible density-ratio models such as neural networks because they try to fit on extremely large density-ratio values [18]. To cope with the unboundedness, the relative uLSIF (RuLSIF) uses the relative density-ratio to uLSIF [49]. Since RuLSIF can obtain the optimal parameters as a closed-form solution, it is computationally efficient and stable and performs well in various applications when sufficient instances are available [49, 30, 6, 37, 47]. Since fast and effective adaptation to support in-
stances is essential in meta-learning as explained in the end of this section, we incorporate RuLSIF in our framework. Although neural network-based DRE methods have been recently proposed [18, 35], they cannot perform well when training instances are quite small due to overfitting. Although the proposed method uses neural networks for the instance embeddings, it can accurately perform relative DRE from a few instances by learning how to perform few-shot relative DRE with related datasets.
DRE is used for transfer learning or covariate shift adaptation [40, 42, 45, 22, 7]. To transfer knowledge in a training dataset to a test dataset, these methods estimate the density-ratio between training and test densities that is used for weighting labeled training instances. These methods use only two datasets to estimate the density-ratio. In contrast, the proposed method uses multiple datasets to accumulate transferable knowledge and uses it for the relative DRE on two new datasets, which can be used in various applications including covariate shift adaptation as described in Section 1.
Meta-learning methods have been recently attracting a lot of attention [8, 41, 9, 34, 3, 16]. These methods train a model such that it generalizes well after adapting to support instances using multiple datasets. In this framework, fast adaptation to support instances is essential since the result of the adaptation is required to train the model in each iteration of training [3, 34]. Encoder-decoder methods such as neural processes [9, 10] perform quick adaptation by forwarding support instances to neural networks. However, they have difficulty working well for any dataset since the adaptation is approximated by only the neural networks. Gradient-based methods such as model-agnostic metalearning (MAML) [8] adapt to support instances by using an iterative gradient descent method and are widely used. These methods require higher-order derivatives and to retain all optimization path of the iterative adaptation to backpropagate through the path, which imposes considerable computational and memory burdens [3]. Thus, they must keep the iteration number small and it prevents effective adaptation. In contrast, the proposed method quickly and effectively adapt to support instances by solving a convex optimization problem, where the global optimum solution can be quickly obtained as a closed-form solution. Although few methods adapt to support instances by solving convex optimization problems for fast and effective adaptation [3, 26], they consider classification tasks. To the best of our knowledge, no meta-learning methods have been designed for DRE, and thus, existing meta-learning methods cannot be applied to our problems.

heading: 3 Preliminary
text: We briefly explain a relative density-ratio. Suppose that instances {xn}Nn=1 are drawn from a distribution with density pnu(x) and instances {x′n}N ′
n=1 are drawn from another distribution with density pde(x). Density ratio r(x) is defined by r(x) :=
pnu(x) pde(x) . Here, “nu” and “de” indicate the numerator and the denominator. The aim of DRE is to directly estimate r(x) from both instances {xn}Nn=1 and {x′n}N ′
n=1. However, r(x) is unbounded and thus can take extremely large values when the denominator pde(x) takes a small value. This causes robustness and stability issues [29]. To deal with this problem, a relative density-ratio has been proposed [49]. For 0 ≤ α < 1, relative densityratio rα(x) is define by rα(x) :=
pnu(x) αpnu(x)+(1−α)pde(x) . Relative density-ratio rα(x) is bounded since
rα(x) ≤ 1α for any x. rα(x) is always smoother than r(x). rα(x) can replace r(x) and is used for various applications [49, 30, 6, 37, 47]. When α = 0, rα(x) is reduced to density-ratio r(x). Thus, the relative density-ratio can be regarded as a smoothed and bounded extension of the density-ratio.

heading: 4 Proposed Method

heading: 4.1 Problem Formulation
text: Let Xd = {xdn}Ndn=1 be a d-th dataset and xdn ∈ RM be the M -dimensional feature vector of n-th instance in the d-th dataset. Instances {xdn}Ndn=1 are drawn from a distribution with density pd. We assume that feature dimension M is the same across all datasets, but each distribution can differ. Suppose that D datasets X := {Xd}Dd=1 are given at the training phase. Our goal is to estimate a relative density-ratio rα(x) from two target datasets that consist of a few instances, Sdnu = {xdnun} Ndnu n=1 and Sdde = {xdden} Ndde n=1 , where dnu and dde are not included in {1. . . . , D}, that are given at the test phase.

heading: 4.2 Model
text: In this subsection, we use notations Snu and Sde instead of Sdnu and Sdde , respectively for simplicity. Similarly, we use pnu and pde instead of pdnu and pdde , respectively. We explain our model that estimates the relative density-ratio from S = Snu ∪ Sde, called support instances. First, our model calculates a latent representation of each dataset using permutation-invariant neural networks [50]:
znu := g
( 1
|Snu| ∑
x∈Snu
f(x) ) ∈ RK , zde := g ( 1 |Sde| ∑
x∈Sde
f(x) ) ∈ RK , (1)
where f and g are any feed-forward neural network. Since summation is permutation-invariant, the neural network in Eq. (1) outputs the same vector even though the order of instances in each dataset varies. Thus, the neural network in Eq. (1) is well defined as functions for set inputs. Since latent vector z is calculated from the set of instances in a dataset, z contains information of the empirical distribution of instances in the dataset. The proposed method can use any other permutation-invariant function such as summation [50] and set transformer [25] to obtain latent vectors of datasets.
The proposed method models the relative density-ratio by the following neural network,
r̂α(x;S) := w>h([x, znu, zde]), (2)
where [·, ·, ·] is a concatenation of vectors, h : RM+2K → RT>0 is a feed-forward neural network, and w ∈ RT≥0 is linear weights. The non-negativeness of both the outputs of h and w ensures the non-negativeness of the estimated relative density-ratio. h([x, znu, zde]) represents the embedding of instance x. Since h([x, znu, zde]) depends on both znu and zde, the embeddings reflect the characteristics of two datasets. Such embeddings are learned by using source datasets X so that they lead to accurate DRE given the target datasets, which will be described in the subsection 4.3.
Linear weights w are determined so that the following expected squared error between true relative density-ratio rα(x) and estimated relative density-ratio r̂α(x;S), Jα, is minimized:
Jα(w) := 1
2 Eqα(x)
[ (rα(x)− r̂α(x;S))2 ] = α
2 Epnu(x)
[ r̂α(x;S)2 ] +
1− α 2 Epde(x) [ r̂α(x;S)2 ] − Epnu(x) [r̂α(x;S)] + Const., (3)
where E is expectation, qα(x) := αpnu(x) + (1− α)pde(x), and Const is a constant term that does not depend on our model. By approximating the expectation with support instances S and excluding the non-negative constraints for w, we obtained the following optimization problem:
w̃ := arg min w∈RT
[ 1
2 w>Kw − k>w + λ 2 w>w
] , (4)
where k = 1|Snu| ∑ x∈Snu h([x, znu, zde]) and K = α |Snu| ∑ x∈Snu h([x, znu, zde])h([x, znu, zde])
>+ 1−α |Sde| ∑ x∈Sde h([x, znu, zde])h([x, znu, zde])
>. In Eq. (4), the third term of r.h.s. is the `2-regularizer to prevent over-fitting, and λ > 0 is a positive real number. The global optimum solution for Eq. (4) can be obtained as the following closed-form solution:
w̃ = (K + λI) −1 k, (5)
where I is the T dimensional identity matrix. This closed-form solution can be efficiently obtained when T is not large. Note that (K + λI)−1 exists since λ > 0 makes (K + λI) positive-definite. Some learned weights w̃ can be negative. To compensate for this, following previous studies [17], the solution is modified as ŵ = max(0, w̃), where max operator is applied for each element of w̃. The closed-form solution enables fast and effective adaptation to support instances S. By using the learned weights, the relative density-ratio estimated with support instances S can be obtained as
r̂∗α(x;S) := ŵ>h([x, znu, zde]). (6)

heading: 4.3 Training
text: We explain the training procedure for our model. In this subsection, symbols S = Snu ∪ Sde are used as support instances in source datasets. In our model, the parameters to be estimated, Θ, are
Algorithm 1 Training procedure of our model. Require: Source datasets X , support instance size NS , query instance size NQ, relative parameter α Ensure: Parameters of our model Θ
1: repeat 2: Sample two datasets d and d′ from {1, . . . , D} with replacement 3: Select support instances Snu and Sde with size NS from Xd and Xd′ , respectively 4: Select query instances Qnu and Qde with size NQ from Xd and Xd′ , respectively 5: Calculate linear weights w̃ with the support instances by Eq. (5) to obtain Eq. (6) 6: Calculate the loss J̃α(Q;S) in Eq. (8) with the query instances 7: Update parameters with the gradients of the loss J̃α(Q;S) 8: until End condition is satisfied;
neural network parameters f , g, h, and regularizer parameter λ. We estimate these parameters by minimizing the expected test squared error of relative DRE given support instances, where support instances S = Snu ∪ Sde and test instances Q = Qnu ∪Qde, called query instances, are randomly generated from source datasets X:
Ed,d′∼{1,...,D} [ E(Snu,Sde),(Qnu,Qde)∼Xd×Xd′ [ J̃α(Q;S) ]] , (7)
where (U, V ) ∼ Xd×Xd′ denotes that instances U and V are selected fromXd andXd′ , respectively, and J̃α(Q;S) is the approximation of expected squared error Jα with query instances Q,
J̃α(Q;S) = α 2|Qnu| ∑
x∈Qnu
r̂∗α(x;S)2 + 1− α 2|Qde| ∑ x∈Qde r̂∗α(x;S)2 − 1 |Qnu| ∑ x∈Qnu r̂∗α(x;S). (8)
The pseudocode for our training procedure is illustrated in Algorithm 1. For each iteration, we randomly select two datasets with replacement (Line 2). From the datasets, we randomly select support instances S = Snu ∪ Sde and query instances Q = Qnu ∪ Qde (Lines 3–4). We then calculate the relative density-ratio with the support instances (Line 5). Using the estimated relative density-ratio, we calculate loss J̃α(Q;S) with the query instances (Line 6). Lastly, the parameters of our model are updated with the gradient of the loss (Line 7). This training procedure trains the parameters of our model so as to explicitly improve the relative DRE performance after estimating the relative density-ratio with a few instances. Thus, the learned model makes accurate DRE from target support instances. Since the close-form solution for adaptation in Eq. (5) is differentiable w.r.t. the model parameters, this training can be performed by using gradient-based methods such as ADAM [20] . Although we use the squared error for the objective function of query instances in Eq. (8), our framework can use any differentiable loss function for query instances, such as KullbackLeibler divergence [43] since we do not require closed-form solutions for query instances. A more intuitive explanation of the proposed method is described in the supplemental material.

heading: 5 Experiments
text: In this section, we demonstrate the effectiveness of the proposed method with three problems: relative DRE, dataset comparison, and inlier-based outlier detection. All experiments were conducted on a Linux server with an Intel Xeon CPU and a NVIDIA GeForce GTX 1080 GPU.

heading: 5.1 Proposed Method Settings
text: For all problems, a three(two)-layered feed-forward neural network was used for f (g) in Eq. (1). For f , the number of output and hidden nodes was 100, and ReLU activation was used. For h in Eq. (2), a three-layered feed-forward neural network with 100 hidden nodes with ReLU activation and 100 output nodes (T = 100) with the Softplus function was used. Hyperparameters were determined based on the empirical squared error for relative DRE on validation data. The dimension of latent vectors z was chosen from {4, 8, 16, 32, 64, 128, 256}. Relative parameter α was set to 0.5, which is a value recommended in a previous study [49]. We used the Adam optimizer [20] with a learning rate of 0.001. The mini-batch size was set to 256 (i.e., NQ = 128 for numerator and denominator instances). In training with source datasets, support instances are included in query instances as
in [9, 10]. The squared error on validation data was used for early stopping to avoid over-fitting, where the maximum number of training iterations was 10,000. This setup was used for all neural network-based methods in subsequent subsections. We implemented all methods by PyTorch [32].

heading: 5.2 Relative Density-ratio Estimation
text: We evaluate the relative DRE performance of the proposed method. We evaluated the squared error in Eq. (3) with α = 0.5 on test instances ignoring the constant term that does not depend on models.
Data We used one synthetic data and two real-word benchmark data (Mnist-r1 and Isolet2), which have been commonly used in transfer or multi-task learning studies [11, 27, 23]. In the synthetic data, each dataset Xd was generated from a one-dimensional Gaussian distribution N (µd, σ2d). Datasetspecific mean µd and standard deviation σd were uniform randomly selected from [−1.5, 1.5] and [0.1, 2], respectively. Each dataset consists of 300 instances that are generated from each distribution. We created 600 source, 3 validation, 20 target datasets and evaluated the mean test squared error of all target dataset pairs when the number of target support instances was NS = 10. Mnist-r, which was derived from MNIST, consists of images. Mnist-r has six tasks, where each task is created by rotating the images in multiples of 15 degrees: 0, 15, 30, 45, 60, and 75. Each task has 1000 images, which are represented by 256-dimensional vectors, of 10 classes (digits). Isolet consists of letters spoken by 150 speakers, and speakers are grouped into five groups (tasks) by speaking similarity. Each instance is represented as a 617-dimensional vector. The number of classes (letters) is 26. For both benchmark data, we treat each class of each task as a dataset, and thus, Mnist-r and Isolet have 60 and 130 datasets, respectively. We randomly chose one task and then chose 10 target datasets from the task. From the remaining datasets, we randomly chose 10 validation sets and used the remaining as source datasets. We created 10 different splits of source/validation/target datasets and evaluated the mean test squared error of all target dataset pairs.
Comparison methods We compared the proposed method with RuLSIF [49] and RuLSIF-FT. RuLSIF trains a kernel model with only target support instances for relative DRE. We used the Gaussian kernel and Gaussian width was set to the median distance between support instances, which is a useful heuristic (median trick) [39]. Since neural network-based models performed poorly in our experiments due to small instances, we used the kernel model as in the original paper. RuLSIF-FT uses a neural network for modeling the relative density-ratio. RuLSIF-FT pretrains the model using source datasets and fine-tunes the weights of the last layer with target support instances. Note that the pretrained model in RuLSIF-FT cannot estimate the relative density-ratio for target datasets without fine-tuning. We used the same network architecture as the proposed method, i.e., the four-layered feed-forward neural network. For RuLSIF and RuLSIF-FT, regularization parameter λ was chosen from {0.0001, 0.001, 0.01, 0.1, 1}, and the best test results were reported.
Results Figure 2 shows five illustrating examples of relative DRE in the synthetic data. The proposed method was able to accurately estimate the relative density-ratio from small target support instances. The mean test squared errors without the constant term of Ours, RuLSIF, and RuLSIF-FT were -0.613, -0.559, and -0.551, respectively (lower is better). Table 1 shows the mean test squared
1 https://github.com/ghif/mtae 2 http://archive.ics.uci.edu/ml/datasets/ISOLET
errors ignoring the constant term with different target support instance sizes in Mnist-r and Isolet. The proposed method clearly outperformed RuLSIF and RuLSIF-FT. Since RuLSIF does not use source datasets, it performed worse than the proposed method. Although RuLSIF-FT uses source datasets, it did not work well since it does not have mechanisms for few-shot relative DRE. In contrast, the proposed method trains the model so that it explicitly improves test relative DRE performance after adapting to a few instances, and thus, it worked well.
Table 3 shows the results of an ablation study of the proposed method. NoLatent is our model without latent vectors for datasets z. NoSadapt is our model without adapting to support instances with the closed-form solution ŵ in Eq. (5). NoSadapt learns dataset-invariant linear weights w, and uses only latent vectors of target datasets z to estimate the relative density-ratio for the datasets. Thus, NoSadapt can be categorized into encode-decoder meta-learning methods. NoSadapt-FT finetunes the liner weights w in the model learned by NoSadapt with target support instances. Note that our model without both latent vectors and adapting to support instances cannot perform relative DRE for target datasets because it cannot take any information of target datasets. The details of these models are explained in the supplemental material. Although all methods performed better than RuLSIF and RuLSIF-FT, the proposed method outperformed the others. This result indicates that considering both latent vectors and adaptation to support instances is useful in our framework.

heading: 5.3 Dataset Comparison
text: We evaluate the proposed method with a dataset comparison problem. The aim of this problem is to determine if two datasets that consist of a few instances come from the same distribution. The proposed method outputs the score of whether two datasets come from the same distribution by calculating relative Pearson (PE) divergence [49], which is calculated using the relative density-ratio.
Data We used Mnist-r and Isolet described in the previous subsection. We regard two datasets as coming from the same distribution if they are from the same class of the same task. We used all target dataset pairs for evaluation. Since the numbers of the same and different pairs in the target datasets are imbalanced (10 same and 90 different pairs), we used the area under ROC curve (AUC) as an evaluation metric because it can property evaluate the performance in imbalanced classification problems [2].
Comparison methods We compared the proposed method with six methods: RuLSIF, uLSIF [17], deep direct DRE (D3RE) [18], maximum mean discrepancy (MMD) [12], RuLSIF-FT, and D3REFT. uLSIF is a DRE method, which is equivalent to RuLSIF with α = 0. For RuLSIF, uLSIF, and RuLSIF-FT, the setting is the same as those of subsection 5.2. D3RE is a recently proposed neural network-based DRE method. We used the LSIF-based loss function and the same network architectures as the proposed method, i.e., the four-layered feed-forward neural network. D3RE-FT pretrains the model with source datasets and fine-tunes the weights of the last layer with target support instances. For D3RE and D3RE-FT, hyperparameter C was chosen from {0.1, 0.5, 1, 10}, and the best test results were reported. MMD is a non-parametric distribution discrepancy metric, which is widely used since it can compare distributions without density estimation. We used the Gaussian kernel and Gaussian width was determined by the median trick. For the proposed method, RuLSIF, and RuLSIF-FT, relative PE divergence was used for the distribution discrepancy metric. For uLSIF, D3RE, and D3RE-FT, PE divergence was used. Although the proposed method, RuLSIF-FT, and D3RE-FT use source datasets for training, the others do not. Note that no methods use any information of similarity/dissimilarity of two datasets during training.
Results Table 2 shows the mean test AUCs with different target support instance sizes. The proposed method showed the best/comparable results for all cases. RuLSIF performed better than uLSIF since relative DRE is more stable than DRE with a few instances. D3RE performed worse than the proposed method since target support instances were too small to train its neural network. When support instance size was small, the proposed method outperformed the others by a large margin. This is because it is difficult for RuLSIF, uLSIF, D3RE, and MMD to compare two datasets from only a few target instances. In contrast, the proposed method was able to accurately compare two datasets from a few instances because it learns to perform accurate relative DRE with a few instances. Since RuLSIF-FT and D3RE-FT were not trained for few-shot DRE, they did not perform well. Table 4 shows the results of the ablation study. Similar to the results in subsection 5.2, the proposed method performed better than the others by considering both latent vectors and adapting to support instances.

heading: 5.4 Inlier-based Outlier Detection
text: We evaluate the proposed method with an inlier-based outlier detection problem. This problem is to find outlier instances in an unlabeled dataset based on another dataset that consists of normal instances. By defining the density-ratio where the numerator and denominator densities are normal and unlabeled densities, respectively, we can see that the density-ratio values for outliers are close to zero since outliers are in a region where normal (unlabeled) density is low (high). Thus, we can use the negative density-ratio value as outlier scores [1, 14]. Similarly, the relative density-ratio values of outliers are close to zero, and thus, we can also use them as outlier scores [49]. In this problem, each dataset consists of normal and unlabeled instances Xd = Xnord ∪Xund . Along with this, we use a slightly modified sampling procedure of Algorithm 1. Specifically, for each iteration, we sample one dataset from the source datasets and create support and query instances from the dataset. The details of the algorithm are described in the supplemental material. We assume that the number of target normal support instances NnorS is small since labeling cost is often high in practice such as normal behavior-based outlier systems for new users [24].
Data We used three real-world benchmark data: IoT3, Landmine4, and School5. These benchmark data are commonly used in outlier detection studies [23, 15]. IoT is real network traffic data, which are gathered from nine IoT devices (datasets) infected by malware. Landmine consists of 29 datasets, and each instance is extracted from a radar image that captures a region of a minefield. School contains the examination scores of students from 139 schools (datasets). We picked schools with 100 or more students, ending up with 74 datasets. The average outlier rates in a dataset of IoT, Landmine, and School are 0.05, 0.06, and 0.15, respectively. The details of the benchmark data are described in the supplemental material. For IoT, we randomly chose one target, one validation, and seven source datasets. For Landmine, we randomly chose 3 target, 3 validation, and 23 source datasets. For School, we randomly chose 10 target, 10 validation, and 54 source datasets. For each source/validation dataset in IoT, Landmine, and School, we chose 200, 200, and 50 instances, respectively, as normal and the remaining as unlabeled instances. For each target dataset, we used all instances except for
3 https://archive.ics.uci.edu/ml/datasets/detection_of_IoT_botnet_attacks_N_BaIoT 4 http://people.ee.duke.edu/ lcarin/LandmineData.zip 5 http://multilevel.ioe.ac.uk/intro/datasets.html
target normal support instances as unlabeled instances (test instances). For each benchmark data, we randomly created 10 different splits of target/validation/source datasets and evaluated the mean test AUC on the target datasets.
Comparison methods We compare the proposed method with 13 outlier detection methods: RuLSIF, uLSIF [14], D3RE, local outlier factor (LOF) [5], isolation forest (IF) [28], autoencoder (AE) [38], deep support vector description (SD) [36], AE-S, SD-S, fine-tuning methods for AE and SD (AE-FT and SD-FT), RuLSIF-FT, and D3RE-FT. LOF and IF use only target unlabeled instances to find outliers. AE and SD use target normal instances for training. AE-S and SD-S use source normal instances for training. AE-FT and SD-FT fine-tune models trained by AE-S and SD-S with target normal instances, respectively. Note that although AE and SD-based methods can use unlabeled instances as well as normal instances for training, they performed worse than them trained with only normal instances. Thus, we used only normal instances for training. RuLSIF, uLSIF, and D3RE use target normal and unlabeled instances. RuLSIF-FT and D3RE-FT use source normal and unlabeled instances as well as target normal and unlabeled instances. Note that no methods use any information of outliers for training. The details of the implementation such as neural network architectures and hyperparameter candidates are described in the supplemental material.
Results Table 5 shows the mean test AUCs with different target normal support instance sizes. The proposed method showed the best/comparable results for all cases. Density-ratio methods such as RuLSIF and uLSIF tended to show better results than the other comparison methods by using information from both target normal and unlabeled instances. The proposed method was able to further improve performance than RuLSIF and uLSIF by incorporating the mechanism for few-shot relative DRE. Table 6 shows the results of an
ablation study. The best method can vary across benchmark data since each benchmark data has different properties. For example, in IoT, all datasets are relatively similar, which is validated by the fact that test AUCs are high [23]. Thus, the dataset-invariant embedding function h in NoLatent is sufficient for adaptation. Nevertheless, the proposed method (Ours) showed the best average AUCs over all benchmark data. In the supplemental material, we additionally showed that the proposed method outperformed the PU learning method [21].

heading: 5.5 Dependency of Relative Parameter α
text: We investigated the dependency of relative parameter α > 0 in the proposed method. Relative parameter α determines the upper bound of relative density-ratio value since rα(x) ≤ 1α for any
x. Table 7 shows results with α = 0.1, 0.5, and 0.9 for all three problems. The proposed method consistently outperformed RuLSIF over different α values. This result suggests that the proposed method is relatively robust against the relative parameter value. Note that, in relative DRE, comparison between different α values is meaningless since the scale of the evaluation metrics is different. Besides, various additional results such as investigation of the dependency of the dimensions of latent vectors are described in the supplemental material.

heading: 5.6 Computation Cost
text: We investigated the computation time of the proposed method. Table 8 shows the computation time of each method for dataset comparison with Mnist-r. We used a computer with a 2.20GHz CPU. The support instance size in each target dataset was set to five. We omitted D3RE since it requires to train the neural network for each target dataset comparison, which is quite time-consuming compared to the others. Although the proposed method took time for training with source datasets, it was able to compare datasets with relative DRE as fast as other methods.

heading: 6 Limitations
text: The proposed method uses multiple source datasets to improve relative DRE performance on target datasets. However, when source and target datasets are significantly different, there is a risk of degrading the performance on the target datasets. This phenomenon is called “negative transfer”, and is a common challenge in general transfer/meta-learning methods. Developing methods to automatically remove negative effects of such datasets is one of the important research directions.

heading: 7 Conclusion
text: In this paper, we proposed a meta-learning method for relative DRE. We empirically showed that the proposed method outperformed various existing methods in three problems: relative DRE, dataset comparison, and outlier detection. As future work, we plan to incorporate other DRE models such as telescopic DRE [35] in our framework. We describe a potential negative social impact of our work. The proposed method needs to access datasets obtained from multiple sources like almost all meta-learning methods. When each dataset is provided from different owners, sensitive information in the dataset risks being stolen and abused by malicious people. To evade this risk, we encourage research for developing meta-learning methods without accessing raw datasets.

Review summaries:
The paper shows how one can estimate density ratios when very few data points of the two sets of interested are available. In order to do that, the idea is to leverage other datasets that contain related tasks. The ratio is estimated via a linear model to a latent space that represents each dataset.

This paper addresses the estimation of the relative density ratio using only a few instances from two datasets.  The authors propose a meta-learning method in which a neural network is trained to output the embeddings reflect the characteristics of two datasets.  In addition, they proposed to use the closed-solution of a linear model on the embedded space.  The experimental results show that the proposed method outperforms conventional methods on dataset comparison problems and outlier detection when a few instances are available.

The authors are proposing to use meta-learning to estimation the density ratio which is a very important problem in machine learning. Density ratio estimation is a very challenging problem as it involves matching an infinite number of statistical moments, which can cause stability issues. The authors have constrained the problem so that they are addressing so that they have limited data from the source domain to estimate the density. The density-ratio is expressed using a permutation-invariant neural network.  The authors derive the loss function for the density ratio that is represented by a neural network. Then show that the parameters of the neural network can be estimated by splitting the data set into support instances and test instances. The authors have demonstrated its capability for relative density ratio estimation with applications to inlier-based outlier detection. Their results have shown to be superior under all matrix.