Meta-Learning for Relative Density-Ratio Estimation
Authors: Atsutoshi Kumagai
Tomoharu Iwata
Yasuhiro Fujiwara
Venue: NeurIPS 2021 Poster (2021)
Keywords: density-ratio estimation
relative density-ratio estimation
meta-learning
neural networks

Abstract: The ratio of two probability densities, called a density-ratio, is a vital quantity in machine learning. In particular, a relative density-ratio, which is a bounded extension of the density-ratio, has received much attention due to its stability and has been used in various applications such as outlier detection and dataset comparison. Existing methods for (relative) density-ratio estimation (DRE) require many instances from both densities. However, sufficient instances are often unavailable in practice. In this paper, we propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge in related datasets. Specifically, given two datasets that consist of a few instances, our model extracts the datasets' information by using neural networks and uses it to obtain instance embeddings appropriate for the relative DRE. We model the relative density-ratio by a linear model on the embedded space, whose global optimum solution can be obtained as a closed-form solution. The closed-form solution enables fast and effective adaptation to a few instances, and its differentiability enables us to train our model such that the expected test error for relative DRE can be explicitly minimized after adapting to a few instances. We empirically demonstrate the effectiveness of the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.

Meta-Learning for Relative Density- -Ratio Estimation
Atsutoshi Kumagai
Tomoharu Iwata
NTT Computer and Data Science Laboratories .wwploaaiiikumiii........ kumagai ht@hho................ .tt..ntt... jo.
NTT Communication Science Laboratories
atsutoshi
tomoharu. iwata. www.tiagaaaaaae...... gy@hco.ntte .tt. co. jp
Yasuhiro Fujiwara
NTT Communication Science Laboratories masuhiro.fujiwary kh@hco ntt. co.
Abstract
The ratio of two probability densities, called density-ratio is vital quantity in machine learning. In particular, a relative density-ratio, which is a bounded extension of the density-ratio, has received much attention due to its stability and has been used various applications such as outlier detection and dataset compari- Existing methods for (relative) density-ratin estimation (DRE) require many instances from both densities. However, sufficient instances are often unavailable practice. In this paper, we propose a meta-learning method for relative DRE, which estimates the relative density-ratio from a few instances by using knowledge related datasets. Specifically, given two datasets that consist of few instances, our model extracts the datasets' information by using neural networks and uses it to obtain instance embeddings appropriate for the relative DRE. We model the relative density-ratio by linear model on the embedded space, whose global optimum solution can be obtained as closed-form solution. The closed-form solution enables fast and effective adaptation to few instances, and its differentiability enables us to train our model such that the expected test error for relative DRE can be explicitly minimized after adapting to few instances. We empirically demonstrate the effectiveness the proposed method by using three problems: relative DRE, dataset comparison, and outlier detection.
Introduction
The ratio two probability densities, called a density-ratin has been used in various applications
such as outlier detection [], dataset comparison [49], covariate shift adaptation [42), change point
detection (30, positive and unlabeled (PU) learning 19 18] density estimation [46], and generative
adversarial networks [47]. Thus, density-ratio estimation (DRE) is attracting a lot of attention. A
naive approach to DRE is to estimate each density and then take the ratio. However, this approach
does not work well since density estimation is hard problem [48] Therefore, direct DRE without
going through density estimation has been extensively studied
Although direct DRE is useful, its fundamental weakness is that the density-ratio is unbounded, i.e., it can take infinity, which causes stability issues [29]. To cope with this problem, a relative density-ratin has been proposed, which is a smoothed and bounded extension of the density-ratio 49). In the above applications, the density-ratio can be replaced with the relative density-ratio, and relative DRE has
shown excellent performance 49
47
Existing methods for (relative) DRE require many instances from both densities. However, sufficient instances are often unavailable for various reasons. For example, it is difficult to instantly collect many instances from new data sources such as new users or new systems. Collecting instances is
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

expensive in some applications such as clinical trials or crash tests, where DRE can be used for dataset comparison to investigate the effect of drugs/car conditions. In such cases, existing methods cannot work well.
In this paper, we propose meta-learning method for relative DRE. To estimate the relative density-ratio from a few instances in target datasets, the proposed method uses instances in different but related datasets, called source datasets. When these datasets are re- lated, we can transfer useful knowledge from source datasets to target ones [31. Figure shows our prob- lem formulation.
Training phase
Relebive density-ratio
Lest phase
for target cotesets
(4)
2000 (u Jo(X)
Learned model
Max
X22
Xi
Xay
X
Mi
Xaz
We model the relative density-ratio by using neural networks that enable us to perform accurate DRE thanks their high expressive capabilities. Since each dataset has a different property. incorporating it the model is essential. To achieve this, given two datasets that consist of a few instances, called support instances, our model first calculates latent vector representation of each dataset. This vector calculated by using permutation invarrinnt neural networks that can take a set of instances as input [50). Since the vector obtained from a set of instances in the dataset, it contains information of the dataset.
1 Source dataset J
Xu
X
Source dataset D
Taraet cotaset A
Terget dataneet
Figure I: Our problem formulation. In train- ing phase, our model is trained with source datasets. In test phase, the learned model estimates relative density-ratte
(x) = (0 S a < 1) with tar- get datasets A and B that are generated from densities p.A (x) and pn respectively.
(x
ap (1-a)xx))))
With the two latent vectors of datasets, each instance is non-linearly mapped to an embedding space that is suitable for relative DRE the datasets. Using the embedded instances, we perform relative DRE, where the relative density-ratio is represented by a linear model on the embedded space. With the squared loss, the global optimal solution of the linear model can be obtained as a closed-form solution, which enables us to perform more stable and faster adaptation to support instances than numerical solutions.
The neural networks of our model are trained by minimizing the expected test squared error of relative DRE after adapting to support instances that is calculated using instances in the source datasets. Since the closed-form solution the linear model is differentiable, this training can be performed by gradient-based methods such as ADAM (20). Since all parameters of our model are shared across all datasets, which enables knowledge to be shared between all datasets, the learned model can be applied to unseen target datasets. This training explicitly improves the relative DRE performance for test instances after estimating the relative density-ratio using support instances. Thus, the learned model can accurately estimate the relative density-ratic from a few instances.
Our main contributions are as follows: (1) To the best of knowledge, our work is the first attempt at meta-learning for (relative) DRE. (2) We propose model that performs accurate relative DRE from a few instances by effectively adapting both embeddings and linear model the instances. (3) We empirically demonstrate the effectiveness of the proposed method with three popular problems: relative DRE, dataset comparison, and outlier detection.
2 Related Work
Many methods for direct DRE have been proposed such as classifier-based methods 4
33 Kull-
back-Leibler importance estimation [43), kernel mean matching [13). and unconstrained lquares importance fitting (uLSIF) [1]] Although these methods are useful, they can suffer from the un- bounded nature of the density-ratio. That is, an instance that is in the low density region of the denominator density may have an extremely large value of ratio, and DRE can be dominated by such points, which causes robustness and stability issues 29). This problem is particular serious when using flexible density-ratio models such as neural networks because they try to fit on extremely large density-ratio values [88] To cope with the unboundedness, the relative uLSIF (RuLSIFF) uses the relative density-ratio to ul SIF [49] Since Rul can obtain the optimal parameters as closed-form solution, computationally efficient and stable and performs well in various applications when sufficient instances available [49 30 6 37 47 Since fast and effective adaptation to support in-

---

stances is essential in meta- meta-learming as explained in the end of this section, we incorporate RuLSIF in our framework. Although neural network-based DRE methods have been recently proposed [8] 351, they cannot perform well when training instances are quite small due to overfitting. Although the proposed method uses neural networks for the instance emheddings, it can accurately perform relative DRE from few instances by learning how to perform few-shot relative DRE with related datasets.
To transfer knowledge in a training dataset to a test dataset, these methods estimate the density-ratio between training and test densities that is used for weighting labeled training instances. These methods use only two datasets to estimate the density-ratio. In contrast, the proposed method uses multiple datasets to accumulate transferable knowledge and uses it for the relative DRE on two new datasets, which can be used in various applications including covariate shift adaptation as described in Section 1.
DRE is used for transfer learning or covariate shift adaptation 140
42 45
Meta-learning methods have been recently attracting lot of attention 88 41
16]. These
methods train a model such that it generalizes well after adapting to support instances using multiple datasets. In this framework, fast adaptation to support instances essential since the result of the adaptation is required to train the model in each iteration of training [ 34). Encoder-decoder
methods such as neural processes ( 10] perform quick adaptation by forwarding support instances to neural networks. However, they have difficulty working well for any dataset since the adaptation is approximated by only the neural networks. tradient-based methods such as model-agnostic meta- learning (MAML) [8] adapt to support instances by using an iterative gradient descent method and are widely used. These methods require higher-order derivatives and to retain all optimization path of the iterative adaptation to backpropagate through the path, which imposes considerable computational and memory burdens [ Thus, they must keep the iteration number small and it prevents effective adaptation. In contrast, the proposed method quickly and effectively adapt to support instances by solving a convex optimization problem, where the global optimum solution can be quickly obtained as a closed-form solution. Although few methods adapt to support instances by solving convex optimization problems for fast and effective adaptation [3] 26). they consider classification tasks. To the best of our knowledge, no meta-learning methods have been designed for DRE, and thus, existing meta-learning methods cannot be applied to our problems.
3 Preliminary
We briefly explain relative density-ratio. Suppose that instances J. { are drawn from distribution with density (x) and instances (x,) are drawn from another distribution with density (x). Density ratio r(x) is defined by r(x) == P=)(x) Here, "nu" and "de" indicate the numerator and the denominator. The aim of DRE is to directly estimate r(x) from both instances A and {x However, r(x) is unbounded and thus can take extremely large values when the denominator ( takes small value. This causes robustness and stability issues (9). To deal with this problem, a relative density-ratio has been proposed [4). For S a < 1, relative density- ratio (x) is define by ra (x) == (XPm(()+(I-- (Pa Relative density-ratio ra (x) is bounded since
(x)
{x,)
(x)
(x) S o for any x. To (x) is always smoother than rxx). ra (x) can replace r(x) and is used for various applications [49 30 5 37 47). When a = 0, ra (x) is reduced to density-ratio r(x). Thus, the relative density-ratio can regarded as a smoothed and bounded extension of the density-ratio.
4 Proposed Method
4.1 Problem Formulation
Let X2 [xan)] be d-th dataset and Xdn. E R be the Mddimensional feature vector of n-th instance in the d-th dataset. Instances {xán) Ne are drawn from distribution with density Pa- We assume that feature dimension M is the same across all datasets, but each distribution can differ. Suppose that 1 datasets X := {Xa)& are given at the training phase. Our goal is to estimate a relative density-ratio (x) from two target datasets that consist of few instances, Sa,.. = (Xdm..m W and Sda. = { Na4,, where dnu and da are not included in (1.....), D).
that are given at the test phase.
3

---

4.2 Model
In this subsection, we use notations Smn and Sae instead of Sd... and Sa.. respectively for simplicity. Similarly, we use Pnu and Pde instead of Pd... and Pda.. respectively. We explain our model that estimates the relative density-ratio from S = Sau USe. called support instances. First, our model calculates latent representation of each dataset using permutation-invare neural networks [50]
Σ [(x)
E R
Σ [(x)
RRK
Zen =
Zie ==
S
S XESac
XESmu
where f and g are any feed-forward neural network. Since summation is permutation-invariann the neural network in Eq. (I) outputs the same vector even though the order of instances in each dataset varies. Thus, the neural network in Eq. ① is well defined functions for set inputs. Since latent vector z is calculated from the set of instances in a dataset, z contains information of the empirical distribution of instances in the dataset. The proposed method can use any other permutation-invel -nvvrraan function such as summation [50] and set transformer (25] to obtain latent vectors of datasets.
The proposed method models the relative density-ratio by the following neural network,
fx; 8) =- w h[[x, Znu, Zdo)),
where [,,,.] is concatenation of vectors, h M+2K - TR is feed-forward neural network, and w E R 20 is linear weights. The non-negativeness of both the outputs of h and w ensures the non-negativeness of the estimated relative density-ratio. l[[x, 2mn; Zdel) represents the embedding of instance x. Since h[[x, au 7-de)) depends on both zm and Zde the embeddings reflect the characteristics of two datasets. Such embeddings are learned by using source datasets X so that they lead to accurate DRE given the target datasets, which will be described in the subsection 4.3
Linear weights w are determined so that the following expected squared error between true relative density-ratio ra (x) and estimated relative density-ratee ratio ia (x; S), Ja, is minimized:
Ja (w) = 2 (x)) [(ra(x)) fa S))
2 E (m) [xa(x; +
(a) (xx S)²] Epmn())
+ Const.,
(3)
where E is expectation, (x) axpu + a Pde (x), and Const is constant term that does not depend on our model. By approximating the expectation with support instances and excluding the non-negative constraints for w, we obtained the following optimization problem:
W == arg min WERT
2
w Kw kT w
+
w
(4)
where k- Ban Exes 1[[x, Znu, Zdel) and K S Exes... h[[x, Zau. ((x)) Zon zael)
1-a Exesa. h[[x, mu, 2 )[(())((())))))))) n TTal) In Eq. (4) the third term of r.h.s. is the e2-regularizer
to prevent over-fitting, and À > is positive real number. The global optimum solution for Eq. can be obtained as the following closed-form solution:
(4)
(K + AI
(5)
where I is the T' dimensional identity matrix. This closed-form solution can be efficiently obtained
when T is not large. Note that (K + AI) -1 exists since X > makes (K + AI) positive-defiitee Some learned weights w can be negative. To compensate for this, following previous studies [17]. the solution is modified as w max(0, w), where max operator is applied for each element of w. The closed-form solution enables fast and effective adaptation to support instances S. By using the learned weights, the relative density-ratio estimated with support instances S can be obtained
P% (x: S) = w TThhhle h[[x, Znu; Zaol).
4.3 Training
We explain the training procedure for our model. In this subsection, symbols Smn U Sa. are used as support instances in source datasets. In our model, the parameters to be estimated, θ are

---

Algorithm Training procedure of our model.
Require: Source datasets X, support instance size Ns. query instance size No. relative parameter oK
Ensure: Parameters of our model θ
1: repeat
2: Sample two datasets d and d' from {L..... D} with replacement
3: Select support instances Sau and Sae with size Ns from Xx and X respectively
4: Select query instances Qnu and Qde with size No from X and Xa', respectively
5: Calculate linear weights w with the support instances by Eq. (5) to obtain Eq. (6)
6: Calculate the loss Ja( Q: S) in Eq. (8) with the query instances
7: Update parameters with the gradients of the loss (Q:
8: until End condition is satisfied;
neural network parameters f. 9. h, and regularizer parameter A. We estimate these parameters by minimizing the expected test squared error of relative DRE given support instances, where support instances S = Spa L Sae and test instances Q Qmu U Qde, called query instances, are randomly generated from source datasets X:
Ed.d'~(nnn D) B(Sm (aa [Ja(e:s)]]
where VU X x X denotes that instances U and V are selected from Xa and Xar, respectively, and (2: is the approximation of expected squared error Ja with query instances 2,
(2, 2
a
Σ S);
Σ XE
(x;
Σ (x; X Q.
(8)
2
2
XE@...
The pseudocode for our training procedure is illustrated in Algorithm [ For each iteration, we randomly select
randomly select two datasets with replacement (Line 2). From the datasets, support instances $ = Smm U Sae and query instances Q = Qm U Qae (Lines 3-4). We then calculate the relative density-ratio with the support instances (Line 5). Using the estimated relative density-ratie
calculate loss Ja (Q: with the query instances (Line 6). Lastly, the parameters of our model are updated with the gradient of the loss (Line 7). This training procedure trains the parameters of our model as explicitly improve the relative DRE performance after estimating the relative density-ratio with a few instances. Thus, the learned model makes accurate DRE from target support instances. Since the close-form solution for adaptation in Eq. (5) is differentiable w.r.t. the model parameters, this training can be performed by using gradient-bass based methods such as ADAM (20] Although we use the squared error for the objective function of query instances in Eq. (8. our framework can use any differentiable loss function for query instances, such as KullbackLeible divergence [4] since we do not require closed-form solutions for query instances. A more intuitive explanation of the proposed method is described in the supplemental material.
5 Experiments
In this section, we demonstrate the effectiveness of the proposed method with three problems: relative DRE, dataset comparison, and inlier-based outlier detection. All experiments were conducted Linux server with an Intel Xeon CPU and a NVIDIA GeForce GTX 1080 GPU.
5.1 Proposed Method Settings
For all problems, three(two)-layered feed-forward neural network was used for f (g) in Eq. ( For
f. the number of output and hidden nodes was 100, and ReLU activation was used. For h in Eq. (2). a three-layered feed-forward neural network with 100 hidden nodes with ReLU activation and 100
output nodes (T 100) with the Softplus function was used. Hyperparameters were determined
based on the empirical squared error for relative DRE on validation data. The dimension of latent
vectors z was chosen from (4, 8, 16, 32, 64, 128, 256). Relative parameter & was set to 0.5, which
a value recommended in previous study 49 We used the Adam optimizer [20] with a learning rate of 0.001. The mini-batch size was set to 256 (i.e., No = 128 for numerator and denominator instances). In training with source datasets, support instances are included in query instances as
5

---

Figure 2: Illustrating examples of relative density-ratio estimation when 10 support instances are used in each target dataset. Horizontal and vertical axes represent and relative density-ratio values, respectively. Blue line denotes true relative density-ratio. Orange, green, and red lines represent estimated relative density-ratios by Ours, RuLSIF, and RuLSIF-FT, respectively.
The squared error on validation data was used for early stopping to avoid over-fitting, where the maximum number of training iterations was 10,000. This setup was used for all neural network-based methods in subsequent subsections. We implemented all methods by PyTorch 33
in 99 10).
5.2 Relative Density-ratio Estimation
We evaluate the relative DRE performance the proposed method. We evaluated the squared error in Eq. 3) with 0.5 test instances ignoring the constant term that does not depend models.
Data We used one synthetic data and two real-word benchmark data (Mnist- + and Isole which have been commonly used in transfer multi-task learning studies [1]
27 23 In the synthetic data, each dataset Xa was generated from one-dimensional Gaussian distribution N ((ns (()),02). Dataset- specific mean jaa and standard deviation aa were uniform randomly selected from [-1.5, 1.5) and 20. respectively. Each dataset consists of 300 instances that generated from each distribution. We created 600 source, 3 validation, 20 target datasets and evaluated the mean test squared error of all target dataset pairs when the number of target support instances was Ns 10. Mnist-r, which was derived from MNIST, consists of images. Mnist-r has six tasks, where each task is created by rotating the images in multiples of 15 degrees: 0, 15, 30, 45, 60, and 75. Each task has 1000 images, which are represented by -i6-dimensional vectors, of 10 classes (digits). Isolet consists of letters spoken by 150 speakers, and speakers grouped into five groups (tasks) by speaking similarity. Each instance is represented as a 617-dimensional vector. The number of classes (letters) is 26. For both benchmark data, treat each class of each task dataset, and thus, Mnist-r and Isolet have 60 and 130 datasets, respectively. We randomly chose one task and then chose 10 target datasets from the task. From the remaining datasets, we randomly chose 10 validation and used the remaining as source datasets. We created 10 different splits of source/ pource/validation/targge datasets and evaluated the mean test squared error of all target dataset pairs.
Comparison methods We compared the proposed method with RuLSIF 449 and RULSIF-FT. RuLSIF trains a kernel model with only target support instances for relative DRE. We used the Gaussian kernel and Gaussian width
to the median distance between support instances, which a useful heuristic (median trick) [39]. Since neural network-based models performed poorly in our experiments due to small instances, uses neural network for modeling the relative density-ratio. RuLSIF-FT pretrains the model using source datasets and fine-tunes the weights of the last layer with target support instances. Note that the pretrained model in RuLSIF-FT cannot estimate the relative density-ratio for target datasets without fine-tuning. We used the same network architecture feed-forward neural network. For RuL.IF and RUF-FIFFT,, regularization parameter À was chosen from (0.0001, 0.001, 0.01, 0.1, 1), and the best test results were reported.
used the kernel model as in the original paper. RuLSIF-FT
the proposed method, i.e., the four-layered
Results Figure 2 shows five illustrating examples of relative DRE in the synthetic data. The proposed method was able accurately estimate the relative density-ratio from small target support instances. The mean test squared errors without the constant term of Ours, RuI, and RIF-FIF were -0.613, -0.559, and -0.551, respectively (lower better). Table the mean test squared
https://gituub.com/ghh//maa

---

Table I: Results for relative DRE: Av- erage test squared errors ignoring the constant term with different target sup- port instance sizes. Boldface denotes the best and comparable methods ac- cording to the paired t-test (p 0.05).
Table 2: Results for dataset comparison: Average test AUCs [%]] with different target support instance sizes. Boldface denotes the best and comparable methods according to the paired t-test (p- 0.05).
RuL.IF
DERE
STF
Data Mnist
Ns
Ours
RuLSIF
D3RE
MMD
Data
Ns
Ours
RuLSIF
.IT
83,53
47.86
55.01
64.88
45.14
69.43
63.83
Maist
-0.671
0.543
0.503
-r
2
93.00
84.46
78.09
73.26
85 09.00
78.
68.29
-0.748
-0.581
-0.513
93.86
87.83
75.30
89.18
85.66
-0.772
0.600
-0.518
4
96.49
93,90
83.51
80.19
93.08
87.31
71.79
-0.784
-0.597
-0.520
5
97.54
98.23
90.66
82,96
98.01
86.09
78.62
-0,793
-0.578
-0.520
Avg.
92.88
82.46
77.22
75.32
K2.10
81.36
70.10
Avg.
-0.754
0,580
00.515
Isolet
96,28
50,18
59,38
81.48
48.11
79.50
81.50
Isolet
-0.873
-0.656
00.588
2
98.32
94,28
89,70
88.20
94,57
81.62
87.76
-0.893
-0.676
-0.512
99.23
97.22
94.27
89.69
97.79
85.93
-0,900
-0.693
-0.514
4
99.37
99.01
96.54
91.80
98.96
83.57
91.12
-0.903
-0.695
-0.514
5
99.60
99.61
96.77
93.86
99.43
83.19
92.74
-0,905
-0.683
-0.515
Avg.
98.56
88.07
87.33
89.00
87.77
82.76
88.30
Avg.
-0,895
-0.681
-0,513
Table 3: Ablation study for relative DRE. Average test squared errors without the constant term over different target support instance sizes.
Table 4: Ablation study for dataset comparison. Average test AUCs [%] over different target sup- port instance sizes.
No
NoSadapt
No
NoSadapt
Data
Ours
Latent
Sadapt
FT
Data
Ours
Latent
Sadapt
-FT
Mnist-r
-0,754
-0.739
-0.724
-0.663
Mnist-r
92.88
91.66
88.46
89,92
Isolet
-0.895
-0.888
-0856
-0.723
Isolet
98.56
98.31
94.73
87.49
Avg.
0.825
0.814
-0.79
-0.093
Avg.
95.72
94.99
91.60
88.7I
errors ignoring the constant term with different target support instance sizes in Mnist-r and Isolet. The proposed method clearly outperformed RuLSIF and RuLSIF-FT. Since RuLSIF does not use source datasets, it performed worse than the proposed method. Although RuLSIF-FT uses source datasets, it did not work well since it does not have mechanisms for few-shot relative DRE. In contrast, the proposed method trains the model so that it explicitly improves test relative DRE performance after adapting few instances, and thus, it worked well.
Table shows the results of an ablation study of the proposed method. NoLatent is our model without latent vectors for datasets NoSadapt is model without adapting support instances with the closed-form solution w in Eq. (5) NoSadapt learns dataset-invariant linear weights w, and uses only latent vectors of target datasets
estimate the relative density-ratio for the datasets. Thus, NoSadapt can be categorized into encode-decoder meta-learning methods. NoSadapt-FF finetunes the liner weights in the model learned by NoSadapt with target support instances. Note that our model without both latent vectors and adapting to support instances cannot perform relative DRE for target datasets because cannot take any information of target datasets. The details of these models are explained in the supplemental material. Although all methods performed better than RuLSIF and RuLSiF-FT, the proposed method outperformed the others. This result indicates that considering both latent vectors and adaptation to support instances is useful in our framework.
5.3 Dataset Comparison
We evaluate the proposed method with a dataset comparison problem. The aim of this problem is to determine if two datasets that consist of a few instances come from the same distribution. The proposed method outputs the score of whether two datasets come from the same distribution by calculating relative Pearson (PE) divergence 49 which is calculated using the relative density-ratio.
Data We used Mnist-r and Isolet described in the previous subsection. We regard two datasets as coming from the same distribution if they are from the same class of the same task. We used all target dataset pairs for evaluation. Since the numbers of the same and different pairs in the target datasets are imbalanced (10 same and 90 different pairs), we used the area under ROC curve (AUC) as an evaluation metric because it can property evaluate the performance imbalanced classification problems (2).
7

---

Comparison methods We compared the proposed method with six methods: RuLSIF, uLSIF [7]. deep direct DRE (D3RE) [8. maximum mean discrepancy (MMD) [2]. RuLSIF-FT, and D3RE- FT. ULSIF is DRE method, which is equivalent to RuI.SIF with a O. For RuLSIF, ul SIF, and RTF-FFF the setting is the same as those of subsection 5.2 D3RE a recently proposed neural network-based DRE method. We used the LSIF-based loss function and the same network architectures as the proposed method, i.c., the four-layered feed- worward neural network. D3RE-FT pretrains the model with source datasets and fine-tunes the weights of the last layer with target support instances. For D3RE and D3RE-FT, hyperparameter C' was chosen from {0.1, 0.5, 1, 10), and the best test results were reported. MMD is non-paramttric distribution discrepancy metric, which is widely used since it can compare distributions without density estimation. We used the Gaussian kernel and Gaussian width was determined by the median trick. For the proposed method, RuI.SI, and FIF-FF relative PE divergence was used for the distribution discrepancy metric. For uLSIF, D3RE, and D3RE-FT, PE divergence was used. A Although the proposed method, RTF-FIF. and D3RE-FT use source datasets for training, the others do not. Note that no methods use any information of similaritydissiillrtty of two datasets during training.
Results Table 2 shows the mean test AUCs with different target support instance sizes. The proposed method showed the best/comparable results for all cases. Rul performed better than SIFSI since relative DRE is more stable than DRE with a few instances. D3RE performed worse than the proposed method since target support instances were too small to train its neural network. When support instance size small, the proposed method outperformed the others by a large margin. This is because it is difficult for RuLSIF, uLSIF, D3RE, and MMD to compare two datasets from only a few target instances. In contrast, the proposed method was able to accurately compare two datasets from a few instances because it learns to perform accurate relative DRE with a few instances. Since RuLSIF-FT and D3RE-FT were not trained for few-shot DRE, they did not perform well. Table 4 shows the results of the ablation study. Similar to the results in subsection [5.2, the proposed method performed better than the others by considering both latent vectors and adapting to support instances.
5.4 Inlier-based Outlier Detection
We evaluate the proposed method with an inlier-based outlier detection problem. This problem is to find outlier instances in an unlabeled dataset based on another dataset that consists of normal instances. By defining the density-ratio where the numerator and denominator densities are normal and unlabeled densities, respectively, we can see that the density-ratio values for outliers are close to zero since outliers are in region where normal (unlabeled) density is low (high). Thus, we can use the negative density-ratio value as outlier scores [4]. Similarly, the relative density-ratio values of outliers are close to zero, and thus, we can also use them outlier scores [49]. In this problem, each dataset consists of normal and unlabeled instances X = Xx U X.". Along with this, we use a slightly modified sampling procedure of Algorithm 1. Specifically, for each iteration, we sample one dataset from the source datasets and create support and query instances from the dataset. The details of the algorithm deseribed in the supplemental material. We assume that the number of target normal support instances Nggor is small since labeling cost is often high in practice such as normal behavior-based outlier systems for new users (24)
Data We used three real-world benchmark data: IoT Landmine and Schoo These benchmark data are commonly used in outlier detection studies 23 15). IoT is real network traffic data, which are gathered from nine IoT devices (datasets) infected by malware. Landmine consists of 29 datasets, and each instance is extracted from radar image that captures a region of minefield. School contains the examination scores of students from 139 schools (datasets). We picked schools with 100 or more students, ending up with 74 datasets. The average outlier rates in a dataset of loT, Landmine, and School are 0.05, 0.06, and 0.15, respectively. The details of the benchmark data are described in the supplemental material. For 1oT, we randomly chose one target, one validation, and seven source datasets. For Landmine, we randomly chose 3 target, 3 validation, and 23 source datasets. For School, we randomly chose 10 target, 10 validation, and 54 source datasets. For each source/validation dataset in IoT, Landmine, and School, we chose 200, 200, and 50 instances, respectively, as normal and the remaining as unlabeled instances. For each target dataset, we used all instances except for
hutps:/arcciveeicss http:://www..r..ooo.....///////s cctu:m//datasst//deeettttooooonn Loriiioooo__oooooooo______ _oo_oooTTottttttcckke N_Boo http://people..e ee www.ppppe.................. /uw..edu/ learin/LandmineData.oo 5 http://muttlleeel.cc.. http:/ttiieeeel.......o///nnaaae html
8

---

Table 5: Results for inlier- outlier detection: Average test AUCs [%) with different target normal support instance sizes N Boldface denotes the best and comparable methods according to the paired t-test (p- = 0.05). Second column denotes the number of target normal support instances N
AE
SD
RuiSif
D3RE
Data 1011
Ours
RuI.SIP
u1.SIF
D3RE
AE
SD
1.F
1F
AE-S
SD-$
-FT
-FT
.FT
97.28
95.75
95.87
85,
93.09
91.90
93.30
41.32
43.59
12
66.05
53,50
93,15
84,51
97.81
93.96
94.09
87.48
92.41
92.20
45.84
43.60
34.63
75.21
67.65
88.53
89.57
96.39
94,64
95.18
84,33
89,48
90.86
93.34
40.79
43.61
36.57
76.88
72.82
89.38
88.97
97.05
94.46
93.43
81.74
90.75
89.76
92.82
42.66
43.60
40.26
80.53
67.27
89.01
89.45
96.17
94.43
95.29
81.49
89.01
87.49
92.87
40.14
43.65
38.67
80.25
73.56
88,96
89.
Avg.
96,94
94.65
94.81
84.05
90.95
90.44
93.12
42.15
43.61
38.05
75.78
67.36
89.80
88.33
68.70
53.69
53.84
49.54
52,91
52.60
55.80
52.79
50.36
55.49
53.95
60.85
52.15
mine
64.92
55,56
55.21
52.54
50,39
53.00
45.17
56.08
52.79
51.16
52.08
52.35
62.55
55,15
63.66
54.74
54.41
51.84
49.50
49.16
45.17
56.90
52.80
50.97
55.43
53.67
61.36
53.34
66.24
54.94
53.98
52.21
49.65
51.74
45.12
55.73
52,80
50.08
55,40
52,40
62.04
51.90
5
63.05
55.46
53.42
53.32
50.88
51.08
45.19
56.35
52.79
50.08
54.43
54.34
62.62
54,46
Avg.
65.31
54.88
54.17
51.89
50.67
51.52
45.15
56.17
52.80
50.53
54.37
53.34
61.89
53.40
Sch
62,98
55.00
54.99
53.05
56.27
54.63
53.94
57.44
58.32
56.36
59,07
56.26
56,26
52.46
nol
62.18
56.34
54.81
53.79
57.20
56.24
53.73
57.11
58.27
56.64
59.27
58.53
56.02
53.47
64.30
56.69
55.93
54.81
57.54
56,71
54.26
57.10
58.28
56.51
59.51
57.25
55,36
54.28
63.70
58.15
$7.42
54.78
58.46
57.63
54,10
$7.09
58.25
56.15
59.70
55.66
55.60
55.61
64.61
57.76
57.71
54.89
58.54
57.01
54.12
56.92
58.24
56.75
59.33
56.84
56.11
56.02
Avg. 63,55
56.79
56.17
54,26
57.60
56.45
54,03
57.13
58.27
56.48
59.38
56.91
35.87
54.37
target normal support instances unlabeled instances (test instances). For each benchmark data, randomly created 10 different splits of targer/validation//oure datasets and evaluated the mean test AUC the target datasets.
Comparison methods We compare the proposed method with 13 outlier detection methods: RuL- SIF, SIF [14, D3RE, local outlier factor (LOF) [5), isolation forest (IF) (28), autoencoder (AE) (88. deep support vector description (SD) (3 AE-S, SD-S, fine-tuning methods for AE and SD (AE-FT and SD-FT), RuLSIF-FT, and D3RE-FT. LOF and IF use only target unlabeled instances to find outliers. AE and SD use target normal instances for training. AE-S and SD-S use source normal instances for training. AE-FT and SD-FT fine-tune models trained by AE-S and SD-S with target normal instances, respectively. Note that although AE and SD-based methods can use unlabeled instances as well as normal instances for training, they performed worse than them trained with only normal instances. Thus, we used only normal instances for training. RuLSIF, uLSIF, and D3RE use target normal and unlabeled instances. RuLSIF-FT and D3RE-FT use source normal and unlabeled instances as well as target normal and unlabeled instances. Note that no methods use any information of outliers for training. The details of the implementation such as neural network architectures and hyperparameter candidates are described in the supplemental material.
Results 5 the mean test AUCs with dif- ferent target normal support instance sizes. The pro- posed method showed the best/comparable results for all cases. Density-ratio methods such as RuI.SIF and SI. tended to show better results than the other com- parison methods by using information from both target normal and unlabeled instances. The proposed method was able to further improve performance than RuL- SIF and uLSIF by incorporating the mechanism for few-shot relative DRE. Table 6 shows the results of an Avg.
Table 6: Ablation study for outlier detection. Average test AUCs [%] over different target normal support instance sizes.
No
No
NoSadapt
Data
Ours
Latent
Sudapt
-FT
loT
96.94
97.63
95.17
94.44
Landmine
65.31
63.08
68.83
65.83
School
63.55
63.40
60.04
62.21
75.27
74.70
74.49
74.16
ablation study. The best method can vary across benchmark data since each benchmark data has different properties. For example, in IoT, all datasets relatively similar, which is validated by the fact that test AUCs are high [23] Thus, the dataset-invariant embedding function h in NoLatent is sufficient for adaptation. Nevertheless, the proposed method (Ours) showed the best average AUCs over all benchmark data. In the supplemental material, we additionally showed that the proposed method outperformed the PU learning method (1
5.5 Dependency of Relative Parameter
We investigated the dependency of relative parameter a > in the proposed method. Relative parameter determines the upper bound of relative density-ratio value since Ta (x) < 1 for any

---

Table 7: Investigation of dependency of relative parameter a in the proposed method. Values in relative DRE represent average test squared errors ignoring constant terms over different target support instance sizes and all benchmark data, Values in dataset comparison and outlier detection
represent average test AUCs [%] over different target support instance sizes and all benchmark data.
Ours
Ours
Ours
RuLSIF
RuLSIF
RuLSIF
Problem
a= 0.1
a=5.5
=.9..
a=0.1
a= 0.5
0.9.9
relative DRE
-2.80
-0.83
-0.53
-0.64
-0.63
-0.48
datasct comparison
92.81
95.72
95.03
85.51
85.27
86.34
outlier detection 74.77
75.27
74.98
68.47
68.77
68.69
Table 8: Computation time in seconds for each method on dataset comparison. Ours (train) and RuLSIF-FT (train) represent training time with source datasets for Ours and RuLSIF-FT, respectively. Ours (test), RuLSIF-FT (test), RuLSIF, uLSIF, and MMD represent test time for 100 target dataset comparisons.
Ours (train)
Ours (test)
RuF
uL.II
MMD
RuI-SIF-FT (train)
Rul STF-FT (test)
137.93
0.32
0.38
0.34
0.12
50.32
0.24
X. Table shows results with a = 0.1, 0.5, and 0.9 for all three problems. The proposed method consistently outperformed RuLSIF over different values. This result suggests that the proposed method is relatively robust against the relative parameter value. Note that, in relative DRF, comparison between different a values is meaningless since the scale of the evaluation metrics is different. Besides, various additional results such as investigation of the dependency of the dimensions of latent vectors described in the supplemental material.
5.6 Computation Cost
We investigated the computation time of the proposed method. Table& the computation time of each method for dataset comparison with Mnist-r. We used computer with a 2.20GHz CPU. The support instance size in each target dataset was to five, We omitted D3RE since it requires train the neural network for each target dataset comparison, which is quite time-consuming compared to the others. Although the proposed method took time for training with source datasets, was able to compare datasets with relative DRE as fast as other methods.
Limitations
The proposed method uses multiple source datasets to improve relative DRE performance target datasets. However, when source and target datasets are significantly different, there is a risk of degrading the performance on the target datasets. This phenomenon called "negative transfer". and is a common challenge in general transfer/mett-learnnng methods. Developing methods to automatically remove negative effects of such datasets is one of the important research directions.
7 Conclusion
In this paper, we proposed meta-learning method for relative DRE. We empirically showed that the proposed method outperformed various existing methods in three problems: relative DRE, dataset comparison, and outlier detection. As future work, we plan to incorporate other DRE models such as telescopic DRE (35] our framework. We describe potential negative social impact of our work. The proposed method needs to access datasets obtained from multiple sources like almost all meta-learning methods. When each dataset is provided from different owners, sensitive information in the dataset risks being stolen and abused by malicious people. To evade this risk, we encourage research for developing meta-learning methods without accessing raw datasets.
References
[1] M. Abe and M. Sugiyama. Anomaly detection by deep direct density ratio estimation. 2019.
10

---

[2] M. Bekkar, H. K. Djemaa, and T. A. Alitouche. Evaluation measures for models assessment over imbalanced data sets. J Inf Eng Appl, 3(10), 2013.
[3] L. Bertinetto, J. F. Henriques, P. Torr, and Vedaldi. Meta-leaming with differentiable closed-form solvers. arXiv preprint
2018.
[4] S. Bickel, M. Brückner, and T. Scheffer. Discriminative learning for differing training and test distributions. In ICML, 2007.
[5] M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander. Lof: identifying density- -based local outliers. In SIGMOD, 2000.
[6] B. Dong, Y. Gao, S. Chandra, and L. Khan. Multistream classification with relative density ratio estimation. In AAAI, 2019.
[7] Fang, N. Lu, G. Niu, and M. Sugiyama. Rethinking importance weighting for deep learning under distribution shift. In NeurlPS, 2020.
[8] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.
[9] M. Garnelo, D. Rosenbaum, C. Maddison, T. Ramalho, D. Saxton, M. Shanahan, Y. W. Teh, D. Rezende, and S. A. Eslami. Conditional neural processes. In ICML, 2018.
[10] M. Garnelo, J. Schwarz, D. Rosenbaum, F. Viola, D. J. Rezende, S. Eslami, and Y. W. Teh. Neural processes. arXiv preprint arXiv:1807.01622 2018.
[1]] M. Ghifary, W. Bastiaan Kleijn, M. Zhang, and D. Balduzzi. Domain generalization for object recognition with multi-task autoencoders. In /CCV, 2015.
[12] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012.
[13] A. Gretton, A. Smola, J. Huang, M. Schmittfull, K. Borgwardt, and B. Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning, 3(4):5, 2009.
[14] S. Hido, Y. Tsuboi, H. Kashima, M. Sugiyama, and T. Kanamori. Statistical outlier detection using direct density ratio estimation. Knowledge and information systems, 26(2)309-3336, 2011.
[15] T. Idé, D. T. Phan, and J. Kalagnanam. Multi-task multi-moda models for collective anomaly detection. In ICDM, 2017.
[16] T. Iwata and A. Kumagai. Meta-leaming from tasks with heterogeneous attribute spaces. In NeurlPS, 2020.
[17] T. Kanamori, S. Hido, and M. Sugiyama. least-squares approach to direct importance estimation. The Journal of Machine Learning Research, 10:1391-1445 2009.
[18] M. Kato and T. Teshima. Non-negative bregman divergence minimization for deep direct density ratio estimation. In ICML, 2021.
[19] M. Kato, T. Teshima, and Honda. Learning from positive and unlabeled data with a selection bias. In ICLR, 2018.
[20] D. P. Kingma and J. Ba. Adam: method for stochastic optimization. arXiv preprint 1rXiv:1412.6980, 2014.
[21] R. Kiryo, G. Niu, M. C. d. Plessis, and M. Sugiyama. Positive-unllbeled learning with non- negative risk estimator. In NeurtPS, 2017.
[22] A. Kumagai and T. Iwata. Learning latest classifiers without additional labeled data. In IJCAI, 2017.
[23] A. Kumagai, T. Iwata, and Fujiwara. Transfer anomaly detection by inferring latent domain representations. In NewrIPS, 2019.

---

[24] T. Lane. Machine learning techniques for the computer security domain of anomaly detection. 2002.
[25] J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh. Set transformer: A framework for attention-bac based permutation invaaaaraiiinnnnniiatt neural networks. ICML, 2019.
[26] K. Lee, S. Maji, A. Ravichandran, and S. Soatto. Meta-learning with differentiable convex optimization. In CVPR, 2019.
[27] C. J. Yan, F. Wei, W. Dong, Q. Liu, and H. Zha. Self-paced multi-task learning. In AAAI, 2017.
[28] F. T. Liu, K. M. Ting, and Z.-H. Zhou. Isolation forest. In JCDM, 2008.
[29] S. Liu, A. Takeda, T. Suzuki, and K. Fukumizu. Trimmed density ratio estimation. In NeurIPS, 2017.
[30] S. Liu, M. Yamada, N. Collier, and M. Sugiyama. Change-point detection in time-series data by relative density-ratio estimation. Neural Networks, 43:72-83, 2013.
[31] S. J. Pan and Q. Yang. A survey transfer learning. IEEE Transactions on knowledge and data engineering, (3110)1345-1330 2009.
[32] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.
[33] J. Qin. Inferences for case-control and semiparametric two-sample density ratio models. Biometrika, 85(3):619-66 6300, 1998.
[34] A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine. Meta-leaminn with implicit gradients. In NeurlPS, 2019.
[35] B. Rhodes, K. Xu, and M. U. Gutmann. Telescoping density-ratio estimation. In NeurIPS, 2020.
[36] L. Ruff, N. Görnitz, L. Deecke, S. A. Siddiqui. R. Vandermeulen, A. Binder, E. Müller, and M. Kloft. Deep one-class classification. In ICML, 2018.
[37] T. Sakai and N. Shimizu. Covariate shift adaptation on learning from positive and unlabeled data. In AAAI, 2019.
[38] M. Sakurada and T. Yairi. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MI.SDA 2014 2nd Workshop Machine Learning for Sensory Data Analysis, page 4. ACM, 2014.
[39] B. Schölkopf, A. J. Smola, F. Bach, et al. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
[40] H. Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
[41] J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In NeurIPS. 2017.
[42] M. Sugiyama, M. Krauledat, and K.-R. MÅžller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 9955y)995--1055, 2007.
[43] M. Sugiyama, S. Nakajima, H. Kashima, P. Buenau, and M. Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. NeurIPS. 2007.
[44] M. Sugiyama, T. Suzuki, and Kanamori. Density ratio estimation machine learning. Cambridge University Press, 2012.
12

---

[45] M. Sugiyama, T. Suzuki, S. Nakajima, H. Kashima, P. von Bünau, and M. Kawanabe. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60(4):699-746, 2008.
[46] H. Takahashi, T. Iwata, Y. Yamanaka, M. Yamada, and S. Yagi. Variational autoencoder with implicit optimal priors. AAAI, 2019.
[47] M. Uehara, Sato, M. Suzuki, K. Nakayama, and Y. Matsuo. Generative adversarial nets from a density ratio estimation perspective. arXiv preprint arii: 2016.
[48] V. Vapnik. Statistical learning theory. Wiley, 1:624, 1998.
[49] M. Yamada, T. Suzuki, T. Kanamori, H. Hachiya, and M. Sugiyama. Relative density-ratio estimation for robust distribution comparison. Neural computation, 25(5):1324-1370, 2013.
[50] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. NeurIPS, 2017.
13


Reviews:
[{"review_id": "k0A24el0ZV", "reviewer": "Reviewer_VnfC", "summary": "In this paper, a meta-learning method for relative density ratio estimation is proposed. The proposed method uses a permutation-invariant network to learn a network that outputs the relative density ratio of source and target data, and then uses the learned network to output the relative density ratio of a new source-target pair.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper is well motivated and well written so easy to follow.\nI acknowledge the importance of relative density ratio estimation, its difficulty in a data-scarce environment, and the idea of meta-learning for addressing the problem. As far as I know, this is the first attempt to solve the relative density ratio estimation problem in the meta-learning framework. It is also highly regarded that the proposed method empirically outperforms conventional methods. However, it is unclear what is mechanism enables to transfer of knowledge.\nAs briefly mentioned in lines 144 to 146, it is contributed to the sharing of the network structure, but there is no theoretical or experimental evaluation of when/how it will work and when it will not. There are no guarantees in terms of model selection or learning theory.\nOverall, the present paper lacks theoretical underpinning and the contribution is not enough for acceptance to this venue.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Wh_yaGCVHCY", "reviewer": "Reviewer_Q8XR", "summary": "The paper shows how one can estimate density ratios when very few data points of the two sets of interested are available. In order to do that, the idea is to leverage other datasets that contain related tasks. The ratio is estimated via a linear model to a latent space that represents each dataset.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In general, the paper is well written and the problem is relevant. However, the notation is a bit convoluted and could be improved. For instance, $S_{nu}$ is used to denote target datasets not used for training (Section 4.1), however, the training algorithm itself also uses $S_{nu}$ to denote a subset of one of the training sets.  \n\nMajor comments:\n\n- The results are encouraging overall, however, I miss comparisons to simpler approaches that would be more appropriate for small datasets than kernel-based and nnets methods. For instance, knn (see Kremer, Jan, et al. \"Nearest neighbour density ratio estimation for large-scale applications in astronomy.\" Astronomy and Computing 12 (2015): 67-72.) and (parametric) probabilistic classification-based ratio methods would provide a more fair baseline for Section \"5.2 Relative Density-ratio Estimation\". \n\n- As expected, all methods in Table 2 improve their performance as $N_S$ grows. In particular, MMD starts to give better results for $N_S=4$ (even though it does not use the auxiliary datasets). I think it is important to understand what are the limitations of the method. If $N_S$ increases even further, would the other methods that don't use auxiliary datasets be more competitive?\n\n- Regarding the ablation study (especially Table 4): I struggle to understand how your method without latent vectors (\"No Latent\") is able to learn something meaningful. If the ratio estimator has the shape $w^t h(x)$ (which is what I understand by \"no latent\"), how can such an estimate adapt to the new datasets? In particular, won't it give the same estimated ratio if you swipe the dataset to be used on the denominator with the one to be used on the numerator? Or are w and h retrained with the support instances of the true target datasets? If the answer is the latter, the text and Algorithm 1 are not clear, and a follow-up question would be whether the AUC results are based on data points not used in this last step.\n\n- I also miss a discussion about the intuition behind the method. For instance, how should the auxiliary datasets relate to the target datasets in order for this procedure to work?\n\nMinor comments:\n\n- Why is NoSadapt-FT worse than NoSadapt? Shouldn't it improve NoSadapt because of the fine-tuning?\n\n- The code has not been submitted (and they say it won't be available because it is proprietary), so it is not possible to check all details about the experiments, nor it is possible to reproduce the examples.\n\n- Why is $N_S$ set to 10 on the toy example but to at most 5 in MNIST and ISOLET?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations mentioned in \"Main Review\" were not addressed in the paper; however other limitations were addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "5_JMNS5bkx", "reviewer": "Reviewer_r99y", "summary": "This paper addresses the estimation of the relative density ratio using only a few instances from two datasets.  The authors propose a meta-learning method in which a neural network is trained to output the embeddings reflect the characteristics of two datasets.  In addition, they proposed to use the closed-solution of a linear model on the embedded space.  The experimental results show that the proposed method outperforms conventional methods on dataset comparison problems and outlier detection when a few instances are available.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \nThe few-shot estimation of the relative density ratio is a novel problem setting and valuable for applications.\nAlthough there are attempts to integrate deep neural networks into the estimation of the relative density ratio, the application of meta-learning is also novel for the DNN-based estimation of the relative density ratio.\n\nQuality: \nThe proposed approach is reasonable, and empirical experiments prove its effectiveness.\nThe paper also provides an ablation study to confirm the effectiveness of the specific design choices, such as the representation learning of two datasets.\n\nClarity: \nThe paper is well-organized.\nAlthough most of the paper is clearly written, integrating the closed-form solution of w into the training of DNN and the estimation of the relative density ratio is not clear enough to implement the proposed method.  I think it will be more clear if the details of NoSadapt and NoSadapt-FT are explained.\n\n\nSignificance: \nThe few-shot estimation of the relative density ratio is an important problem for real-world applications.   The proposed meta-learning approach can be a baseline for future work.\nSince the few-shot setting in the experiments is artificial, I would like to see a real-world application.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations described in the Conclusion and the supplemental material are adequate.\n", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "2EdVlO8-cPp", "reviewer": "Reviewer_Yp9Q", "summary": "The authors are proposing to use meta-learning to estimation the density ratio which is a very important problem in machine learning. Density ratio estimation is a very challenging problem as it involves matching an infinite number of statistical moments, which can cause stability issues. The authors have constrained the problem so that they are addressing so that they have limited data from the source domain to estimate the density. The density-ratio is expressed using a permutation-invariant neural network.\n\nThe authors derive the loss function for the density ratio that is represented by a neural network. Then show that the parameters of the neural network can be estimated by splitting the data set into support instances and test instances. The authors have demonstrated its capability for relative density ratio estimation with applications to inlier-based outlier detection. Their results have shown to be superior under all matrix.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the paper is quite well written. The technical part is well explained and well structured. I do think there is some novelty in introducing meta-learning for relative density ratio estimation. My only comment is that there could be stronger experimental results. For example, the authors have compared their results to a number of approach. They have claimed that their approach is a convex optimization, so that there should be no variation in the AUC. However, other approaches may not be convex, it may be good to show some variance in the model after each run, with different initializations or parameters.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews are mostly positive and the general sentiment is that the meta-learning approach (for estimating density ratios when there are few instances) is well-motivated and the results encouraging. There were some concerns of lack of theoretical underpinnings (Vnfc), the method limited in terms of applications (Q8XR), and experiments being artificial (r99y). The authors need to address the last two concerns in a revision, and include the additional experimental support in e.g. Supplementary Materials.
Decision: Accept (Poster)