Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Given samples from two distirbutions, how can we estimate the KL divergence between them? More generally, how can we estimate the ratio of the two densities at given points (e.g, on the samples from one of the dists)? The authors propose a novel method for density ratio estimation (DRE). DRE is challenging when the two distributions have different supports or the density ratio is very high. The authors propose a solution that builds on the previous work, namely Rhodes et al..,  which instead of computing the ratio directly, it writes it as multiplication of a series of density ratios between some intermediate distributions (telescoping). 

The authors improve over Rhodes et al. by accounting for the distribution shift in a systematic way and also slightly changing the way the intermediate distributions are constructed. The empirical results are promising and show the benefit of the approach compared to the baselines.

main_review: The paper is easy to read and generally well-written. However, parts of the paper seem inaccurate or not very rigorous. 

After offering Lemma 1, the authors conclude that "Lemma 1 shows that a multi-class logistic regression leads to a multi-class density ratio estimator that minimizes the sBD..." However, Lemma 1 just shows the connection between regression and DRE, and does not really talk about "logistic regression". At least in the standard ML literature logistic regression is a linear classifier with logistic loss. Therefore I see no reason that logistic regression work, unless the distributions are nicely separated (like isotropic Gaussians with different means). 

Also, in Lemma 1, $\hat{\eta}$ has not been defined properly. After the statement of Lemma 1, we read "....the reference density C". This is quite confusing, since C seems to be just the number of classes (and not a density). Moreover, $r(x)$ in the lemma is a number so how is $D(r(x)||\hat{r}(x))$ defined?

It will help if the authors formally define the problem of density ratio estimation, and its measure of success. The discussion of the proposed method in section 4 are at times vague.  It will help to have a clear algorithm, pseudocode, etc. rather than just explaining things.

I could not follow Section 4.3, in particular the argument that says replacing log(m/q) with -log(q/m) is useful. I also found the arguments like "TRE is only defined when p<<m<<q..." somewhat sloppy. Why is this true? What goes wrong if m=10q in some parts of the domain? Can you make the argument somewhat more formal? At the end of the same section, the authors make another argument about the benefits of using multiclass logistic regression. However, the connection between consistency of ranking-based noise constrastive estimation and the current paper is not really established. Therefore I don't find the argument well supported.

summary_of_the_review: The paper proposes a novel DRE method which addresses some of the issues that the previous methods (e.g., TRE) had. The paper is easy to read but some of the arguments are weak and not sufficiently justified. More formal arguments/statements will help with the accuracy and readability of the paper. The paper is not theoretically strong, but offers a useful and practical approach which can be impactful.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work estimates the ratio of two densities using intermediate densities that have sufficient overlap with the target densities. The proposed method is based on the scaled-Bregman Divergence and it can be equivalently formulated as a multi-class logistic regression problem. By combining the idea of telescoping, the proposed approach mitigate the issue of FOD/HOD density-chasm and show superior performance on a variety of datasets.
main_review: Strength:

+ The method is clearly motivated, and is theoretically sound.

+ The paper is written clearly.

+ The proposed method shows significant advantages over other methods, especially when both FOD and HOD density-chasm issues are present.

Limitations/questions:

- The novelty of the proposed method seems incremental. Both Lemma 1 and the idea of telescoping are not new.

- Assume that two densities have disjoint support, then wouldn’t the density ratio estimation be a trivial problem as the ratio is always 0 (or infinity) ? Also, even though two target densities overlap, if one is a mean-shifted version of the other (and they are far apart), intuitively logistic regression would give a better classification due to better separation. Would this contradict the challenge of FOD that you claim? I may have misunderstood some setting of the problem so I hope the authors can help clarify it.

- Most experiments are on normal/Gaussian distributions. Very limited results are given on the robustness to other families of well-known distributions. I wonder how well the proposed method performs given more complicated distributions, e.g. mixture of Gaussian (as a single density) or other distributions of multiple modes.


Other comments: 
On page 5, in the definition of Xm, x_p should be x_p^i


summary_of_the_review: In summary, I like the idea of the paper, but most examples and experiments are on Gaussian distributions which may limit the scenarios of real applications. I would encourage the authors to include more results on more “malicious” densities, or at least explain why such suggested experiments do not make sense.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper addresses one of the essential tasks in machine learning, the problem of density ratio estimation (DRE). In this paper, the authors consider applying the binary classification method to DRE. When the samples are well separated, this problem becomes difficult. To tackle this problem, this paper proposes a framework called scaled Bregman divergence. Under this framework, the density ratio can be estimated successfully. The authors also present some applications in experiments.
main_review: Main review:
- This paper attempts to contribute to solving an important problem in the context of DRE.
- The proposed method seems to be an extension of Rhodes et al. (2020), but has a novelty.
- The authors propose a novel concepts: FOD and HOD, which are also interesting and meaningful in this context.

Questions:
1. What is the data-generating process?
2. What is the definition of $D_f$? $D_f(a||b) = B_f(a, b| M)$ for some $M$?
3. What is $\hat{\eta}$ in the denominator of $\hat{r}(x) = \frac{\hat{\eta}(x)}{\hat{\eta}}$? 
4. Can the authors compare their proposed method with more methods, such as plain DRE methods and the method proposed by Kato and Teshima (2021)?

Minor points:
1. I could not understand the following sentence:
> However, closed-form estimation of density ratios is impossible in most problems of interest, as it requires knowledge of the functional form of the underlying densities.
We can approximate the density ratio via a linear-in-parameter model with some kernel function. Combining with the linear-in-parameter model with certain methods yields an estimator with a closed form. For example, the uLISF has a closed form solution in the sense that we can compute the estimator analytically. In the other words, I could not catch the definition of "closed form." 

summary_of_the_review: This paper attempts to contribute to solving an important problem in the context of DRE. While the attempt is novel and very interesting, there is insufficient comparison with existing methods and justification of the proposed method. Therefore, I vote for weak reject. However, depending on the rebuttal, there is a possibility to raise this rating.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposed to estimate the (log) density ratio, $\log [p/q]$ by introducing an intermediate density $m$, and rewrite it as $\log [p/q] = \log [p/m] - \log [q/m]$. The authors then estimate each ratio by a generative classifier (logistic regression). Some simulations were conducted to argue the benefit of the proposed method.
main_review: 1. The descriptions and titles should be improved. 
The paper does not seem to be written in a clear way. For instance, the title in 2.1 is logistic regression but the contents are general results. The title of 2.2 is density chasm but the authors have never explained what it is. 
Also, while the authors kept mentioning 'logistic regression' in a couple of places (such as Sec 4.1), the results are not limited to logistic regression. 
The logistic regression places a linear model of the log-odds (i.e., $\log \eta(x)$ in the paper), which is a restricted approach in some sense. 
Also, there is a typo in Lemma 1 that $\hat r(x)$ should be $\frac{\hat \eta(x)}{1-\hat \eta(x)}$.
Moreover, the quantity $\pi_C$ in Lemma 1 was never defined.
In the first paragraph below Lemma 1, it stated that 'the reference density $C$'. But shouldn't $C$ be the index rather than a density?

2. Misleading statement: "This empirically confirms that, unlike TRE, the SDRE estimator is theoretically guaranteed ..." (Page 5). 
This statement does not make any sense. 
Theoretical guarantee is not from empirical analysis. It should be derived from symbolic analysis (e.g., mathematical derivation).


3. Lack of novelty? 
It seems to me that the only novel part was to use a 'single' intermediate density to estimate the ratio and replace the $\log [m/q]$ by $-\log [q/m]$. 
I don't think this is something that novel. 
sDRE has considered a more general version (a sequence) of this idea (See Sec. 4.3). And replacing the log by -log is an elementary approach that many people did.
The paper has to provide more analysis and justifications on its contribution. 


4. Choice of the measure M needs more analysis.
This part is the key to this paper since the paper is built on a single intermediate density. 
This paper will be improved a lot if the authors can provide a detailed analysis and argument on how to choose a good measure $M$ and even prove some useful theorems (a better convergence rate, an elegant representation, an efficient algorithm).
However, the authors only mentioned three ideas without much analysis on each of them. 


5. Estimation/Learning of $\eta(x)$.
It was never explicitly explained how the authors estimate the class-probability $\eta(x)$. 
From the contents, I guess the authors apply a logistic regression method. But again, $\eta(x)$ can be estimated by other approach such as a tree classifier. Is the logistic regression offers any benefits on the later analysis? Or can we use other generative classifiers? Questions like this should be answered. 


6. A bit of redundant of FOD and HOD.
It was not clear to me why the authors introduce the concept of FOD and HOD. First, these concepts were not well-defined beyond Gaussian. Second, these concepts were not related to the proposed method and were only used in the experiments. Why not just call them 'mean-shift' Gaussian problems and `variance-shift' Gaussian problems?


summary_of_the_review: This paper lacks several important analysis on the proposed methods and the writing was misleading and confusing. The paper requires a lot of work to improve.
review_title: null

Meta-review: The paper introduces a technique to improve density ratio estimation. This is an important problem and very relevant to the ICLR conference. The main idea is to consider density ratios with respect to intermediate distributions to “scale” the densities and make the ratios easier to estimate by training a suitable discriminative model (classifier). Reviewers found the idea interesting but there was a consensus the paper is not ready for publication.
Decision: Reject