Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: In this paper, authors propose a continual learning for density-ratio estimation. The formulation of the CDRE Eq.(5)is quite intuitive, and it makes sense. Then, a log-linear model is employed for a density-ratio model, and it is estimated by using a density-ratio estimation algorithm. Through experiments, the authors show that the proposed algorithm outperforms existing methods.

The paper is clearly written and easy to read. The density-ratio estimation algorithm for continual learning is new and interesting.

Detailed comments:
1. I am pretty new to the continual learning. The formulation of CDRE is interesting. However, I am still not that certain whether the setup is realistic. In the introduction, authors describe that we cannot obtain data points due to privacy or limited cost budget. More specifically, if it is a privacy issue, we may not be able to use the model trained by the private data as well. Also, could you give me a couple of examples of the limited cost budget case?

2. In this paper, authors employed the log-linear model. If we use another model, performance can be changed?


review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: ######### Updated Review ###########

I'd like to thank the author(s) for their rebuttal. However, I am still on the same boat with R1 and recommend rejection for this submission. 


################################


This submission seeks to evaluate generative models in a continual learning setup without storing real samples from the target distribution.  The main technique the author(s) have used is the likelihood-ratio trick. I question the scope of this paper, as this is not a topic of general interest to the community. Additionally, the density ratio estimation technique is fairly standard. I vote to reject this submission for the lack of highlights and relevant potential applications. 

My main argument for rejection. 
While continual learning is a trendy topic in the AI community, it's less well-received in the context of generative modeling, probably for the lack of real applications. Such works, including this one, fail to address any real challenge, as the hypothesized scenario is unrealistic. For example, I am not convinced of the significance of using f-div to evaluate model performance. And since importance sampling is notorious for its variance issues (the essential mathematical tool used in this model), the estimate is not expected to be reliable, say subsequent tasks q_t and q_{t-1} differ somehow. This submission feels more like playing a game with the rules defined by the author(s), not driven by practical considerations. 


review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: This submission proposes a continual density ratio estimation method and suggested its application to evaluate a continually learned generative model without requiring the previous training data. The basis of the continual density estimation is based on a recursive relationship between the density ratio at step t and that at t - 1. Continually estimating the density ratio without storing the raw data is an interesting topic and could be useful for continual learning. To my knowledge, I have not seen this in earlier publications.

However, I give reject to this paper because of the following reason: 

The writing of this paper is not easy to follow. 

- The beginning of section 3 (CDRE in continual learning), I found it difficult to understand why the model q needs to be updated (indexed by t) while p(x) is not dependent on t. As far as I know, under the continual learning setting the data distribution p(x) is also conditioned on t. I interpret it as a general introduction on how density ratio could be estimated continually.
- The Lagrange multiplier and the bias / variance statements need elaboration, I don't understand how it is affecting the bias and variance.
- In the second part of section 3, the continual learning setting is introduced (in equation 11), however, it is no longer reasonable to use the symbol r_t in equation 12 which was initially defined in equation 5. 
- A loss for continual VAE is proposed in seciton Feature generation for CDRE, however, the p(x) is again independent of t. And I'm also suspicious that equation 13 is the correct way of adjusting VAE's objective with VCL. In VCL, the KL divergence is on the parameter distribution, which could help prevent forgetting, however, here the KL is between VAE's approximate posteriors, which alone is not sufficient for keeping the information of previous tasks.
- There's lack of analysis / interpretation of results for section 4.1, e.g. what is the motivation of the experiments and what is the conclusion.
- Through out section 4.2 - 4.3, it is not explained what is the source of variance in the experiment results.

review_title: Official Blind Review #1

Meta-review: The paper seems technically correct and has some novelty, but the relevance of the paper is questionable. Considering the selectiveness of ICLR, I cannot recommend the paper for acceptance at this point. 

In more detail: the authors propose a technique for estimating density rations between a target distribution of real samples and a distribution of samples generated by the model, without storing samples. The method seems to be technically well executed and verified. However, there was major concerns among multiple reviewers that the addressed problem does not seem relevant to the ICLR community. The question addressed seemed artificial, and it was not considered realistic (by R2 and also by R1 in the confidential discussion). R3 also expressed doubts at the usefulness of the method. 

Furthermore, some doubts were expressed regarding clarity (although opinions were mixed on that) and on the justification of the modification of the VAE objective to the continual setting. 
Decision: Reject