Venue: ICLR 2017 conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper provides an exposition of multiple ways of learning in implicit generative models, of which generative adversarial networks are an example. The paper is very clear, the exposition is insightful, and the presented material is clearly important.

It is hard to assess "novelty" of this work, as the individual pieces are not novel, and yet the exposition of all of them in the same space with clear outline of the connections between them is novel.

I believe this work is significant - it provides a bridge for language and methods used in multiple parts of statistics and machine learning. This has the potential to accelerate progress.

I recommend publishing this paper at ICLR, even though it is not the "typical" paper that get published at this conference (in that it doesn't offer empirical validation, nor makes a particular claim about relative merits of different methods).
review_title: An important paper

final_score: rating: null
review_content: The authors present a great and - as far as I can tell - accurate and honest overview of the emerging theory about GANs from a likelihood ratio estimation/divergence minimisation perspective. It is well written and a good read, and one I would recommend to people who would like to get involved in GANs.

My main problem with this submission is that it is unclear what precisely the novelty is - beyond perhaps articulating these views better than other papers have done in the past. A sentence from the paper "But it has left us unsatisfied since we have not gained the insight needed to choose between them.” summarises my feeling about this paper: this is a nice 'unifying review’ type paper that - for me - lacks a novel insight.

So my assessment is mixed: I think this is a great paper, I enjoyed reading it. I was left a bit disappointed by the lack of novel insight, and this is why I’m not highly confident this has to be a conference paper. I am open to be convinced either way.

Detailed comments:

I think the authors should probably discuss the connection of Eq. (13) to KLIEP: Kullback-Leibler Importance Estimation by Shugiyama and colleagues.

I don’t quite see how the part with equation (13) and (14) fit into the flow of the paper. By this point the authors have established the view that GANs are about estimating likelihood ratios - and then using these likelihood ratios to improve the generator. These paragraphs read like: we also tried to derive another particular formulation for doing this but we failed to do it in a practical way.

Typo in Csiszar divergence

Equation (15) is known (to me) as Least Squares Importance Estimation by Kanamori et al (2009).

I have stumbled upon the work of Sugiyama and collaborators on direct density ratio estimation before, and I found that work very insightful. Generally, while some of this work is cited in this paper, I felt that the authors could do more to highlight the great work of this group, who have made highly significant contributions to density ratio estimation, albeit with a different goal in mind.

I think the hypothesis testing angle is a bit oversold in the paper.  I’m not sure what additional insight is gained by mixing in some hypothesis testing terminology. Other than using quantities that appear in hypothesis testing as tests statistics, his work does not really talk about hypothesis testing, nor does it use any tools from the hypothesis testing literature. In this sense, this paper is in contrast with Sutherland et al (in review for ICLR) who do borrow concepts from two-sample testing to optimise hyperparameters of the divergence used.
review_title: Review: Great paper, clear overview of state of theory, but I felt it lacks novel insight.

final_score: rating: 7: Good paper, accept
review_content: Thank you for an interesting read.

Given the huge interest in generative modelling nowadays, this paper is very timely and does provide very clear connections between methods that don't use maximum likelihood for training. It made a very useful observation that the generative and the discriminative loss do **not** need to be coupled with each other. I think this paper in summary provides some very useful insights to the practitioners on how to select the objective function to train the implicit generative model.

The only reason that I decided to hold back my strong acceptance recommendation is that I don't understand the acceptance criteria of ICLR. First this paper has the style very similar to the Sugiyama et al. papers that are cited (e.g. presenting in different perspectives that were all covered in those papers but in a different context), making me unsure about how to evaluate the novelty. Second this paper has no experiment nor mathematical theorem, and I'm not exactly sure what kinds of contributions the ICLR committee is looking for.
review_title: This is a very good paper for connecting methods, but I'm not exactly sure if that fits to the ICLR conference

final_score: rating: 6: Marginally above acceptance threshold
review_content: I just noticed I submitted my review as a pre-review question - sorry about this. Here it is again, with a few more thoughts added...

The authors present a great and - as far as I can tell - accurate and honest overview of the emerging theory about GANs from a likelihood ratio estimation/divergence minimisation perspective. It is well written and a good read, and one I would recommend to people who would like to get involved in GANs.

My main problem with this submission is that it is hard as a reviewer to pin down what precisely the novelty is - beyond perhaps articulating these views better than other papers have done in the past. A sentence from the paper "But it has left us unsatisfied since we have not gained the insight needed to choose between them.” summarises my feeling about this paper: this is a nice 'unifying review’ type paper that - for me - lacks a novel insight.

In summary, my assessment is mixed: I think this is a great paper, I enjoyed reading it. I was left a bit disappointed by the lack of novel insight, or a singular key new idea which you often expect in conference presentations, and this is why I’m not highly confident about this as a conference submission (and hence my low score) I am open to be convinced either way.

Detailed comments:

I think the authors should probably discuss the connection of Eq. (13) to KLIEP: Kullback-Leibler Importance Estimation by Shugiyama and colleagues.

I don’t quite see how the part with equation (13) and (14) fit into the flow of the paper. By this point the authors have established the view that GANs are about estimating likelihood ratios - and then using these likelihood ratios to improve the generator. These paragraphs read like: we also tried to derive another particular formulation for doing this but we failed to do it in a practical way.

There is a typo in spelling Csiszar divergence

Equation (15) is known (to me) as Least Squares Importance Estimation by Kanamori et al (2009). A variant of least-squares likelihood estimation uses the kernel trick, and finds a function from an RKHS that best represents the likelihood ratio between the two distributions in a least squares sense. I think it would be interesting to think about how this function is related to the witness function commonly used in MMD and what the properties of this function are compared to the witness function - perhaps showing the two things for simple distributions.

I have stumbled upon the work of Sugiyama and collaborators on direct density ratio estimation before, and I found that work very insightful. Generally, while some of this work is cited in this paper, I felt that the authors could do more to highlight the great work of this group, who have made highly significant contributions to density ratio estimation, albeit with a different goal in mind.

On likelihood ratio estimation: some methods approximate the likelihood ratio directly (such as least-squares importance estimation), some can be thought of more as approximating the log of this quantity (logistic regression, denoising autoencoders). An unbiased estimate of the ratio will provide a biased estimate of the logarithm and vice versa. To me it feels like estimating the log of the ratio directly is more useful, and in more generality estimating the convex function of the ratio which is used to define the f-divergence seems like a good approach. Could the authors comment on this?

I think the hypothesis testing angle is oversold in the paper.  I’m not sure what additional insight is gained by mixing in some hypothesis testing terminology. Other than using quantities that appear in hypothesis testing as tests statistics, his work does not really talk about hypothesis testing, nor does it use any tools from the hypothesis testing literature. In this sense, this paper is in contrast with Sutherland et al (in review for ICLR) who do borrow concepts from two-sample testing to optimise hyperparameters of the divergence used.
review_title: Review

final_score: rating: null
review_content: Hello Authors,

Congratulations on the acceptance of the paper.

I've just reread parts of the revised paper and noticed a few things that you might want to consider and change before the camera-ready deadline.

* You now include a reference to KLIEP after Eqn. (16), but this procedure is in fact known as least-squares importance estimation.
* in turn, Eqn. (14) is actually more akin to KLIEP, the main difference being the use of the unnormalised form of the KL-divergence. So I think you meant to put the KLIEP reference here.

Further comment on making Eqn. (14) practical:
If you read the KLIEP paper, they formulate the procedure as a constrained optimisation problem:
maximise
$$
E_{p^*} log r_\phi(x)
$$
subject to the constraint that
$$
E_{q_\theta} r_\phi(x) = 1
$$

Compare this constrained optimisation to your solution, it is easy to make the connection: if you introduce a Lagrange multiplier to handle the constraint, one obtains the following unconstrained optimisation problem:

seek stationary points of

$$
\ell(\phi, \lambda) = E_{p^*} log r_\phi(x) - \lambda E_{q_\theta} (r_\phi(x) - 1)
$$

I do think that solving this unconstrained optimisation problem is actually possible, you can do that via stochastic gradient descent, and it does not include your nasty cross-entropy term.

What am I missing?

Thanks,

Rev1
review_title: please correct: references KLIEP in the wrong place + comment on Eqn. (14)

final_score: rating: null
review_content: Thank you for an interesting read. 

I guess all the theory presented are already around but I still like the presentation which walked through each route in a very clear way.

The only question from a practitioner perspective is: which method should I choose when I want to learn an implicit model? Or can you comment a bit on the difficulty of training for different objectives?
review_title: any guidlines for practitioners?

Meta-review: This paper provides a unifying review of various forms of generative model. The paper offers some neat perspectives that could encourage links between areas of machine learning and statistics. However, there aren't specific new proposals, and so there is no empirical evaluation either. The PCs thus believe this contribution is more appropriate for the Workshop Track.
Decision: Invite to Workshop Track