Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper presents an amortised MAP estimation method for SR problems. By learning a neural network which learns to project to an affine subspace of SR solutions which are consistent with the LR method the method enables finding propoer solutions with by using a variety of methods: GANs, noise assisted and density assisted optimisation.
Results are nicely demonstrated on several datasets.

I like the paper all in all, though I feel the writing can be polished by quite a bit and presentation should be made clearer. It was hard to follow at times and considering the subject matter is quite complicated making it clearer would help. Also, I would love to see some more analysis of the resulting the networks - what kind of features to they learn? 
review_title: Interesting paper

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: Sincere apologies for the late review.

This paper argues to approach Super-Resolution as amortised MAP estimation. A projection step to keep consistent HR-LR dependencies is proposed and experimentally verified to obtain better results throughout. Further three different methods to solve the resulting cross-entropy problem in Eq.9 are proposed and tested. 

Summary: Very good paper, very well written and presented. Experimental results are sufficient, the paper presents well chosen toy examples and real world applications. From my understanding the contributions for the field of super-resolutions are novel (3.2,3.3,3.4), parts that are specific for the training of GANs may have appeared in different variants elsewhere (see also discussion). I believe that this paper will be relevant to future work on super-resolution, the finding that GAN based model training yields most visually appealing results suggests further work in this domain. 

Manuscript should be proof-read once more, there were some very few typos that may be worth fixing.
review_title: Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper presents a new framework to solve the SR problem - amortized MAP inference and adopts a pre-learned affine projection layer to ensure the output is consistent with LR. Also, it proposes three different methods to solve the problem of minimizing cross-entropy. Generally, it is a great paper. However, I still have several comments:

1) The proposed amortized MAP inference is novel and different from the previous SR methods. Combined with GAN, this framework can obtain plausible and good results. Compared with another GAN-based SR methods - Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network, question may arise as to what this new formulation adds to the latest state-of-the-art.

2) Using an affine projection architecture as a constraint, the model do not need any corresponding {HR, LR} image pairs for training. However, when training the affine projection layer, we still need the {HR, LR} image pairs. Does it mean that we merely transfer this training procedure to the training of affine projection?

3) The paper presents many results of the framework, including the results of natural images from ImageNet. Can the author also provide the results of Set5, Set14 or BSD100, which are conventional test dataset for SR, so that we can perform a fair comparison with previous work.

4) I see that the size of the results of nature images presented in this paper are limited to 128*128. Can this framework perform well on images with larger size? Because SR will encounter input with arbitrary size.

5) A normal GAN will have a noise term as a latent space, so that it can be better illustrated as learning a distribution. Do the author try the noise vector?

Overall, this paper provides a new framework for SR with solid theoretical analysis. The idea is novel and the author explore many methods. Though there still exist questions like the necessity and more experiments are needed. I think this work will will provide good inspiration to the community.
review_title: Novel methodology

Meta-review: All the reviewers agreed that the paper is original, of high quality, and worth publishing.
Decision: Accept (Oral)