Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper investigates the performance of invertible generative models for solving inverse problems. They argue that their most significant benefit over GAN priors is the lack of representation error that (1) enables invertible models to perform well on out-of-distribution data and (2) results in a model that does not saturate with increased number of measurements (as observed with GANs). They use a pre-trained Glow invertible network for the generator and solve a proxy for the maximum likelihood formulation of the problem, where the likelihood of an image is replaced by the likelihood of its latent representation. They demonstrate results on problems such as denoising, inpainting and compressed sensing. In all these applications, the invertible network consistently outperforms DCGAN across all noise levels/number of measurements. Furthermore, they demonstrate visually reasonable results on natural images significantly different from those in the training dataset.

The idea of using invertible networks for estimating a specific forward process is not new, as the authors also pointed out. The contribution of this paper is that they use a pre-trained invertible model as a prior in various tasks not known in training time and support their technique with experimental results and therefore I would recommend accepting this paper. 

Since one of the main arguments in the paper is how the lack of representation error benefits the Glow prior compared to DCGAN prior, it would be interesting to see the representation error quantitatively for the DCGAN results and how it contributes to the total error.  Moreover, demonstrating the comparison results in other metrics than PSNR (MSE, SSIM) would be interesting and more comprehensive.
review_title: Official Blind Review #4

final_score: rating: 3: Weak Reject
review_content: Authors extend the invertible generative model of Kingma et. al. to image inverse problems. Specifically, they use Generator trained within the Glow framework as an image prior for de-noising, inpainting and compressed sensing tasks.
During training, a heuristic adjustment to the objective is made allowing optimization of latent variable norm instead of image log likelihood. This seemed critical for convergence of image inverse tasks. The use of Glow prior was shown to be beneficial for all inverse tasks. Experiments were limited to face images from celebA database. While the proposal demonstrates improved empirical performance, it seems to be the only contribution of this paper. Taking an existing model and applying it to a problem where similar extensions have been tried (GAN etc) does not seem quite worthy of a full paper.
review_title: Official Blind Review #2

final_score: rating: 1: Reject
review_content: This paper proposes to employ the likelihood of the latent representation of images as the optimization target in the Glow (Kingma and Dhariwal, 2018) framework. The authors argue that to optimize the ''proxy for image likelihood'' has two advantages: First, the landscapes of the surface are more smooth; Second, a latent sample point in the regions that have a low likelihood is able to generate desired outcomes. In the experimental analysis, the authors compare their proposed method with several baselines and show prior performance.

This paper has three major flaws and should be clearly rejected. 
First, the novelty of this paper is trivial, in my opinion, the Eq. 2 is the only contribution of this paper.
Second, the experimental results are not convincing, almost all the methods proposed after 2015 have better performance compared to these baseline methods.
Third, there are a lot of claims in this paper have been made without clarification, I have huge troubles in understanding certain sentences.
 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Update: I have read the other reviews and the author response and have not changed my evaluation.

Recent work has shown that GANs can be effective for use as priors in inverse problems for images such as compressed sensing, denoising, and inpainting. A drawback is that GANs may have the problem of inexact reconstruction, and strongly reflect the biases in the training set yielding poor performance on out-of-distribution data. This paper shows that the exact inverses available to normalizing flow models and their broad assignment of likelihoods allows for better reconstructions especially on out-of-domain data.

As far as I know, this is the first work to use normalizing flows for inpainting and compressed sensing. The approach and application is very natural, although it’s a bit surprising that using the likelihood as a prior term directly did not work very well. The results of this work show that invertible generative models have utility for inverse image problems even when the quality of raw samples is substantially below GANs. In my opinion the main advantage in this method is not on having low reconstruction error on observed pixels, which becomes less of a problem for more powerful GAN models, but rather the good performance on out of domain data which is somewhat surprising. The authors are reasonably thorough, testing their model on a variety of problem settings and perform ablation studies on hyperparameters.

As additional baselines for compressed sensing and denoising, it would be good to compare to the Deep Image Prior since there is effectively no out-of-distribution input for this untrained model and it performs well with moderate image corruption. Additional discussion about the two could be useful, as for the Deep Image Prior a similar patter is observed where denoising requires explicit regularization (early stopping or gradient noise for DIP) but image completion and compressed sensing do not. Also, there have been many improvements to DCGAN over the years that might ameliorate the problems that were observed in reconstruction, but I don’t fault the authors much for this as it can be difficult training models like StyleGAN even at 64x64 sizes.

It might also be interesting to know whether the good performance on out-of-distribution inputs is due to the exact invertibility or the log-likelihood objective, although I would guess that it is the latter. On way to test this would be training the GLOW model with an adversarial objective instead of NLL as done in [1].

Minor Comments:

Figure 4 would probably be better with a logarithmic scaling for # of measurements

I did not understand the comment about sublevel sets of the data misfit term being inverse images of cylinders, maybe this could use some elaboration.

[1] https://arxiv.org/abs/1705.08868

review_title: Official Blind Review #3

Meta-review: This paper studies the empirical performance of invertible generative models for compressive sensing, denoising and in painting. One issue in using generative models in this area has been that they hit an error floor in reconstruction due to model collapse etc i.e. one can not achieve zero error in reconstruction. The reviewers raised some concerns about novelty of the approach and thoroughness of the empirical studies. The authors response suggests that they are not claiming novelty w.r.t. to the approach but rather their use in compressive techniques. My own understanding is that this error floor is a major problem and removing its effect is a good contribution even without any novelty in the techniques. However,  I do agree that a more thorough empirical study would be more convincing. While I can not recommend acceptance given the scores I do think this paper has potential and recommend the authors to resubmit to a future venue after a through revision.
Decision: Reject