Paper:
Neural Proximal Gradient Descent for Compressive Imaging
Recovering high-resolution images from limited sensory data typically leads to a serious ill-posed inverse problem, demanding inversion algorithms that effectively capture the prior information. Learning a good inverse mapping from training data faces severe challenges, including: (i) scarcity of training data; (ii) need for plausible reconstructions that are physically feasible; (iii) need for fast reconstruction, especially in real-time applications. We develop a successful system solving all these challenges, using as basic architecture the recurrent application of proximal gradient algorithm. We learn a proximal map that works well with real images based on residual networks. Contraction of the resulting map is analyzed, and incoherence conditions are investigated that drive the convergence of the iterates. Extensive experiments are carried out under different settings: (a) reconstructing abdominal MRI of pediatric patients from highly undersampled Fourier-space data and (b) superresolving natural face images. Our key findings include: 1. a recurrent ResNet with a single residual block unrolled from an iterative algorithm yields an effective proximal which accurately reveals MR image details. 2. Our architecture significantly outperforms conventional non-recurrent deep ResNets by 2dB SNR; it is also trained much more rapidly. 3. It outperforms state-of-the-art compressed-sensing Wavelet-based methods by 4dB SNR, with 100x speedups in reconstruction time.

Review:
While my concerns were given significant attention in the rebuttal, I feel they were not fully addressed. In particular, regarding comparison with deep ADMM-net and LDAMP, the authors argue that these methods need more training data/training time.  However, training time is normally not a big issue (you only train your model once, does it matter if it takes 2 hours or 10?). The *size* of the training data is however important, but no experiments are provided to show superior performance of the proposed method with respect to the the size of training data. This is surprising given that in l. 62. the authors say they use "much less training data" (addressing the challenge of "scarcity of training data" mentioned in l.4 in abstract), without referring back to this claimed contribution anywhere in the paper!  Additionally, for the additional super resolution experiments in Table 1 of the rebuttal, I am left again confused. Instead of reporting on a common SR dataset (like one of the datasets SRResnet[12] reports on), they authors report on CelebA and compare with their own implementation of SRResnet on this dataset. The authors do not show a standard baseline, like Bicubic interpolation, meaning the context for these numbers is missing. When looking up in the literature for results on CelebA 8x upscaling (32x32 -> 128x128), (Wavelet-SRNet: A Wavelet-based CNN for Multi-scale Face Super Resolution, Huang et. al., ICCV 2017) gives wildly different numbers. There we see Bicubic having 29.2dB PSNR (higher than the 28.X reported in the rebuttal!), but SSIM of the s.o.t.a. being 0.9432 (lower than the 95.07 SSIM in the rebuttal...). Either the setting is somehow different, or the experiments are somehow flawed, but it is hard to say given that the authors show no standard baselines!  On the other side, Reviewer 3 gives a quite different perspective on the paper, focusing on the contraction analysis and the accompanying experiments. While I'm not qualified to comment on the correctness/applicability of this part of the paper, I think Reviewer 3 has valid points that such analysis are valuable in general and should be more often done.  ============================================================ The paper proposes an inversion algorithm for ill posed linear measurements of images. That is, from an image x, we are given few (compared to dim(x) )  (noisy) linear measurements from which we want to recover x, relying on the assumption that x lives in some low dimensional manifold. This problem encapsulates for example MRI/CT imaging as well as super-resolution.  The proposed approach is an adaptation of the proximal gradient algorithm for deep networks. While I'm not an expert on proximal learning, my understanding is that the authors take the iterative procedure of proximal learning (eq (2)) and swap out the proximal operator (which has closed form solutions for simple priors but otherwise requires solving a convex optimization problem at each iteration), with a deep network. The intuition is that normally the proximal operator can be viewed as taking a step towards the manifold/prior, which is an operation a DNN could in principle learn.  This means that instead of deriving the operator from the (known) prior, it needs to be learned from data, motivating the loss (P1) in l. 133.   However, this loss is not very clearly explained. Why do we need beta and the (second) measurement consistency term? How does that relate to classical proximal learning? Why not just ask different iterations to also reconstruct the image? How does this term come into the contraction analysis?  When the authors jump to the neural proximal training, they essentially just obtain a constrained recursive image restoration network (in the sense that the relationship between x_t and x_t+1 is the composition of a generic DNN and a fixed linear operator g (that depends on the measurement matrix)).  Previously recursive approaches for image restorationhave been studied, such as ( Deeply-Recursive Convolutional Network for Image Super-Resolution, Kim et al. CVPR 2016 ) and the authors should discuss the differences. Aside from the fixed g, the training losses differ, but again the loss does not have a clear motivation from proximal learning theory as far as I understand.  Experimentally, the paper could also be more convincing.  The MRI experiments, while impressive, seem to be done on a private dataset with limited comparisons. Why can't the authors e.g. compare with (Deep ADMM-Net for Compressive Sensing MRI, Sun et al, NIPS 2016) or  ( Learned D-AMP: Principled Neural Network Based Compressive Image Recovery, Metzler et al, NIPS 2017)?  The Super-Resolution experiments (a field I know well) are extremely limited. The authors claim they only do them to analyse the contraction properties, but do not explain why they need to switch to SR and cannot do this for the MRI experiments. For me, the experiments rather rather raise the question on why the method works so poorly for SR, since this is a problem that falls within the problem formulation (linear measurement matrix). Note I am not saying the authors need to be state of the art in SR, but at least they should report results in a standard manner (e.g. on standard datasets such as Set14, see e.g. Image Super-Resolution Using Dense Skip Connections, Tong et al, ICCV 2017 for a recent s.o.t.a. SR paper).  Overall, I think the goal of the paper is worthy (generalizing proximal algorithms to deep net), and the paper is well written in general. My main concern is the execution, both in terms of how the proximal method actually motivates the training objective and the overall experimental evaluation.   Minor Comments: l 105: the definition of g is just the argument to the proximal operator in eq (2). I would either define g earlier or refer to (2) in l105. l 129: Shouldn't it be \hat{x}_i = x_i^T instead of x_T^i?