Paper:
Deep neural networks provide state-of-the-art performance for image denoising, where the goal is to recover a near noise-free image from a noisy image. The underlying principle is that neural networks trained on large datasets have empirically been shown to be able to generate natural images well from a lowdimensional latent representation of the image. Given such a generator network, or prior, a noisy image can be denoised by finding the closest image in the range of the prior. However, there is little theory to justify this success, let alone to predict the denoising performance as a function of the networks parameters. In this paper we consider the problem of denoising an image from additive Gaussian noise, assuming the image is well described by a deep neural network with ReLu activations functions, mapping a k-dimensional latent space to an n-dimensional image. We state and analyze a simple gradient-descent-like iterative algorithm that minimizes a non-convex loss function, and provably removes a fraction of p1´Opk{nqq of the noise energy. We also demonstrate in numerical experiments that this denoising performance is, indeed, achieved by generative priors learned from data.

Review:
The paper analyzes the recovery accuracy of a "tweaked" gradient descent algorithm for imaging denoising and compressive sensing under deep generative priors. In particular, when assuming Gaussian randomness of the network weights and extremely stringent conditions of network sizes, they demonstrate a specific denoising rate of O(k/n), with k and n being the input and output dimension of the generative network. This is seemingly optimal in terms of the dependence on the latent code dimensionality and the signal dimensionality and is the first result of this kind. 
Two papers are closely related, but are not sufficiently discussed in the introduction. [Bora et al., 2017] does not require Gaussian randomness of the network weights, but achieves only O(1) error bound assuming the empirical risk minimization problem can be solved to optimality.  [Hand & Voroninski, 2018] showed that under same assumptions as in this paper, the nonconvex empirical risk minimization problem exhibits a nice geometric landscape - no spurious stationary points. This implies that virtually anything reasonable would converge to global optimum. Combing both facts, it is not surprising to arrive at the results in this paper.
While the paper makes some novel theoretical contributions, two concerns stand out. First, there is a lack of intuition or justification of the tweak in gradient descent - flipping the sign of the iterate at times.  The author argued that around approximately -x*, the loss function is larger than around the optimum x* . So simple gradient descent is likely to get stuck in this region, so the negation check is needed. I am not so convinced by the argument. There could be other critical points that are not necessarily in the negative regime of true optimal, right?  So why would this be sufficient or necessary for global convergence? Second, even ignoring the unrealistic Gaussian assumption on the network weights, the theorem requires very narrow regimes for the expansivity condition and the noise variance bound. It's hard to verify whether these conditions can be satisfied at all. 
The experiment on denoising with learned prior from MNIST data is interesting, as it suggests that the theoretical assumptions are not necessary in practice to observe the optimal recovery rate. It would be more convincing if more experiments are provided, especially for the compressive sensing application.