Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper considers an important problem in denoising diffusion models (DPM), that is, how to accurately conduct posterior sampling given a pre-learned score function and an inverse problem. Previous works address this problem by circumventing the calculation of $p(y|x^t)$ and resorting to projections onto the measurement subspace. However, these works either achieve unsatisfactory performance or are not elegant in terms of theoretical interpretation. Different from prior methods, diffusion posterior sampling (DPS) proposes to approximately compute $p_t(y|x^t)$ for each diffused image $x^t$, and conduct posterior sampling using the discretized version of the following equation
$$dx=[-\frac{\beta(t)}{2}x-\beta(t)(\nabla_{x_t}\text{log} p_t(x_t) + \nabla_{x_t} \text{log} p_t(y|x_t))]dt + \sqrt{\beta{t}} d\bar{w}$$

In summary, the contribution of this work is:
1. It proposes a feasible approach to compute $p_t(y|x_t)$ by using Tweedie's formula and Laplace's method. In particular, 
$$\nabla_{x_t} \text{log} p_t(y|x^t) \sim \nabla_{x_t} \text{log} p_t(y|\hat{x}_0),\quad\text{where}\quad \hat{x}_0 = \text{TweedieFormula}(x_t)$$
2. It experimentally shows that DPS achieves better results than the state-of-the-art algorithms on multiple inverse problems
strength_and_weaknesses: *Strength*:
1) The manuscript is well-written and easy to follow.
2) The figures are illustrative and useful for readers to understand the problem and the approach.
3) The proposed DPS is novel and provides a new way to compute $p_t(y|x^t)$ 
4) Clear experiments with sharp results

*Weakness & Questions*:
1. The connection between Proposition 2 and Theorem 1 is not clear in the text. Please provide more explanation on how Proposition 2 helps derive Theorem 1. I notice the proof in the appendix, what I suggest is to include some high-level explanation.
2. Fig. 3 is very illustrative. However, it can mislead readers to interpret that DPS sticks to each manifold $\mathcal{M}_n$ perfectly, which I don't think is true as multiple layers of approximation are applied. I suggest the authors emphasize in the caption that this is just a conceptual illustration of DPS.
3. In the experiments, performance is only evaluated by using perception loss rather than PSNR/SNR/SSIM. Is it because diffusion models generally have bad performance under these metrics? It will be interesting to the PSNR/SSIM comparison because reconstruction accuracy is much more important than perceptual quality in imaging inverse problems.
4. What is the denoiser of PnP-ADMM? If it is not a deep denoiser, I suggest comparing DPS with a deep denoiser.

*Minors:
1. Eq. 11 and Alg. 1 & 2 are not consistent.
2. Please clarify the definition of $\simeq$.

clarity,_quality,_novelty_and_reproducibility: *Clarity:* well-written and easy to follow
*Quality:* the overall quality of the manuscript is high
*Novelty:* the work is novel
*Reproducibility:* looks like it could be reproduced

summary_of_the_review: I think this paper makes concrete contributions to the literature on diffusion models with application to inverse problems. The method DPS is novel, and the theoretical interpretation in the work is insightful.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: this submission deals with posterior sampling from diffusion models for inverse problems. For general inverse problems with nonlinearity and measurement noise, sampling the conditional posterior needs the conditional score \log \nablap(y|x_t), that is very complicated and can be quite far from Gaussian, at a generic time step t. The idea in this submission is to (first-order) approximate it around the optimal MMSE estimate at each time t, namely \hat{x}_0. Several experiments, provided for linear and nonlinear inverse tasks, show significant quality (FID & LPIPS) improvement over SOTA with this simple approximation.   
strength_and_weaknesses: Strength
- extensive evaluation results and good improvements over SOTA methods
- addressing an important and timely problem
- simple approach but seems effective
- well written for the most parts


Weakness
- The main result in Theorem 1 includes quite strict assumptions that may not hold true for distributions with scores with order beyond linear such as Gaussian. This can really limit the scope of Theorem 1. 

- The result in Theorem 1 seems to be a simple first order approximation, but it’s been derived in a complex manner. The Laplace approximation to end up with the result in Theorem 1 seems not necessary.

- The approximation can be very loose for large noise levels at the later steps of the diffusion process because MMSE estimator will be quite far from the ground-truth x_0


Questions and comments
- The assumptions in Theorem 1 seem to be quite restrictive, which does limit the scope of the results and they don’t even include Gaussian distribution. First of all, what's the intuitive meaning of the assumptions? Second, the authors need to elaborate when these assumptions are satisfied and how limited the scope of the distributions satisfying these assumptions would be.

- in section 3.2. it is not quite clear, how p(y|x_0) is related to p(y|\hat{x}_0)? This seems to be coming from proposition 1, but it's not clear since x_0 and \hat{x}_0 could have completely different distributions. Is that exact or is there an additional hidden assumption here? This needs to be clarified in the paper. 

- The choice of step size seems to be very specific in this work. Did you try other step sizes such as the constant or diminishing step size rule? it seems that the step size rule adopted here is the key to suppress noise? what if one uses the step size coming from eq. 12, namely 1/sigma^2? an ablation on the choice of step size would be needed to clarify the sensitivity of the algorithm to step size.

- better to move tables/algorithms to the top part of each page
- page 3, last paragraph, line 6, is there a typo in p(x|y)? shouldn’t it be p(y|x)?
- page 3, last line, remove “the” from “.... both lines of the works …”


clarity,_quality,_novelty_and_reproducibility: The paper is well written for the most parts. Some steps of derivations need more clarity. 
summary_of_the_review: This paper deals with an important and timely problem. The idea is original, and the experimental results are solid. However, the main result in Theorem 1 seems to be quite restrictive to a small class of distributions that limits the scope of the theorem.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a framework based on denoising diffusion model for solving inverse problem in image processing. The main contribution is the characterization of the forward process for non-linear models which include many well known image formation models. The proposed method has the specificity to both be easier to use than previous models (e.g no SVD) and relies on automatic differentiation for complex computation. The results shows great improvement compare to others methods based on diffusion models. 
strength_and_weaknesses: ## Strength

The proposed methods has several interesting contributions,
1. The forward process is easy to implement and don't ask for heavy operations like SVD.
2. All the framework is well motivated and the explanation are clear.
3. Results are impressive compared to state-of-art.
4. The full framework seems very extensible and then may interest a large part of image processing community.

## Weaknesses

There is no clear weakness. My only regrets is there is no comparison against more classical methods (e.g image denoising with sparse prior and ADMM as solver).
clarity,_quality,_novelty_and_reproducibility: ## Clarity

The paper is straightforward and so very clear. 

## Quality

This is good work.

## Novelty

While the denoising diffusion models are not new, their application to inverse problems is still work in progress. The proposed framework is a clear contribution on this subject and is very novel.

## Reproducibility

All the work is reproducible. The paper is very clear on the parameters, context and networks. Moreover the code is available.
summary_of_the_review: This paper proposes a denoising diffusion process for inverse problems. The main contribution is on the forward process that is well described and easy to implement. The experiments show promising results for image processing problems.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a framework for general Bayesian noisy inverse problems using diffusion probabilistic models. The unknown data distribution is expressed as a posterior distribution. The updated reverse diffusion process now involves a generally intractable likelihood term. The authors propose the use of Laplace approximation of the likelihood term to circumvent the intractability issue. Results are presented that show the usefulness of the proposed framework.
strength_and_weaknesses: The main selling point of the paper is that it proposes a rather straightforward framework for solving general Bayesian noisy inverse problems using diffusion probabilistic models. The numerical results show that by using the framework one can obtain high quality results for a diverse set of problems. It is worth mentioning that the proposed method works on top of existing diffusion probabilistic models.

Its main weakness is that overall, it brings little to the table in terms of new knowledge. The paper is interesting, however, and with the risk of oversimplifying a bit the contributions, it is a collection of existing models and/or techniques. While the use of the Laplace approximation is clever in allowing a straightforward approximation of the likelihood term in the reverse diffusion, it is arguably a standard technique in a statistician and ML practitioner's toolbox.
clarity,_quality,_novelty_and_reproducibility: The paper is well written. The authors correctly place their work in the with respect to existing bodies of work. Sufficient details are given in my opinion to reproduce the results, not to mention that the authors also provide a link to the code. The novelty is somewhat limited, the authors propose more or less an application of diffusion probabilistic models.
summary_of_the_review: The paper is overall interesting, the issue is the novelty factor which is low. The paper could still appeal to a sufficiently large audience. I'm not convinced on accepting the paper, but I'm not convinced on rejecting it either. I'm looking forward to reading the rebuttal on the novelty factor.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper shows how to leverage a pretrained diffusion model as the prior to solve general noisy inverse problems. Combining a pretrained score estimator and an estimator of the gradient of the log likelihood, it arrives at a reverse diffusion sampler for sampling from the posterior distribution. The key technical contribution is how to use the Laplace approximation to better approximate the intractable log likelihood $\log p(y|x_t)$. Empirical results, involving both linear and nonlinear corruptions, show that the proposed method provides an effective solution for a variety of noisy inverse problems. The main weakness is that assembling a collection of existing techniques, the proposed method does not bring much new knowledge on how to leverage a pretrained diffusion model to solve image inverse problems. All four reviewers recognize the contributions of the paper in identifying a key place for improvement on using pretrained diffusion models for solving image inverse problems and achieving state-of-the-art results. 
justification_for_why_not_higher_score: Using standard approximation techniques, the paper provides an incremental improvement of an existing diffusion model-based inverse problem pipeline. 
justification_for_why_not_lower_score: Identifying a key place for improvement under an existing inverse-problem pipeline, the proposed solution is simple and effective.
Decision: Accept: notable-top-25%