Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper introduced a new approach called Pseudoinverse-guided Diffusion Models to deal with problem agnostic image inverse problems. Based on the recently popular diffusion models, it added the Pseudoinverse guidance by using a problem agnostic model S_theta(x;sigma_t) to approximate the score. Then the authors shows for 4 inverse problems their method performed on-par with the problem-specific method
strength_and_weaknesses: Strength:

The paper is well written with a good story flow, making it much easier to follow for the audience.
The method is well motivated by the lack of problem-agnostic inverse methods and provided their solution by approximating the guidance term using the Pseudoinverse of the measurement matrix.
The ablation study of swapping the Psy-G with AW is solid evidence of Psy-G contributing to the performance robustness.

Weakness:
1. Although the problem motivation is clear, to me it has some problems: The solution is not really "problem agnostic." The Pseudoinverse guidance involves informing the algorithm of the measurement function H or h(x). Is it difficult to come up with such a matrix for different inverse problems?

2. How is the time performance (both training and testing) of such an algorithm compared to the baselines?

3. How many times are the experiments repeated? How many test samples are combined to report the FID scores in Table 2&3. Is there a standard deviation to be reported?
clarity,_quality,_novelty_and_reproducibility: Overall, the paper is of high quality and clear. 

It is novel from my limited knowledge to the newest literature. 

The reproducibility seems good with the code.
summary_of_the_review: Not knowledgeable enough about the recent publications in the area, but the paper is novel and technically solid for me. Therefore, I recommend "accept."
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes Pseudoinverse-Guided Diffusion Models, which uses problem-agnostic diffusion models to reach the empirical performance of problem-specific ones. The proposed approach directly estimates conditional scores without additional training. In particular, it can address inverse problems with noisy and non-linear measurements, in contrast to existing approaches that are limited to noiseless linear ones. The authors illustrate the empirical performance of the proposed model on different image restoration tasks.  

strength_and_weaknesses: Strength: (1) The work is overall well-written and joyful to read. The contributions of this work are clearly stated. (2) In general, the idea of this work is natural and interesting. The authors provide enough material to support their idea. 
(3) The experiments are solid and complete, including super-resolution, inpainting, and JPEG restoration. The experimental results are encouraging and well support the effectiveness of the proposed approach. 

Weakness: (1) It is not straightforward for the readers to know why the authors can approximate the score function via Eq. (3), Eq. (5), and Eq. (6). The authors should provide more details and insights.
(2)  In Eq. (6), the authors derive the approximation of the score function. A natural question is how tight this approximation is. It seems that there are no guarantees. 
(3) It seems that this method is a heuristic approach. 

clarity,_quality,_novelty_and_reproducibility: The work is well-written and joyful to read.
In general, the idea of this work is natural and interesting. 
The reproducibility can be improved if code can be released.

summary_of_the_review: The work overall is interesting and well presented.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Pseudoinverse-guided Diffusion Models ($\Pi$GDM), that use pretrained diffusion models (problem-agnostic) to solve inverse problems. The proposed $\Pi$GDM works with noisy, non-linear, or even non-differentiable measurements, and demonstrates promising results.
strength_and_weaknesses: ## strengths
- The paper is well-written and well-organized.
- The problem of using pre-trained diffusion model to solve inverse problems is of great interest to the community. The paper demonstrates promising results and the discussion of related works is comprehensive.
- The proposed pseudoinverse guidance has nice properties that can work with noisy, non-linear, or even non-differentiable measurements.
## weakness and questions
- Minor issues:
	- Table 1 is not referred to in the main text.
- Questions:
	- Moore-Penrose: is there a specific reason to employ Moore-Penrose Pseudoinverse here? Is it possible to consider another more general pseudoinverse?
	- Adaptive weights: how does the adaptive weight compare to the weight used in [1], $1/||y-H \hat{x}_t||$?
	- Non-linear extension: the paper discussed two examples of non-linear H, is it possible to extend it further to neural networks like a contrastive encoder, a classifier, or an autoencoder?

[1] Chung, Hyungjin, Jeongsol Kim, Michael T. Mccann, Marc L. Klasky, and Jong Chul Ye. 2022. “Diffusion Posterior Sampling for General Noisy Inverse Problems.” arXiv [stat.ML]. arXiv. http://arxiv.org/abs/2209.14687.
clarity,_quality,_novelty_and_reproducibility: - The paper is clearly and well-written.
- The presented method is novel.
- Pseudo-code is provided in the Appendix.
summary_of_the_review: Good paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposes a pseudoinverse-guided diffusion model for solving (nonlinear) inverse problems using model-agnostic pre-trained score-based diffusion model (SDM) networks. The key component of the proposed in $\Pi$GDM is the approximation of the conditional probability $p_t(y|x_t)$, which is based on Tweedie's formula and Gaussian assumption. With known $p_t(y|x_t)$, $\Pi$GDM can the term as guidance to guide the generation of the image.

In summary, I think the contributions of the work are:
1. Propose an approach to approximately calculate $p_t(y|x_t)$.
2. Demonstrate the efficacy of the model on various (nonlinear) inverse problems, including JPEG restoration, image impainting, and image super-resolution.


strength_and_weaknesses: **Strength:**
1. Open code and data
2. Thorough supplement
3. Proposal of a new approach to compute $p_t(y|x_t)$

**Weakness:**

*Statement:*
1. The statement "To the best of our knowledge, ΠGDM is the first approach based on problem-agnostic models to achieve this quality on ImageNet." might be too ambitious. I found a concurrent paper (https://arxiv.org/abs/2209.14687) that also uses model-agnostic DPM for solving the general inverse problem and achieves very good results. A thorough comparison with existing methods needs to be presented to support this argument.
2. I am not very clear what are the contributions of this work by reading the manuscript. I felt that using 'guidance' to condition DPM has already become a common strategy and many papers have investigated this problem. Perhaps it is due to my misunderstanding, but the text seems to imply that making model-agnostic pre-trained DPM work for specific inverse problems is one novelty/contribution tied to the paper.

*Methodology:*

3. The underlying assumption for ΠGDM to approximate $p_t(y|x_t)$ is that $p_t(x_0|x_t)$ is Gaussian, which is strong. For example, assume $p_t(x_t|x_0)$ is Gaussian and $p(x_0)$ is Gaussian as well. According to Baye's rule, 
$$p(x_0|x_t) = \frac{p_t(x_t|x_0)p(x_0)}{p(x_t)}$$
for which a Gaussian approximation is too strong.

4. Therefore, the practicality of the model might be severely limited.

*Experiment:*

5. In the experiments, the baseline methods are restricted to model-specific methods, failing to include other model-agnostic methods such as plug-and-play prior (PnP). It will be also helpful to see if ΠGDM improves over existing model-agnostic methods.
clarity,_quality,_novelty_and_reproducibility: The manuscript is not clear about the contributions of the model. Some statements are too general.
summary_of_the_review: I have some concerns about the technical correctness of the proposed method. I am looking forward to the authors' clarification.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Given a pretrained difffusion model, the paper proposes Pseudoinverse-guided Diffusion for solving inverse problems.

 Authors show the merit of the methods on mage restoration tasks, including super-resolution, inpainting and JPEG restoration and compare it to state of the art methods designed for these particular task and show that their method is competitive with SOTA. 

The paper was discussed in an  email thread with the reviewers. Using generative models in inverse problems is not new however authors specialize  it to diffusion model and devise few approximations and heuristics to make the likelhood of the observation given the input tractable. During the rebuttal authors clarified the heuristic used and the reviewers were satisfied with the rebuttal.  
justification_for_why_not_higher_score: Using diffusion models in inverse problems is not new but  authors showed a new simple projection method that is competitive  with SOTA 
justification_for_why_not_lower_score: The simplicity of the approach and the good results benefits ICLR community.
Decision: Accept: poster