Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to synthetize and edit realistic images based on stocahstic differential equations (SDE). The method is based on diffusion generative model that iteratively denoises through SDEs and demosntrates that this SDE prior increases photorealism of the images, compared to state-of-the-art GAN-based approaches. 
main_review: Stroke-based image editing algorithms have been widely-adopted and popular for many artistic design tools and softwares. Many of those are (conditional) GAN-based methods, and can be often challenging to balance the tradeoff between use control vs. result faithfulness. 

The proposed method has a novel technical contribution, that utlizes iterative SDE denoising for guided stroke sketch input + perturbed noise, with photorealistic output via the reverse of the SDE. The paper has compared with a few state-of-the-art baselines and demosntrate its effectiveness in producing more faithful editing results.

One question I have is that, how good the guided stroke sketch needs to be? I don't see ablation study on this, but I think it is an important factor for practical applications. 

Paper writing and references seem fine to me. 
summary_of_the_review: Overall I am leaning towards a weak accept. The paper has clear novelty regarding stroke-based image editing via perturb + reverse SDEs. Comparisons and ablation study look good to me except for the one on the level of user sketch control.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposesed an image editing and synthesis system (SDEdit) with SDE. By injecting the guided image into the reverse process with appropriate $t_0$, the results can have a trade-off between realism and faithfulness.
main_review: Strengths:
The whole idea is interesting, effective and simple. The SDEdit perturbs the input image with the forward SDE and generates edits by reversing the SDE, the SDE is pretrained on unlabeled data, so the SDEdit does not require collecting pair images. Meanwhile, since the excellent generation performance of SDE, the SDEdit can generate the content with high quality. The mask strategy can help the user edit the regions where they want.

Weaknesses:
The SDEdit can not edit the image flexibly enough, such as closing eyes, opening mouth, changing lip color. Meanwhile, the guidance can not influence the results well sometimes (Fig.1 Stroke-based Editing), there is no brown forest in the region where the user draws the brown stroke. Finally, the Stroke-based Editing is not complex and some methods can do this such as [1][2], the SDEdit should compare with them.

[1]DeFLOCNet: Deep Image Editing via Flexible Low-level Controls. CVPR2021

[2]Sc-fegan: Face editing generative adversarial network with user’s sketch and color. ICCV 2019
summary_of_the_review: The paper is well-written and easy to follow. The results are impressive. Although Stroke-based Editing is not complex, especially on the face,  I think this is a good start of using the SDE to do the image editing task.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposed Stochastic Differentiable Image Editing method which iteratively diffuse the human edited image  to real images, using the pretrained score-based SDE models (the edit is in a form of manipulating image pixel values, e.g. stroke, color paint, or image patch).The method achieves both realistic image editing and faithfulness to human input through user studies. 
main_review: Strength:
1. The SDE formulation of image editing is an interesting approach, it's not only able to generate image with realism (through diffusion process) but also capture the real input (by controlling the time variable t0 that runs the diffusion process).

2. The proposed method is able to produce a diverse set of predictions given user input by sampling different Gaussian noise in the process. 

3. The paper is well written and easy to follow.

Weakness

1. The paper is overclaimming. It is argued that the proposed method is able to diffuse the image "given a user guide of any type", however, it seems the proposed method is only able to handle the user guide in a form of manipulating pixel rgb values (strokes, color patch, or image patch), there are many other user guides exist, such as semantic mask (GauGAN), or text (using CLIP model), and it's not clear how the proposed method generalize to these user guides.

2. The proposed method need to run iteratively to obtain the final image, which makes the inference time inevitably long. It would be great to include the running time of the proposed method in comparing with other methods. 

3. The main technique in the paper (The SDE process) is previous method (Song et. al. 2020, 2021), the paper only proposed to use the human edited image as initialization of SDE at a particular time t0, thus the novelty is not great enough. 
summary_of_the_review: The paper proposed an interesting technique to edit image using the SDE. The generated image is not only faithful but also relistic. However, considering the technique novelty and the overclaiming, I vote for a weak reject initially and am willing to listen to the authors and other reviewers.
review_title: null

Meta-review: Thank you for your submission to ICLR.

This paper presents a technique for image synthesis based on stochastic differential equations and a diffusion model.  This looks to be a very nice idea with good results.  After discussion, the reviewers converged and all agreed that the paper is ready for publication---the most negative reviewer raised their score after the author rebuttal, from a weak reject to weak accept.  The rebuttal clearly and concisely addressed several concerns of the reviewers.

I'm happy to recommend accepting the paper.
Decision: Accept (Poster)