Kati8/ICLR_2023_2025 huggingface dataset:
authors: Chong Mou
authors: Xintao Wang
authors: Jiechong Song
authors: Ying Shan
authors: Jian Zhang
reviews: 
Comments: In this paper, the authors propose a method of drag-style image manipulation with Diffusion Model. They addressed this through the guidance based on feature matching and also conducted comparisons with previous studies such as DragGAN and DragDiffusion. The effects of using layers of various scales were analyzed, and attempts were made to preserve the content of the original image using visual cross attention. In addition, various applications such as object moving, object resizing, appearance replacing, and object pasting were also demonstrated.
Strengths: - The training time is short and FID score is better compared to DragGAN and DragDiffusion
- Many applications are conducted like object moving, object resizing, appearance replacing, and object pasting
Weaknesses: - The problem targeted by this paper is not clear. Therefore, it is unclear why diffusion feature matching, drag-style editing, memory bank, and visual cross-attention strategy were introduced, giving an incremental feel.
- If the paper is focused on the problem of drag-style image manipulation, more experimental results should be presented. For example, it is unclear why the FID score is higher compared to DragGAN. There is no related ablation study for that part.
- Despite the introduction of the visual cross-attention strategy, it feels like that the identity or content of the image is not sufficiently preserved.
- The choices of hyper-parameters seems heuristic. The experiment from multiple combination of hyper-parameter set could be helpful to address this issue.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 3 good
reviews: 
Comments: This paper introduces DragonDiffusion, an image editing method that allows for Drag-style manipulation on Diffusion models. By utilizing feature correspondence, this approach transforms image editing into gradient guidance. It incorporates multi-scale guidance that takes into account both semantic and geometric alignment, as well as visual cross-attention for consistency. The proposed method demonstrates promising performance across a range of image editing tasks.
Strengths: - The energy motivation that originates from classifier guidance is interesting. It motivates the design of the energy function for correspondence in diffusion models.
- The visualization figure vividly demonstrates the editing effect.
Weaknesses: - The clarity of how the memory bank is meaningful is not evident in this draft. As the memory bank is proposed as a contribution, the authors should provide a more comprehensive ablation study, including both quantitative and qualitative analysis.
- How the energy design makes it works is not clear, the authors should provide more details numerical studies.
- The inference time is too slow, approximately 15.93 in Table 1, which makes the solution incomparable with dragGAN.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good
reviews: 
Comments: The paper presents a method enabling user 'dragging' style motion control image editing, similar to DragGAN but with diffusion models. To achieve this, DDIM inversion is performed first on the input image, while the intermediate features from the UNet are saved. During the forward image editing pass, starting from the DDIM-inverted noise, at each diffusion step, three terms are calculated: 1. A cosine similarity score between the dragging patch of the DDIM inversion features and current generation features to gain local consistency, 2. A cosine similarity score between the mean features of the dragging patch of the DDIM inversion features and current generation features to gain global appearance consistency, and 3. Similarity between the unchanged features. The final score gradient is calculated by perturbing the original gradient with the gradient of a weighted sum of these similarities constraints to zt.
Strengths: * The task of user-defined handles is challenging and well-motivated -- – supported by various applications shown in the paper.
* Evaluations were done with reasonable metrics and against SOTA methods, and decent improvements can be observed, especially the efficiency compared with DragDiffusion. Nice qualitative results are shown.
* The method has significantly less complexity comparing with prior works, but seems to work well.
Weaknesses: * Compared with prior (and concurrent) works such as DragGAN and DragDiffusion, way too few samples are shown. The paper and supplementary do not present enough challenging and diverse qualitative samples and comparisons.
* The ablation is a bit incomplete. E.g. it will be nice to see some ablations on the usefulness of S_global.
* Some flickering still happens in the no-change areas, e.g. clouds in the sun example and background in the apple example. If this is because of the balance between different losses, some ablation could be very helpful.
* The identity preservation also seems a bit off, e.g. patterns of the apple. However, I think it is a relatively minor issue as other works also cannot completely fix this issue.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 3 good
reviews: 
Comments: This paper tackles a suite of image editing task (dragging editing in particular) via gradient guidance to drive the sampling from the inversed latent towards the editing target. To ensure consistency between the editing output and the original input, the KV values are cached to form a memory bank that retains the semantics of the original image. The gradient guidance formulation can deal with multiple tasks like dragging edit, object removal, object resizing, appearance replacing and object pasting. The method is mainly compared with prior dragging-based image editing approaches and shows improved quality. The results on other image tasks are also quite impressive.
Strengths: - While gradient guidance has been explored extensively, using this idea as a general approach to accomplish multiple image editing tasks is cool. The qualitative results as shown in Figure 12 is stunning. And, all of these are achieved without the use of any auxiliary model.

- Caching the memory bank for improved image information preservation is a useful technique.

- It is welcome to report the detailed model inference time as shown in Table 1. 

- The proposed method works well for real images, while the editing on real images is usually challenging for many prior methods.
Weaknesses: - First of all, the paper proposes to use gradient guidance sampling for a bunch of tasks, but the paper writing and the experiments mainly focus on dragging-based editing. This will narrow down the scope of the paper quite a lot. It is suggested to formulate the paper as a general solution and equally treat multiple tasks. 

- Also, the experiment is not thorough enough. It is suggested to conduct comparisons on other tasks besides dragging edit. For example, for object pasting, it is suggested to compare against the work "paint by example". For appearance replacement, it is suggested to compare against "Diffusion Self-Guidance for Controllable Image Generation" and "Null-text Inversion".

- There are some typos in the paper. For example, Equation 2 is not correct. 

- There is no limitation analysis for the proposed method.
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 4 excellent
Clarity: 3 good