GDrag:Towards General-Purpose Interactive Editing with Anti-ambiguity Point Diffusion
Authors: Xiaojian Lin
Hanhui Li
Yuhao Cheng
Yiqiang Yan
Xiaodan Liang
Venue: ICLR 2025 Poster (2025)
Keywords: Interactive editing
dragging-based image manipulation
diffusion models
Primary area: generative models

Abstract: Recent interactive point-based image manipulation methods have gained considerable attention for being user-friendly. However, these methods still face two types of ambiguity issues that can lead to unsatisfactory outcomes, namely, intention ambiguity which misinterprets the purposes of users, and content ambiguity where target image areas are distorted by distracting elements. To address these issues and achieve general-purpose manipulations, we propose a novel task-aware, training-free framework called GDrag. Specifically, GDrag defines a taxonomy of atomic manipulations, which can be parameterized and combined unitedly to represent complex manipulations, thereby reducing intention ambiguity. Furthermore, GDrag introduces two strategies to mitigate content ambiguity, including an anti-ambiguity dense trajectory calculation method (ADT) and a self-adaptive motion supervision method (SMS). Given an atomic manipulation, ADT converts the sparse user-defined handle points into a dense point set by selecting their semantic and geometric neighbors, and calculates the trajectory of the point set. Unlike previous motion supervision methods relying on a single global scale for low-rank adaption, SMS jointly optimizes point-wise adaption scales and latent feature biases. These two methods allow us to model fine-grained target contexts and generate precise trajectories. As a result, GDrag consistently produces precise and appealing results in different editing tasks. Extensive experiments on the challenging DragBench dataset demonstrate that GDrag outperforms state-of-the-art methods significantly. The code of GDrag will be released upon acceptance.

Published as a conference paper at ICLR 2025
GDRAG: TOWARDS GENERAL-PURPOSE INTERAC- TIVE EDITING WITH ANTI- POINT DIFFU- SION
Xiaojian Lin'*, Hanhui L Yuhao Cheng', Yiqiang Yan", Xiaodan Liang2.2.41
Shenzhen Campus of Sun Yat-Sen University
2Guangdong Key Laboratory of Big Data Analysis and Processing $ Lenovo Research 'Peng Cheng Laboratory
linxj68 @mail2. sysu. edu. 11hh77@mail. edu.
chengyh5@lenovo com yanyqelenovo com 328 com
ABSTRACT
Recent interactive point-based image manipulation methods have gained consid- erable attention for being user-friendly. However, these methods still face two types of ambiguity issues that can lead to unsatisfactory outcomes, namely, in- tention ambiguity which misinterprets the purposes of users, and content ambi- guity where target image areas are distorted by distracting elements. To address these issues and achieve general-purpose manipulations, we propose novel task- aware, training-free frree framework called GDrag. Specifically, GDrag defines a tax- onomy of atomic manipulations, which can parameterized and combined unit- edly to represent complex manipulations, thereby reducing intention ambiguity, Furthermore, GDrag introduces two strategies to mitigate content ambiguity, in- cluding an anti-ambiguity dense trajectory calculation method (ADT) and self- adaptive motion supervision method (SMS). Given an atomic manipulation, ADT converts the sparse user-defined handle points into dense point set by select- ing their semantic and geometric neighbors, and calculates the trajectory of the point set. Unlike previous motion supervision methods relying single global scale for low-rank adaption, SMS jointly optimizes point-wise adaption scales and latent feature biases. These methods allow us model fine-grained tar- get contexts and generate precise trajectories. As result, GDrag consistently produces precise and appealing results in different editing tasks. Extensive ex- periments on the challenging DragBench dataset demonstrate that GDrag outper- forms state-of-the-arr methods significantly. The code of GDrag available https: / /tt///gttuue com com/DaaaY-coder
INTRODUCTION
With the recent advances in generative models (Rombach al., 2022 many notable image edit- ing methods (Zhang et al.; 2023b Brooks et al. | Kawar et al. 2023 Chen et al. 2023) have emerged, enabling users generate or edit image content effectively. Among these methods, new paradigm that edits images by interactively dragging points on images (Pan et al.) (023b), provides an intuitive and convenient way to obtain desirable images. Specifically, to edit an image, a user places several points the image and drags them to target positions, as shown in Figure These points are considered as handle points, and our task is to move image content along the trajectories of handle points to seamlessly blend the dragged content into its context. To this end, methods of this type alternatively conduct two key processes in iterations, i.e., motion supervision that mod- ifies image content guided by tailored optimization objectives, and point tracking that adjusts the dragging trajectories by calculating intermediate positions handle points.
Bott authors contributed equally.
1 "Correponndigg author.

---

Published as a conference paper at ICLR 2025
DrasDiflaso
Modken
Daftece Inneeier
Dessuing
Latent Fentaee RIXED uow sky
Tal-agnoii
Updiod
Iepat
Sperse Fastares
Cuppe
Selsutaptive Mation Superriston
framida
& Loweroei StaltionSaalee
Deretezg
Inqux
RANDOM Line
Latre
Upizad Danes Terma
Oxtpat
Sceciie
Prin
Art anllariigity
Dones Trajecooie vinee
Figure 1:
Paradigm comparison between previous dragging methods (left-top) and the proposed GDrag method (left-bottom). GDrag estimates task-aware dense trajectories and adopts more fine- grained motion supervision. Hence, GDrag obtains smoother, more reasonable trajectories (labeled in green) and appealing results.
Although current point-based diffusion methods (Shi et al. 2024b Mou et al. 2024 Ling et al., 2024) already achieve relatively good editing effects, they still face certain limitations. Specifically, as existing methods model various editing tasks implicitly, they suffer from two types of ambigu- ities, i.e., intention ambiguity that mixes multiple possible editing tasks into a single trajectory, and content ambiguity that fails to identity and preserve the targets. For example, DragDifflusion (Sh et al. 2024b) updates the positions of handle points after each optimization step regardless of the task, which tends to yield less reasonable and drifted trajectories. FreeDrag (Ling et al. 2024) alleviates this issue by constraining the trajectories to straight lines. However, such 2D trajectories cannot fully represent 3D manipulations like out-of-plane rotations. Regarding content ambiguity, most existing methods rely on a user-specific denoising time step to select latent features of han- dle points for optimization. However, the information encoded latent features different time steps varies significantly. Since different editing tasks require different levels of information, it is impractical to generate satisfying results by exploiting latents only at one time step.
To address the above ambiguity issues in current methods, we propose to model editing tasks ex- plicitly in this paper. Intuitively, we can divide a complex manipulation into multiple basic tasks, each with intents that are clear and easy to perform. This paradigm not only reduces the difficulty of editing but also provides an opportunity to optimize edited images accordingly. For example, in task that involves simply moving a target to another position, we can encourage the model to preserve the target's 's features as much as possible, Conversely, for tasks like target removal, it is more reasonable to update the target features rapidly.
Therefore, we propose a novel task-aware, optimization-based framework for general-purpose in- teractive editing, named GDrag. GDrag categorizes point-based image manipulations into three atomic tasks: relocation, rotation (both in-plane and out-of-plane), and non-rigid transformation (e.g., scaling, content creation/removal), Specifically, given an input image, GDrag first requires the user to specify the task through simple interactions (e.g., a single click). We then introduce an anti-ambiguity dense trajectory estimation method (ADT), which addresses the lack of contextual information in the sparse handle points selected by users. Based on the specific task, ADT selects the semantic and geometric neighbors of handle points as the context and calculates their dense corresponding trajectories. As we will demonstrate in the following sections, our dense trajecto- ries alleviate the poseddessssss of 2D lines in representing 3D deformations, thereby providing more reliable guidance for subsequent optimizations. Additionally, we introduce a self-adaptive motion supervision method (SMS) to optimize latent features based on their dense trajectories. SMS dif- fers from previous motion supervision methods in two key aspects: First, it aims optimize biases for latent features randomly sampled from all denoising time steps, better exploiting the generative capabilities of diffusion models. Second, it optimizes scaling maps that control the effects of low- rank adaptation (Hu et al. (021). Thus, it can preserve target contents at different granularity levels and alleviate content ambiguities. Consequently, as shown in Figure T, GDrag can achieve more reasonable trajectories and higher-quality images compared to conventional methods.
Our contributions are summarized follows:
2

---

Published as a conference paper at ICLR 2025
We propose GDrag, the first optimization-aann based framework that explicitly models editing tasks to handle ambiguities.
Based on different task types, propose the ADT method to construct dense dragging point set and calculate their trajectories, which can offer more comprehensive and reasonable prior knowledge for image editing.
We propose the SMS method that introduces task-aware, fine-gratie ine-grained optimization parameters to refine latent features. This allows us to address content ambiguities and improve the quality of edited images.
By showcasing different experimental results, demonstrate the powerful editing capabilities of GDrag, achieving superior performance on DragBench.
2
RELATED WORK
2.1 IMAGE EDITING
Image editing has always been hot topic and widely applied in various fields. Previous methods
based on generative adversarial networks (GANs) (Goodfellow et al., 2014 Karras et al.
2020)
already achieve considerable performance (Afifi al. [021]] Parihar et al. 2022] Shen et 2020
Zhou et al. 2022)). Recently, with the rapid development of large-scale diffusion models (Rombach
2022), diffusion- based methods become the mainstream for image editing (Epstein et al,
2023
Mao et al. 2023 Hertz et al.) 2022; Tumanyan et al. 2023). Especially, text-based image
synthesis and editing (Brooks et al.] 2023] Chang et al., [223] Li et al. 2024a, Cheng et al., 2024]
Gao et al. [0244] Ahn 2024) that exploit the learned correspondences between texts and images from pretrained large multi-modal networks (Radford et al.) 202T) provides a convenient scheme for related tasks. For instance, Broook et al. (2023) propose instruction- based method, in which editing intentions can be depicted by natural instructions. Autostudio (Cheng et al. (2024) introduces framework that supports multi-turn text-based editing. However, texts are high-level summaries of user intentions, they inevitably result in more ambiguities compared with interactive points.
2.2 INTERACTIVE POINT-BASE MANIPULATION
Interactive point-based image editing allows users to easily edit images by manipulating points in images. pioneering method of this type is GragGAN (Pan et al. 2023a), which proposes the idea of motion supervision and point tracking. However, due to the utilization of GANs and optimizations
on ID latent features, the editing ability of GragGAN is limited in complex scenarios. With the
recent success of diffusion models in image generation, the quality of synthesized images generated
by dragging-bant based methods (Mou et al.) 2024 Shi et al. 20246] Ling et al.) [2024] Liu et al.) 2024 Zhang et al., (022)) has been improved notably. For example, DragonDiffusion (Mou et al. 2024
introduces a seraddiet-guidance-bbeee based energy function to model diffusion sampling, which can better perverse the characteristics of targets. A similar framework explored in DragDiffusion (Shi al. 2024b), which also proposes the DragBench dataset to facilitate related research. To mitigate handle point deviations, FreeDrag (Ling et al. 2024) constrains updates of point positions straight lines to generate more stable editing results. DragNoise (Liu et al. (024) utilizes feature maps from intermediate layers of the denoising network alleviate gradient vanishing. GoodDrag (Zhang
al. 2024) proposes to alternate between drag and denoising steps to achieve finer controls.
Dragging-based methods can also benefit from diffusion models that can generate content beyond images. For instance, DragAnything (Wu et al. 2024), DragNUWA (Yn et al.) 2023), and Motion- 12V (Shi et al., 2024a) exploit video generators to achieve point-based video editing. DragAPart (Li et al. 2024b) learns motion priors in parts of 3D objects for dragging. However, emphasized, most current methods do not seek the explicit interpretation of user intentions, and hence they still suffer from intention and content ambiguity.
have
3

---

Published as a conference paper at ICLR 2025
Seif Self-adaptive Motion Supervision (SMS)
Iterative Optimization
Dittusion Insersion
LeRA Branch
Sampling
Nosy Lacal Feature Maps of Time Steps
Lateni Et
Upskzed Features Danse Trijectorics
candars line waep
Japor
Latest Biase
Outper
Anti-ambiguity Dense Trajectory (ADT) Estimation
Non-cijil
Relocation
Rocation
Tiurs Ourttoo
& (p, B)
Tauk-spezifie
Paruinttrrs
Trajecocies
Seymentation Madel
Semontt Madi
Eilipeedd
Povrndary Bond
Figure 2: The proposed GDrag framework. The key idea of GDrag is to reduce intention and con- tent ambiguities, which is accomplished by the proposed anti-ambiguity dense trajectory estimation method (ADT) and the self-adaptive motion supervision method (SMS). Given user-specific sparse handle points and the editing task, ADT selects a dense set of points that encode rich contextual information and estimates the corresponding task-aware trajectories. Utilizing these trajectories and pair of task-specific parameters, the SMS method adjusts the positions and latent features of the dense points by optimizing latent feature biases and scaling maps, thereby achieving fine- fine-graing grained editing results.
3 METHODOLOGY
In this section, we present the details of the proposed GDrag method. We start by introducing the overall framework of GDrag and formulating our task. Next, we present the two core components of GDrag, including dense trajectory estimation method (ADT) to address intention ambiguities and aelf-adaptive motion supervision method (SMS) to solve content ambiguities.
3.1 GDRAG FRAMEWORK
Conventional dragging- based methods, which lack explicit modeling of editing intentions, inevitably create ambiguities. These ambiguities not only increase the difficulty of editing but also reduce the quality of the edited images. Therefore, the key idea of our GDrag method define set of atomic manipulation tasks with clear and explicit intentions. These atomic tasks should be parameterized and guide subsequent processing, enabling us to complete diverse editing tasks within unified framework. To this end, we propose the GDrag framework as shown in Figure
Specifically, we categorize point-based manipulations into three types, including reloca- tion, rotation, and non-rigid transformation.
Table 1: Main notations within GDrag.
Symbol
Definition
Relocation involves translating the target. Rotation includes both in-plane and out-of-plane rotations, meaning the target rotates within or out of the image plane. Non-rigid transformation com- prises isotropic and anisotropic scaling, as well as the removal or creation content within the target. To obtain satisfying sults, it is reasonable to perform manipulations with different
Length trajectory.
N
Number of optimizations after each movement.
T
Number denoising steps.
Latent features at the t-th denoising step.
2
2; after n optimization steps.
b
Optimizable latent feature biases. Optimizable low-rank adaption scales.
s
M
Target mask.
Tx & G
Dense points and their trajectories. UNet and its low-rank side-branch for denoising. Task-condittoned parameters.
U, U
P.3
4

---

Published as a conference paper at ICLR 2025
degrees of freedom and calculate different point trajectories for these three tasks. For instance, in the relocation task, we should preserve the characteristic of the target as much as possible, while allowing flexible deformations in the non-rigid transformation task that may alter the target significantly.
Therefore, given target image and user-specific trajectories of handle points, assume these tra- jectories belong to an atomic task and manually specify its type. Note that complex manipulations can be divided into atomic tasks (as shown in Figure ( we focus on a single atomic task in the following sections. Since the trajectories of handle points are sparse and limited in representing the context during editing, we propose the ADT method identify the semantic and geometric neigh- bors of dragging points to construct a dense point set and calculate their trajectories based on the task type. Unlike previous point-tracking methods (Pan et al., 2023b Shi et al.] 2024b; Liu et al.) 2024) that update point positions by searching pre-defined neighboring areas, we move our dense points only on their own trajectories, which feasible as their trajectories are less ambiguous. To generate the desired edited image, our SMS method optimizes the latent features of dense points ex- tracted by a pretrained diffusion model during their movements, Besides, a pair of task-conditioned parameters used in our optimization process to balance target preservation and editing flexibility.
Discussion. Similar to our method, a few concurrent approaches (Zhang et al. 2024 Cui et al. 2024) also mention intention ambiguities. However, they either conduct post-editing evaluations (Zhang jtt al. 2024) or utilize pnource-intensive large language models (Cui et al. 2024). In contrast, method is explicitly conditioned tasks, which can be achieved through few user-friendly interactions, such as single click on option tabs in a graphical user interface, as demonstrated in Appendix A.9 Consequently, our method preserves the efficiency of point-based interactive editing while sigmificantly reducing ambiguities.
3.2
ANTI-AMBIGUITY DENSE TRAJECTORY
The goal of our proposed ADT method is to obtain sufficient contextual information and reli- able task-aware trajectories. Specifically, given M user-specific handle points P {pm (Im, Um]]m = O.... M - 1} and their corresponding end positions Q = {9m = (Im. Fm | = O,..., M 1}, we first employ the off-the-shelf promptable segmentation model (Kirillov et al., 2023) to obtain the semantic segmentation mask of the target M. We then construct the dense point set P and estimate point trajectories & based on the atomic task as follows:
Relocation. In the relocation task, points within the target move coherently with similar directions and distances. Therefore, we calculate P and G by,
1 (g)) = = + 4 )
pa = M,
G=
A 1
4m
Pm
M
where 0 ÓSiS (Pt), 1, and I, is the number of steps that move each point its target. A denotes the mean displacement of handle points.
Rotation. Since recovering 3D rotations from 2D image-plane trajectories ill-posed, we utilize an ellipsoid E as the alternative tool to calculate trajectories. Specifically, as shown in Figure we first place& E near the center of the target, which the axis lengths and angles can be adjusted to coarsely cover the target. Let (EE) denote the orthographic projection of the surface points of & onto the image plane. We approximate P by finding their nearest neighbors from Φ(E) (denoted as Pe), and then rotate & around one of its three axes so that the projection of Pe close Q. Hereby, obtain p and G for the rotation task as follows,
P' = MM 4(E),
G - = +d,,)
where d, denotes the geographical displacement (i.e., the ellipsoidal surface) of Ps.j after the rotation. Consequently, our generated trajectories can better represent 3D rotations compared to those calculated directly by handle points the image plane. Moreover, trajectories implicitly
5

---

Published as a conference paper at ICLR 2025
encode the geometric information, such varying distances of surface points of the target, which can guide subsequent optimizations.
Non-rigid transformation. To complete non-rigid transformations such scaling, focus the boundary points, as they determine the shape and pose of the target. Particularly, we extract a boundary band from M with fixed width of To minimize the influence of boundary points that are distant from P and achieve fine-grained, local transformations, we retain only boundary points that are among the & nearest neighbors of any point in P. Let Qu (M) denote the of selected boundary points, we define p and G as follows,
p' = (MaMM)
(3)
G G- - pj + (aī P3) -
where the starting and end positions of pJ are set as those of its nearest handle point.
With the above definitions of task-aware dense trajectories, GDrag can clarify user intents. Figure demonstrates the visual examples of our generated dense points and trajectories for different tasks. In the next section, optimize the features of P along trajectories G, thereby accomplishing various manipulations within unified framework.
3.3 SELF-ADAPTIVE MOTION SUPERVISION
Motion supervision (Pan al. 2023b is the process that optimizes contextual features of handle points along their trajectories gradually. In current diffusion- based methods (Liu et al.
Input
Mask Ellipsoid
Dense Trajectory
2024 Shi et al. 2024b Ling et al., 2024), mo-
tion supervision requires user-specific denois- ing time step to choose the latent features for optimization. However, recent studies (Choi ett al. 2022, Croitoru et al., (203) indicate that diffusion models tend to generate high-level
content during the early denoising steps and finer details in the later stages. Therefore, op- timizing latent features at single time step insufficient for producing desirable content. To address this issue, we introduce the SMS method in this section.
Following existing methods (Shi et al. 2024b Mou al. 2024), we employ DDIM inversion (Song et al. 2021) to transform the input image into a series of latent feature maps } within T diffusion steps, where < t S T rep- resents an arbitrary time step. Zu denotes the clean feature maps while 2T-1 is the noise that can be easily sampled from certain distributions (e.g., (O, 1)). This diffusion process can be formulated as,
Figure 3: Visual examples of our generated dense trajectories for diverse tasks, including relocation (top), rotation (middle), and non-rigid transforma- tion (bottom).
dre, where ag is the Gaussian variance governing the transition probability of the Markov chain in DDIM and (O, 1). To recover and generate the edited image, we can alter a certain latent and iteratively remove noise according to Eq.
2: a.z0
The original motion supervision method assumes that the latent feature maps of the desired edited image adhere to the same data priors as those of the input image. Therefore, moves the handle points step-by-step and minimizes the difference between the corresponding features of the control point before and after a small movement. Given the dense trajectories & consisting of L steps, if we naively through all T' denoising time steps exploit sufficient latent features, the computational complexity is OTL). which is a significant burden. Hence, we propose to perform optimizations N times (N T) with randomly selected time steps after each movement.

---

Published as a conference paper at ICLR 2025
Specifically. let 0 S N denote arbitrary optimization step after small movement on the dense trajectories. For the concise presentation, here we ignore the indexes of movement steps on the dense trajectories and denote the latent features randomly selected denoising time step t as x.. By substituting 2p into Eq. (4). we can obtain the optimized latent feature maps 2; 2,+1
as,
2 2n+1_ 2a,28 + vI atttt
(5)
where e, is predicted by the pre-trained diffusion model. To calculate 20+1, we first obtain 20 from the unaltered latents as follows.
Zi+1
(6)
01++1
We then introduce the following self self-adaptive scheme for updating 20+1;
2 2p+1 (1 p)gم + pb".
Here, b" denotes latent feature biases for optimization, which initialized as zo to ensure it follows the same data distribution of the original image. is a task-specific weighting factor, e.g., in the relocation task, a larger p is preferred, as generally leads to better preservation of the original latents.
Additionally, common practice to utilize low-rank adaption (LoRA) (Shi et al., 2024b) Ling et al. 20244 Liu et al.) 2024), which fine-tunes the denoising UNet of the diffusion model with low-rank side branch. As the bottleneck feature dimension (rank) of the LoRA branch is much smaller than that of the original diffusion model, fine-tuning this branch reduces the feature space of the model to be compactly around the target image, consequently helping to preserve the identity of the target. Specifically, let U and U' denote the original UNet and its fine-tuned LoRA branch, respectively. The outputs of U and ''' are combined linearly with a single trade-off scalar, i.e., u(2 +. ). Nevertheless, such global combination is insufficient model fine-grained and heterogeneous feature transformations. Hence, we also introduce an optimizable scaling map s" in SMS as follows.
f(2,+1) = =(( + 5 @ (2;
(8)
where c denotes the (broadcast) Hadamard product and s" is of the same spatial resolution as 2,
The desirable latents of the edited image can be obtained now by jointly optimizing b and s super- vised by the following loss function,
L= (1 Calig + BL + XC
where the first loss term Lalign measures the discrepancy between original features and those moved along the dense trajectories. The second term Lamooth is calculated on the corresponding features before and after a motion supervision step, which introduced encourage smooth feature trans- formations. Finally, Liaas calculates the contextual deviations of latent feature maps within the target mask, helping to maintain the homogeneity of the latent feature maps. B is another task- specific parameter while X is set the same value for all tasks. The detailed definitions of these three terms are as follows:
Calign = [|
fgo -
(10)
where fz( denotes the latent features of dense points in G. retrieved from via feature interpo- lation, e.g., Jo (2 means the dense point features at the 1 + position along the trajectories after n + optimization steps. Similarly,
Lamooth = ||.fgri (2 - fgr (2 ))
(11)
and,
Lmask
[[ 2,) o (1 - (1-M)l I|.
(12)
In practice, we find that including all intermediate features in the denoising UNet to optimize Eq. p leads to better performance.
Task-aware point tracking. At last, we update the positions of the dense points after every N motion supervision steps. Thanks to the reliability of our task-aware trajectories, for each point in its corresponding trajectory that minimizes Eq. (IO. rather than searching the neighboring areas of the trajectories. This allows our handle points to move along the trajectories and avoid undesirable drifts.
P'. determine its new position the
7

---

Published as a conference paper at ICLR 2025
4 EXPERIMENTS
In this section, we provide quantitative and qualitative analysis of GDrag. Due to the page limitation, more experimental results can be found in the Appendix.
4.1 SETUP
Implementation details. All our experiments are conducted with an NVIDIA GeForce RTX 4090 graphical card (24 GB). We use Stable Diffusion 1.5 (Rombach et al.) 2022) as the base diffusion model. Following (Liu et al., 2024), to ensure the consistency of image content before and after editing, we perform LoRA fine-tuning on the input image, with 80 fine-tuning steps and the adaptor rank is 16. SMS uses the Adam optimizer with a learning rate of 0.01 for b and 0.02 for s, and the parameter À is set to 0.2. In addition, the values of parameters L, T, and A are 50, 35, and 5, respectively. We set r = 5 and k = |MI/10 to calculate the boundary band in the non-rigid transformation task. In the relocation and in-plane rotation tasks, the value of p is set 1.0, while in the non-rigid transformation and out-of-plame rotation tasks, the value of is 0.2. We calculate the value of adaptively based on the specific task and the details can be found in Appendix A.10
Baselines and evaluation metries. We compare GDrag with state-of-the-art methods on the Drag- Bench dataset (Shi al. 2024b) validate its effectiveness. We select four baselines with publicly available implementations for comparison, including DragDiffusion (Shi et al. 2024b)) DragonDif- fusion (Mou al.) 2024). FreeDrag (Ling et al.) 2024). and DragNoise (Liu al. (024). Following these methods, we use LPIPS (Zhang al. 2018) 8) and mean distance (MD) as our evaluation metrics. I.PIPS is widely used to access image quality. The second metric is the average of the Fuclidean distances between the target and actual positions of handle points, which reflects how well the edited results align with the user intents.
4.2 COMPARISON WITH SEATE-OF-THE-AEE
Quantitative analysis. Table 2 reports the performance of GDrag and the other baseline methods. GDrag achieves the lowest LPIPS value (0.0915), which suggests its edited images have the best quality. Furthermore, GDrag obtains the lowest mean distance (26.49), which we owe to our anti- ambiguity dense trajectories.
Beyond these two objective metrics, we also conduct human study to further evaluate our method. We invite 20 volunteers to complete a questionnaire consisting 30 questions. We show group of five edited images produced by GDrag the four basclines in each ques- tion, and ask the volunteers select the most satisfying one. The results in Table show that most volunteers (about 60.33%) choose GDrag, indicating the significant advantages of GDrag.
Table 2: Quantitative comparison with state-of-
the-arts on DragBench.
Method
MDJ
LPIPS4.
User Study 1
DragDiffusion
33.91
0.0940
7.83%
Shi et al.
2024b
DragonD Di Jusion
31.63
0.1033
10.5%
Mouu al.
2024
FreeDras
27.41
0.0996
8.67%
Qualitative analysis. Figure 4 shows several visual examples generated by GDrag and the baseline methods. These results show that the baselines still produce noticeable artifacts or fail to meet the editing requirements. For in- stance, in the rotation task, DragDiffusion and FreeDrag struggle with large out-of-plane rota- tions, while DragonDiffusion and DragNoise fail to preserve the targets effectively. Even in the seemingly simple relocation task, some targets moved by the baselines lag behind the handle points. These examples support our view that implicitly representing various editing tasks leads to severe ambiguities and performance degradation. On the contrary, GDrag explicitly models the editing tasks and generates more appealing images.
Ling et al.
2024
Noise
29.56
0.1017
12.67%
Liu et al.
2024
GDrag
26.49
0.0915
60.33%
(Ours)
8

---

Published as a conference paper at ICLR 2025
Input
DragDiffusion
DragenDiflusion
FreeDrag
DragNoisc
GDrag (Ours)
Rolation
Relocation
N.R. Transfurm.
LIF
Figure 4: Visual comparison between GDrag and state-of-the-art methods. Blue dots denote the final positions of handle points. GDrag achieves more precise manipulations and satisfying results across various editing tasks, including rotation, relocation, and non-rigid transformation.
4.3 ABLATION STUDIES
We conduct ablation studies on the two key components of our framework, i.c., ADT and SMS.
Effects of anti-ambiguity dense trajectory.
As shown in Table 3 the mean distance metric of the baseline decreased by approxi- mately 20. 1%. when augmented by the pro- posed ADT method. This indicates that incor- porating the task prior and rich context (in- cluding semantic and geometric neighbors) can significantly improve the precision of ma- nipulations.
Table 3: Effects of the proposed components.
ADT
SMS
MDJ
LPIPS
33.91
0.0940
27.09
0.0948
v
26.49+0.07
0.0915++7.3e-7
Effects of self-adaptive motion supervi-
sion. Table 3 shows the performance of the baseline can be further improved by the SMS method consistently in both the mean distance and LPIPS metrics. This is reasonable, as the optimizable latent features biases and scaling maps better represent local and fine-grained modifications. Fur-

---

Published as a conference paper at ICLR 2025
Iapu:
User Edit
User Bdil
Oupes
Figure 5: Visual ablation study the proposed Figure 6: Visual examples of combining atomic components.
manipulations to achieve complex editing.
thermore, the task-aware trajectories and parameters (p and () help to reduce the difficulty of op- timizations. Figure [ also suggests that with both ADT and SMS, our method obtains the most appealing results.
Moreover, since we randomly select denoising time steps for optimizations in SMS, we report the standard deviations seeds. As shown in Tables 3 & [ and Figure 15] the overall performance of GDrag is stable and
the two evaluation metrics across five experiments with different random
hence our random sampling strategy is considerable.
Compositional editing. Figure 6 shows that our defined atomic manipulation tasks can be com- bined to accomplish complex intents. This validates that GDrag is general-purpose solution for interactive point-based image editing.
CONCLUSION
In this paper, present GDrag, general-purpose optimization-baser framework that tackle diverse interactive point-based image editing tasks. contrast to conventional methods, GDrag defines three atomic editing tasks to calculate task-aware trajectories and reduce ambiguities. Fur- thermore, by introducing the self-adaptive motion supervision strategy, which contains optimiz- able biases and scaling maps for randomly sampled latent features arbitrary denoising steps, our method better balances content consistency and editing flexibility. Our GDrag method is validated thoroughly on the DragBench dataset and achieves of-th-of-ee-rrr performance terms of manip- ulation precision and image quality.
ACKNOWLEDGEMENTS
This work was supported by National Key Research and Development Program of China(2044VFE00003 National Natural Science Foundation of China (NSFC) under Grants No.62372482 and No. No.62476293, Nansha Key R&D Program under Grant No. N0.20222D014, and Gen- eral Embodied AI Center of Sun Yat-sen University.
REFERENCES
Mahmoud Afifi, Marcus A Brubaker, and Michael S Brown. Histogan: Controlling colors of gan- generated and real images via color histograms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7941-7950, 2021.
10

---

Published as a conference paper at ICLR 2025
Namhyuk Ahn, Junsoo Lee, Chunggi Lee, Kunhee Kim, Daesik Kim, Seung-Hun Nam, and Kibeom Hong. Dreamstyler: Paint by style inversion with text-to-image diffusion models. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 674-681, 2024.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, José Lezama, Lu Jiang, Ming- Misan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image gen- eration via masked generative transformers. In International Conference on Machine Learning, pp. 4055-4075, 2023.
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero- shot object-level image customization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2023.
Junhao Cheng, Xi Lu, Hanhui Li, Khun Loun Zai, Baiqiao Yin, Yuhao Cheng, Yiqiang Yan, and Xi- aodan Liang. Autostudio: Crafting consistent subjects in multi-turn interactive image generation. arXiv preprint arXiv:2406.00 01444066088888 38888, 2024.
Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11472-11481, 2022.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9): 10850-10869, 2023.
Xing Cui, Peipei Li, Zekun Li, Xuannan Liu, Yueying Zou, and Zhaofeng He. Localize, understand, collaborate: Semantic-aware dragging via intention reasoner. arXiv preprint (tt::20060004332 2024.
Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffusion self- guidance for controllable image generation. Advances in Neural Information Processing Systems, 36:16222-16635 16239, 2023.
Xiang Gao, Zhengbo Xu, Junhan Zhao, and Jiaying Liu. Frequency-contreoee diffusion model for versatile text-guided image-to-image translation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1824-1832, 2024.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Nearal Informa- tion Processing Systems, 27, 2014.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. arXiv preprint SXXi:22008001266 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. International Confer- on Learning Representations, 2021.
Tero Karras, Samuli Laine, Mikk Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Ana- lyzing and improving the image quality of stylegan. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1018819, 2020.
Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 6007-6017, 2023.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE International Conference Computer Vision, pp. 4015-4026, 2023.
11

---

Published as a conference paper at ICLR 2025
Dongxu Li, Jumnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for trollable text-to-image generation and editing. Advances Neural Information Processing Sys- tems, 36, 2024a.
Ruining Li, Chuanxia Zheng, Christian Rupprecht, and Andrea Vedaldi. Dragapart: Learning part-level motion prior for articulated objects. Proceedings of the European Conference on Computer Vision, 2024b.
Pengyang Ling, Lin Chen, Pan Zhang. Huaian Chen, and Yi Jin. Freedrag: Feature dragging for reliable point-based image editing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024.
Haofeng Liu, Chenshu Xu, Yifei Yang. Lihua Zeng, and Shengfeng He. Drag your noise: Interactive point-based editing via diffusion semantic propagation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024.
Jiafeng Mao, Xueting Wang, and Kiyoharu Aizawa. Guided image synthesis via initial image editing in diffusion model. In Proceedings of the ACM International Conference on Multimedia, pp- 5321-5329, 2023.
Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. Dragondiffusion: Enabling drag-style manipulation on diffusion models. In International Conference on Learning Represen- tations, 2024.
Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image mani- fold. In Proceedings of the ACM SIGGRAPH Conference, pp. 1-11, 2023a.
Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your gan: Interactive point-based manipulation on the generative image mani- fold. In Proceedings of the ACM SIGGRAPH Conference, pp. 1-11, 2023b.
Rishubh Parihar, Ankit Dhiman, and Tejan Karmali. Everything there in latent space: Attribute editing and attribute style manipulation by stylegan latent space exploration. In Proceedings of the ACM International Conference on Multimedia, pp. 1828-1836, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning.
pp. 8748-8763. 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjøm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9243 9252, 2020.
Xiaoyu Shi, Zhaoyang Huang, Yun Wang. Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and con- trollable image-to-video generation with explicit motion modeling. In Proceedings of the ACM SIGGRAPH Conference, pp. 1-11, 2024a.
Yujun Shi, Chuhui Xue, Jiachun Pan, Wenqing Zhang, Vincent YF Tan, and Song Bai. Dragdiffu- point-based
sion: Hareessing diffusion models for interactive point- based image editing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2024b.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- fional Conference on Learning Representations, 2021.
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven taage-to-immae translation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1921-1930, 2023.
12

---

Published as a conference paper at ICLR 2025
Wejia Wu, Zhuang Li. Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for anything using entity representation. arXiv preprint 2xXv:2403.07420, 2024.
Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Drag- nuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.0800 2023.
Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim, Sung-Ho Ho Bae, Seungkyu Lee, and Choong Seon Hong. Faster segment anything: Towards lightweight sam for mobile applications. arXiv preprint:2306. 14289, 2023a.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE International Conference Computer Vision, pp- 3836-3847, 2023b.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586-595, 2018.
Zewei Zhang, Huan Liu, Jun Chen, and Xiangyu Xu. Gooddrag: Towards good practices for drag editing with diffusion models. arXiv preprint arXiv:2404.07206. 2024.
Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Chris Tensmeyer, Tong Yu, Changyou Chen, Jinhui Xu, and Tong Sun. Tigan: Text-based interactive image generation and manipulation. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3580-3588, 2022.
13

---

Published as a conference paper at ICLR 2025
A APPENDIX
The outline of the Appendix is as follows:
More visual results compared with state-of-the the- methods in A.1
User study on intention understanding in A.2
Effects of segmentation models A.3
Computational cost analysis of the proposed method in A.4
Ablation study the proposed anti-ambiguity dense trajectory (ADT) calculation method in A.5
Ablation study on the proposed self-adaptive model supervision (SMS) strategy in A.6
Comparison between using random and fixed denoising time steps in A.7:
Edited images with different random seeds in A.8
An example of our graphical user interface in A.9
Our strategy of calculating the task-related parameter /3 in A.10
Ablation Study on the task-aware parameters and in A.11
A.1
MORE VISUAL RESULTS
Here we provide more qualitative results of GDrag in Figure 9 which further demonstrate that GDrag outperforms state-of-the-ar methods across various editing tasks, including rotation, reloca- tion, and non-rigid transformation.
In Figure 7 we show examples of the proposed GDrag method in completing complex manipula- tions. Each of these manipulations involves motions of multiple joints/parts and incorporates more than one atomic task. For example, the first row, our goal to transform barking dog into smiling one, which requires us to first close its mouth and then lift the corners of its lips. In these examples, we separate each manipulation into two steps and show the intermediate and final edited images. These results demonstrate that the edited images generated by our GDrag method better align with user intentions and have fewer artifacts compared with the baseline.
In Figure 8, we demonstrate few failure cases of the proposed method. Although GDrag moves handle points to their target positions more accurately compared with other methods, it generates a few realistic but unnecessary details around the targets, e.g., the leaves on the left side of the rose in the bottom row of Figure These failure cases are mainly caused by the limited inpainting ability of our base generator (Stable Diffusion 1.5), which can be addressed by more advanced generators.
A.2 USER INTENTION UNDERSTANDING
One of our key assumptions in this paper is that 2D dragging inevitably causes intention ambiguity, which can be verified
levels: First, given a dragging- based manipulation, will most users recognize it as the same editing task? (ii) If so, can this editing task be recognized using current methods?
To answer these two questions, we conduct an additional user study. In this study, we invite 20 volunteers to complete questionnaire consisting of questions. each question, we present sample from the DragBench dataset and ask the respondents to identify the corresponding user intention by selecting one of the following four choices: (a) Relocation, (b) Rotation, (c) Non-rigid transformation, and (d) Not sure, may be ambiguous.
To evaluate the consistency of the respondents' votes, we adopt the variation ratio (VR) which is defined as 1- F' k. where F is the maximum frequency for choice in question and K is the number the respondents. The higher the VR, the more dispersed the choices are, indicating that the corresponding user intention is more ambiguous.
The average VR of all samples our user study is 0.47, with the VR for each sample varying between 0.3 and 0.7. Figure 10 also shows some samples of the study. These results validate
14

---

Published as a conference paper at ICLR 2025
DragDiffusion
GDrag (Ours)
Input
Intermediute
Final
Intermediate
Finul
Figure 7: Visual examples of the proposed method in addressing complex motions.
Input
DragDiffusion
DiflssDi
FreeDrag
DragNoise
GDrag (Ours)
Figure 8: Visualization of failure cases.
that even for the same 2D dragging manipulation, different people have varied interpretations of its intention and are not always consistent. Therefore, the concept our method, which represents user intentions through atomic tasks, is necessary.
We then investigate whether user intentions can be predicted. Impressed by the powerful reasoning abilities of modern multi-modal large language models (MILMss, we propose to use them to inter- pret user intentions. We use images with drawn dragging trajectories and the following prompt for MLLM reasoning: "The white arrow in the figure represents the drag editing starting from the red dot to the blue dot that the user needs to implement. Please select which type of task the user is most likely to want to implement and why, from the following three task types: relocation, rotation, or non-rigid transformation.""
We Qwen2.5, one of the most advanced MLLMs that is open and free, to complete the afore- mentioned questionnaire as well. We find that only 40% of its choices are consistent with those selected by most respondents. T0
10 demonstrates some of its results, we can see that even for
15

---

Published as a conference paper at ICLR 2025
Inpur
DraDiffusion
DragonDiftusion
FreeDray
DragNoise
GDrag(Ous)
Retedion
Rotation
Relocttonn
Relocaiio
NR. Teeefoom
N.R. Trasfom
Figure More visual comparison between GDrag and other state-of-the-an methods.
a less ambiguous sample like the rotation task in the first row, the MLLM mistakenly considers the non-rigid transformation. With these results, believe that is still challenging for current methods to understand user intentions given dragging trajectories. Consequently, our method that explicitly specifies atomic tasks is effective and efficient.
A.3 EFFECTS OF SEGMENTATION MODELS
Our GDrag utilizes semantic segmentation to calculate dense trajectories. To investigate how the segmentation results affect the per- formance of GIrrag except for SAM (Kir-) illov et al. 2023), we also employ Mobile- SAM (Zang et al.) 2023a) for evaluation, which is the light- version of SAM.
Table 4: Ablation study the segmentation model.
Method
MDJ
LPIPS
GDrag w/ MobileSAM
26.74
0.0959
GDrag w/ SAM
26.49
0.0915
Considering that MobileSAM uses only 1.5% of the parameters of SAM, it is reasonable that the masks predicted by MobileSAM are less accurate. However, from the quantitative results in observe that GDrag still performs well with MobileSAM. The mean distance and I.PIS
4
16

---

Published as a conference paper at ICLR 2025
Input
Human
MLLM
A: The user is most likely trying implement non-rigid transformation. Here's why:
5%
10%
15%
Starting Point (Red Doty: The rod dot positioned near the alpaca's nosc.
Ending Point (Bluue Doty: The blue dot positioned slighty to the right and higher up from the red dot.
Given movement path, suggests the user wants to manipulate the shape around the alpaca's
70%
possibly to make appear smaller adjust its position.
This kind of manipulation best achieved through non riiid cansformation, which allows for local deformations without affecting the entire image uniformly.
Rotattoo N.R. Transfoem Not Sure
Relocation
Therefore, the intended task is most likely non-rigid transformatee
A: The white arrow in the image points directly leflwards, indicating a straighiforward movement along a single axis. This suggests simple change position without any rotation or deformation. Therefore, the user is most likely trying to implement a relocation fask.
10%
25%
Here's reasoning for each option:
points directly left, indicating clear intention the object (in this case, the bicycle wheel) to new location borizontally.
Relocation: The
90%
Rotation: There is no indication of circular motion or angular displacement, so rotation unlikely.
Nea-Rigid Transformations: These involve changes in shape or form, such as stretching or bending, which are not suggested by the straight, borizonial arrow.
Relecation
Rotation
Transform
Given these considerations, the most appropriats choice is relocation.
A: The white arrow in image points directly leftwards, indicating straightforward movement along a single avis This suggests simple change position without any rolation or deformation. Therefore, the user is most likely trying to implement a relocation task.
55
Here's reasening for each option:
Relocation: The arrow points directly left, indicating clear intention move the object (in this case, the car tire) a new location horizontally.
Rotation: There indication of circular molion angular displacement, so rolation
42%
unlikely.
Nea-Rigid Transformation:: These involve changes in shape or form, such as stretching or bending, which are not suggested by the straight, borizontal arrow:
Relocatin
Rotation
N.R. Transfom
Not Sure
Given these considerations, the most apprupriate chuice is relocation.
The white arrow in imuge points directly upwards, indicating a straightforward movement along single axis. This suggests simple change in position without any rotation or deformation. Therefore, the user is most likely trying to implement a relocation task.
20%
25%
Here's the reasening for each option:
Relocation: The arrow points directly up, indicating clear intention to move the object (in case, the person) to a new location vertically.
25%
30%
Rotation: There indication of circular molson angular displacement, so rotation unlikely.
Non-Rigid Transformatione These involve changes in shape or form, such as stretching or bending, which are not suggested by straight, upward arrow.
Relocation N.R. Transfomm
Rotation Sure
Given these considerations, the most appropriate choice is relocation.
Figure 10: Visual samples of the study on understanding user intentions.
of GDrag with MobileSAM are 26.74 and 0.0959, respectively, while those of GDrag with SAM are 26.49 and 0.0915. Figure [ also shows that, despite some artifacts like holes and disconnected regions in the predicted masks, GDrag still generates high-quality edited images.
A.4 COMPUTATIONAL. COST ANALYSIS
Our computational cost analysis is conducted using an NVIDIA GeForce RTX 4090 GPU with 24 GB of memory. We calculate the mean processing time per image and the GPU memory consump- tion on the DragBench dataset (Shi et al.. 2024b), with a fixed image size of 512 x 512 and 250 optimization steps. We select DragDiffusion (Shi et al.) 2024b) as the baseline for comparison as also adopts Stable Diffusion 1.5 as the base generator. The computational cost of the baseline is approximately 12 GB of memory and 132 seconds per image, including 40 seconds for employ- ing low-rank adaption (LoRA) and 92 seconds for optimization. In comparison, GDrag consumes about 16 GB of memory and 192 seconds per image, including 40 seconds for LoRA, 2 seconds
17

---

Published as a conference paper at ICLR 2025
Input
MobileSAM Mask
Result MebileSAM
SAM Mask
Result SAM
Figure 11: Visual comparison of the proposed method with different segmentation models.
for segmentation, and 150 seconds for optimization. We find that the additional computational cost proportional to the task difficulty. For easier tasks like relocation and in in-plane rotation, the pro- cessing time of GDrag is almost the same the baseline, while more complex manipulations like oft-of-plane rotation and non-rigid transformation take longer. Considering that our task prioritizes image quality over processing speed and that most current methods real-time, the proposed GDrag remains competitive.
A.5 ABLATION STUDY ON ADT
The proposed ADT method converts sparse handle points specified by users into task-aware dense trajectories to better exploit contextual information. To ensure the fairness of com- parison and validate that the advantages of the proposed method do not merely rely on more handle points, we conduct that replaces inputs the baselines with dense trajectories. The results reported in Ta- ble [5] show that not all baselines benefit from dense trajectories. Although DragDiffusion (Shi et al. (024b) and DragonDiffusion (Mou al.j (024) with dense trajectories achieve lower mean distances, the performance of Free-
Table 5: Quantitative comparison of different methods with ADT.
Method
MD4
LPIPS
ablation study
DragDiffusion
33.91
0.0940
DragDiffusion w/ ADT
33.48
0.1559
DragonDiffusion
31.63
0.1033
DragonDiffusion w/ ADT
29.16
0.1109
FreeDrag
27.41
0.0996
FreeDrag w/ ADT
35.16
0.1061
DragNoise
29.56
0.1017
DragNoise w/ ADT
30.00
0.1399
Drag (Ling et al. 2024 and DragNoise (Liu
GDrag (Ours)
26.49
0.0915
et al. 2024) deteriorates. This is because Free- Drag devises its own strategy for trajectory cal- culation, while DragNoise conducts high-level feature optimizations and already adopts whole seg- mentation masks for guidance. Therefore, we can conclude that the performance gains of GDrag mainly stem from its method, instead of using more handle points.
This can also be validated by the visual examples shown Figure 12 In these examples, the end positions of handle points of the baselines with dense trajectories are closer to the targets. However, their results still have noticeable artifacts compared with the proposed method, such as the warped fire extinguishers in the last example.
A.6 ABLATION STUDY ON SMS
Our SMS method introduces learnable latent biases and scaling maps to balance the modifi- cation and preservation of latents. To further validate its effectiveness, we perform an ab- lation study considering five variants of SMS, including: (i) "Latent" which optimizes la-
18

---

Published as a conference paper at ICLR 2025
Input/GDrag (Ours)
DragDiffusion
DragonDiffusion
FreeDrag
DragNoise
w/o w/o ADT
w/ ADT
w/O w/O ADT
w/ ADT
Figure 12: Visual examples of the ablation study on the proposed ADT method.
tent features directly; (ii) "Bias" which only learns latent biases; (iii) "Scaling Scalar" that learns scaling scalar for the LoRA side branch as in conventional methods; "Scaling Map" that converts scalar in (iii) a learnable map; (v) "Bias + Scaling Scalar" that com- bines (ii) and (iii). We employ the proposed ADT method with all these five variants.
Table 6: Quantitative comparison of SMS variants.
Table 6 reports the quantitative comparison among the variants SMS. We observe that, compared with optimizing latents directly, "Bias", "Scaling Scalar", and "Bias + Scal- ing Scalar" reduce the LPIPS values signifi- cantly yet perform badly when evaluated by the mean distance metric. This suggests that while these variants tend to preserve the orig- inal images and generate high-fidelity images, they are less effective at following user inten- tions. The "Scaling Map" variant performs better than "Latent" on IPIPS, as it allows to employ LoRA in a finer manner. Nevertheless, without the optimization of latents or latent bi- ases, this variant is still inferior for diverse modifications and falls behind "Latent" the mean distance. Finally, with both optimizable components, our SMS method outperforms the latent-based optimization on both metrics consistently.
Method
MD
LPIPS
Latent
27.09
0.0940
Bias
47.11
0.0522
Scaling Scalar
47.92
0.0486
Scaling Map
28.62
0.0873
Bias + Scaling Scalar
47.15
0.0516
SMS
26.49
0.0915
The above analysis further validated by the qualitative examples shown Figure 13 observed that the images synthesized by "Bias". "Scaling Scalar", and "Bias Scaling Scalar" almost identical to the input images. The "Latent" and "Scaling Map" variants modify the images certain extent but they also result in distorted target structures.
It can be
19

---

Published as a conference paper at ICLR 2025
Iepui
Latent
Bisa
Scaling Stalar
Scaling Map
Biaa Scaling Siater
SMS
Figure 13: Visual examples of the ablation study on the proposed SMS strategy.
Input
1-10
1-20
--30
3-35
4-0
Random
Figure 14: Visual comparison between randomly sampled and fixed denoising steps.
A.7 OPTIMIZATION RESULTS UNDER DIFFERENT DENOISING STEPS
Another key idea of our SMS method that previous methods selecting fixed denoising time step é are insufficient to complete various manipulations. This because large generally results large struc- tural changes, while a small t is more suitable for small modifications. Hence, we propose sampling time steps randomly during optimization to cover as many variations possible and reduce the compu- tational cost of exhausting all time steps. To validate this, we compare SMS with a variant that adopts a fixed time step. As the maximum denoising time step in our base model T = 50, we report the re- sults of the variant with f 10, 20, 30, and 40. Both the quantitative comparison in Table 7 and the qual- itative comparison in Figure 14 show that our SMS method with random time steps outperforms the fixed one significantly. Hence, our SMS overcomes the limitations of selecting the time step manually and is more practical for real-world applications.
Table 7: Quantitative comparison with dif- ferent denoising steps.
Method
MD4
LPIPS
10
43.72
0.0628
20
35.05
0.0739
= 30
30.36
0.0873
35
29.37
0.0908
t = 40
31.66
0.0928
Random
26.49
0.0915
20

---

Published as a conference paper at ICLR 2025
Input
Sead
Seed=30
Seed-112
Seed-121
Seed=166
Figure 15: Visual results under different random seeds. These examples are satisfactory and validate that our method is stable.
A.8 OPTIMIZATION RESULTS UNDER DIFFERENT RANDOM SEEDS
In GDrag, an arbitrary time step is randomly selected for optimization. Therefore, demonstrate the sta- bility of our method, we visualize the edited images under five different random seeds, as shown in Fig- ure 15 We can see that our method is robust to the choice of random seeds, e.g., in the first row, the ro- tations of the faces are almost identical, and all re- sults are satisfactory. We also report the quantitative performance of our method with different random seeds T which further validates the stability of our method.
Table 8: Quantitative results under different
random seeds. MD stands for mean distance.
Random Seed
MDJ
LPIPS.
0
26.56
0.0907
30
26.25
0.0929
42
26.33
0.0916
121
26.97
0.0919
166
26.33
0.0905
A.9 THE GRAPHICAL USER INTERFACE
Figure 16 shows an example of our graphical user interface (GUT), where guide the user complete the rotation task. This GUI allows the to define complex rotations easily, e.g., by dragging an ellipse that roughly covers the editing area and rotating the ellipse mimic out-of-plane rotations.
A.10 THE VALUE OF 3 BASED DIFFERENT TASKS
Here we will provide a detailed definition of the value of / in different tasks:
1) Relocation:
8 = exp(
n N
align
50) * 0.1,
(13)
Samoth
H W
where H and W are the height and width of the input image.
2) In-plane Rotation:
n-1 align Cn-1 smooth
[M
= exp(
0.7 + 0.3.
(14)
N
H W
3) Out-of-plane Rotation:
C
3" exp(
n
align ca-1
(15)
N
H + W
Non-rigid Transformation:
cn-1 align
[M|
8th exp(
25) * 0.2.
(16)
N
Ln-1
smooth
21

---

Published as a conference paper at ICLR 2025
(4)
Center 257
Center Y
X Range
Y Range
0.51
Angle
Z'Aggle
Draw Ellipsold
(c)
Angle
Axts X Coord
Rot Axis Coord
Undo Point
Dmg Ellipooid
The graphical user interface of rotation task. (a) The user selects the input image and specifies the editing task. (b) The user uses the promptable segmentation model locate the editing area, with prompts like positive points and bounding boxes. (c) The user drags and customizes an ellipsoid roughly cover the target, of which parameters like the center coordinates and the lengths of axes are recorded. (d) The user selects an axis and drags it to rotate the ellipsoid, and we record the rotation angle calculate the corresponding trajectories. (e) The edited image synthesized by our method.
Figure 16:
In all tasks, 3 decreases gradually n increases. This is because more optimization steps will lead to more image information loss at the initial position. Consequently, we need to decrease the value of 3 to increase the weight of Latign as the step of optimization increases. In order to more balance the weights of Lalign and Lemooth during the optimization process, also consider Lalign and Lamooth in the previous optimization into the formula. The intuition is that if the value of Cymooth is relatively large in the previous optimization, should focus more on the Lsmooth in this optimization, which also applies to Lalign-
Additionally, for different inputs, adjust the rate of B reduction based the proportion of the editing area relative to the entire image. Specifically, a larger ratio requires a faster reduction, as the larger the ratio of editing area is, the more easily the image information the initial position lost. Finally, for different tasks, the range of 3 values will vary. For example, in the relocation task, needs be restricted to relatively small range. This ensures that our model focuses more on Callga. meaning it pays more attention to preserving the initial image information of the edited object during the dragging process. In contrast, the in-plane rotation task requires generating new image information, so needs to be within larger range.
A.11 ABLATION STUDY ON TASK-SPECIFE PARAMETERS
We investigate the effects of our task-related parameters P and 3 the non-rigid trans- formation task as it is the most challenging among all three atomic tasks. Table 9 reports the results of our method with different values of p. From the results, we observe that the low- est mean distance is achieved when = 0.4 and gradually increases as the value of in- creases. This is reasonable since the role of p to preserve the original images. Hence. the larger / is, the fewer modifications are exhib-
Table 9: Ablation study p.
MDJ
LPIPS
Baseline
33.86
0.0733
P = 0.2
31.26
0.0679
P = 0.4
30.61
0.0682
P = 0.6
33.52
0.0642
P = 0.8
34.74
0.0632
P = 1.0
34.39
0.0647
22

---

Published as a conference paper at ICLR 2025
ited in the edited images. The LIPIPS scores of p with different settings are similar, which suggests that the optimization process of GDrag and the image quality of its outputs stable.
As for 3, we vary its base scaling factor Eq. (T from 0.2 1.0 to analyze its effects. The results in Table [0] show that the overall trend of in affecting the performance of method is similar to that of As discussed in A.10 we use 3 to control the weight of Caligns consequently preserving the initial image in- formation before editing. From the results, we can see that B also fulfills our goal, as the mean distance increases (indicating fewer modifica- tions) as the value of ß increases.
Table 10: Ablation study on B
MDJ
LPIPS
Baseline
33.86
0.0733
B 0.2
30.08
0.0663
B 0.4
30.61
0.0682
B = 0.6
33.52
0.0642
Ø 0.8
34.74
0,0632
B = 1.0
34.39
0.0647
23


Reviews:
[{"review_id": "8W3SFNgUkn", "reviewer": "Reviewer_54bB", "summary": "This paper proposes GDrag, a general-purpose optimization-based framework to tackle diverse interactive point-based image editing tasks. GDrag introduces two strategies, an anti-ambiguity dense trajectory calculation method (ADT) to calculate the trajectories, and a self-adaptive motion supervision method (SMS) to refine latent features. The experiment performance demonstrates the powerful editing capabilities of GDrag.", "questions": "1.\tThe \"rotation\" in the middle row of Figure 3 is somewhat unclear. Is it intended to rotate the back edge of the bottle to the front? Please provide a clearer explanation here.\n2.\tThe reasoning behind the use of low-rank adaptation in line 324 is somewhat vague. Please provide a more detailed explanation here.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tOriginality: The paper addresses the issue of intent ambiguity in dragging-based image editing tasks with an anti-ambiguity dense trajectory calculation method, and specifically constrains movement on dense trajectories, demonstrating good originality overall.\n2.\tQuality: The paper clearly outlines its innovations, presents a reasonable discussion, provides sufficient experimental results, and demonstrates good quality.\n3.\tClarity: The discussion is mostly clear.\n4.\tSignificance: The method proposed in this paper facilitates users in expressing their intentions, enhances interaction, and achieves competitive results, making it of considerable significance.", "weaknesses": "The clarity could be improved. For instance, the term fP* mentioned in lines 345-346 does not appear in Equation 10.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "chX85ZjfY2", "reviewer": "Reviewer_1gRT", "summary": "The paper introduces a task-aware, optimization-based framework for general-purpose interactive editing (GDrag).  GDrag categorizes point-based image manipulations into three atomic tasks based on user intents, and convert sparse trajectories into dense trajectories via a carefully designed graphical user interface. Based on the converted dense trajectories, a set of fine-grained optimization parameters are applied for motion supervision. GDrag achieves superior performance on DragBench.", "questions": "1. The ablation study in Table 3 shows that the proposed Self-adaptive Motion Supervision (SMS) barely contributes to the performance improvements. It will be better to provide comparison between traditional motion supervision and SMS.\n2. Since GDrag uses augmented dense trajectories for motion supervision, the qualitative comparison among other state-of-the-art methods seems a little bit unfair. What do the results of these methods look like using denser trajectories?\n3. The computational complexity in line 299 seems not right. The compared methods usually select a fixed timestep and optimize N times, which is much less than O(TL).\n4. Is it necessary to use features of the original UNet? Why not use features of UNet w/ LoRA directly?\n5. Is there any experiments on the variation of hyper-paramters \\rho and \\beta?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. GDrag focuses on solving the ambiguity of user intents. The proposed Anti-ambiguity Dense Trajectory (ADT) Estimation, i.e., the graphical user interface is interesting, especially for rotation task.\n2. The paper is easy to understand.", "weaknesses": "1. There are not enough experiments to prove the effectiveness of the proposed Self-adaptive Motion Supervision (SMS) module.\n2. The definition of symbols is confusing.\n3. The experiments are not enough to show the superiority of the propose method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "s0N65NfKRQ", "reviewer": "Reviewer_wdkj", "summary": "The paper introduces GDrag, a novel task-aware, optimization-based framework designed for interactive image editing. This method addresses the limitations of existing point-based diffusion models, particularly the challenges of intention ambiguity and content ambiguity in image editing tasks. Existing point-based methods, such as DragDiffusion and FreeDrag, struggle with accurately modeling diverse editing tasks, often leading to mixed or unclear trajectories (intention ambiguity) and a lack of precise target identification (content ambiguity). Current approaches also face challenges in representing 3D manipulations, relying too much on single denoising time steps.\n\n To overcome these issues, GDrag introduces three atomic editing tasks—relocation, rotation (both in-plane and out-of-plane), and non-rigid transformations (like scaling or content creation/removal). This task-aware design allows the system to simplify complex manipulations by breaking them into smaller, specific tasks. ADT (Anti-ambiguity Dense Trajectory Estimation): This component improves the precision of 3D edits by selecting the semantic and geometric neighbors of handle points, allowing the creation of dense, contextually informed trajectories rather than simple 2D lines. SMS optimizes latent features by sampling them from various denoising steps, enabling a more detailed control of motion and addressing content ambiguity. Additionally, SMS applies low-rank adaptation techniques, allowing GDrag to preserve target details at multiple granular levels.", "questions": "- A detailed definition of the Distance metric would be beneficial, particularly in distinguishing it from the Mean Distance metric used in DragDiffusion [1]. Providing a more comprehensive explanation of the Distance metric, including its calculation and interpretation, could help clarify its role in evaluating model performance. Supplementing these details in the Appendix would enhance readability for new readers, offering them a clearer understanding.\n\n\n\n[1] DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing. Shi et al.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "## Originality\nThe paper identifies and addresses key limitations in existing point-based image editing methods, specifically intention ambiguity and content ambiguity, offering a well-defined solution to these issues. \n- By categorizing editing actions into distinct tasks (relocation, rotation, non-rigid transformations), GDrag allows for more targeted and effective edits. This task-oriented approach enables clearer control over image transformations and reduces the complexity of handling diverse editing requirements.\n- Innovative Dense Trajectory Estimation (ADT): ADT is a significant advancement in managing trajectory information. It enhances the precision and reliability of editing by creating a dense point set with contextual information, which is especially valuable for complex 3D manipulations\n- Self-Adaptive Motion Supervision (SMS): The SMS method introduces a robust way to optimize latent features across multiple denoising steps. This enhances generative models’ performance by allowing finer-grained control and preserving target details at various levels, improving the overall quality of the edits.\n\n## Quality\nGDrag is evaluated on DragBench, a benchmark that effectively demonstrates its advantages over existing methods. The results show both quantitative and qualitative improvements in trajectory accuracy and image quality, providing strong empirical evidence for its effectiveness. Compared to other baseline models, GDrag consistently delivers more precise and visually appealing edits, highlighting its promising performance in the interactive image editing.\n\n## Clarity\nThe overall organization and writing of the paper are well-executed. It clearly articulates the significant limitations of current methods in point-based image manipulation task and presents an effective solution. The logic and details of the proposed method are well thought out, with no apparent flaws.\n\n## Significance\nI believe the main work of this paper makes a substantial contribution to the task, particularly in defining the issues of intention and content ambiguity and proposing an effective solution. The overall approach is highly intuitive and, in my opinion, offers valuable insights that could inspire future research in this area.", "weaknesses": "- I personally think that adding more details about the core modules in the ablation study section could make it easier for readers to follow and better grasp the key aspects of these modules.\n- In lines 299-302, the authors employ a random optimization step. I suggest that adding a comparison between fixed and variable optimization steps would enhance the persuasiveness of the paper, providing a clearer understanding of the benefits of using a variable approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IT3yunW1o3", "reviewer": "Reviewer_DgdH", "summary": "This paper proposes a system referred to as GDrag to resolve intention based on user given drag. GDrag introduces two strategies to mitigate content ambiguity, including an anti-ambiguity dense trajectory calculation method and a self-adaptive motion supervision method.", "questions": "My questions are based on weakness. Please give me a rebuttal on them.\nUser intention may highly differ from drag, is there any analysis of understanding human intention based on input drag?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "[1] Proposed problem (intention-awareness) for drag-based editing is challenging issue.\n\n[2] Proposed method is reasonable\n\n[3] Figure is illustrative", "weaknesses": "[1] Introduction: The problem statement lacks persuasiveness due to unclear writing. Begin by explaining what \"handle points\" in drag-based diffusion are. Don’t assume that all readers are already familiar with the task of drag-based diffusion and the terminology used. Without an explanation of handle points, readers may struggle to understand the trajectories derived from them, which ultimately undermines the intent behind the examples illustrating the problem.\n\n[2] Intention Understanding Model: Although the paper is presented as a model for understanding intention, the method proposed seems more focused on effectively executing predefined intentions rather than genuinely understanding them.\n\n[3] Practicality: Since the approach relies on heuristic methods based on categorical intentions, its practical applicability appears limited.\n\n[4] Computational Analysis: Functionally, placing dense points is likely to lead to significant computational overhead. However, there is a lack of experimental validation, as no computational analysis has been conducted to assess this aspect.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QVaUeKloOs", "reviewer": "Reviewer_jimP", "summary": "The primary goal of this work is to address two issues inherent in current point-based image manipulation: intention ambiguity and content ambiguity. To tackle intention ambiguity, the paper defines a taxonomy of atomic manipulations that can be combined to form complex actions. For content ambiguity, the authors introduce the Anti-Ambiguity Dense Trajectory calculation method (ADT) and a Self-Adaptive Motion Supervision method (SMS). In ADT, each atomic manipulation is defined using a dense point set and corresponding point trajectories from an image segmentation model, allowing for better specification of the motion direction for specific tasks. SMS enhances performance by jointly optimizing point-wise adaptation scales and latent feature biases. The proposed methods demonstrate significant advantages in both quantitative and qualitative comparisons.", "questions": "Including a discussion of some failure examples would make the paper more comprehensive.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing is clear, engaging, and easy to understand.\n2. The objectives of addressing intention ambiguity and content ambiguity are both reasonable and aligned with practical needs.\n3. The process of decomposing drag-based manipulation into three atomic manipulations is concise and effective, facilitating future research.\n3. The approach of optimizing $z_0$ to replace optimizing $z_t$ demonstrates a degree of novelty.", "weaknesses": "My primary concern is whether the estimation of dense points and point trajectories proposed in the paper will be affected by the performance of the semantic segmentation model, and whether there might be significant deviations for more complex motions, such as the transition of a hand from an open to a closed position.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)