DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Authors: Siwei Xia
Li Sun
Tiantian Sun
Qingli Li
Venue: ICML 2025 poster (2025)
Keywords: Computer Vision
Generative Model
Diffusion Model
Image Editing
Primary area: applications->computer_vision

Abstract: Drag-based editing within pretrained diffusion model provides a precise and flexible way to manipulate foreground objects. Traditional methods optimize the input feature obtained from DDIM inversion directly, adjusting them iteratively to guide handle points towards target locations. However, these approaches often suffer from limited accuracy due to the low representation ability of the feature in motion supervision, as well as inefficiencies caused by the large search space required for point tracking. To address these limitations, we present DragLoRA, a novel framework that integrates LoRA (Low-Rank Adaptation) adapters into the drag-based editing pipeline. To enhance the training of LoRA adapters, we introduce an additional denoising score distillation loss which regularizes the online model by aligning its output with that of the original model. Additionally, we improve the consistency of motion supervision by adapting the input features using the updated LoRA, giving a more stable and accurate input feature for subsequent operations. Building on this, we design an adaptive optimization scheme that dynamically toggles between two modes, prioritizing efficiency without compromising precision. Extensive experiments demonstrate that DragLoRA significantly enhances the control precision and computational efficiency for drag-based image editing. The Codes of DragLoRA are available at: https://github.com/Sylvie-X/DragLoRA.

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Siwei Xia
Li Sun 12
Tiantian Sun
Qingli Li
User Request
30 sheps
57 shaps
80 steps
User Reqquest
20 steps
60 steps
80 sbeps
DragDiffusio
DrigDettuiion
MinD Curve
1903
Vow
Dragl DragLoRs oRAA
User Roquest (176 13e) steps (31e +6+) steps (49. +9+ steps
User Request
(11e+99e stops (27 +53e) stops (39 +110s) steps
Figure 1. Visual comparison between DragDiffusion (Shi al., 2024b) and DragLoRA at cach step. For given image and user request, we present dragged images at four intermediate steps. In DragLoRA, the steps are divided into two types: DOO+ILFA (Red) or ILFA ILFA-only (Blue), where DOO stands for Obje-Obeective Optimization 80 optimization steps but produces results with lower fidelity. In contrast, DragLoRA achieves more precise deformations with fewer optimization steps and more total steps, which consumes time thanks to the high-efficiency ILFA. The middle winD curves demonstrate that DragL A achieves hetter point tracking, preserving source details more accurately than DragDiffusion.
ILFA stands Input Latent Feature Adaptation. DragDiffusion requires
Abstract
tion ability of the feature in motion supervision, as well as inefficiencies caused by the large search space required for point tracking. To address these limitations, present DragLoRA, novel frame- work that integrates LoRA (Low-Rank Adapta- tion) adapters into the drag-based editing pipeline. To enhance the training of LoRA adapters, we introduce an additional denoising score distilla- tion loss which regularizes the online model by aligning its output with that the original model. Additionally, we improve the consistency of mo- tion supervision by adapting the input features using the updated LoRA, giving more stable and accurate input feature for subsequent operations. Building on this, we design an adaptive optimiza- tion scheme that dynamically toggles between two modes, prioritizing efficiency without compromis- ing precision. Extensive experiments demonstrate
Drag-based editing within pretrained diffusion model provides precise and flexible way to ma- nipulate foreground objects. Traditional meth- ods optimize the input feature obtained from DDIM inversion directly, adjusting them itera- tively to guide handle points towards target lo- cations. However, these approaches often suffer from limited accuracy due to the low representa-
Shanghai Key Laboratory of Multidimensional Informa- tion Processing. *Key Laboratory of Advanced Theory and Application in Statistics and Data Science, East China Nor- mal University, Shanghai, China. Correspondence Li Sun <sunli @ www..cc................
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
that DragLoRA significantly enhances the control precision and computational efficiency for drag- based image editing. The Codes are available at: DoR
noising score (DDS) loss to regularize the online training. Specifically, LoRg first predicts a clean feature from the DDIM-inverted input at timestep of t, which is then perturbed with noise at a randomly sampled timestep t'. The DDS loss is computed as the difference between the noise predictions of the original UNet and the DragLoRA enhanced UNet for the perturbed feature. By jointly mini- mizing the extra loss with the original motion supervision objective, our method preserves semantic fidelity while en- abling flexible deformations. Furthermore, to ensure motion consistency across iterative edits, we adapt the input fea- ture to accumulate deformation effects. At each step, the input feature is denoised from to t 1 using DragLoRA's predicted noise, then re-noised back to with random pertur- bations, for the next optimization. This cycle progressively aligns the feature with the accumulated deformation trajec- tory, propagating handle point adjustments into the latent space and stabilizing motion supervision through coherent feature updates.
1. Introduction
The Stable Diffusion (SD) model has demonstrated remark- able capabilities in synthesizing high-quality images from textual prompts. Based on the pre-trained SD model, many works (Hertz et al., 2022; Zhang al., 2023; Hertz al., 2023) have sought to enhance control over the generated images, often relying on detailed text prompts or reference images to specify generation conditions. While these meth- ods offer various types of editing, they typically require users to provide complex instructions to describe their de- sired outputs, which can be cunbersome and restrictive in scenarios where only minimal modification is required.
Recent advances in based image editing enable intu- itive point-driven manipulation within pre-trained genera- tive models. By specifying pairs of source and target points, users can interactively guide object deformation, iteratively "dragging" content from the source location toward the tar- get. These methods typically operate through two sequential stages: motion supervision, which computes directional gra- dients to align features with the desired movement, and point tracking, which updates handle positions based on the evolving feature space. While this paradigm reduces reliance on complex textual or reference inputs, existing approaches often struggle with precision and efficiency. Di- rect optimization of latent features, such as DDIM-inverter representations, introduces instability due limited fea- ture expressiveness in motion supervision, while the itera- tive tracking process incurs high computational costs from searching large spatial regions.
In practice, we observe that handle points can be driven toward target positions through input adaptation alone, even without explicit motion supervision. This occurs because accumulated gradients from previous optimizations utilized for moving handle points at the new positions with- out extra driving force. In each gradient step, although the specific tasks are not exactly the same, they share a low-variance handle feature and a common direction. Dra- gLoRA can learn these commonalities and generalize, which is comparable with meta-learmin To leverage this, we em- play an adaptive optimization strategy: when point tracking achieves sufficient quality, LoRA updates are bypassed to prioritize efficiency. Conversely, if tracking deviates (e.g., due to occlusions or ambiguous textures), motion supervi- sion is triggered to refine the LoRA parameters, ensuring robust deformation control. By dynamically toggling be- tween motion supervision and input adaptation, DragLoRA enables efficient handle localization with minimal optimiza- tion steps, as it selectively optimizes LoRA only when nec- essary.
be
In this work, we address these limitations by augmenting the SD model with DragLoRA, a dynamically optimized adapter integrated into all attention layers of the pre-trained Unet. Unlike prior methods that directly optimize latent fea- tures, our method performs online adaptive learning during drag-based editing, enabling adjustments of LoRA parame- ters tailored to user interactions. The expanded optimization space enhances the model's capacity to represent foreground deformations, allowing precise alignment of handle points with target positions at each motion supervision step. By decoupling deformation control from static latent representa- tions, DragLoRA mitigates the instability caused by limited feature expressiveness, hence reducing reliance on iterative large-scale searches.
The contributions of this paper lie in following aspects.
We propose DragLoRA, a parameterized adapter en- ables online optimization following user's 's interactions. By replacing direct latent feature optimization with dynamic model adaptation, DragLoRA enhances fine- grained deformation capacity while preserving the pre- trained diffusion priors.
We introduce a dual-objective framework that com- bines drag loss with a DDS loss, computed by com- paring noise predictions of the original and LoRA- augmented Unets on perturbed features. Coupled with a cyclic denoise-renoise process, which iteratively propagates handle point adjustments into the latent
However, we observe that unrestricted LoRA optimization guided solely by drag loss can lead to deviations from the original image. To address this, we introduce a delta de-
2

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
space, this framework ensures semantic fidelity with the source image and stabilizes motion supervision through accumulated deformation trajectories.
els. DragDiffusion (Shi al., 2024b) adapts the point-based dragging technique to the SD model, enhancing generative quality and realizing precise spatial control. SDE-Drag (Nie et al., 2024) presents unified probabilistic formula- tion for diffusion-based image editing, including dragging. DragNoise (Liu et al., 2024) utilizes U-Net's noise predic- tions for efficient point-based editing while maintaining semantic coherence. FreeDrag (Ling et al., 2024) improves stability and efficiency by narrowing the search region for handle points and incorporating adaptive feature updates. EasyDrag (Hou et al., 2024) simplifies the user interaction process, making image editing more intuitive and accessible. GoodDrag (Zhang et al., 2024) introduces an alternating drag and denoising framework, improving result fidelity and reducing distortion. StableDrag (Cui et al., 2024) ad- dresses challenges in point tracking and motion supervision by developing discriminative point tracking method and confidence- based latent enhancement strategy, resulting in more stable and precise drag-based editing. AdaptiveDrag (Chen et al., 2024) proposes a mask-fre free point- point-based editing approach, leveraging super-pixel segmentation for adaptive steps. ClipDrag (Jiang et al., 2024) leverages CLIP for text-guided editing, offering semantic control over image content, while DragText (Choi al., 2024) facilitates text- guided drag by optimizing text embeddings alongside image features. FastDrag (Zhao et al., 2024) enabled quick image modifications without the need for iterative optimization. GDrag (Lin et al., 2025) categorizes point-based manipula- tions into three atomic tasks with dense trajectory, achieving less ambiguous outputs.
We design an adaptive optimization strategy that dy- namically toggles between two modes. By evaluating tracking quality, DragLoRA prioritizes efficient input feature updates when tracking succeeds and activates motion supervision for LoRA refinement when devia- tions occur, minimizing redundant optimization steps.
2. Related Works
Applications of diffusion model on image editing. Diffu- sion models (Ho et al., 2020; Song et al., 2020b; Dhariwal & Nichol, 2021) are initially designed for iterative denoising in the pixel domain and later accelerated by operating in the latent feature space (Rombach et al., 2022). Trained on large-scale image-text datasets, Stable Diffusion can gen- erate high-quality images conditioned on input text. These models enable various image editing tasks with little to no additional training. Typically, the source image is noised to an intermediate timestep suitable for editing, then denoised back with target conditions to modify the content. SDEdit (Meng et al., 2022) applies random noise directly following the DDPM schedule, while DDIM inversion (Song et al., 2020a) and its advanced versions (Mokady et al., 2023; Miyake et al., 2023) have been shown to better preserve details from the source image.
Among various editing techniques, text-based editing is one of the most explored. P2P (Hertz et al., 2022) achieves pre- cise modifications by adjusting the cross-attentio matrix in the UNet, while PnP (Tumanyan et al., 2023) and Masac- trl (Cao et al., 2023) focus on modifying the self-attention layers. Optimization-bu bbased methods, on the other hand, are typically trained on a single image (Kawar et al., 2023; Valevski et al., 2023; Hertz et al., 2023) or a small set of images (Gal et al., 2022; Ruiz et al., 2023; Kumari et al., 2023), allowing the model learn detailed appearances of the editing target and adapt arbitrary text prompts. In contrast, reference-basee based editing directly feeds reference image into the model, often requiring a large dataset for effective learning (Wei et al., 2023; Ye et al., 2023). The ref- erence can take various forms, including a standard image or structural information extracted from the target image (Zhang et al., 2023).
Besides point-based drag editing, DragonDiffusion (Mou et al., 2023), DiffEdiior (Mou et al., 2024), and RegionDrag (Lu et al., 2024) extend drag editing to regions or incorporate a reference image for editing. Additionally, InstantDrag (Shin et al., 2024) and Lightning (Shi et al., 2024a) train models for drag-based based editing, enabling rapid adaptation across various tasks and datasets.
The proposed DragLoRA is point-based drag editing method. Compared to other methods in the same category, it achieves state-of-the-ar performance on key metrics while reducing time costs. In contrast to general models like Light- ningDrag, DragLoRA avoids the heavy burden of offline training and delivers better results.
3. Method
The proposed DragLoRA framework, as is shown in Figure 2, augments the pretrained Stable Diffusion (SD) model with a dynamically optimized adapter to enable interactive drag-based image editing. Our method has two modes for driving handle points, which are LoRA training with motion supervision and input feature adaptation without it. And we design adaptive scheme to combine them for drag-based
Dragged- based image editing has gained significant atten- tion for its intuitive approach to modifying images through user-defined handle and target points. DragGAN (Pan et al., 2023) is the first to demonstrate the feasibility of "drag- ging" in pre-trained StyleGAN models, introducing motion supervision and point tracking mechanisms. Subsequent advancements have expanded this concept to diffusion mod-
3

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
K. stops
DragLoRA Optimization
Ioput Image X
LoRA
LoRA
Inversion
+Noise
User Roquest
VLoos learingtrayy a
30
Point Tracking
Larea
Dragged Image
LoRA
LoRA
LoRA
x
case
+Noise
Point Tricking
Denoising
Input daptation
good
Figure 2. Overview of our proposed DragLoRA. Given an inversion code 2; at t 35 from a source image Zp. we incorporate a LoRA adapter and optimize it online using Lang and Lons Lang primarily guides handle point movement, while Lons constrains LoRA remain close to the original model, preserving the fidelity of the edited image. Additionally, the input feature to the UNet undergoes a denoise-renoise cycle the foreground region, allowing the LoRA adapter to drive handle points even without gradient updates. DragLoRA dynamically switches between motion supervision and input-adaptation-onk, modes based on point tracking quality, cnsuring both efficiency stability the editing process.
editing.
This inversion maps 2g to editable latent time t for different tasks, e.g. #35 for drag editing.
3.1. Preliminaries: Diffusion Model and Drag-based Image Editing
Drag-based image editing is typically performed in pre- trained generative model, such as GAN or diffusion- model. In this task, user provides set source points P. and corresponding target points gi, where pa Hi.g: IR? represent 2D pixel coordinates in the image plane. For SD, LoRA adapter is first fine-tuned on the input image to ensure that the edited results retain a high similarity to the 20 original. The latent feature, 235 at timestep / = 35, is then obtained via DDIM inversion and serves as the optimization target. During editing, two stages alternate:
Diffusion models (Ho et al., 2020; Rombach et al., 2022) have the forward process, in which an input image or latent representation is progressively perturbed by Gaussian noise over timesteps:
24 = Vai 2--1-1 + VI a4 ct.
Vo 20 + vI E,c, ELCC~ N(O,I). I),
where a, = 1-3 and ā. IT', (1- (1-B,) is the cumulative noise retention coefficient, and {3r} defines fixed noise schedule. As T, ãe decays zero, and 2, becomes pure noise. The reverse process aims to recover zn by iiterativeyy denoising 2t Denoising Diffusion Implicit Mod- els (DDIM) (Song et al., 2020a) propose non-Man Non-Markovian sampling process:
Motion Supervision: A gradient-based objective adjusts 2a5 morph the neighborhood of temporal target points h, + d, into the corresponding regions around handle points h, using features from a selected UNet layer to compute directional guidance, as is shown in (4).
au co(2),)) St-1 0 -i-vIn
) ae-1
Ldrag Σ [F h.)) F(235. h, + d (4)
Vāu
(2)
giving an accelerated generation. Crucially, DDIM supports deterministic inversion: given 2p, the noisy latent 2n at any t can be reconstructed via:
where F(-) extracts features from the specified UNet layer and sg(-) detaches the possible gradients. N denotes total number of handle points specified by user. d, is the nor- malized displacement vectors d
| b. To mitigate s. --
2t vT de co co(2),))
2141 Vort1
+VI aef (o(2))))
cumulative errors, we adopt the initial latent input zBs and handle point hς = P; to obtain fixed target features, which
(3)

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
is different from (Pan al., 2023; Shi et al., 2024b). To preserve the original content outside the edited regions, optional constraint LMsk is applied:
makes the model size unchanged compared to (Shi et al., 2024b).
However, we find that optimizing LoRA exclusively for drag-based editing leads to performance degradation, as iterative fine-tuning causes the LoRA-enhancee model to deviate significantly from the original pretrained model. To address this, we propose a dual-objective optimization scheme, combining the drag loss drag with a delta denoising score (DDS) loss LDDS (Hertz al., 2023; Arar al., 2024). Specifically, we first use LrA to predict the clean sig- nal Eg based on the feature 235 from DDIM inversion. The forward process is then applied, adding noise according to (I), transforming 2g into ②y, where t' is a random timestep. Finally, & is passed through both the LoRA-enhanced and pretrained models compute the discrepancy between their noise predictions edrag and e The gradient of the DDS loss is computed as follows:
Mask = 11(334 - = (1-M)lh,,
(5)
where 234 is denoised from 235 by (2) through a forward pass of the model, and M is a given binary mask indicating editable regions. This ensures that non-target areas remain unchanged during optimization.
The updated 2αs is reprocessed through the UNet to locate the new handle positions h,+1, which serve as new guidance for subsequent motion supervision. Here, 12 12) defines a rectangular search area centered at the previous handle point ht, with r2 controlling its size.
Point Tracking:
h = arg min qER(h (2))
|| [[225, ①) - F(255) hb)l, (6)
V AOLLDS (cori - edrgg) Ozo
(8)
dAo
Additionally, the new handle point h h can be used to assess the quality of drag. We compute the best matching distance between h,t and h2, as defined in (7). A lower value of min indicates higher confidence in point tracking and the success of the previous optimization for motion supervision.
Note that this gradient only takes effect through Eg. The total loss can be expressed as L = Larng + > Laakk + LDDS While the drag loss aligns handle points with their target positions, the additional gradient ensures consis- tency with the pretrained model. This -ual-obeective opti- mization (DOO) effectively balances precise deformation control with fidelity to the original model, mitigating the instability introduced by unrestricted LoRA optimization.
minD = [2a5. -
F F(235 11
3.3. Input Latent Feature Adaptation (ILFA)
Furthermore, we evaluate a geometric metric, the Euclidean distance d(hftt1 hy +d,), between h and h +d, Intu- itively, this distance should be small, as motion supervision requires F (=235, by + d:) to align closely with F(2 h?) from the previous optimization step. Thus, the tracked point should not deviate significantly from it. These additional metrics are not used in previous methods and are specific to our proposed online adaptive optimization strategy. Further details can be found in Section 3.4.
To enhance the stability of motion supervision, we introduce a cyclic denoise-renoise process that adapts the input latent feature for drag-based editing. At each iteration, the input feature 235 at timestep t 35 is first denoised to f 1 using DragLoRA's predictions, then re-noised back to t with random Gaussian noise. This cycle propagates handle point adjustments into the latent feature space, ensuring coherent updates on LoRA parameters across iterations.
For denoising, we use the full model, including LoRA, to perform one step of DDIM denoising as defined in (2). Al- though DDIM can also be used for re-noising, we find that it produces inferior results compared to the DDPM schedule defined in (1). Moreover, this scheme is only carried out within the foreground mask, and background region keeps untouched. The proposed ILFA scheme, combined with the dual-objective loss, achieves robust and stable optimization, effectively addressing the challenges of unrestricted LoRA fine-tuning
3.2. DragLoRA and Its Online Optimization
Building on existing works, we introduce DragLoRA, novel framework that improves the precision and efficiency of user-guided deformations through online optimization of LoRA adapters. Instead of directly optimizing latent feature 235, DragLoRA dynamically adjusts the parameters AO of LoRA, which are integrated into the UNet parameterized by 0. This approach increases the capacity of the model and decouples deformation control from latent features, giving fine-grained adjustments while preserving semantic fidelity. To minimize additional computation, we initialize the LoRA with the weights from reconstruction fine-tune tunitunnng on the input image, without introducing extra LoRA modules. This
3.4. Adaptive Optimization Scheme with Two Modes
The input adaptation scheme works effectively with motion supervision. some cases, it can drive the handle points
5

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Algorithm 1 Drag Updates
Input: inverted latent code 21, mask M, unet 0, recon- struction lora 2rrec source points p. target points g Initialize A0 Afrecr =f 2t, h p. k
Handle HainlePointt Target Point.g Interpolatedd
Grid Point
gi
F ModplyModel(0 =0,=4)
while k < K and |// gll2 1 do
F AppplyModel + 20,
h
h, min (F,, F0 .P. p.p.. g.h)
if Asss and min < dj and || - n|12 < l2 then
(a) Neighbourhood Region
(b) Distance- Region
while minD < d2 and |/ gll2 > 1 do
gi
Adapt 24 F =ApplyModel(( 20, 24) h, minD -ointTrrckk( P, F p, 0,
end while
by
else
(c) Angle-closer Region
(c) Linear Region
d
= 9 1g
n
= + d
Optimize A by losses defined as (8))5)))
Figure 3. Comparison among different point tracking schemes. (a) A common strategy locates new point in a square neighborhood around current handle point. (b) and (c) reduces the search region using target point, only grid points in the intersection region are considered as candidate.
Adapt 24
k+ =
end if' end while
(d) uses linear line and needs feature interpolation. (b) and (d) are initially proposed in (Jiang et al., 2024) and (Ling et al., 2024), respectively.
toward their targets even without LoRA updates. This is be- cause DragLoRA learns to move the handle points towards the desired direction through previous optimization steps, re- ducing the need for further LoRA adjustments. This strategy has high efficiency, does not need backpropagation.
reduce the search region, and outperforms traditional methods in terms of efficiency. Unlike conventional neigh- borhood searching, the distance-closer (Jiang et al., 2024) and angle-closer regions constrain the candidate points for handle tracking, preventing unnecessary location reversion. Specifically, the target current handle point serves the center of circle, with their distance defining the radius. Only the grid points within the intersection of the circle (or sector) and the neighboring region are considered as candi- dates for comparison with the original source point feature. A more aggressive search strategy, proposed in (Ling et al., 2024), follows a straight line from the handle to the tar- get point. In DragLoRA, and angle-closer strategies for point tracking, as the former provides the best performance and the latter is the most effi- cient. Additionally, to prevent insufficient optimization due to rapid advancement of point coordinates, we determine whether to proceed or retain the previous handle point based on the minD. Further details are provided in Appendix B.3.
To balance efficiency and robustness, we propose an adap- tive switching scheme (ASS) between two modes, DOO plus ILFA and II FA-lly,,y, based on the quality of point tracking. In the ILFA-only mode, DragLoRA updates the la- tent feature to guide the handle points to the target positions. When point tracking locates confident handle point with small enough minD defined in (7), and is not far from the temporal target h + cl ILFA-only is activated. How- ever, in challenging situations such as occlusions or texture ambiguities, point tracking may degrade, requiring further refinement. In these cases, DOO plus ILFA begins so that a gradient-based objective used to adjust LoRA parameters and stabilize the deformation. This ensures that the handle points remain accurately positioned even difficult scenar- ios. The proposed ASS dynamically toggles between these two modes based on point tracking quality. This adaptive scheme makes DragLoRA to efficiently handle a variety of scenarios while maintaining robustness. Details about are provided in Algorithm
choose the distance-closed
4. Experiments
4.1. Implementation Details
3.5. Efficient Point Tracking (EPT)
We use Stable Diffusion 1.5 (Rombach et al., 2022) as the base model. Following DragDiffusion (Shi et al., 2024b), we train reconstruction LoRA for each image in 80 steps with a learning rate of 0.0005. Given a total of 50 timesteps,
To further enhance the efficiency of drag editing, we investi- gate different point tracking strategies for handle points, as shown in Figure 3. We find that Loa can effectively
6

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
we optimize DragLoRA at = 35 with a learning rate of 0.0001. The ranks of both LoRAs are set 16, and DragLoRA is initialized from RecLoRA, We employ the Adam optimizer, and set Aask = 0.1, ADDS 50, K = 80, kimn = 10, 1 = 1, 1,12 = 1.4, d1 = 1. = 1.3. In terms of EPT, DragLoRA A primarily utilizes the distance-closer region. To speed up the dragging process, the angle-closer region is used in DragLoRA-Fast. After drag updates, we apply DragLoRA to all the remaining timesteps enhance the drag effects. If not specified otherwise, all of our ex- periments are conducted on a single NVIDIA RTX 4090 GPU.
Input
DragDiffusion
Drageoise
GrodDrag
DragLoRA
4.2. Qualitative Evaluation
To validate the effectiveness of our proposed DragLoRA, we conduct extensive experiments on DragBench (Shi et al., 2024b), 1rag100 (Zhang et al., 2024), VChoo- et al,, 2021) and private collections. We compare Dra qualitatively with existing drag methods: DragDiffusion (Shi et al., 2024b), DragNoise (Liu et al., 2024), GoodDrag (Zhang et al., 2024). The visual results are presented in Figure 4. Our method and GoodDrag both demonstrate superior editability, while DragDiffusion and DragNoise fail to achieve the target specified by the input annotations, such as not being able to close the duck's mouth (third row). Additionally, compared to GoodDrag, our DragLoRA maintains higher fidelity. For example, after moving the camera, our method generates a more natural- natural-looking face (first row).
Figure 4. Qualitative comparisons with (Shi ct ct.,, 2024b; Liu et al., 2024: Zhang et al.. 2024). The proposed DragLoRA outperforms existing approaches in both perceptual quality and the accuracy of drag editing.
Evaluation Metrics: Following (Shi et al., 2024b), we adopt I-LPIPS (Zhang et al., 2018) and MD (Mean Dis- tance) to assess consistency with the original image and edit- ing accuracy, respectively. The former calculates I minus feature difference between the original and edited images using AlexNet (Krizhevsky al., 2012), while the latter computes coordinate discrepancy between handle points in the edited image and target points using DIFT (Tang et al., 2023). However, We find that using DIFT to search point coordinates across the entire image may not be ac- curate, leading to unreliable metrics. Similar to (Lu et al., 2024), we introduce m-MD (mas which restricts the DIFT search into the edit region specified by input mask M, thereby reducing uncertain errors. While m-MD is nu- merically lower than MD, it may yield optimistic scores in cases of image distortion, where the mask constraint forces final handle points closer to the target. Thus, the two met- rics complement each other. As demonstrated in Table 1, DragLoRA achieves the best MD and competitive m-MD, with enhanced efficiency, compared to the state-of-the-an method GoodDrag. And our DragLoRA-Faon which adopts angle-closer EPT, demonstrates significant time efficiency among optimization-basee approaches while maintaining strong editability.
4.3. Quantitative Evaluation
To further compare with more methods (Cui et al., 2024;
Hou et al., 2024; Ling et al., 2024; Choi et al., 2024; Chen et al., 2024; Jiang et al., 2024; Lin et al., 2025; Shin et al., 2024; Shi et al., 2024a; Nie et al., 2024; Zhao et al., 2024), we conduct metric evaluations on DragBench. We use pub- liely available code from other works, except for ClipDrag (Jiang et al., 2024) and GDrag(Lin et al., 2025), which has not open-sourced its code, and categorize all drag meth- ods into three types: optimization-basee (Optim), encoder- based(Enc), and training-free (TrFree). We test DragText (Choi et al., 2024) based on DragDiffusion (Shi et al., 2024b). We evaluate the number of trainable parameters, the steps required to pre-train the reconstruction LoRA, the maximum step set for online optimization, and the asso- ciated time. Note that since the reconstruction LoRA can be trained offline and reused for multiple drag editings by various annotations of points, the time spent on this pro- cess is not included, which is about 48s per image over 80 steps on one NVIDIA 4090 GPU. As EasyDrag (Hou et al., 2024) requires more computational resources, we individually evaluate it on one NVIDIA A40 GPU.
Drag-Back Evaluation: As the extent of editing increases, 1-LPIPS naturally decreases. Especially in cases of signifi-
7

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Table Quantitative comparisons on DragBench. Bold: best, underline: second hest, *: open-sourced. DragI.off .oRA adopts distance-
closer EPT while DragLoRA Fast utilizes angle-closer EPT. Our methods demonstrate state-of oate--of-the-see art editing quality and the lowest time consumption among optimization baaseattttt---dddddddee approaches.
METHODS
I-LPIPS
MD
M-MD
TIME(S)
PARAMS(M)
RECSTEPS
DRAGSTEPS
CATEGORY
DRAGDDEFUSION
0.88
32.13
30.71
32.93
0.07
80
80
OPTIM
DRAGNOISE
0.89
35.17
30.66
30.53
0.33
200
OPTIM
STABLEDRAG
0.91
36.46
35.88
37.21
0.07
80
80
OPTIM
EASYDRAG
0.91
38.28
37.67
44.69
0.07
0
80
OPTIM
GOODDRAG
0.87
24.26
21.86
56.97
0.07
70
210
OPTIM
FREEDRAG
0.90
32.30
30.37
51.42
0.07
200
300
OPTIM
DRAGTEXT
0.87
32.87
29.59
42.93
0.12
OPTIM
ADAPTIVEDRAG
0.86
35.70
32.94
77.73
0.07
80
300
OPTIM
CLIPDRAG*
0.88
32.30
/
0.07
160
160
OPTIM
GDRAG*
0.91
26.49
152
0.08
80
250
OPTIM
DRAGLORA
0.87
23.77
22.70
29.84
3.19
80
OPTIM
DRAGLORA-FAST
0.88
27.55
26.90
22.89
3.19
80
80
OPTIM
INSTANTDRAG
0.88
31.51
28.67
1.44
914
ENC
LIGHTNINGDRAG
0.89
29.10
26.77
1.95
933
0
0
ENC
SDE-DRAG
44.48
41.53
62.74
100
TRFREE
FASTDRAG
0.86
31.62
28.29
5.13
0
0
TRFReE
cant edits, it fails to reflect the consistency. Following (Ling et al., 2024), we take a drag-back pipeline to simultaneously measure editability and consistency. After one round of editing, train reconstruction LoRA on the edited image, swap the source points and target points, and perform second round of drag editing. The discrepancy between the second-edited image and the original image is measured by LPIPS and CLIP (Radford et al., 2021). The CLIP metric calculates the similarity of features extracted by the CLIP image encoder from two images. As shown in Figure and Table 2, lower LPIPS and higher CLIP indicate that both rounds of drag editing effectively preserve the original image information, and also suggest that the first round of editing brings the image sufficiently close to the target. Our DragLoRA achieves better trag-back results both visually and quantitatively.
Our baseline starts by optimizing LoRA instead of input latent feature 23s, based on DragDiffusion. Due to the exces- sive updates of LoRA, distortive changes occur in the image, resulting in poor performance. Applying DOO effectively stabilizes the drag updates, improving both LPIPS and CLIP. ILFA helps strengthen the edit accuracy by aligning the lay- out information of input features with LoRA weights. The EPT and ASS are primarily designed to enhance efficiency, The former mitigates the interference from error- error-tracked tracked points, while the latter makes the training process more adaptive to diverse scenarios, thereby also improving ed- itability. More visible results be found in Appendix B.4.
Table 3. Ablation study DragLoRA.
METHODS
I-LPIPS
CLIP
MD L
M-MD +
BASELINE
0.87
9.47
48.55
41.00
Table 2. Drag Back evaluation on DragBench.
+ DOO
0.93
9.8
36.88
34.21
+ ILFA
0.88
9.74
26.99
26.78
+ EPT
0.88
9.74
25.45
25,24
METHODS
LPIPS (x10)
CLIP t
+ ASS
0.87
9.73
23.77
22.70
DRAGDIFFUSION
1.39
0.96
DRAGNOISE
1.64
0.94
DRAGLORA
1.33
0.96
5. Conclusion
GOODDRAG
1.37
0.97
We introduce DragLoRA, novel framework for drag- image editing that improves precision and efficiency through online optimization of LoRA adapters. By replacing latent feature optimization with dynamic model adaptation, Dra- gLoRA enables finer deformations while preserving seman- tic fidelity. The dual-objective optimization, combining drag loss and DDS loss, ensures alignment with pretrained diffu- sion priors, addressing instability from unrestricted LoRA
4.4. Ablation Study
To systematically evaluate the contributions of different modules to the overall performance, we conduct an ablation study, starting from the baseline and incrementally adding modules. We assess their impact on performance metrics 1-L.PIPS and MD. The results are summarized in Table 3.
8

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Impact Statement
Input
DragDiffusion
DragNoise
GoodDrag
Dra
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Drayy
DrapBark
References
Arar, M., Voynov, A., Hertz, A., Avrahami, O., Fruchter, S., Pritch, Y., Cohen-Or, D., and Shamir, A. Palp: prompt aligned personalization of text-to-image models. In SIG- GRAPH Asia 2024 Conference Papers, pp. 1-11, 2024.
DREES
Droc
Cao, M.. Wang, X., Qi. Z., Shan, Y., Qie, X., and Zheng, Y. Masactrl: Tuning- mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 225600, 2023.
Figure 5. Comparisons in Drag-Bate Back pipeline. Following (Ling al., 2024), we perform two symmetric drag edits: the first ad- heres to the input drag annotation and the second reverses it. We focus on the similarity between the drag-back image and the input to validate image fidelity and edit accuracy.
Chen, D., Chen, B., Geng, Y., and Bo, L. Adaptivedrag: Semantic-driven dragging on diffusion--ased image edit- ing. arXiv preprint arXiv:2410. 12696, 2024.
Choi, G., Jeong, T., Hong, S., Joo, J., and Hwang, S.J. Drag- text: Rethinking text embedding in point-based image editing. arXiv preprint arXiv:2407.17843, 2024.
Choi, S., Park, S., Lec, M., and Choo, J. Viton-hd: High- resolution virtual try-on via misalignment-awve aware normal- ization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14131- 14140, 2021.
fine-tuning. The cyclic input feature adaptation and adap- tive optimization further stabilize motion supervision and boost efficiency. Experiments show DragLoRA outperforms existing methods in both precision and runtime, making it a powerful tool for interactive image editing. Future work will extend this framework support flexible drag tasks, in- cluding ogion-bbsed drag, incorporating reference images, and various types of drag adaptation.
Cui, Y., Zhao, X., Zhang, G., Cao, S., Ma, K., and Wang, L. Stabledrag: Stable dragging for point- based image editing, In European Conference Computer Vision (ECCV), 2024.
Linitations. Our work has several limitations that we leave for the future work. First, considering the quantitative met- rics are not direct and accurate, we plan to conduct a user study in the future. Second, there are challenging cases where DragLoRA does not perform optimally, producing edited image with low fidelity, such as moving the camera down to show up the face behind it, which also challenges other methods. We believe advanced generative models can be applied to improve the quality of edited images.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208. 210601088,.. 2022.
Acknowledgements
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. Prompt-to-prompl im- age editing with cross attention control. arXiv preprint arXi::2208.06222 2022.
This work supported by the Science and Technol- ogy Commission of Shanghai Municipality under Grant No.22511105800 195111200 1120800 and 22DZ2229004, the AI project from the economic and information commission of shanghai (Grant No. 1244--ZL-GZZ--000 and ECNU Multifunctional Platform for Innovation (001).
Hertz, A., Aberman, K., and Cohen-Or, D. Delta denoising score. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2328 2328-2337, 2337, 2023.

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851 2020.
Miyake, D., Johara, A., Saito, Y., and Tanaka, T. Negative- prompt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint (rXiv:2305.16800 2023.
Hou, X., Liu, B.. Zhang, Y., Liu, J., Liu, Y., and You, H. Easydrag: Efficient point-based manipulation on diffu- sion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 8404-8413, 2024.
Mokady, R., Hertz, A., Aberman, K., Pritch, Y., and Cohen- Or, D. Null-text inversion for editing real images us- ing guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6038-6047, 2023.
Jiang, Z., Wang, Z.., and Chen, L.. Combing text-based and drag-based editing for precise and flexible image editing. arXiv preprint 0rXiv:2410.0309., 2024.
Mou, C., Wang, X., Song, J., Shan, Y., and Zhang. Dragon- diffusion: Enabling drag-style manipulation on diffusion models. arXiv preprimi (0..i::330.02222.. 2023.
Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I,, and Irani, M. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6007-6017, 2023.
Mou, C.,, Wang, X., Song, J., Shan, Y., and Zhang, J. Diffed- itor: Boosting accuracy and flexibility diffusion-based image editing. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 2024.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
Nie, S., Guo, H. A., Lu, C., Zhou, Y., Zheng, C., and Li, C. The blessing of randomness: Sde beats ode in general diffusion- image editing, In International Confer- ence Learning Representations, 2024.
Kumari, N.. Zhang, B., Zhang, R., Shechtman, E., and Zhu, J.-Y. Multi-concept customization of text-to-image diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1931- 1941, 2023.
Pan, X., Tewari, A., Leimkühler, T., Liu, L., Meka, A., and Theobalt, Drag your gan: Interactive point-based ma- nipulation on the generative image manifold. In ACM SIG- GRAPH 2023 Conference Proceedings, pp. 1-11, 2023.
Poole, B., Jain, A., Barron, J. T., and Mildenhall, B. Dream- fusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.1423 14988, 2022.
Lin, X., Li, H., Cheng, Y., Yan, Y., and Liang, X. Gdrag: Towards general-purpos interactive editing with anti- ambiguity point diffusion. In The Thirleenth International Conference on Learning Representations, 2025.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
Ling, P., Chen, L.,, Zhang, P., Chen, H., Jin, Y., and Zheng, J. Freedrag: Feature dragging for reliable point- image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6860- 6870, 2024.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference computer vision and pattern recognition, pp. 10684-10695, 2022.
Liu, H., Xu, C., Yang, Y., Zeng, L.,, and He, Drag your noise: Interactive point-based editing via diffusion semantie propagation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. -7552, 2024.
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Pro- ceedings of the IEEF/CVE conference on computer vision and pattern recognition, pp. 22500-22511 2023.
Lu, J., Li, X., and Han, K. Regiondrag: Fast region-based image editing with diffusion models. In European Con- ference on Computer Vision (ECCV), 2024.
Meng, C., He, Y., Song, Y., Song. J., Wu, J., Zhu, J.-Y., and Ermon, S. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022.
Shi, Y., Liew, J. H., Yan, H., Tan, V. Y. F., and Feng, J. Lightningdrag: Lightning fast and accurate drag-based based image editing emerging from videos. arXiv preprint arXiv:2405. 13722, 2024a.
10

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Shi, Y., Xue, C., Liew, H., Pan, Yan, H., Zhang, W., Tan, V. Y., and Bai, S. Dragdiffusion: Harnessing diffu- sion models for interactive point-based image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8839-8849, 2024b.
Zhao, X., Guan, Fan, C., Xu, D., Lin, Y., Pan, H., and Feng, P. Fastdrag: Manipulate anything in one step, 2024.
Shin, J., Choi, D., and Park, J. Instantdrag: Improving interactivity in drag-based image editing. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-10, 2024.
Song, J., Meng, C., and Ermon, S. Denoising diffusion im- plicit models. arXiv preprint arXiv:2010.02502, 2020a.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 1..00011134500 13456, 2020b.
Tang, L., Jia, M., Wang, Q. Phoo, C. P., and Hariharan, B. Emergent correspondence from image diffusion. Ad- vances in Neural Information Processing Systems, 36: 2023.
Tumanyan, N., Geyer, M., Bagon, S., and Dekel, T. Plug- and-play diffusion features for text-driven image-to- image translation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 1921-1930 2023.
Valevski, D., Kalman, M., Molad, E., Segalis, E., Matias, Y., and Leviathan, Y. Unitune: Text-driven image editing by fine tuning diffusion model single image. ACM Transactions Graphics (TOG), 42(4):1-10, 2023.
Wei, Y., Zhang, Y., Ji, Z., Bai, J., Zhang, L., and Zuo, W. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15943-15953, 2023.
Ye, H., Zhang, J., Liu, S., Han, X., and Yang, W. Ip-adapter: Text compatible image prompt adapter for toxt-to-image diffusion models. arXiv preprint (xXi:2308.06621. 2023.
Zhang, I.., Rao, A., and Agrawala, M. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Zhang, Z., Liu, H., Chen, J., and Xu, X. Gooddrag: Towards good practices for drag editing with diffusion models. arXiv preprint arXiv:2404 07220. 2024.

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Input
DrffSsiuusoon
DragNoise
GoodDrag
InstantDrag
LightningDrag
Fast
DragI.of LoR
Figure Qualitative comparisons with more methods (Shi et al., 2024b; Liu al., 2024; Zhang et al., 2024; Shin al., 2024; Shi al., 2024a; Zhao et al., 2024).
A. Additional Results
More qualitative comparison results given in Figure 6. It can be observed that DragLoRA achieves the most accurate drag-edit results. As evidenced by Figure 7, the minD of DragLoRA averaged on 205 images from DragBench reaches the lowest through 80 steps, which demonstrates the stability and reliability of our proposed optimization process. Notably, minD-related strategies (ASS and EPT) are not designed to decrease minD, and minD curve comparisons between DragLoRA and other methods are fair. When ASS and EPT are ablated (DragLoRA-wole) wo/minD, dark purple), the observed minD is even lower than DoaA To further evaluate the efficiency of point displacement, we compute the Fuclidean distance between
12

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
handle points and target points d(h gg) each drag step, and report the value averaged over all points and images, which is noted dT. Results in Figure confirm that DragLoRA drives handle points to target positions more efficiently than various existing methods.
DragDiffusion
0.11
DragNoise
DragLoRA
2.0
0.10
GoodDrag
0.09
1.5
0.08
1.0
0.07
DragDiffusion
GoodDrag
0.06
DragNoise
AdaptiveDrag
DragText
stableDrag
0.05
DXagLoRA-wo/minD
EasyDrag
0.0
DragLoRA
0.04
Drag Steps
Drag Steps
Figure Comparisons minD and dl' across different methods. each benchmark image, we record minD dl' from point tracking after each optimization step and compute the average curve across all images. DragLoRA achieves the lowest min and dT, indicating superior dragged results.
B. More discussion
In the main paper, we have studied the effects of Dual-Object Optimization (DOO), Input Latent Feature Adaptation (ILFA), Adaptive Switching Scheme (ASS) and Efficient Point Tracking (EPT). In this section, we provide more analysis on DOO, ILFA and details bout EPT. Furthermore, we provide qualitative ablation study as a supplement the quantitative results.
B.1. Dual-Ohject Optimization
In this section, we analyze the difference between the loss introduced (Hertz et al., 2023) and our LDdS
In the original DDS paper, VoLDDS (Endit erri Ose is applied guide the image towards the semantic editing direction, 86
which points from the original text the target text. In work, however, employ Larug for the editing operation and
use V VDSLDDS (cori drug) &ég 020 regularization term. This regularization constrains the edited model remain close to the original model's generative capability, thereby preventing excessive updates. In essence, our application of the DDS loss is intentionally reversed relative to the original purpose. Therefore, the backpropagated gradient in our method carries an additional negative sign compared to the original DDS loss function.
B.2. Input Latent Feature Adaptation
In this section, we analyze the necessity and theoretical feasibility of ILFA discussed in Section 3.3, building a connection with Score Distillation Sampling (SDS) (Poole et al., 2022) and delta denoising score (DDS) (Hertz et al., 2023). Furthermore, we apply ILFA to DragNoise (Liu et al., 2024) to validate its generalization.
Necessity. Maintaining fixed input feature 235 while solely optimizing DragLoRA preserves the initial spatial layout. This constraint forces the LoRA to generate the growing displacements as handle points approach their targets, creating fundamental conflict with the small-step motion supervision paradigm. As visualized in the cross-attention maps (Figure 8), this mismatch manifests unclear outline of duck's 's beak and suboptimal editing results. To resolve this instability, we propose Input Latent Feature Adaptation (ILFA), which dynamically aligns 235 with the LoRA-learned motion through a denoise-renoise update strategy.
Formula Analysis. ILFA, we denoise the input latent feature 2,, 35 to 2 by Equation (2) and then renoise it to
13

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Input
Cross-attention Mask
Edited Image
wo
ILFA
ILFA
Figure Visualization cross-attention map to reveal layout conflicts between fixed input latent feature and DragLoRA. ILFA mitigates these conflicts and produces more precise editing results.
obtain the new o₁ by Equation (I), We can combine the two equations and rewrite them into a single form as:
Vai Vau-1
vI c0+s0(2e,t)
/ a cots0(2)1)) vI
Vār
(9)
21 + v ar 6 (v1 σe va: - σc)) =0+
Essentially, ILFA involves taking the weighted difference between DragLoRA prediction and random noise, and then using this difference as the editing direction to guide input feature. This similar to SDS, which takes the difference as grad optimize the input feature. If renoise 2 with DDIM inversion as Equation (4), the guidance form looks like DDS. As shown in Table 4, the SDS form works better in DragLoRA. We attribute this the fact that random noise is more capable of altering the inherent layout information and creating suitable information that needs to be filled in.
Table 4. Comparison on ILFA forms.
METHODS
1-LPIPS
MD
ILFA-DDS
0.91
29.33
ILFA-SDS
0.87
23.77
Generality. ILFA is portable and can be applied to other methods like DragNoise, which optimizes the intermediate feature of Unet. As shown Figure 9, ILFA improves the editability and reduces the ambiguity caused by layout conflicts between the input and the intermediate layer.
B.3. Efficient Point Tracking
As introduced Section 3.5, the common neighborhood region is so large that the tracked points could be misled by ambiguous points, causing dragging to be stuck. To improve efficiency and eliminate error- error-prone reverse-direction points, we tested distance-closer (Jiang et al., 2024), angle-closer and linear (Ling et al., 2024) region on DragLoRA, with decreasing number of candidate points. The stricter the constraints on the candidates, the more likely the tracked points are to move towards the targets, which may ultimately lead to under-optimization. Therefore, we introduce an additional confidence check. Specifically, when minD > d2, we determine that the tracked point cannot accurately reflect the current editing state, so we retain the previous point coordinates. As shown in Table 5, the Distance-closer method achieves the best drag edits while the other two cost less time. Since we combine DragLoRA with linear point searching in the FreeDrag way, we also compare with FreeDrag and find that Lra surpasses FreeDrag in every aspect.
14

---

DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model
Input
DragNoise
+ILFA
Figure 9. Comparison on DragNoise-based ILFFA.
Table 5. Comparison on strategies in EPT.
METHODS 1-I.PIPS
MD.
M-MD
TIME(S) 4
DISTANCE-CLOSER
0.87
23.77
22.70
29.84
ANGLE-CL.OSER
0.88
27.55
26.90
22.89
LINEAR
0.90
30.61
30.21
20.79
FREEDRAG
0.90
32.30
30.37
51.42
B.4. Ablation Study
As shown in Figure 10, simply optimizing LoRA accroding to the strategies of DragDiffusion leads to unexpected extreme edits. We increase the image fidelity by applying DOO to restrict the parameters of Dragl from deviating largely. As we successively incorporate ILFA, ASS and EPT, DragLoRA significantly enhances the degree of editing, and these significant edits are stable and reliable.
Input
Baseline
+DO0
+ILFA
+ASS & EPT
Figure 10, Visual ablation study.
15


Reviews:
[{"review_id": "BFsLOUSHAN", "reviewer": "Reviewer_iyEZ", "summary": "This paper introduces DragLoRA, a framework that integrates LoRA into drag-based editing. Instead of optimizing the input feature obtained from DDIM inversion, the paper claims to improve accuracy and efficiency by utilizing lora adaptation. A denoising score distillation loss is proposed to align the outputs with the original model , ensuring stable and accurate features for motion supervision. An adaptive optimization scheme enhances both precision and computational efficiency. The experiments show that DragLoRA boosts efficiency in drag-based editing benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback regarding the clarity of our Preliminaries section and symbolic expressions. We appreciate the opportunity to clarify our approach.\n\n### **More details about drag-based image editing, particularly its motion supervision and point tracking.**\n\nDrag aims to make the semantic positions of the handle points $\\mathbf{h}\\in \\mathbb{R}^2$ reach targets $\\mathbf{g}\\in \\mathbb{R}^2$ through online optimization, breaking down the journey of $\\mathbf{h}$ into small steps. Each step consists of **motion supervision** and **point tracking**:\n* **Motion supervision** relies on $L_\\text{drag}$ in eq.(4) to optimize input latent feature $z_{35}$, obtained by DDIM inversion, transforming the region around temporal target points $\\mathbf{h} + \\mathbf{d}$ into handle points region, where $\\mathbf{d}$ means a small displacement, and implicitly filling the handle point region by utilizing the generation capability of model. \n* **Point tracking** locates the new handle points $\\mathbf{h}$ on updated deep features in Unet, as defined in eq.(6), which gives a new guidance for next-step motion supervision. \n\nThis optimization is performed per image, eliminating the need for a training set.\n\n### **Difference between DragLoRA and LoRA finetuning.**\n\nFor our work, the DragLoRA is online optimized to control the dynamic motion, gradually editing local image features towards the final goal, which is quite different from simply fine-tuning the Unet with LoRA based on a set of images or a single image. Instead, we intend to control the optimization and use LoRA to steer the handle points.\n\n### **Motivation behind each module.**\nBesides, as shown in Fig.10 and Tab.3, only changing the trainable parameters from $z_{35}$ to LoRA $\\Delta \\theta$, which serves as our baseline, proves to be unstable. To address this, we devise a series of strategies based on problem analysis:\n\n1. The primary challenge of optimizing LoRA is that excessive deviation in LoRA parameters can compromise the generative capability of model. Regarding this, we introduce $L_\\text{DDS}$ as Eq.(8) to build **dual-objective optimization (DOO)**, which aligns the generative capability of the optimized model with the pre-trained model.\n\n2. Meanwhile, the fixed input feature $z_{35}$ retains its initial spatial layout, which requires the LoRA to generate the growing displacements for handles as they move towards the targets, and it conflicts with the small step during drag, contributing to instability of optimization. This phenomenon can be observed in cross-attention map visualizations (see Fig. 1 at: https://imgur.com/a/Cb01npK), where the conflicts lead to unclear mask of duck's beak and suboptimal results. To mitigate this, we design **input latent feature adaptation (ILFA)** to align the input $z_{35}$, updating it with the motion learned by LoRA in a denoise-renoise way.\n\n3. Through experiments, we observe that once LoRA is well optimized, applying multiple ILFA steps alone can facilitate the movement of handle points. Since ILFA is training-free, it is faster than DOO. This suggests the potential benefits of a balanced integration of ILFA and DOO. To achieve this balance dynamically, we propose an **adaptive switching strategy (ASS)** that:\n(1) Solely applies ILFA when LoRA has learned the accurate motion.\n(2) Reactivates LoRA optimization when ILFA alone becomes unreliable.\n\n4. While ASS enhances motion supervision efficiency, we further propose an **efficient point tracking (EPT)** with limited search region and minD-based retreat, achieving an balance between computational cost and editing effectiveness.\n\nEach module in our design is introduced progressively based on problem analysis and experimental observations. Rather than being arbitrary design choices, they address specific limitations of directly applying LoRA. We hope this clarification provides a stronger motivation for our method and demonstrates the necessity of our contributions.\n\n### **Comparisons with other works.**\nThe combination of modules above achieves an much higher editing accuracy than most of the methods, and our results are comparable with GoodDrag. While GoodDrag optimizes input $z$ over multiple timesteps, sacrificing efficiency, our method costs the least time among optimization-based methods.\n\nThank you again for your insightful feedback. We eagerly await your feedback on our rebuttal.", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "The paper proposes three modules: A dynamically lora adapter integrated into UNet for optimizing the editing task, a DDS Loss that claimed by the paper to establish \"semantic fidelity\" by comparing noise predictions of the original and LoRA-enhanced UNet, and \"Adaptive Optimization\", which is a strategy that dynamically switches between efficient input updates and motion supervision based on tracking quality.   \n\n1. The dynamically lora adapter is simply using lora on Unet for fine-tuning the diffusion model, which seems incremental and naive. \n2. The paper proposes three modules with complex design, while I do not see strong motivations to do so.  I personally do not like papers that introduce intricate methods without clear justification.  Furthermore, the experiments do not show that the method significantly overperforms other methods by a notable margin.  I think this field still lacks high-quality data at scale, and fancy tricks or heuristic design can only bring incremental improvements.", "theoretical_claims": "No theoretical claims given.", "experimental_designs_or_analyses": "The experimental result is based on a well-established benchmark: DragBench and Drag100.  The paper also conducts a sufficient comparison with other methods.  The ablation study also verifies the effectiveness.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "The paper has discussed essential references", "other_strengths_and_weaknesses": "I also highly recommend to make the Preliminaries section easy-to-understand to help readers who are unfamiliar with Motion Supervision and Point Tracking.", "other_comments_or_suggestions": "target points \"t\" and timestep \"t\" should used different denotation.", "questions_for_authors": "N/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L0L69DerKo", "reviewer": "Reviewer_xAC9", "summary": "This paper introduces the DragLoRA framework to enhance point tracking in drag-based image editing, thereby improving editing precision. It proposes a DDS loss combined with drag loss, along with a cyclic denoise-renoise process to maintain semantic fidelity with the source image. Additionally, an adaptive optimization scheme is employed to minimize further LoRA adjustments, improving overall efficiency.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback. We address your concerns below:\n\n### **1. Test data details.**\nWe mainly conduct our experiments on DragBench and Drag100, which are proposed by DragDiffusion and GoodDrag. The qualitative comparisons in Fig.4. and Fig.6 are mainly based on these two test sets. The qualitative comparisons on DragBench has been presented in Tab.1 while results on Drag100 are given below:\n| Methods       | MD↓    | m-MD↓  | 1-LPIPS↑ |  Time(s)↓    |\n|---------------|----------|----------|----------|----------|\n| GoodDrag      | 26.72 | 25.26 | 0.86  |  72.06 |\n| DragLoRA      | **26.31**  | **25.20**  | **0.87**|37.93  |\n| DragLoRA-Fast | 29.18 | 27.78 | **0.87** | **24.32**|\n\nCompared to GoodDrag, DragLoRA achieves better MD and m-MD on Drag100, and also reduces editing time by 47.4%, while DragLoRA-Fast reduces editing time by 66.2%.\n\nTo enrich the test scenarios, we select a few images from VITON-HD$^{[1]}$ dataset and our private collections, and manually annotate them with handle-target points and masks. Qualitative comparisons on these constructed pairs are presented in row 5 of Fig.4 and rows 3–4 of Fig.6. These original images and results will be released on our Github. \n\n### **2. MD v.s. m-MD.**\nWe observe that applying DIFT to search final handle points $h_f$ around the entire edited image may mislead the found points to excessively deviate from editing region, especially when there are semantically similar points with the source handle in the image, resulting in high MD. E.g., points on the left and right hands are often confused by DIFT. To eliminate these cases, given the user input mask $M$ to indicate the editing region, we treat points within $M$ as candidate set $\\Omega_\\text{mask}$. Formally, the searching process is:\n\\begin{equation}\nh_f = \\arg\\min_{p \\in \\Omega_\\text{mask}} d(F_{ori}(h), F_{edit}(p))\n\\end{equation}\nwhere $F_{ori}, F_{edit}$ are feature maps extracted by Stable Diffusion 2.1 from original and edited images, $d$ is a cosine distance metric, and $h$ is the given handle points in original image.\nFollowing MD, we calculate the mean Euclidean distance between target points and their corresponding final handle point to derive m-MD.\nWhile m-MD is numerically lower than MD, it may yield optimistic scores in cases of image distortion, where the mask constraint forces final handle points closer to the target. Thus, the two metrics complement each other.\nNotably, the inherent limitations of both MD and m-MD affect all methods uniformly, making them fair for comparison. \n\n### **3. Quantitative comparison.**\nIn terms of both MD/m-MD and visual editing quality, our DragLoRA performs comparably to GoodDrag and shows significant advantages over other methods. Meanwhile, our approach achieves the highest efficiency among optimization-based methods, substantially reducing time compared to GoodDrag, which is presented in Tab.1. \n\nBesides, as mentioned in Sec. 4.3, 1-LPIPS inherently favors under-edited results. StableDrag and EasyDrag achieve high 1-LPIPS by insufficiently moving points (see their larger MD/m-MD in Tab.1). As a supplement, we introduce the DragBack pipeline (Sec. 4.5) to measure the similarity between the edited image and original image. After two symmetric edits, there is no doubt that the higher the similarity, the better. As shown in Tab.2, DragLoRA achieves the best LPIPS while GoodDrag achieves the best CLIP, both showing high consistency with original image, which can not be verified in Tab.1.\n\n### **4. Figure Rendering Issues.**\nPerhaps the \"?\" symbols in Figs. 2–3 are related to rendering engine version of Edge browser. We reconfirm that:\nsymbols render correctly in Chrome, macOS Preview and other PDF viewers.\n\n### **5. Limitations.**\nWe acknowledge that a more detailed discussion of our limitations would improve the manuscript. In the revised version, we will add a section that presents the following points: (1) considering the quantitative metrics are not direct and accurate, we plan to conduct a user study in the future. (2) There are challenging cases where DragLoRA does not perform optimally, producing edited image with low fidelity, such as moving the camera down to show up the face behind it, which also challenges other methods. (3) Due to computational constraints, we have not integrated the latest generative models with our method.\n\nThank you once again for your valuable comments and suggestions. We eagerly await your feedback on our rebuttal.\n\n[1] Choi S, Park S, Lee M, et al. Viton-hd: High-resolution virtual try-on via misalignment-aware normalization[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 14131-14140.", "claims_and_evidence": "All claims are supported by clear evidence.", "methods_and_evaluation_criteria": "The method and evaluation criteria make sense for the problem.", "theoretical_claims": "No theoretical claims.", "experimental_designs_or_analyses": "In the experiments section, the paper selects DragDiffusion, DragNoise, and GoodDrag as the main comparison methods. For DragDiffusion and DragNoise, the same dataset and metrics are used, whereas GoodDrag includes an additional dataset and new metrics. However, these are missing from the quantitative comparison.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "Essential references are all discussed.", "other_strengths_and_weaknesses": "Strength: \n\n1.\tThe DragLoRA framework enhances the point tracking stage, which is crucial for achieving high image quality and precise drag-editing results.\n2.\tVarious point tracking schemes for drag editing are discussed, and the proposed search strategy is more advanced compared to the vanilla method\n3.\tIn quantitative comparisons, the proposed method is evaluated against many approaches, demonstrating both high image fidelity and time efficiency.\n\nWeakness:\n\n1.\tThe paper mentions other selected images, more details are needed for this new dataset.\n2.\tFor m-MD, further clarification is required on how it is computed and why it is an improvement over MD.\n3.\tBased on Table 2, which presents the quantitative comparison on DragBench, the paper claims to achieve state-of-the-art editing quality. However, for 1-LPIPS, STABLEDRAG and EASYDRAG outperform the proposed method, while for m-MD, GoodDrag performs better. In Lines 366–367 of the Evaluation Metrics section, the paper states that m-MD reduces uncertainty errors compared to MD. If m-MD is indeed superior to MD, then the proposed method does not achieve state-of-the-art performance in any metric presented in Table 2.", "other_comments_or_suggestions": "1.\tIn Figures 2 and 3, some characters appear as \"?\".\n2.\tThe paper lacks discussion about its limitations.", "questions_for_authors": "No additional questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fun33WGtte", "reviewer": "Reviewer_TdyS", "summary": "The authors presented the DragLoRA method for efficient and more accurate drag-editing, which is specified by a mask of the area to be edited and pairs of points specifying the direction of shift editing. The method is based on the use of LoRA adaptors for simultaneous gradual shifting towards target points, but at the same time keeping the model prediction with adaptors close to the original model ones. For this purpose, the authors use dual-objective LoRA optimization (DOO), combining feature shift loss and DDS loss. In addition, the authors propose an adaptive switching scheme (ASS), the idea of which is to choose a strategy with or without updating the adapter weights depending on how far the handle point is from the target point. The authors also propose Input Latent Feature Adaptation (ILFA) mechanism, which is to renoise the latent once by 1 step and then noise with Gaussian noise. The authors do this adaptation at each step of the algorithm regardless of whether there is an optimization step, and the goal is to ensure coherent updates on LoRA parameters across iterations. Finally, the authors use Efficient Point Tracking (EPT) to efficiently select the next handle points, reducing the complete enumeration. The proposed method requires fewer optimization steps compared to existing methods and preserves the details of the original image better.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and constructive feedback. Below are our point-by-point responses:\n\n### **1. Runtime Comparison Fairness.**\nOur reported time excludes the offline LoRA finetuning time for reconstruction (~48s per image over 80 steps on NVIDIA 4090 GPU), which applies to **all methods** with reconstruction steps $>0$. \nWe agree that this is long and it can't be ignored. However, the trained LoRA can be stored and reused for multiple drag editings by various annotations of points. They can also be applied in other customized editing. Here, the exclusion of offline training emphasizes the comparison of time cost on dragging. We will describe this cost in the revised paper.\n\n\n### **2. Clarification on minD.**\nWe propose minD, which measures the distance between feature of the tracked new handle points and initial handle point, to serve as a training-time indicator of the stability of the LoRA optimization. However, lower minD does not definitly result in good drag quality. Metrics such as MD and m-MD are more reliable for evaluating the final results. Therefore, minD curves are used only for reference.\n\nSpecifically, minD is applied in Adaptive Switching Scheme (ASS) and Efficient Point Tracking (EPT). In ASS, minD and coordinate distance assess whether LoRA or input latent feature updates are confident. If updates are confident, the system enters IFLA-only mode; otherwise, $L_{drag}$ continues driving in next step. In EPT, minD checks if tracked handle points accurately reflect the previous updates. If minD is low, the system moves to the next step; otherwise, it stays with additional motion supervision. MinD does not decrease in ASS, and is only lowered down in EPT. When the ASS and EPT strategies are ablated (DragLoRA-wo/minD, dark purple), as shown in https://imgur.com/a/UYhzqfD, the observed minD is even lower than DragLoRA. So we mainly use minD to gauge the online optimization, but do not reduce its value in purpose. The lower minD in DragLoRA mainly comes from the LoRA structure and DOO.\n\n### **3. Incorporation of GDrag and SDE-Drag.**\nThank you for the advices on adding essential references. SDE-Drag presents a unified probabilistic formulation for diffusion-based image editing, including drag, while GDrag categorizes point-based manipulations into three atomic tasks with their dense trajectory, and jointly optimizes point-wise adaptation scales and latent feature biases at a sampled timestep, achieving less ambiguous outputs.\n\nWe adopt the metric values reported by GDrag and evaluate SDE-Drag on DragBench introduced by DragDiffusion. As shown below, DragLoRA achieves better editing accuracy than the others while GDrag achieves better image consistency.\n\n| Methods        | 1-LPIPS↑ | MD↓    | m-MD↓  | Time(s)↓ | Params(M) | RecSteps | DragSteps | Category |\n|----------------|----------|--------|--------|----------|-----------|----------|-----------|----------|\n| SDE-Drag       |  **0.91**   |  44.48 | 41.53 | 62.74|     0     | 100        |     0     | TrFree   |\n| GDrag         | **0.91** |  26.49 |   /   |     /    | 0.08      |  80      | 250       | Optim    |\n| **DragLoRA**   | 0.87     | **23.77**|  **22.70** | **29.84** | 3.19      | 80       | 80        | Optim    |\n\n### **4. Generative model Limitation.**\nSince all methods are based on Stable Diffusion 1.5, we also implement our method on this model for fairness. While Flux/SD3 have higher generative capacity, indicating the potential to boost editing performance, they require more cost on both time and computation sources. We will explore this potential in the future and release code for easy model swapping among SDXL and Stable Diffusion 2.1.\n\n### **5. Parameter Selection in Pseudocode.**\nSince focusing on practical strategy, we do have some threshold params in Alg. 1 determined via empirical analysis: $k_{ini}=10,th_{low}=1,th_{high}=1.3,d_m=1.4$. As shown in Fig.7, after $k_{ini}$ steps, minD plateaus, indicating that LoRA optimization tends to be stable. According to the observation on DragLoRA wo/ ASS, minD $>th_{high}$, usually correlates with unrealistic edits. Besides, we set $minD<th_{low}$ and $||h-n||_2<d_m$ to indicate high confidence to balance the efficiency and stability. However, these two values are still not fully tuned.\n\n### **6. Current Limitations of Drag-Editing Methods.**\nWe acknowledge that a limitation common to current drag-editing methods, including our own, is that in some challenging cases the output may still appear unnatural or low edit-accuracy. In the case of \"moving the camera away to supplement the girl's facial details,\" the editing results can be suboptimal. We believe that incorporating reference face ID additionally could help meet user expectations better.\n\n### **7. Symbol Corrections and Minor Issues.**\nWe have noted the suggested corrections and will address them in the revised version.\n\nThank you again for your valuable comments. We eagerly await your feedback on our rebuttal.", "claims_and_evidence": "The authors provide an extensive comparison with existing methods, both quantitative and qualitative. However, there are questions about the visual quality of the method. In Figure 4, the first and third rows show results that are better than the baselines, but the proposed method outputs a rather unrealistic image. The girl in the first row has unrealistic facial features and the camera has lost its shape. Given that these images are highlighted as the best, there are questions about the real quality of the method and how applicable it is in practice.", "methods_and_evaluation_criteria": "The proposed method solves the drag-editing problem, which has been widely spread recently, as it could simplify the image editing process for real users. The problem has been studied many times and methods that solve it have been proposed, and there are also benchmarks such as DragBench in order to compare the methods proposed in the community. The authors use just this benchmark for comparison. The metrics that the authors use are common and standard for this task.", "theoretical_claims": "For the most part, there is no theoretical evidence in the article. However, the supplementary contains the derivation of formula 9 for latent updating using the Input Latent Feature Adaptation (ILFA) mechanism. I see no inaccuracies in this conclusion.", "experimental_designs_or_analyses": "For the experimental part of the work, the authors chose to compare the proposed method with baselines on the DragBench dataset. The metrics chosen for quantitative comparison are 1-LPIPS, Mean Distance (MD) and masked Mean Distance. Also, a comparison in terms of the running time and the number of parameters is added in Table 1. From the comparison, it is observed that in terms of MD and m-MD metrics, the proposed method is indeed on top. However, I think it is not very fair to compare by runtime. The authors write in Section 4.3 that for DragLoRA, only the time required for finetuning for editing was included in the comparison table, and the time for initializing this finetuning (actually finetuning on reconstruction) was not included. In this regard, there is a question, for the DragDiffision method the corresponding time was also excluded in the comparison? What is the order of the time required for this step in general? Is it possible to neglect it in the comparison on the basis of the fact that it can be done offline, although the user still has to wait for this time to upload his image?\n\nThe authors also offer an interesting analysis on the Drag-Back Pipeline. The image is first edited with a set of point pairs and then edited again in the reverse direction of point shifts. This should ideally result in a reconstruction of the original image. The authors compare a subset of methods on this task and show in Table 2 that in terms of LPIPS and CLIP metrics, which are responsible for image preservation, the proposed method performs almost the best, which is supported by the visual comparison in Figure 5.\n\nFinally, the authors conducted an ablation study (Table 3) in which the authors show the need for each of the four components of the method, which was supported by the visual comparison in Figure. 10.", "relation_to_broader_scientific_literature": "The idea of optimizing weights by gradually shifting points in a given direction is not new and has been proposed as early as DragGan and DragDiffusion. However, the authors proposed a number of modifications that improved the quality of the Pipeline. As mentioned earlier, the proposed method consists of 4 main components.\n\n1. Dual-objective LoRA optimization (DOO)\n\n2. Input Latent Feature Adaptation (ILFA)\n\n3. Adaptive switching scheme (ASS)\n\n4. Efficient Point Tracking (EPT)\n\nDOO uses delta denoising score, which has been proposed earlier but not used as a component of loRA for optimization in drag-editing. The combination of the two losses is rather a new approach for this task.\n\nILFA has also not been presented in the literature before, but there was an attempt to use a close idea in GoodDrag.\n\nASS is an entirely new idea, although it stands out for its simplicity.\n\nFinally, EPT has already been proposed in papers such as CLIPDrag and FreeDrag and is not novel.\n\nOverall the combination of these ideas has shown to be quite good, there are both completely new ideas as well as utilizing existing ones as separate components. The main novelty is in the combination of these parts with each other, which gave a fairly high quality output.", "essential_references_not_discussed": "GDrag [1] is one of the latest published methods that shows high metrics on a drag-editing task.\n\nThe SDE-Drag [2] article showed that using SDE instead of ODE improves the quality of various types of editing, including drag-editing.\n\n[1] GDrag:Towards General-Purpose Interactive Editing with Anti-ambiguity Point Diffusion, Xiaojian Lin, Hanhui Li, Yuhao Cheng, Yiqiang Yan, Xiaodan Liang\n\n[2] The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing, Shen Nie, Hanzhong Allan Guo, Cheng Lu, Yuhao Zhou, Chenyu Zheng, Chongxuan Li", "other_strengths_and_weaknesses": "**Strengths:**\n\nA clear and reasonable idea of the method.\n\n**Weaknesses:**\n\nThe method is based on the outdated sd-1.5 model, the quality of which is initially inferior to such models as SD3, Flux.", "other_comments_or_suggestions": "Typos:\n\n127: InstanceDrag → InstantDrag\n\n305: DragDiffuaion → DragDiffusion\n\n660: Aadption → Adaption", "questions_for_authors": "1. My main question is, as I wrote earlier about measuring the runtime of the method. For the DragDiffision method the corresponding time was also excluded in the comparison? What is the order of the time required for this step in general? Is it possible to neglect it in the comparison on the basis of the fact that it can be done offline, although the user still has to wait for this time to upload his image?\n2. How were the constants used in the pseudocode selected? What is their intuition? How did they get out? Is there an analysis of them?\n3. How do you assess the applicability of the proposed method, taking into account such significant distortions in the visual comparison in Figure 4, as I wrote earlier? Will the user be satisfied with such unrealistic output images?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5PS5LKTfCB", "reviewer": "Reviewer_Bmt5", "summary": "The paper proposes a new method, named DragLoRA, for drag-based image editing using a pre-trained Stable Diffusion model. Specifically, the paper proposes two novel steps: DragLoRA online optimization (DOO) and Input Latent Feature Adaptation (ILFA). In DOO, instead of optimizing the latent representation like the classic DragDiffusion, it optimizes the LoRA with feature drag loss and the DDS loss. In IFLA, the latent representation passes through DragLoRA and is renoised to adapt the latent representation to dragging. Meanwhile, the paper also discovered that IFLA itself could occasionally achieve point moving, even without DOO steps. Therefore, the paper proposes an adaptive switching scheme (ASS) based on point tracking: if IFLA itself can achieve reliable point moving, DOO can be skipped for this step until the result of point tracking reveals that IFLA cannot provide a good move, DOO is resumed to supervise point moving using optimization. DragLoRA is evaluated on the DragBench benchmark against previous methods and shows superior performance in the MD metric with comparable computational time.\n\n## Update after rebuttal\nI appreciate the author's response, and it solved most of my concerns. Therefore, I keep my original rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your insightful and constructive feedback. We appreciate the depth of your analysis and the valuable suggestions you provided. Below, we address your concerns point by point:\n\n### **1. Difference from the Original DDS Loss.**\n\nIn the original DDS paper, $\\nabla_{\\theta} L_{\\text{DDS}}=(\\epsilon^{edit}-\\epsilon^{ori})\\frac{\\partial{z_0}}{\\partial {\\theta}}$ is applied to guide the image towards the semantic editing direction, which points from the original text to the target text. In our work, however, we employ $L_\\text{drag}$ for the editing operation and use $\\nabla_{\\Delta\\theta} L_{\\text{DDS}}=(\\epsilon^{ori}-\\epsilon^{drag})\\frac{\\partial{\\hat{z_0}}}{\\partial {\\Delta\\theta}}$ as a regularization term. This regularization constrains the edited model to remain close to the original model’s generative capability, thereby preventing excessive updates. In essence, our application of the DDS loss is intentionally reversed relative to the original purpose. That's why we apply grad $\\epsilon^\\text{ori} - \\epsilon^\\text{edit}$ to $\\hat{z}_0$.\n\n### **2. Intuition Behind ILFA Without DOO.**\n\nWe observe that after sufficient DOO (Dual-objective LoRA optimization), ILFA (Input Latent Feature Adaptation) can solely move the handle point. In this setting, accumulated DOO gradients from previous optimizations, can be utilized for moving handle points at the new positions without extra driving force. In each gradient step, although the specific tasks are not exactly the same, they share a low-variance handle feature and a common direction. DragLoRA can learn these commonalities and generalize, which is comparable with meta-learning. Note that ILFA-only is not stable enough, so we introduce adaptive switching scheme (ASS). Here we explain this process and intuition behind this:\n\n* **Initialization and Thresholding:** We initialize DragLoRA from a reconstruction LoRA and activate the no-training mode only after LoRA has been optimized for a certain number of steps. This ensures that the LoRA module learns a robust directional signal over multiple iterations, rather than merely shifting from one fixed point to another.\n* **Optimization Confidence Assessment:** We employ minD and coordinate distance of points to evaluate the confidence of previous-step LoRA optimization. When LoRA's guidance is deemed reliable with an effective movement and a smaller value of minD, ILFA can incrementally adjust the nearby handle points in a small, controlled manner, thereby effectively advancing in the intended direction.\n* **Fallback Mechanism:** In cases where LoRA is compromised due to overfitting, ILFA can hardly advance the handle points that LoRA is unfamiliar with or produce large minD. Our ASS strategy can detect these unreliable cases and switch back to DOO with a new supervisory signal.\n\nIn summary, IFLA-only can be potentially used with detection metric as the safty guard. \n\n### **3. Time Efficiency via Adaptive Switching.**\nYou correctly noted that our ASS allows us to bypass several DOO steps by relying on the training-free ILFA mechanism. This substantially reduces the number of optimization steps and, as a result, saves significant time. Specifically, the time consumed by one DOO step is around **0.26s** while one ILFA step costs around **0.05s**, and we apply **54** DOO steps and **87** ILFA steps per image on DragBench.\n\n### **4. Visualization of the Optimization Process.**\nWe appreciate your suggestion to enhance the visualization of our optimization process. In response, we have supplemented our manuscript with two additional figures that more clearly illustrate the progression of the drag editing process. The first image, which separates the ILFA-only steps (displayed in a different color) from the DOO+ILFA steps, clearly indicates how many optimization steps are skipped. And the second image, which calculates the average normalized Euclidean distance $d(h_i^k,t_i)$ on DragBench, indicates how fast the handle points move. You can view these visualizations at the following link: https://imgur.com/a/PVuceIo.\n\n### **5. Incorporating Human Evaluation.**\nWe agree that human evaluation is crucial for fully assessing the quality of editing results. While our current work primarily relies on quantitative metrics, a comprehensive user study would provide further insights. We plan to conduct such evaluations and release the results in our Github.\n\n### **6. Symbol Corrections and Minor Issues.**\nWe have noted the suggested corrections and will address them in the revised version of our manuscript.\n\nThank you once again for your detailed and thoughtful comments. We eagerly await your feedback on our rebuttal.", "claims_and_evidence": "Based on the results of experiments, DragLoRA improves the baseline on MD and m-MD metrics in the DragBench benchmark (Table 1). The effectiveness of each proposed components: DOO, ILFA, ASS and EPT are demonstrated by the ablation study in Table 3 and Figure 10 in the supplementary material. Therefore, the effectiveness of DragLoRA could not be denied. However, some points require further clarification to understand how they work, and I lay out my questions regarding this part in **Methods And Evaluation Criteria** section. \n\nExperiments are thorough in general, and they do substantiate the claims. However, I have some concerns and questions regarding the experiments, and I elaborate on these in **Experimental Designs Or Analysis** section.", "methods_and_evaluation_criteria": "Although the effectiveness of the proposed method is empirically demonstrated, further clarification is required on some points:\n\n1. The paper employs the DDS loss to reduce the performance degradation caused by LoRA optimization. The original DSS loss aims to have a cleaner gradient than the SDS loss in editing direction by deducting bias from the editing SDS loss. Mathematically, it is $\\nabla_{\\theta} L_{DSS} = \\nabla_{\\theta} L_{SDS}(\\hat{z}, c_{edit}) - \\nabla_{\\theta} L_{SDS}(z, c_{ori})$. The bias is $\\nabla_{\\theta} L_{SDS}(z, c_{ori})$ and the intuition is that if the latent of the original image $z$ already aligns with the text $c_{ori}$, the derivative should be zero and any remaining value is bias. If we apply the analogy to this case, shouldn't the DSS loss be $(\\epsilon^{drag} - \\epsilon^{ori})\\frac{\\partial \\hat{z_{0}}}{\\partial \\Delta \\theta}$ rather than $(\\epsilon^{ori} - \\epsilon^{drag})\\frac{\\partial \\hat{z_{0}}}{\\partial \\Delta \\theta} $ as shown in equation 8?\n\n2. The adaptive optimization scheme noticed that after DOO, LoTA can move the handle point toward the targets without DOO. There lacks a intuition behind this phenomenon. Finetuning LoRA for one moving step is very likely to cause LoRA to overfit to that specific step, so it is quite counter-intuitive in this point.", "theoretical_claims": "This paper is an application paper with no significant theoretical contribution.", "experimental_designs_or_analyses": "1. In Table 1, it is shown that DragLoRA achieves better results than its baseline DragDiffusion with a comparable inference time. Given the fact that DragLoRA has a more complicated pipeline and a larger number of parameters to optimize, I guess that the reason for DragLoRA to catch up DragDiffusion in inference time is ASS, which allows DragLoRA to skip several optimization steps. Therefore, it would be best to show how much time and how many optimization steps can be saved by adopting ASS.\n\n2. The curves that compare DragLoRA and DragDiffusion in Figure 1 and Figure 8 are good. It shows the variation of the quality of the handle point during the optimization process. However, the plot is misleading in terms of drag steps. The plot shows that the DragLoRA finishes editing with fewer drag steps than DragDiffusion, implying DragLoRA is faster and more efficient than DragDiffusion. This is not true according to the results in Table 1, which shows that DragLoRA and DragDiffusion have similar inference times. According to Algorithm 1, DragLoRA only increment drag step count $i$ when DOO is triggered. All steps in the while loop are ignored, which makes DragLoRA appear to be much more efficient. A more appropriate curve would be that the drag step still counts ILFA in the while loop but uses a different color for this section, indicating that these steps are faster than the optimization step. This can also reflect how many optimization steps are skipped.\n\n3. Figure 7 in the supplementary material plots MinD vs drag steps for all methods. It would be better if $d(h_{i}^{k}, t_{i})$ vs drag steps can also be plotted in a similar way as it will reflect how fast each method completes the dragging. \n\n4. Although mean distance is widely used to evaluate the dragging result, it cannot always reflect the true quality of the editing. A more appropriate metric is the human evaluation, which is widely adopted in text-to-image and text-to-video generation methods. However, it is inappropriate to ask for a human evaluation during the author-reviewer discussion period, and I simply want to share my opinion on it.", "relation_to_broader_scientific_literature": "The paper addresses the drag-based image editing problem using diffusion models. It is built on top of DragDiffusion, which is one of the pioneering works in this direction. The general pipeline of this work is similar to that of DragDiffusion, and it proposes its own innovation to improve the baseline.", "essential_references_not_discussed": "All relevant works have been included in the literature review.", "other_strengths_and_weaknesses": "All strengths and weaknesses are laid out in the previous sections.", "other_comments_or_suggestions": "There are several minor issues:\n\n1. In the right column of Ln200, \"we adopt the initla latent input $z_{35}^{0}$\" should be \"we adopt the initla latent input $z_{35}^{0} = z_{35}$\"\n\n2. In Section 4.1 Implementation Details, $\\lambda_{Mask}$ and $\\lambda_{DSS}$ are not defined in the previous section.", "questions_for_authors": "I would appreciate it if the authors could address my concerns regarding the Method and Experiment sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces an innovative framework for enhancing the efficiency and quality of drag-based image editing using diffusion models. The proposed DragLoRA framework integrates a dynamically LoRA adapter into the UNet architecture, which is designed to fine-tune the diffusion model for specific editing tasks. This adapter, combined with a DDS Loss that ensures semantic fidelity by comparing noise predictions, and an Adaptive Optimization strategy that dynamically adjusts between input updates and motion supervision, represents a significant advancement in the field.

The framework's strength lies in its enhancement of point tracking, which is vital for achieving high-quality and precise drag-editing results. The authors also provide a comprehensive analysis of various point tracking schemes and propose an advanced search strategy that surpasses traditional methods. The quantitative comparisons demonstrate that the proposed method achieves high image fidelity and time efficiency, making it competitive with other state-of-the-art approaches.

While the paper does have some limitations, such as the need for more details on the new dataset and further clarification on the m-MD metric, the overall contributions are substantial. The DragLoRA framework addresses a critical gap in the current methodology by offering a more efficient and precise approach to drag-based editing. The ablation studies and comparative analyses provided in the paper further support the effectiveness of the proposed method. Despite some concerns regarding the experimental setup and comparisons with other methods, the paper's innovative approach and promising results make it a valuable addition to the field of image editing.
Decision: Accept (poster)