FastDrag: Manipulate Anything in One Step
Authors: Xuanjia Zhao
Jian Guan
Congyi Fan
Dongli Xu
Youtian Lin
Haiwei Pan
Pengming Feng
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion Model;Drag-based Image Editing
Primary area: diffusion_based_models

Abstract: Drag-based image editing using generative models provides precise control over image contents, enabling users to manipulate anything in an image with a few clicks. However, prevailing methods typically adopt $n$-step iterations for latent semantic optimization to achieve drag-based image editing, which is time-consuming and limits practical applications. In this paper, we introduce a novel one-step drag-based image editing method, i.e., FastDrag, to accelerate the editing process. Central to our approach is a latent warpage function (LWF), which simulates the behavior of a stretched material to adjust the location of individual pixels within the latent space. This innovation achieves one-step latent semantic optimization and hence significantly promotes editing speeds. Meanwhile, null regions emerging after applying LWF are addressed by our proposed bilateral nearest neighbor interpolation (BNNI) strategy. This strategy interpolates these regions using similar features from neighboring areas, thus enhancing semantic integrity. Additionally, a consistency-preserving strategy is introduced to maintain the consistency between the edited and original images by adopting semantic information from the original image, saved as key and value pairs in self-attention module during diffusion inversion, to guide the diffusion sampling. Our FastDrag is validated on the DragBench dataset, demonstrating substantial improvements in processing time over existing methods, while achieving enhanced editing performance.

FastDrag: Manipulate Anything in One Step
Xuanjia Zhao', Jian Guan",, Congyi Fan', Dongli Xu', Youtian Lin?, Haiwei Pan', Pengming Feng"
College of Computer Science and Technology. Harbin Engineering University 2 School of Intelligence Science and Technology, Nanjing University Space-Ground
State Key Laboratory of Space- Ground Integrated Information Technology IIdependent Researcher
Abstract
Drag-based image editing using generative models provides precise control over image contents, enabling users to manipulate anything an image with few clicks. However, prevailing methods typically adopt n-step iterations for latent semantic optimization to achieve drag-based image editing, which is time-consuming and limits practical applications. In this paper, introduce novel one-step drag- based image editing method, i.e., FastDrag. to accelerate the editing process. Central to our approach is latent warpage function (LWF), which simulates the behavior of a stretched material to adjust the location individual pixels within the latent space. This innovation achieves one-step latent semantic optimization and hence significantly promotes editing speeds. Mcanwhile, null regions emerging after applying LWF addressed by our proposed bilateral nearest neighbor interpolation (BNNI) strategy. This strategy interpolates these regions using similar features from neighboring areas, thus enhancing semantic integrity. Additionally, consistency-preserving strategy is introduced to maintain the consistency between the edited and original images by adopting semantic information from the original image, saved as key and value pairs in self-attention module during diffusion inversion, guide the diffusion sampling. Our FastDrag is validated DragBench dataset, demonstrating substantial improvements in processing time over existing methods, while achieving enhanced editing performance. Project page: httts: site. wit..github....................t io/
the
Introduction
The drag editing paradigm 28
leverages the unique properties of generative models to implement point-interaction mode of image editing, referred to as drag-based image editing. Compared with text-based image editing methods [18 I0 30), drag-based editing enables more precise spatial control over specific regions of the image while maintaining semantic logic coherence, drawing considerable attention from researchers.
15 20
However, existing methods typically involve n-step iterative semantic optimization in latent space to obtain optimized latent with desired semantic based on the user-provided drag instructions. They focus on optimizing a small region of the image at each step, requiring n small-scale and short- distance adjustments achieve overall latent optimization, leading These optimization approaches can be categorized primarily into motion- based [3] 28 [5] [6] 32 4
significant amount of time.
and gradient- trasee [20] [9 n-step iterative optimizations, as shown in Fig. ((a) n-step iterations
motion- based methods are necessary to avoid abrupt changes in the latent space, preventing image
"Corresponding author: j-guan @ hrbeu.edu.cn
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Motion-
Gradient-
based
based
1, oblained by LWF
Original
ststep Iterative
Desired
Original
One-step Optimization
Desired
semantic
Optimization
semantic
semantic
semantic
(a) Existing methods with n-step iterative optimization.
(h) FastDrag with one-step warpage optimization.
Figure 1: (a) Existing methods usually require multiple iterations to transform an image from its original semantic to desired semantic; (b) Our method utilizes latent warpage function (LWF) calculate the warpage vectors (i... vy) to move each individual pixel on feature map and achieve semantic optimization in one step.
distortions and ensuring stable optimization process. This is exemplified in DragDiffusion [28 and GoodDrag (32). which require 70 to 80 iterations of point tracking and motion supervision for optimization. In addition, gradient-based methods align sampling results with the drag instructions through gradient guidance [6]. In this way, they also require multiple steps due to the optimizer [3
25 needing multiple iterations for non-convex optimization. For instance, DragonDiffusion (20
requires around 50 gradient steps to accomplish the latent optimization. Therefore, existing drag- based image editing methods often suffer from significant time consumption due n-step iterations required for latent semantic optimization, thus limiting the practical applications.
To this end, we present novel one-step drag-based image editing method based on diffusion, i.e. FastDrag, which significantly accelerates editing speeds while maintaining the quality and precision of drag operations. Specifically, novel one-step warpage optimization strategy is proposed to accelerate editing speeds, which can achieve the latent semantic optimization a single step with an elaborately designed latent warpage function (LWF), instead of using motion or gradient- based n-step optimizations, as illustrated in Fig. [(b) By simulating strain patterns in stretched materials, treat drag instructions on the noisy latent as external forces stretching material, and introduce stretch factor in LWF, which enables the LWF to generate warpage vectors to adjust the position of individual pixels on the noisy latent with simple latent relocation operation, thus achieving one-step optimization for drag-based editing. Meanwhile, a bilateral nearest neighbor interpolation (BNNT) strategy is proposed to enhance the semantic integrity of the edited content, by interpolating null values using similar features from their neighboring areas to address semantic losses caused by null regions emerging after latent relocation operation, thus enhancing the quality of the drag editing.
Additionally, consistency peeeeeevipeeeriiggg strategy is introduced to maintain the consistency of the edited image, which adopts the original image information saved in diffusion inversion (i.e., key and value pairs of self-attention in the U-Net structure of diffusion model) to guide the diffusion sampling for desired image reconstruction, thus achieving precise editing effect. To further reduce time consumption for inversion and sampling, the latent consistency model (LCM) (17) is employed in the U-Net architecture of our diffusion-based FastDrag. Therefore, our FastDrag significantly accelerate editing speeds while ensuring the quality of drag effects.
Experiments on DragBench demonstrate that the proposed FastDrag is the fastest drag- baaa- editing method, which is nearly 700% faster than the fastest existing method (i.e.. DiftEddiorr (19), and 2800% faster than the typical baseline method (i.e., DragDiffusion (28). with comparable editing performance. We also conduct rigorous ablation studies to validate the strategies used in FastDrag.
Contributions: 1) We propose novel drag-based image editing approach based on diffusion i.e., FastDrag, where LWF strategy is proposed to achieve one-step semantic optimization, tremendously enhancing the editing efficiency. 2) We propose a novel interpolation method (i.e., BNNI), which effectively addresses the issue of null regions, thereby enhancing the semantic integrity of the edited content. 3) We introduce consistency- preeeecvv--reeiingggg strategy to maintain the image consistency during editing process.
2 Related Work
2.1 Text-based Image Editing
Text-based image editing has seen significant advancements, allowing users to manipulate images through natural language instructions. DiffusionCLIIP [2] adopts contrastive language-image pre-

---

training (CLIP) [24] for diffusion process fine-tuning to enhance the diffusion model, enabling high-quality vero-shot image editing. The study in 7 manipulates the cross-attention maps within the diffusion process and achieves text-based image editing. Imagic [I]] further enhances these methods by optimizing text embeddings and using text-driven fine-tuning of the diffusion model, enabling complex semantic editing of images. InstructPix2Pix [I] leverages pre-trained large language model combined with a -ixt-to-image model to generate training data for a conditional diffusion model, allowing it to edit images directly based on textual instructions during forward propagation. Moreover, Null-text Inversion [8] enhances text-based image editing by optimizing the default null-text embeddings to achieve desired image editing. Although text-based image editing methods enable the manipulation of image content using natural language description, they often lack the precision and explicit control provided by drag- based image editing.
2.2 Drag-based Image Editing
Drag-based image editing achieves precise spatial control over specific regions of the image based on user-provided drag instructions. Existing drag- based image editing methods generally rely
n-step latent semantic optimization in latent space to achieve image editing. These methods fall
into two main categories: motion-based [23
28
32
4
16
9 and gradient-based (20 [9]
optimizations. For example, DragGAN 23 employs generative adversarial network (GAN) for
drag-based image editing with iterative point tracking and motion supervision steps. However, the image quality of the methods using GAN for image generation is worse than diffusion models [5]. Therefore, a series of diffusion-based methods have been proposed for drag-based image editing. For instance, DragDiffusion (28) employs iterative point tracking and motion supervision for latent semantic optimization achieve drag-based editing. Building on this foundation, GoodDrag [32, StableDrag [4) DragNoise [6]., and FreeDrag [15] have made significant improvements to the motion- based methods. Without coincidence, by utilizing feature correspondences, DragonDiffusion (00 and its improved version DiffEditor [9) formulate an energy function that conforms to the desired editing results, thereby transforming the image editing task into a gradient- based process that enables drag- based editing. However, these methods inherently require n-step iterations for latent optimization, which significantly increases the time consumption. Although SDEDrag 22 does not require n-step iterative optimization, it is still time-consuming due to the stochastic differential equation (SDE) process for diffusion. In addition, while EasyDrag [9] offers user- eser-friendship editing, its requirement for over 24GB of memory (i.e., a 3090 GPU) limits its broad applicability. To this end, based on latent diffusion model (LDM) (26), we propose a novel one-step optimization method that substantially accelerates the image editing speeds.
3 Proposed Method
FastDrag is based on LDM [26] to achieve drag-based image editing across four phases. The overall framework given in Fig. 2 and the detailed description of strategies in FastDrag are presented as follows: (1) Initially, FastDrag is based on traditional image editing framework including diffusion inversion and sampling processes, which will be elaborated in Sec. 3.1 (2) The core phase in Sec. 3.2. is one-step warpage optimization, employing LWF and latent relocation operation to simulate the behavior of stretched material, allowing for fast semantic optimization. (3) BNNI is then applied in Sec. 3.3 to enhance the semantic integrity of the edited content, by interpolating the null (4) The consstenccypressene introduced in Sec. 3.4 to maintain the desired image consistency with original image, by utilizing the key and value of self attf-attention in inversion to guide the sampling.
regions emerging after the one-step warpage optimization.
strategy is
3.1 Diffusion-based Image Editing
Similar to most existing drag editing methods 228 model (iee, LDM), including diffusion inversion and diffusion sampling.
32 20
FastDrag is also built upon diffusion
Diffusion Inversion 29 is about mapping given image to its corresponding noisy latent representa- tion in the model's latent space. We perform semantic optimization on the noisy latent 2, E Rexkxe due to it still captures the main semantic features of the image but is perturbed by noise, making it suitable as a starting point for controlled modifications and sampling 288. Here, w. h, e represent the width, height and channel of 2,, respectively. This process for latent variable at diffusion step can
3

---

laage mi
One-step Warpage Optimization
High
Diffusion Inversion
Consistency- Preserving
S (P,D.) Latentt Warpage Function
D-DEDE
V
20
Elog
Original Image
F Fiv V)
Warpuge
(K.V)
Latent Relocation
Vector
of drgg
Jon
Mie g(len
Ja korpage wese y
Erel Wis
(ookg
Opsiniied 5, with call parn
Optinnzod
Bilateral Nearest Neighhor
Desired Image
Diffusion Sampling
Interpolation (HNNI)
(K.P). Key and salee parsoo pwr.oo attertetioon
Figure 2: Overall framework FastDrag with four phases: diffusion inversion, diffusion sampling, one-step warpage optimization and BNNL. Diffusion inversion yields noisy latent 2, and diffusion sampling reconstructs the image from the optimized noisy latent z/. One-step warpage optimization is used for noisy latent optimization, where LWF proposed generate warpage vectors adjust the location of individual pixels the noisy latent with a simple latent relocation operation. BNNI used enhance the semantic integrity noisy latent. consiseency-preseriing strategy is introduced to maintain the consistency between original image and edited image.
be expressed as:
Vot
(2 V1 04-1 62) /1 E2:
where = (1o) denotes the initial latent of the original image Lo from the encoder [14] E(-). ar the noise variance diffusion step f, and e, is the noise predicted by U-Net. Subsequently, perform one-step warpage optimization on in Sec. 3.2
Diffusion Sampling reconstructs the image from the optimized noisy latent 2 by progressively denoising it to the desired latent z/. This sampling process can be formulated as:
V1 vl-vlc
a4 El
= = 01-1
V - at-1 o².
(2)
Voa
where is the Gaussian noise and ø denotes the noise level. By iterating the process from to 1, # is reconstructed, and the desired image can be obtained by I = D(2(), with D(-) being the
decoder [14..
3.2 One-step Warpage Optimization
Building upon the phases Sec3.1 we propose one-step warpage optimization for fast drag-based image editing. The core idea involves simulating strain patterns in stretched materials, where drag instructions on the noisy latent are interpreted as external forces stretching the material. This enables us to adjust the position of individual pixels on the noisy latent, optimizing the semantic of noisy latent one step, thus achieving extremely fast drag-based editing speeds. To this end, we design LWF in Sec. 3.2. to obtain warpage vector, which is utilized by straightforward latent relocation operation Sec. 3.2.2 adjust the position individual pixels on the noisy latent.
3.2.1 Warpage Vector Calculation using LWF
In drag- image editing, each drag instruction d; in set of & drag instructions D (d; 1. 1..... k Z} can simultaneously influence feature point p, on the mask region p {p) = 1 1...... m; E Z} provided by the user. As shown in Fig. the mask region is represented by the brighter areas the image, indicating the specific image area to be edited. To get uniquely determined vector, i.e., warpage vector vj to adjust the position of feature point P5 (will be discussed in 3.2.2,,, we propose a latent warpage function fLWF to aggregate multiple component warpage vectors caused by different drag instructions, i.e., vy with balanced weights to avoid

---

deviating from the desired drag effect. The function is given as follows:
k f = (twr((P,D.)) = Σ Dj
(3)
where w is the normalization weight for component warpage vector of Here, drag instruction d; considered as a vector form handle point 6; to target point e,. During dragging, we aim for the semantic changes around the handle point s; be determined by the corresponding drag instruction d, rather than other drag instructions far from the S,. Therefore, wj is calculated as follows:
1/1p5s:
3 Ex (1/1D588))
(4)
where &; is considered as the "point of force" of d,, and the weight w/ is inversely proportional to the Euclidean distance from s; to Pj.
worth noting that under an external force, the magnitude of component forces at each position within the material is inversely proportional to the distance from the force point, while the movement direction at each position typically aligns with the direction of the applied force [21]. Similarly, the component warpage vector oj each Pj aligns with the direction of drag instruction d, and magnitudes of o5* are inversely proportional to the distance from 8,. Hence, vy" can be simplified as:
vy X3 di
(5)
where A is the stretch factor that denotes the proportion between v/ and d,.
To appropriately obtain the stretch factor X and fa- cilitate the calculation, we delve into the geometric representation of the component warpage vector v) As shown in Fig. of* can be depicted as the guid- ance vector from point to point p5", where p** the expected new position of Pj under the drag effect of d, Recognizing that the content near to mask edge should remain unaltered, O where every 2i* will gradually reduce to 0 as Pj approaches the circle. Consequently, since v and d, are parallel, magnitudes of oj* are inversely pro- portional to the distance from 8; and of* is reduced to 0 circle O, the extended lines from s:p and exp)" will intersect at df circle O. Hence, based on the Eq. 5 and the geometric principle in Fig.
q
P
P
e
d
Si
define reference circle
Figure 3: Geometric representation of o Circle O is the circumseribed circle of the circumscribed rectangle enclosing the mask's shape. Py is the feature point requiring relocation, and pj its new position following the drag instruction d;
3
we calculate X3 follows:
p,pp1 s,ee
|
A3
(6)
d.
Finally, we obtain the warpage vector v, using only d, and two factors as follows:
f.wF(P.D.)) Ewj
w X3. X3-dd
Note that, for the special application of dtra-based editing, such as object moving as shown Fig. g drag editing is degenerated to mask region shifting operation, requiring the spatial semantics of the mask region to remain unchanged. that case, we only process a single drag instruction, and all component drag effects will be set equal to the warpage vector, i..., Vj dj and D {d1).
3.2.2 Latent Relocation with Warpage Vector
Consequently, we utilize the warpage vector Uy to adjust the position of feature point p3 via latent relocation operation Fw g. achieving the semantic optimization of noisy latent for drag-based editing.
5

---

F oR
BNNI
KV
KV
KV
Q
KV
refu
He
=2
len,-1
W
Lilen
N
N
lecurrdd
U-Net in inversion process
x
N
N
NV
Σ Eref he Whe
U-Net in sampling process
leng=2
len
(null point)
BNNI
KY
Self-Attention module
Figure 4: Illustration of bilateral nearest neighbor interpolation.
Figure 5: preserving strategy.
Illustration consistency-
Establishing a Cartesian coordinate system the latent space, let (Kpy: Mp, denote the position of point PA P within this coordinate system. The new location of the point pz after applying the vector by = (uf, uy)) can be written as:
(e, (xp vpy) + (uf, wy)
(8)
Then the new coordinates set C' of all feature points in P can be written as:
C Fw P, V) (157)) Up. (5 }iy)) (Tpy Bb,) (vf, o5);; 1, m),
where V {uyl 1,... m, m E Z). If (Ipj Upy) has already been new position for feature point, it no longer serves as a new position for any other points. Consequently, by assigning corresponding values these new positions, the optimized noisy latent 2 can be obtained as shown in the following equation:
z (Epg May tog Z1 (zpp
(10)
In essence, the latent relocation operation optimizes semantics efficiently by utilizing the LWF- generated warpage vector, eliminating the need for iterative optimization.
However, as certain positions the noisy latent may not be occupied by other feature points, z obtained from one-step warpage optimization may contain regions with null values shown in Fig. 4
leading to semantic losses that can adversely impact the drag result. We address this issue in Sec. 3.3
3.3 Bilateral Nearest Neighbor Interpolation
To enhance the semantic integrity. BNNI interpolates points in null region using similar features from their neighboring areas in horizontal and vertical directions, thus ensuring the semantic integrity and enhancing the quality of drag editing. Let N be a point with coordinate (#N,, yNn) in null regions, identify the nearest points of N containing value in four directions: up, right, down, and left, as illustrated in Fig. 4 which are used as reference points for interpolation. Then, the interpolated value for null point N can be calculated as:
Σ Whoe rrefta
(1I)
loc=u,r,d,1
where ref loc denotes the value of reference point, and loc indicates the direction, with u, d and ( representing up, right, down and left, respectively. Whoe the interpolation weight for each reference point, which is calculated based on its distance to N,, as follows:
1/lentoc
Wioe Loc=uurd 1/lentoc
(12)
where Lemvoe represents the distance between the reference point and N. Such that we can obtain the optimized noisy latent z with complete semantic information by using BNNI to exploit similar semantic information from surrounding areas, further enhancing the quality of the drag editing.

---

User Edit
DragDifTusion
FreeDrag
DragNoise
FastDrag (Ours)
Figure 6: Illustration of qualitative comparison with the state-of-the-ard methods.
3.4
Strategy
Following 20 2 28
we introduce consistency-preserviin strategy maintain the consistency between the edited image and the original image by adopting the semantic information of the original image (i.e,, key and value pairs) saved in self-attention module during diffusion inversion guide the diffusion sampling, as illustrated in Fig. [5 Specifically, during the diffusion sampling, the calculation of self-attention Attentions, within the upsampling process of the U-Net is follows:
Attentions, QSa. Kin, Vin) softmax Qsa Kin
Vi
(13)
Vd
where query Qsa is still used from diffusion sampling but key Kin and value Vi are correspondingly from diffusion inversion. Thus, the consistency between the desired image and original image, ensuring the effect of drag-based editing.
strategy maintains the overall content
4
Experiments
4.1
Qualitative Evaluation
We conduct experiments to demonstrate the drag effects of our FastDrag method, comparing it against state-of-the-art techniques such as DragDiffusion (28), FreeDrag [15], and DragNoise [16). The qualitative comparison results presented in Fig. 6 Notably, FastDrag maintains effective drag performance and high image quality even in images with complex textures, where n-step iterative methods typically falter. For instance, shown in the first row of Fig, 6 FastDrag successfully rotates the face of an animal while preserving intricate fur textures and ensuring strong structural integrity. In contrast, methods like DragDiffusion and DragNoise fail rotate the animal's '' face, and Free disrupts the facial structure.
In the stretching task, FastDrag outperforms all other methods, as shown in the second row of Fig. 6 where the goal to move sleeve to a higher position. The results show that other methods lack robustness slight deviations in user dragging, where the drag point is slightly off the sleeve.
7

---

Table I: Quantitative comparison with state-of-art methods DragBench. Here, lower MD indicates more precise drag sults, while higher 1-LPIPS reflects greater similarity between the generated and original images. The time metric represents the average time required per point based on RTX 3090. Prepa- ration denotes LoRA training. means FastDrag without LCM- cquipped U-Net.
38
-0.87
Image Fideiity (1-Pf
36-
-0.00
Mcan Distance
34-
10.85
0.84
0.88
30
0.02
28
Mean Dettnce
magge Rirtly c1 (1-PPS)
0.80
Approach
Veuse
MD
LPIPS
Time
26
6 10 14
Number of Inversion Steps c
20
30
-0.90
Brighillaion
DragNoise
[0
X8
CVPR2002
CVFRR2IS
33.41
35.70
0.89
0.63
mim (LoRA)
rim (LOEA)
20.41
21.34
FreeDrag [
CVPR2024
35.00
0.70
(LoRA)
52.63
Goodlang 52
arXiv2124
22.96
0.86
(LoKA)
45.83
Figure 7: Ablation study on number of inver- sion steps in terms of quantitative metrics.
DiffEditr 1
CVPR2024
28.46
0.89
21.68
DastDrg (Ours)
33.02
FasiDrag (Ouns
32.23
0.86
3,12
Despite this, FastDrag accurately moves the sleeve to the desired height, understanding the underlying semantic intent of dragging the sleeve.
Additionally, we perform multi-point dragging experiments, illustrated in the fourth row of Fig. 6 Both DragDiffusion and DragNoise fail stretch the back of the sofa, while FreeDrag incorrectly stretches unintended parts of the sofa. Through the LWF introduced in Sec. [32.1] FastDrag can manipulate all dragged points to their target locations while preserving the content in unmasked regions. More results of FastDrag are illustrated in supplementary Sec.
4.2 Quantitative Comparison
To better demonstrate the superiority of FastDrag, we conduct quantitative comparison using Drag- Bench dataset (28). which consists of 205 different types of images with 349 pairs of handle and target points. Here, mean distance (MD) 23 and image fidelity (IF) [1] are employed perfor- mance metrics, where MD evaluates the precision of drag editing, and IF measures the consistency between the generated and original images by averaging the learned perceptual image patch similarity (LPIPS) [31]. Specifically, I-LPIPS is employed as the IF metric in our experiment to facilitate comparison. In addition, we compare the average time required per point to demonstrate the time efficiency of our proposed FastDrag. The results are given in Table [
Apart from 28) (66) (15), two other state-of offtth-art methods, i.e., GoodDrag 32 and DiffEddtor [19], are also adopted for comparison, with DiffEditor being the current fastest drag- based editing method. Due to well-designed one-step warpage optimization and Drag does not require LoRA training preparation, resulting in significantly reduced time consumption (i.e., 3.12 seconds), which is nearly 700% faster than DiffEditor (i.e., 21.68 seconds), and 2800% faster than the typical baseline DragDiffusion (i.e., I min and 21.54 seconds). Moreover, even using standard U-Net without LCM, our method is still much faster than DiftEdditor and far outperforms all other state-of-the-art methods. It is particularly noteworthy that, even with an A GPU, DifiEditor still requires 13.88 seconds according to [19 whereas FastDrag only requires 3. seconds on an RTX 3090.
strategy, Fast-
In addition, our FastDrag also achieves competitive quantitative evaluation metries (i.e., IF and MD) comparable to the state-of-the-ard methods, and even better drag editing quality, as illustrated in Fig. 6 These results demonstrate the effectiveness and superiority of method.
4.3 Ablation Study
Inversion Step: To determine the number of inversion steps in diffusion inversion with LCM-
equipped U-Net, we conduct an ablation experiment with number of inversion steps set as t = 6, 8, 10, 12, 14, 20, and 30, where IF and MD are used to evaluate the balance between the consistency with original image and the desired drag effects. The results are given in Fig. 3 and Fig. 7 where we can see that when / < 6, the generated images lack sufficient detail to accurately reconstruct the original images. Conversely, when L > 6, it can successfully recover complex details such as intricate fur textures and dense stone while maintaining high image quality. However, when > 14, some image details lost, which negatively impacts the effectiveness of the drag effect. By comprehensive

---

Original image
Editing
4 steps
steps
8 steps
steps
12 steps
14 steps
20 steps
steps
Figure 8: Ablation study number of inversion steps in terms of drag effect.
User edit
Original value
interpolation
Random interpolation
BNNT
Figure Ablation study on bilateral nearest neighbor interpolation.
evaluation of both the drag effect and the similarity to the original images, we select 10 as the number of inversion steps for our method with I.CM to balance the drag effect.
To demonstrate the effectiveness of BNNI, compare it with several interpolation methods on null point N,, including maintaining the original value of this position, interpolation by zero-value, and interpolation by random noise, denoted as "original value", "O interpolation", and "random interpolation". respectively. The results are given in Fig. 9 where we can see that, by effectively utilizing surrounding feature values interpolate null points, BNNI address semantic losses, and enhance the quality of the drag editing.
BNNI:
Consistency- Preseeeeii--rrr-ngngn We also conduct experiment to validate the effectiveness of the strategy in maintaining image consistency. The results are illustrated in Fig. 10 where "w/ CP" and "w/o CP" denote FastDragg with and without using consistency- preserving strategy, respectively. obviously that our method with consistency-preservvii strategy effectively preserve image consistency, resulting in better drag editing effect.
User odit
wib CP
CP
edit
wif CP
CP
Figure 10: Ablation study ponssstn strategy.

---

Original Image
User edit
DreeDrag
DragDiffusion
DragNoise
Dag (Ours)
Figure 11: Illustration of failure cases for limitation analysis under extremely long-distance drag editing. Our FastDrag method may lose some detailed information these cases still achieves better editing performance compared to state-of-the-art (SOTA) methods.
Original Image
User edit
FastDrag
Original Image
User edit
Fast
Failed
Sucoed
(c) Desired editing effed: Thin the hair while keeping the face size
(b) Destred editing effect Lengthen the beak
Illustration of the limitation analysis with failed and successful drag editing for highly relying on precise drag instruction. (a) It is best to exclude the face from the mask region. (b) The handle point should ideally placed where the "beak" feature more prominent.
Figure 12:
Limitations
Despite FastDrag's impressive editing speed compared to SOTA methods, it shares some common limitations. 1) Overly Smooth and Finer Details Loss: Similar to other diffusion-basee methods [3]
28.. FastDrag occasionally loses fine textures from the original images, as shown in Fig. 6 row 4.
Despite this, FastDrag outperforms other methods in speed and overall performance. 2) Extremely Long-distance Drag Editing: In such case, object details may be lost due to the lower- lower-dimensional latent space, in which significant changes in detail (i.e., long-drag editing) can disrupt the semantics, making it harder to preserve all details. Nevertheless, FastDrag handles long-distance editing better than other SOTA methods, as illustrated in Fig. [T] where our method successfully achieves long- distance drag editing that others fail to achieve. 3) Highly Relying on Precise Drag Instruction: Achieving optimal results depends heavily on clear drag instructions. As with other SOTA methods, precise input, such as excluding irrelevant areas from the mask (e.g., the face in Fig, [2] row 2) or correctly placing the handle point (e.g., beak in Fig. 12 is essential for better performance.
Conclusion
This paper has presented novel drag- baaed image editing method, i.e., FastDrag, which achieved faster image editing speeds than other existing methods. By proposing one-step warpage optimization and BNNI strategy, our approach achieves high-quality image editing according to the drag instruc- tions in a very short period of time. Additionally, through the ensures the consistency of the generated image with the original image. Moving forward, we plan to continue refining and expanding our approach to further enhance its capabilities and applications.
prreeeeccyireeivingg strategy, it
10

---

7 Acknowledgments
This work was partly supported by Beijing Nova Program (2023048426)).
References
[1] Tim Brooks, Aleksander Holynski, and Alexei A. Efros. InstructPix2Pix: Learning to follow image editing instructions. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023.
[2] Ming Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qic, and Yinqiang Zheng. MasaCtrl: Tuning- free mutual self-attention control for consistent image synthesis and editing. Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2023.
[3]
Hansam Cho. Jonghyun Lee, Seoung Bum Kim. Tae-Hyun Oh, and Yonghyun Jeong. Noise map guidance: Inversion with spatial context for real image editing. In Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[4] Yutao Cui, Xiaotong Zhao, Guozhen Zhang, Shengming Cao, Kai Ma, and Limin Wang. SrableDrag: Stable dragging for point- bast- image editing. arXiv preprint 1rxxiv2403.04433 2024.
[5] Prafulla Dhariwal and Alex Nichol. Diffusion models beat GANs on image synthesis. In Proc. Adv. Neural Inf. Prucess. Syxt. (NeurIPS), 2021.
[6] Dave Epstein, Allan Jabri, Ben Ponle, Alexei A Efros, and Aleksander Holynski. Diffusion self-guidance for controllable image generation. In Proc. Adv. Neural Inf. Process. Syst. (NeurlPS), 2023.
[7] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman. Yacl Pritch, and Daniel Cohen-Or. Prompt- to-prompt image editing with cross attention control. In Proc. Int. Conf. Learn. Represent. (ICLR). 2023.
[8] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2021.
[9] Xingzhong Hou, Boxiao Liu, Yi Zhang, Jihao Liu, Yu Liu, and Haihang You. Easydrag: Efficient point- based manipulation on diffusion models. In Proc. IEEE/CVF Conf. Comput. Vis. Puttern Recognit. (CVPR). pages 1404-8413, 2024.
[10] Xuan Ju, Ailing Zeng, Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Direct inversion: Boosting diffusion- based editing with lines code. In Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[11] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text- based real image editing with diffusion models. In Proc. IEEE/CVF Conf. Compur. Vis. Pattern Recognit. (CVPR), 2023.
[12] Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. DiffusionCLII:: Text-guided diffusion models for robust image manipulation. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognis. (CVPR), 2022.
[13] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412,6980, 2014.
[14] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114 14, 2013,
[15] Pengyang Ling, Lin Chen, Pan Zhang, Huaian Chen, Yi Jin, and Jinjin Zheng. FreeDrag: Feature dragging for reliable point-based image editing. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024.
[16] Haofeng Liu, Chenshu Xu, Yifei Yang, Lihua Zeng. and Shengfeng He. Drag your noise: Interactive point-based editing via diffusion semantic propagation. In Proc. IEEE/CVF Conf. Comput. Vis. Puttern Recognit. (CVPR), 2024.
(17] Simian Luo, Yigin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arX1v:2310.04378, 2023.
[18] Ron Mokady, Amir Hertz, Kfrr Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proc. JEEF/CVF Conf. Comput. Vis. Pattern Recognit, (CVPR), 2023.
[19] Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. DiffEditor: Boosting accuracy and flexibility on diffusion- image editing. In Proc. IEEE/CVF Conf. Comput. Vis. Puttern Recognit. (CVPR). 2024.
[20] Chong Mou, Xintao Wang, Jiechong Song, Ying Shan, and Jian Zhang. DragonDiffusiin: Fiaaliig drag-style manipulation diffusion models. In Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[21] D. Naylor. Theoretical elasticity, by A. E. Green and W. Zerna (Second edition). Clarendon Press, Oxford, 1968. xv + 457 pages. Can. Math. Bull, 12:537-538, 1969.
[22] Shen Nie, Hanzhong Allan Guo, Cheng Lu, Yuhao Zhou, Chenyu Zheng, and Chongxuan Li. The blessing of randommess: SDE beats ODE in general diffusion-based image editing. In Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
[23] Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, and Christian Theobalt. Drag your GAN: Interactive bast- manipulation on the generative image manifold. In Proc. ACM Conf. SIGGRAPH, 2023.
11

---

[24] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. Proc. Int. Conf. Machine Learning (ICML), 2021.
[25]
Herbert E. Robbins. A stochastic approximation method. Ann. Math. Stat., 22:400-2400 1951.
[26]
Robin Rombach, A. Blattmann. Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffuusin models. In Proc. IEEE/CVF Conf. Comput. Vis. Puttern Recognit. (CVPR), 2021.
[27]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022.
[28] Yujun Shi, Chuhui Xue, Jiachun Pan, Wenqing Zhang, Vincent YF Tan, and Song Bai, DragDiffusion: Harnessing diffusion models for interactive point-based image editing. In Proc. IEEE/CVF Conf. Comput.
Vis. Pattern Recognit. (CVPR), 2024.
[29] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. Proc. Int. Conf.
Learn. Represent. (ICLR), 2021.
[30] Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, Joyce Chai. Inversion-free image editing with natural language. In Proc. IEEF/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024.
[31] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features a perceptual metric. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018.
[32] Zewei Zhang, Huan Liu, Jun Chen, and Xiangyu Xu. GoodDrag: Towards good practices for drag editing with diffusion models. arXiv preprint arXiv:2404.077 07206, 2024.
12

---

A
Supplementary Experiments
We conduct supplementary quantitative experiments BNNI and consistency-preserving strategies to further validate their effectiveness. The quantitative metrics used are consistent with those described in Sec.
we compare with several interpolation methods on null point N, including maintaining the original value of this position, interpolation by vero-value, and interpolation by random noise, denoted as "origin". "O-inter", and gundom-inters respectively. As illustrated in Fig. [3 FastDrag with BNNI achieves the best MD levels compared to other interpolation methods, while its TF second only to "origin". However, "origin" can lead to negative drag effects, as shown in Fig. Therefore, by effectively utilizing surrounding feature values to interpolate null points, BNNI can address semantic losses and enhance the quality of drag editing.
BNNI:
Following the setup in Sec 4.3
5.3
Consistency- Preservingg We also conduct experiments to assess the impact of initiating the prreseevnyyy strategy at different sampling steps. The results, as shown Fig. 14 indicate that as the starting step increases ( i.e., the frequency of key and value replacements decreases), the IF decreases, leading to poorer image consistency. Meanwhile, the MD initially decreases and then increases as the starting step increases. It is evident that consistency-pressee preserving strategy can effectively maintain the consistency between the generated images and the original images.
35.5
Rideity (1-1PPP)
0.8700
(mag)) p Fidlitty (1- (1-LPIPS)
Image Fidellyy (1-LPIPS)
35.0
0.8675
35
34.5
34.0
34
0.83
Image Roelty
0,8600
Mcon
33-
Nean Disiance
0.8575
33.0
0.8550
0.81
32.5
0.8525
0.80
origin
D-ittrr
random-inter
BNNI
3
Interpolation Strategy
Start step of Consistency-Preserving
Figure 13: Ablation study on BNNI in terms of quan- titative metrics.
Figure 14: Ablation study on consistency-
preserving in terms of quantitative metries.
B Single-step Diffusion
When integrating DragDiffusion with single- step diffusion model, the editing time is still much longer than that of FastDrag. For DragDif- fusion and FastDrag under diffusion steps of 20, and 50, we calculate the time required for inversion, sampling, and latent optimization respectively. The results provided in Fig. [5 show that even with single diffusion step (i.e., diffusion step set as 1), DragDiffusion still re- quires significantly more time (20.7 seconds) compared to FastDrag (2.88 seconds).
Timets)
1
10
FastDrop()
2astDrag(20)
FasDrag(50)
Drag DragDiffiniont20) DrieDiitsion))
DragDiffissont50)
Methods with different diffusion steps
In addition, as observed in Fig. 15 DragDiffu- sion spends significantly more time on latent op- timization compared to diffusion inversion and time spent
inversion
sumpling
optimization
Figure 15: Overall editing time comparison with differ- ent diffusion steps hetween FastDrag and DragDiffusion. All experiments are conducted on 3090 with diffu- sion step set as 1, 20, and 50 respectively. Optimization means latent optimization.
sampling. Therefore, reducing on latent optimization is crucial for minimiz- ing overall editing time, which is precisely what FastDrag accomplishes.
13

---

C
Statistical Rigor
To further validate the superiority of our work and achieve statistical rigor, we conducted an additional experiment by repeating our experiment 10 times under the same experimental settings. We observed that the variances of the performance metries obtained from 10 realizations of our FastDrag are MD (0.000404), 1-LPIPS (9.44E-11) and Time (0.018), all of which fall within reasonable range. These statistical results further demonstrate effectiveness and stability of our method for drag editing.
D Implementation Details
We utilize a pretrained LDM (i.e., Stable Diffusion 1.5 [27]) as our diffusion model, where the U-Net structure is adapted with LCM-distilled weights from Stable Diffusion 1.5. It is worth emphasizing that the U-Net structure used our model is widely used in image generation methods [28 15
16 32
Unless otherwise specified, the default setting for inversion and sampling step is 10. Following DragDiffusion [28), classifier-free guidance (CFG) [8] is not applied in diffusion model, and optimize the diffusion latent at the 7th step. All other configurations follow that used in DragDiffiusin. Our experiments are conducted on an RTX 3090 GPU with 24G memory.
For the special application of drag-based editing, i.e., object moving, as shown in Fig. 8 significant null region may be left at the original position of the object due to long- distance relocation, posing challenges for BNNI. To ensure semantic integrity in the image and facilitate user interaction, adopted two straightforward strategies. For first strategy. specifically, we introduce a parameter r. 2, centered the unique target point e1, defining rectangular then extract the noised-latent representation within this rectangular area and fill it into the mask, effectively restoring the semantics the original position of the object. For second strategy, we just maintain the original semantic of original position to avoid null region. Under second strategy, object moving will produce the effect of object replication, shown in the third row of Fig. 16
with dimensions 2r. We
E
More Results
We apply FastDrag to ttag-based image editing in various scenarios, including face rotation, object movement, object stretching, object shrinking and so on. The experimental results, as shown in Fig. [6 demonstrate that FastDrag achieves excellent drag-based effects across multiple scenarios.
F Societal Impacts
FastDrag has the potential to bring about several positive societal impacts. Firstly, they offer intuitive and efficient image editing tools, catering to artists, designers, and creators, thereby fostering creativity and innovation. Secondly, their user user-frient friendly nature simplifies the image editing process, increasing accessibility and participation among wider audience. In addition, our FastDrag improves efficiency and saves the user's time and effort, thus increasing productivity. Moreover, the innovative and flexible nature of these methods opens up possibilities for various applications, spanning creation, design, education, and training.
However, along with these positive aspects, FastDrag can also have certain negative societal implica- tions. They may be exploited by unethical individuals or organizations to propagate misinformation and fake imagery, potentially contributing to the spread of false news and undermining societal trust. Furthermore, widespread use of image editing tools may encroach upon individual privacy rights, particularly when unauthorized information imagery is manipulated. Moreover, inappropriate or irresponsible image editing practices could lead to social injustices and imbalances, such as distorting facts misleading the public, thereby influencing public opinion and policies negatively.
14

---

Original image
User odit
Result
Original image
User odit
Result
Figure 16: More visualized results of FastDrag.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly outline the contributions of our approach in both the abstract and the introduction See of our paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the our method in Section 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
Our paper does not involve theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our experimental results are reproducible. We provide detailed description of our method's workflow in See 3 and implementation details in Sec D
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification: We include the implementation code for our method in the supplementary materials submitted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https Cot//ooodeeesiseeeeeeeeee hwtpbbbmsssiooooooooiooo for more details.
/mips.ccc/pulic//uuui
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The main experimental details of our method are discussed in Appendix D
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Our experiments, conducted on the DragBench dataset, yield average metries that are statistically significant and provide a reliable basis for comparing our method with others. And we discuss experimental statistical significance in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
18

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We have provided detailed information about the computer resources used for all our experiments in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have thoroughly reviewed the NeurlPS Code of Ethics and can confirm that our research conducted the paper adheres the NeurIPS Code of Ethics
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws or regulations their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The potential positive socictal impacts and negative socictal impacts the our work is discussed in Appendix
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not introduce any new datasets pretrained models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The models and baselines utilized in our method are detailed in Appendix D of the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/dataseta
latasets.papersstt
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Question:
Answer: [NA]
Justification: Our paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: During our research, we do not employ crowdsourcing or conduct studies involving human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: In our research process, human subjects were not required to assist with the study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "xQOFmXUZUo", "reviewer": "Reviewer_YBFQ", "summary": "This paper presents a drag-based image editing method that uses the Latent Warpage Function to optimize pixel adjustments in a single step, which is an improvement over previous iterative methods. This approach simulates a stretched material in the latent space to allow for fast and accurate pixel adjustments. It also combines bilateral nearest-neighbor interpolation to handle null regions and a consistency-preserving strategy, involving key and value pairs in self-attention during the inversion process, to maintain semantic consistency and coherence.", "questions": "1) Does the reported computational time include the inversion process?\n2) What happens if you integrate DragDiffusion with a single-step diffusion model? How does it compare with Fastdrag?\n3) Why does iterative drag diffusion suffer compared to the one-step approach?\n4) How does FastDrag affect other parts of the image during edits?\n5) Can you perform multi-point drag editing with FastDrag?", "limitations": "Yes in the appendix", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed method reduces computational time by performing one-step optimization, which makes the process faster and more efficient. Additionally, it includes clever tricks using bi-linear nearest neighbor interpolation to fix empty areas. Preserving keys and values during inversion also appears to help maintain semantic coherence.", "weaknesses": "The paper needs to include more comprehensive evaluation metrics beyond visual inspection. The chosen metrics are unreliable, and the evaluation seems to be on a very small scale. Moreover, there are missing standard errors, and it is unclear if the results are cherry-picked or randomly chosen as the numbers on the proposed metrics look very much alike. To improve the overall evaluation of editing accuracy and quality, using interest points and key points like SIFT, SuperPoint, or DUST3R to verify if features at desired locations match the correct points in the edited image would be beneficial. Furthermore, there is a need to explore the effect of the edits on \"other\" parts of the image to understand potential unintended alterations due to the stochasticity in diffusion models. The edited images appear overly smooth and lose finer details, which may impact the overall realism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for the positive feedback and helpful comments. This rebuttal addresses your comments and suggestions for conciseness. Following are our responses to each individual comment (which are highlighted in italics).\n\n# Response for Weaknesses (RfW)\n>*W1: ... needs to include more comprehensive evaluation metrics ...*\n\n**RfW1:** Regarding evaluation metrics, following the SOTA methods [14, 15, 27], we employ ''MD'' (Mean Distance [22]), ''IF'' (i.e.,''1-LPIPS'' [10]) as evaluation metrics for a fair comparison. The metrics suggested by Reviewer, such as SIFT, are primarily used for feature point detection and matching. The suggested metrics may not be appropriate for drag-based editing due to semantic changes within the editing region. It is because that drag editing requires measuring the similarity between the edited result and the desired semantics, rather than comparing the ground truth and the edited result.\n\nDue to the space limitation, please refer to ''Evaluation Metrics'' in the general response to reviewers, where we conduct more detailed analysis and explanation.  \n\n\n>*W2: Miss standard error and it is unclear if the result are ...*\n\n**RfW2:** Our method is evaluated on the widely used DragBench dataset [27] for drag-based editing, following the state-of-the-art methods [4, 14, 15, 27] for a fair comparison. The results presented are the averages across DragBench that includes diverse image categories and drag tasks, which are not cherry-picked or randomly chosen. This has been explained in the General Response to Reviewers, please refer to the Evaluation Metrics in general response to reviewers. \n\nRegarding standard errors, to address your concern, we conducted an additional experiment by repeating our experiment 10 times under the same experimental settings. We observed that the variances of the performance metrics obtained from 10 realizations of our FastDrag are MD (0.000404), 1-LPIPS (9.44E-11), and Time (0.018), all of which fall within a reasonable range. \n\n\n>*W3: ... effect of the edits on \"other\" parts of the image ...*\n\n**RfW3:** There are almost no changes outside the masked editing region using our method.\nIt is because our approach optimizes the latent strictly within the masked region, as described by Equations (8)-(9) in our paper. We also employ a consistency-preserving strategy to maintain the consistency of image content outside the mask region. Note that, this is almost the same for other drag editing methods, as they also conduct latent optimization within the masked region, and adopt different strategy to maintain consistency such as LoRA [14, 15, 27, 31].\n\n>*W4: ... smooth and lose finer details ...*\n\n**RfW4:** The limitation of smooth and losing finer details is common across diffusion-based drag methods, inherent to the diffusion models employed, such as LDM and LCM. This issue arises due to model approximations, randomness, and potential computational errors, which can result in imperfect symmetry between the inverse and sampling processes, leading to a situation where generated image may not be exactly the same as the original [28].\n\nFor instance, DragNoise [15] and DragDiffusion [27] introduce textures not present in the original images, affecting fidelity, as illustrated in Fig. 6 of paper. However, although our method may result in finer details loss, it outperforms others in overall task execution, especially in editing speed. We will discuss the limitation in the camera-ready version. Future research will be conducted to mitigate such effect and enhance the editing performance.\n\n# Response for Questions\n>*Q1: Does the reported computational time include the inversion process?*\n\nYes, the reported computational times include the inversion process, sampling process, and the time for latent optimization.\n\n>*Q2: What happens if you integrate DragDiffusion with a single-step diffusion model? How does it compare with Fastdrag?*\n\nWhen integrating DragDiffusion with a single-step diffusion model, the editing time is still much longer than using FastDrag. For DragDiffusion and FastDrag under diffusion steps of 1, 20, and 50, we calculate the time required for inverse, sampling, and latent optimization respectively. The results provided in Fig. 3 of the attached PDF show that even with a single diffusion step (i.e., diffusion step set as 1), DragDiffusion still requires significantly more time (20.7 seconds) compared to FastDrag (2.88 seconds).\n\n>*Q3: Why does iterative drag diffusion suffer compared to the one-step approach?*\n\nThese methods require $n$-step iterations to achieve semantic optimization, with each step to optimize semantics within small editing area of the image, thus they need $n$ small-scale, short-distance optimizations to achieve overall latent optimization, and require large amount of time to perform $n$ iterations of the optimization. Whereas our method only requires a single short-time computation on the latent to achieve the semantic optimization, thereby significantly reducing editing time. \n\n>*Q4: How does FastDrag affect other parts of the image during edits?*\n\nDuring the editing process, other parts of the image outside the mask remain almost unchanged, as explained in  RfW3 to the reviewer.\n\n>*Q5: Can you perform multi-point drag editing with FastDrag?*\n\nYes, our method can perform multi-point drag editing, which has been demonstrated in Fig. 6 (row 4) of our paper and Fig. 13 (row 6) in Appendix C. \n\nWe will provide further clarifications and explanations regarding RfW2 and Q3 in the camera-ready version of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ubRcVtVO0A", "reviewer": "Reviewer_eL9M", "summary": "This paper introduces a new one-step drag-based image editing method that significantly accelerates the editing process using a LWF function. It also employs a BNNI strategy to handle null regions and a consistency-preserving strategy to maintain the integrity of the edited image. Experimental results demonstrate FastDrag’s fast speed and performance compared to existing methods.", "questions": "See the strengths and weaknesses", "limitations": "See the strengths and weaknesses", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. FastDrag is easy for editing and has a fast editing speed.\n2. BNNI strategy addresses the issue of null regions, maintaining semantic integrity and quality.\n3. It also provides spatial control over specific regions of the image, enabling detailed drag editing.", "weaknesses": "1. If the drag distance is long, will the BNNI still success to maintain high semantic quality, how about the editing speed and complexity for long-distance dragging and latent relocation?\n2. Threre should also be some failed examples to better illustrate the proposed method.\n3. I am curious whether using a better base model can achieve better editing results, and whether there is a trade-off between editing time and editing performance.\n4. More recent works [1] should be included for comparison\n\n[1] EasyDrag: Efficient Point-based Manipulation on Diffusion Models, CVPR 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the positive feedback and helpful comments. Following are our responses to each individual comment (which are highlighted in italics).\n\n# Responses for Weaknesses (RfW):\n\n>*W1: If the drag distance is long, will the BNNI still success to maintain high semantic quality, how about the editing speed and complexity for long-distance dragging and latent relocation?*\n\n**RfW1:** For typical drag tasks, dragging is usually performed over short distances. However, for long-distance dragging, which is generally an object moving task, as illustrated in Figure 13 (row 3) in the Appendix, we do not need to employ the BNNI strategy. Instead, we can fill the semantics around the target location or a manually designated area of the image into the moved object's original location [18, 19], as described in Appendix B of our paper. We will further clarify this in the camera-ready version.\n\nHowever, for extremely long-distance drag editing, our method may lose some details of the object, as explained and discussed in the \"Limitation\" of our general response to reviewers. Please refer to the limitation regarding \"Extremely Long-distance Drag Editing\" in our general response to reviewers.\n\nNonetheless, long-distance editing does not increase the editing speed and complexity of our method. Our method can achieve latent optimization with one-step warpage optimization using Equations (3)-(10), which is not affected by the editing distance.\n\n>*W2: There should also be some failed examples to better illustrate the proposed method.*\n\n**RfW2:** Thank you for your valuable suggestion. We have included failed examples to illustrate the limitations of our method for better understanding, as shown in Figures 1 and 2 of the attached PDF with our author rebuttal. Please refer to the limitation discussion in our general response to reviewers.\n\n>*W3: I am curious whether using a better base model can achieve better editing results, and whether there is a trade-off between editing time and editing performance.*\n\n**RfW3:** Regarding better base model, it may potentially enhance editing results.\nSince the compared state-of-the-art methods all utilize SD 1.5 for drag tasks, our experimental results are solely based on SD 1.5.\nTheoretically, our one-step optimization method is independent of the base model.\nTherefore, FastDrag can perform drag tasks based on more advanced models such as SDXL or SDXL turbo, and its editing performance theoretically varies with the base model used.\nHowever, integrating more advanced models like SDXL would require substantial code base restructuring (we use relatively early version of diffusers), due to time constraints in the rebuttal period, we are not able to implement this in time. \nWe will support more base models in the future.\n\nRegarding the trade-off between editing time and performance, there should be a trade-off between editing time and performance for $n$-step optimization methods, as they all require $n$-step iterative optimization to achieve desired editing performance, though these methods can reduce editing time by decreasing the number of iterations, this typically degrades performance, since less iteration will lead to insufficient latent optimization. Thus there should be a trade-off for $n$-step methods.\n\nIn contrast, FastDrag leverages one-step warpage optimization to achieve latent optimization via Equations (3)-(10), the editing time will not be  influenced by editing task or images. Therefore, our method does not have the trade-off between editing time and performance.\n\n\n\n>*W4: More recent works [ref1] should be included for comparison.* \n\n**RfW4:** Thank you for the suggestion. The study [ref1] did not publish or preprint when we submitted FastDrag, thus we did not compare it. We will discuss and compare it in our camera-ready version.\n\n[ref1] EasyDrag: Efficient Point-based Manipulation on Diffusion Models, CVPR 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FyNlLb7PYw", "reviewer": "Reviewer_hGqW", "summary": "The paper introduces \"FastDrag,\" a novel one-step drag-based image editing method that significantly accelerates the editing process compared to existing n-step iterative approaches. The core of FastDrag is the Latent Warpage Function (LWF), which simulates the behavior of stretched material to adjust pixel locations within the latent space, enabling single-step latent semantic optimization. This results in substantially faster editing speeds while maintaining quality and precision.\n\nThe paper's contributions include:\n1. The proposal of FastDrag, a diffusion-based image editing approach that employs LWF for one-step semantic optimization, greatly enhancing editing efficiency.\n2. The introduction of Bilateral Nearest Neighbor Interpolation (BNNI), a new interpolation method that addresses null regions and enhances the semantic integrity of edited content.\n3. The incorporation of a consistency-preserving strategy that maintains image consistency during the editing process by using original image information to guide diffusion sampling.\n\nExperiments on the DragBench dataset demonstrate FastDrag's superiority in processing time, being nearly 700% faster than the fastest existing method and 2800% faster than the typical baseline method, with comparable editing performance. The paper also includes rigorous ablation studies to validate the strategies used in FastDrag. The authors aim to continue refining and expanding their approach for enhanced capabilities and applications in the future.", "questions": "**Questions for the Authors:**\n\n1. **Latent Warpage Function (LWF) Details:** Could you provide more technical details about the design and implementation of the Latent Warpage Function? Specifically, how is the stretching behavior modeled, and what criteria determine the degree and direction of pixel adjustments within the latent space?\n\n2. **Bilateral Nearest Neighbor Interpolation (BNNI) Robustness:** How does the BNNI strategy cope with complex scenes or objects with varied textures? Are there any specific cases where BNNI might struggle, and how do these instances affect the final image quality?\n\n3. **Consistency-Preserving Strategy and Diffusion Sampling:** Can you elaborate on the process of adopting semantic information from the original image during diffusion inversion, and how exactly these key-value pairs guide the sampling process to maintain consistency? Are there quantitative measures to evaluate the consistency preservation?\n\n4. **Scalability with Image Complexity and Dataset Size:** Your work demonstrates impressive speed improvements, but how does FastDrag's performance scale with the complexity of the image being edited or the size of the underlying dataset? Are there any benchmarks or theoretical analyses to predict computational demands for larger datasets or more intricate images?\n\n5. **Addressing Negative Societal Implications:** Given the potential misuse of FastDrag for generating disinformation or breaching privacy, have you explored or considered integrating any technical safeguards directly into the tool to mitigate these risks? For instance, adding watermarks, traceability features, or built-in detection mechanisms for manipulated content.\n\n6. **Limitations and Future Work:** In Appendix D, you discuss the limitations of your method. Could you elaborate on the most critical limitations that future research should address to improve FastDrag, and what are your thoughts on potential directions for overcoming these limitations?\n\n7. **Error Analysis and Sensitivity to Initialization:** How sensitive is FastDrag to the initial state of the latent space? Have you conducted experiments to analyze the variability in output quality based on different starting points, and if so, could you share insights on the stability of the method?\n\n**Suggestions for Improvement:**\n\n1. **Quantitative Evaluation of Semantic Integrity:** Consider including a quantitative evaluation of the semantic integrity of edited images, perhaps through metrics that assess feature similarity or structure preservation, to complement qualitative assessments.\n\n2. **User Study or Survey:** Conduct a user study or survey to gather feedback from artists, designers, and other potential users to understand the real-world usability and satisfaction with FastDrag. This could validate the claimed benefits of intuitiveness and efficiency.\n\n3. **Expand on Mitigation Strategies:** Further elaborate on potential mitigation strategies for the negative societal impacts mentioned. Discuss how these strategies could be practically implemented, and if possible, provide examples of successful implementations in similar contexts.\n\n4. **Code Availability and Reproducibility:** Since the computational efficiency and resource requirements are crucial aspects of FastDrag, consider making the code publicly available with clear documentation to facilitate reproduction of your results and encourage further development by the community.\n\n5. **Additional Experimentation on Diverse Datasets:** Extend the evaluation to include a variety of datasets covering different image types and complexities to strengthen the generalizability of FastDrag's performance claims.", "limitations": "The authors have addressed the limitations of their work in the \"Limitations\" section of the paper, which is in line with the NeurIPS Paper Checklist guidelines. They are encouraged to explicitly state any assumptions made and discuss how these assumptions could be violated in practical scenarios, as well as the potential implications of such violations. For instance, if the FastDrag method is highly dependent on certain image characteristics, such as resolution or lighting, the authors should clarify these dependencies and discuss the possible decrease in performance under varying conditions.\n\nRegarding potential negative societal impacts, the authors should discuss if there are any direct paths from their research to unintended or malicious uses, as outlined in the guidelines. Given that FastDrag is an image editing tool that could potentially be misused for creating disinformation or deepfakes, it is essential for the authors to acknowledge this possibility and outline strategies to mitigate such risks. This could include discussing ethical release strategies for the model, such as gated access, user guidelines, or incorporating detection mechanisms to identify manipulated content.\n\nIf the authors have not done so already, they should also reflect on the broader ethical implications of their work, such as fairness considerations (ensuring the technology doesn't unfairly impact specific groups), privacy concerns (ensuring personal data is protected), and security considerations (preventing unauthorized use). The authors should also consider including a discussion on how the technology could be used as intended but still give incorrect results, or how it might be intentionally or unintentionally misused, along with possible mitigations.\n\nIn summary, if the authors have not adequately addressed these aspects, they should revise their paper to include a clear limitations section that outlines any assumptions, constraints, or areas where the method may fall short. Additionally, they should discuss potential negative societal impacts and propose strategies to mitigate these risks. The NeurIPS guidelines emphasize that authors should be transparent about limitations and negative impacts, and doing so will not negatively affect their review but instead demonstrates responsible research conduct.", "rating": 4, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality:** The paper introduces \"FastDrag,\" a novel one-step drag-based image editing method that significantly accelerates the editing process compared to conventional n-step iterative methods. By proposing a Latent Warpage Function (LWF), the authors innovate by simulating the behavior of a stretched material to instantly adjust pixel locations within the latent space, marking a departure from traditional iterative latent optimization. Additionally, the Bilateral Nearest Neighbor Interpolation (BNNI) strategy for handling null regions and the consistency-preserving mechanism that utilizes self-attention key-value pairs add further novelty to the approach. The integration of these components into a diffusion model framework for image editing showcases a creative combination of existing techniques to solve a known problem more efficiently.\n\n**Quality:** The quality of the work is demonstrated through rigorous experimentation on the DragBench dataset, which validates FastDrag's efficiency and performance. The paper shows that FastDrag is not only faster but also maintains editing quality, being nearly 700% faster than the fastest existing method and 2800% faster than typical baseline methods. A series of ablation studies further reinforces the effectiveness of the individual components of the proposed method. The computational efficiency and scalability implications are considered, even though specific computational requirements are not fully detailed in the abstracted content. \n\n**Clarity:** The paper is structured coherently, presenting the FastDrag method systematically across its phases: diffusion-based editing, one-step warpage optimization, BNNI for semantic integrity, and the consistency-preserving strategy. The abstract and introduction effectively outline the contributions, setting clear expectations for the paper's content. Theoretical assumptions and limitations are openly discussed, complying with the NeurIPS Code of Ethics. The use of visual aids like Fig. 2 likely contributes to illustrating the method's workflow, although the figure itself is not accessible in the abstracted content.\n\n**Significance:** The significance of FastDrag lies in its potential to democratize and streamline image editing tasks, enhancing productivity in various domains such as art, design, education, and training. By offering an intuitive and efficient tool, it empowers a broader range of users to engage in creative image manipulation without requiring extensive technical skills. The societal impact discussion highlights both the positive outcomes of fostering creativity and the potential negative implications like misuse for disinformation or privacy breaches, showing a balanced perspective. The authors' consideration of mitigation strategies underscores a proactive approach to responsible innovation.\n\nOverall, FastDrag represents a marked advancement in image editing technology, combining originality in method design, high-quality empirical validation, clarity in presentation, and significant implications for both the research community and practical applications. Its contribution to reducing latency in image editing workflows can have far-reaching effects on digital content creation workflows and user experiences.", "weaknesses": "The paper titled \"FastDrag: Manipulate Anything in One Step\" presents a novel one-step drag-based image editing method that aims to accelerate the editing process compared to existing n-step iterative optimization methods. While the paper makes significant contributions to the field of image editing, there are several areas where the work could be improved:\n\n1. **Theoretical Depth**: The paper does not appear to include theoretical results or proofs to support the claims made. While it is stated that the paper does not involve theoretical results, providing some theoretical grounding could strengthen the contribution, such as explaining the mathematical properties of the proposed Latent Warpage Function (LWF) or the bilateral nearest neighbor interpolation (BNNI) strategy.\n\n2. **Experimental Design**: While the paper includes qualitative and quantitative evaluations, there could be a more extensive range of experiments to further validate the robustness of the method. For instance, testing the method on a broader variety of image types and editing tasks could provide a more comprehensive understanding of its capabilities and limitations.\n\n3. **Statistical Rigor**: The paper does not report error bars or other statistical measures that would provide insight into the variability and reliability of the results. Including such measures would enhance the credibility of the experimental findings.\n\n4. **Comparison with State-of-the-Art**: The paper compares its method with existing techniques but could benefit from a more detailed analysis of where FastDrag stands in terms of the trade-offs between speed and quality. A deeper dive into how the method's performance compares across different types of image content and editing tasks would be valuable.\n\n5. **User Study**: The paper lacks a user study that could provide insights into the usability and user experience of the FastDrag method. Including feedback from potential end-users, such as artists or designers, could highlight practical aspects that may not be evident from quantitative metrics alone.\n\n6. **Limitations Discussion**: While the paper does discuss some limitations, it could be more explicit about the scenarios where the method may not perform as expected. Providing more concrete examples and discussing potential workarounds or future work to address these issues would be beneficial.\n\n7. **Reproducibility**: The paper claims that the results are reproducible and that the code is included in the supplementary materials. However, it would be helpful to provide more detailed instructions on how to set up the environment and run the experiments, ensuring that other researchers can easily replicate the results.\n\n8. **Ethical Considerations**: The paper touches on societal impacts but could expand on the ethical considerations of using such technology, especially regarding the potential for misuse in spreading misinformation or altering reality.\n\n9. **Technical Details**: The paper could benefit from a more thorough explanation of the technical details, such as the specific design choices in the U-Net architecture and how the latent consistency model (LCM) is integrated into the FastDrag framework.\n\n10. **Long-Term Performance**: The paper could address how the method performs over time, especially considering the potential for the model to degrade in quality as more edits are made to the same image.\n\nBy addressing these points, the authors could strengthen the paper's contributions and provide a more comprehensive understanding of the FastDrag method's capabilities and potential areas for future research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for the positive feedback, helpful comments, and high praise and recognition of our work. Below are our responses to each individual comment (highlighted in italics).\n\n# Response for Weaknesses\n> *W1: Theoretical Depth*\n\nOur one-step optimization strategy is developed by simulating strain patterns in stretched materials, inspired by [20]. The core of this strategy is LWF, which is strictly derived by formula (3)-(10) in our paper. \n\nFurthermore, our effort in this study is to significantly reduce the editing time, the experiments we conducted have fully verified the contribution we claim. Specially, the results in Table 1 of our paper show FastDrag's ultra-fast editing speed compared to SOTA methods.\n\nAdditionally, the principle of BNNI is based on the most fundamental interpolation principles in the field of image processing.\n\n> *W2, 3 and 4: Experimental Design, Statistical Rigor and Comparison with State-of-the-Art*\n\nThe evaluation methods on DragBench [27] dataset are widely used in the field of drag-based editing, following the state-of-the-art methods [4, 14, 15, 27]. The results presented are averages across DragBench, which includes diverse image categories and drag tasks. Thus, the results on it are sufficiently demonstrating FastDrag’s superiority and robustness. \n\nAdditionally, for Statistical Rigor, please refer to the ''Statistical Rigor'' of the General Response.\n\nRegarding the trade-offs between speed and quality, please see the response RfW3 to Reviewer eL9M.\n\n> *W5: User Study*\n\nThank you for your valuable suggestions. We plan to conduct a user study through surveys in the released project demo to ensure the reliability of the results. The outcomes of this study will be published following the demo release.\n\n> *W6: Limitations Discussion*\n\nRegarding Limitations, please refer to ''Limitation'' in General Response to reviewers.\n\n> *W7: Detailed Code Instructions*\n\nThanks for your valuable suggestion. We will improve it and provide more detailed tutorials in the released code.\n\n> *W8: Ethical Considerations*\n\nIt is a common concern with generative methods. We will enforce strict open-source licensing in the publicly released code to limit unethical use.\n\n> *W9: Technical Details of U-Net and LCM*\n\nOur diffusion structure's implementation entirely follows the mainstream baseline DragDiffusion [27].\nSpecifically, the U-Net used in our model is widely used in image generation [4, 14, 15, 27, 31]. As stated in Appendix B, the U-Net structure is adapted with LCM-distilled weights from LDM (i.e., Stable Diffusion 1.5). We will further clarify this in the camera-ready version.\n\n> *W10: Long-Term Performance*\n\nThank you for your interesting idea. We conducted experiments and found that repeated editing using diffusion models can degrade image quality over time due to accumulated errors. This may be mitigated by refined training techniques or corrective algorithms. Due to PDF space constraints, we are unable to include the figure in this rebuttal but will include related discussions in the camera-ready version.\n\n# Response for Questions\n> *Q1: Latent Warpage Function (LWF) Details*\n\nIn Section 3.2.1, we provide a detailed explanation of the derivation process for the Latent Warpage Function (LWF).\n\n1) We model the stretching behavior by Equation (3), which normalizes and aggregates ''component warpage vectors'' $p_jp_j^{i\\ast}$ caused by multiple drag instructions into a single warpage vector for subsequent latent optimization, as shown in Fig. 3. The $p_jp_j^{i\\ast}$ are modeled by Equation (5), with the stretch factor modeled by Equation (6), designed to simulate the behavior of stretched materials.\n\n2) The degree of pixel adjustments is inversely proportional to the distance of the pixel from handle point $s_i$.\n\n3) The direction of pixel adjustments is the sum of $p_jp_j^{i\\ast}$, whose direction is the same as corresponding drag instruction's direction $|\\overrightarrow{s_i e_i}|$.\n\n> *Q2: BNNI in Complex Scenes*\n\nThank you for your valuable opinion. BNNI may struggle with cases involving detailed textures, as discussed in our response RfW1 to Reviewer eL9M. However, BNNI effectively handles most cases. Results on the DragBench dataset demonstrate its comparability with SOTA methods. We will try to improve it in future work.\n\n> *Q3: Consistency-Preserving Strategy Details*\n\nKey-value pairs contain semantic information through attention mechanism during diffusion inversion, which has been proved by many researches [2, 18, 19]. Via cross-attention mechanism in each step, key-value pairs containing ori-image's semantics guide the sampling process to maintain consistency. We use the ''1-LPIPS'' metric to evaluate consistency preservation. Please refer to ''Evaluation Metrics'' in General Response.\n\n\n\n> *Q4: Scalability with Image Complexity and Dataset Size*\n\nThere is no existing drag-based image editing benchmark with a focus on intricate images or larger datasets. Although DragBench is not as large as some traditional datasets, it is the most complex dataset in its field—featuring more classes and a mix of real and generated images compared to those used in [14, 21, 27], which contain less image number and class or miss key drag instructions, such as mask. It also supports our primary contribution of improving editing speeds.\n\n> *Q5: Negative Societal Implications*\n\nThanks for your valuable suggestion. About this issue, please see response in RfW8.\n\n> *Q6: Limitations and Future Work*\n\nRegarding Limitations and Future Work, please refer to ''Limitation'' in General Response to reviewers.\n\n> *Q7: Sensitivity to Initialization*\n\nFastDrag uses DDIM inversion strategy to get the initial state of the latent space as same as other drag-based editing methods [14, 15, 31].\nSince our latent optimization is decoupled from the diffusion inversion and sampling process, Fastdrag is not sensitive to the initial state of the latent space.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XLGSXurZTo", "reviewer": "Reviewer_Sik5", "summary": "This paper presents a method that enables fast drag-based image editing using diffusion models. The proposed method uses a latent warpage function to obtain the dragged latent representation. The additional nearest neighbor interpolation and content-preserving strategy further improve the result.", "questions": "- I wonder why the interpolation is called “Bilateral” nearest neighbor interpolation. For me, “Bilateral” term should be used for a scheme that considers the value as well. And the proposed interpolation scheme is just find the nearest non-null value. This is not critical but I think it is misleading.\n- It is unclear to me what the brighter region around the drag arrow means. Is that a mask? Are those region/mask included and used in drag instructions?\n- I am confused about Figure 8. What is the desired effect and result of the drag edit? Semantically, the user is moving “the dog’s” hand, not “the cat’s”. However, the result seem to generate cat’s hand for all different steps. Is this some limitation of all drag-based method? If yes, I think it is worth discussing.\n- I wonder if there are metrics or evaluation processes that can evaluate the content preserving quantitatively. Regarding image editing, I think this is a very important factor and the current paper only provides a couple of images as evidence.", "limitations": "As I mentioned above, I think the limitation discussion is not enough (I also raised another potential limitation). Moreover, I strongly recommend including the limitation discussion in the main paper, not supplemental material.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper are:\n- The proposed method seems straightforward, and the results are convincing.\n- A large speed performance gain compared to the previous method.\n- The evaluations support the paper’s claim.", "weaknesses": "The weaknesses of this paper are:\n- I think the discussion of limitation is too shallow. I recommend providing more failing cases and discussing the potential directions to enhance the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for the positive feedback, helpful comments, and the support of our work. \nFollowing are our responses to each individual comment (which are highlighted in italics).\n\n# Response for Weaknesses (RfW)\n>*W1: I think the discussion of limitation is too shallow. I recommend providing more failing cases and discussing the potential directions to enhance the results.*\n\n**RfW1:** Thank you for the constructive suggestion. We have provided more failing cases to illustrate the limitations of our study, as shown in Fig. 1 and Fig. 2 of the attached PDF with our author rebuttal. We also provided in-depth discussion about these limitations for potential future directions, please see the detailed analysis of the ''Limitation'' section in General Response to reviewers. \n\n\n# Response for Questions\n>*Q1: I wonder why the interpolation is called ''Bilateral'' nearest neighbor interpolation. For me, ''Bilateral'' term should be used for a scheme that considers the value as well. And the proposed interpolation scheme is just find the nearest null value. This is not critical but I think it is misleading.*\n\nWe apologize for any confusion caused by our terminology. The term ''Bilateral'' was chosen to emphasize that the interpolation considers nearest neighbors in two directions (i.e., both the X and Y axes), highlighting that the interpolation process involves adjacent non-null data across two dimensions. We will explain this terminology in the camera-ready version for better clarity.\n\n>*Q2: It is unclear to me what the brighter region around the drag arrow means. Is that a mask? Are those region/mask included and used in drag instructions?*\n\nYes, you are correct, the brighter region is a mask, indicating the image area to be edited.\nIt is used as drag instruction with the drag arrows, which is widely adopted in drag-based editing [4, 18, 19, 27].\n\nWe will explain this in the camera-ready version for better clarity.\n\n>*Q3: I am confused about Fig.8. What is the desired effect and result of the drag edit? Semantically, the user is moving ''the dog’s'' hand, not ''the cat’s''. However, the result seem to generate cat’s hand for all different steps. Is this some limitation of all drag-based method? If yes, I think it is worth discussing.*\n\nWe apologize for any confusion regarding Fig.8.\nThis case was randomly selected to illustrate an ablation study on number of inversion steps in terms of drag effect, which is used to determine the number of inversion steps in diffusion inversion.\nIt is not intended to demonstrate a specific editing result. To avoid such confusion, we will replace Fig. 8 with a more proper case in the camera-ready version.\n\nRegarding your question, the observed phenomenon is common for diffusion models. In this case, when ''the hand'' is moved onto ''the cat'', it semantically associates with ''the cat'', resulting in the generation of ''the cat’s hand''. This is because the U-Net architecture used in diffusion models contains numerous CNN layers, which fuse features of the cat and the hand in the latent space when their positions are close together. For generative artificial intelligence, the ability to generate image with contextual semantic association is actually considered a strength of diffusion models, such as Stable Diffusion.\n\n>*Q4: I wonder if there are metrics or evaluation processes that can evaluate the content preserving quantitatively. Regarding image editing, I think this is a very important factor and the current paper only provides a couple of images as evidence.*\n\nIndeed, we have provided evaluation results with regard to content preserving in terms of ''1-LPIPS'' metric, as shown in Table 1 of our paper. ''1-LPIPS'' metric is  widely used to for image content consistency evaluation for drag editing [4, 14, 15, 27]. Our experiments are conducted on DragBench dataset, and the results can demonstrate the effectiveness of our method, as explained in general response (i.e., evaluation metrics and statistical rigor) to reviewers.\n\n# Response for Limitations\n>*As I mentioned above, I think the limitation discussion is not enough (I also raised another potential limitation). Moreover, I strongly recommend including the limitation discussion in the main paper, not supplemental material.*\n\n\nThank you for the constructive suggestions. \nDue to page limitation, we did not include limitations in the main paper of the submitted paper. However, we acknowledge the importance of this discussion and will try our best to enhance and include it in the main paper of the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# General Response to Reviewers

We would like to thank the reviewers for the positive feedback and valuable comments. We are elated that the reviewers found our paper well-written, the presentation clear and excellence in ultra-short drag-based editing time compared to state-of-the-art (SOTA) methods. This rebuttal addresses reviewers' main concerns and suggestions for conciseness. For more detailed questions and answers, please refer to the individual responses to each reviewer.

**1. Evaluation Metrics (To Reviewers YBFQ, hGqW, Sik5)**

In our study, we employ the widely used performance metrics in the field of drag-based editing, i.e., "MD'' (Mean Distance [22]), "IF'' (i.e., "1-LPIPS'' [10]), following the SOTA methods [4, 14, 15, 27].  MD assesses how well the approach moves the semantic contents to the target points.  1-LPIPS quantifies the similarity between the original and edited images (i.e., consistency), as detailed in Section 4.2 of our paper. 

The metrics that Reviewer YBFQ suggested (i.e., SIFT, SuperPoint and DUST3R) are primarily used for feature point detection and matching. These may not be appropriate for evaluating performance of drag-based editing due to semantic changes within the editing region. For example, turning the head or thinning the face not only changes the image's content but also its semantics, which cannot be effectively measured by these metrics such as SIFT, as illustrated in Figure 13 of our paper. 

Besides, our effort in this study is to significantly reduce the editing time for drag-based editing, thus we adopt the evaluation metrics following these SOTA methods for a fair comparison, and the results effectively support our contributions, i.e., ultra-short editing time with competitive quantitative metrics.

However, as a novel task, existing mainstream metrics (i.e., MD and IF) in this field may not be  perfect to evaluate drag performance. We will make efforts to develop appropriate metric for drag editing in our future study.

**2. Statistical Rigor (To Reviewers YBFQ, hGqW, Sik5)** 

Our method is evaluated on the most widely used DragBench dataset [27] for drag-based editing, following the SOTA methods [4, 14, 15, 27] in this field, for a fair comparison. DragBench is a diverse compilation encompassing more than 10 types of images,  including 205 images with 349 pairs of handle and target points.

The comparison results presented are the averages across DragBench that includes diverse image categories and drag tasks. Thus, the results in our study are sufficiently to demonstrate superiority and robustness of our method, as compared with the state-of-the-arts.

To address the reviewers' concerns, we conducted an additional experiment by repeating our experiment 10 times under the same experimental settings. We observed that the variances of the performance metrics obtained from 10 realizations of our FastDrag are MD (0.000404), 1-LPIPS (9.44E-11), and Time (0.018), all of which fall within a reasonable range. These statistical results further demonstrate the effectiveness and stability of our method for drag editing. We will provide these statistical results in the camera-ready version for better clarity.

**3. Limitation (To all Reviewers)**

We will provide a more in-depth analysis of the limitations of our method in the camera-ready version from following three aspects:

**Overly Smooth and Finer Details Loss:** This is a common issue across diffusion-based drag editing methods, inherent to the diffusion models employed, such as LDM and LCM. For instance, DragNoise [15] and DragDiffusion [27] introduce textures not present in the original images, affecting fidelity, as illustrated in Figure 6 of our paper. 

Though our method may also result in some loss of finer details, it outperforms other state-of-the-art methods in overall task execution, particularly in editing speed.

**Extremely Long-distance Drag Editing:** When conducting extremely long-distance drag editing, our method may lose some details of the dragged object. As illustrated in Figure 1 of the attached PDF, though our method can achieve the long-distance drag editing, part detail information of the objects are missing, i.e., window on post box (row 1), and window on house (row 2). This is because our optimization is conducted on a lower-dimensional latent space. If the feature changes in the details within the latent space are too large (i.e., long-drag editing), the semantics of these details will be severely disrupted, making it more difficult to complete or maintain the details, resulting in the missing details of drag editing.

However, our FastDrag can still achieve better editing performance than these SOTA methods, as illustrated in Figure 1 of the PDF. 
The reason why these methods perform inferior to our FastDrag is that they require $n$-step optimizations, with each step to optimize semantics within small editing area of the image, thus they need $n$ small-scale, short-distance optimizations, to achieve overall latent optimization. When performing extremely long-distance drags, each step requires to optimize semantics within much larger editing area, resulting in difficulty for desired semantic optimization, leading to inferior performance to our FastDrag. 

**Highly Relying on Precise Drag Instruction:**  It is worth noting that achieving precise performance relies on clear drag instructions. FastDrag optimizes the latent space based on these instructions, which is also common for the SOTA methods. Therefore, providing clear instructions is crucial for desired performance. As illustrated in Figure 2 of the attached PDF, when the goal is to "thin the hair while keeping the face size", it is best to exclude the face from the mask region (row 2 of Figure 2). Similarly, when the task is to "lengthen the beak", the handle point should ideally be placed where the 'beak' feature is more prominent.

Meta-review: The final rating of this paper is Accept, Weak Accept, Borderline Accept, and Borderline Reject. The paper introduces a drag-based image editing method using the Latent Warpage Function (LWF) for single-step pixel adjustments, which significantly improves speed and accuracy over iterative methods. It combines bilateral nearest-neighbor interpolation (BNNI) and a consistency-preserving strategy to maintain semantic coherence. While the method is efficient and shows significant speed improvements, the reviews highlight concerns about the lack of comprehensive evaluation metrics, detailed analysis of unintended alterations, and the potential trade-off between editing time and performance. The reviews suggest including more recent works for comparison, providing more failing cases, and exploring the effects on other image parts. The authors’ rebuttal has properly addressed these concerns, resulting in an increase in rating. During the reviewer discussion phase, the AC urged Reviewer hGqW to respond, but there was no reply. The AC has read the paper and concurs that this paper has made significant contributions in drag-based image editing, warranting acceptance at NeurIPS. Please incorporate the materials in rebuttal into the final version of this paper.
Decision: Accept (poster)