Enhancing Consistency of Flow-Based Image Editing through Kalman Control
Authors: Haozhe Chi
Zhicheng Sun
Yang Jin
Yi Ma
Jing Wang
Yadong MU
Venue: NeurIPS 2025 poster ()
Keywords: Image Generation
Rectified Flows
Generative Modeling
Optimal Control
Image Editing
Primary area: applications

Abstract: Flow-based generative models have gained popularity for image generation and editing. For instruction-based image editing, it is critical to ensure that modifications are confined to the targeted regions. Yet existing methods often fail to maintain consistency in non-targeted regions between the original / edited images. Our primary contribution is to identify the cause of this limitation as the error accumulation across individual editing steps and to address it by incorporating the historical editing trajectory. Specifically, we formulate image editing as a control problem and leverage the Kalman filter to integrate the historical editing trajectory. Our proposed algorithm, dubbed Kalman-Edit, reuses early-stage details from the historical trajectory to enhance the structural consistency of the editing results. To speed up editing, we introduce a shortcut technique based on approximate vector field velocity estimation. Extensive experiments on several datasets demonstrate its superior performance compared to previous state-of-the-art methods.

Enhancing Consistency of Flow-Based Image Editing through Kalman Control
Haozhe Chi Zhicheng Sun Yang Jin Yi Ma Jing Wang Yadong Mu
!Peking University, CCentral Media Technology Institute, Huawei
Abstract
Flow-based generative models have gained popularity for image generation and editing. For instruction-based image editing, it is critical to ensure that modifi- cations are confined to the targeted regions. Yet existing methods often fail to maintain consistency in non-targeted regions between the original edited images. Our primary contribution is to identify the cause of this limitation as the error accumulation across individual editing steps and to address it by incorporating the historical editing trajectory. Specifically, we formulate image editing as a control problem and leverage the Kalman filter integrate the historical editing trajectory. Our proposed algorithm, dubbed Kalman-Eda reuses early-stage details from the historical trajectory to enhance the structural consistency of the editing results. To speed up editing, we introduce shortcut technique based on approximate vector field velocity estimation. Extensive experiments on several datasets demonstrate its superior performance compared to previous state-of of-the-art methods.
Introduction
Diffusion models 17 have revolutionized the field of image and video generation, bringing unprecedented advancements. The recent development of the Diffusion Transformer (39 has enabled current diffusion- based models to scale up their parameters, achieving higher level of generative capability. Notable works such as Stable Diffusion 3 [10] and Sora (5) demonstrate the remarkable potential of diffusion models in generating complex and intricate scenarios in both images and videos. Furthermore, researchers have explored the potential of diffusion models in editing tasks. For instance, DDIM inversion [45] progressively matches the target image distribution back to the original latent space, allowing for the generation of edited images by incorporating new prompt conditions. Additionally, some studies have modified attention maps 18 14] during the generation process to directly alter specific characteristics of objects. More recently, efforts have been made to integrate pretrained modules for more precise editing. For example, existing work (29] utilizes SAM (25) to extract specific regions requiring modification. Nevertheless, many diffusion-based editing methods still face challenges related to imprecise editing, primarily due to the non-linear nature of the generation trajectory.
special class of diffusion models, transforms random noise into the target distribution by linear interpolation between the two distributions. These models achieve distribution matching by constructing a velocity field, resulting in an efficient trajectory. Recent advancements have extended rectified flow models image editing tasks. However, existing methods often struggle to balance structural consistency and editing quality effectively. For instance, Wang ef al. 5 employ a new sampler to achieve more precise inversion and freeze specific attention values to preserve the overall semantics of edited images. While this approach maintains structural consistency, it sacrifices editing flexibility, as we will demonstrate later. Similarly, Rout et al. [43] utilize Linear Quadratic Regulator (LQR) control (2l) to guide the editing process, and Kulikov et al. [27] propose using
Rectified flow 30 31 22
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

a. quality-consistere dilemma in controller-based editing
b.
Enhancing editing consistency through Kalman control
Sorigt
Xaris
Inversion
Generaton
Without controllrr
Kolman centrol
3
hortzut estmmaton
Puck
Xnii
Anght
editing
over-ediling
eider-edilng
over-editing
C.
Taié
Tright
Pt
Figure 1: Conceptual illustration of previously-develope controller-based method 43 (left) and our method (right). 2, represents original image, while & and 2i represent edited images. The controller- based method derives an optimal control strategy for different stages of the generation process. However, as illustrated in the figure for Tief, excessive control leads to failed edits, whereas Fright demonstrates that insufficient control results in significant structural inconsistencies. Additionally, Tmid reveals that moderate control intensity produces ambiguous image semantics. Building on these key observations, we introduce Kalman control to further suppress irrelevant semantics while maintaining structural consistency. We also propose shortcut estimation to eliminate the need for second inversion process. The figure underscores the effectiveness of our approach.
an estimated velocity transform to directly map original images to the target space. However, both methods exhibit limitations in editing quality, highlighting the need for further refinement in this area.
The primary technical contribution of this work lies pinpointing the cause of inconsistency in image editing (i.e., notable change over non-targeted image areas) as the accumulation of errors (i.e., small inaccuracies can propagate forward and lead to significant deviations in the output) over individual editing steps and mitigating it by integrating the historical editing trajectory. We propose Kalman- Edit (along with its accelerated variant Kalman "diut""" ), a method that addresses the Linear Quadratic Gaussian (LQG) problem (23 in optimal control theory. Our approach assumes that each velocity step in the LQG process incorporates a noise term, which can be estimated and reduced through the integration of Kalman filter [22] Specifically, we design inversion algorithm that transforms the original image latent into a mixed (i.e., encapsulates features from both the original and edited images). This mixed latent corresponds prompts. Subsequently, we invert the mixed latent and apply the Kalman filter to preserve structural information in the background. Our contributions are summarized follows: (1) We introduce image editing. (2) Based on observations of direct
semantic blend between the source and target
the Kalman filter approach to controller-basee LQR control, propose a two-stage method that better unleashes the potential of Kalman control and achieves more flexible control.
(3) Through experiments, our method shows high structural consistency and good editing flexibility on various editing tasks.
2
Related work
remark- able progress has been made image editing via diffusion model inversion 46) To address the time-consuming inversion computations and compounding estimation errors, various sampling and es-
Image editing with diffusion models. Following advances of diffusion models 44
47 17
timation strategies are developed [33 9
addition, several work [42 24 28 introduce optimization methods to achieve better editing quality. Alternative editing algorithms
15
35
52
36
54
20).
include attention map controls [16 50) and masking strategies (37 However, their editing quality
and efficiency remain to be improved.

---

Image editing with rectified flow. Rectified flow B0, 31 2 learns linear interpolation between
souree and target distributions to enable more efficient sampling of diffusion models. In addition to significant advances in rectified flow- how-based generation models [ 4 its potential in image editing is gaining increasing attention [27
38 53 43 Kulikov al. [27] and Patel al. [38 take derivatives
to produce more precise edits. Wang ef al. [3 share attention scores in Transformer blocks to
improve editing consistency. Rout et al. [43 use optimal controller to guide the generation trajectory.
However, these efforts have not yet achieved good balance between editing quality and consistency,
as evidenced in our experiments later.
Image editing with control theory. Inspired by the connections between optimal control and SDEs [5T 48 [2]. various sampling strategies control theory are introduced diffusion models for more controllable generation. Koo et al. [26] apply posterior sampling strategy for linear inverse problems. Rout el al. [43 utilize conditional sampling strategy for optimal control in the vector field. Our method advances this line of study with Kalman filter for more precise image editing.
Methodology
3.1 Preliminaries
Rectified flow. Flow-based generative models [49 55 30 aim to learn probability path between the source distribution go and the target distribution gi with velocity field The flow starting from 2o 2o to qi is formulated as the following ordinary differential equation (ODE):
dæ: di
V (xx,)) t);
where the velocity function takes the timestep t E [0, 1] and the latent variable x, as input. Rectified flow (30 31 2 is a special class of flow-based models defined by the linear interpolation between start point To and endpoint 21:
Tt = - () + (x)
(2)
Note that rectified flow is empirically able generate high-resolution images with fewer sampling steps 32
making it prevailing in image generation tasks.
Editing with rectified flow. To edit a given image Is with source prompt cs and a target prompt G, a straightforward approach is to perform ODE inversion using Eq. [ Let V Vzz,, t, c) denote the velocity function conditioned on prompt c, then editing follows two-stage procedure:
dx
Inversion:
dt (z),tt,),,
dxf dt V(tt,t,a)
Generation:
where we first invert the image x, to structured noise Tg. and then perform sampling with target prompt & for generation. However, directly applying this could lead to significant deviation from the desired result (see ž; in Fig. [
Optimal control. To improve editing controllability, an effective approach is to formulate it as optimal control [R] Both inversion and generation processes can be viewed as continuous-time linear system defined over i in [0, 1].
dx: dt
Ax, Bug,
(4)
where Ie represents the state the system, u serves as the controller of the system, and A, B are coefficient matrices. In optimal control theory, the objective is to determine optimal controller My that guides the drift path to minimize the energy cost. A choice the energy cost is a quadratic function, which corresponds to the Linear Quadratic Regulator (LQR) problem [1] follows:
1= x. Fx1 + / (z, Qxi + ug Rue) dt,
(5)
3

---

where F, Q and R are coefficient matrices of the system. As shown in previous work the problem for rectified flow in Eq. produces the optimal controller:
43 solving
I.
Zz.
(6)
where v, represents the original image for reference. Although this controller reduces the overall error during editing, it fails preserve detail consistency for background regions, illustrated in Fig.
3.2 Harnessing history with Kalman filter
To improve the consistency between the edited and original images, we make a key observation that the history sequence in inversion can effectively rectify the generation trajectory. As illustrated Fig.[ applying control signals to latents at different steps results distinct semantic and detail- level changes. A comparison between F'id and Fright reveals that latents from later generation steps are particularly effective recovering fine details of the original image. Additionally, we observe that carly inversion latents also contribute significantly restoring structural information. To fully leverage these latents for improved consistency, it is essential to maintain sequence of observed latents guide and refine the editing process. This approach naturally aligns with Kalman control, where observations are used to refine estimations and achieve more accurate outcomes.
Inspired by optimal control theory, reformulate the editing process Linear Quadratic Gaussian (LQG) problem [23] and introduce the Kalman control method [22]. To be specific, in LQG control theory, the system's evolution at each timestep includes noise terms, which can be mitigated by refining the trajectory using Kalman filter, i.e.,
dr. Ax, + Bu, + W/s Vt H + Ce: dt
where yt represents the measurement sequence, A, B and H are the coefficient matrices of the system, while wi, and or denote the noise terms of system state estimations. With the application of the Kalman filter, the total cost function to be minimized is given by:
J2
1 Fxi +
(xt Qu, + un Rus) di
(8)
where E refers to the expectation of the following terms. Importantly, expectation is necessary in this context, as our goal is to mitigate the impact of noise terms when minimizing the cost function J2. To accurately estimate the expectation in J2, we utilize the following Kalman filter equations:
K₁ P (HP HT +T)
Jk = Aœ + Bux + KR(u - Hxx-1),
Ps PK-)
In specific, given the covariance matrix Pk-1, state Tk-1, noise T, controller ux and measurement yk. it first computes the Kalman gain Kg. These values are then used recursively to obtain the updated terms Tg and Pg. Consequently, all the filtered latents Tg can be computed using these equations. Here, the noise term is estimated and corrected by multiplying the Kalman gain K₃ with the innovation term (yk - H 1). Through proper derivation and analysis, we arrive at the following proposition (see Appendix for details):
Proposition 1. With proper initialization of system coefficients (Po. H, T), the Kalman control process shown in Eq. converges.
As shown Fig. I achieving both high generation quality and structural consistency presents a significant challenge in current controller-based editing methods. Addressing this issue requires establishing an effective measurement sequence. To sufficiently incorporate historical information, we carefully construct the measurement sequence {y V of length i, integrating latents from the inversion process:
V/ Inv k), 1<= <=
(10)
where Inv(r, k) denotes the th latent obtained during the inversion process described in Eq. 3 and the resulting measurement sequence {yx)k= are incorporated in the computation of the Kalman filter to control the generation process.

---

Algorithm Kalman-Edit and Kalman-Edit"
Input: original image F.,, total step N, source prompt C., target prompt Inversion function (-v
Stage I: editing with optimal controller
Forigl +- [nv N)
Generate mid from Farigt with controller in Eq. (6)
# Stage 2: editing with Kalman filter
if Kalman-Edii then
Torig2 <- Inv N) else Kalman- Edit" then
Forig2 Forig1 + Fmid 2z
shortcut in Section 3.4
Compute measurement {yee [y according to Eq.
Generate 2; from Furig2 with Kalman filter in Eq.
Output: 2₁
Following this formulation, the entire Kalman control process proceeds as follows: First, we construct the measurement sequence using Eq. (TO). Next, iteratively update the Kalman gain K₂ and compute the corresponding innovation term y - to regulate the generation sequence Tę using Eq. (9). This way. the generation process is conditioned on measurement from the original image before editing, thus preserving more details.
3.3 Proposed algorithm: Kalman-Edit
Applying Kalman control to flow-based editing presents two primary challenges: (1) As shown in Fig.[ determining the appropriate timesteps for applying the Kalman filter is challenging. too many timesteps undergo the filtering process, editability may be compromised, leading to edited results that fail to faithfully reflect the target prompt. (2) Artifacts and blurring often arise when directly applying Kalman control through a single inversion and forward process. This occurs because the filtering equations can inadvertently guide the trajectory toward an intermediate state between the original image distribution and the target distribution. Such a middle state often corresponds poor image quality, resulting in undesirable visual artifacts.
Two-stage image editing. To address both challenges, we propose two-stage algorithm for generating high-quality edited images using Kalman control, as outlined in Algorithm [ In the first stage, our goal is generate intermediate latent that encapsulates the semantics of both the original and target prompts. Next, we generate the intermediate latent Zmid by applying the controller appropriate timesteps. In the second stage, we first invert Tiid to obtain the second original latent Torig2. We then apply the Kalman filter, as described in Eq. (9) to the generation process in order to filter out undesired semantic information irrelevant to the target prompt, and reintroduce history information from the original image 2s. This approach results in target image that retains the structural information of x, while adhering to the target prompt G, as desired.
We further adapt the measurement sequence construction to the two-stage editing scheme. Since the original image v, and edited intermediate image Wmid both contain desired information (i.e., the original detail and target semantics), we curate the measurement sequence by collecting inversion trajectories from both images x; and 2iidd Specifically, we introduce hyperparameter and define the measurement sequences {yx } and {yx ) capture structural information from the first and second inversion processes, respectively. They are computed by the following inversion processes:
Inv k), <= <= 5-1
(1I)
V/: Inv( k), 8 <<=k<=
By incorporating the two inversion sequences and integrating them into mcasurement sequence, it enables the recovery of diverse structural and semantic information through Kalman control. Note that this construction scheme is also compatible with our accelerated version without two inversion passes (Section 3.4), in which case we simply set the 8 value to / /++1 1.
Kalman filter phases. Another crucial consideration in our approach is selecting the appropriate timesteps for applying both the controller and the Kalman filter effectively. key observation that rectified flow models exhibit behavior similar to traditional diffusion models. As noted in prior
5

---

Table Quantitative comparison on SFHQ datasets among flow-based editing models. See the main text for the definitions of the performance metrics. The highest value in each column is highlighted in bold.
Face Rec. L
CLIP-I
LPIPS L
CLIP-T
DreamSim
RF-Edit
0.4051
0.8984
0.1562
0.2910
0,1591
RF-Inversion
0.4325
0.8927
0,1720
0.3012
0.1889
Flow Edit
0.4856
0,8579
0.1687
0.2905
0,2375
FlowChef
0.4013
0.8769
0.1401
0.2832
0,1487
Kalman- Fdit
0.3958
0.9167
0.1332
0.2921
0.1408
Kalman-Edit"
0,8871
0,1892
0.2936
0.2227
Table 2: Quantitative comparison HQ datasets among flow-based editing models.
CLIP-TT
CLIPIT f
DINO 1
DreamSim
RF-Edit
0,1842
0.9141
0.2383
0.8197
0.1492
RE-Inversion
0.1825
0.7963
0.1662
FlowEdit
0.1877
0,8813
0.2846
0.7467
0.2238
FlowChef
0.1928
0.8053
Kalman Edit
0.1943
0,9062
0.2345
0.7929
0.1353
Kalman- Editt
0,1870
0.8696
0.3615
0.7123
0.2276
work (56]. image generation in diffusion models progresses through distinct phases: the early stages primarily establish semantic content, while finer details are refined in the later stages. This pattern is also evident in rectified flow models. Applying the controller across too many timesteps during the refinement phase would overly constrain the output distribution, making it too similar the original image and thereby limiting effective editing. To ensure high-quality editing, we apply the controller primarily during the early phase of generation when semantic information is being formed. Likewise, to maximize the effectiveness of Kalman filtering, we apply it during the refinement stage, where it can best aid in recovering structural details.
Implementation for flow models. In the above derivation we consider the evolution of zs. However, flow-based generative models are often parameterized by velocity instead of x-prediction. Therefore, in practice, we apply the Kalman control updates in the following manner:
Tlet1 Ft, (tx+1 tk tg)
(12)
2/ 1ut2 + Xuee + (1 μ A Hz
where U/2 = (Z tx) is the predicted velocity, uhe is the optimal controller given by Eq. (6). and μ and X are coefficients balancing their contributions. See Algorithm for complete update details.
3.4 Kalman-Edit": acceleration with shortcut
To avoid the computational cost of performing the inversion process twice, we accelerate the esti- mation of Torig2 by leveraging the parallelogram law of vectors. This approach is justified in many editing scenarios, such as local area modifications, where the difference between T'idd and x, is minimal. Under the assumption that both source and target latents are normalized and exhibit similar variance, a first-order approximation the vector field yields a sufficiently accurate estimate of oooig
Torig2 Ferigl + (Xmid x.))
(13)
This avoids the second inversion process and turns out be efficient through experiments.
4 Experiments
4.1 Evaluation protocols
Datasets. The experimental evaluation is conducted across four widely used datascts: SFHQ B]] HQ 19], ZONE [29] and DIV2K (T. The SFHQ dataset consists of 425,000 high-quality human facial images. The HQ datase is synthetic editing benchmark containing approximately 200,000

---

Table 3: Quantitative evaluations on ZONE and DIV2K datasets. See main text for more details about the performance metrics.
CLIP-TT
CLIPTT 1
LPIPS
DINO
DreamSin
SDEdit
0.2754
0.1908
0.8547
0.1148
P2P
0,2773
0.9209
0.1568
0.8186
0.1519
MasaCul
0.3103
0,9179
0,1580
0.8397
0.1635
DDPM Inv
0.2847
0.9063
0.1734
0.8215
0.1742
RI-Edic
0.2964
0.8926
0.7986
0.1776
RF Inversion
0.2844
0,8919
0.2491
0.7974
0.1536
FlowEdit
0.3096
0,8687
0.2269
0.7671
0.2319
FlowChef
0.3025
0.8831
0.2563
0.7456
0.2471
Kalman- Edit
0.2957
0.1407
0.9141
0,0793
Kalman Edit*
0.3220
0,8986
0.2488
0.8237
0.1454
Table 4: Comparison of CLIP-I (left) and LPIPS scores(right) for different Kalman filter strengths and steps evaluated on the ZONE dataset.
Filter strength / Added steps
15-18
15-22
15-27
Filter strength / Added steps
15-18
15-22
15-27
0.1
0.8770
0,9219
0.9346
0.1
0.2433
0.2035
0.1487
0.2
0.9043
0,9282
0.9226
0.2
0.2325
0.1944
0.1521
0.3
0.9014
0.8921
0.9079
0.3
0.2284
0,2008
0.1886
images generated through DALL-E 3 and GPT-4V. The ZONE dataset features 100 images designed for object insertion, editing, and removal tasks. DIV2K serves standard benchmark for super- resolution tasks, comprising 1,000 real-world images. Due to practical computational constraints, we evaluate our approach on the subsets of these benchmarks, including 1, 1,200 images from SFHQ, 320 images from HQ, and 105 images from ZONE and DIV2K. Following the setting of Rout et al. [43 we employ an instruction prompt that adds glasses all face images in the SFHQ dataset.
Metrics. Six metrics are employed to evaluate both editing quality and consistency. For editing quality, CLIP-T [40] is adopted to measure the semantic adherence between edited image and input prompts. Meanwhile, we also use Face Rec. metric to quantify identity similarity on the face-specifi SFHQ dataset. Regarding editing consistency, CLIP-I and DINO (7] measure high-level semantic similarity, while LPIPS 57] captures low-level similarity such as pixel-level details. Moreover,
Dreamsim 13 is responsible for evaluating mid-level similarity, including image layout.
Baselines. Our method is compared against eight image editing baselines spanning rectified flow and diffusion models. For rectified flow-based editing, we consider RF-Edit [53], RF-Inversion [ FlowEdit (27 and FlowChef (38). For diffusion- -baeed editing counterparts, we compare against
SDEdit 34 P2P [66. MasaCtrl (6] and DDPM-Inv [8]. To ensure fair comparison, we follow the original recommended hyperparameter settings (e.g., where to add the controller) for all baselines.
Implementation details. For flow- editing, we use FLUX,1 1 dev 4 with N = 28 sampling steps. For diffusion-buas based editing, we use Stable Diffusion 1.4 [41] with N 50 sampling steps. The measurement length I is set to 14, and 8 is 6 by default. More details are provided in Appendix B
4.2 Experimental results
Quantitative results. As demonstrated in Table [ and Table 2 our method maintains high facial similarity after editing on the SFHQ dataset and effectively adheres to complex editing prompts on the HQ dataset. The basic version of Kalman-Edh demonstrates strong performance, achieving state- of-the-art results in most metrics. And Kalman-Edit" (the accelerated variant) produces comparable results on SFHQ and HQ datasets. This discrepancy can be attributed to the fact that Kalman- " directly estimates the second original latent, which may result in the loss of low-level structural information. As shown in Table 3 Kalman- Edit and Kalman- also outperform the four baseline methods across most metrics ZONE and DIV2K datasets, showing the effectiveness of our method.
Ablation analysis. this section, we conduct ablation experiments to determine the optimal filter strength and steps at which the filter is applied, as well as to highlight the importance of Kalman control in structural preservation. Following previous research [56), we observe that the generation process of rectified flow can also be divided into two stages: semantic formation and refinement.
7

---

Original Imege
RF-Bit
RP-Javerston
Ous
a girl
beside the
mon
Add a
mooldrssh
the
Changy
N
flosers
Change
modern cothr
city
Figure 2: Comparison of structure preservation and editing quality on ZONE and DIV2K dataset. The top three rows demonstrate that our method effectively preserves local details, ensuring strong structural consistency. Meanwhile, the bottom two rows highlight its ability to adhere to the target prompt, accurately incorporating elements such flowers and brick walls. They illustrate that method achieves both better structure preservation and editing quality. Better viewing when enlarged.
Table 5: Ablation study of Kalman filter in LQR-bbaed control method on ZONE dataset.
Metries
CLIP-- t
CLIP-I
LPIPS 4
w/o Kalman filter
0.2952
0.8784
0.2695
w/ Kalman filter
0.2961
0.9346
0.1487
Since the Kalman filter is designed to refine structural details, apply Kalman control during the refinement stage, which corresponds to the latter half of the generation steps. As shown Tables 5nd the best CLIP-I and LPIPS scores achieved when using relatively small filter strength combined with a large number of added steps. Also, from each row of Table 4 we conclude that more steps of Kalman control help recover more pixel-level details. From each column of Table 4 observe that small strength helps to generate structural details more efficiently and smoothly, while large strength causes significant performance drop. This indicates that longer measurement sequence enhances structural details, while lower filter strength steers the trajectory toward higher-quality distribution. Overall, our method is effective across fairly wide range of hyperparameters.
To validate the effectiveness of our approach, we conduct ablation experiments to assess the impact of Kalman control. Specifically, compare method against the RF-Inversion bascline without the Kalman control. As shown in Table 5] our method achieves higher CLIP-I and LPIPS scores, indicating superior structural consistency. Furthermore, the CLIP-T metric confirms that our approach maintains high editing quality, demonstrating its advantages in both structure and semantics.
Qualitative results. Our qualitative comparison results are presented in Figures and 3 showcasing both real-world and synthetic images to demonstrate our method's ability to maintain high structural consistency and editing quality. In particular, Fig. 1ese2cooppp compares the structural preservation capabilities of our method against baseline approaches. As shown, our method retains more local details than the baselines. For instance, in the second row of Fig. 2 only our method successfully recovers the
and

---

Oragnall
itrntes
Ri-reen
Figure 3: Qualitative results obtained using human face images from SFHQ dataset. The first two rows are edited with target prompt "A person wearing glasses" and the last row is edited with target prompt "A person with beard". The edited results are compared with baseline methods, demonstrating our approach's 's superior ability to preserve the structural details of human faces (i.e., our method produces edited images higher fidelity, recovering facial features more accurately than baseline approaches).
traffic cones on the road. Similarly, the last row, our edited results accurately incorporate multiple elements specified in the target prompt, demonstrating our method's flexibility in handling longer and more complex prompts. In contrast, baseline methods struggle to recover structural details while maintaining overall editing quality. Compared to the baselines, our method significantly outperforms in preserving the structure of the original images while effectively editing the desired areas. Further- more, to demonsirate the effectiveness of our method in preserving structural information across various image structures, we present editing results on human faces Fig. 3 Specifically, we first modify the images by adding glasses to each face, following the approach in 43) To further validate the generality of our method, we then edit the images by adding beards to each face. The edited results maintain high degree of structural similarity to the original faces, highlighting our method's s ability to preserve facial structural consistency. For additional qualitative results on tasks such as style transfer and scene editing, please refer to Appendix D
Computational efficiency. Figure Saareeecomme compares the time usage of our approach against flow- based editing methods. Since RF-Edit takes much time to load models, we report the time
Model Running Comparison
monnre time becarou
cost from the starting point of inversion to the
Ctt
time editing is completed to ensure the fairness. From the comparison, we observe that FlowChef has the fastest editing speed. Our proposed Kalman-Edit" is slower than RF-Inversion and FlowChef, but faster than all other baselines. This indicates that our algorithm strikes a com- petitive balance between editing performance and efficiency.
Howdrdd
RF-Ine
Kalman-dii Galman-diii
Models
Figure 4: Running time comparison of different flow-based methods highlighting efficiency.
Failure cases. For tasks such as object removal, our method requires hyperparameter tuning to achieve optimal results. This necessity arises from our design that leverages historical inversion latents as the measurement sequence to rectify the final generation process. As a consequence, the original details and residual artifacts from the original image tend to appear in edited images the default control strength settings, as illustrated by the boat example in Appendix G
Style transfer and Scene editing. To further evaluate our method on wider range of tasks, such as style transfer and scene editing, we present additional qualitative results. As illustrated in Fig. 5 our method demonstrates strong capability in transforming an old rusty room with cement floor into a simple and elegant room with a wooden floor, converting castle into a Disney-style cartoon scene, transforming a house into white church with stained-glass windows, and replacing a beach background with snow-covered mountains. Moreover, our method effectively preserves structural consistency, maintaining both local details and the overall spatial layout. For example,

---

Simple and elegont style) emply with wooden floor und view of the /impressionism beack]
Change beach the background snow mountain
Change to Disney cartoon style bridge
cle
$11
[Stained glass window off white [cartoon] church sits a hill in field
Figure 5: High-recs nigh-resolution qualitative results of style transfer and scene editing tasks. The left image is the original input and the right one is the edited result. As illustrated above, our method achieves precise prompt adherence and delivers high-quality editing outcomes.
the edited result in the top case of Fig. retains the room's geometric structure, while in the case, it accurately preserves the road direction and car position. These results demonstrate that our method can successfully handle broad variety of complex tasks and produce high-quality, structure- preserving outputs. Additional discussions on broader tasks and high-resolution visualization results are provided in Appendix D and Appendix
5 Conclusion
In this paper, we propose Kalman-Edi training-free flow-based image editing method based on optimal control theory. Existing rectified flow editing methods struggle to balance structural consistency and editing quality. To address this challenge, we derive fundamental equations from Linear Quadratic Gaussian (LQG) control, effectively utilizing history information in the editing trajectory with Kalman filter-based algorithm. Through extensive experiments, we demonstrate that Kalman-Edi achieves superior structural consistency while maintaining high editing quality.
10

---

Acknowledgement:
This work is supported by a grant from Huawei (No. TC20240821013 _1).
References
[1] Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 126-135, 2017.
[2] Michael Albergo and Eric Vanden- Vanden-Eijnden. Building normalizing flows with stochastic inter- polants, In International Conference on Learning Representations, 2023.
[3] David Beniaguev. Synthetic faces high quality (sfhq) dataset, 2022.
6
[4] Black Forest Labs.
FLUX. https://gttub...cmmm////
2024.
7 22
[5] Tim Brooks, Bill Peebles, Connor Holmes, Will DePuc, Yufci Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video gen- eration models as world simulators. https://opent www...co//reeeeeeeee comm/eeseach/
sissmmmmtmaorsss. 2024. !
[6] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages
22560 22570,, October 2023.
[7] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. Proceedings of the IEEF/CVF international conference computer vision, pages 9650-9660, 2021.
[8] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. In ICLR 2023 (Eleventh International Conference on Learning Representations), 2023. 2
[9] Adham Elarabawy, Harish Kamath, and Samuel Denton. Direct inversion: Optimization-free text-driven real image editing with diffusion models. arXiv preprint ar arXiv:221 2022. P
[10] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024. I
[11] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. In International Conference on Machine Learning, pages 22606-12633, 2024. 3
[12] Wendell Fleming and Raymond Rishel. Deterministic and stochastic optimal control, volume 1. Springer Science & Business Media, 1975.
[13] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. Advances in Neural Information Processing Systems, 36, 2024. 7
[14] Michal Geyer, Omer Tal. Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing, 2023. [
[15] Chunming He, Kai Li, Yachao Zhang, Longxiang Tang, Yulun Zhang, Zhenhua Guo, and Xiu Li. Camouflaged object detection with feature decomposition and edge reconstruction. In Proceedings of the IEFF/CV conference on computer vision and pattern recognition, pages 22046-22055, 2023. 2

---

[16] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfrr Aberman, Yael Pritch, and Daniel Cohen- or. Prompt-to-prompl image editing with cross-attentin control. In The Eleventh International Conference on Learning Representations, 2022. 2 22
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, pages 6840-6851, 2020. 2
[18] Inbar Huberman- Spiegelllsss Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 12469-12478, June 2024.
[19] Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang. Yuyin Zhou, and Cihang Xie. Hq-edit: A high-quality dataset for instruction- image editing, 2024. 6
[20] Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd:
A native skeleton- guideddgueddddoeo diffusion model for human image generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15988-15998, 2023. 2
[21] RE Kalman. Contributions to the theory of optimal control. Boletin Sociedad Matematica Mexicana, 5:102-109, 109, 1960.
[22] R.E. Kalman. A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82(1):35 1960.
[23] RE Kalman and RS Bucy, New results in linear filtering and prediction theory. Journal of Basic Engineering, 83(1)995-108, 1961. 4
[24] Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2426-24355 2022. 2
[25] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Lo, et al. Segment anything. In Proceedings of the IEEF/CVF International Conference Computer Vision, pages 4015-4026,
2023. 1
[26] Juil Koo, Chanho Park, and Minhyuk Sung. Posterior distillation sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13352-13366
2024. 3
[27] Vladimir Kulikov, Matan Kleiner, Inbar Sineerman-SSiegggglla and Tomer Michaeli. Flowedit: Inversion-free text-based editing using pre-trained flow models. arXiv preprint arXiv:2412.08629 2024.
1 3
[28] Gihyun Kwon and Jong Chul Ye. Diffusion- based image translation using disentangled style and content representation. In The Eleventh International Conference on Learning Representations, 2023.
2
[29] Shanglin Li, Bohan Zeng, Yutang Feng, Sicheng Gao, Xiuhui Liu, Jiaming Liu, Lin Li, Xu Tang, Yao Hu, Jianzhuang Liu, et al. Zone: Zero-shot instruction-guite local editing. In Proceedings of the JEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6254 6263, 2024.
16
[30] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In International Conference on Learning Representations, 2023.
[31] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations, 2023.
[32] Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. InstaFlow: One step is enough for high-quality diffusion- based text-to-image generation. In International Conference on Learning Representations, 2024.
12

---

[33] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver+t: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211000095, 2022.
[34] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022. 22
[35] Daiki Miyake, Akihiro Iohara, Yu Saito, and Toshiyuki Tanaka. Negative-prond preeatie--romplt inversion: Fast image inversion for editing with text-guided diffusion models. arXiv preprint arXiv:2305. 16807, 2023. 2
[36] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 6038 6088, 2023. ]
[37] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image genera- tion and editing with text- text-guided diffusion models. In International Conference on Machine
Learning, pages 16784-16804. PMLR, 2022. 2
[38] Maitreya Patel, Song Wen, Dimitris N Metaxas, and Yezhou Yang. Steering rectified flow models in the vector field for controlled image generation. arXiv preprint arX1v:2412.00100, 2024.
[39] William Peebles and Saining Xie. Scalable diffusion models with transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023. |
[40] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8766 8763. PMLR, 2021. 7
[41] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022. 22
[42] Litu Rout, Yujia Chen, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen- Sheng Chu. Beyond first-order tweedie: Solving inverse problems using latent diffusion. In 2024 IEEF/CVF Conference Computer Vision and Pattern Recognition, 2024. 2
[43] Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, and Wen- Sheng Chu. Semantic image inversion and editing using rectified stochastic differential equa- tions. In International Conference on Learning Representations, 2025.
24
[44] Jascha Sohl-Dickstein. Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pages 2256-2265 2015.
[45] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representutions, 2021.
[46] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representutions, 2021.
[47] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pages 11918-11930, 2019.
2
[48] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
13

---

[49] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[50] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven maage-to-image translation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921-1930, 2023. 2
[51] Belinda Tzen and Maxim Raginsky. Theoretical guarantees for sampling and inference in generative models with latent diffusions. In Conference on Learning Theory, pages 3084-3114 PMLR, 2019. 3
[52] Bram Wallace, Akash Gokul, and Nikhil Naik. Edict: Exact diffusion inversion via coupled transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22532 22532-22541, 2023. 2
[53] Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint arXiv:2411.0477 2024.
[54] Chen Henry Wu and Fernando De la Torre. A latent space of stochastic diffusion models for zero-shot image editing and guidance. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 7378-7387, 2023. 2
[55] Yilun Xu, Ziming Liu, Max Tegmark, and Tommi Jaakkola. Poisson flow generative models. In Advances in Neural Information Processing Systems, pages 16782-16795, 2022. 3
[56] Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training- free energy-guided conditional diffusion model. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 23174-23184, 2023.
[57] Richard Zhang, Phillip Isola, Alexei Efros, Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018. 7
14

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
We summarize our contributions correctly in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations as failure cases in the experiment section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
We have provided detailed proof of theoretical results in Appendix A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have disclosed all necessary settings in Appendices and
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We have released the link to our code in Appendix G
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https:/nnips.co
/togge/eedeeeuuaaaeaaae ssionE PalioooPolllyyy olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https) wttpcc///ppciicgiiieeeeoceeeeeee wtt.pp/cc/ouuuiic/////////////c/
//nips cc/
CooeeeooeeSSae ubmi is aessonPoolise
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental setting could be found in Appendix B Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The datasets we use are large, and conducting comprehensive statistical analysis over the entire datasets would be prohibitively time-consuming and computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the information of compute resources in Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
Our research follows the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts in Appendix G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We would include safeguards in our released data or models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have properly cited all relevant works and follow the public licenses.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paper thcode com dataaetttett has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: Our core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: http://eeeii..p..ee http://rrr.....ceoeeeerrrreeeeee http://cccccccoeeeeeeee/////ee/e 2025/ LLM) for what should should not described.
2025/
21

---

A Additional theoretical results: Proof of convergence for Kalman control iteration
We show that the error covariance sequence Ps: decreases at each iteration, ensuring convergence. According to the Kalman iteration Eq. the update rule is given by:
Pk [I (K
(14)
K P 5%=²
(15)
Sk = 1 HH + T.
(16)
Substituting Eq. (15) into Eq. (14) we obtain:
Pk = Px-1 - PR- HT S S HPPP--
(17)
Defining M Pk-1HTS) ST. HP we need to show M is positive semidefinite to conclude Pk P
Assuming Po positive definite and Pa-1 remains positive definite, and also that is positive semidefinite and T' is positive definite with proper initialization, we first show that S is positive definite. According to Eq. (16). since Pa-1 positive definite, - is positive semidefinite. As T is positive definite, their sum S& is also positive definite, implying S/ is positive definite.
To establish M ž 0, for any nonzero vector I, we compute
f' x² PA S HPh
(18)
Letting y H this simplifies to
T M ST'
(19)
Since Sx= is positive definite, S y 2 O, proving M 2 0. This guarantees Ps: 3 Pa- showing that the error covariance sequence decreases. If the system stable, Pk converges to steady-state value.
Table 6: Experiment hyperparameters.
Steps
Base model
CFG scale
Control strength
SDEdit
50
SD
P2P
50
SD 1.4
7.5
MasaCtrl
50
SD 1.4
7.5
DDPM-Inv
50
SD 1.4
RF-Edit
28
FLUX.1 1 dev
RF-Inversion
28
FLUX.1 1 dev
(0.7, 0.95)
FlowEdk Edit
28
FLUX.1 1 dev
(1.5,5.5)
FlowChet
28
FLUX.1 1 dev
Ours
28
FLUX.1 dev
3.5
0.95
B Additional details for experiment settings
First, we show the detailed hyperparameters setting for all bascline methods in Table As demon- strated, for diffusion-basse based editing methods SDEdit [34] and P2P [16). test both of them with 50 sampling steps and use StabeDiffusion 1.4 [41 as the base model. For all flow-based editing methods, we test all of them with 28 steps and use FLUX.1 dev 4 as base model. The timesteps are determined by Euler discrete scheduler. We also summarize the CFG scale and control strength the table. These CFG values follow the default settings recommended in each baseline's implementation.
Next, we explain the hyperparameter settings for our proposed method. We set steps to add Kalman filter to 14, which is half of the total steps. And we set the steps L. to be the later half steps of the generation (i.e., steps to 28). The hyperparameter 5 determining the two types measurement sequences is 6 by default. The coefficient hyperparameters μ and À are set to 0.7 and 0. by default.
22

---

We also set the matrices A, B, and H in Eq. (9) to be identity matrices, which is computationally efficient and has proven be effective in experiments. We also set the initial covariance matrix Po to be an identity matrix. The noise term is approximated by an identity matrix multiplied by small All of our
coefficient 0.1 or 0.01. The complete procedure of our algorithm is given in Algorithm 2 experiments are conducted on single NVIDIA A40 GPU.
Algorithm/ 2 Detailed procedure Kalman-Edi
Input: original image Zs, timesteps {t, Y source prompt ca. target prompt Ct. strength coefficients (A) i= and {u) step sets S1, S2 for adding controller, and L for adding Kalman filter (with [L] - l).
Init: Ttg f Jst M+ 0
* Phase Backward Denoising with Source Prompt
for i = N to 1 do
Vt, - (z,,, Ca) if t Si then
-1 t Dt, + (t/-1 - ti) (2 + (1- - A) 2)ee.)
else
31; 1 t Iτ, + (Li-1 - (2)
then
Add t; to M
Construct measurement sequence (vili
Forigl
Fto
1 Phase Forward Denoising with Target Prompt */
for í 0 to N do vt t- - VV( 4i,
if 1 S₁ then
T1i11 t I2, + (ti li+ Ave. + (1 - A) us,)
else
Itst f- Dt, + (t, 1+41) Ut.
Fmid
Fty
Middle latent
1t Phase 3: Backward Refinement with Controller S2 */ for i = to 1 do
Ut. Va( (i, cs) 14 S2 then
-t.- It, + (ti-1 - -ti) Ur, + (1 A) ves
else
It,-1 Fit + (ti-1 - t;) Uts
Torig2 +- Ito
>> Alternatively, one can use:
Torig2 + Forigl + (2mid - Iis)
** Phase 4: Forward Refinement with Target Prompt and Kalman Filter */
for i 0 to N do Ut. -- - VV 4i,
if t, S₂ then
Iti4 + It, + (t, (441) (vee + (1- 2) 11.s)
else 1 then
Thi Ft; + (li 1++1) uur; + - ko.))) heyj Kalman filter terms (see Eq. (12)
else
Z144 Fz, + (t, t++1) Ut.
Output: edited image In +- Biy
23

---

C Detailed Kalman-Edit algorithm
The detailed editing algorithm of our approach is presented in Algorithm 2 Phases and 2 correspond to the first stage, where we construct the measurement sequence, while Phases 3 and 4 make up the second stage, where Kalman control is applied. Concretely, in Phase I, we apply ODE inversion using rectified flow construct the first part of the measurement sequence, {y: 1 from early inversion latents and obtain Zorigt- In Phase 2, we build the second part of the measurement sequence, fu.)=s5 using generation latents from later timesteps and acquire Timid- We then compute Torig2 either via shortcut estimation or a second inversion. In Phase 4, the Kalman filter is applied to produce more accurate results that align with both the target prompi and the structural integrity of the original image. For hyperparameters, we define filter strengths {A} and {u],-0 control the guidance strength in the Kalman control process. We also specify step sets S1 and S2 to indicate where controllers should be applied, and step set L to identify where the Kalman filter should be used. All hyperparameters are set to default values but may require tuning for different editing tasks.
D Illustration of more capabilities: Style transfer, Scene editing and large area modification
We showcase additional capabilities of Kalman-Edit, including style transfer, complex scene editing, and large-area modifications. Our method effectively adapts images to various styles, edits intricate backgrounds, and modifies extensive regions within an image. For instance, in Fig. [ all four cases demonstrate our method's capability to edit complex background scenes while producing high-quality results. Figure 7 presents additional style transfer examples, such as converting a room into a medieval setting and performing large-area modifications, exemplified by the parrot case. In Fig. 8 we further highlight another large-area modification involving a giraffe. Moreover, Fig. 9 and Fig. IT illustrate various high-resolution style transfer cases, including transforming a picnic scene into a cartoon style and rendering a woman in the style of Van Gogh. Overall, these examples demonstrate the flexibility and effectiveness of our method in handling diverse and complex image editing tasks.
Fig. 10
E
Additional qualitative results
We present additional qualitative results, including real-world image edits and diverse editing tasks. We provide additional examples to further demonstrate the strong structural consistency and high editing quality achieved by our method. In Fig. 12 we present edited results on real-world images sampled from the DIV2K dataset. Our method effectively preserves the overall structure in non-target regions while accurately adhering to the editing prompts. For instance, in the boat insertion example, the foggy atmosphere is well preserved as a boat is naturally integrated into the scene. Other examples similarly exhibit high editing fidelity and strong structural consistency. Furthermore, Fig. [3] includes more diverse editing tasks, such as changing the breed of a dog. These results highlight the versatility and robustness of our approach in handling various types of image edits. In addition, Fig. 14] presents ablation studies on the effect of controller placement. The first row of images shows that applying controllers to early generation steps primarily influences the semantic structure, steering the output toward the original content (e.g.. generating cat) but missing background details. Conversely, the second row demonstrates that applying controllers to later steps refines visual details more effectively. These observations support our understanding of the controller's 's impact at different stages and help guide the selection of hyperparameters in Algorithm
F Discussion of controller-based methods
In this section, we briefly discuss controller-based approaches, which are grounded in optimal control theory. As noted in [43, such methods can be applied to both diffusion- based and flow-based generative models (i.e., they can operate on both stochastic differential equations (SDEs) and ordinary differential equations (ODEs)). With recent advances in flow-based models like Flux, there has been growing interest in applying control techniques specifically to ODE-based systems. From the perspective of optimal control theory, ODEs offer more favorable mathematical properties than SDEs, often enabling the derivation of more effective optimal controllers. For this reason, our proposed
24

---

Original Image
Edit
RF-Edit
Ours
Change the skatehourd to hike, change the scene to skatepark, and add muitiple riders porforming tricks in the background while muinkaining the overall structure of the picture
Change nity to a sandy overall structure
mony skyrevapers flying background while maintaining
Change city street enviromment, illuminated by bright lights while maintaining
picture
Change the child onimate style wearing grey hoodie and blue jeans, stuolyngg with books, and odd yellow chair and a teddy bear on the floor while maintaining the overall structure of the picture
Figure 6: Illustration of intricate scene editing. We present complex scene editing cases, such as adding and modifying multiple room elements in the last-row example, and altering background scenes from a city track sandy beach and from city street racetrack in the second-row example. Compared with baseline methods, our approach exhibits significantly stronger prompt adherence. These cases further demonstrate the robustness and versatility of our method in handling intricate editing tasks.
method adopts a flow-based framework. Nevertheless, similar to other controller approaches, method can also be extended to diffusion-based models. Kalman-Edit presents principled solution for achieving more accurate and consistent image editing through optimal control.
G Broader impact and limitations
Broader impact. While our Kalman-back based method advances the quality and consistency of image editing methods, such techniques should be treated with caution due to their increasing potential for malicious use. Noteworthy, our method is training-free and does not rely on any private datasets for evaluation, thereby posing data privacy concerns associated negative impacts. To facilitate
25

---

Original Image
FlowEdit
RF-Edit
Ours
Change to rustic wooden sable in modernal style hrckes with stone walls and amtique kuchorvare while maintaining everall struture the picture
on Flpypiion hieroghysise surral with gymbboly greem couch, a wonden cuffiee toble with save datrios and rod apple on it while maintzining the overall structure of the picture
Change snowy landecape clyff, sinucture the picture
while maintaining
Figure 7: Illustration of large-ares arrge-areaaoo modification. We present examples including room style transfor- mation (first-row case) and the addition of multiple new elements (second -ow row case). Furthermore, we demonstrate case involving substantial background modification, where forest scene is replaced with icy cliffs and polar bears (last-row case).
further open research into its practical uses and any potential societal impacts, our code would be open sourced at https://github.com http:://gthub.com/nnnnmoouu com/anonymous- 138384/Kalman- -dit---ttotthhh
Limitation. Due our design focus leveraging historical inversion latents as the measurement sequence to rectify the final generation process, the method requires additional hyperparameter tuning for tasks such object removal. We illustrate these failure cases in Fig. 15] In addition, the noise and artifacts from the original image tend to appear in the edited images at our default control strength.
26

---

Original Image
RE-Edi
Ours
Replace the clock with giraffe. Original Image
Original Image
Edit
Edt
Ours
RI--nveerson
Ri-Iaversion
Replace the chameleon with snowy owl.
Add parrot on the cat.
Figure 8: Illustration of large-area modification and complex scene editing. The giraffe example demonstrates our method's ability to preserve image structure, as evidenced by the chandelier remaining correctly positioned. The owl example features a complex background, and our result retains most structural details in the non-target regions. In the cat example, our method maintains strong structural consistency while accurately adhering to the target prompt.
27

---

The Faurrealiom styee wolf wcaring armor and has crown on his head
[Cartoon style) pienic table with bottle of wine and fruit it
Figure 9: Additional qualitative results on the style transfer task. We present examples of style transfer in surrealism and cartoon styles.
28

---

freadssm stylej and bunay
[Cartron off
Evan gogh atylle
woman in
white sitting
on bench
Figure 10: Additional qualitative results on the style transfer task. We present examples of style transfer in realism and van gogh styles.
29

---

cup with flowers while maintain the overall structure the picture
and tok sketch off wind turbine the sumset
Additional qualitative results on the style transfer task. We present an example of style transfer to a pen-and-ink style, and additionally include bathroom editing case.
Figure 11:
30

---

Bouprks
bard HOWs DrOOAR
Add
Add
Panda
Racoom
Add aurara
Add Aulloon
Wol
Rodk ---- Branch
Additional results on real-world image editing. We present examples involving complex scenes such as urban environments and beaches, as well as images with intricate structures like the boat and the panda. These results highlight the flexibility and effectiveness of our method in handling diverse and challenging editing tasks.
Figure 12:
31

---

Add
glasses
glasses
Change
Black dog to
Change
Roses
to
tulips
Husky dog
Add
Add
bread
chairs
BA
Add
Add
A
dog
stick
Change
Child to Albert Einstein
Figure 13: Additional results on diverse image editing tasks. We present more examples with a wide range of target prompts, such as changing dog's breed and transforming roses into tulips. These results further demonstrate the strong performance and versatility of our approach.
32

---

Step 0-6
Step 0-8
Step 0~10
Step 0-12
Step 15-20
Step 15-22
Step 15-24
Step 15-27
Figure 14: Additional ablation results on controller placement. We evaluate the effects of adding controllers at different stages of the generation process. These insights inform the hyperparameter choices in our proposed algorithm. For detailed analysis, please refer Appendix E
Original Image
Edit with large strength
Edit with small strength
Remove the boat
Change the bike to a Vespa
Figure 15: Limitation of method. Improper hyperparameter settings may cause our method fail in some editing cases, such as certain object removal tasks. See Appendix G for detailed explanation.
33


Reviews:
[{"review_id": "4Ws45X4tmX", "reviewer": "Reviewer_7uLu", "summary": "This paper formulates image editing as an optimal control problem, aiming to satisfy the target condition while minimizing editing cost. It introduces the classical Kalman filter into the image editing pipeline to better preserve details from the original image. The method is seamlessly integrated into the standard forward and inverse processes of diffusion models. Additionally, the authors propose a shortcut technique based on approximate vector field velocity estimation, which enhances editing quality without incurring significant computational overhead.", "questions": "How sensitive are the final results to different initializations or hyperparameter settings of the Kalman filter?", "limitations": "Authors discuss the limitations in the final chapters and this paper has no potential negative societal impact.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n(1) The paper presents a compelling perspective by framing image editing as a control problem and effectively incorporates the classical Kalman filter to demonstrate tangible improvements.\n\n(2) The motivation is well-articulated, and the proposed method achieves significant performance gains.\n\n(3) The approach is firmly grounded in optimal control theory and is supported by a formal convergence proof of the Kalman filtering process.\n\n\nWeakness:\n(1) The method is sensitive to hyperparameters such as filter strength and step selection, particularly in tasks like object removal, where improper settings may introduce artifacts.\n\n(2) The accelerated variant, Kalman-Edit*, relies on the assumption of linear behavior in latent space, which may not hold in more complex or nonlinear editing scenarios.", "was_revised": true, "final_justification": "After reading comments from authors and other reviewers, I maintain my rating for acceptance. I believe the value of this paper lies in connecting a theory-grounded control literature to the task of image editing. I agree with bsYB that this paper should be improved with better visual results and more quantitative evaluation. Hoping the authors could improve on those in the final version.", "rebuttal": "Thank you for your valuable feedback. Below, we provide our detailed responses to the concerns you raised.\n\n[W1/Q] The sensitivity of our method to parameters\n- The term \"sensitive\" in our method refers to the introduction of additional Kalman control parameters compared to previous controller-based approaches. In practice, as long as a few key parameters, such as the added timestep and filter strength, are properly set, the overall editing performance remains strong. \n- To better show the sensitivity of our method to parameters, we conduct additional experiments. As shown in the table below, the previous controller-based method, RF-Inversion, even shows more sensitivity to strength and added steps. As these two parameters vary, the values produced by RF-Inversion show greater fluctuations compared to those generated by our approach. This suggests that the sensitivity of our method to parameter changes is lower than that of RF-Inversion.\n\n**CLIP-I Score Ablation Comparison on Zone Dataset**\n| Method                 | Filter or Controller Strength / Added Steps | 15-18  | 15-22  | 15-27      |\n| ---------------------- | ----------------------------- | ------ | ------ | ---------- |\n| **Kalman-Edit**  | **0.1**                       | 0.8770 | 0.9219 | **0.9346** |\n| **Kalman-Edit**  | **0.2**                       | 0.9043 | 0.9282 | 0.9226     |\n| **Kalman-Edit**  | **0.3**                       | 0.9014 | 0.8921 | 0.9079     |\n| **RF-Inversion** | **0.1**                       | 0.8613 | 0.9221 | **0.9311** |\n| **RF-Inversion** | **0.2**                       | 0.8921 | 0.9191 | 0.9259     |\n| **RF-Inversion** | **0.3**                       | 0.9174 | 0.8813 | 0.9063     |\n\n[W2] Difficulty of Kalman-Edit* handling complex editing scenarios\n- Kalman-Edit* indeed may not become practical in some editing scenarios. But it turns out to be efficient in most editing tasks, such as style transfer and scene editing (Please see the Appendix for more visualization results)."}, {"review_id": "Tuw3HhRytj", "reviewer": "Reviewer_YwVg", "summary": "This paper introduces a new method to enhance the consistency and relevance of image editing problem. The proposed method exploits Kalman control concept in the flow-matching equation and optimize the image editing into 2 stages. It starts with a inversion process to get the noise from the input image and then followed by a Generation process to produce the target image. The proposed method also provide a faster version to skip the second inversion process with approximation to accelerate the image generation throughput. Extensive experiments on qualitative and quantitative evaluation demonstrate the effectiveness of the proposed method.", "questions": "This paper proposes an efficient and performing method to control the feature editing using Kalman-control with strong solid formulation and reasonable assumptions. The authors are suggested to address minor issues raised in the weakness during rebuttal period.", "limitations": "yes, with failure cases and limitation in supplementary material.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strength\n1. The qualitative experiments show that the proposed method is really efficient and performing. It keeps the identity of the original person very well. \n2. The proposed method comes with a verifiable code implementation for results reproducing. \n3. The proposed method is technically sound with equation derivation and proofs. \n4. Extensive experiments show that the proposed method outperforms the baseline state of the art methods significantly and consistently on the benchmarking datasets.\n\n### Weakness\n1. What is the reason that Kalman-Edit* shortcut has significant higher performance on CLIP-T compared to Kalman-Edit in Table 3 ?", "was_revised": true, "final_justification": "I would like to keep my initial ratings as accept because this paper proposes a novel theory to do flow matching using kalman filtering and comes up with good experiment results to demonstrate the effectiveness of the theory. The rebuttal messages and additional experiment results have addressed the previous concerns.", "rebuttal": "We sincerely appreciate your valuable feedback. Our response is as follows.\n\n[W/Q] Reason of Kalman-Edit* owning higher CLIP-T performance\n\n- Kalman-Edit* achieves significantly higher performance on CLIP-T in Table 3, primarily because the evaluation datasets, Zone and DIV2K, are dominated by editing tasks involving local modifications like object addition or replacement. Though the second-pass inversion utilizes more structural information, it is more constrained and lacks flexibility when tackling small area modifications. \n- By contrast, the shortcut estimation in Kalman-Edit* introduces greater flexibility in adhering to the target prompt as it provides a looser estimation method that utilizes a first-order approximation in the vector field. When handling the local modifications task, this method would yield a relatively small estimation error. To verify this point, we conduct an additional quantitative experiment showing the local area similarity. As shown in the table below, our approach achieves superior performance across all evaluation metrics, showing that our approach has relatively minor errors in estimating the local area in different images. This leads to better target prompt adherence, as many prompts in the evaluation datasets contain local area descriptions. As a result, Kalman-Edit* demonstrates superior performance on the CLIP-T metric.\n\n**Quantitative evaluation of local area similarity on Zone dataset**\n| Method            | DreamSim ↓ | CLIP-I ↑   | LPIPS ↓    |\n| ----------------- | ---------- | ---------- | ---------- | \n| **RF-Inversion**   | 0.2172     | 0.8558     | 0.1618     |\n| **RF-Edit**       | 0.1743     | 0.8831     | 0.1473     |\n| **Kalman-Edit**\\* | **0.1508** | **0.9012** | **0.1322** |"}, {"review_id": "S7OMDbodzh", "reviewer": "Reviewer_bsYB", "summary": "This paper formulates the image editing task with *Rectified flow* model as a control problem and proposes *Kalman-Edit*, which uses *Kalman filter* to leverage the historical editing trajectory. For enhancing image consistency during the editing process, *Kalman-Edit* reuses early-stage representation from the overall sampling trajectory. Through this work, the authors aim to address the issues of structural consistency and editing quality inherent in previous approaches. The authors conducted extensive experiments on four datasets—SFHQ, HQ, ZONE, and DIV2K—to evaluate the performance of the proposed method.", "questions": "Q1: In Section 3.3, the “two-stage image editing” approach is introduced, but no ablation study is provided for it. Why was an ablation analysis for this component omitted, and why is there no ablation on the number of stages?\n\nQ2: In the RF-Edit, video editing is demonstrated; could the same method be applied to video editing tasks as well?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The authors clearly identified the problems using rectified flow models for editing and articulated the motivation with clarity.\n2. This paper establishes a link between image editing and control theory, introducing a novel approach that leverages the Kalman filter.\n3. Extensive experiments were conducted across multiple datasets, providing the evaluation of the proposed approach’s performance.\n\n**Weaknesses**\n1. The qualitative results lack a sufficient set of baseline comparison models. In addition to *RF-Edit* and *RF-Inversion*, visualizations comparing against *SDEdit*, *P2P*, *MasaCtrl*, and *DDPM-Inv*—which the authors referenced in their quantitative results—are also necessary. Additionally, visual results for *Kalman-Edit*, one of the paper’s contributions, are needed.\n2. The baseline comparison experiments focus solely on changing or adding specific objects; additional comparisons on image stylization are necessary.\n3. Overall, the qualitative results are not good. In *Figure 2*, the reported edits involve tiny objects or mere texture changes, and the output quality appears poor (e.g., plant → flower, wall → brick walls). In terms of consistency, RF-Inversion preserves the original image far better in *Figure 3*. Moreover, in *Figure 5* cartoon-style transformation, a teddy bear is added unexpectedly; in *Figure 6* snow-covered tree addition, the image fails to reflect the change properly; in the two-dimensional Egyptian style transformation, there is no proper alteration; and cup addition in *Figure 9*, the image does not show the cup effectively—suggesting that the proposed method’s performance is not good and robust.\n4. The images in the qualitative results are presented at low resolution, making it difficult to tell which model performs better, even when zoomed in. Uploading higher-resolution images would help readers understand the performance of the proposed method more effectively.", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thoughtful suggestions and respectfully address your primary concerns below:\n\n[W1/W2] Lacking diffusion-based qualitative comparison and image stylization results\n- We will add an additional visual comparison between diffusion-based methods and ours. Since we can not upload the visualization result here, we conduct an additional quantitative experiment on the PIE benchmark [1] to show the effectiveness of our method in the image stylization task. Our method owns superior performance on the image stylization task across different metrics, compared to both diffusion-based and flow-based methods.\n\n[W3] Concern about editing quality\n- For Figure 2, our approach achieves good editing quality and maintains strong structural consistency. To verify this, we conduct an additional experiment to evaluate the local area similarity. As shown in the table below, our approach achieves the best similarity results across all evaluation metrics. This demonstrates the strong structural consistency of our method. Some of the changes in Figure 2 may appear minor because the prompt specifically targets modifications in a small local area. To better illustrate the results and demonstrate the effectiveness of our method, we will add more high-resolution image results to the Appendix.\n- For Figure 3, we would like to highlight that our approach preserves more face structural consistency and has higher face similarity. We qualitatively evaluate the average Face Rec metric of the images in Figure 3. Our results have 0.4021 average Face Rec, which is lower than RF-Inversion with 0.4213 average Face Rec. This shows that our method preserves better facial consistency.\n- For Figures 5 and 6, all the images are from the HQ dataset and edited with long and complex prompts. In both Figures, we only sketch the main editing area, which does not reflect the whole target prompt. We are sorry about the confusion. For example, the teddy bear case is edited with the target prompt “A teenager in a grey hoodie and blue jeans is sitting on a yellow chair at a wooden desk studying from a book and using a laptop. The room has blue walls, a shelf with books and a clock, a large framed space poster on the wall, and a teddy bear on the floor next to a backpack and scattered school supplies.” Therefore, the teddy bear appears in the target prompt and should be included in the edited image as expected. We will add the whole target prompts of each editing case to the Appendix. \n- For Figure 9, most cases can reflect the effectiveness of our approach. Regarding the case of the mentioned cup, since the editing area is very small and the image is in low resolution, it does not show up effectively. We sincerely thank you for the reminder and would add high-resolution cases here.\n\n[W4] Lacking high-resolution images\n- We will add higher-resolution images to our paper. Thank you for your valuable advice.\n\n[Q1] Lacking ablation study results\n- We have conducted the ablation study between the previous one-stage controller-based method (i.e. w./o. Kalman filter) and our two-stage approach (i.e. w./ Kalman filter) in our paper. The ablation results demonstrate the effectiveness of our approach. Please refer to Table 5 for more details and Section 4.2 for more analysis.\n\n[Q2] Concern about extension to video editing\n- Our work focuses on the image editing task. The effectiveness of our approach for video editing remains to be explored. Compared to RF-Edit, our method does not require the modification of attention values and is expected to have better video editing flexibility and quality. We would try to extend our method to video editing in future work.\n\n**Quantitative comparison on the PIE benchmark image stylization task**\n| Method            | CLIP-T ↑   | DreamSim ↓ | CLIP-I ↑   | LPIPS ↓    |\n| ----------------- | ---------- | ---------- | ---------- | ---------- |\n| **SDEdit**           | 0.2842     | 0.1124     | 0.9230     | 0.2575     |\n| **P2P**    | 0.2988     | 0.1010     | 0.9375     | 0.1691     |\n| **MasaCtrl**       | 0.3041     | 0.1504     | 0.9150     | 0.1607     |\n| **DDPM-Inv**       | 0.2913     | 0.1103     | 0.9204     | 0.1461     |\n| **RF-Edit**       | 0.2845     | 0.1212     | 0.9271     | 0.1592     |\n| **FlowEdit**      | 0.2876     | 0.1686     | 0.8802     | 0.1623     |\n| **Kalman-Edit**   | 0.2903     | **0.0912** | **0.9451** | **0.1402** |\n| **Kalman-Edit**\\* | **0.3135** | 0.1347     | 0.9381     | 0.1652     |\n\n**Quantitative evaluation of local area similarity in Figure 2**\n| Method            | DreamSim ↓ | CLIP-I ↑   | LPIPS ↓    |\n| ----------------- | ---------- | ---------- | ---------- | \n| **RF-Inversion**   | 0.3322     | 0.8179     | 0.2128     |\n| **RF-Edit**       | 0.2646     | 0.8467     | 0.1546     |\n| **Kalman-Edit** | **0.2318** | **0.8640** | **0.1127** |\n\n\n---\n[1] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. (2024). PnP Inversion: Boosting Diffusion-based Editing with 3 Lines of Code. ICLR 2024."}, {"review_id": "LUXbdYlnxS", "reviewer": "Reviewer_ndBN", "summary": "Flow-based image editing suffers from structural inconsistencies in non-target areas, due to the accumulation of small errors. To address this, the editing process is formulated as an LQG control problem, and Kalman-Edit is introduced, which integrates a Kalman filter to exploit historical inversion trajectories for preserving fine details. Additionally, it proposes a faster variant, Kalman-Edit*, that approximates the second inversion step to reduce computation while incurring only minimal quality loss.", "questions": "- Why does the proposed method require a two-stage approach? Section 3.3 (lines 146–153) offers some explanation, but it remains unclear why a single-stage Kalman control fails while a two-stage approach succeeds. First, selecting appropriate timesteps for Kalman filtering is inherently challenging, and the proposed method still relies on selected ranges of timesteps. Second, the authors claim that applying Kalman control in one inversion and forward pass causes blurring and artifacts, yet they do not explain why the filter in stage 2 avoids these issues. An ablation study directly comparing a single-stage Kalman control setup with the two-stage pipeline would help clarify this choice.\n\n- Unlike other methods, the proposed approach requires two generation passes, even with the approximate second inversion in Kalman-Edit*. Therefore, comparison of inference times against competing techniques is necessary.\n\n- Tables 1 and 2 only include flow-based baselines, and the qualitative results omit any diffusion-based methods. Since diffusion models are inherently slower, demonstrating comparable performance between the proposed method and diffusion-based approaches would underscore the efficiency of the proposed framework.\n\n- What inversion method do P2P and MasaCtrl use? DDIM inversion yields poor reconstruction quality when editing real images. Both could potentially benefit from more advanced inversion schemes, such as Direct Inversion [1], NMG [2], or SPDInv [3]. It would strengthen the evaluation to compare against these advanced methods.\n  \n*Minor Suggestions*\n- Provide additional implementation details: how are editing prompts formulated for the DIV2K dataset, and which timesteps are used for S1 and S2 in Algorithm 2?  \n\n\n[1] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. Direct inversion: Boosting diffusion-based editing with 3 lines of code. ICLR 2024.  \n[2] Cho, H., Lee, J., Kim, S. B., Oh, T. H., & Jeong, Y. (2024). Noise map guidance: Inversion with spatial context for real image editing. ICLR 2024.  \n[3] Li, R., Li, R., Guo, S., & Zhang, L. (2024, September). Source prompt disentangled inversion for boosting image editability with diffusion models. ECCV 2024.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**[Strengths]**\n- Reformulates the editing process as an LQG control problem and integrates a Kalman filter to improve structural consistency.\n- Proposes an accelerated variant, Kalman-Edit*, which omits a second inversion to cut computation time while maintaining high-quality results.\n\n**[Weaknesses]**\n- The motivation for the two-stage design is not thoroughly explained.\n- Evaluation is limited, relying mainly on comparisons with flow-based editing baselines.", "was_revised": true, "final_justification": "The author's response resolve my concern, so I raise my score accordingly", "rebuttal": "We greatly appreciate your constructive comments. Below, we provide our responses to the concerns raised.\n\n[W1] Motivation for the two-stage design\n- This design is motivated by the observation that previous controller-based methods have difficulty preserving editing consistency. And we find that harnessing history trajectory could mitigate this issue, which naturally aligns with Kalman control. Therefore, we propose the two-stage method: stage 1 corresponds to the controller-based method that collects the history trajectory, while stage 2 corresponds to Kalman control utilizing that history trajectory. The two-stage approach better utilizes the structural information in the trajectory and enhances the editing consistency.\n \n[Q1] Necessity of the two-stage design\n- The necessity of the two-stage design arises from the editing dilemma as presented in Figure 1. Figure 1.a shows that previous controller-based methods tend to reach under-editing or over-editing area and lead to editing failure. With the two-stage approach, we can better utilize the structural information in the history trajectory with the help of the Kalman filter and succeed in editing, as shown in Figure 1.b. Moreover, we have conducted the ablation study comparing vanilla controller-based methods and our method with Kalman filter added.  Kalman filter significantly improves the performance of the controller-based method, demonstrating the effectiveness of our two-stage design. Please refer to Table 5 in the Experiment section for more details.\n\n[Q2] Time cost comparison\n- We have presented the running time comparison of different methods. Our proposed Kalman-Edit* is slower than RF-Inversion and Flowchef, but faster than all other baselines. This shows that our approach strikes a competitive balance between editing performance and efficiency. Please refer to Figure 4 in the paper for more details.\n\n[W2/Q3/Q4] Quantitative comparison with more recent diffusion-based methods\n- We conduct additional experiments on the three listed diffusion-based methods. The results are shown in the two tables below. Our method still shows better performance across most metrics on SFHQ and HQ datasets compared to more advanced inversion schemes Direct Inversion [1], NMG [2], and SPDInv [3]. This shows the effectiveness of our approach.\n\n**Quantitative comparison on the SFHQ dataset**\n| Method            | Face Rec. ↓ | CLIP-I ↑   | LPIPS ↓    | CLIP-T ↑   | DreamSim ↓ |\n| ----------------- | ----------- | ---------- | ---------- | ---------- | ---------- |\n| **NMG**           | 0.4437      | 0.8264     | 0.1906     | 0.2820     | 0.1825     |\n| **Direct-Inv**    | 0.4276      | 0.8350     | 0.1703     | 0.2686     | 0.1535     |\n| **SPD-Inv**       | 0.4625      | 0.8561     | 0.1618     | 0.2646     | 0.1480     |\n| **RF-Edit**       | 0.4051      | 0.8984     | 0.1562     | 0.2910     | 0.1591     |\n| **RF-Inversion**  | 0.4325      | 0.8927     | 0.1720     | **0.3012** | 0.1889     |\n| **FlowEdit**      | 0.4856      | 0.8579     | 0.1687     | 0.2905     | 0.2375     |\n| **FlowChef**      | 0.4013      | 0.8769     | 0.1401     | 0.2832     | 0.1487     |\n| **Kalman-Edit**   | **0.3958**  | **0.9167** | **0.1332** | 0.2921     | **0.1408** |\n| **Kalman-Edit**\\* | 0.4696      | 0.8871     | 0.1892     | 0.2936     | 0.2227     |\n\n**Quantitative comparison on the HQ dataset**\n| Method            | CLIP-T ↑   | CLIP-I ↑   | LPIPS ↓    | DINO ↑     | DreamSim ↓ |\n| ----------------- | ---------- | ---------- | ---------- | ---------- | ---------- |\n| **NMG**           | 0.1879     | 0.8916     | 0.2826     | 0.8310     | 0.1672     |\n| **Direct-Inv**    | 0.1676     | 0.9153     | 0.2866     | 0.8188     | 0.1834     |\n| **SPD-Inv**       | 0.1864     | **0.9221** | 0.2756     | **0.8237** | 0.1548     |\n| **RF-Edit**       | 0.1842     | 0.9141     | 0.2383     | 0.8197     | 0.1492     |\n| **RF-Inversion**  | 0.1825     | 0.9033     | 0.3074     | 0.7963     | 0.1662     |\n| **FlowEdit**      | 0.1877     | 0.8813     | 0.2846     | 0.7467     | 0.2238     |\n| **FlowChef**      | 0.1928     | 0.9023     | 0.2925     | 0.8053     | 0.1537     |\n| **Kalman-Edit**   | **0.1943** | 0.9062     | **0.2345** | 0.7929     | **0.1353** |\n| **Kalman-Edit**\\* | 0.1870     | 0.8696     | 0.3615     | 0.7123     | 0.2276     |\n\n---\n[1] Ju, X., Zeng, A., Bian, Y., Liu, S., & Xu, Q. Direct inversion: Boosting diffusion-based editing with 3 lines of code. ICLR 2024.\n\n[2] Cho, H., Lee, J., Kim, S. B., Oh, T. H., & Jeong, Y. (2024). Noise map guidance: Inversion with spatial context for real image editing. ICLR 2024.\n\n[3] Li, R., Li, R., Guo, S., & Zhang, L. (2024, September). Source prompt disentangled inversion for boosting image editability with diffusion models. ECCV 2024."}]

Author rebuttal:

Meta-review: This paper proposes Kalman-Edit, framing rectified flow editing as a control problem and applying Kalman filtering to improve structural consistency. The connection to control theory in image editing is novel and well-motivated. Results across several datasets, with added ablations and baselines in rebuttal, show the method is effective.

The main weakness is qualitative: edits are often small, figures are low resolution, and comparisons are limited. While the rebuttal helped, the paper would benefit from clearer visual results and stronger baselines.

Overall, the contribution is solid and the method promising. The AC agrees with the majority of reviewers and recommends acceptance. Authors are encouraged to improve the qualitative results, provide higher-resolution figures, and include broader baseline comparisons in the camera-ready version.
Decision: Accept (poster)