Training-Free Reward-Guided Image Editing via Trajectory Optimal Control
Authors: Jinho Chang
Jaemin Kim
Jong Chul Ye
Venue: ICLR 2026 Poster (2026)
Keywords: reward-guided editing
diffusion models
optimal control
Primary area: generative models
TL;DR: We propose a novel image editing method towards a given reward, using the solution of reward-optimal control problem.

Abstract: Recent advancements in diffusion and flow-matching models have demonstrated remarkable capabilities in high-fidelity image synthesis. A prominent line of research involves reward-guided guidance, which steers the generation process during inference to align with specific objectives. However, leveraging this reward-guided approach to the task of image editing, which requires preserving the semantic content of the source image while enhancing a target reward, is largely unexplored.
In this work, we introduce a novel framework for training-free, reward-guided image editing. We formulate the editing process as a trajectory optimal control problem where the reverse process of a diffusion model is treated as a controllable trajectory originating from the source image, and the adjoint states are iteratively updated to steer the editing process. Through extensive experiments across distinct editing tasks, we demonstrate that our approach significantly outperforms existing inversion-based training-free guidance baselines, achieving a superior balance between reward maximization and fidelity to the source image without reward hacking.

Published as a conference paper at ICLR 2026
TRAINING-FREE REWARD-GUIDED IMAGE EDITING VIA TRAJECTORY OPTIMAL CONTROL
Jinho Chang' Jaemin Kim Jong Chul Ye
Graduate School of Artificial Intelligence
Korea Advanced Institute of Science and Technology
Seoul, South Korea
{jinhojsk51m k jjm 98 981 ,95, jong. jong.ye)ekaist.com ac.kr
Human Pr Preference
Ibl Style Transter
imageResserd
Counterfactual Generation
Idi Text guided Editing
306"
Rlwer
Reward-guided image editing samples with unconditional diffusion and flow- matching models. Reward-guided edited samples various tasks, such (a) Human pref- erence, (b) Style transfer, (c) Counterfactual generation, and (d) Text-guided image editing.
Figure 1:
ABSTRACT
Recent advancements in diffusion and now-matching models have demonstrated remarkable capabilities in high-fidelity image synthesis. A prominent line of re- search involves reward-guided guidance, which steers the generation process dur- ing inference to align with specific objectives. However, leveraging this reward- guided approach to the task of image editing, which requires preserving the se- mantic content of the source image while enhancing target reward, is largely un- explored. In this work, we introduce novel framework training-free, reward- guided image editing. We formulate the editing process control problem where the reverse process of a diffusion model is treated as a controllable trajectory originating from the source image, and the adjoint states are iteratively updated to steer the editing process. Through extensive experiments across distinct editing tasks, demonstrate that approach significantly out- performs existing inversion-based training-free guidance baselines, achieving a superior balance between reward maximization and fidelity to the source image without reward hacking.
trajectory optimal
INTRODUCTION
Following the advancement of diffusion and flow-matching models that led to remarkable success high-fidelity image synthesis (Ho et al., 2020; Dhariwal Nichol, 202T Lipman et al. 2022). various methods have been developed to edit real-world images (Meng et al. 2021 Hertz et al. 2023) by leveraging their pre-trained image priors. However, most editing techniques remain limited concepts that exist within the model's pre-trained distribution (i.e., applying "Van Gogh style" is only possible if the model has been trained in such styles). While text-to-image models (Rombach al. 2022 Esser et al. 2024) provide diverse conditional distributions, abstract human preferences or subtle stylistic nuances are often difficult to specify clearly using natural language.
"Equal contribution this work.

---

Published as a conference paper at ICLR 2026
Iterative Trajectory Optimization
Update Adjaitt State
Optimized Trajectory (xf h
Update Control
Source image
Update Trajectory (X!T)wr
Trajectory
Inital Trajectory (ReleuT
Dnsto soalel
Reward gewarddguided Edited Image xi"
Inital Trajectory Lx,].ur
Figure 2: Methodology overview. Given a source image 21, our method first generates its corre- sponding initial trajectory. We then progressively refine this trajectory by solving a reward-guide optimal control problem. This process steers the path into an optimized trajectory, whose endpoint is the final edited image æ"
Meanwhile, reward-guided sampling methods have been proposed a promising, training- framework that operates during inference (Chung et al. 2023; Yu et al. 2023 Song et al. 2023 Ye et al.) 2024, Geng & Owens 2024), which leverages off off-the-shelf differentiable reward functions to steer the generation process toward a desired objective. The primary advantage of this approach its ability to generate images toward novel target distribution defined by the reward function, moving beyond the original sample distribution.
Despite the progress of gwwrdd-guiedd image generation, its potential for image editing techniques has been under-explored and there is room for improvement. Reward-guide editing is more chal- lenging, as requires both maximizing reward and preserving the core identity of the source image. The most intuitive approach is to first invert the source image into the noise space and then apply a reward- -uidee generation algorithm during the reverse process. Unfortunately, this method often fails because most guidance techniques rely the reward gradient to the intermediate noised image or one-step approximation
the clean image, but for complex and non-linear reward functions, this indirect guidance degrades the structural faithfulness of the source image (Chung al. 2023 Yu Ye et al. 2024)..
et al. 2023
To address these challenges, here we propose training-free framework by reformulating reward- guided image editing as trajectory optimal control problem (Figure 2. Specifically, we treat the reverse diffusion process, originating from the source image, as controllable trajectory. Our goal is then to find the optimal control signal that steers this entire trajectory to terminal state that maximizes the reward. To solve this control problem, algorithm based on the principles of Pontryagin's 's Maximum Principle (PMP) (Levine) 1972), We comprehensively demonstrate the effectiveness of our approach across four distinct editing tasks (Figure []. By optimizing the entire path, our approach shows that the resulting edits are not only effective terms of the target reward but also structurally coherent with the source image. Our main contributions are threefold:
develop an iterative adjoint-state update
We present training-free reward-guided image editing framework by formulating it trajectory optimal control problem, applicable to both diffusion and flow- cow-mathing models.
Based on the PMP necessary conditions, we develop an iterative adjoint-state optimization procedure to find the optimal trajectory that maximizes the target reward.
Through extensive experiments across diverse tasks, we demonstrate that our method out- performs existing inversion-basee guidance baselines, achieving superior results without reward hacking or structural degradation.
2 RELATED WORKS
2.1 TRAINING-FREE IMAGE EDITING WITH DIFFUSION AND LLOW-MATCHING MODELS
The exploitation of the pre- mre-trained distribution from pre-trained models enabled various techniques for the image editing task. One of the most popular approaches is inversion based methods (Meng
al. 2021 Mokady et al. 2023 Lluberman-Spiegelgee et al. 2024), which map source image
2

---

Published as a conference paper at ICLR 2026
to noise space through the forward process and then edit it through a modified reverse trajectory. Another direction employs distillation- based optimization (Poole et al.
2023 Hertz et al., 2023
Nam et al., 2024), which guides the source image without explicit sampling step. Some used empirical feature alignment, such as cross-attention map (Hertz et al. sure the sampling output retains the source image feature. More recently, such as FlowEdit (Kulikov et al.) 2024) achieve optmizzttion-fie editing by directly steering text- conditional flows. Leveraging the semantic coverage of large-scale text-to-image models like Stable Diffusion (Rombach et al., 2022 Esser et al.) 2024), editing is typically specified with natural lan- guage prompts. However, these approaches are restricted by the scope of the model's 's pre-trained distribution, making it difficult to edit beyond the concepts it has trained.
2022 Cao et al. 2023), to en-
approaches
2.2 REWARD-GUIDED IMAGE GENERATION
Recently, modifying the generative process to align with user-defined objective, often encapsulated by reward function, is central goal in controllable generation. While this can be done by explicit training for the reward-aligned distribution (Black et al.] 2023) Wallace et al. 22024) several works aim apply training-free guidance that steers the sampling process (Chung al. 2023 Ye al.) 2024 Yu et al. 2023 He et al. 2023: Song et al. 2023). Leveraging off-the-shelf differentiable pre- dictors (e.g., a classifier or reward model), these approaches modify the denoising samples or their posterior mean during inference to achieve higher reward at the end of the sampling. Nevertheless, their potential for editing has been underexplored since these methods were fundamentally designed for sampling from the noise distribution.
2.3 STEERING GENERATIVE MODELS WITH OPTIMAL CONTROL
Leveraging the iterative sampling process of diffusion and flow-machhing models, recent works (Rout et al. 2024 2025 Zhu et al., 2025) have employed optimal control perspectives to modify the sampling trajectory to satisfy certain desired properties such as style personalization, generalized Doob's h-trransfomm and inversion proximal to the given endpoint. For reward-alignment model training, Adjoint Matching (Domingo-Enrich et al.) 2025) formulated a Stochastic Optimal Control (SOC) problem, where the goal is to maximize the terminal reward while regularizing the control term. While optimal control has been successfully applied to model fine-tuning and sam- pling, its application to the training-free editing task has been relatively under-explored. Our work is to adapt the principles of the trajectory optimal control problem for editing a given source image, steering its sampling trajectory towards target reward without model updates.
PRELIMINARIES
3.1 DIFFUSION AND LLOW-MATCHING MODELS
Diffusion Models. Diffusion models (Dhariwal & Nichol
2021 1 Song et al. 202 are a class of
generative models trained to reverse the predefined forward process that gradually injects Gaussian
noise into clean data Ij over time interval [0, 1 The diffusion model is trained by
denoising score matching (DSM) objective (Vincent 2011 Ho et al. 2020) to predict the injected
noise from the perturbed sample x; (( az T) where { f is set of parameters to
control the noise level. The reverse sampling to generate ar+dtt from 2, using the following,
aepat - of o,E, <~ (0, (1)
Va.
where o, controls the stochasticity (Song al. 202
Esser et al., 2024) define their sampling processes through interpolating between known prior and the target data distribution. The sampling process is typically governed by an Ordinary Differrntial Equation (ODE) over the time interval [0, 1]:
Flow-matching Models. Flow-matching models (Lipman et al., 2022 Liu et al. 2022
dx; = (x t)dt,
Cp (0,
(2)
Instead of the notation typically used in diffusion models, employ the notation used in flow-matching models, where the timestep spans from 0 (noise) to (data) with evenly spaced interval.
3

---

Published as a conference paper at ICLR 2026
The parameterized velocity field (z) is trained to approximate the marginal derivative of pre-defined reference flow across the training data, typically the form I; = Br + a, with (a,, (b)) satisfying boundary conditions ap 81 0 and a Po 1. The most common setting lets 0g = and 3 = 1 t. This training objective ensures that the solution of the sampling ODE has the same marginal distributions the reference flow, thereby guaranteeing that 21 follows the target data distribution.
Unified SDE Framework. Although diffusion and flow-matching model originates from different theoretical foundations, their sampling processes can be unified with a Stochastic Differential Equa- tion (SDE). Leveraging the SDE perspective of the diffusion reverse process (Song et al. 2021b) and the Fokker-Planchs equation, the sampling dynamics for both models can be expressed as:
dæ; b( t)dt + o,dB;, 20 (O,,,,,
where b(z,, t) is the drift term, o, is an arbitrary time-dependent diffusion coefficient, and dB, is Brownian motion. With the diffusion model scheduler {ā=)== | and flow-matching model setting of Og = and Br = - t, the drift term can be further specified as (Domingo Enrich et al. (2025):
0tt
o
en (1x. 1) aa
UDitffusion (It, t) 3t -
(4)
20,
2
vI
to
1
Mloo-Matching (Frt, 1) (11,1 1)
10 1) -
(5)
2(1
where å; denotes dā Under this framework, diffusion models correspond to particular choices of 0z) that recover the DDIM samplers (Song al.) 2021a),, while flow-matching models are recovered in the deterministic limit Ø, = 0 or its stochastic extension. This unified formulation allows us to analyze and manipulate both model types using a single theoretical perspective, and provides a way for control-theorets interventions.
di
3.2 OPTIMAL CONTROL PROBLEM
Optimal control (OC) is a mathematical framework for finding an optimal strategy to steer a dynam- ical system minimize functional. While OC encompasses wide range of problem formu- lations, we focus on the quadratic cost and additive control problem, starting from the given initial
state Z0 gd. Consider continuous-time dynamics at Eq. (3). where b I2d [0, > R and
Te E R.. The OC problem aims to find the additional optimal control term u Rd x [0, 1] + Rd that minimizes the following cost functional
[ (bll((a),)) +]| + f(x)")) dt + g(x²²)
min E NEl
(6)
s.t. dxy" = (b(x), () + azu(x; f)) dt + oidB,, x = 20p
where f is the running cost and is the terminal cost. This optimal control problem has been exten- sively studied in both deterministic and stochastic settings, and analytical tools such as the Hamil- non-Jacobi-Bellaat (HJB) equations (Fleming & Rishel 2012) and Pontryagin's 's Maximum Princi- ple (PMP) (Levine 1972) provide necessary and, in some cases, sufficient conditions for optimality.
4 METHODS
4.1 MOTIVATION: FROM GRADIENT ASCENT TO TRAJECTORY CONTROL
Assuming a differentiable reward function r(-), the most direct approach for editing a given image 21 to maximize r(-) is to perform Gradient Ascent (GA) in the pixel space. While this provides the steepest direction to optimize the image, it disregards the underlying image prior, leading to ad- versarial and out-of distiibibtttion results that are perceptually unrealistic (Goodfellow et al.) 2014)) An alternative to prevent this is leverage the generative model's 's prior, by first performing deter- ministic inversion into noise space (Mokady et al. 2023) and then applying reward-guided sampling
Thhe expectation over the Brownian motion in Fq. 6 can he removed by setting ae = 0. or fixing the Brownian motion with certain realization of dB as constant.
4

---

Published as a conference paper at ICLR 2026
methods during the reverse process. However, reward-guided sampling fundamentally constrained by its reliance on approximated guidance; since any noiseless image is not available in the sampling process, samples are optimized to increase the reward the posterior mean (Efron 2011) of clean images from the given noised image. As the reward function becomes more complex and non-linear, corrupt the global consistency of the image structure. More- over, previous guided sampling methods cannot provide a theoretical justification for the selection of the guidance scale, and require careful hyperparameter tuning find their optimal performance.
this guidance be ineffective
To overcome these limitations, we propose a novel image editing methodology for the guidance term that is both effective and minimizes off- aff-mmanfoodd phenomenon, by rephrasing the problem as the optimization of the entire generation trajectory with optimal control.
4.2 PROBLEM FORMULATION
Let's say {ailirr is given as an initial trajectory sampled from Eq. (3). where æ1 denotes the given source image and 7 E [0, 1) is the starting noise depth. Even for real-world images that were generated by the model, there are methods to get an initial trajectory { that ends at the given image, which are further discussed in Section Our goal introduce an additional control term uz into the drift and find the optimized trajectory {æ:?" 1=rr that still starts from Zy but produces an edited image æt that remains realistic and faithful to a source image #1 while maximizing the reward r(-). Formally, we solve the following optimal control problem,
min [ Ellu(x); t)l rxx") well s.t. dxy"" (b(x="' () + u(x t))dt + o,dB,, 2 TT,
where the Brownian component will be replaced by the fixed realization according to the given {æ+) ter since we only focus on the optimization of the single trajectory. Since both the base drift term and reward functions are complex and non non-linear, it is impractical to find closed-form solution for u(xx" 1) that guarantees the global minimum of the cost. Nonetheless, PMP states the necessary condition that the optimal control term of Eq. 7 satisfies. Specifically, by introducing a Hamiltonian
H( u, Pt, t) = P. (b(æ,,.) t) u) + 1 ||/u/l where Pt often called the adjoint state, the optimal trajectory satisfies three coupled differential equations:
dx;
V M(xi t.. = b(æ; t) + ui
2T = ZT
(8)
dt
dpi
di V., H(a, u², .P,,)) = [V O)] Pt: pi = -v,
(9)
uz = arg min H( u, t)
(10)
wEll
Therefore, our goal is find the optimal control u² to construct the trajectory that satisfies these optimality conditions. After we find the optimized trajectory {ar²" H we take the terminal point at reward-guided editing result of æ2. Compared to Adjoint Matching (Domingo-Enrire Enrich et al. 2025), which had to formulate its goal into stochastic optimal control problem to tune the entire model's marginal distribution, our formulation directly targets the single-image editing.
4.3 ITERATIVE TRAJECTORY OPTIMIZATION VIA ADJOINT GUIDANCE
However, jointly optimizing 2z, de. and P: across all time steps is computationally impractical. Therefore, we propose an iterative approach analogous to Coordinate Descent (Wright 2015). In each iteration, we sequentially update each component to better satisfy the PMP conditions:
1. Compute Adjoint State py: With the current trajectory and control {x,, uHler fixed, we solve the adjoint equation Eq. ( backward in time to compute the adjoint states {pe)
2. Update Control ux: We then update the control {wr) towards ppr according to the optimality condition of Eq. (10).
3. Update Trajectory æ,; With the updated control, we simulate a new, updated trajectory {#x)_r using Eq. (8)
5

---

Published as a conference paper at ICLR 2026
Algorithm Image Editing via Trajectory Optimization Control
Require: Source image 21, Depth T 1, Number of iteration N, Unconditional base model
0. Learning rate X, Reward function r(.), Reward weight w
1: [ [B.] simulate_tresee bamullatetrajector((a,,.
2: {ur]er :=
3: for ifer to
4: {pr)i_ compute. adjoaat(joojj.ooo/ee
O, wr(-))
> Compute pt from current
5: Ut = U M(u, + pt) for 1....
Update uy towards Pe b Get with updated de
6: Iz+dtt It + (bbl 1) + us det + B, for t T..... 1 dt
7:
end for
8:
return 21
This iterative process is repeated. progressively refining the trajectory until it converges to a path that locally satisfies the optimality conditions, yielding a final edited image xf that achieves higher reward while maintaining high fidelity to the source image, illustrated in Figure
Algorithm I describes the proposed image trajectory optimization process. We denote the function that generates the initial image trajectory simulate trajectory. For our primary results, utilized deterministic DDIM inversion for diffusion models and the time-reversed ODE for flow- matching models, noiseless trajectory with o, 0. We discuss alternative stochastic methods for initial trajectory generation in Section 66] Note that compared to previous methods with empir- ical guidance scale search YYe et al.) 2024). the guidance scale in all steps can be controlled by weight parameter w on the terminal reward function r(-). More specified algorithms for diffusion and flow-matching models are detailed Appendix [A.]] Furthermore, we discuss the advantage of our method over the previously suggested guided sampling methods in Appendix B.2 by the link between their guidance terms and the optimal control problem.
EXPERIMENTS
In this section, we evaluated our method and several baselines to edit the given images to improve the desired reward. We designed four scenarios with different reward objectives: Human Preference, Style Transfer, Counterfactual Generation, and Text-guided Image Editing.
5.1 EXPERIMENTAL SETUP
Models and Baselines. We used StableDiffusion 1.5 (Rombach et al. 2022) and StableDiffusion 3 (Esser et al. 2024) as primary unconditional diffusion and flow-matching model, respectively. The main results are reported using the diffusion model, while the results for the flow-matching model are provided in Appendix [B. We compared our method against categories of baselines: Naive Gradient Ascent (GA), which directly adds the reward gradient to the source image, Sec- ond, we adapt an image inversion followed by several haward-guided sampling methods, including DPS (Chung et al. 2023), FreeDoM (Yu et al., 2023). and TFG (Ye et al. 2024). For all experi- ments, we only utilized unconditional models isolate the effect of the reward guidance from any text conditioning. Detailed hyperparameter settings for each method are provided in Appendix A.2
Datasets. We prepared diverse of datasets depending on the task: Images sampled with the prompts from REFI. (Xu et al. (224) for human preference, Pick-a-Pic (Kirstain et al.] 2023) for style transfer, ImageNet- 1k (Deng et al. (2099) for counterfactual generation, and CelebA-HQ (Kar- ras et al. 2017) for text-guided facial editing. Each evaluation is performed on 300 randomly selected images from the respective datasets.
Evaluation Metrics. Our metrics are designed quantify three aspects: (1) Effectiveness of the method to increase the target reward. (2) The output's generalizability beyond target reward overfit- ting. which measured with different reward functions for the same quality. (3) Preservation of the content and structure of the source image, which we mostly employed LPIPS (Zhang et al. 2018)
and CLIP cosine similarity (Radford et al. (021) between the source and edited images (CI.IP-1are)

---

Published as a conference paper at ICLR 2026
Source
GA
Inversion DPS Inversion FreeDoM Inversion TFG
Ours
(a)
+Hummn preference
HCC-LL FL5592
FR-1 11--1-244
111
(b)
+ Stylc
12.006
SMi 5367
"Wlyens"
(d)
+ "smiling mon-
S
piis pI: or c
+
t PH: O
CLIF
CLIP
5425
CLIPS
2333
CLIF 2446
Figure 3: Qualitative comparison (a) Human preference, (b) Style transfer, (c) Counterfactual
generation, and (d) Text-guided image editing. Each image's target reward is written in yellow.
Target reward Reward []
Validation metrics CLIPScore[[]
Source preservation
Method
HPSv2[1]
Aesthetic[t]
LPIPS[[]
CLIP-Lara(1]
None
0.1542
0.2385
0.2887
6.0516
0.0000
1,0000
Gradient Ascent
L,9088
0.2247
0.2877
5.5775
0.1474
0.9195
Inversion+DPS
0.2650
5.8276
0.8505
Inversion-FreeDoM
1.5995
0.2226
0,2356
3.4951
0,5503
0.7225
Inversion+TFG
1.7053
0.2362
0.2727
5.6331
0,2927
0.8401
Ours
1.894
0.2526
0.2904
6.1088
0.1717
0,9242
Table Quantitative results for higher human preference. Bold: best, underline: second best.
5.2 RESULTS
We discuss the performance of method across different scenarios, where several examples and the qualitative comparison with baselines are shown in Figure aand Figure respectively.
Human Preference. Human preference captures composite concept of image quality, prompt alignment, and other subjective factors. Although it's difficult to express through explicit condi- tions such text, several proxy metries have been proposed. We adopt the ImageReward (Xu al.) 2024) between the image and its corresponding text prompt as the target reward function, which is trained predict human preference scores. We evaluate HPSv2 (Wu al. 2023). image-text CLIP- Score (Radford et al., 202I), and Aesthetic Score (Schuhmann 2022 as similar validation metrics for the generalizability.
The first row of Figure shows a qualitative comparison across methods. GA leaves the source image mostly unchanged while introducing severe artifacts, which are clear indication of reward hacking. This is further shown in Table [ where GA achieves the highest target reward, but its generalization to other human preference metrics is limited. Meanwhile, guided-sampling-bases methods deviate the image more than ours through their sampling process, which doesn't regard the source image. Moreover, the result often has severe structural degradation. This stems from the importance of high-frequency details of the reward function, where the guidance on the blurred posterior mean can be ineffective or harmful. In contrast, our approach achieves better target reward and source image fidelity than guided sampling baselines, with generalized performance that also

---

Published as a conference paper at ICLR 2026
Targel reward ||G]||[[]]
Validation metries
Source preservation
Method
CLIP-Lay [T]
CI.IP-
None
12.190
0.4757
0.1236
1,0000
Gradient Ascent
4.8742
0.5270
0.1953
0.8374
Inversion-DPS
6,8435
0.5395
0.1693
0.6858
Inversion-FroeDoM
5.4619
0.5629
0.2250
0.6207
Inversion=TFG
6.2641
0.5455
0.1938
0.7076
Ours
5.0185
0,5782
0.2467
0,7169
Table 2: Quantitative results on style transfer. AG denotes the difference between the Gram matrix of the editing output and the style reference image. Bold: best, underline: second best.
Target reward Logit
Validation metries CLIPScore [t]
Source preservation
Method
LPIPS[J]
CLIP-I.wdd11
None
4.8722
0.1452
0.0000
1.0000
Gradient Ascent
24.875
0.1908
0.2246
0.8483
Inversion+DPS
20.378
0.1811
0.3251
0.7305
Iiverion+FFreeDMM
17.891
0.1736
0.4801
0.6411
Inversion+TFG
18.854
0.1757
0.2972
0.7607
Ours
23.372
0.1936
0.2251
0.8256
Table 3: Quantitative results on counterfactual generation. Bold: best, underline: second best.
increases the validation metrics. This suggests that by optimizing the entire trajectory, our method avoids reward hacking and produces more coherent, high-quality editing results.
The goal is to edit a source image with the artistic style of a reference image while preserving its original content. The target reward is defined as the negated Frobenius norm of the Gram matrix difference (| ||] FF) extracted from the edited image and the reference. Style reference images were selected from Hertz al. (2024). Following previous works on style transfer (Rout et al., 2024 Hertz et al. 2024), style alignment is validated with CLIP cosine similarity (CLIP- Isty) and DINO cosine similarity (DINOzty) (Caron al. (021) between the output and the style reference image. The source image preservation is only measured by CLIP-Isre since LPIPS wasn't instructive for the task that changes the entirety of the image.
Style Transfer:
Again, our method achieves the highest validation metrics as shown in Table 2 while the effect of GA only limited its target reward. Guided campling-based methods unavoidably distort the source image's 's content in the process of stylization. The second row of Figure 3 illustrates that our trajectory optimization offers both stylistically faithful and structurally coherent images.
Counterfactual Generation. Counterfactuals are widely used in explainable AI, they reveal what minimal changes are sufficient to alter the decision of the classifier, offering human- interprrateee insights into the model's is reasoning (Verma et al. 2024; Kim et al. 2025b). In this section, we edit the image to alter the classifier's 's decision with minimal structural change. Using a pre-trained robust classifier ImageNet- 1k (Santurkar et al. 2019), define the reward the logit value (Logit:g:) of a new target class different from the source image. The target class was selected to be close to the original class based the Boottoc (009 ImageNet-1k hierarchy. We the CLIPscore for the generalizability, with the text prompt of "a photo with [class]".
As presented in Table 3 and the third row of Figure 3 our method effectively generates counter- factual examples by sufficiently increasing the target class logit while preserving the overall ap- pearance of the image. Note that GA shows better validation metrics and image quality compared to other baselines in this task, only because the reward function is highly robust to adversarial at- tacks. In contrast, our method achieves better or comparable reward optimization and source image preservation throughout various tasks, without any assumptions restrictions on the objectives.
Text-guided Image Editing. Unlike most approaches that rely on models trained to learn a text- conditional distribution, we frame the classic task of text-guided editing within our reward-based framework. Following prior work on reward-driven text-based editing (Liu et al.) 2023), we de- sign our scenario on the CelebA-HQ (Karras et al. 2017) dataset. We use CLIPScore between the edited image and a target text prompt (e.g.. "A smiling man.") ") as a reward. The target text prompts randomly generated for each image to change of its features according to the CelebA-HQ attributes (Na et al., (002). We additionally use ImageReward and HPSv2 as separate metrics to evaluate image-text alignment.
8

---

Published as a conference paper at ICLR 2026
Target reward CLIII11I
Validation metries
Source preservation
Method
ImageReward[[1
HPSv2 [1]
LPIPSIUJ
None
0.1760
-0,2404
0.2233
0.0000
1,0000
Gradient Ascent
0.3567
-0,2331
0.2193
0,1250
0,6660
Inversion+DPPS
0.3173
.0.2923
0,2032
0.3658
0.5300
DoM
0.3158
-0.1100
0.2094
0,4492
0.5147
TFV
0.3260
-0,2801
0.2040
0.3745
0.5282
Ours
0.0976
0.2252
Table Quantitative results on text-guided image editing. Bold: best, underline: second best.
0.6
Ours
1.8
1.8
4.7
TPG
12-
--
Ours
Ours
0.3
0.8
freeOOM
FreDDoM
TFG
TFG
0.6
0.7
0.5
0.4 0.3
0.1
200
1000
200
1000
TFLOP
TELOPS
Figure 4: (a) Trade-off between target reward and source image fidelity with different guidance scale hyperparameters. (b) Evolvement of the target reward and source image fidelity with increasing computational cost.
Our approach achieves the best alignment with the textual description in both quantitative measures (Table 4 and perceptually (the last row of Figure 3. While inversion- based sampling methods can also produce appropriate results, they inevitably lose more of the information from the source images (e.g., the letters in the background), leading lower LPIPS and CLIP-
User Study. To validate the perceptual quality, we conducted user study with 42 different participants who were asked across different categories: align- ment with the target reward, faithfulness to the source im- age, and quality of the edited image. Each participant viewed 50 images and rated them on 5-point scale. The results the Table 5 demonstrated that our model significantly outper- formed the baseline models in terms of perceptual quality.
Method
Reward align.
Faith- fulness
Quality
Gradient Ascent
2.75
3.37
2.53
Inv. + DPS
Inv. + FreeDoM
2,90
2.45
2.42
Inc TFG
3.02
2.94
2.74
Ours
3.67
3,60
3.36
Table User study result.
6 DISCUSSION
Reward-Fidelity Tradeoff with Different Guidance Scale. While our method has been shown to provide superior editing performance across various scenarios, the inherent trade-off between reward alignment and source fidelity is present in all guidance- based approaches. For a fairer comparison, evaluated the performance of our method and the baselines across range of guidance scales. Figure (a) plots the target reward (ImageReward) against source fidelity (LPIPS) on 100 REFL prompt images. Our approach achieves dominant Pareto front, indicating a better editing method for any given level of editing scale.
Performance over Different Guidance Scale. To verify that the superior performance of our method does not simply result from using more computation, we examined the trade-off between computational efficiency and performance for both method and the baselines. The results are summarized in Figure (b): the baselines can apply additional optimization steps by increasing Nrecur and Naters but they still achieve lower reward and source preservation than our model at the same FLOPs. We also observe that, unlike ours, excessive guidance for the baselines leads to reward decrease. Note that increasing Nater our method does not correspond to stronger guid- ance, but better convergence toward the optimal trajectory. consequently, LPIPS does not increase with larger Nrecur, but rather decreases.
Impact of Initial Trajectory Generation Strategy. Our main experiments use initial trajectories {#t)!-r via simulating a noiseless reverse sampling path. An alternative to generate a stochastic Markovian trajectory by applying the forward SDE process to the source image (Song et al. 202
Rout et al. 2025). This approach simulates a sampling path with a different noise schedule, with a fixed realization of the Brownian motion term B, e O. While both effective as shown in Figure

---

Published as a conference paper at ICLR 2026
Stndhaiie Determiniek
SD1.5
0: SD3
5
t-0.0 D-0.5
t-1.0
Figure 5: Selection of different initial trajectory generation strategies on different model types, with the plot of or for each model.
we found that the Markovian trajectory is more sensitive hyperparameters and more prone to image degradation, especially in flow-mttching models. This is likely because the sampling of the Markovian path has a chance introduce an infeasible Brownian term for real-world images, where this error magnifies on flow-matching models with high o, See Appendix A.T for more detailed analyses on the choice different initial trajectories.
7 CONCLUSION
In this work, we proposed a novel reward-guided image editing by formulating the task as a tra- jectory optimal control problem. Unlike previous guidance methods that typically rely on step- wise corrections with posterior mean, which can compromise the global structure of the image, our method treats the entire reverse diffusion trajectory as the object of optimization. Notably, our framework is training-free and broadly applicable across diffusion and flow-matching models. Our experiments across human preference optimization, style transfer, counterfactual generation, and text-guided editing demonstrate that this approach not only achieves substantial gains reward objectives but also mitigates common pitfalls such as reward hacking and structural collapse.
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation Korea under Grant No. RS-2024- 00336454. This work was supported by the Institute for Information & communications Technology Planning & Evaluation(IITPP) grant funded by the Korea government(MSIT), (RS-2019-I190077) Artificial Intelligence Graduate School Program(KAIST) (RS-2025-02304967, Al Star Fellow- ship(KAIST)) This research was supported by the Al Computing Infrastructure Enhancement (GPU Rental Support) User Support Program funded by the Ministry of Science and ICT (MSIT), Republic of Korea (RQT-25-120217).
REFERENCES
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 13300, 2023.
Michael Bostock. Imagenet hierarchy, 2019.
URL http:://bservablehhg
@mbosttck//ime haapee///mmmaeeet-eeeaaaaee
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. ICCV, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEFF/CVF international conference on computer vision, pp. 9650-9660, 2021.
Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Mare Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scabe scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255 leee, 2009.
10

---

Published as a conference paper at ICLR 2026
Yingying Deng, Xiangyu He, Changwang Mei, Peisong Wang, and Fan Tang. Fireflow: Fast inver- sion of rectified flow for image semantic editing. arXiv preprint arXiv:2412.07517, 2024.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs on image synthesis. In A. Beygelzimer, Y. Dauphin, Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021.
Carles Domingo-Enrien Enrich, Michal Drozdzal, Brian Karrer, and Ricky TQ Chen. Adjoint matching: Fine-tuning flow and diffusion generative models with memoryless stochastic optimal control. The Thirteenth International Conference on Learning Representations, 2025.
Bradley Efron. Tweedic's formula and selection bias. Journal of the American Statistical Associa- tion, 106(496):1602-1614, 2011.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Wendell Fleming and Raymond W Rishel. Deterministic and stochastic optimal control, vol- ume 1. Springer Science & Business Media, 2012.
Daniel Geng and Andrew Owens. Motion guidance: Diffusion-based image editing with differen- tiable motion estimators. arXiv preprint arXiv:2401.18085 2024.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Yutong He, Naoki Murata, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei- Hsiang Liao, Yuki Mitsufuji, Zico Kolter, Ruslan Salakhutdinov, et al. Manifold preserving guided diffusion. arXiv preprint arXiv:2311.1643 2023.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prome image editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022.
Amir Hertz, Kfir Aberman, and Daniel Cohen-Or. Delta denoising score. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2328-2337, 2023.
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4775-4785 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in 2020.
Neural Information Processing Systems,
Inbar Huberman-Spiee Spiegelllss, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12469-12477 2024.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv: 1016, 10196, 2017.
Jaemin Kim, Bryan Sangwoo Kim, and Jong Chul Yc. Free2guide: Training-free free text-to-video align- ment using image Ivlm. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 17920-17929, 2025a.
Won Jun Kim, Hyungjin Chung, Jaemin Kim, Sangmin Lee, Byeongsu Sim, and Jong Chul Ye. ferivative-free diffusion constrande gradient for unified xai. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 23795-23805, 2025b.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances neural information processing systems, 36:36652-36663. 2023.

---

Published as a conference paper at ICLR 2026
Vladimir Kulikov, Matan Kleiner, Inbar Huberman- Saaeerrman-Siieegglas and Tomer Michaeli. Flowedit: Inversion-fiee text-based editing using pre-trained flow models. arXiv preprint arXiv:2412.08629, 2024.
W Levine. Optimal control theory: An introduction. IEEE Transactions on Automatic Control, 17 (3):423-423, 1972.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.0274:5 2022.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003 2022.
Xingchao Liu, Lemeng Wu, Shujian Zhang, Chengyue Gong, Wei Ping, and Qiang Liu. Flowgrad: Controlling the output of generative odes with gradients. In Proceedings of the IEEF/CVF Con- ference Computer Vision and Pattern Recognition (CVPR), pp.
June 2023.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.00 01073, 2021.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6038-6047, 2023.
Dongbin Na, Sangwoo Ji, and Jong Kim. Unrestricted black-box adversarial attack using gan with limited queries. In European Conference Computer Vision, pp. 467-4827 Springer, 2022.
Hyelin Nam, Gihyun Kwon, Geon Yeong Park, and Jong Chul Ye. Contrastive denoising score for text-guided latent diffusion image editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9192-9201, 2024.
Maitreya Patel, Song Wen, Dimitris N Metaxas, and Yezhou Yang. Steering rectified flow models in the vector field for controlled image generation. arXiv preprint arXiv:24/2.00100 2024.
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. The Eleventh International Conference on Learning Representations, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal,
Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Patlern Recognition, pp. 10684-10695, 2022.
Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Rb-modulation: Training- free personalization of diffusion models using stochastic optimal control. arXiv preprint 2rXhv:240..17400 2024.
Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Semantic image inversion and editing using rectified stochastic differential equations. The Thirteenth International Conference on Learning Representations, 2025.
Shibani Santurkar, Andrew llyas, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with a single (robust) classifier. Advances in Neural Information Pro- cessing Systems, 32, 2019.
Christoph Schuhmann.
Laion-acsthetics.
https://lainnaai/blod
aeiion-esttheticss/ 2022. Accessed: 2023-11-10.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- fional Conference on Learning Representations, 2021a.
12

---

Published as a conference paper at ICLR 2026
Jiaming Song, Qinsheng Zhang, Hongxu Yin, Morteza Mardani, Ming- Yu Liu, Jan Kautz, Yongxin Chen, and Arash Vahdat. Loss-guided diffusion models for plug-and-play controllable generation. PMLR, 2023.
In International Conference on Machine Learning, pp.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th Interna- tional Conference on Learning Representations, ICLR, 2021b.
Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan Hines, John Dickerson, and Chirag Shah. Counterfactual explanations and algorithmic recourses for machine learning: A review. ACM Computing Surveys, 56(12):1 2024.
Pascal Vincent. connection between score matching and denoising autoencoders. Neural compu- tation, 23(7):1661-1674 2011.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Ste- fano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model aligmment using direct preference optimization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228 8238, 2024.
Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint arX0::241..04766 2024.
Stephen J Wright. Coordinate descent algorithms. Mathematical programming, 151(1):3-34, 2015.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306.0944 09341, 2023.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
Haotian Ye, Haowei Lin, Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Y Zou, and Stefano Ermon. Tfg: Unified training-free guidance for diffusion models. Advances Neural Information Processing Systems, 37:22370-22441 2024.
Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training- free energy-guided conditional diffusion model. In Proceedings of the IEEE/CVF International Con- ference Computer Vision, pp. 231 23174-23184, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Kaizhen Zhu, Mokai Pan, Yuexin Ma, Yanwei Fu, Jingyi Yu, Jingya Wang, and Ye Shi. Unidb: uni- fied diffusion bridge framework via stochastic optimal control. arXiv preprint arXiv:2502.05449, 2025.
13

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 MORE DETAILED ALGORITHM FOR DIFFUSION AND
MODELS
In this section, describe more detailed processes of image editing with trajectory optimal control, as specified instances of Algorithm I for diffusion models (Algorithm) 2 and flow-matching models (Algorithm D. respectively.
Simulate Trajectory. We suggest two possible implementations of simulate.trajjecorry/e. for each model family to generate plausible sampling trajectory:
Deterministic trajectory for given source image 21 be obtained by deterministic DDIM Inversion Eq. (11) for diffusion models and time-reversed ODE Eq. (22) for flow-
matching models:
Et
vI p--V--rcolae. aicolot..t) + VI (0o
Lt-dt da
(11)
Vā
Zt-dt = xz Vo t)dt.
(12)
These methods simulate the sampling process that leads to æ without any stochasticity.
Markovian trajectory for a given source image Ø1 can be obtained by simulating the for- ward SDE that retains the same marginal probability of p(z,). Diffusion models readily
utilize their forward process as Eq. (13), and flow-matching models also have the corre-
sponding forward SDE as Eq. [4) with the same marginal distribution (Rout et al. 2025)
as follows:
Tt-dt Vas-dt VI Ct-dEE
(13)
2(1 - 1)dl
Tt-dt = Eidt +
E~ N(,,1,
(14)
where dt a is the single-step noise schedule. This simulates the sampling process
&
by Eq. (4) with at = v & for diffusion models, and Eq. (5) with 0z v 2(1-() for
the
flow-matching models. The difference between &x+dd == E [x]] and the simulated trajectory Ie-di can be considered as the realization of the Brownian term B..
B. Dz+dt - (Vāezae
ct vI - âgea (22,
+ v ac+dt Ex(t,
(15)
B, = Ii+di + (20((2), - x2)) dt
(16)
where 7/ = or V dl.
Deterministic trajectory guarantees the model will generate the source image following the obtained trajectory. On the other hand, obtaining Markovian trajectory via noise injection requires the as- sumption that the source image follows the distribution learned by the pre-trained model. As the real-world image deviates from the modeled distribution, the calculated B, becomes more infeasi- ble as a Brownian term dB This error is often exaggerated in flow-matt dow-matching models with high oa (see Figure Instead, multiple Markovian trajectories can generated from a single source image, enabling diverse editing results with the same setting.
Compute Adjoint. In each iteration of our trajectory optimization, we compute the set of adjoint states {p. Jp.r using the process pampute_adjoin (t 0, wr(-)), given the current trajec- tory, reward function r and a weight parameter w. This is achieved by iteratively solving the partial differential equation (PDE) in Eq. (9) backward in time from
to T. Notably, the reward weight w globally scales the magnitude of the adjoint states, thereby controlling the overall strength of the guidance applied to the trajectory.
Update Control. Rather than directly applying the optimal control condition u, = -P: from Eq. 100.. we employ wradient-based update scheme. We update the control at each iteration by
14

---

Published as a conference paper at ICLR 2026
Algorithm 2 Image Editing via Trajectory Optimization Control with Diffusion Model
Require: Source image æ1, Depth 0 < T 1, Number of iteration N. Base model 0, Learning rate X, Reward function r(-), Reward weight w, mode E ('Deterministic', 'Markovian')
1- (1 - an)
1: 7) =
mode = 'Deterministic" else
1-ae
2: Define dt = (Vā++at (==VI-areu(eedd) + V1 a1+ - 11 (222)))
vār
3:
if made 'Deterministic" then
4:
{
i= (DIM.Inversion(1).00 [B:
5:
6: else
7:
3:-dt di + v1 ae-die for 1.... dt
8:
B: = Ii+dl Et+ dtdl for t = T..... 1 dt
9: end if
10: (u))er 0)
11: for iter = to do
12:
P1 -w V 20, T(x1)
13:
p: = Pi+dt + P2+ Va. (ie+ - Dz) for = dt,
14:
ue = ur X(u; + pt) for = !,..,
15:
I++d dt/e u B, for T, ,,... dt
16: end for
17:
return 21
Algorithm 3 Image Editing via Trajectory Optimization Control with Flow-Matching Model
Require:
Source image Depth 0 < T < 1, Number of iteration N, Base model 0, Learning rate X, Reward function r(-), Reward weight w. mode E ("Deterministicc, 'Markovian')
2(1-8)
1: = 0 if mode = 'Deterministic" else to, 2: Define @itdt]i := x; + (vs( 1) 2(1-1) (vo( 1) -aaa)) dt
3: if mode 'Deterministic' then
4:
ft 2, (1 t)dt for 1..... T + dt
5:
{B:}i=T
6: else
It-di xt 1 z,dt + / /2(1-e)dt e 1,.... T dt
7:
8:
B, = tt de/e for / = T,...,
9: end if
10: {ur)i-r 0
for ifer to N do
12:
P1 = - V æ1 r(x1)
Pt Pitdi + Pi+dt Va. (ic+ 2x) for
dt, .... T
u, u, X(u + pt) for 1, 1....,' T
15:
Iz+d 2ctdt/e + utd + B, for = T,..... dt
end for
17: return æ1
taking a gradient step with learning rate À to minimize Lg distance ||u, + Pell2- Note that while describe the most naive gradient ascent in Algorithm 2land Algorithm more advanced optimizers can also be utilized for more stable optimization. Empirically, we find that even single optimiza- tion step iteration is sufficient to achieve stable optimization while maintaining alignment with the PMP conditions.
A.2 HYPERPARAMETER SELECTION
Table 6 lists the detailed hyperparameters for our method and baselines on different image editing scenarios. The hyperparameter notation for the guided sampling baselines follows TFG (Ye al.
15

---

Published as a conference paper at ICLR 2026
StableDiffusion 1.5 (Image Resolution: 512 512)
Method
Gradient Ascent
Inversion + DPS
Inversion + FreeDoM
Inversion TFG
Ours
Inversion depth=0.7.
Human Preference
Inversion 0.p Pt 3.0
Inversion depth=0.77. Newour 2,0: = 1.0
Nyess 1,0: 1.0,
T 0.5, NN 20,
N=100. A=2.0
Na 4,4, 0.5,
w 500
7 7= 0.1
Inversionddepth-4 depth-0.7.
Inversion 0.7 Pt 15.0
Inversion depth=0.7. Necour 2, 7.5
Nescur 1,0c 10.0,
T 0.5, 20,
Style Transfer
N=100, A=3.0
Nace 4,44 1.0.
200
7 0.1
Inversion depth=0.7.
Counterfactual Generation
Inversion depth=0077. 1.0
Inversion depth=0.7. Neweu 2, = 0.4
Nrecer 1 1.0,
T 0.3, N=
N=100. A=1.0
Na 4, Pt 0.1,
@ 50
Inversion depth-0.7,
Text-Guided Image Editing
Inversion depth=0.7. Pt 40.0
Inversion depth=0.7. Neeewr p, 20.0
Nescur 1, 30.0,
T 0.5, 20,
N=100, A=1.5
Nacr 4. 2.5.
1000
0.1
StableDifftssion 3 (Image Resolution: 768 x 768)
Method
Gradient Ascent
Inversion + DPS
Inversion + FreeDoM
Inversion TFG
Ours
Inversion depth=0.7.
Human Preference
Inversion d.pth=0.7, Pr = 5.0
Inversion d.pth=0.7,
Nyeesr 1. 5.0,
T N.7.N 15,
N=100, A=2.0
Neceur 2,Pg = 5.0
Nater 4, 1.0,
10 500
7 0.1
Inversion depth=0.7.
Inversion depth=077. Pr 30.0
Inversion depth=0.7. Nescur 2, 20
N Nesour 1, = 40.0,
T 0.7, 15,
Style Transfer
N=100, Am3.0
Nair 1, 2.5.
1000
7 0.1
Inversion depth=0.7.
Counterfactual
Inversion depth=0.7, Pr 7.0
Inversion d,p Newcur 2, = 3.0
Naecur 1. 7.0,
T 0.7, 15, w 200
N=100, A=1.0
Generation
Na 1, = 0.5,
7
Inversion depth=0.7.
Text-Guided Image Editing
Inversion depth=0.7.
Inversion depth=0.7. Nreeur p, 30.0
Nevow 1, = 60.0,
T = 0.7, = 15, 1000
N=100, A=1.5
Pt
Nace 4,14 2.5.
0.1
Table 6: Hyperparameter settings for the quantitative results in Section 5.2 on different base models, methods, and experiment scenarios.
Inversion depth refers to the noise level, the same as T in ours. Pt and H4, denote the guidance strength multiplied by the v and V r(@1)t), respectively, where øije denotes the pos- terior mean. Niter represents the number of guidance updates performed in single timestep, and Nrecur the number of times the same timestep is repeated with a forward noise injection. 7 is the noise scale injected Z1je for TFG, which is fixed at O.1.
2024).
Robustness to Hyperparameters. We analyzed the impact of key hyperparameter selection in our algorithm in Algorithm namely the inversion depth T and the number of optimization iterations N. As shown in Figure 6 the inversion depth T controls the trade-off between editing strength and source consistency, aligning with observations in previous image editing literature. When T (i.e., shallow noise), the editing effect is minimal as the trajectory has little room to deviate. As T + 0 (i.e., pure noise), the potential for editing increases, but the risk of losing fidelity the source image. Crucially, the number of iterations N governs the convergence of the output trajectory rather than the guidance strength. As illustrated in Figure 6 the final result is not highly sensitive to N provided that N is sufficient for convergence. However, omitting the iterative process entirely (N = with x 1.0) leads to significant artifacts. This is because the control computed from the initial trajectory is no longer optimal for the modified states. Our iterative refinement is therefore essential to ensure the trajectory converges to produce high-reward images.
To further investigate stability, we visualize the behavior of our method and baselines under increas- ing guidance scales (i.e., reward weight w for ours, and Pts f4 for baselines) in Figure 7 While the quantitative trade-off was shown in Figure (a), these qualitative results highlight a distinct difference in robustness. As the guidance scale increases, baselines begin to exhibit severe degrada- tion, including color saturation, artifacts, and structural corruption. In contrast, our method achieves
16

---

Published as a conference paper at ICLR 2026
7-0.9
7-0077
7-0.5
7-0.3
7-0.1
(bo Itteration
N=110
N=15
Human Preference
Figure 6: Qualitative ablation study different choices of hyperparameters for the depth T and the number of iterations N. The text prompt for the alignment is "colorful painting. river flowing grass field with flowers.".
DPS
FreeDoM
TFC
Human Preference
pi Hembel
P.
Ours
strong
Figure 7: Qualitative results on varying guidance scale. The other hyperparameters except {pes Mes w follow the configuration in Table 6 The text prompt for the alignment is "pirate ship, flowing through cosmic nebula.".
significantly higher target rewards while demonstrating a smooth and progressive emphasis on the objective, without compromising image quality.
A.3 USER STUDY
We conducted human-subject study with 42 participants from the general population recruited via an online platform. As shown Figure g each participant was presented with a series of source-edited image pairs and asked rate the edited images along three criteria:
17

---

Published as a conference paper at ICLR 2026
Medhoc (6)
Source image
Edited image
(a)
Target direction: "Hyena"
vay poor
2: poor
3: dkay
good
5 very good
0
0
Smilaity
0
0
0
Figure 8: An example question from our user study survey.
1. Change: Does the edited image show meaningful and noticeable changes in the target direction (e.g., category shift, text description, style transfer)?
2. Similarity: Does the edited image remain faithful to the source image, including back- ground and other non-target regions?
3. Quality: Does the edited image look realistic without obvious artifacts or distortions?
They were instructed to rate each image on a 5-point Likert scale (1 = very poor, very good). Table 5] summarizes the user ratings across the three criteria. These results confirm that our model produces edits that are not only aligned with the intended modifications but also visually convincing and coherent.
B ADDITIONAL RESULTS
B. RESULTS ON FLOW-MATCHING MODELS
While the main manuscript focuses on the performance our methods on the diffusion- mod- els, this section presents qualitative results a state-of-the-art flow-matching model, StableDif- fusion 3, to validate the generality of our method. The experimental protocol remains identical to the experiments in the main paper. Note that all of the baseline methods were originally suggested for diffusion models, and we re-implemented their calculation of the posterior mean and forward noise process analogous flow-matching models. As shown in Table 7 our method maintains its superior performance on the flow-matching model, exhibiting consistent behavior across both model families.
B.2 CONNECTION BETWEEN OPTIMAL CONTROL TERM AND GUIDED SAMPLING
In this section, discuss how the suggested method can be related to the guided sampling methods. In the diffusion model sampling process with the noisy sample é., DPS and many of the suggested
guided sampling variations (Chung et al. 2023) Yu al. 2023 He et al. 2023 Ye al. 2024) calculate the gradient of the objective function at the posterior mean i with respect to Z;, and this guidance term is added into the denoising direction.
Here, we show that this guidance term suggested in DPS ((+1e) can be explained from per- spective of the solution of the optimal control problem:
18

---

Published as a conference paper at ICLR 2026
Human Preference
Target reward ImageReward[11
Validation metrics CLIPScore[11
Source preservation
Method
HPSv2[11
Aesthetic|11
I.PPS[L]
CLIP--,...l"
None
0.1542
0.2385
0.2887
6.0516
0.0000
1.0000
Gradient Ascent
1.9088
0.2247
0.2877
5.5775
0.1474
0,9195
Inversion+DPS
1.4169
0,2189
0.2552
5.7227
0.3767
0.7896
Pversion+FreeDoMt
1.5887
0,2288
0.2305
5.7446
0,5460
0.6893
Inversion+TFG
1.5162
0.2216
0.2745
5.6072
0,3083
0.8537
Ours
1.8529
0.2400
0.2890
6.1730
0.2475
0.9013
Style Transfer
Target reward ||]|||[]]
Validation metrics
Source preservation CLIP-1
Method
CLIP-Lay [T]
DINOsty
None
12.190
0.4757
0.1236
1.0000
Gradient Ascent
4.8742
0,5270
0.1953
0.8374
Inversion+DPS
5.3983
0.5553
0.1774
0.6617
Inversion=FreDoM
4.9643
0,5466
0.2091
0.6365
Inversion+
5.4176
0.5495
0.1922
0.6758
Ours
4.5333
0,5633
0.2201
0.7666
Counterfactual Generation
Target reward Logitey(f1
Validation metrics CLIPSSore [t]
Source preservation
Method
LPIPS[[]
CLIP-1greef11
None
4.8722
0.1452
0.0000
1.0000
Gradient Ascent
24.875
0. 1908
0.2246
0.8203
Inversion+DIPS
21.628
0.1874
0.3852
0,6498
Inversion-FreeDoM
23.085
0.1984
0.4017
0.6241
Inversion+TFG
21.538
0. 1872
0.3846
0.6506
Ours
24.572
0.2044
0.2743
0.7040
Text-guided Image Editing
Target reward CLIPScore(1)
Validation metrics
Source preservation
Method
ImageReward(11
HPSv2 (t]
LPIPS[[]
CLIP-
None
0,1760
-0,2404
0,2233
0.0000
1.0000
Gradient Ascent
0.3567
-0,2331
0.2193
0.1250
0.6660
Inversion+DPS
0,2915
-0.4124
0.2106
0.3262
0.5665
Lnversion+FrreDoM
0,3060
-0,2091
0.2242
0.4571
0.5281
Inversiont TFG
0.2944
-0.3118
0.2093
0.3386
0,5597
Ours
0.3491
-0.1308
0.2272
0.2439
0.6011
Table 7: Quantitative performance of the proposed method and baselines with StableDiffusion 3. Bold: best, underline: second best.
Proposition The guidance term by DPS is equivalent the negative adjoint state Pp under the optimal control problem in Eq. ( calculated with one-step sampling trajectory xt--
proof. Note that one-step sampling from H; clean image domain(e, g., 1) gives T terminal point. When the adjoint state is calculated in this one-step trajectory according to Eq. (9)
we get
Po V 21 (wr (wr($11)))
(17)
P: Po + Va, [
vx.) T po]
(18)
(1 + Jæ. (m11e) T 1))o
(19)
( Po
(20)
ami where x is i-th element of æ.
where Jz. (11 denotes a Jacobian matrix, defined as Ji Sse When put Eq. (17) to Eq. (20). from the chain rule,
19

---

Published as a conference paper at ICLR 2026
Gradient Ascent
Inversion +DPS
Inversion +FreeDoM
Inversion +TFG
Ours
StableDiffusion 1.5
23.74
14.77
23.09
37.26
60.63
Required time [s/image]
StableDDifussion 3
27.55
13.26
20.31
30.50
41.97
StableDiifussio 1.5
277.91
155.93
284.59
543.47
590.93
FLOPs [image
StableDiffusion
592.51
353.95
676.09
863.84
1950.26
Table 8: Required time and FLOPs for each method with different base models. The hyperparameters the Human Preference row of Table 6 with the ImageReward reward function were used. We ran experiments with StableDiffusion half-precision floating point format(float16).
P: = -2, ( Văne (wr(#111)) = (
(21)
(22)
where Eq.
22 is equivalent to the guidance term utilized by DPS with a sign reversed.
This perspective of previous guided sampling methods emphasizes the advantage of our method; utilizes multi-step trajectory that ends with a fully detailed source image endpoint. It also iteratively refines the control term to balance the optimization and the guidance term regularization, where previous guidance terms cannot provide theoretically appropriate guidance strength.
C DISCUSSION ON RELATED FLOW-BASED EDITING METHODS
Recent works have explored the steering and editing of Rectified Flow (ReFlow) models, which share conceptual motivation with our control-based approach. We first summarize the related works and discuss to clarify the distinct contributions of our paper.
(Wang et al. 2024) RF-solver is proposed to reduce inversion and reconstruction errors using a higher-order ODE sampler based on Taylor expansion. Then, RF-Edit is used for editing by storing and sharing self-attention features from the inversion path to the editing path.
RF-Solver
FireFlow (Deng et al. 2024): FireFlow addresses the computational cost of high-order solvers by introducing efficient second-order solver that reuses stored mid-point velocities calculated from the previous step.
FlowChef (Patel et al. 2024): FlowChef proposes inversion-free free framework for steering ReFlow models. It applies inference-tine
guidance by optimizing the trajectory at each step. This is achieved by estimating the final output ên and obtaining the gradient of the loss functions (e.g., mask-based 1.2 loss or classifier loss) to update the current state Zt.
Our work differs from prior approaches in two key aspects. Unlike existing methods, which focus on text-prompt-based editing within the ReFlow family, our framework addresses the more general setting of reward-guided editing without text conditioning and can incorporate any differentiable reward signal (e.g., human preference scores, aesthetic models, classifier logits). Methodologically, our framework formulates editing as a trajectory optimal control problem: starting from an initial inversion, we optimize the entire generation path by solving adjoint-state equations from PMP to up- date a time-var marrinaying control. This yields a trajectory-level optimization that does not rely on attention modulation or user-provided masks to edit images.
D LIMITATIONS AND FUTURE WORK
Our framework, grounded in trajectory optimal control, has several inherent limitations: First, it fun- damentally requires the reward function to be differentiable, as the computation of the adjoint state relies its gradient. While this assumption is shared for most guided sampling methods, this prereq- uisite limits our method to directly applying to objectives that are non- con-dffrrentiable or discrete, such direct human feedback. However, this limitation represents promising avenue for future work;
20

---

Published as a conference paper at ICLR 2026
(a) Human Preference
(b) Style Transfer
Imaye
Image
Reward
on
Imwge
(c) Counterfactual Ceneration
(d) Text-guided Editing
938t class (CoullRower)
"man with beard"
130% Class (Flamingo)
"old man"
292 (Tiger)
ssd
960 (Carbonara)
winking"
Figure 9; Additional qualitative image editing examples of our method and source images.
the framework could be extended to black-box settings by employing Zeroth gradient estima- tion techniques approximate the reward gradients numerically (Kim et al. 2025a). Second, the number model evaluations in our method proportional to and N, leading to about 40 60% more required time than guided sampling- sampling-based based methods, shown in Table Nevertheless, as demonstrated in our efficiency-perfoonne performmneeee trade-off analysis (Figure 4 our method establishes superior Pareto frontier compared to baselines. This indicates that the performance gain justifies
21

---

Published as a conference paper at ICLR 2026
the additional computational cost, and method maintains its advantage when baselines given an equivalent computational budget. Finally, while this work comprchensively validates the framework for 2D image editing, its generalization to other domains like video, 3D models, audio remains for future research. Additionally, investigating optimal reward-aware trajectory initialization strategies beyond deterministic inversion could be valuable for further accelerating convergence.
E THE USE OF LARGE LANGUAGE MODELS (LLMS)
LLMs were not involved in research ideation methodological design and were only used for the purpose of minor expression refinement. The authors retain full responsibility for all scientific content.
22


Reviews:
[{"review_id": "IL8uH2Ixgs", "reviewer": "Reviewer_WApa", "summary": "This paper introduces a training-free, reward-guided image editing framework built atop pretrained diffusion and flow-matching models. Instead of relying on adversarial gradients or retraining reward models, the authors reformulate reward-guided editing as a trajectory optimal control problem, where the reverse diffusion trajectory is treated as a controllable path. Using Pontryagin’s Maximum Principle (PMP) and an iterative adjoint-state optimization, the method updates control signals to steer the entire trajectory toward higher rewards while preserving source fidelity.", "questions": "* What is the wall-clock or compute comparison (FLOPs or inference time) between the proposed method and FreeDoM/DPS? \n* Could the authors provide a Pareto frontier (compute vs. performance) plot across baselines to strengthen the argument of practical utility? \n* How does the proposed method relate to ReNO [5], which performs inference-time noise optimization guided by reward models? * Conceptually, is ReNO equivalent to or distinct from gradient ascent in noise space?\n* Could the method be extended to VLM-based editing with text prompts? This seems promising given the reward-agnostic formulation.\n\n\n[5] “ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization,” NeurIPS 2024.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper presents a novel and theoretically grounded formulation of reward-guided editing via stochastic optimal control, bridging generative modeling and control theory.\n* The adjoint-based iterative optimization offers a principled alternative to heuristic reward gradients, mitigating adversarial guidance effects.\n* Experiments cover diverse tasks and datasets and compare against good baselines..\n* The method is model-agnostic, applicable to both diffusion and flow models (via ODE & SDE conversions).\nThe paper is well written and easy to follow.", "weaknesses": "* The proposed iterative adjoint optimization appears computationally demanding. While FreeDoM and DPS operate in O(n), the proposed trajectory-level optimization is $O(n^2)$, limiting scalability.\n* A stronger comparison on efficiency–performance trade-offs is needed (e.g., Pareto plots already hinted at in Fig. 4a). It would help to contextualize the computational gains relative to improvement in reward or fidelity.\n* It remains unclear how much the improvement arises from multiple refinements versus the control-theoretic formulation itself. Could repeated low-guidance FreeDoM runs approximate similar results?\n* Missing analysis of hyperparameter sensitivity, especially the impact of reward weights and guidance terms, both for baselines and the proposed method.\n* Grounded text-editing benchmarks (e.g., PIE-Bench [1]) are absent, which limits the community relevance of the text-guided editing evaluation.\n* The paper omits discussion of related optimal-control-inspired methods like FlowChef [4], FireFlow [3], and Taming Rectified Flow [2], which are conceptually close in motivation.\n\n[1] “Direct Inversion: Boosting Diffusion-based Editing with 3 Lines of Code,” ICLR 2024.\n\n[2] “Taming Rectified Flow for Inversion and Editing,” ICML 2025.\n\n[3] “FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing,” ICML 2025.\n\n[4] “FlowChef: Steering of Rectified Flow Models for Controlled Generations,” ICCV 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tRqNBLfQa2", "reviewer": "Reviewer_oo4y", "summary": "This paper introduces a novel, training-free framework for reward-guided image editing by reformulating the process as a trajectory optimal control problem. Existing methods (like DPS or FreeDoM) calculate the gradient of the final reward $r(x_1)$ with respect to the posterior mean $\\hat{x}_{1|t}$. They then push the noisy intermediate image $x_t$ in that direction. The paper's key insight is to treat the entire path from the noisy starting point $x_T$ back to the clean image $x_1$ as a single, adjustable trajectory. The authors use the principles of PMP to develop an iterative adjoint-state update algorithm, achieving good results.", "questions": "1. To calculate the gradient through the reward function, I assume it also requires gradient backprop through the decoder and the diffusion model? How much memory cost is it?\n2. Is it possible to us RL (e.g., REINFORCE) to adapt the framework to incorporate non-differentiable feedback signals?\n3. Since the method is formulated as trajectory optimal control, what guarantees can be provided about finding the global optimum, or are the iterative updates susceptible to finding poor local minima?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The formulation is noval and techinically sound, bypassing the issue of using posterior mean for approximation.\n2. The method works with pre-trained models (both diffusion and flow-matching) without requiring any model fine-tuning.\n3. The writing is easy to follow and the logic flows reasonable.", "weaknesses": "1. The proposed method is computationally expensive because it requires a high number of model evaluations.\n2. The quality of the output depends on hyperparameters like inversion depth ($T$) and the number of iterations ($N$), which requires careful tuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ve5ShUJS1k", "reviewer": "Reviewer_DZiS", "summary": "This paper introduces a training-free framework for reward-guided image editing using trajectory optimal control. The authors regard the image editing procedure as the optimal control of a reverse sampling trajectory from a diffusion or flow-matching generative model. Leveraging Pontryagin’s Maximum Principle, the method iteratively updates adjoint states and control terms to steer the generation trajectory, producing edited images that balance reward maximization with fidelity to the source image. Experiments on various tasks such as human preference optimization, style transfer, counterfactual generation, and text-guided editing demonstrate the superior performance compared to baselines.", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Treating the reverse diffusion process as a controllable trajectory and solve the control problem by using Pontryagin’s Maximum Principle is novel to my knowledge.\n- The derivation and mathematical formulation is solid and intuitive. \n- The method is training-free, and can be applied on various base models such as SD1.5 and SD3.\n- The paper is clear and well-written.", "weaknesses": "- As stated in the paper, the proposed method requires the reward function to be differentiable and the gradient need to be calculated. Would this cause notable extra computational overhead? Authors are expected to provide the comparison of inference time between the proposed method and baselines.\n- Generally, I think the experiments are insufficient, which are still not enough to illustrate the effectiveness of the proposed method:\n    1. Authors are expected to provide the results on FLUX, which is one of the most powerful image generation foundation models.\n    2. Missing baselines. Currently, there emerges a number of image editing methods based on flow-matching-based models such as [1,2,3], authors should compare the image editing performance between the proposed method and these baselines.\n    3. The qualitative results are not enough and some flaws still exists (for example, in second row in Figure 3, the ears of the cat are not clear in the image produced by the proposed method). Authors are expected to provide more comprehensive qualitative results, which are important to evaluate the performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RK0Xiqazdj", "reviewer": "Reviewer_HoKy", "summary": "This paper introduces a novel, training-free framework for reward-guided image editing applicable to both diffusion and flow-matching models. The core innovation is formulating the editing process as a trajectory optimal control problem. This approach treats the reverse diffusion process, starting from a source image, as a controllable trajectory. By leveraging the principles of Pontryagin's Maximum Principle (PMP), the authors develop an iterative adjoint-state optimization algorithm that steers the entire trajectory toward a terminal state that maximizes an arbitrary differentiable reward function, while explicitly minimizing the control effort to maintain fidelity to the source image structure.\n\nThe method is validated across four diverse editing tasks: Human Preference Optimization, Style Transfer, Counterfactual Generation, and Text-guided Editing. Experiments show that this trajectory-level optimization approach achieves a superior balance between reward maximization and source image fidelity compared to existing inversion-based guided sampling techniques, effectively mitigating issues like \"reward hacking\" and structural degradation.", "questions": "above", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Principled Formulation based on Optimal Control (OC): The paper provides a strong theoretical foundation by framing image editing as a trajectory optimal control problem and deriving the iterative update rules using PMP. This approach is superior to heuristic, step-wise guidance methods as it optimizes the entire path, leading to better structural coherence and a more theoretically grounded way to balance editing (reward) and fidelity (control regularization).\n\nBroad Applicability (Generality): The framework is designed to be training-free and is unified under the Stochastic Differential Equation (SDE) framework, making it applicable to both Diffusion Models (SD 1.5) and Flow-Matching Models (SD 3). This generality significantly broadens its impact and utility.\n\nSuperior Fidelity-Reward Trade-off: Quantitatively and qualitatively, the method consistently establishes a dominant Pareto front between reward alignment and source fidelity (Figure 4a). In contrast to baselines which often suffer from structural collapse or reward hacking, this method achieves high rewards while maintaining high LPIPS and $CLIP-I_{src}$ (Tables 1, 2, 3, 4).\n\nMitigation of Common Failure Modes: The trajectory optimization successfully avoids common failure modes of guided sampling methods, such as reward hacking (where GA introduces severe artifacts) and structural degradation (where stepwise guidance corrupts high-frequency details). The user study (Table 5) confirms the superior perceptual quality and faithfulness of the generated edits.", "weaknesses": "Fundamental Dependence on Differentiable Rewards (Critical Limitation): The entire framework hinges on the reward function $r(\\cdot)$ being differentiable, as the adjoint state calculation explicitly requires $\\nabla_{x_1} r(x_1)$. This strictly limits its application to objectives that are inherently smooth or can be approximated by differentiable proxies, excluding direct application of discrete or non-differentiable objectives (like direct human feedback ratings). This restricts its practical use in true preference learning settings.\n\nHigh Computational Cost and Efficiency Concerns: The method is computationally expensive. As shown in Table 8, the required time for the optimization is substantially higher (approximately 80% more time than guided sampling baselines) because the iterative process necessitates multiple, full, backwards and forwards passes over the model's trajectory (proportional to $T$ and $N$). While the results are better, the practical deployment cost is a major constraint.\n\nSensitivity to Optimization Hyperparameters: The method introduces new critical hyperparameters, notably the inversion depth $T$ (where the optimization starts) and the number of iterations $N$. As shown in Figure 5, the performance is highly sensitive to $T$ (controlling the fidelity-edit trade-off) and $N$ (critical for convergence and stability). The need for careful tuning of $T$ and $N$ for every new task or model reduces the \"training-free\" simplicity.\n\nSimplistic Trajectory Initialization: The paper primarily relies on deterministic DDIM/ODE inversion for the initial trajectory. While the effects of Markovian (stochastic) initialization are discussed, the deterministic approach, while simple, may place the starting trajectory on a rigid path that is far from the optimal reward direction, potentially leading to slow convergence in complex editing scenarios. Further investigation into optimal reward-aware initialization could be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)