Training-Free Text-Guided Color Editing with Multi-Modal Diffusion Transformer
Authors: Zixin Yin
Xili Dai
Ling-Hao Chen
Deyu Zhou
Jianan Wang
Duomin Wang
Gang YU
Lionel Ni
Lei Zhang
Heung-Yeung Shum
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion
DiT
Image Editing
Video Editing
Color Editing
Primary area: generative models
TL;DR: We introduce ColorCtrl, a training-free method for text-guided color editing in images and videos. It enables precise, word-level control of color attributes while preserving geometry and material consistency.

Abstract: Text-guided color editing in images and videos is a fundamental yet unsolved problem, requiring fine-grained manipulation of color attributes, including albedo, light source color, and ambient lighting, while preserving physical consistency in geometry, material properties, and light-matter interactions. Existing training-free approaches provide broad applicability across editing tasks but struggle with precise color control and often introduce visual inconsistency in both edited and non-edited regions. In this work, we present ColorCtrl, a training-free color editing method that leverages the attention mechanisms of modern Multi-Modal Diffusion Transformers (MM-DiT). By disentangling structure and color through targeted manipulation of attention maps and value tokens, our method enables accurate and consistent color editing, along with word-level control of attribute intensity. Our method modifies only the intended regions specified by the prompt, leaving unrelated areas untouched. Extensive experiments on both SD3 and FLUX.1-dev demonstrate that ColorCtrl outperforms existing training-free approaches and achieves state-of-the-art performances in both edit quality and consistency. Furthermore, our method surpasses strong commercial models such as FLUX.1 Kontext Max and GPT-4o Image Generation in terms of consistency. When extended to video models like CogVideoX, our approach exhibits greater advantages, particularly in maintaining temporal coherence and editing stability. Finally, our method generalizes to instruction-based editing diffusion models such as Step1X-Edit and FLUX.1 Kontext dev, further demonstrating its versatility. Here is the website.

Published as a conference paper at ICLR 2026
TRAINING-FREE TEXT-GUIDED COLOR EDITING WITH MULTI-MODAL DIFFUSION TRANSFORMER
Zixin Yin Xili Dai
Ling-Hao Chen
Deyu Zhou
Jianan Wang
Duomin Wang
Gang Yu Lionel M. Nil
Lei Zhang*
Heung-Yeung Shum!
I The Hong Kong University of Science and Technology
2 International Digital Economy Academy
3 The Hong Kong University of Science and Technology (Guangzhou)
Tsinghua University "Astribot SSeepFun
ABSTRACT
Text-guided color editing in images and videos fundamental yet unsolved problem, requiring fine-grained manipulation of color attributes, including albedo, light source color, and ambient lighting, while preserving physical consistency geometry, material properties, and light-matter interactions. Existing training-free approaches provide broad applicability across editing tasks but struggle with pre- cise color control and often introduce visual inconsistency in both edited and non- edited regions. In this work, we present ColorCtrl, a training-free color editing method that leverages the attention mechanisms of modern Multi-Modal Diffusion Transformers (MM-DiT). By disentangling structure and color through targeted manipulation of attention maps and value tokens, our method enables accurate and consistent color editing, along with word-level control of attribute intensity. Our method modifies only the intended regions specified by the prompt, leaving unrelated areas untouched. Extensive experiments on both SD3 and FLUX.1- 1- dev demonstrate that ColorCtrl outperforms existing training-free approaches and achieves mate-of-the-aa performances in both edit quality and consistency. Fur- thermore, our method surpasses strong commercial models such FLUX.1 Kon- text Max and GPT-4o Image Generation in terms of consistency. When extended to video models like Cog VideoX, our approach exhibits greater advantages, partic- ularly in maintaining temporal coherence and editing stability. Finally, our method generalizes to instruction-bar based editing diffusion models such as Step --dit and FLUX.1 Kontext dev, further demonstrating its versatility. Here is the website
INTRODUCTION
Film industry-level color changing in images and videos based on textual instructions is a funda- mental yet challenging task in deep learning and visual editing. Here, "color" encompasses not only object albedo (i.e., intrinsic surface color independent of material properties), but also the color of light sources and ambient illumination. Color editing is an ill-conditioned problem, it requires explicit or implicit 3D reconstruction of the entire scene, including correct illumination. During edit- ing, it is essential to modify only the intended color attributes while preserving material properties, ensuring accurate reflections and refractions, and keeping non-editing regions unchanged.
Traditional image processing methods have been widely commercialized in professional software such as Photoshop. serving billions of users worldwide. However, the steep learning curve and significant manual effort required make it difficult to achieve widespread accessibility. Moreover, such tools are not well-suited for automated batch processing and cause problems when applied to video-related tasks. Recently, diffusion models have demonstrated remarkable capabilities in generating high-quality images that adhere to physical principles of color and illumination. This has spurred growing interest in leveraging their generative power to address the above challenges, with controllability emerging critical factor. Although many methods (Magar al. 2025 Zhang jet al.) 2025) tne-tune diffusion models for controllable editing, they usually require large-scale datasets and complex training pipelines, and often constrained narrow domains or specific edit

---

Published as a conference paper at ICLR 2026
"red... ... blue"
"green joll""
"hright morning"
red ball"
ddrk green"
"yellow
green"
"red Jelly"
dark night
yellow ball"
dark) green"
Figure 1: Text-conditioned color editing. Our method, ColorCtrl with FLUX.I-dew edits colors across multiple materials while preserving light- Light-matter interactions. For example, in the fourth case, the ball's color, its water reflection, specular highlights, and small droplets the glass have all been changed. It also enables fine-grained control over the intensity of specific descriptive terms.
types. On the other hand, training-free methods (Jiao et al. 2025) have gained popularity for a wide range of image editing tasks due to their generality and ease of use. Despite their success in many scenarios, they still struggle with fine-grained color control and often introduce inconsistencies in edited and non-edited regions, making precise color editing an unresolved challenge.
The recent architectural shift in diffusion models from U-Net (Rombach et al. 2022) to the Multi- Modal Diffusion Transformer (MM-DiT) (Esser al.) 2024) brings new opportunities for training- free color editing. for two main reasons: (1) the new architecture allows for scaling up both data and model capacity, enabling better adherence to physical priors; and (2) the improved fusion of text and vision modalities supports more flexible and precise attention-control editing strategies.
In this work, we propose ColorCtrl, a training-free open-world color editing method that effectively leverages pre-trained MM-DiT models to perform natural and precise color modifications, while preserving all other visual attributes, such as geometry, material, reflection and refraction, light source position, illumination direction, and light intensity, shown in Fig. [ For example, when editing the color of a ball, ColorCtrl accurately adjusts not only the ball itself but also its reflection in the water, the specular reflections surface, as demonstrated in the fourth example of Fig. [ Furthermore, ColorCtrl supports fine- grained, word-level control over the strength of color attributes while preserving the other visual attributes mentioned above. Thanks to our precise and robust control, ColorCtrl requires no manual parameter tuning and can be directly applied across all attention layers and inference timesteps.
both sides, and even the small water droplets
the glass
To verify the effectiveness of our method, we conduct extensive experiments showing that Col- orCtrl outperforms existing training-free approaches on both Stable Diffusion Medium (a.k.a,, SD3) (Esser et al. (2024) and FLUX. (Labs) 2024)). To better understand the gap between open- source and commercial models, we further compare ColorCtrl with two strong commercial base- lines: FLUX.1 Kontext Max (Black Forest Labs 2025) and GPT-4o Image Generation (a.ka.,, GPT- 40) (OpenAl, 2025). ColorCtrl achieves more natural color editing and substantially better preser- vation of visual consistency, including background and structural fidelity. In addition, method model-agnostic and can be seamlessly extended to video models such as VideoX (Yang al.) 2024]. where the performance gap becomes even more pronounced. Finally, ColorCtrl can be in- tegrated into instruction- based editing models such as Step (Liu et al. (025b) and FLUX.1 Kontext dev (Black Forest Labs 2025). demonstrating strong compatibility.
In summary, the main contributions can be listed as follows.
We propose ColorCtrl, a training-free method for color editing that supports modification of albedo, light source color and ambient lighting, while preserving the consistency of geometry, material and light-matter interaction.
We present extensive experiments demonstrating that ColorCtrl achieves state-of-the-art art performance among training-free methods on SM--DT-baaed models. Compared com-
2

---

Published as a conference paper at ICLR 2026
mercial models (i.e., FLUX.1 Kontext Max and GPT-4o Image Generation), method delivers significantly better consistency preservation and produces more natural edits.
ColorCtrl generalizes well across multiple MM-DiT-based models, including video and instruction-basse editing models, highlighting its broad applicability and extensibility.
2 RELATED WORK
Text-to-image and video generation. Diffusion models with a U-Net backbone (Ho
2020
Rombach et al. 2022 Guo et al. (204) have largely replaced early GAN systems (Reed et al.
2016
Yu et al. 2023 Wang et al,, 2023) due to superior image fidelity. However, the U-Net design scales
poorly, leading to the adoption of Diffusion Transformers (DiT) (Peebles & Xie 2023). Among
these, MM-DiT (Esser et al. 2024) has emerged as a widely adopted backbone for recent state-
of-the-art models (Esser et al. 2024 Al 2024 Labs
2024 Yang et al.) [022| Kong et al. 2024
Liu et al. 2025a Ma et al. 2025| Black Forest Labs 2025),
such as SD3 (Esser et al. 2024) and
#LUX.1-dev (Labs) 2024) for image generation, well as CogVideo.) (Yang et al. (0224) for video
generation. In this work, we propose an attention control that plugs into any MM-Di1 DiT model.
Text-guided editing. Training-free text-guided editing methods based on pre-trained diffusion
models offer flexibility and efficiency. Existing approaches fall into two groups: (i) sampling-
based methods, which steer generation by injecting controlled noise or inversion (Tiao et 2025
Huberman-SSpieellee
et al. [024]] Xu et al.] [003] Kulikov et al., [024] Yan et al. 2025 ) (ii)
allention-bused methods, which modify attention maps, starting with Prompt-to-Promee Prompt (Hertz
et al. 2023) and its image/video variants (Chen et al. 2024; Wang et al. 2025 Liu et al. 2024
Cao et al. 2023 Cai et al.) 2025, Rout et al.J [0255] Xu et al. 2025; Yin et al. 2025). Among
these, DiTCtrl is the only method exploring attention control in MM-DiT, but ColorCtrl differs in
key ways: DiTCurl only applies mask extraction during long video generation, not editing, while improved method is more robust. Additionally, DiTCtrl's re-weighting disrupts attention con- sistency, causing inconsistent geometries, and TexiCrafter (Du et al. 2025) exhibits a similar issue, while ours avoids this. Furthermore, ColorCtrl works across all layers and timesteps, unlike DiTCtrl, which requires careful layer selection. Different from DiTCtrl, we operate in attention maps rather than key and value tokens. More recently, methods (Liu et al.) 2025b; Brooks al., (023) like FLUX.1 Kontext (Black Forest Labs] (205) and GPT-4o (OpenAl] 2025) have shown convenient creative workflows in training with synthetic instruction-responee pairs for fine-tuning a diffusion model for image editing.
Despite advancements in text-guided editing, we demonstrate that current methods still face signif- icant challenges in accurately changing colors while preserving geometry, material properties and light-matter interaction. Also, all aforementioned training-free methods rely on selectively manipu- lating specific inference steps attention layers, which limits their robustness and consistency with respect to the source. In contrast, our approach requires no manual selection of steps or layers.
3 METHOD
followed by revisit of the attention mech- anism MM-DiT blocks in Sec. [.2]] Sec. 3.3 describes our approach for preserving geometry, material properties, and light-matter interactions. In Se4. we introduce method for preserving colors in non-editing regions, which are automatically detected by the model. Together, Sec. Sec. and Sec. constitute the core of our method. Finally, Sec.3.3. i.5. attributes via attention re- weiieigh,, serving as an optional add-on for fine- mne-grained attribute control.
We first formulate the color editing problem in Sec. 3.1
introduces word-level control over color
3.1 TASK FORMULATION: TEXT-GUIDED COLOR EDITING
Rendering process. To provide a clearer and more precise description of the conditions required for film industry-level color changing, define the rendering process of a frame by
I = R(G, L, A, S, C).
(1)
where the notations of rendering inputs are defined as,
R(-) the rendering function,
3

---

Published as conference paper at ICLR 2026
(a) Attention in MM-DIT Blocks
(c) Color Preservation
Klu
vislon-ta ta- ta-vision vision
to-fext
Prsision
griston
Mext-teeext
Qrea
texi-tovvision
gaoer
frees
(b) Structure Preservation
(d) Attribute Re- Weightigng
Krino
Kreur
Risiiaion
Kr
R Rvision
Rioar
R
Riear
Qenmi
Q
Q |
Great |
Q
M
M
Figure 2: Pipeline of ColorCtrl. (a) Visualizes the attention mechanism in MM-DiT blocks. (b) Enables color editing while maintaining structural consistency. (c) Preserves colors in non-editing regions. (d) Applies attribute re-weighting to specific tokens. Symbols in the source branch have no superscript. Symbols with superscript indicate the target, and hats (e.g.. V. A)) denote outputs.
G = .Gi,..., G a set of object geometries, egg. mesh topology for N objects,
L {Lenv Lı.... Lx] the ambient illumination Lew and K light sources (each with posi- tion, intensity, and spectrum),
A (A Ax} } the albedos (base colors) each object,
S {81 ...,, Sx} material parameters (roughness, specularity, and normal maps) for each object that are color-independent,
C camera intrinsies and extrinsics.
Editing task. According to the predefined image rendering process in Eq. ( we formulate the consistent editing task this work as follows. Given a source image and text prompt pairs & before and after editing" that specifies which objects lights recolor and the desired target color, aim to learn the following editing function f(-)
(I. i= (=R
such that satisfies:
(CI) Geometry/view consistency and remain fixed with the source image I, preserving object layout and perspective.
(C2) Illuiinaiion consistency
light positions and scalar intensities remain fixed. Specifically, changes may occur only the target spectral components (i.e., RGB channels) of L.
(C3) Material consistency the object material S remain fixed. The edits apply only to the albedo components A of specific objects.
In the scope of the defined editing task in Eq. 2 the editing function f(-) needs to internally infer the underlying scene parameters and localize the editing objects, lights, and regions to perform precise, constrained color changes.
3.2 PRELIMINARIES: ATTENTION IN MM-DIT BLOCKS
The attention mechanism for vixtovision fusion in MM-DIT (Esser et al. 2024) differs from that in U- Net (Rombach al. 2022 In U-Net, cross anssetttention used for text guidance, while self-attention
example of q: "white fox." "orange fax." (source and target prompt respectively).
4

---

Published as a conference paper at ICLR 2026
Source
Ours
wio Color Preservation
text-to-tent Swap
w/
w/ pm Swap
Mask Extracted from Mark Estracted from vision-lo-test
tect-Io-risoon
FLUX I-dev
SD3
(a) Comparison of editing strategles
(b) Mask visualkzaionn w/o threshold
Figure 3: Top row: SD3 results; bottom row: FLUX.I-dev results. (a) The edit prompt is "white fox" + "orange fox".. Left to right: source image, our full method, without color preservation, with swapped text-to-text part in structure preservation, and with swapped / in color preservation. (b) The generation prompt is "a white fox in a forest", and the token for mask extraction "fox". From left to right: the mask extracted from vision-to-text parts, and from text-to-vision parts.
focuses visual content interactions, separately. In contrast, MM-DiT integrates text and vision by concatenating their tokens together and processing them jointly via sole self-attention. As illustrated in Fig. (a), vision and text tokens are fed into the i-th MM-DiT block at timestep during inference. After modulation, the block produces an attention map M and value tokens V, which are used to generate the updated tokens for the next block and timestep. The resulting attention map M can be divided into four parts: vision-to-vision (upper-left), vision-to-text (upper-right), text-to- vision (lower-left), and text-to-text (lower-right). These quadrants correspond different query-key token pairings. For example, the text-to-vision region represents attention scores computed between query tokens from the text modality Q and key tokens from the vision modality K vvisson, Similarly, the value tokens V are composed of a vision part Vvision and a text part Vieu The functional roles of each region are discussed in detail in the following sections.
3.3 STRUCTURE PRESERVATION
Following Hertz al. 2023 y Cao et al. (003)), we divide the editing process into a source branch and target branch. The source branch follows the original generation process, producing source image and storing intermediate attention outputs. The target branch reuses these stored variables to generate edited results. Starting from fixed random seed, the model can generate the desired ob- ject without applying any editing method. However, the resulting image often diverges significantly from the source, making it unsuitable for meaningful editing tasks that require structural consis- tency. To address this, all subsequent modifications are applied exclusively on the target branch, aiming improve consistency without disrupting the intended edit. The process of keeping G, S, C. light positions, and scalar intensities fixed (as in constraints preservation. We observe that the vision-to-vision part of the attention map inherently encodes rich knowledge about the parts of the scene that must remain unchanged. Given source attention map M, its vision-to-vision part transferred from M the corresponding part in the target atten- tion map M producing an edited attention map M that fully respects the structure preservation constraints, as described in Fig. (b).
(C3) is referred to as structure
3.4 COLOR PRESERVATION
Despite enforcing structure preservation, we observe that undesired changes such as color shifts can still occur in regions unrelated to the edit, as shown in Fig. (a). To further localize the edit and reduce inconsistencies, edits should be confined to the intended regions, while preserving all other areas. Motivated by the approach Ca et al. 2025), we first extract binary mask m from vision-to-text parts of attention maps with threshold which indicates the target editing region. In contrast to Cai et al. (2025). which averages the vision-to- -s-teetto-text and text-to-vision parts to obtain the mask, use only the vision-to-ter parts, they provide superior spatial localization for the target, unlike the eext-to-vision parts (see Fig. (b)). Based on m, we copy the value tokens from the non-editing regions in the vision part of the source V into the corresponding regions of the target
5

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative image results compared with training-free methods on PIE-Bench. Re-
sults for FireFlow on SD3 are omitted due to their consistency being worse than using fixed seeds.
SD3
FLUX.1-6
Method
Canny SSIM
BG Preservation
Clip Similarit
Canny SSIM
BG Preservation
Clip Similarity 1 Edited
PSNKT 1
SSIM
Whols
Edited
PSNK 7
SSIM
Whole
Fix seed
0,5787
20.44
0.8411
29.17
27.54
0.7180
22.32
0.8877
27.72
25.76
FizeFlow (Deng et
2025b
0.6078
19.19
28.63
27.24
0.8322
35.87
0.9717
25.84
23.56
RF-Solve Waan o 2015
0.6711
23.30
0.8906
27.43
25.96
0.8394
36.25
0.9715
25.83
23,58
SDEdi: MMmnr
0.6353
27.78
0,8699
25.98
24.47
0.8285
32.62
0.9605
25.54
23,17
DITCul C3 al 2025
0.8119
35.40
0.9812
26.21
24.67
0.8306
34.48
0.9791
25.89
23.58
FlowEdir (Kirkoo 2024
0.7852
34.24
0,9704
26.13
24.97
0.8639
32.64
0.9774
25.95
23,63
UniEdir Uiior
0.8016
36.31
0.9774
26.08
24.67
0.8498
37.57
0.9777
23.44
Ours
0.8473
42.93
0,9960
28.32
26.96
0.9196
39,49
0.9936
27.34
24.90
V*, yielding the final value tokens V, as shwon in Fig. 2 (c). We refer to this procedure as color
preservation. As shown in Fig. 3 (a). including the text part of value tokens Viesi during value exchange significantly weakens text guidance in FLUX.11dde, and leads to severe artifacts in SD3.
3.5 ATTRIBUTE RE- NE-WEIGHTING
So far, our method enables powerful color editing capabilities. However, the granularity of control via text prompts remains limited. For example, if the user specifies color like "dark yellow", there is no way to explicitly control the degree of darkness. Existing re-weighting techniques in U-Net-based models primarily follow two approaches: (1) Scaling the text embedding of specific tokens before the diffusion process (Perry) 2023). However, this method is designed for CLIP- based (Radford al. (202I) text encoders and is incompatible with MM-DiT-based models, which typically use T5 (Raffel et al., 2020).
(2) Scaling the attention scores of specific tokens in the aotssnattnntoon map (Hertz et al.) 2023). This is also inapplicable to MM-DiT, which relies solely on self- attentoteniio. Moreover, both existing approaches fail to maintain structural consistency during scaling, limiting their utility high- figh-fideiity editing tasks.
To support more fine-grained and user-friendly control in MM-DiT-based models, we introduce mechanism to modulate the strength of specific attribute words. Specifically, we scale the attention scores in the text-to-vision parts of the attention map corresponding to the selected word tokens before the softmax operation, as illustrated in Fig. 2 (d). This modification can be applied either directly on the original source attention maps or on the target attention maps that have already undergone structure preservation, allowing flexible integration in above editing pipeline.
EXPERIMENTS
4.1
SETUP
Baselines. We compare method against several training- -frne approaches built upon MM-DIT,
including FireFlow (Deng et al. 2025b), RF-Solver WWang et al. 2025), ) SDEdit (Meng et al. 2022
DiTCurl (Cai et al. 2025), FlowEdit (Kulikov al. 2024). and Unitdit Flow (Jiao al. 2025) We
focus exclusively on MM-DiTTbasee baselines, as prior work (Jio et al. 2025 Deng et al.] 2025b)
has shown that U-Net-based methods perform significantly worse. Accordingly,
exclude methods
that cannot be adapted to the MM-DiT architecture. For commercial baselines, we compare with
FLUX.1 Kontext Max (Black Forest Labs 2025) and GPT-4o Image Generation (OpenAl 2025).
Implementation. We conduct experiments on SD3 (Esser et al. 2024) and FLUX.I-dev (Labs) exweeteeepeeeeeeee
2024] for image generation, and on VogVideoX-2B YYang et al.J (024) for video generation. For instruction-based image editing, we use XtepiX-Edit (Liu et al. 2025b) and FLUX.1 Kontext dev (Black Forest Labs 2025). For FLUX, Idev. attention control to the single-stream attention layers, following Deng et al. (2025b). Unless other- wise noted, we use the Euler sampler and adopt UniEdit- (Jioo et al. 2025) for image inversion. Maintaining balance between fidelity to the original image and the strength of the applied edits is a well-known trade-off in generative editing. To ensure a fair comparison across methods, we care- fully tune the hyperparameters for each baseline. Additional details are provided in Appendix A
and FLUX. Kontext dev, we apply

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative image results compared with commercial models PlE-Bench.
Method
Canny SSIM
BG Preservation
Clip Similaniy
Canny SSIM
BG Preservation
Clip Similrity Whole
PSNR
SSDM
Whole
Edited
PSNR 1
SSIM
Edited
FLUX.. Kontext Max [Back Forest Labs 2025
0,73015
31.97
0.9254
28.93
27 30
0.7607
26.77
0.9165
28.36
26.10
GPT-44 Image Generation
28.08
Ours
GPT-4u
UniE
FireFlow
2000""
Taklee
1
Tnd
erzage
tnte'
abae
Figure 4: Qualitative image results compared with training-free methods and commercial mod- els PIE-Bench. The top three rows are generated using FLUX.I-dev, while the bottom two generated using SD3. Best viewed with zoom-in.
Benchmark. While prior editing methods (Hertz et al.) 2023 Cao 2023 Cai al. 2025)
typically lack standardized benchmark evaluation, we adopt prompts from the Change Color task in PlE-Bench (Ju al. 2024), which includes 40 editing pairs, better showcase the capabilities of our method. Although our approach is fully compatible with inversion methods, we adopt a noise- to-image setting during benchmark evaluations to better isolate and evaluate editing performance, removing the influence of reconstruction and inversion. This also allows to reuse the same set of benchmark prompts for video diffusion models, enabling a consistent and fair comparison across benchmark
both image and video domains. To further scale up the evaluation, called ColorCtri-Bench, consisting of 300 prompt pairs in the same style as PlE-Bench (see Ap- pendix A.4 for details). For all baselines, adopt fixed sampler and identical random seeds to ensure that source images are consistent, enabling reliable comparison across methods.
introduce
Evaluation protocol. Unlike the original PIE-Bench, which evaluates structural similarity using structural distance (Tumanyan al. 2022), we adopt the Structural Similarity Index (SSIM) (Wang et al. 2004) computed on Canny edge maps (Canny 1986), following the approach of Zhao et al. (2023). for more accurate assessment. To evaluate the preservation of non-edited regions (a.k.a., BG Preservation). we compute PSNR and SSIM exclusively on those regions, which are annotated using Grounded SAM (Ren et al. 2024) with dilation. Semantic alignment of the edits assessed using CLIP similarity (Radford al. 2021), applied to both the entire image and the edited regions.
4.2 COMPARISON WITH TRAINING-FREE METHOODS (IMAGES)
Tab. and Tab. 5 report benchmark results on both SD3 and FLUX.1-den comparing method with other training-fri free baselines. Our method achieves state-of-the-art performance, delivering su- perior results in both preserving source content and executing accurate edits. Fig, 4 further supports this finding: other methods exhibit limited capacity for color editing and often introduce visual in- consistencies, while produces coherent and faithful edits. Additional results are in Appendix

---

Published as a conference paper at ICLR 2026
4.3 COMPARISON WITH COMMERCIAL MODELS (IMAGES)
Tab. 2 and Tab. 6 compare our method (based on SD3 and FLUX.I-dew with two commercial
models: FLUX.1 Kontext Max (Black Forest Labs 2025) and GPT-4o Image Generation (OpenAl 2025). Despite being based on open-source models, our approach achieves superior layout and detail consistency, as well as better preservation of non-edited regions. While the CLIP similarity scores of our method are slightly lower, visual results in Fig. 2 reveal that the commercial models often rely on over-saturated, unrealistic edits to better align with prompts. For example, in the top row, FLUX.1 Kontext Max recolors the entire mouse, including its magic wand, in solid purple, while GPT-40 produces a dark, dissonant shade. In contrast, our method applies a harmonious color. In the second row, only our method preserves the shirt's s semi-transparency, whereas the commercial models render it as opaque yellow, igmoring material properties. In the third row, our method respects the muted tone of "green tea" and edits the ice cream to natural reddish-brown "red tea" color. The commercial models, however, apply an unnaturally pure red. In the final row, although the prompt requests a "yellow kitten",, instead generates a kitten with the closest plausible fur color, aligned with real-world appearances, unlike the commercial models, which apply flat, high-saturation tones that appear unnatural. These results demonstrate that higher CLIP similarity does not necessarily indicate better edit quality. It often stems from prompt overfitting while compromising realism and consistency. Overall, our method consistently produces more faithful, controllable edits, even built on open-source models.
naturally occurring cat has truly pure yellow coat. Our method
4.4 COMPARISON WITH TRAINING-FRES METHOSSS (VIDEOS)
Thanks to the training-free nature of our method, it can be seamlessly extended to video
Table 3: Quantitative video results compared with baselines on -BEcBe.ch.
editing tasks. Since FLUX.1 Kontext Max
and GPT-4o do not support video editing, we compare only with other training- training-free free methods. FlowEdit excluded
Methed
Canay SSIM
PENK
BG Pieserution
NS0M
Clip Simiartt
Whole
Ediitt
Fix
0.6728
20:73
0.8012
27.97
26.63
it only applies to rec- tified flow models rather than diffusion- -bssed models. Tab. 3 and Tab. 8 presents benchmark results Cog VideoX-2B (Yang et al.) (0244).
FieFFow
RF-Soluz Woate
Dane
50250
012673
0.2517
30.74
0.9683
0.97%)
25:40
25.42
24.08
24.18
Mbnr
0.7500
35.46
0.9699
34/64
21:31
D:TCa
23.70
27.00
25.57
Unitidn
25.25
24.13
25,06
Similar to image editing, our method outperforms all baselines. Notably, the performance gap be-
comes even more pronounced due to the added temporal dimension. Visualization results in Fig. 6 further highlight the effectiveness of ColorCtrl. Consistent with its performance in image editing, our method also handles challenging cases in video, such accurately reflecting the color change of the ice cream in the bowl's reflection. Please refer to Appendix B for additional comparisons.
4.5 ATTRIBUTE RE- WEIGHTING ANALYSIS
Although DiTCtrl (Cai et al., 2025) in- cludes a mechanism for attribute re- weighting, both the paper and results in Tab. suggest limitations. Specifically, the method struggles to achieve the in- tended color changes while maintaining consistency with the source image. More- over, their approach of scaling attention weights after the softmax operation violates the assumption that attention scores should sum to one, which leads to incorrect at- tention behavior. Since none of the base- lines can smoothly adjust attribute strength while simultaneously satisfying the three constraints (C3) we present results of our method alone in 5 These results show that our method not only supports re- weighting a single attribute within the same prompt, but also allows adjusting attribute
blue"
"dack
yallow
"light
greeat"
"derk
lilkt
Figure 5: Examples of attribute re-weighting. The top two rows are generated using FLUX.I-dey, while the bottom one generated using SD3.
8

---

Published as a conference paper at ICLR 2026
0 Ours
DiTCtrl
UniE dit-Flow
FireFlow
RF-Solver
Figure 6: Qualitative video results compared with training-free methods on PIE-Bench. The edit prompt is "green tea" "red tea"" Each shows three frames.
strength across different prompts (second row). Moreover, our method can re-weight multiple at- tributes simultaneously (third row). Overall, these results demonstrate that ColorCtrl enables smooth and controllable transitions in attribute strength, while preserving structural consistency across the image and maintaining color fidelity in non-edited regions, both SD3 and F1.UX.1-dev.
4.6 ABLATION STUDY
According to Tab. 4 starting from fixed random seed generation, we observe the highest CLIP similarity due to the lack of consistency constraints. However, this comes at the cost of very low scores Canny SSIM, as well as PSNR and SSIM in non-edited regions, indicating poor structural and visual consistency. Introducing the structure preservation component significantly improves all consistency metrics, suggesting that the geometric and material attributes are effectively maintained, also illustrated in Fig- [3 Adding the color preservation component completes method, which further enhances consistency, particularly in non-edited regions, while sacrificing almost no CLIP similarity. Overall, the results validate the effectiveness of each component in method.
4.7 COMPATIBILITY AND DISCUSSION
Real image editing. To apply ColorCtrl to real input images, integrate the inver- sion method from UniEdit-Flow (Jiao et al.] 2025) and replace the original editing mod- ule with our method. As shown in Fig. 7 our approach generalizes well to real-world inputs and matches its noise-to-image tomiiage perfor- mance on both SD3 and FLUX. 1-ev, pre- serving fine-grained consistency (e.g., subtle fabric wrinkles and shadows) while deliver- ing strong editing performance. Notably, in the top row, even when editing black cloth- ing, method accurately distinguishes ma- terial shading from cast shadows, resulting in Illuiination-consitter edits.
ELOLSEN
Mad
Figure 7: Examples of real image editing, Results generated with SD3 (left) and FLUX. 1-Uvx.1-dev (right).
Generalization to instruction- editing diffusion models. In addition to text-to-image and text-to-video models, ColorCtrl is also compatible with instruction-based editing diffusion models, such SteplX-Edi (Liu et al. 2025b) and FLUX.1 Kontext dev (Black Forest Labs 2025). Given a real input image and a target editing instruction, the model performs edits accordingly. However, performing second round of editing using the original model alone often leads to structural in- consistencies, such as distortions or shifts shadows and edges. By incorporating our method, the model can further refine color edits while preserving structural fidelity in both edited and non-edited

---

Published as a conference paper at ICLR 2026
Real Euput Image
Real Input Image
ColeeCtri CoboC
ColorCtrl
Figure 8: Examples of results generated with StepIX-Edit (left) and FLUX.1 Kontext dev (right). Green arrows: first edit using the editing model. Blue arrows: second edit directly using the editing model conditioned on the result of the first edit. Orange arrows: second edit using the editing model with ColorCtrl. Top left: red diamond is added to the neck, then changed to blue. Bottom left: an orange cap added, then changed to purple. Top right: blue butterflies are added, then changed to yellow. Bottom right: a flashlight with blue light is added, then changed to green.
Table 4: Ablation study evaluating the effectiveness of each component on
SD3
FLUX.
Method
Canny SSIM
BG Preservation
Clip Simalarity
Canny SSIM
BG Preservation
Clip Similarity
PSNR
SSI
Whole
Edited
PSNR
SSIM
Whole
Edited
Fix seed
0.5787
20.44
0.8411
29.17
27.54
0.7180
22.32
0.8877
27.72
25:76
Structure Preservation
0.7312
24.77
0.9201
28.41
27.29
0.9019
30.20
0.9719
27.44
25.22
Color Preservation (Ours)
0.8473
42,93
0.9960
28.32
26.96
0.9196
39.49
0.9936
27.34
24.90
regions. As shown in Fig. 8 compared to using the base model alone, our approach achieves better outline consistency and improved preservation of subtle visual cues like shadows.
Additional results. Appendix B presents benchmark evaluations on ColorCtrl-Bench. limitation analysis, user studies, and downstream applications.
Discussion. Our method integrates structure preservation, regional color preservation, and word- level attribute intensity control into a unified, training-fiee pipeline. These components work to- gether to enable precise and consistent text-driven color editing. Each part of the attention com- putation plays distinct role: the vision-to-vision part of M preserves structure, the vision-to-text part is used for mask extraction, the text-to-vision part enables attribute reiihtth,, the vision part of V supports color preservation, and the text-to-text region of M along with the text part of V provides crucial textual guidance. As demonstrated by the degradation observed Fig. 3 (a) when the text-to-text region of M or the text part of V is altered, preserving the integrity of these parts is essential for maintaining robust and coherent generation that aligns with the target prompt.
CONCLUSION
We have introduced ColorCtrl, a training-free method for text-guided color editing that achieves fine-grained, physically consistent control over albedo, light source color, and ambient illumination. Our method is designed to edit only the intended visual attributes specified in the prompt, leaving all unrelated regions untouched. Built upon diverse MM-DiT-based diffusion models, such as SD3 and FLUX.1-dew, our approach enables fine-grained control over attribute intensity, while preserving geometry, material properties, and light-matter interactions. ColorCtrl not only outperforms prior training-free methods and achieves state-of-the-an results but also delivers stronger consistency than commercial models, i.e., FLUX.1 Kontext Max and GPT-4o Image Generation, on both quantitative and qualitative evaluations. Moreover, its model-agnostic design generalizes naturally to video dif- fusion models (i.e,, Cog VideoX) and instruction- baase editing diffusion models (i.e., Step1 X-Edit and FLUX.1 1 Kontext dev), highlighting its broad applicability. We believe ColorCtrl paves the way for scalable, figh-fideitty and controllable color editing in both research and practical deployment.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Advances in image editing technologies bring not only new capabilities but also ethical challenges. While our method improves the precision of color editing via text-based control, it may also be misapplied to produce deceptive or harmful visual content. To mitigate such risks, we advocate for responsible usage practices that prioritize transparency, user awareness, and informed consent in real-world applications. Moreover, since our method relies on pretrained models, it inherits any biases embedded in these models. Such biases may surface in the edited outputs in subtle or umin- tended ways. We consider this an open area for further investigation and support ongoing efforts to identify and reduce algorithmic bias. All human studies were conducted with voluntary participants who were informed of the goals of the study and provided explicit consent.
REPRODUCIBILITY STATEMENT
To support reproducibility, provide comprehensive details regarding the inference process and evaluation protocols in Sec. 3 Sec. 4.1 and Appendix A We are committed to releasing the com-
plete source code, including implementation scripts and necessary dependencies, upon acceptance. This will allow other researchers to reproduce our experiments and extend ColorCtrl future work.

---

Published as a conference paper at ICLR 2026
REFERENCES
Stability AI Stable diffusion 3.5.
2024.
Accessed: May 2025.
Black Forest Labs. Flux.1 1 kontext: Flow matching for in-context image generation and editing in latent space.
2025. Accessed:
2025-06-13.
Tim Brooks, Aleksander Holynski, and Alexei Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 18392-18402, 2023,
Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditctrl: Exploring attention control in multi-modal diffusion transformer for tuning-fiee foiing-free multi- prompt longer video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7763-7772 2025.
John Canny. A computational approach edge detection. IEEE Transactions on pattern analysis and machine intelligence, 8(6):679 698, 1986.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Mas- actrl: Tuning-free free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEF/CVF international conference on computer vision, pp. 22560-22570, 2023.
Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5343-5353, 2024.
Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, Xinghang Li, Jirong Liu, Yang Liu, Zhuoyan Luo, Jinsheng Wang, Wenxuan Wang, et al. Emu3. 5: Native multimodal models are world learners. arXiv preprint arXiv:2510.2658; 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie. Ziang Song, et al. Emerging properties in unified multimodal pretraining, arXiv preprint arXiv:2505. /4683, 2025a.
Yingying Deng, Xiangyu He, Changwang Mei, Peisong Wang, and Fan Tang. Fireflow: Fast inver- sion rectified flow for image semantic editing. In Forty-second International Conference on Machine Learning. 2025b.
Nikai Du, Zhennan Chen, Shan Gao, Zhizhou Chen, Xi Chen, Zhengkai Jiang, Jian Yang, and Ying Tai. Textcrafter: Accurately rendering multiple texts in complex visual scenes. arXiv preprint arXiv:2503.222 2...6. 2025.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning. 2024.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image dif- fusion models without specific tuning. In The Twelfth International Conference Learning Representations, 2024.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-or. Prompt-to-prompe image editing with cross-attention control. The Eleventh International Con- ference Learning Representations, 2023.
Jonathan Ho and Tim Salimans. Classifier-fre free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
12

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 6851, 2020.
Inbar Huberman- Spieeellllie Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 12469-12478, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
Guanlong Jiao, Biqing Huang, Kuan-Chich Wang, and Renjie Liao. Uniedit-flow: Unleashing in- version and editing the era of flow models. arXiv preprint arXiv:2504. 13109, 2025.
Xuan Ju, Ailing Zeng. Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pup inversion: Boosting diffusion-based editing with 3 lines of code. In The Twelfth International Conference on Learning Representations, 2024.
Raphi Kang, Yue Song, Georgia Gkioxari, and Pietro Perona. Is clip ideal? no. can we fix it? yes!
arXiv preprint 2rXhv:2503.08723, 2025.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. CoRR, 2024.
Max Ku, Dongfu liang, Cong Wei, Xiang Yue, and Wenhu Chen. Viescore: Towards explainable metrics for conditional image synthesis evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 12268-12290, 2024.
Vladimir Kulikov, Matan Kleiner, Inbar Huberman- Spieemmaln--ieggllas.... and Tomer Michaeli. Flowedit: Inversion-free text-based editing using pre-trained flow models. arXiv preprint arXi::4412.08622 2024.
Black Forest Labs. Flux. htttpss//gghhab..mmmlalll------e cessed: May 2025.
2024. Ac-
Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 8599-8608, 2024.
Shaoteng Liu, Tianyu Wang. Jui-Hsien Wang, Qing Liu, Zhifei Zhang, Joon- Lee, Yijun Li, Bei Yu, Zhe Lin, Soo Ye Kim, et al. Generative video propagation. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 17712-17723 2025a.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Steplx-edit: A practical framework for general image editing, arXiv preprint arXiv:2504. 177611, 2025b.
Yue Ma, Zexuan Yan, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung- Yeung Shum, et al. Follow-your-emoj-fasteeee Towards efficient, fine-controllable, and expressive freestyle portrait animation. arXiv preprint 1XXX:25509116300 2025.
Nadav Magar, Amir Hertz, Eric Tabellion, Yael Pritch, Alex Rav-Acha, Ariel Shamir, and Yedid Hoshen. Lightlab: Controlling light sources in images with diffusion models. In Proceedings of the Special Interest Group Computer Graphics and Interactive Techniques Conference Con- ference Papers, pp. 1-11, 2025.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022.
OpenAl.
Gpt 4o image generation. introducing-4e0 image - generation 2025. Accessed: 2025-06-13.
https://openai.com
13

---

Published as a conference paper at ICLR 2026
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Damian Perry. Compel: Prompt parser and conditioning tuning for diffusion models, https:
//gtthub.com// http.//../mmm.aan1///coaoaaoooo 2023. Accessed: 2025-07-01.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748 8763. PmLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-00 67, 2020.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In International conference machine learning, pp. 1060-1069. PMLR, 2016.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, al. Grounded sam: Assembling open- world models for diverse visual tasks. arXiv preprint arXiv:2401.14155 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Litu Rout, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Semantic image inversion and editing using rectified stochastic differential equations. In The Thirteenth International Conference on Learning Representations, 2025.
Narek Tumanyan, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Splicing vit features for semantic appearance transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10748-10757, 2022.
Duomin Wang, Yu Deng, Zixin Yin, Heung- Yeung Shum, and Baoyuan Wang. Progressive disentan- gled representation learning for fine-graing controllable talking head synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17979-17989, 2023.
Jiangshan Wang, Junfu Pu. Zhongang Qi, Jiayi Guo, Yue Ma. Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. In Forty-second International Conference on Machine Learning, 2025.
Zhou Wang and Alan C Bovik. Modern image quality assessment. Synthesis Lectures on Image, Video, and Multimedia Processing, 2(1):11156, 2006.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324 2025a.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang. Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint arXiv:2506. 18871, 2025b.
Pengcheng Xu, Boyuan Jiang, Xiaobin Hu, Donghao Luo, Qingdong He, Jiangning Zhang, Chengjie Wang, Yunsheng Wu, Charles Ling, and Boyu Wang. Unveil inversion and invariance in flow transformer for versatile image editing. In Proceedings of the Computer Vision and Pattern Recog- nition Conference, pp. 28479-28488 2025.
14

---

Published as a conference paper at ICLR 2026
Sihan Xu, Yidong Huang, Jiayi Pan, Ziqiao Ma, and Joyce Chai. Inversion-free image editing with natural language. CoRR, 2023.
Wufeng Xue, Lei Zhang, Xuanqin Mou, and Alan C Bovik. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index. IEEE transactions on image processing, 23(2): 684-695, 2013.
Zexuan Yan, Yue Ma, Chang Zou, Wenteng Chen, Qifeng Chen, and Linfeng Zhang. Eedit: Rethinking the spatial and temporal redundancy for efficient image editing. arXiv preprint arXiv:2503.10270 2025.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint arXiv:2408.0601 06002, 2024.
Zixin Yin, Ling-Hao Chen, Lionel Ni, and Xili Dai. Consistedit: Highly consistent and precise training-free visual editing. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers, pp. 1-11, 2025.
Zhentao Yu, Zixin Yin, Deyu Zhou, Duomin Wang, Finn Wong, and Baoyuan Wang. Talking head generation with probabilistic audio-to-visual diffusion priors. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7645-7655, 2023.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion-based illumination harmonization and editing by imposing consistent light transport. In The Thirteenth International Conference on Learning Representations, 2025.
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:1 1122-1 11 150, 2023.
15

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 INFERENCE SETTINGS
For benchmarking, we use 28 inference steps for both SD3 (Esser et al. 2024 and FLUX. with the classifier-free guidance (CFG) scale (Ho & Salimans 2021) set 7.5. All images are generated at resolution of 1024 x 1024. For VogVideoX-2B. we use 50 inference steps and set the CFG scale to generating videos at 49 frames with 720 480 resolution. A fixed random seed of 42 is used for all benchmark experiments. For real image editing, we use the latest inversion method from UniEdit-Flow (Jiao et al. 2025) and set the CFG scale according to the method.
The target object for mask extraction is determined using the "blended_ wblrnded_word" keywords provided by PIE-Bench (Ju et al. (024) and ColorCtrl- We use a mask threshold the same as DiTCtrl, which consistently yields strong performance across SD3, FLUX. 1-de,, and CogVideoX-25 2B. Despite being relatively coarse, this threshold is sufficient due to the strong global adaptation ability of modern generative models, which enables them to propagate edits from sparse color cues to semantically aligned regions.
= 0.1 which is
For Step Ix-Edii (Liu et al., 2025b) and FLUX.1 Kontext dev (Black Forest Labs 2025 their official code and setting with resolution 1024 x 1024.
we use
Image generation is performed RTX 4090 GPU, while video generation uses an A100 GPU.
A.2 SAMPLING DETAILS
To avoid redundant computation and accelerate inference, the source branch first executed to cache attention maps and value tokens for reuse, following similar strategy to that in Wann et al. (2025). During this stage, the editing mask m is also computed the actual editing phase, the cached features and mask are directly loaded, eliminating the need to recompute the source branch. This approach ensures that the editing process remains as efficient as standard sampling methods, without introducing any additional computational cost.
indicate the regions to be modified. In
A.3 IMPLEMENTATION OF COMPARED METHODS
Several compared methods lack official implementations for SD3 or CogVideoX or do not pro- vide compatible sampling code. To ensure fair comparison, we reimplement these methods SD3, I.UXX.Idey, and Cog VideoX-2B by faithfully following their original designs and carefully tuning hyperparameters to match the reported performance. Detailed implementations are as follows:
DiTCtrl (Cai et al. 2025): : For SDs-based image editing, we set the editing range from timestep 2 to 17, modifying the last 5 blocks. For FLUX.11dew) we edit from timestep 2 to 11 across the last 6 blocks. For video editing with Cog CogVideoX22B the official implementation used. During editing, key annd value (V) tokens from the source branch are copied into the attention layers of the target branch.
FlowEdit (Kulikov et al., 2024): We adopt the official image editing setting for FLUX-1-dev For SD3, we rescale the n n_max factor to account for the change in inference steps. Specifically, since the number of steps has changed from 50 to 28, we set n_max 28 : 50 x 33 x 18, while keeping all other parameters unchanged.
UniEdit-Floo Flow (Jiao et al., 2025): The official version supports SD3 and FLUX. 1 1-dev but provides the w parameter only for CFG Following the similarity transformation introduced in the paper, we use w = 5 + 7.5 R 0.6 and set = 0.6 for SD3, 0.85 for FLUX. Ieeev, and a 0.8 for video generation, which yields comparable performance to the original.
FireFlow (Deng et al., 2025b): In SD3, we observe that it is difficult to select suitable end timestep for FireFlow, as setting it too high often leads to artifacts and generation failures, as shown in Fig. 9 To mitigate quality degradation caused by excessive editing steps, we restrict editing to timesteps O through 3 across all blocks in SD3. In contrast, FLUX.1-dev does not exhibit this issue, and using the same range (timesteps 0 3) yields results comparable to those reported in the original paper. For video generation, the editing ends at timestep 9. During editing, value tokens (V) are copied from the source the target.
16

---

Published as a conference paper at ICLR 2026
RF-Solver (Wang al. 2025)
RF-Solver exhibits similar issue to FireFlow when applied to SD3, where higher editing timesteps lead to artifacts and degraded generation quality, as illustrated in Fig. 9 To address this, we limit editing timesteps 0 to 7 in the second half of the SD3 blocks. For FL.UX. 1-de, we set the end timestep to 4, which yields stable results without noticeable artifacts. Value tokens (V) copied from the source to the target. For video generation, editing is performed up to timestep 9.
SDEdit (Meng et al. 2022): We set ln 0.6 and apply editing to both generated and real input images or videos. The same parameter is applied across SD3, FLUX. 1-dee, and Cog C-
FLUX.1 Kontext Max (Black Forest Labs, 2025): Results are obtained using the official API, with source images and instruction prompts taken from PIE-Bench (Ju et al. 2024).
GPT-40 Image Generation (OpenAL, 2025): Results are generated through the official client
using the source images and instruction prompts from PIE-Bench.
Firef Result
FireFlow Resalt
Solver Result (Selected End T'meestp))
Ro-vvlr Result
Source
(Selected End limestep)
(Higher End l'meettp)
(Higher End Timestep)
"ublte fox"
"oannge fox"
Figure 9: Examples of FireFlow and RF-Solver with difference end timesteps on SD3. The selected end timestep refers to the setting used in the benchmark evaluation, while the higher end timestep denotes larger value chosen for comparison purposes.
A.4 COLORCTRL-BENCH CONSTRUCTION
Here, we provide details on the construction of ColorCtri-Bench. Following a similar approach to that in Ju al. (202)), prompt, a target prompt, subject token, and an instruction, consistent with the format used in PIE- Bench. These tuples represent an image before and after a color editing operation, along with an instruction describing the transformation and a subject token indicating the object to be modified. The exact prompting template is shown in Fig. 23
GPT generate dataset of tuples, each consisting of a source
A.5 EVALUATION DETAILS
For editing region masks, we first detect the target object with Grounded SAM (Ren 2024)
using the "blended. wordd in the PIE-Bench and ColorCtrl-Beer as the detection keyword. T he raw mask is then dilated to compensate for the uncertainty of the boundary, slightly enlarging the edited region. A marginally eroded mask is still adequate for BG preservation metrics, and a marginally dilated foreground does not bias CLIP similarity scores. Hence, this dilation neither undermines the consistency check nor suffers from the boundary instability of Grounded SAM 2. For video
evaluation, we compute each metric frame-wir wise and report the average over all frames as the final score for the video.
B MORE RESULTS AND ANALYSIS
B. DOWNSTREAM APPLICATION: SYNTHETIC COLOR EDITING AND FINE-TUNING
We generate 40k pairs of color-editing examples by applying ColorCtrl to FLUX.1 1-Xv,, and use this synthetic corpus to fine-tune Step1X-Edee 1-Edit-v1.1 (Liu et al., 20256) for 2,300 H800 GPU-hours. This downstream model shows clear gains on color-editing tasks, especially under complex light-matter interactions such as reflections and refractions, while better preserving object details. Leveraging the multilingual capability of Step Step1X-Ead EddEtit-vl..... with Chinese edit instructions, as shown in Fig. I0 This highlights our potential of building pairwise color editing data in the future.
also compare the original and fine-tuned models
17

---

Published as a conference paper at ICLR 2026
Diner
Source
1 1EA RDA CO
Instruction
Translation
woman siver gny hair
Clange the elderly
Change the newn sign's magenta cobalt biue
Replace the holognathi Earths eyon blee glow with nibyyrad
Change the donut's coloe silver
Edit this image: make the wall color cark
wilt prnk
Diner
StepIX-Edis v1.1
Diner
After
Fine-tune
Figure 10: StepIX-Edit-v' fine-tuned on Color Celegeettleee color-editing data vs. original. Results are produced with Chinese edit instructions and English translations are shown below.
Table Quantitative image results compared with training-free methods on Ctrl--BechB.h. Results for FireFlow SD3 omitted due consistency worse than using fixed seeds.
FLUX
Method
Canny
DG Preservation
Clip Similanty Edited
Canny SSIM
BG Preservabon
Clip Similanity
PSNR
Whole
Edited
Fix seed
0.6805
16.93
0,8303
28.27
26.65
0.7569
19.54
0.8552
27.70
26.25
FizeFlow
Deng al.
2025b
0.6625
13.87
0.7664
27.45
25.95
0.8553
32.80
0.9594
25.05
23.69
RF-Solve Wilni mim Pilas
0.7329
18.69
0.8596
26.70
25.19
0.8832
36.25
0.9736
24.89
23.49
23.77
DITCrl Carer 2023
0.8465
31.65
0.9723
25.25
23.82
0.8699
31.72
0.9707
24.90
23.53
FlowFdir Kallkow 2024
0.8261
30.77
0.9622
25.53
24.24
0.8704
29.29
0.9620
25.09
23.52
Unildit Now Dla al 2025
0.8503
32.74
0.9725
25.24
23.81
0.8725
34.85
0.9676
25.04
23.61
0.8775
38.16
0.9896
28.07
26.69
0.9324
37.96
0.9901
26.53
25.26
B.2
USER STUDY
total 24 expert participants conducted pairwise evaluations on image cases from PIE-BBenhh and ColorCtrl-Benee For each comparison, presentation order placement randomized and method identities were blinded. Raters chose the preferred result according to a predefined rubric (edit success, naturalness, background preservation). Overall, ColorCtrl was preferred in 64.71% of comparisons, outperforming all baselines (Tab. 7). The user-study interface is shown in Fig. 22
B.3 MORE RESULTS OF IMAGE EDITING
Tab. and Tab. present quantitative results of image editing tasks ColorCtri-Bench, compar- ing our method against both training-free baselines and commercial models. Visual examples are shown in Fig. 17 Consistent with our findings PIE-Bench, ColorCtrl outperforms other methods in maintaining object consistency and achieves accurate color edits with proper illumination and reflection. The similar conclusion drawn from this larger-scale benchmark further highlights the robustness and effectiveness of our approach.
Fig. 18 presents additional image editing results PIE-Bench, comparing method with both Trainin-ffres and commercial models. In the first column, only ColorCtrl successfully preserves the texture of the rocks, while other methods either fail to change the color or introduce texture distor-
18

---

Published as a conference paper at ICLR 2026
Table 6: Quantitative image results compared with commercial models ColorCtri-Bencls
FLUXX
Canny SSIM
Preservition
Clip Similaniyy
Caney
Similrity
PSNR
SSIM
Whols
Edhied
SSIM
PSNR T
S3IM
Whole
Edlied
FLUX.1 Kontial Maa. [Black Forest Labs 2025
0.8016
30.40
0.9552
28.23
26.83
0.8032
27.18
0.8854
27.56
26.24
GPPT-4 Image Genention OpenAl
0.6988
21.69
0,8030
28.34
26.66 66
0.7227
23,05
0.8371
27,65
26.24
Ours
0,8775
38,16
0.989
28.07
26.69
0.9324
37.96
0.9901
26.53
25.26
Table 8: Quantitative video results compared with base- lines on ColorCtrl-Benne Bench.
Table 7: User study results.
Method
Preference
2023
Method
SSIM
Canny
PSNR
BG Preservation
SSDM
Clip Similarity
Whole
EShied
Solver
Fix seed
0.6671
19.49
0.8489
26.01
26.66
UniEdii-Taww
FlowEdi
Kalhow Belle
FLUX. Komner Smt (blck K Fores Laie
GPT-4a Image Gesemfoo Opennt 2025
DrTChi
Cal 97763
2025
23.53 1446
11.76 831
UniEdit- FXW
SDEan
DTC
Mena of
2014
0.7270
0.8022
082281
35.33
26.14
34.30
0.1665
0.9613
0.9626
25.39
24.74
26.54
23.46
24.14
25.14
Ous
64.71
the morning"
"cyan shirt"
"orange light"
rrdd blue"
I
"in the night"
"purple shirt"
"blue light"
"yellow green"
Figure 11: Examples of image editing results.
In the third column, FLUX.1 Kontext Max and GPT-4o alter the pose and fabric wrinkles, whereas other training-fit free methods fail to perform the intended edit.
tions.
Fig. II shows additional image results with FLUX.I-des
B.4 MORE RESULTS OF VIDEO EDITING
Tab. shows quantitative results of video editing tasks on ColorCtri-Bente Fig. 19 and Fig. 20 present video editing results on ColorCtri -olorCtri-Bench, while Fig. 21 shows additional results on PIE- Bench. Our method consistently outperforms all baselines, achieving superior color editing accuracy and better structural consistency across frames.
B.5 ADDITIONAL. EVALUATION PROTOCOL
B.5.1 PRBUCTURRA--RRESEVVAAE METRICS
Beyond the Banny-SSIX, metric used in the main paper, we report two additional gradient- metrics to further verify edge and fine-detail consistency under strong color edits:
19

---

Published as a conference paper at ICLR 2026
Table 9: Additional quantitative image results compared with training-free methods on PIE-Bench
SD3
FLUX.1-4 --d
Method
Y-GCS 1
Y-GMST
SC
PQ1
Y-GCS
Y-GMS
SC 1
PQ f
Fix seed
0.5430
0.6978
5.300
7.475
0.49U5
0.7579
4.775
8.325
FireFlow (Dngg et al.
2025b
0.6105
0.7271
5,600
7.650
0. 0,8020
0.9107
1.875
8.550
RF-Solver (Wang al.
2025
0.7339
0.7891
5.525
8.075
0,8068
0.9127
1.825
8.675
SDEdit Meng al.
0.7339
0.8109
1,950
7.975
0.8797
0.8967
0.350
7,900
DiTCur (Ca
0.9074
0,9281
2,000
7.925
0.8553
0,9049
2.275
8,425
FlowEdit Kulikoy
2024
0,9027
0,9183
2.800
7.925
0,9061
0,9337
1.625
8.175
UniEdit-Flow 1100 et al.
2025
0.9034
0,9264
2,075
8.150
0.8320
0,9261
1.825
8.675
Ours
0.9244
0.9316
7.725
8.350
0.9555
0.9467
7.100
9.025
Table 10: Additional quantitative image results on PIE-Bench compared with commercial models on PIE-Bench.
SD3
FLUX.1 d-vev
Method
Y:GCS 1
YGGMS
SC 1
PQT
V.GCST
V-GMS
PQT
Fix sced
0,5430
0.6978
5,300
7.475
0.4905
0.7579
4.775
8,325
FLUX.I Kontext Max (Black Forest Labs 2025
0,7533
0.8104
8.750
7.393
0,5674
0.8079
8,525
7.625
GPT-4o Image Generation OppeaT 2025
0.5901
0.7435
8,923
6.769
0.5547
0,7906
8,900
7,050
0.9244
0.9316
7.725
8.350
0.9555
7,100
Y-Gradient Cosine Similarity (Y-GCS), where we convert both images YUV, keep only the luminance channel Y. compute Sobel gradient magnitudes, normalize them by the shared global maximum, and take the cosine similarity between the two gradient maps, capturing global alignment of edge and fine-detail patterns while being largely insensitive color changes.
Y-Gradient Magnitude Similarity (Y-GMS), where on the same luminance gradients we compute a per-pixel Gradient Magnitude Similarity (GMS) (Xue et al., (001)) map and average it over the image, providing a more localized and standard gradient-based IQA measure (Wang & Bovik 2006) of structural consistency.
B.5.2 EDITING-QUALITY METRICS
Beyond the standard CLIP similarity used in PIE-Bench, we observe that CLIP-based scores often exhibit over-saturated preferences and suffer from inaccurate attribute binding (Kang et al. 2025). making them suboptimal for disentangling the success of color-specific edits. Instead, following
recent instruction based image editing works (Liu et al. 20256 Wu al., [025a] Deng et al. 2025a
Wu et al. 2222bb] Cui et al. 2025), we adopt the VIEScore metrics (Ku et al. (024) from GEdit- Bench (Liu al. 2025b). which are originally designed evaluate a wide range of instruction- based editing tasks, including color editing. Therefore, we directly reuse their definitions without any modification. Concretely, we use:
Semantic Consistency (SC), which measures whether the intended edit has been correctly applied.
Perceptual Quality (PQ), which reflects the naturalness of the result and the absence of artifacts or over-saturation.
Each score ranges from 0 to 10 (higher is better) and is produced by a state-of-the-as multimodal L.LM evaluator, GPT-4c3 (Hurst et al. (2024). For video inputs, we uniformly sample three frames per clip, compute SC and PQ for each frame, and report the mean over the three frame-wise scores as the final scores for that video.
In combination, the prreerrallpressevattionn metrics (Y-GCS, Y-GMS) diagnose whether edits pre- serve structure and details, while SC specifically evaluates whether the color edit successful and PQ captures potential over-saturation unnatural editing artifacts.
API access of Nov. 2025
20

---

Published as a conference paper at ICLR 2026
Table 11: Additional quantitative video results compared with baselines on PIE-Bench.
Method
Y-GCS f
Y-GMS f
SC
PQT
Fix seed
0.5537
0.6532
4.375
6.708
FireFlow (Deng et al.
2025b
0.8278
0.8419
2.508
6.967
RF-Solver Wang
2025
0,8517
0,8545
2.542
6.850
SDEdit Meng et al. 2022
0.9167
0,9102
0.833
6.167
DiTCtrl CCa el al. 2025
0.7027
0.8396
4.933
6.258
UniEdit- Flow (Jao ct al.
2025
0.8446
0,8599
3.317
6.867
Ours
0.9383
0.9299
7.550
6.975
Table 12: Quantitative real image results compared with training-free free models on PIE-Bench.
Canny SSIM
BG Preservation
Clip Similarity
Method
Y-GCS
Y-GMS
SC
PQ
PSNR f
SSIM f
Whole
Edited
Pnplaversion Ju al. 2024
21.60
0.7785
26.10
21.41
0.8142
0.8106
3,650
FlowEdit Kahkkw 2024
0.6524
21.46
0.8324
25.71
21.05
0.7463
0.8222
3.275
7.700
Ours
0.7683
23.87
0.8890
25.88
20.52
0.9366
0.8969
5.450
8.425
B.5.3 RESULTS
Tabs. and report additional quantitative metries on PIE-Bench for SD3, FLUX.1-dex and CogVideoX-22 2B, comparing method against training-free baselines. Across all settings, ap- proach consistently achieves the highest Y-GCS and Y-GMS scores, further highlighting its superior ability to preserve edge structure and local details under strong color edits. In contrast, the low SC scores of the other baselines indicate that they rarely perform the intended color changes correctly, which is consistent with the qualitative comparisons in the main paper. This also shows that CLIP similarity alone is insufficient to reveal our advantage specifically on color editing.
Tab. 10 compares our method with commercial models. Although they can obtain relatively high SC, their lower PQ scores support our observation that they tend to over-edit and introduce over- saturated, unrealistic edits. Their low Y-GCS and Y-GMS further demonstrate that they fail to maintain structural and textural consistency.
Overall, these results show that existing training-free methods do not handle color editing well, highlighting that accurate color editing is a challenging problem. Our method is able to achieve high-quality color changes while strongly preserving structure without training.
B.6 MORE RESULTS REAL IMAGE EDITING
We further compare our method with the latest MM-DiT-based FlowEdit (Kulikov al. 2024 and the classic U-Net-based PnP- Invernoen (Ju et al. 2024) on real images from PlE-Bench. For both baselines, we observe that obtaining reasonable results requires carefully tuning which attention layers to edit and which timesteps to intervene at. The optimal choices vary across backbones (SD1.5, SDXL, SD3, FLUX. 1-dev and settings (noise-to-image vs. real-image editing), making their robustness and usability rather limited. For fairnsss, we therefore report results using the official implementations with their recommended default hyperparameters.
In contrast, our method can be applied to real images with exactly the same configuration as in the noise-to-image setting: we simply edit all layers and timesteps, without any changes. To the best of our knowledge, this is the first attention-contre editing method that does not require any hand-tuned hyperparameters when changing backbones or settings.
Quantitatively, Tab. 12 shows that our approach consistently and substantially outperforms both FlowEdit and InP-Inversion on sincttral-preservaaie metrics, including Canny-SSIM, Y-GCS, and Y-GMS, well as gegrroundd-reseervattie higher CLIP similarity, this must be interpreted with caution in light of the discussion in Sec. (.5.2 CLIP-based scores often exhibit over-saturated preferences and cannot reliably capture artifacts
metries. While PnP-Inversion sometimes achieves
21

---

Published as a conference paper at ICLR 2026
Source
Ours
FlowEdl
PnpInversion
white shirt"
'blue shirt"
"white mouse"
purple mouse"
"white
wall"
'red
wall"
Figure 12: Qualitative real image results compared with training-fire frriingg-eeeeee models PIE-Bench.
"Jeft red pills right blue pills"
"blue rabbit purple ball"
"white bshirt...
green dress"
"yellow and green chess pieces"
left green pills right... orange pills"
pink rabbit orange ball"
cyan t-shirt...
"red and blue chess pieces"
Figure 13: Examples of multi-object editing cases.
troduced by edits. Once consider SC and PQ, it becomes clear that Inversversoon in fact per- forms poorly at changing colors as instructed and often produces visually unnatural results, as illus- trated in Fig. 12 Overall, method remains state-of-the-art for real-image color editing, matching the strong structural preservation seen in the noise-to-image setting while achieving significantly better edit quality without any modification the inference settings.
B.7 MORE RESULTS OF MULTI-ORIECT EDITING
In addition to the first case in Fig. and the last case in Fig which already demonstrate multi- object editing ability, we provide further examples in Fig. As shown, our method can robustly handle scenes with many objects, such the first example where separate piles of pills, each
22

---

Published as a conference paper at ICLR 2026
Source
0.05
0.1 (ours) 0.2
Source
0.05
U.1 (ours)
0.2
"gronn Abirs"
"red hat
Min
Figure 14: Ablation study of the mask extraction threshold e, For better visibility, we visualize mask.
"eallsticc style"
made of rubber"
brand-new bicycle"
"a glass of water"
1
"cartoon style"
"ade of bronze"
"rusty bicycle"
"a glass milk"
Figure 15: Examples of additional applications enabled by reducing the preservation strength.
containing many pills of different color, are edited simultaneously. In the second example, multiple objects are spatially close even overlapping, yet our method still correctly assigns different target colors to each object. In the final example, we not only edit multiple chess pieces but also accurately update their corresponding reflections on the board, further illustrating precise multi-object editing.
B.8 ABLATION STUDY OF MASK THRESHOLD
The threshold for mask extraction is the only hyperparameter in ColorCtrl. We conduct an ablation study in Fig. 14 evaluate the sensitivity of our method to As shown, the performance not very sensitive to this parameter: around our default choice e = 0.1, using stricter threshold (E = 0.2) or more relaxed one 0.05) leads only minor visual differences. Even when we completely remove the mask extraction and color preservation modules (e 0), the structural details of the background remain unchanged. The main effect of using too small a threshold slight hue shift in some background regions, but the overall results are still visually coherent.
B.9 MORE APPLICATION AND ABILITY
ColorCtrl is designed to edit color while keeping geometry material and light-matter interaction strictly fixed. However, this does not mean it can only produce extremely conservative edits. In fact. by appropriately reducing the number of effective timesteps of structure preservation (i.e., weakening the preservation strength), we can obtain a variety different applications and effects while still preserving structure, illustrated in Fig. 15 including style transfer, material change,
23

---

Published as a conference paper at ICLR 2026
Edited Result
Edited Result (FLUX.1 dev)
Source
Source
(SD3)
red trees"
lipstick"
trees"
groen lipstick"
Figure 16: Examples of failure cases. Left: results generated with SD3. Right: results generated with FLUX.I-dev.
texture change, and transparency change. In the first example, real-image style turned into cartoon style while preserving geometry extremely well, without requiring separately trained ControlNet (Zhao et al. 2023). In the second example, smooth rubber pig is transformed into metallic bronze pig, showing characteristic metallic texture and reflections while its shape remains unchanged. the third example, rust-like textures are added to an originally brand-new bicycle. In the last example, change the transparency of the liquid in a glass, turning clear water into opaque milk, while keeping both the glass structure and the non-edited regions well preserved.
B.10 LIMITATIONS
The generation quality and the precision of text-guided localization in our method are fundamen- tally limited by the capabilities of the underlying generative models. As shown in Fig. [6] failure case with SD3 demonstrates that the model fails to correctly detect "red trees" and instead mod- ifies all trees, including green ones that should remain unchanged. Similarly, in the example with FLUX.I-dew. the model misinterprets "lipstick" and edits the casing rather than the lipstick itself. As foundation models continue improve, expect the performance and applicability of our method to advance accordingly.
Furthermore, editing real images and videos remains challenging due the limitations of current inversion and reconstruction techniques. Although our method performs reliably on data that lies within the training distribution of the generative model, handling weal-world inputs requires first mapping them accurately into the latent space the model. This mapping process still difficult and highly sensitive to the quality of the inversion.
C LARGE LANGUAGE MODEL (LLM) USAGE
We used an LLM to correct grammar, draft the web UI for the user study, and generate the ColorCtrl-Bench prompt list, All outputs were manually reviewed and edited by the authors.
24

---

Published as a conference paper at ICLR 2026
SD
Ro-SSller
FireF low
UniEdit-Floo Fow
FlowEdit
Cir
FLUX.1 Kontext
GPT-4o
Ours
Source
"bright
"orange umbrella*
"grenn plate"
pink gollen necklaoe"
"green plate"
"bright blue"
black umbrella
turqunise
plete"
"range
silver necklace"
plate"
Qualitative image results compared with training methods and commercial models on Ceenhh. The left four columns are generated using F1..X.1-dev, while the right three are generated using SD3. Best viewed with zoom-in.
Figure 17:
25

---

Published as a conference paper at ICLR 2026
SDEdit
RF-Solver
FireF low
dit-FittFloo
FlowE dit
Crr
FLUX.1 Kontext
GPT-4o
Ours
Source
bllck rocks"
"purple lilas"
shirt"
black bird"
brown cabin"
*white rocks
"orange liles"
"blue shirt"
"geen hird"
golden cabin"
Figure 18: Additional qualitative image results compared with training-free methods and com- mercial models on PlE-Bench. The left three columns are generated using FLUX. 1-dev, while the right two generated using SD3. Best viewed with zoom-in.
26

---

Published as a conference paper at ICLR 2026
SDEdit
RF-Solver
FireFlow
UniEditt
DITC
Ours
Sourre
Figure 19: Qualitative video results compared with training-free models on ColorCtri-Bende Left: "yellow bus" "green yellow". Right: "green backpack" "yellow backpack". Each shows three frames.
27

---

Published as a conference paper at ICLR 2026
SDEdit
RI Solvve
FireFlow
Uni Frrdit-FFlow Flow
DiTCH
Ours
Source
Figure 20: Additional qualitative video results compared with training-free models on
ColorCtrl-Bench. Left: "gray sofa" "brown sofa". Right: "brown suitcase" "black suit- case". Each shows three frames.
28

---

Published as a conference paper at ICLR 2026
SDE dit
R.F-Solver
11
FireFlow
UniEdit-Flon Flow
DIIC
Ours
Source
Figure 21: Additional qualitative video results compared with training-free models on PIE-
Bench. Left: "white shirt" + "blue shirt". Right: "white lamb" - "blue lamb". Each shows three frames.
29

---

Published as a conference paper at ICLR 2026
ColorCtri Image Editing User Study
This exsuates difeoe rethede fee levgel
Jearcs utonal imae
Anarymaa insults nasuaa affaret serbccu (aceed thougss unecar slccteg
gropp
Oriemal cevint new bumeen nes infortiiul of shoule oecomvve
inoud loot rssent irablicc
tedar cendarin
fighaa Fromat:
pessostion sollem under on socooad
Eeltina ierruction
Coonge
salling
Meched Peratss Solect Besk One
Eameleo T2d 24
Cannc's
Figure 22: User interface for user study.
30

---

Published as a conference paper at ICLR 2026
Please generate a JSON list of 300 sets. Each set consists of: a source prompt, a target prompt, a instruction, and subject token.
The source prompt describes source image. The target prompt describes the source image after the color of an object has been changed.
The instruction is description of what needs to be changed to go from the source to the target prompt.
The subject word is the noun that refers to the object that changed color, a single word that appears in the source prompt.
Here is an example:
"sre :rre_propt": "A person wearing a blue shirt sitting on a chair", "tgt_ : "A person wearing a yellow shirt is sitting on a chair", "subject word": "shirt",
"instruction": "Change the color the shirt from blue to yellow",
Only generate examples where there is clearly only one possible object to be changed, so it can be tagged correctly. Write as a JSON list yourself. Please DO NOT write code; Return only the JSON list.
Figure 23: The prompt used to generate ColorCtri- with GPT.
31


Reviews:
[{"review_id": "mC4PxyZos4", "reviewer": "Reviewer_Ld1H", "summary": "This paper proposes a training-free color editing method for images and videos based on pretrained T2I/T2V MMDiT models. It disentangles the color and structure within the edited region, and achieves accurate and natural color editing without touching unrelated regions or attributes. The proposed method is evaluated on extensive examples with various base models, surpassing strong commercial models. A new benchmarking protocol is proposed by adopting the PIE-bench prompts on generated images and adjusted metrics. The proposed method also applies to videos and instructional image editing models seamlessly.", "questions": "- For the failure cases described in Sec. B.5, would there be additional approach to improve? For example, would it be feasible to locate the editing region with the full object in the source image caption, i.e. use \"red trees\" or \"red lipstick\" instead of \"trees\" or \"lipstick\" to locate the desired region?\n\n- Would the proposed method be applied for other attribute editing like color that are uniform and doesn't change structures, e.g. textures, reflection/transparency? If so the scope of this paper could be largely extended.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The proposed method is dedicatedly designed for color editing and exclude other factors, achieving clean color editing results with faithfully maintained content.\n\n- The proposed method and benchmark highlights the importance of producing reasonable color edit with similar lighting etc. environmental conditions, over the traditional standard semantic CLIP alignment, making the output results more realistic and natural.", "weaknesses": "- [Major] The only editing quality metric is still CLIP similarity, which doesn't align with the claim that CLIP usually leads to over saturated preference as it lacks details. The major upgrade of metrics only focus on the structure preservation. Given the focus of the paper, some improved editing metrics are necessary to solidate the evaluation. For example, at least aesthetics/harmony can be tested to show the edited colors fit in the environment well. Maybe other color spaces like HSV could also help to decompose? Texture preservation can also be considered (e.g. semantic not impacted), while Canny is relatively rough especially when the threshold is not low enough. For CLIP similarity, it could also potentially help if it is calculated separately between color words and object words etc.\n\n- [Major] The proposed method is tested mainly on generated images instead of real images, and the proposed benchmark also emphasizes this. Although a dedicated section is provided for application on real images, the major quantitative results are compared on synthetic data. It is claimed that testing on generated images can calibrate the impact of inversion quality etc., but in practice it's still important to evaluate on real data, in terms of both performance and methodology. The performance of all methods will be impacted by inversion, while some methods might cooperate better or worse with inversion, and the difference matters for real usage.\n\n- UNet-based models sometimes feature their tighter spatial correspondence over transformers. One or two strong baselines could be involved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "io5mSWJtQ4", "reviewer": "Reviewer_QUHF", "summary": "This paper introduces ColorCtrl, a training-free framework for text-guided color editing using Multi-Modal Diffusion Transformers (MM-DiT). The method leverages attention maps from MM-DiT to extract semantic masks and reweight attention for localized color edits. The authors claim improved controllability and consistency without requiring model fine-tuning.", "questions": "1. Could you elaborate on how the semantic mask is extracted from MM-DiT attention maps? Which layers are used? What criteria or thresholds are used to binarize or localize the mask?\n2. How is the attention reweighting applied during inference? Is it uniform across all heads/layers or selectively applied?\n3. How sensitive is the editing quality to the mask threshold and prompt?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Originality: The idea of using MM-DiT attention maps for color-specific editing is novel in the context of training-free pipelines.The integration of word-level control adds granularity not commonly seen in prior works.\n2. Quality: The method is well-implemented and evaluated across multiple domains (images, videos, instructions). Results show high semantic consistency and localized edits, outperforming several baselines.\n3. Clarity: The paper is generally well-written, with clear motivation and structured methodology. Visual examples and comparisons are effective in illustrating the method’s capabilities.\n4. Significance: The approach is impactful for real-world applications where retraining is costly or infeasible. It contributes to the growing field of training-free multimodal editing, pushing the boundaries of what can be achieved with pre-trained models.", "weaknesses": "1. Limited Novelty: While the use of MM-DiT is novel for color editing, similar pipelines have applied attention-based editing in other contexts. TextCrafter also leverages attention maps from MM-DiT to extract semantic masks and reweight attention for image editing. Add-It also leverages attention maps from MM-DiT to extract semantic masks. \n2. Insufficient Analysis of Key Components: \n(1) The mask extraction process is underexplained: How are attention maps selected? What thresholding strategy is used? How robust is the mask across different prompts?\n(2) The attention reweighting mechanism lacks detailed discussion: How is the reweighting computed?\n(3) A more thorough ablation study would help isolate the contribution of each component (e.g., mask quality, reweighting strategy).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RKvMm0AW3P", "reviewer": "Reviewer_45WZ", "summary": "The paper presents ColorCtrl, a training-free, text-guided color editing method for images and videos built on Multi-Modal Diffusion Transformers (MM-DiT). The core idea is to disentangle “what should stay” (structure, materials, lighting geometry) from “what should change” (albedo or light color) by (i) swapping the vision-to-vision quadrant of the attention map from a source branch to a target branch to preserve geometry and view, (ii) extracting an edit mask from vision-to-text attention and copying vision value tokens outside the mask to preserve non-edited colors, and (iii) enabling word-level attribute intensity control by pre-softmax scaling in the text-to-vision attention region. Extensive comparisons on SD3 and FLUX.1-dev, plus CogVideoX for video, indicate state-of-the-art training-free performance on PIE-Bench and a new ColorCtrl-Bench; the method reportedly surpasses strong commercial systems (FLUX.1 Kontext Max, GPT-4o Image Generation) on consistency while remaining model-agnostic and compatible with instruction-based editors (Step1X-Edit, FLUX.1 Kontext dev).", "questions": "* How sensitive is performance to the mask threshold ϵ and to the choice of the “blended word” subject token? Please provide a small sensitivity analysis (e.g., ±0.05 around 0.1) and results for ambiguous subjects or multi-instance scenes. \n\n* The benchmark uses noise-to-image generation to isolate editing. Can you complement this with a real-image inversion benchmark that reports reconstruction error and edit success jointly, to reflect common editing workflows?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Originality: The paper adapts attention-control editing to MM-DiT with a clear decomposition of attention quadrants: vision-to-vision for structure preservation, vision-to-text for mask extraction, and text-to-vision for controllable attribute strength. This differs from U-Net cross-attention methods and prior MM-DiT controls (e.g., DiTCtrl) by operating directly on attention maps and value-token routing without training.   \n\n* Quality: The mechanism is well specified: two-branch unrolling with cached source features; mask from vision-to-text attention; copying only vision value tokens for non-edited areas; and pre-softmax token-specific scaling for attribute re-weighting. Ablations show that structure preservation and color preservation cumulatively improve consistency metrics with minimal loss in CLIP alignment.   \n\n* Clarity: The paper provides a task formulation grounded in a rendering-style decomposition (G, L, A, S, C), detailed pipeline figures, and explicit implementation/inference settings (steps, CFG, mask threshold, fixed seeds). These details aid reproducibility.", "weaknesses": "* Masking and subject detection reliance: The evaluation and parts of the pipeline hinge on subject keywords and a fixed attention-threshold ($\\epsilon=0.1$) for mask extraction; robustness to threshold choice, ambiguous subject words, or multi-object scenes is not deeply analyzed.  \n\n* Claims versus limitations: The paper acknowledges failures when the base model mislocalizes targets or confuses attributes (e.g., trees or lipstick casing). More systematic characterization of such failure modes—especially under crowded scenes, glossy materials, or colored lighting—would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)