LightCtrl: Training-free Controllable Video Relighting
Authors: Yizuo Peng
Xuelin Chen
Kai Zhang
Xiaodong Cun
Venue: ICLR 2026 Poster (2026)
Keywords: video relighting; controllable video editing
Primary area: generative models
TL;DR: LightCtrl relight a given video via a user-specific light trajectory in a zero-shot manner.

Abstract: Recent diffusion models have achieved remarkable success in image relighting, and this success has quickly been reproduced in video relighting. Although these methods can relight videos under various conditions, their ability to explicitly control the illumination in the relighted video remains limited. Therefore, we present \name, the first controllable video relighting method that offers explicit control over the video illumination through a user-supplied light trajectory in a training-free manner. This is essentially achieved by leveraging a hybrid approach that combines pre-trained diffusion models: a pre-trained image relighting diffusion model is used to relight each frame individually, followed by a video diffusion prior that enhances the temporal consistency of the relighted sequence. In particular, to enable explicit control over dynamically varying lighting in the relighted video, we introduce two key components. 
First, the Light Map Injection module samples light trajectory-specific noise and injects it into the latent representation of the source video, significantly enhancing illumination coherence with respect to the conditional light trajectory. 
Second, the Geometry-Aware Relighting module dynamically combines RGB and normal map latents in the frequency domain to suppress the influence of the original lighting in the input video, thereby further improving the relighted video's adherence to the input light trajectory. 
Our experiments demonstrate that \name can generate high-quality video results with diverse illumination changes closely following the light trajectory condition, indicating improved controllability over baseline methods. The code will be released at: https://github.com/GVCLab/LightCtrl.

Published as a conference paper at ICLR 2026
LIGHTCTRL: TRAINING-FREE CONTROLLABLE VIDEO RELIGHTING
Yizuo Pengl. Xuelin Chen', Kai Zhang' Xiaodong Cun
Tsinghua University, "GC Great Bay University, "Adobe Research pengyz23emails, tsinggua....... edu. en, cun gbu edu.cn
Source
Relighted
Relighted
curty natural Lighting, kare otsosphere"
the grass, moonlight glov
Source
Seurce
Relighted
Relighted
white black
coin on the desk, natural Lighting
Figure I:
LightCtrl can relight input video to produce high-quality results with strong temporal consistency and, particularly, illumination that closely follows the user-specified light trajectory.
ABSTRACT
Recent diffusion models have achieved remarkable success in image relighting. and this success has quickly been reproduced in video relighting. Although these methods can relight videos under various conditions, their ability explicitly con- trol the illumination in the relighted video remains limited. Therefore, we present LightCtrl, the first controllable video relighting method that offers explicit control over the video illumination through a user-supplied light trajectory in a training- free manner. This essentially achieved by leveraging hybrid approach that combines pre-trained diffusion models: pre-trained image relighting diffusion model is used to relight each frame individually, followed by a video diffusion prior that enhances the temporal consistency of the relighted sequence. In partic- ular, to enable explicit control over dynamically varying lighting in the relighted video, we introduce two key components. First, the Light Map Injection module samples light trajectory-specifi noise and injects it into the latent representation of the source video, significantly enhancing illumination coherence with respect to the conditional light trajectory. Second, the Geometry-Awand Relighting mod- ule dynamically combines RGB and normal map latents the frequency domain to suppress the influence of the original lighting in the input video, thereby further improving the relighted video's adherence to the input light trajectory. Our ex- periments demonstrate that LightCtrl can generate high-quality video results with diverse illumination changes closely following the light trajectory condition, indi- cating improved controllability over baseline methods. The code will be released at: http:///ggaaaaeeeec/////aaaaaaaa
"Corresponding authors

---

Published as a conference paper at ICLR 2026
INTRODUCTION
The ability to controllably edit illumination in images and videos is highly desired feature in visual content creation. With the rapid advancement of generative models, image relighting has achieved
remarkable success. These image relighting pipelines (Sun et al. 2019; Nestmeyer al., 20200 Pandey et al.
2021, Zeng et al. 2024 Bharadwaj et al. [2024] He et al. 2024 Ponglertnapakorn eet 2023 Ren et al. 2024; Kim et al. (024) typically involve a complex traditional synthesis pipeline that adheres to the desired relighting conditions. More recently, the state-of-the-art im-
age relighting model IC-Light (Zhang ct al., 2025) directly fine-tunes a pre-trained diffusion-baser
image generator conditioned various input lighting scenarios.
The success is quickly reproduced in video relighting, where, analogically, video diffusion mod- els (VDMs) are repurposed for relighting existing videos. RelightVid (Fang et al.] (025) constructs a high-quality video dataset featuring diverse lighting conditions to train the VDM. Nevertheless, the data crafting is often prohibitively expensive. This limitation has prompted the development of training-free methods, such as Light-A- Sigdt---Vidoo (Zhou et al. 2025), which progressively integrates in- dividually relighted frames, produced by pre-trained image relighting model (Zhang et al. 2025).
into the denoising process of a VDM.
These existing approaches (Fang al. 2025 Zhou al. 2025 often overlook the subtle yet crucial impact of localized light adjustments on the narrative and emotional expression within videos. We argue that specific local light change in video frames can significantly enhance the storytelling atmosphere, as it allows for more nuanced manipulation of visual focus and mood. For instance, a gradual dimming of light in a particular scene corner can emphasize a character's 's inner tension, while a sudden beam of light can highlight pivotal plot moment. To address this gap and mock the dynamic changes of light and shadow in videos, we define a controllable video relighting task. This task aims to provide fine-grained control over frame-wist lighting, enabling creators to tailor light effects to the narrative flow of each scene. By introducing procedural synthesis techniques, we seek to achieve precise, frame-by by-ffrmme light control, which can simulate real-world lighting dynamics such as the slow transition of dawn or the specific highlight region.
We thus present LightCtrl, the first controllable video relighting method that produces high-quality results with explicit control over the video illumination through a user-supplied light trajectory. Sim- ilar to previous work (Zhou et al. 2025 Meng al. 202I), LightCtrl based on the video editing pipeline and adopts a hybrid approach that combines pre-trained diffusion models: a pre-trained image relighting diffusion model is used to relight each frame individually, followed by a video diffusion prior that enhances the temporal consistency of the relighted sequence. In particular, to achieve controllability of video relighting through user-defined light trajectory. Light Map Injec- tion module is devised. Specifically, we begin by synthesizing sequence of light maps that align with the lighting trajectory defined by the user. This can be straightforwardly achieved through pro- cedural synthesis, or users have the flexibility to manually author the light map sequence. Then, we inject these light maps as control signals into the noise latent of both the single-frame relighting model and the video diffusion model. The Light Map Injection module significantly improves the adherence of the lighting in the generated video with respect to the input light trajectory. On the other hand, the illumination in the source video could greatly affect the relighting results, hence we introduce the -earetry-Aware Relighting module, which takes as input the estimated normal map sequence of the source video, to suppress the influence of the source illumination. Additionally, to preserve the details in the source video, we dynamically fuse the RGB video and normal maps latents in the frequency domain, obtaining the best of both worlds. Experiments demonstrate that our training- mareeng-free method enables coherent and controllable video relighting across a wide range of source videos, faithfully following diverse lighting trajectories specified by the user.
2 RELATED WORK
CoomiinationCControll Visual Generation. Controllable illumination synthesis has become crucial research direction in visual generation. Early efforts employed deep neural networks to con- trol illumination light stage data (Sun et al. 2019) and enhanced neural network relighting by incorporating physical priors (Nestmeyer et al., 2020). Total Relighting (Pandey et al. (02T)) op- timized the Phong model using HDR lighting maps. With the advent of diffusion-based models,
2

---

Published as a conference paper at ICLR 2026
methods such DiLightNet (Zeng et al., 2024), Gen (Bharadwaj et al. 2024). DifFRelight (He)
et al. 2024). and DiFaReli (Ponglertnapakom et al. 2023) have emerged, achieving detailed relight-
ing. However, applying image-based methods such as Relightful Harmonization (Ren et al.) 2024),
Switchl (Kim et al., 2024), and IC-Liight (Zhang et al. 2025) to videos often results in tem- poral inconsistency. In video illumination control, methods like EdgeRelight360 (Lin et al. 2024) and ReliTalk (Qiu et al., 2024a) addressed temporal consistency using 3D-aware models. Genera- tive portrait-relightnng works such as Lumos (Yeh et al. 2022) and related methods (Zhang et al. 2021 Cai et al.) 2024) also achieve impressive relighting results by injecting illumination through environment maps. However, their strong dependence on environment-me map inputs limits practical applicability. Training-based methods like RelightVid (Fang et al. 2025) achieve coherent video re- lighting through bigh-quality datasets, while training-free methods like Light-A- Light-A-Video (Zhou al.) 2025) leverage image relighting techniques and video diffusion models' motion priors. Despite these advances, no existing method can yet achieve flexible video relighting based on user-defined lighting trajectories.
Video Editing and Video Difftusion Models. Recent years have witnessed substantial advance- ments in video generation, particularly through the application of video diffusion models (Blattmann) et al. [2023] Guo et al.) [203]] Yang et al. 2024; Xing et al. [024)) which are capable of produc- ing high-quality, physically plausible videos. In the domain of text-to-video generation, various strategies have been explored: some methods integrate motion modeling modules into existing text-
to-image (T2I) architectures (Guo et al.] 2023] Chen al.] 2023 2024; [Wng et al. (023a) to capture temporal dynamics, while others focus on learning vidco-specific priors from scratch (Yang al. 2024). These innovations have also spurred interest in video editing techniques. where some
approaches optimize models through additional training (Mou et al., 2024 Cheng et al. 20233
while others achieve editing without requiring further training (Ku et | [024]] Ling et al. 2024).
Additionally, when adapting image-based methods to video, researchers have employed different
strategies, such as leveraging pre-trained T2I networks (Ma et al. 2024b Wang et al. 2023b Wu
et al. 2023) or utilizing pre-trained optical flow models (Yang al. 2023 Cong et 20233 to hance temporal coherence. However, despite these notable achievements, precise control over video lighting remains significant challenge, with existing methods falling short in maintaining lighting motion consistency, preserving scene details, and ensuring the rationality of character lighting.
Controllable Video Generation. With the rapid development of video generation models, sig- nificant potential has been brought to the field of controllable video generation. Recently, some methods have used basic conditions such depths and sketches control video content gener- ation (Chen et al., 2024) Wang et al., 2023c). There are also methods based on motion (Geng & Owens 2024). optical flows (Zhao al.) (023), and those related to human poses (Ma et al.) 2024b). Another category of methods (Wang et al. 2024a b) provides user-defined bounding boxes or trajectories to achieve flexible control over the foreground content in videos. The emergence of most of these methods is largely due to the appearance of the ControlNet (Zhang et al. (2033) archi- tecture. However, these methods often require us to construct high-quality training data to train the corresponding modules to achieve controllable generation. Accordingly, some wraining-free methods have emerged, such as Trailblazer (Ma et al.) (224a)), which enhances attention on specific regions via cross-atted atteentttn,,onn Peekaboo (Jain et al. (024). which uses masked attention for control without additional training, and FreeTraj (Qiu et al. 2024b), which aligns trajectories with given boxes using frequency fusion.
METHOD
The proposed LightCtrl a training-free controllable video relighting method that provides control-
lability of the video illumination through the given light map conditioning. We begin by introducing
the background of the image relighting method, IC-Light (Zhang et al. 2025) in Sec. 3.1 Then
elaborate the overall pipeline of LightCtrl in Sec. 3.2 along with its two key components: Light Map Injection module (Sec. 3.2.2) and Geometric- Aware Relighting module (Sec. 3.2.3).
3.1 PRELIMINARY: IC-LIGHT (ZHAGG ET AL.. 2025) FOR SINGLE IMAGE RELIGHTING
IC-Light (Zhagg et al. 2025) is the state-of sanee-of-the-att art method for conditional image relighting, which is fine-tuned based pre-trained text-to-image diffusion model, Stable Diffusion (Rombach et al.,
3

---

Published as a conference paper at ICLR 2026
VDM
VAB Encoder
VAB Decoder
Single Step Frame-wise Relighting
Element Multi.
Goometry- Award Relighting Module
Nommal
Element Sum
Extimation
Frmoowise
Light Map ML
Trajectory Generation
Relighted Torget
Add Notoe
Progressive Fusiio
Initial Detail Resadial ard
Notey Loterns
Coneistont Target &
Aee
Fasion Target
Light Map
V
Fusion LasentsZ
Injection Module
Noise F'ree Component Porc
Ousput
Framoraise Mask Randon Nuisc Franstor Injected Labents Amn
VDM Denoising
Figure 2: Overall Pipeline. We perform controllable video relighting with a user-provided light trajectory. Where we inject the light map on the noisy latent of VDM using the light map injection module. Then, each denosing step, we design geometry-aware relighting module to produce the relighted results wise. Thus, the VDM can help to generate consistent video results with controllable lighting.
2021 During training, the target relighted image Tt is encoded into latent space via pre- trained VAE encoder E, where the noise is added iteratively via diffusion model theory (Ho et al. 2020). Given the
conditions, including time step background reference and the input degradation La, the network learn to predict the added noise e of each step. Besides the original diffusion progress, their key innovation is the light transport consistency constraint, which enforees the linear properties of light transport by minimizing the difference between the appearance under mixed illumination and the sum of appearances under individual illuminations, so that IC-Light can be trained via:
L Adiffusion ((E(T)), T., ET, Acomsistency| || @ (EL,+ (((la,6122)1
where M is the foreground mask, @ denotes pixel-wise multiplication, and x(-- is learnable function. The final loss function is a weighted combination of the vanilla diffusion loss and the light transport consistency loss, with weights Adifiusioon and Acomsistency. respectively. During the inference stage, we can obtain the corresponding controllable image relighting results by inputting an image along with an appropriate background reference.
Specifically, IC-Light shows very interesting application by manually defining gradient mask as the potential light map, as the background reference. Then, can be used generate the light- controllable results in a specific direction. This light map reflects the positional and directional preference information of the illumination the relighting results, laying the foundation and inspir- ing controllable video relighting task.
3.2 LIGHTCTRL
Unlike image relighting methods that focus on single images and the specific lighting direction, approach targets high-quality video relighting, particularly with continuously varying light map, ensuring coherence and naturalness throughout the video. Specifically, given source video, a relighting prompt, and light map condition, our model aims to generate a new illumination for the video while preserving its original content. We begin by presenting the overall pipeline of the proposed framework, followed by a detailed description of its two key components: the Geometry- Aware Relighting module and the Light Map Injection module.
3.2.1 OVERALL PIPELINE
LightCtrl follows a similar approach to the training-fre free video editing paradigm (Meng et al. 2021), where the source video is first encoded into noisy latent, and controllable relighting is achieved
4

---

Published as a conference paper at ICLR 2026
through progressive denoising in the latent space. As shown in Fig. first encode the frame source video V. into a latent space žo, and add Tm steps of noise to obtain the noisy latents 2m- Subsequently, according to the trajectory defined by the user, the corresponding frame- wise mask sequence is generated. For controllable relighting, firstly, we use the Light Map Injection module to obtain the trajectory-injected latents 2. from 2m, so that the following T.. steps of denoising have the prior of generating the corresponding light trajectory. Then, similar to previous work (Zhou et al., 2025), we perform a single frame relighting and utilize the VDM for temporal consistency. In detail, during each denoising step t, we first predict the noise-free free latent at 0 step Zu+- Here, each frame of Zue-t is decoded to the RGB space, which serves as the video consistent target I, = (zuu-r) using the pre-trained VAE decoder D() for frame-wise relighting. To address detail loss, we calculate the initial detail residual Ad between the decoded latent in the first step In and the source video V, pixelwisely and add it into I as the input of Geometry-Aware Relighting module for frame- wise relighting. obtaining the relighted target I for the t-th denoising step. Finally, we employ progressive fusion strategy with fusion weight d that decreases denoising progresses. The fusion target I for the step is calculated as:
I, = (1 -,
The fusion latent Zo-t = EI,) guides the next step of the denoising process towards the relight- ing direction. This ensures temporally coherent lighting, addressing temporal inconsistencies and enhancing the visual quality of the relighted video.
3.2.2 USER-DEFINED TRAJECTORY AS LIGHT MAP CONTROL SIGNAL
As mentioned above, the first key component of our framework enables the injection of a user- defined light trajectory, allowing the generated video to exhibit temporally varying lighting across frames that follows the specified trajectory. For example, with circular light sources, the user manu- ally specifies light trajectory along with the starting and ending radius. We then linearly interpolate the radius and position of the light for each frame to construct a frame-wise light map. This control signal is subsequently integrated into both the single- frame relighting process and the video diffu- sion denoising loop. Firstly, in the single frame relighting, as discussed in Sec. [.1] IC-Light (Zhang eet al. 2025) provides a manual background map to generate the relighting results in the correspond- ing direction. Inspired by this finding. we expand this directional background light source to the local dynamic for relighting video with differences as shown in the Fig. 3
Although the above method can achieve some kind of controllability at the frame level, it still shows limited effects when facing the multi-step VDM denosing. Therefore, to produce more controllable results, inspired by the previous bounding box injection method (Qiu et al. (024bb on leveraging the noise guide the motion of objects video generation, we adapt the concept to control the movement of illumination and then propose the Light Map Injection module.
During the video diffusion processes, given the initial noisy latent êm of the source video, we ini- tialize a random Gaussian noise Grandom based on the area defined by the input light masks M. However, directly replacing the noisy latents 2m with the masked local latent Grandom caused severe visual artifacts. To mitigate this, we introduced a weighted fusion strategy. Mathematically, for each frame k the new initial noise
* is calculated as:
2th
if M [ jl 0
z w. Grandom + (1 w) 2 if M [i 1,
where 2t, represents the noisy latent variable corresponding to frame k. and Mj denotes the input light mask at frame k. My[r.j] = when the position [i. j] inside the mask area of the light map, 0 otherwise. w is the fusion weight carefully tuned to balance the influence of the
and je] local random noise and the original noise, ensuring a smooth transition and minimizing artifacts. Based on the above strategy, the light trajectory be successfully injected into both the video diffusion model and the image diffusion model for controllable video relighting.
3.2.3 GEOMETRY-AWARE RELIGHTING MODULE
Directly using the proposed Light Map Injection module can achieve controllable video relighting. However, the light environment in the source video often causes undesirable leakage into the final
5

---

Published as a conference paper at ICLR 2026
3D PM
Nociaal Soquesce
agint, natural lighting. warn atmosphere
IFFT
Light Map Latenix
HIPE
CALEEREREEEETTE CHETEEEETEE Tager
Iepat Laterts
[C LC-Light
Relight Target
This module is designed for frame-wiss wise relighting. Firstly, the normal sequence of the source video and each consistent target derived from the demoising steps are encoded into the latent space and further process them in the frequency domain. Then, their spatio-temporal low-frequency components are dynamically fused with the high-frequency components of the consistent latents, with the cut-off frequency varying each denoising step to add details. Finally, these lantents are transformed back to the video space and produce the relighted frames using frame-wise IC-Light Zhang et al. (2025).
Figure 3: The framework of Geometry-Aware Relighting Module.
output. For instance, when the illumination direction changes from left right, artifacts may per- sist where the left side of the subject's ' face remains incorrectly lit. To address this, we propose incorporating geometry informationnsuuface normal maps) as part of single-frame relighting.
Similar to the RGB image, we find that the latent of the normal map contains stronger structural in- formation and can also be utilized in zero-shot setting. However, directly integrating this geometry information before each step of relighting would bring the blurring results caused by the pre-trained model, thereby severely affecting the quality and details of the final relighted video. Thus, we in- volve the frequency features for better fusion. As shown in Fig. 3 specifically, in each step of VDM denoising, we firstly use pre-trained satte-of-the-az normal estimation model, Stable Normal (Ye et al. (2024), to predict the frame-wiss normal map N from the source video. Given the consistent target I, as input to the relight model, we first encode into the latent space to obtain the consistent latents Zoe--- We perform the similar VAE encoding to the predicted normal N, obtaining Znormal- Then, we transfer them into the frequency domain via the Fast Fourier Transformation FFT 3D in the spatial and frequency domain (Wu et al. 2024). Whereas we define dynamic 3D Butterworth filter Ha (t) with cut-off frequency at a to control the high-frequency and low- fwe.ueequccc details be- tween two latents. The detailed definition of the filter will be provided in Appendix B. After that, we combine the low-frequency latents and high-frequency latents and recover them into the spatial latent domain via inversed 3D Fast Fourier Transformation IFFT 3D, which serves as the input latents 2, for frame-wise IC-Light relighting. Formally, this process can be written as:
2 IFFT 3D (FFT aD aD (noormal) Ha (t) + FFT (D @ (1 (t)),
(4)
As the denoising loop progresses, we linearly decrease the cut-off frequency a so that the dynamic filter (t) gradually changes from all-pass to low-pass. In this way, we completely adopt the normal latents as input at the beginning of denoising. In the later stage of denoising, we utilize more of the low-frequency part of the normal latents and gradually add the high-frequenc, information from the consistent latents. This module enables us to effectively leverage the normal information to eliminate interfering lighting distributions while preserving details.
EXPERIMENTS
4.1 EXPERIMENTAL DETAILS
Baselines. As baseline, we adopt IC-Light (Zhang et al.) 2025). a state-of-the-an controllable image relighting model that relights each frame individually based on its corresponding light map from the input lighting trajectory. To balance frame-wise relighting controllability with temporal consistency, we also develop another baseline method based on SDEdit (Meng et al. (02T). where the frame-wise relighted video produced by IC-Light is perturbed with noise (using two levels: 0.2 and 0.6) and subsequently denoised. Finally, we compare against a variant based on the recent video relighting method Light-A- Video (Zhou al. (025). which refer as LAV-Traj. in which the light map from the input lighting trajectory is provided as the background image to the frame-wiis wise image relighting model used in Light-A-Video..

---

Published as a conference paper at ICLR 2026
Ingur Vidao
Mapht
KC-iggh
IC IC-Lisht
IC-igh -S008000401
LAV-Tag
Eighh
"a girl Listening music, natural Lighting, atnosphere"
girl, ragic lit, sci-fi RGB gloving, Lightiin"
Qualitative comparison of light trajectory control. We compare our LightCtrl with IC- Light, IC-Light SDEdit-0.2, IC-Light + SDEdit-0.6, and LAV-Traj. Compared to baselines, LightCtrl produces high-quality, temporally coherent, and controllable relighting results, closely following the input light trajectories.
Figure 4:
Datasets. We construct a test dataset comprising 50 videos, primarily sourced from Pixabay (pix- abay 2025), which offers a diverse collection of high-quality footage. For each video, we apply six predefined light trajectories featuring variations in light source radius and trajectories, includ- ing left-to-right, top-to-bottom, circular motion, and more. Additional results and comparisons are provided in Appendix D.
We conduct quantitative evaluations to assess the relighting controllability with respect to the input light trajectory, as well as the visual quality. For the light trajectory control- lability, we report the PSNR, in the y channel(the luminance channel) in the light map mask region between the results of the baseline methods and pure white video to measure the controllability of illumination. When the result the mask area of the light map is closer to a pure white result in the luminance channel, we have reason to believe that there is more lighting in that area, which also reflects better ability to control the video lighting according the lighting trajectory. Then, directly overlay the light map the original video and calculate the PSNRtight between the over- lay results and all baseline methods evaluate the controllable temporal consistency of lighting and foreground. Secondly, to evaluate the video quality of the results, we adopt aesthetic quality in Vbench (Huang al., 2024) assess the aesthetic quality of videos, and calculate the Frechet Video Distance (FVD) (Unterthiner et al. 2018) scores between all video results of different basclines and the input videos.
Evaluation Metrics.
Finally, we conduct user study with 40 volunteers. We randomly select 24 videos from the 50 videos generated by each method, where these 24 sets of videos include 6 predefined trajectories by users, as well as source videos with different characteristics and variety of lighting prompts. They assessed the performance our method by comparing it with baseline methods across four key as- pects. Specifically, they examined Video Smoothness (the coherence and naturalness of appearance

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative Comparisons with Other Methods.
Video Quality
Control Abiliry
User Study
Methods
AQ
FVD L
PSNR,
VS
LC
LQ
ALT 1
IC-Light al. Meng et al.
2025
0.5937
1018.5
11.059
15.850
1.00%
15.00%
3.005
9.73%
IC-Light + SDEdi1-
2021
0,5907
1134,8
11.009
16.249
2.50%
2.00%
0.50%
3.00%
IC-Lishl + SDEdit-0.. Meng clal
2021
0.5681
1630.9
10.980
16.385
4.75%
1.09%
0.50%
3.27%
7ho al. 2025
0.6157
1077.4
11.043
17.755
23.50%
4.00%
20.003
10.27%
LightCtrl (Ours)
0.6114
993.1
11.768
18.532
68.25%
77.91%
74.86%
73.73%
and lighting changes in the video), Lighting Controllability (the alignment of lighting movement with the given trajectory), Lighting Quality (the overall quality and rationality of the lighting ef- fects), and Aligmment between Lighting and Text (the match between the generated lighting and the provided relight prompt). The best results from the baseline methods for each dimension are chosen by the volunteers and used as preference metries.
Implementation Details. In our experiments, following by (Zhou et al. 2025 we adopt IC- Light /Zang et al. (205)) as the image relight model and employ AnimateDiff (Guo et al. 2023) as the detault VDM. We typically set Tm to 25. Then we set X = 1 t/Tm in the progressive fusion strategy, where the fusion weight )e decreases as denoising progresses. In the geometry-aware relighting module, we set the cut-off frequency = A so that as the denoising loop progresses, is linearly decreasing. More implementation details are in Appendix A.
4.2 COMPARE WITH THE STATE-OF-THE--AE METHODS
Fig. presents the control ability of illumination of all methods. The IC-Light per-frame model, while capable of producing high-quality relighting results on frame-by-frame basis, performs poorly in terms of temporal consistency when dealing with continuously varying lighting trajec- tories. This results in significant flickering and discontinuity in the overall visual output. However, ICLight + SDEdit, although introducing motion priors through VDM, suffers from inconsistency in lighting after relighting at noise level of 0.2. and excessive smoothing of appearance at noise level of 0.6. Under the PLF fusion strategy, LAV-Traj has demonstrated strong capabilities in terms of temporal consistency in relighting results. However, when it comes to handling the diverse lighting trajectory variations we require, its controllability still falls short and fails to achieve controllable results for a variety of lighting trajectories.
The quantitative comparison of our method with other baseline methods is shown in Table The IC-Light results still demonstrate their effectiveness terms of the controllability of lighting each frame. However, the obvious temporal inconsistency greatly reduces the aesthetic quality AQ score of the results. By combining SDEdit with different noise levels, temporal consistency is improved, but since it only smooths the video by changing the noise level and cannot achieve a consistency constraint on lighting, the video quality scores AQ and FVD, as well as the controllability metrics PSNR, are all low. L.V-Trra achieves the best aesthetic quality AQ score, but its strong consis- tency constraint greatly reduces its lighting controllability based on the given trajectory. In contrast, LightCtrl achieves a low FVD score and high AQ score. While maintaining high temporal con- sistency in relighting quality, also achieves the best scores in both controllability metrics PSNR,,. PSNR demonstrating outstanding lighting control capability. Furthermore, we demonstrate method's superiority under static light conditions in Appendix C.
4.3 ABLATION STUDY
We conduct an ablation study to evaluate the importance of different components. Considering the original LAV + Traj as baseline (Zhou et al. (225). due to the strong temporal consistency constraint, it is unable to handle controllable continuous variations in lighting trajectories. As shown in Fig. 5 the Geometry-Awart Relighting module makes the lighting results human faces more reasonabie. independent of the lighting distribution in the source video. Meanwhile, the lighting distribution brought by the source video the lawn the right side causes the LAV-Traj the background (as shown in the red box) when the light reaches the lower part of the frame, even when method does not have the GAR module. In addition, by injecting lighting trajectories into
still highlight
8

---

Published as a conference paper at ICLR 2026
the initial noise before VDM denoising, LightCtrl can generate controllable relighting videos while preserving the overall details of the video(as shown in the right part of Fig. 5). Due to the space limitation, give more ablation experiments in Appendix B.
Lighht Imput Video
Welo LAV-TT)
LightCiri
"handsone with glasses, sunlight through the blinds" "a bost floating the evening noon
Figure 5: Ablation Study. Results of controllable video relighting with the Giomeery-Awan Re- lighting (GAR) module the Light Map Injection (LMI) module removed. The red box indicates the light distribution brought by the input video.
4.4 LIMITATION AND FUTURE WORK
Since our method is based existing image relighting models and VDM models, although our approach can achieve controllable lighting results for different light paths under the zero-shot setting, its performance is still limited. For example, when the light path passes over the foreground, it can cause abnormal flickering on the face. Moreover, our method currently cannot achieve 3D perception information,
of the light path. Despite the use of geometry the influence obvious shadows in the source video. As shown in Fig. 6 the shadow on the right side of the cat remains after relighting. To address these limitations, we will adapt the latest video
still cannot effectively remove
Input
Ours
Figure 6: Failure Case. We show an example result where the light map is blended into the original input video. Our method struggles to perform effectively in scenarios involving strong shadows, such as the "cat" case with the relight prompt "a cat, warm sunshine."

---

Published as a conference paper at ICLR 2026
generation base models and design an advanced network in our future work to achieve higher-quality controllable lighting.
5 CONCLUSION
We introduce LightCtrl, a controllable video relighting method using pre-trained video and im- age diffusion models. Thus, the image relighting model produces high-quality relighting results, whereas the video diffusion model provides robust temporal consistency and control over the entire video via user-supplied trajectories. To achieve this goal, we propose two key modules, Light Map Injection and Geometry-Awan Relighting, to boost controllability by enhancing illumination coher- ence and reducing the impact of initial lighting. LightCtrl precisely generates high-quality videos with diverse illumination changes according to light trajectories, outperforming baseline methods in controllability as well as maintaining the temporal consistency.
Acknowledgments. This work was financially supported in part by National Natural Science Foun- dation of China (Project No. 62506064) and Guangdong Provincial Regional Joint Fund (Project No. 2024A15151 10052)..
REFERENCES
Shrisha Bharadwaj. Haiwen Feng, Victoria Abrevaya, and Michael Black. Genlit: Reformulating single-image relighting as video generation. arXiv preprint 0rXiv:2412.11224, 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15122 2023.
Ziqi Cai, Kaiwen Jiang, Shu-Yu Yu Chen, Yu-Kun Lai, Hongho Fu, Boxin Shi, and Lin Gao. Real-time 3d-aware portrait video relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6221-6231, 2024.
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. Videocrafterl: Open diffusion models for high- quality video generation. arXiv preprint
(tt.:233...9512... 2023.
Haoxin Chen, Yong Zhang, Xiaodong Cun, Menghan Xia, Xintao Wang, Chao Weng, and Ying Shan. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7320-7320, 2024.
Jiaxin Cheng, Tianjun Xiao, and Tong He. Consistent video-to-video transfer using synthetic dataset. arXiv preprint arXiv:231 1.002111011111111111111 2023.
Yuren Cong, Mengmeng Xu, Christian Simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. Flatten: optical flow-guided attention for consistent text-to-video editing. arXiv preprint arXiv:2310.05922, 2023.
Ye Fang, Zeyi Sun, Shangzhan Zhang, Tong Wu, Yinghao Xu, Pan Zhang, Jiaqi Wang, Gordon Wetzstein, and Dahua Lin. Relightvid: Temporal-consisten diffusion model for video relighting. arXiv preprint arXiv:2501. 16330, 2025.
Daniel Geng and Andrew Owens. Motion guidance: Diffusion-bastt image editing with differen- tiable motion estimators. arXiv preprint arXiv:2401.18085, 2024.
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Mancesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffu- sion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023.
Mingming He, Pascal Clausen, Ahmet Levent Taşel. Li Ma. Oliver Pilarski, Wenqi Xian, Laszlo Rikker, Xueming Yu, Ryan Burgert, Ning Yu, et al. Diffrelight: Diffusion- based facial perfor- mance relighting. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-12, 2024.
10

---

Published as a conference paper at ICLR 2026
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840- 6851, 2020.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 21807-2181 2024.
Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peckaboo: Interactive video generation via masked-diffusion In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8079-8088, 2024.
Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, and Sanghyun Woo. Switchlight: Co-design of physics-driven architecture and pre-training framework for human portrait relight- ing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 25096-251066 25106, 2024.
Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning- free frame- work for any video-to-video editing tasks. arXiv preprint wrXhv:2403.14466 2024.
Min-Hui Lin, Mahesh Reddy, Guillaume Berger, Michel Sarkis, Fatih Porikli, and Ning Bi. Edgere- light360: Text-conditioned 360-degree hdr image generation for real-time on-device video por- trait relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 831 831-840, 2024.
Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training-free motion cloning for controllable video generation. arXiv preprint arXiv:2406.052:
2024.
Wan-Duo Kurt Ma, John P Lewis, and W Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion- basedd video generation. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024a.
Yue Ma, Yingging He, Hongfa Wang, Andong Wang, Chenyang Qi. Chengfei Cai, Xiu Li, Zhifeng Li, Heung-Yeen Yeung Shum, Wei Liu, et al. Follow-your-click: Open-domain regional image anima- tion via short prompts. arXiv preprint arXiv:2403.08220 2024b.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun- Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint 200::21088.0073,, 2021.
Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, and Jian Zhang. Revideo: Remake a video with motion and content control. Advances in Neural Information Processing Systems, 37:18481-18503 2024.
Thomas Nestmeyer, Jean-François Lalonde, Tain Matthews, and Andreas Lehrmann. Learning physics-guided face relighting under directional light. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 5124-5133, 2020.
Rohit Pandey, Sergio Orts-Escolano, Chloe Legendre, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul E Debevec, and Sean Ryan Fanello. Total relighting: learning to relight portraits for background replacement. ACM Trans. Graph., 40(4):43-1, 2021.
pixabay. http://pixabay.com/videee 2025. Utt/tttss//ppiiaaayycom /ttp://pxaaaay..coo//deeeoe////
Puntawat Ponglertnapakorn, Nontawat Tritrong, and Supasorn Suwajanakorn. Difareli: Diffusion face relighting. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 22646-22657, 2023.
Haonan Qiu, Zhaoxi Chen, Yuming Jiang. Hang Zhou, Xiangyu Fan, Lei Yang, Wayne Wu, and Ziwei Liu. Relitalk: Relightable talking portrait generation from a single video. International Journal of Computer Vision (3208):773322725 2024a.

---

Published as a conference paper at ICLR 2026
Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Free- traj: Tuning-frre free trajectory control in vidco diffusion models. arXiv preprint arXiv:2406. 16863, 2024b.
Mengwei Ren, Wei Xiong, Jae Shin Yoon, Zhixin Shu, Jianming Zhang. HyunJoon Jung, Guido Gerig, and He Zhang. Relightful harmonization: Lighting- aware portrait background replace- ment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 6452-6462, 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Tiancheng Sun, Jonathan T Barron, Yun-Ta Tsai, Zexiang Xu, Xueming Yu, Graham Fyffe, Christoph Rhemann, Jay Busch, Paul E Debevec, and Ravi Ramamoorthi, Single image portrait relighting. ACM Trans. Gruph., 38(4):79-1, 2019.
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: new metric & challenges. arXiv preprini arXiv: MXXiv:88120011111 2018.
Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, and Hang Li. Boximator: Generating rich and controllable motions for video synthesis. arXiv preprint arXiv:2402.01566, 2024a.
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elscope texi-to-video technical report. arXiv preprint arXiv:2308. 06571, 2023a.
Wen Wang, Yan Jiang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chun- hua Shen. Zero-shot video editing using off-the-shelf image diffusion models. arXiv preprint arXiv:2303. 2023b.
Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and lingren Zhou. Videocomposer: Compositional video synthesis with motion trollability. Advances in Neural Information Processing Systems, 36:7594-7611, 2023c
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-11, 2024b.
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang. Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tunc-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7623-7633, 2023.
Tianxing Wu, Chenyang Si, Yuming Jiang, Ziqi Huang, and Ziwei Liu. Freeinit: Bridging initializa- tion gap in video diffusion models. In European Conference on Computer Vision, pp. 378-394. Springer, 2024.
Jinbo Xing. Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open- open-domain images with video diffusion priors. In European Conference on Computer Vision, pp. 399-417. Springer, 2024.
Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender video: Zero shot text-guided video-to-video translation. In SIGGRAPH Asia 2023 Conference Papers, pp. 1-11, 2023.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint 24v::240.006777 2024.
Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal. ACM Transactions Graphics (TOG), -388 2024.
12

---

Published as a conference paper at ICLR 2026
Yu- Ying Yeh, Koki Nagano, Sameh Khamis, Jan Kautz, Ming- Yu Liu, and Ting- Chug- Wang. Learn- ing to relight portrait images via a virtual light stage and synthetic- to-real adaptation. ACM Trans- actions Graphics (TOG), 41(6))1-21, 2022.
Chong Zeng. Yue Dong Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained lighting control for diffusion-based image generation. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-12, 2024.
Longwen Zhang, Qixuan Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Neural video portrait relighting in real-time via consistency modeling. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 802-812, 2021.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 3836-3847, 2023.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport. In The Thirteenth International Conference on Learning Representations, 2025.
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36: 127-11150, 2023.
Yujie Zhou, Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Qidong Huang, Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, et al. Light-a-video: Training-free video relighting via pro- gressive light fusion. arXiv preprint (xxXx::5002.83900, 2025.
A IMPLEMENTATION DETAILS
Inference time. We automatically generate light masks consistent with the video length based on user-defined trajectory patterns. On our A6000 GPU, the inference time for 512x512 video with 16 frames is approximately 240 seconds.
Details of user study. We used Tencent survey create questionnaire for all volunteers to evaluate the generated 24 sets of videos from four aspects.
i) Video Smoothness: herently, naturally, and smoothly, without noticeable sudden flickers or jumps?
Does the appearance and illumination changes in the video flow co-
ii) Lighting Controllability: Does the movement illumination the video align with the given illumination trajectory?
iii) Lighting Quality: How is the overall lighting quality of the foreground and the entire frame, and is the lighting reasonable?
iv) Alignment between Lighting and Text: Is there consistency between the generated illu- mination the video and the given relight prompt?
Fig. 7 illlstrrates an example of the questionnaire utilized in our study. For comprehensive assess- ment, we recruited 40 participants, all participants were students working in video generation or computer vision related fields. Before the study, all participants received concise tutorial explain- ing the relighting task and the meaning of each evaluation metric, including lighting controllability, so that they shared consistent and informed understanding of what they were rating. the provided study example, we also explicitly state the definition of each metric. To enhance efficiency, users were guided to compare multiple methods within each evaluation dimension. Prior to comparing our method with the baseline approaches, we randomly shuffled the order of videos for each evaluation metric. Participants were then asked to select the more favorable result for each comparison. To en- sure the rationality and objectivity of the lighting controllability evaluation, lighting map in each comparison video. This allows users to refer to the movement trajectories of the lighting, which based on six predefined lighting patterns, each with distinct characteristics.
include the current
13

---

Published as a conference paper at ICLR 2026
02 Video Group2-'s car driving on the street, neon light"
Vicaal
Modi
1.iite smeociness: Whather the appearancs and itatination changes wkioo coneront, natural, smooth, without dbvious suddon
jumos.
2.1 lighting controlaniiity Whener the movemmntt of numioniion in the video conslstent
llumiiation trajectory.
Lighting quality: The overal lightin quality the foregroune and the ondire framro, as
Villo Brout
the retionality
4.Al gramett butween Lighting Twet: Whither consistancy betooen
Lighing Die th
0
generated llumination the video and the givon ilumiration desoriction.
The conditioo tiona for the ilumniation map are followas
Ligttig Oue by
Mpproont bn eveen snp
Figure 7: Visualization of the questionnaire for user study. Users are asked to compare the all baselines at each aspect and select their preferred one.
B MORE ABLATION STUDIES
The definition of the dynamic filter in the GAR module. In the GAR module, to better fuse the normal map and the RGB latent, adopt a strategy of dynamic fusion in the frequency domain. The definition of this dynamic filter Ha (t) is as follows:
when D w) S d1(
(t) =
1+
he (e
(5)
0
when ht h w) > dt(()
where the time-varying cut-off frequency: d.(1) = de(() = a 11 Tim the frequency distance calculation is updated to:
V (d.(t)
(t) (t)
2t T 1
1)
2
2h H
2
2w W
D(t, h, w) ())),,,))//
Through the above method, we have defined a 3D dynamic filter, whose cut-off frequency a value that changes with the denoising step t. By gradually adjusting the high-frequency and low-frequency information, we have effectively injected the normal map.
Impact on feature selection of the GAR module. In our GAR module, we introduce geometry feature information to mitigate the bias lighting distribution brought by the source video. In our experiments, adopt two types of features: delight images and normal maps. However, since use pre-trained models for these features, the domain--seccinn models' performance significantly fects the quality, impacting the subsequent relighting results. As shown in Fig. & models to remove the lighting distribution in the source video. While both the StableDelight and In- ArythicAnything models can mitigate the illumination distribution in the input video to some extent, noticeable artifacts (highlighted by the red boxes) emerge in the generated results after injecting in- formation via the GAR module, as evident from the third and fourth rows. In contrast, our method, by incorporating normal information, produces more plausible and high-quality outputs. Moreover, as demonstrated in the third frame, our approach excels in terms of lighting quality in the generated results. Overall, our results can better reflect the impact of lighting changes in the video. Therefore, in our experiments, we chose the normal map with better feature quality.
delight
Impact on injection method of the GAR module. After selecting the normal map as the input for the GAR module among delight images and the normal map, we need to consider how to integrate this feature into our model. Here, we conducted an ablation experiment on the injection method. We attempted direct weighted fusion, using half of the consistent target and half of the normal map the input. However, due to the loss of details in the normal map, the result of direct fusion showed significant blurriness (the third left row of Fig. 99. Then, we fused the two in the frequency domain, combining the tigh-frequency information of the consistent target, which is rich in detail,
14

---

Published as a conference paper at ICLR 2026
Inpar Video
Light Map
Intrinii Anything StableDelight
StableNormal (Ours)
"A man in the classroon, sunshine"
Figure 8: Ablation Study on Feature Selection of GAR module.
Iaput Video
Iaput Videe
Light Map
Light
Cur-of-00775
0.5-03
02-08
Cat-ult-bbl
Ours
"handsone glesses, sunlight through blinds"
girl Listening music, natural Lighting, etnosphere"
Figure 9: Ablation Study on Injection Method of GAR(Left) and LMI module(Right).
with the low-frequency information the normal map. But from the results (the fourth and fifth left rows of Fig. 9. we found that matter how we adjusted the cut-off frequency of the filter, we couldn't effectively remove the highlight in the red box on the left side the image. Based on this, decided to adopt dynamic fusion method. injecting more normal map information in the early stage denoising and supplementing details with the consistent target in the later stage. Finally, achieved good balance between detail preservation and prevention of light
15

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of Normal Injection Strategies on Video Quality
Method
AQ
FVD 4
0.5 0.5
0.5892
1010.9
Cut-off-0.25
0.6097
998.7
Cut-off-lbd
0.6114
993.1
leakage(the last left row of Fig. 9). To further validate the benefit of this design, we additionally include another set of comparisons between latent space and frequency space integration as the right part of Fig. IT We evaluate video quality under both strategies, and the results are consistent with the visual observations: the frequency -omennn method yields markedly superior performance across all quantitative metrics, as reported in T
Impact on injection method of the LMI module. When injecting the light map. to strike bal- ance between detail preservation and post-injection lighting controllability, we explored two fusion methods: direct fusion in the latent space and frequency-domain fusion of local noise and the orig- inal noisy latent, separating high and low frequencies. Our experiments revealed an issue with the rrequencydoomate approach. When using the original noisy latent to supplement high frequencies after injecting local noise to maintain details, the edges of the girl's sweater under sunlight exhibited excessive (as shown in the third row of the right image in Fig. 99. This ever-enhancement of details deviated from our desired outcome. Consequently, we adopted a direct weighted fusion strategy in the latent space. We conducted multiple experiments to determine the optimal weights for local and original noise. Initially, when the original noise accounted for 0.8 (as shown in the fourth row of the right image in Fig. 9. while the excessive outlining was mitigated, lighting controllability decreased. For instance, in the last figure, although lighting was present in the bottom-right corner, the corresponding illumination effect was not properly reflected in the image. To address this, we increased the weight of local noise, ultimately settling on ratio of 0.8 local noise to 0.2 original noise. This configuration effectively reduced excessive outlining while ensuring lighting controlla- bility, making the lighting in the image more consistent with expectations(as shown in the last row of the right image in Fig. 9. It enables us to preserve image details while achieving precise adjustment and control of the lighting distribution.
Table 3: Quantitative Results between Raw Frames and Decoded Frames + Residual.
Methods
AQ
FVD 4
Raw Frames
0.5946
1037.4
With Residual
0.6114
993.1
Raw Frames Decoded Frames Residual, This design compensates for the loss of local high- frequency information introduced by the VDM denoising process. The residual computed once at the first step restores these fine details while preserving the relighting effects progressively estab- lished through latent space denoising. Using the raw frames directly would break this balance. As demonstrated in Fig. [0] directly injecting raw frames causes the output video inherit substantial illumination patterns and low-frequency structures from the input, which preserves appearance but ultimately fails to achieve correct relighting. In contrast, the residual formulation enables us re- plenish structural details during the early denoising stages, while its influence gradually diminishes later stages where lighting appearance is formed. This yields the desired balance: retaining neces- sary fine-grained details without interfering with the generation of high-quality. trajectory-controlleed controlled relighting. We also compute AQ and FVD for both, as shown in Table further quantifying the im- provement in vidco quality achieved by our design.
The Effects of Progressive fusion.Th. progressive fusion mechanism is key to achieving temporal consistency, As illustrated in the left part of Fig. [1] compared with the per-frame results produced by IC-Light, progressive fusion ensures that both the foreground and background remain noticeably coherent over time. In particular, the red-boxed background region shows that without progres- sive fusion, the background exhibits clear frame-to toaee-to-frame fluctuations with no temporal stability, whereas our fusion strategy effectively suppresses such inconsistencies and maintains stable appear-
16

---

Published as a conference paper at ICLR 2026
Table 4: Effect of Progressive Fusion Temporal CLIP--coree
Methods
CLIP-Score f
w/o Progressive Fusion
0.9200 0.9634
With Progressive Fusion
ance throughout the sequence. To objectively evaluate the improvement in temporal consistency brought by progressive fusion, additionally report the average CLIP score across consecutive video frames, which reflects semantic and structural coherence over time. In the Table with pro- gressive fusion, the model achieves higher CLIP scores, confirming its contribution to smoother and more stable video relighting.
light map
Input
Use raw frames
Use residual Ours
Figure 10: Ablation Study on Raw Frames Residual.
C MORE QUANTITATIVE RESULTS
The effectiveness of static light conditions. We have also conducted a experiment evaluating illu- mination control in static scenes (Specifically, keep the light map in the same position through- out the entire video) using our established PSN Right metric (defined in Section 4.1 and used for temporal consistency). The results show method achieves 17.494 dB, exceeding all baselines (next-best Zhou et al. (2025) 16.630 This quantitative evidence confirms our method's 's lighting control superiority even in static conditions, supporting our original claim. The complete results are presented as shown in Table 5
D MORE QUALITATIVE RESULTS
More results based on CogVideoX. In Fig. 12 we present some results on the DIT-based video diffusion model, such Cog VideoX. These results also demonstrate that model can be adapted
17

---

Published as a conference paper at ICLR 2026
light map
light map
in latent space
W/O PLF
in frequency space
WIT PLI
Figure 11: Ablation Study on Progressive Fusion and Injection method of Normal.
Table 5: Comparison of PSN Rliht (dB) under static light conditions
Methods
IC-Light
SDEdit-0.2
SDEdit-0.6
LAV-Traj
Ours
PSN Riight
12.112
13.349
13.686
16.630
17.494
to multiple video diffusion models. Since the model of Cog VideoX uses the DDIMScheduler, while main model, based AnimateDiff, uses the mallcrAnesttriiisseeechhhuuu the intensity of the lighting colors in the final results is largely affected by the different schedulers. The lighting results from the DDIMScheduler tend be lighter. This difference in color intensity is also reasonable phenomenon.
More results of LightCtrl. In Fig. amples show that our model can generate controllable video relighting under different customized trajectories in various scenes.
present visual results of LightCtrl. These
E LARGE LANGUAGE MODEL USAGE
In the preparation of this work, the authors used LLMS (e.g., GPT-4, Claude, etc.) solely for text polishing, including grammar checking and sentence refinement. The LLM did not contribute to the scientific content, and the authors assume full responsibility for the work.
18

---

Published as a conference paper at ICLR 2026
Inpat
Light Map
Relighttd
Vdm prompt: man is standing front window
Relight prompt: "a mon standing in front of window, warm sunlight through window" "
Light
Relighted Video
Vdm prompt: "a girl listening a music"
Relight prompt: girl listening music, natural lighting, atmosphere" "
Figure 12: More results based on VogVideoX.
19

---

Published as a conference paper at ICLR 2026
Figure 13: More Results with Diverse Lighting Trajectories. The first and third columns show the results under random polyline illumination trajectories. The second column presents the results of two light sources moving independently, one from left right and the other from top bottom. The fourth column shows the results of two light sources moving in opposite directions along circular paths.
20

---

Published as a conference paper at ICLR 2026
Input Video
Light Map
Re Relighted Video
"a person in a yellow raincoat fishing by the shore, a golden sunrise."
Input Video
Relighted Video Light Map
"a house, the crimson sunset."
Input Video
Relighted Video Light Map
"a beach scene, dusky evening "
Figure 14: More results of LightCtrl in light trajectory-condittooned video relighting
21


Reviews:
[{"review_id": "DWM6rgF1OM", "reviewer": "Reviewer_bAHR", "summary": "This paper tackles the controllable video relighting task in a training-free manner. For this, LightCtrl builds on a pre-trained single-image relighting model, i.e., IC-Light, and tailors it to video relighting with several designs. First, it proposes to conduct frame-wise relighting on the video diffusion model's paired VAE decoder. Besides, it proposes to progressively fuse the source video and relit video during the diffusion process to maintain temporal coherence. Further, a user-defined light trajectory injection module is applied. Finally, they use off-the-shelf normal estimation to provide the video's normal maps to enable a geometry-aware relighting module. Experiments demonstrate the effectiveness of the proposed approach.", "questions": "See \"weakness\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- originality-wise: the idea of using a single-image relighting model to enable video relighting is interesting.\n- quality-wise: the qualitative and quantitative results are promising.\n- clarity-wise: the presentation is good.\n- significance-wise: the video relighting is important for a lot of downstream tasks, e.g., content creation.", "weaknesses": "I feel the ablations in the paper are quite inadequate. Even though there are some ablations in the appendix that are good, they are not the core part of the model design.\n\nFor example, can authors provide both **quantitative and qualitative** results to show the gradual improvement from the pre-trained IC-Light? From my understanding, there are several enhancements, but I have no clue which one contributes. Feel free to add things that I miss.\n- decoded frames + residual instead of raw frames (L216)\n- progressive fusion in Eq. (2)\n- geometry-aware feature in latent space\n- geometry-aware in frequency space.\n\nI am actually confused why not directly use the raw frames in the source video? Since the authors add back the difference between the raw videos and the decoded videos (L216), isn't this the same as just using the original video?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S4ukxi7GCc", "reviewer": "Reviewer_shr2", "summary": "This paper proposes a new method for video relighting that is training-free and allows users to provide a light trajectory as lighting conditions for the video. The authors make changes to the existing video diffusion framework as well as the image relighting framework (i.e., IC-Light) to allow them to use the user-provided light trajectory. The quality of the video relighting is evaluated with comparison to IC-Light and Light-A-Video. A user study is also conducted with 40 volunteers on video smoothness, lighting controllability, lighting quality, and alignment between lighting and text aspects of the relighting videos.", "questions": "1. What is the supporting evidence to indicate that having a user-provided light trajectory is a desired feature for relighting tasks? \n\n2. Can the authors provide more details about the user study? Please see the weaknesses above for questions. \n\n3. Can the authors discuss more about the significance of the approach, given the ablation study? If the authors can provide more rigorous ablation with more insight, it will be better.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The idea of designing a training-free method to control the lighting of a video generation model is interesting.\n\n- It is known that the common metrics for relighting might not faithfully reflect the visual quality of the relighted results. The authors design and conduct a user study with 40 volunteers, making the evaluation more rigorous.", "weaknesses": "- It is unclear how significant it is to have a user-provided light trajectory for relighting. The authors explain their motivation for using manually labeled light trajectories in lines 069-072, but there is no supporting evidence to indicate that this is a desired or much-needed feature for users. \n\n- The user study is very important in evaluating the performance of the relighting methods. But many important details are not there in the current draft: how are those volunteers selected? Do they understand the relighting task? How are they trained on the provided metrics (e.g., lighting controllability)? How reliable are their answers? Is the result statistically significant? \n\n- The proposed method is designed to handle light trajectories in the relighting video task and is evaluated with the same type of light trajectory data (lines 315-318). This testing seems to be limited. There are many other ways to control lighting in relighting methods (e.g., environment maps or added light sources at certain locations in the image). It is currently unclear how this method compares to those more general cases of lighting conditions. \n\n- The writing of the paper could be improved. There are typos and formatting issues in the paper. In particular, in line 208 it says the initial noisy latent is $\\hat{z}_m$, but line 243 says the initial noisy latent is $z_m$.\n\n- Finally, being training-free is of course attractive, but the current design mostly uses existing video diffusion model and IC-light to do the heavy-lifting while the new parts are preparing the trajectory to the video diffusion model. The difference between the variants of LightCtrl (Figure 5, last three rows) is not that substantial. This makes it unclear about the significance of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qf8pxnJYi9", "reviewer": "Reviewer_v3AC", "summary": "This paper introduces LightCtrl, a training-free controllable video relighting framework that enables explicit, fine-grained control of video illumination via a user-specified light trajectory. The method combines two pre-trained diffusion models an image relighting model (IC-Light) and a video diffusion model (VDM, e.g., AnimateDiff) to achieve temporally coherent and controllable lighting effects without retraining.\n\nThe method achieves strong quantitative and qualitative improvements over baselines such as IC-Light, SDEdit, and Light-A-Video, showing superior controllability, temporal coherence, and visual quality.", "questions": "1. GAR module. Empirically, the low-frequency fusion could still introduce temporal blurring or artifacts around illumination edges. A systematic perceptual analysis or patch-level FVD comparison would strengthen confidence in this design\n\n2. Light trajactory. Can this method be applied to : 1. Abrupt trajectory changes 2.Multiple moving light sources 3. Unaligned control inputs? Such tests would reveal whether LightCtrl’s latent-space conditioning is robust to domain shifts or natural illumination variance.\n\n3. Transferability across video diffusion backbones: Have you tested the method on alternative video diffusion priors (e.g., CogVideoX, VideoCrafter2)? Does controllability degrade or improve when switching to models with different motion representations or schedulers?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Originality: The proposed combination of LMI and GAR modules represents a creative hybridization of diffusion-based image and video models, enabling spatio-temporal control without additional data or model fine-tuning. \n\n2. Technical Quality. The methodology is clearly formulated. The Light Map Injection method is grounded in the diffusion process and introduces a principled way to guide illumination through noise manipulation. The Geometry-Aware Relighting component is well-justified. Leveraging normal maps via frequency-domain fusion to mitigate original light leakage is both technically sound and physically motivated.\nExperiments are comprehensive and clear.\n\n3. Clarity. The paper is well-written and visually rich, with clear figures illustrating the pipeline and ablation effects.\n\n4. Significance. This paper advances controllable generation from static to dynamic illumination domains, bridging image and video diffusion paradigms.", "weaknesses": "1. Physical Realism and 3D Awareness. The paper acknowledges limited 3D understanding of illumination. The method cannot simulate light scattering, occlusion, or volumetric effects, which restricts realism when the light trajectory crosses 3D geometry.\n\n2. Geometry-Aware Relighting (GAR) module. Although the GAR module suppresses source illumination, it fuses normal and RGB latents via a frequency-domain filter with dynamically decreasing cutoff. But there is no quantitative evidence that this schedule balances structure preservation and lighting suppression optimally.\n\n3. Limited Analysis of Control Robustness. The method only test on linear, circular and top–bottom light trajectory. More trajectorys should be tested.\n\n\n.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tz9qphozCa", "reviewer": "Reviewer_fsKh", "summary": "This paper proposes a training-free method for light trajectory editing in videos. The approach extends the single-image relighting model, IC-Light, to the video domain by incorporating priors from video diffusion models to ensure temporal consistency. The core technical contributions include a light map injection module, which introduces trajectory-aware noise into the VDM's latent space, and a geometry-aware relighting module. This second module processes both RGB frames and corresponding normal maps estimated via StableNormal to guide the relighting process. The results are visually compelling and demonstrate good adherence to the specified lighting trajectories.", "questions": "1. In Eq. (3), what exact $\\omega$ values are used across videos and trajectories? Is it fixed or tuned per video?\n2. Could you elaborate on the temporal stability of the StableNormal predictions? Are the normal maps computed independently per frame and fed directly into the video VAE, or is some form of temporal smoothing or other consistency enforcement applied?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper tackles a relatively unexplored task of lighting control in video diffusion models rather than just global style. The direction, ideas, and results are promising and the problem is quite interesting.\n2. By building on existing, well-known T2I and T2V models, the method is accessible and its components are easier to understand and potentially replicate.\n3. The strategy of injecting trajectory-aware noise into the VDM's initial latent space appears effective for guiding the lighting. This concept may be adaptable to other conditional video generation tasks.\n4. The design of the geometry-aware relighting module, which dynamically blends RGB and normal map information, is technically sound. It correctly reflects that surface geometry should remain invariant during a relighting task.", "weaknesses": "1. The use of PSNR_y against a pure white reference is not convincing. Relighting is a complex task involving light, geometry, and material, and does not necessarily mean \"the brighter the better\". This metric mainly captures pixel-wise brightness in 2D space and fails to account for the geometric and directional correctness of the illumination. It cannot distinguish excessive lighting and ignores geometry and material reflectance.\n2. The evaluation is somewhat limited. The dataset contains only 50 videos. This small scale and likely limited diversity are insufficient to robustly validate the method's generalizability. The paper would significantly benefit from showcasing more results across a wider variety of video content and lighting conditions.\n\nMinor issues:\n\nThere are plenty of papers about generative portrait relighting. Though they deal with portrait videos, I think they are still related to this topic. It is better to cite and discuss these papers. Following are some examples:\n\n+ Lumos: Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation\n\n+ Neural Video Portrait Relighting in Real-time via Consistency Modeling\n\n+ Real-time 3D-aware Portrait Video Relighting", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)