TC-Light: Temporally Consistent Relighting for Dynamic Long Videos
Authors: name: Yang Liu, name: Chuanchen Luo, name: Zimo Tang, name: Yingyan Li, name: Yuran Yang, name: Yuanyong Ning, name: Lue Fan, name: Junran Peng, name: Zhaoxiang Zhang

Editing illumination in long videos with complex dynamics has significant
value in various downstream tasks, including visual content creation and
manipulation, as well as data scaling up for embodied AI through sim2real and
real2real transfer. Nevertheless, existing video relighting techniques are
predominantly limited to portrait videos or fall into the bottleneck of
temporal consistency and computation efficiency. In this paper, we propose
TC-Light, a novel paradigm characterized by the proposed two-stage post
optimization mechanism. Starting from the video preliminarily relighted by an
inflated video relighting model, it optimizes appearance embedding in the first
stage to align global illumination. Then it optimizes the proposed canonical
video representation, i.e., Unique Video Tensor (UVT), to align fine-grained
texture and lighting in the second stage. To comprehensively evaluate
performance, we also establish a long and highly dynamic video benchmark.
Extensive experiments show that our method enables physically plausible
relighting results with superior temporal coherence and low computation cost.
The code and video demos are available at
https://dekuliutesla.github.io/tclight/.