STAR: Spatial-Temporal Augmentation with Text-to-Video Models for
  Real-World Video Super-Resolution
Authors: name: Rui Xie, name: Yinhong Liu, name: Penghao Zhou, name: Chen Zhao, name: Jun Zhou, name: Kai Zhang, name: Zhenyu Zhang, name: Jian Yang, name: Zhenheng Yang, name: Ying Tai

Image diffusion models have been adapted for real-world video
super-resolution to tackle over-smoothing issues in GAN-based methods. However,
these models struggle to maintain temporal consistency, as they are trained on
static images, limiting their ability to capture temporal dynamics effectively.
Integrating text-to-video (T2V) models into video super-resolution for improved
temporal modeling is straightforward. However, two key challenges remain:
artifacts introduced by complex degradations in real-world scenarios, and
compromised fidelity due to the strong generative capacity of powerful T2V
models (e.g., CogVideoX-5B). To enhance the spatio-temporal quality of
restored videos, we introduce~\name
(Spatial-Temporal Augmentation with T2V models for
Real-world video super-resolution), a novel approach that leverages
T2V models for real-world video super-resolution, achieving realistic spatial
details and robust temporal consistency. Specifically, we introduce a Local
Information Enhancement Module (LIEM) before the global attention block to
enrich local details and mitigate degradation artifacts. Moreover, we propose a
Dynamic Frequency (DF) Loss to reinforce fidelity, guiding the model to focus
on different frequency components across diffusion steps. Extensive experiments
demonstrate~\name~outperforms state-of-the-art methods on both
synthetic and real-world datasets.