SeedVR2: One-Step Video Restoration via Diffusion Adversarial
  Post-Training
Authors: name: Jianyi Wang, name: Shanchuan Lin, name: Zhijie Lin, name: Yuxi Ren, name: Meng Wei, name: Zongsheng Yue, name: Shangchen Zhou, name: Hao Chen, name: Yang Zhao, name: Ceyuan Yang, name: Xuefeng Xiao, name: Chen Change Loy, name: Lu Jiang
Organization: ByteDance Seed

Recent advances in diffusion-based video restoration (VR) demonstrate
significant improvement in visual quality, yet yield a prohibitive
computational cost during inference. While several distillation-based
approaches have exhibited the potential of one-step image restoration,
extending existing approaches to VR remains challenging and underexplored,
particularly when dealing with high-resolution video in real-world settings. In
this work, we propose a one-step diffusion-based VR model, termed as SeedVR2,
which performs adversarial VR training against real data. To handle the
challenging high-resolution VR within a single step, we introduce several
enhancements to both model architecture and training procedures. Specifically,
an adaptive window attention mechanism is proposed, where the window size is
dynamically adjusted to fit the output resolutions, avoiding window
inconsistency observed under high-resolution VR using window attention with a
predefined window size. To stabilize and improve the adversarial post-training
towards VR, we further verify the effectiveness of a series of losses,
including a proposed feature matching loss without significantly sacrificing
training efficiency. Extensive experiments show that SeedVR2 can achieve
comparable or even better performance compared with existing VR approaches in a
single step.