TVG: A Training-free Transition Video Generation Method with Diffusion
  Models
Authors: name: Rui Zhang; name: Yaosen Chen; name: Yuegen Liu; name: Wei Wang; name: Xuming Wen; name: Hongxia Wang
Published: 2024-08-24 00:33:14

Abstract: Transition videos play a crucial role in media production, enhancing the flow
and coherence of visual narratives. Traditional methods like morphing often
lack artistic appeal and require specialized skills, limiting their
effectiveness. Recent advances in diffusion model-based video generation offer
new possibilities for creating transitions but face challenges such as poor
inter-frame relationship modeling and abrupt content changes. We propose a
novel training-free Transition Video Generation (TVG) approach using
video-level diffusion models that addresses these limitations without
additional training. Our method leverages Gaussian Process Regression
(GPR) to model latent representations, ensuring smooth and dynamic
transitions between frames. Additionally, we introduce interpolation-based
conditional controls and a Frequency-aware Bidirectional Fusion (FBiF)
architecture to enhance temporal control and transition reliability.
Evaluations of benchmark datasets and custom image pairs demonstrate the
effectiveness of our approach in generating high-quality smooth transition
videos. The code are provided in https://sobeymil.github.io/tvg.com.