StyleMaster: Stylize Your Video with Artistic Generation and Translation
Authors: name: Zixuan Ye; name: Huijuan Huang; name: Xintao Wang; name: Pengfei Wan; name: Di Zhang; name: Wenhan Luo
Published: 2024-12-10 18:44:08

Abstract: Style control has been popular in video generation models. Existing methods
often generate videos far from the given style, cause content leakage, and
struggle to transfer one video to the desired style. Our first observation is
that the style extraction stage matters, whereas existing methods emphasize
global style but ignore local textures. In order to bring texture features
while preventing content leakage, we filter content-related patches while
retaining style ones based on prompt-patch similarity; for global style
extraction, we generate a paired style dataset through model illusion to
facilitate contrastive learning, which greatly enhances the absolute style
consistency. Moreover, to fill in the image-to-video gap, we train a
lightweight motion adapter on still videos, which implicitly enhances
stylization extent, and enables our image-trained model to be seamlessly
applied to videos. Benefited from these efforts, our approach, StyleMaster, not
only achieves significant improvement in both style resemblance and temporal
coherence, but also can easily generalize to video style transfer with a gray
tile ControlNet. Extensive experiments and visualizations demonstrate that
StyleMaster significantly outperforms competitors, effectively generating
high-quality stylized videos that align with textual content and closely
resemble the style of reference images. Our project page is at
https://zixuan-ye.github.io/stylemaster