TransPixar: Advancing Text-to-Video Generation with Transparency
Authors: name: Luozhou Wang; name: Yijun Li; name: Zhifei Chen; name: Jui-Hsien Wang; name: Zhifei Zhang; name: He Zhang; name: Zhe Lin; name: Yingcong Chen
Published: 2025-01-06 13:32:16

Abstract: Text-to-video generative models have made significant strides, enabling
diverse applications in entertainment, advertising, and education. However,
generating RGBA video, which includes alpha channels for transparency, remains
a challenge due to limited datasets and the difficulty of adapting existing
models. Alpha channels are crucial for visual effects (VFX), allowing
transparent elements like smoke and reflections to blend seamlessly into
scenes. We introduce TransPixar, a method to extend pretrained video models for
RGBA generation while retaining the original RGB capabilities. TransPixar
leverages a diffusion transformer (DiT) architecture, incorporating
alpha-specific tokens and using LoRA-based fine-tuning to jointly generate RGB
and alpha channels with high consistency. By optimizing attention mechanisms,
TransPixar preserves the strengths of the original RGB model and achieves
strong alignment between RGB and alpha channels despite limited training data.
Our approach effectively generates diverse and consistent RGBA videos,
advancing the possibilities for VFX and interactive content creation.