CCEdit: Creative and Controllable Video Editing via Diffusion Models
Authors: name: Ruoyu Feng; name: Wenming Weng; name: Yanhui Wang; name: Yuhui Yuan; name: Jianmin Bao; name: Chong Luo; name: Zhibo Chen; name: Baining Guo
Published: 2023-09-28 15:03:44

Abstract: In this work, we present CCEdit, a versatile framework designed to address
the challenges of creative and controllable video editing. CCEdit accommodates
a wide spectrum of user editing requirements and enables enhanced creative
control through an innovative approach that decouples video structure and
appearance. We leverage the foundational ControlNet architecture to preserve
structural integrity, while seamlessly integrating adaptable temporal modules
compatible with state-of-the-art personalization techniques for text-to-image
generation, such as DreamBooth and LoRA.Furthermore, we introduce
reference-conditioned video editing, empowering users to exercise precise
creative control over video editing through the more manageable process of
editing key frames. Our extensive experimental evaluations confirm the
exceptional functionality and editing capabilities of the proposed CCEdit
framework. Demo video is available at
https://www.youtube.com/watch?v=UQw4jq-igN4.