VideoGrain: Modulating Space-Time Attention for Multi-grained Video
  Editing
Authors: name: Xiangpeng Yang, name: Linchao Zhu, name: Hehe Fan, name: Yi Yang

Recent advancements in diffusion models have significantly improved video
generation and editing capabilities. However, multi-grained video editing,
which encompasses class-level, instance-level, and part-level modifications,
remains a formidable challenge. The major difficulties in multi-grained editing
include semantic misalignment of text-to-region control and feature coupling
within the diffusion model. To address these difficulties, we present
VideoGrain, a zero-shot approach that modulates space-time (cross- and self-)
attention mechanisms to achieve fine-grained control over video content. We
enhance text-to-region control by amplifying each local prompt's attention to
its corresponding spatial-disentangled region while minimizing interactions
with irrelevant areas in cross-attention. Additionally, we improve feature
separation by increasing intra-region awareness and reducing inter-region
interference in self-attention. Extensive experiments demonstrate our method
achieves state-of-the-art performance in real-world scenarios. Our code, data,
and demos are available at https://knightyxp.github.io/VideoGrain_project_page/