Scaling Instruction-Based Video Editing with a High-Quality Synthetic
  Dataset
Authors: name: Qingyan Bai; name: Qiuyu Wang; name: Hao Ouyang; name: Yue Yu; name: Hanlin Wang; name: Wen Wang; name: Ka Leong Cheng; name: Shuailei Ma; name: Yanhong Zeng; name: Zichen Liu; name: Yinghao Xu; name: Yujun Shen; name: Qifeng Chen
Published: 2025-10-17 15:31:40

Abstract: Instruction-based video editing promises to democratize content creation, yet
its progress is severely hampered by the scarcity of large-scale, high-quality
training data. We introduce Ditto, a holistic framework designed to tackle this
fundamental challenge. At its heart, Ditto features a novel data generation
pipeline that fuses the creative diversity of a leading image editor with an
in-context video generator, overcoming the limited scope of existing models. To
make this process viable, our framework resolves the prohibitive cost-quality
trade-off by employing an efficient, distilled model architecture augmented by
a temporal enhancer, which simultaneously reduces computational overhead and
improves temporal coherence. Finally, to achieve full scalability, this entire
pipeline is driven by an intelligent agent that crafts diverse instructions and
rigorously filters the output, ensuring quality control at scale. Using this
framework, we invested over 12,000 GPU-days to build Ditto-1M, a new dataset of
one million high-fidelity video editing examples. We trained our model, Editto,
on Ditto-1M with a curriculum learning strategy. The results demonstrate
superior instruction-following ability and establish a new state-of-the-art in
instruction-based video editing.