OmniInsert: Mask-Free Video Insertion of Any Reference via Diffusion
  Transformer Models
Authors: name: Jinshu Chen, name: Xinghui Li, name: Xu Bai, name: Tianxiang Ma, name: Pengze Zhang, name: Zhuowei Chen, name: Gen Li, name: Lijie Liu, name: Songtao Zhao, name: Bingchuan Li, name: Qian He

Recent advances in video insertion based on diffusion models are impressive.
However, existing methods rely on complex control signals but struggle with
subject consistency, limiting their practical applicability. In this paper, we
focus on the task of Mask-free Video Insertion and aim to resolve three key
challenges: data scarcity, subject-scene equilibrium, and insertion
harmonization. To address the data scarcity, we propose a new data pipeline
InsertPipe, constructing diverse cross-pair data automatically. Building upon
our data pipeline, we develop OmniInsert, a novel unified framework for
mask-free video insertion from both single and multiple subject references.
Specifically, to maintain subject-scene equilibrium, we introduce a simple yet
effective Condition-Specific Feature Injection mechanism to distinctly inject
multi-source conditions and propose a novel Progressive Training strategy that
enables the model to balance feature injection from subjects and source video.
Meanwhile, we design the Subject-Focused Loss to improve the detailed
appearance of the subjects. To further enhance insertion harmonization, we
propose an Insertive Preference Optimization methodology to optimize the model
by simulating human preferences, and incorporate a Context-Aware Rephraser
module during reference to seamlessly integrate the subject into the original
scenes. To address the lack of a benchmark for the field, we introduce
InsertBench, a comprehensive benchmark comprising diverse scenes with
meticulously selected subjects. Evaluation on InsertBench indicates OmniInsert
outperforms state-of-the-art closed-source commercial solutions. The code will
be released.