GenCompositor: Generative Video Compositing with Diffusion Transformer
Authors: Shuzhou Yang
Xiaoyu Li
Xiaodong Cun
Guangzhi Wang
Lingen Li
Ying Shan
Jian Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion Models
Video Editing
Video Compositing
Primary area: generative models
TL;DR: GenCompositor is capable of effortlessly compositing different videos guided by user-specified trajectories and scales.

Abstract: Video compositing combines live-action footage to create video production, serving as a crucial technique in video creation and film production. Traditional pipelines require intensive labor efforts and expert collaboration, resulting in lengthy production cycles and high manpower costs. To address this issue, we automate this process with generative models, called generative video compositing. This new task strives to adaptively inject identity and motion information of foreground video to the target video in an interactive manner, allowing users to customize the size, motion trajectory, and other attributes of the dynamic elements added in final video. Specifically, we designed a novel Diffusion Transformer (DiT) pipeline based on its intrinsic properties. To maintain consistency of the target video before and after editing, we revised a light-weight DiT-based background preservation branch with masked token injection. As to inherit dynamic elements from other sources, a DiT fusion block is proposed using full self-attention, along with a simple yet effective foreground augmentation for training. Besides, for fusing background and foreground videos with different layouts based on user control, we developed a novel position embedding, named Extended Rotary Position Embedding (ERoPE). Finally, we curated a dataset comprising 61K sets of videos for our new task, called VideoComp. This data includes complete dynamic elements and high-quality target videos. Experiments demonstrate that our method effectively realizes generative video compositing, outperforming existing possible solutions in fidelity and consistency. Project is available at https://gencompositor.github.io/

Published as a conference paper at ICLR 2026
GENCOMPOSITOR:
GENERATIVE VIDEO COMPOSIT-
ING WITH DIFFUSION TRANSFORMER
Shuzhou Yang!
Xiaoyu Li Xiaodong Cun
Guangzhi Wang"
Lingen Lix
Ying Shan
Jian Zhangl.3f
School of Electronic and Computer Engineering, Peking University 2 ARC Lab. Tencent PCG
Guanggong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology
AGCC Lab, Great Bay University
"The Chinese University of Hong Kong
ft. video
realts hp. video
tt- video
bp. video
reutts
Figure I: GenCompositor is capable of effortlessly compositing different videos following user- specified trajectories and scales. It could preserve the background video content and also seamlessly integrate the dynamic foreground elements into the background video, which not only strictly fol- lows user-given instructions but also physically coordinates with background environments.
ABSTRACT
Video compositing combines live-action footage create video production, serv- ing as a crucial technique in video creation and film production. Traditional pipelines require intensive labor efforts and expert collaboration, resulting lengthy production cycles and high manpower costs. To address this issue, we automate this process with generative models, called generative video composit- ing. This new task strives to adaptively inject identity and motion information of foreground video the target video an interactive manner, allowing users customize the size, motion trajectory, and other attributes of the dynamic el- ements added in final video. Specifically, we designed novel Diffusion Trans- former (DiT) pipeline based its intrinsic properties. To maintain consistency the target video before and after editing, we revised a lightt DiT-based background preservation branch with masked token injection. As to inherit dy- namic elements from other sources, a DiT fusion block is proposed using full self-attention, along with simple yet effective foreground augmentation for train- ing. Besides, for fusing background and foreground videos with different layouts based on user control, developed novel position embedding, named Extended Rotary Position Embedding (ERoPE). Finally, we curated dataset comprising 61K sets of videos for our new task, called VideoComp. This data includes com- plete dynamic elements and high-quality target videos. Experiments demonstrate that our method effectively realizes generative video compositing, outperform- ing existing possible solutions in fidelity and consistency. Project is available at Ittps: httpss///gnccoooosiioogoo wwt.eiitooriiiiiiiio.ei.oi github.ioi...
Project leader.
Corresponding author.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Video compositing aims to edit target background video by adding foreground video from other sources, creating visually pleasing video assets. Compared to text- or image-guided video edit- ing, this process edits video productions based on real-world captured videos, acting as a bridge between raw live-action video footage and final video work. However, classical process is labor in- tensive, which requires collaborative efforts of animators, videographers, and special effects artists. In this paper, we present generative video compositing, new video editing task that attempts to automate compositing process with generative models (Ho et al. 2022 Blattmann et al. 2023; Li et al. 2025a). It allows direct editing of background videos using video footage under user control, aligning with traditional video creation process in video compositing stage.
To composite visually appealing video results, three main challenges explored in this paper: 1) Ensuring background consistency of the video before and after editing. 2) Preserving the identity and motion of the injected dynamic elements while harmonizing with background content. 3) Enabling flexible user control, such as controllable size and motion trajectories. However, existing solutions cannot resolve these issues well. Controllable video generation (Hul 2024 et al., 2026; Hu al., 2025 Wang et al.) 2025b) produces videos based on external conditions, including user-defined trajectories, images, and texts. Here, images provide identity information, and texts describe the motion to be generated. But images and text cannot precisely control the added elements at the pixel level, and current methods do not support external video condition. In Contrast, video harmonization methods (Lu et al.) 2022 Chen et al. 2024 Xiao et al.) 2024) directly paste foreground element onto the target video frame- frame-by-frame by-frame, and adjust its RGB values to blend with background. However, these cannot adaptively specify the position, size, motion trajectory. and other attributes of the added element. Moreover, pasting foreground video requires accurate corresponding segmentation masks, which may not always be feasible in practical utilization.
To this end, we propose the first generative video compositing method, GenCompositor, aiming to automatically composite background video with dynamic foreground elements. As shown in Fig. [ given a foreground video, a background video, and a user-provided trajectory in background (depicted by red line). GenCompositor injects foreground element to background video faithfully following trajectory. Trajectory can be specified by dragging a line or only clicking point. For the latter, we track the movement of point with optical flow to simulate drag line. Our model can even predict realistic interaction between added element and background. For example, in Fig. (a), the explosion effect influences the background content, i.e., the car's fuel tank disappears after explosion in the last frame. In Fig. (b), GenComposing predicts realistic shadow of the added butterfly across frames, whose direction and effect consistent with background lighting.
We realize this by curating the first high-quality dataset for training, and proposing novel Diffu- sion Transformer (DiT) pipeline specifically tailored for generative video compositing. Our pipeline consists of three main components. Firstly, lightweight DiT-based background preservation branch is designed to ensure the consistency of the background in the edited results with the input videos. Secondly, inject dynamic foreground elements, propose DIT fusion block with full self- attention to fuse tokens from foreground elements with that of background videos, and conduct detailed ablation study to demonstrate the advantages of this design over current cross-altention injection approaches (Ye et al., 2023 Cao et al. 2023: Wang et al., 2024a|c) & Yang et al.) 2025a). Considering that layouts of added foreground elements should follow user control and often differ from those of background videos, directly applying RoPE to foreground tokens introduces leakage artifacts. Therefore, we propose a novel position embedding method, Extended Rotary Position Em- bedding (ERoPE), to adaptively adjust the positions and scales of foreground tokens, which enables high-quality user-specified conditional generation even under layout-unaligned conditions. This technique is theoretically applicable to other tasks that exploit layout-unaligned video conditions.
Moreover, we develop some practical operations to enhance generalization and robustness. On the one hand, luminance augmentation is applied to foreground video for training, which strengthens the model's 's generalization to diverse foreground conditions. On the other hand, we propose mask inflation to feed inaccurate masks to the model, enabling realistic interaction between newly added objects and environment. Experiments show that GenCompositor enables high-quality video com- positing and can be used to make video effects. Overall, our contributions are follows:
2

---

Published as a conference paper at ICLR 2026
We propose new practical video editing task, generative video compositing, and the first feasible solution that can automatically inject dynamic footage into the target video a generative manner, utilizing the proposed novel diffusion architecture.
We design some novel techniques targeted the unique characteristics of video composit- ing, including revised position embedding, a full self-attention DiT fusion block, and lightweight background preservation branch. These provide references for future research.
To train this model, we elaborate data curation pipeline and develop some practical train- ing operations, such as mask inflation and luminance augmentation, to improve the gener- alization ability and robustness of algorithm.
Experiments prove that the proposed method outperforms existing potential solutions, ables effortless generative video compositing based on user-given instructions, and can be used for automatic video effects creation. This inspires future research in this area.
2 RELATED WORK
2.1 DIFFUSION-BASED VIDEO EDITING
In the field of Al-generated content, video editing (Brooks et al.
2023 Esser al.
2023; Shin
ctt al.
2024) has made great progress with the success of diffusion models (Yng al.
2025d: Li
jct al. 2025b Kong et al. 2024 Wang et al. 2025a; Ju et al. 2025 Li et al. 2025c). Early works
attempted to use priors from pre-trained models (Ceyla et al.
2023
Yang al. 2024
2025b Ma
jt al. 2025). Such as finetuning T2I models (Wu et al. (023) or designing attention mechanisms
tailored to video (Cai et al.) 2025 Ql et al. 2023] Wang et al. 2023 Yatim et al.) 2025
Guo al.
2024a). Recently, some methods trained specialized video editing models to realize more effective
and prolific effect. [Mou al. (2024a) trained two ControlNet (Zhang et al.
2023) branches to
specify motion and inject ID, respectively, Tu et al. (2025) followed similar strategy, but enabled more fine-gratd tine-grained control through denser key points and trajectories. [ et (2025) trained two
control branches to guide spatial and temporal editing respectively. |Bian et al. (2025) trained a video inpainting model, and edited videos through masking the target area, using Flux (Blackforestlabs 2024) to edit the first frame, and painted subsequent frames to propagate the editing effect.
In general, training a dedicated model achieves better performance than training-free methods. How- ever, existing approaches mainly edit videos based on images or textual prompts, making it hard to precisely control the appearance and motion details of the editing effects. To this end, we first pro- pose generative video compositing task, which directly edits videos based dynamic footage from other sources, realizing more practical and fine- ane-ggrined video editing.
2.2 VIDEO HARMONIZATION
Video harmonization is classical low-level vision task (Li et al., 2024 Dong al. 2024 Tan
et al. 2025), which aims to adjust the lighting of added foreground element in the composited video make it visually harmonious. Similar other low-level methods (Liu et 2022 Yang al. 2023 Huang et al. (220)) first introduced adversarial training to video harmonization. Lu et al.
(2022 collected a dedicated dataset for training. Ke et al. (2022) attempted to work in a white-box manner. which regresses the image-level filter argument to predict harmonized results. Guo et al. (2024b) directly trained Triplet Transformer to simultaneously resolve multiple low-level video tasks, including harmonization. However, all of these methods focus only on adjusting the color of added elements. They all operate on composited videos and require foreground video and its accurate and pixel-aligned mask input. Besides, these methods do not allow users to freely modify the size and motion trajectory of the added elements the final video. To address these limitations, introduce generative video compositing task that accommodates arbitrary mask inputs. Using foreground vidcos, source background videos, and user-defined trajectories and scales, our model autonomously generates composited videos that adhere all conditions.
3

---

Published as a conference paper at ICLR 2026
3 TASK DEFINITION
Here, we define the task of generative video compositing. The inputs to this task are: a background video vo be edited, a foreground video v to be injected, and user-given control c. The objective is to composite Va and vy following c, resulting in the final output Zo-
Compared with other video editing methods, there some unique characteristies of this problem. First, the layout of the videos to be edited V, is highly consistent with editing results Zo. Meanwhile, layouts of condition vy are not pixel-aligned with Vg. We should consider these two different condi- tions to generate results Zo. Latent Diffusion Model (LDM) (Rombach et al.) 2022) is employed to realize this, starting from a random latent noise Zr, we aim denoise Zy Zy based on the three in- puts mentioned above. For training, given the Ground Truth video Zo containing desired foreground element. We add noise e of various scales to it following a predefined schedule defined as:
2 v Z
vI ājc.
Training process aims to predict the added noise e in Z, by network ea(-), objective function is:
min Ex. Woi. c0(z, vo. vy,c, f)
where t is sampling step. Vo. Vy. and are conditions. After training, ca(-) receives random noise Zr, denoises step by step, and decodes final latent through VAE decoder for composited results.
4 METHOD
We introduce input conversion in Sect. showing how we convert and augment user-given instruc- tions. Then, explain the proposed pipeline, including background preservation branch (Sect. 4.2)
DiT fusion block (Sect. (4.3) and Extended Rotary Position Embedding (ERoPE) (Sect. 4.4).
4.1 INPUT CONVERSION
For utilizzation, our inputs include a background video Vo. a foreground footage Vy, as well as the user-given scale factor s and trajectory curve. We denote the user control as u {7,s}, where
y 2D trajectory on the first frame of vs. We rasterize into per-frame binary mask video
M = (M. M, E (0. H) and masked video X = (X.) defined by
x, (1- Vo.e. t=
(3)
where denotes element-wise multiplication. In Eq. 2 we thus set the conditioning variable to = (M, X), i.e., c0(2; v6, V,,Y,,e, c.t, = c Vo, y. M. X, t). In this way, the user trajectory and scale are encoded entirely through the mask and masked background videos fed into the network.
As shown in the left of Fig. 2 our compositing starts from two input videos (background and fore- ground). Given a background video to be edited, users drag trajectory on its first frame, denoting the movement of added elements. Alternatively, users can click point (as shown in Fig. [ (a)), and our method will automatically track the trajectory of this point in subsequent frames based on video optical flow. On this basis, the remaining problem is to determine the size of the newly added element. For the foreground input, we get its corresponding binary mask video through Grounded SAM al. 2024), then adjust its region size according to the user-given rescale factor. Fi- nally, we adjust the position of this rescaled mask video based on the designated trajectory curve. In this way, adaptively rescale and reposition foreground masks produce mask video.
To integrate with environment realistically, a Gaussian filter is used to inflate mask. Given the binary foreground mask video Mhin (Mbin) J mask is inflated define buffered editable band around the object. We smooth Mhin with a 2D Gaussian kernel Go and then apply a threshold:
M, G. Mbin
M. = 1M, r]. t=
(4)
where denotes convolution. Intuitively, M. slightly extends the foreground region beyond the object boundary, creating narrow buffer band in which the model is allowed to modify background content and hallucinate interactions (e.g., shadows, glow, motion blur). At the same time, when SAM2 fails, this inflated mask explicitly absorbs pixel-level misalignments to support imperfect
4

---

Published as a conference paper at ICLR 2026
Prichily
taken
Bg
Preservation Branch
scatep2.6
DVT
Bluck
DIT
Bllek
b9
& pet tra)
Tellutton
Pachhfy
VAE Deonde
DV1 Fusco Block
DVT Fusioooo Block
VILE Eacodee
video Vi.
Basked video
Fosim
x21
x21 ...
Luminana Augmentation
Pauchiyy
4g Tekee
composited Figure 2: Workflow of Com GenCompositor takes a background video and a fore- ground video imput. Users specify the trajectory and scale of the added foreground elements. We firstly convert user-given instructions to model inputs, then generate with a background preser- vation branch and a foreground generation mainstream, consisting of the proposed ERoPE and DiT fusion blocks. Following this, our model automatically composites input videos.
fg. video V
Fig. Generation Mainstream
DiT' Block
x(1
Bask video
SLEEI HOULE
Hotal
fg. token noisy token
DiT Fusion Block
Layer Nomn
5D Salf Sol--atemioon
Layer Nomn
Feed Forward Net
RoPE
Cate
Cate
2h
Figure 3: DiT fusion block receives concatenated
ERoPE
tokens, some are to-be-generate tokens (blue) and some unaligned conditional tokens (green). It fuses the two via pure self-attention. The final to- kens with gradient color represent the mixture of generated tokens and masked conditional tokens from the BPBranch.
Figure 4: RoPE and ERoPE. (A) RoPE assigns labels to embeddings of vidcos. (B) For videas with inconsistent layout, our ERoPE extends RoPE by assigning unique labels to each embedding from dif- ferent videos, avoiding interference.
masks. We also train our model with masks of SAM2, and the trained end-to-end model has a certain mask correction capability when utilization. Finally, masked video can be obtained by simply masking the object in the source video with mask video for training. Our model actually receives (i) foreground video, (ii) inflated mask video, and (iii) masked video, to generate realistic output. This forces the model, during training, to explain pixels in the whole band around the object using both foreground and background cues, enabling local interaction.
4.2 BACKGROUND PRESERVATION BRANCH
As a video editing task, our main goal is to preserve non-editing content and only edit desired regions of va. We propose Background Preservation Branch (BPBranch) to ensure consistency between composited result and vg. Considering the overall layout of edited result is pixel-aligned with that of vo. Directly plus the latent v, with model latent has been able to faithfully preserve non-editing content, which is proven by previous ControlNet-like methods (Mou et al. 2024b; Ju et al. 2024).
Intuitively, only using masked video has been able to inject background. As shown in Fig. in masked video, the area where we aim to insert the element is marked black. However, some back- ground videos naturally contain black content, which not the desired masks and could confuse the network. Hence, we concatenate the masked video with its corresponding mask video as input. This guides the branch focus on background preservation.A As mask video and masked video are pixel-aligned, we apply the same Rotary Position Embedding (RoPE) (Su et al., 2024) to both, as shown in Fig. A). This strictly aligns the positions of the two and enables precise mask guidance.
Subsequently, we inject these to foreground generation mainstream for compositing As BPBranch designed to inject background only, its main purpose is to align features of masked video with that of mainstream model. Instead of deeply extracting masked features, simply design lightweight control branch, which consists of two normal DiT blocks, to align with the latent of mainstream. Meanwhile, only want use the background region, masked token injection method is
5

---

Published as a conference paper at ICLR 2026
applied to prevent interference from BPBranch to foreground region, which is formulated as:
2x 2x (1 M) @ ZBP Branch,
(5)
where is the latent from mainstream, and Branhh is the output of BPBranch. This process is visualized detail in the upper part of Fig.
4.3 FOREGROUND GENERATION MAINSTREAM
To composite foreground footage, our goal is to faithfully preserve the identity and dynamic features of foreground elements from other sources in composited results. cross-attention is commonly used inject conditions (FU al., 2025 Yu et al. (022)), such texts camera poses. However, we find that although cross-attention could address semantic conditions, it does not effectively utilize low- level conditional information for our task. To faithfully inherit foreground condition, we propose to concatenate the tokens of foreground with the tokens to be denoised as shown in Fig, 2 then calculate its self-attention fully fuse these two messages through the DiT fusion block.
As depicted in Fig. B given the tokens of noisy latent and foreground condition, DiT fusion block concatenates them a token-wise manner, instead of classical channel- wise concatenation. This is because the layout features of foreground condition and generated results are not pixel-aligned. Roughly concatenating their features in channel dimension will cause severe content interference, which leads to training collapse. DiT fusion block then predicts the noise in noisy latent by calculat- ing self-attention on the concatenated tokens, which contain both foreground condition and masked background. Note that our generated results the processed latent that is boxed by red in Fig. 3 Hence, we fuse the tokens of BPBranch only with the processed noisy tokens and pass them to the next block. Finally, we only decode the part corresponding to the input noisy tokens to obtain composited video, as shown in the right of Fig. 2
Meanwhile, newly added content should visually coordinate with background. Some of its attributes (e.g., lighting), need to be properly adjusted during generation. To enable our model to learn this coordination adaptively, we develop luminance augmentation strategy for training. In each it- eration, we use gamma correction to the foreground video, with the gamma parameter randomly selected from range of 0.4 1.9. This changes the lightness of foreground condition to offset from source video. Consequently, based on our DiT fusion block that fully fuses foreground ele- ments with model latent, foreground generation mainstream automatically learns the capabilities of foreground harmonization. Notice that luminance augmentation is only used in the training process.
4.4 EXTENDED ROTARY POSITION EMBEDDING
We distinguish two types of conditions with respect to the target layout. Given the background and target layout (Vs, zo). we call condition video v layout-aligned if v(z, v) and zo (I. v) refer to the same spatial location in the image plane, and layout--maligned otherwise. In Fig. 2 the mask video M and masked video X are layout-aligned with (vt, 2o), while the foreground video y is layout-unaligned, since its dynamic content is centered and follows its own camera frame.
Layout-aligned conditions can be well uti- lized by sharing the same RoPE. However, this causes content interference for layout- unaligned conditions. As shown in Fig. 5 "w/ RoPE" directly shares RoPE between layout- aligned (ve. X)) and snyout-unaligned v,,
background video foregeound video mask videe Figure ERoPE is superior in fusing layout- unaligned videos. As dynamic content in fore- ground footage centered, using RoPE leads to content interference as shown in red box of "w/ RoPE", while our ERoPE resolves this issue well.
w/ RoPt
which leads to obvious artifacts in composited results (boxed in red). Whose shapes and posi- tions are consistent with that of foreground ele- ments. Another optional way is cross-attention, which extracts abstract semantics to support unaoun-dd control signals. However, such a high-level feature cannot faithfully inherit detailed appearance and action features of fore- ground videos and is not suitable for video compositing, as proved in Sect. [5.
To this end, we developed a new embedding strategy tailored to the unaligned nature between foreground footage and background video, named ERoPE. It still takes the same four inputs

---

Published as a conference paper at ICLR 2026
ranually paste
Harmonizer
Video ripp riplet Fransforner
GenCorpositor
Figure 6: Visual comparison of video harmonization. The compared methods cannot achieve satisfactory results with jagged artifacts at the edges of foreground elements, inconsistent color or lighting, while our method achieves better performance.
(Ve, M, X, vÿ) ) as in Fig. 2 but changes the position embedding: we assign distinct positional labels to layout-unaligned tokens so that foreground and background videos can be fused without occupy- ing the same spatial positions
the latent space. Concretely, RoPE (Su et al. 2024), each token at ID position index has its query/key components divided into 2D pairs and rotated by an angle proportional to For the k-th frequency wi and 2D query component (928> 928+1). we have:
g2k+1
L2k
R(0p.))
)
92k
Og,s = wkP, R(0)
cos sin 0
sin 0
cos 0
and the same to the key component. Resulting attention score depends only on the relative position:
(gg k₂) (f(w(p--))))
which implicitly assumes all tokens lie a single spatio-temporal grid shown in Fig. A
In our setting, tokens come from ForeGround and BackGround videos s E (BG, FG) with incom- patible spatial coordinates. ERoPE introduces stream-specific shift on top of RoPR:
s.ppk = wx(p As).
(BG,
(8)
where Ag is constant offset for stream Let (s.p R and ks', E R denote the unrotated query and key for a token from stream at position p and token from stream s' at position respectively. Here s, E (BG, FG] indicate the source streams of the query and key tokens. They may be equal for within-stream attention or different for cross-stream attention. We write Ls,p and K/, for their ERoPE-rotated versions obtained by applying R00.pp.)) and R(0s.q..)) to each (2k, 2k + 1) pair. Using the identity R()) R(()) R(á 0) and writing qs.p.A = (gs.p.2k, (x.p.2k+1) ks..g. (I a.2k Ke n. 2x.4+1) T for 2D slices of qx and ki .a attention score between these two tokens is:
K. Σ TT. R(fx (s.p.k) korgq.
E k R(w Axr) (p (p+) ky'.
In other words, the contribution of each frequency depends only on the relative effective positions (p + A.) and (q + Agr), and hence on their difference wp + A,) (q+ Ar.)]. Choosing large, distinct shifts A, for BG and FG prevents "same-positie collisions across streams. while still al- lowing attention to learn meaningful cross-stream interactions. As shown Fig. 4B). ERoPE implemented as stream-specific shifts A, on top of standard RoPE, introduces no additional param- eters, and is crucial for stable fusion of layout-unaligned foreground and background videos in our compositing setting. As shown in Fig. 5 this strategy efficiently increases compositing performance and eliminates artifacts caused by interference of unaligned content. Moreover, considering that there three optional shift directions for our ERoPE (e.g., width, height, and timing), this paper conducts ablation Sect. of appendix, and demonstrates their equality. We believe ERoPE is a practical tool for numerous aayout-unaligned editing tasks, and here use it for video compositing.

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS
Given that there are no existing works for generative video compositing like ours, we compare with two related tasks, video harmonization, and trajectory-controlled video generation. We first introduce implementation details in Sect. P.I. showcase comparisons in Sect. 5.2 and Sect. 5.3 To validate the effectiveness of our key components, we conduct an ablation study in Sect. 5.4
Table 1: Quantitative comparison with video harmonization methods. The best results are
Quantitative results of comparison with trajectory controlledd generation. The best results are highlighted in bold.
Table 2:
highlighted in bold.
Metries
Harmonizer
VTT
GenCompositor
Metrics
Tora
Reviden
VACE
GenCompositer
PSNR 1
39.7558
40.0251
42.0010
Subject
88,442 48.44%
88.02%
89.51%
89.75%
SSIM
0,9402
0.9297
0.9487
Motion
Background
92.45%
98.03%
92.90%
96.85%
98.21%
92.63%
98.69%
93.43%
CLIP T
0,9614
0.9564
0.9713
Aesthetic 1
49.33%
48.56%
49.30%
52.00%
LPIPS
0.0412
0.0455
0.0385
FVD
1402.82
1342.56
942.52
$35.71
94.94
120.92
5.1
IMPLEMENTATION DETAILS
GenCompositor is a DIT model with 6B Transformer, consisting of the proposed DiT fusion blocks and background preservation branch. We reuse pre-trained VAE module of CogVideoX to generate latent space. We train our new architecture on H20 GPUs from scratch. In inference, GenCom- positor takes over 65s to generate a video at 480 720 with 49 frames within 34GB VRAM.
5.2 COMPARISON WITH VIDEO HARMONIZATION
Considering the limited number of open-source methods in video harmonization, we compare our method with two recent approaches whose codes are available: Harmonizer (Ke et al. 2022) and VTT (Guo et al., 2024b). As these methods cannot control the motion trajectory, we only compare the ability to harmonize foreground elements. We also manually paste foreground footage from other sources into the background video for comparison. As shown in left of Fig. 6 noticeable jagged artifacts appear at the edges of added elements in manually paste and Harmonizer, due to the imperfection of the segmentation mask of the foreground video. Additionally, the color style of the newly added explosion effect does not harmonize well with background video in these harmo- nization approaches. In contrast, our method effectively addresses these jagged artifacts caused by inaccurate masks and produces more harmonious results. In other examples of Fig. 66, we manually adjust the lighting of the foreground elements and test the video harmonization ability. Our method consistently outperforms the other methods, demonstrating its superiority.
For quantitative comparison, we use the well-known YouTTube (Lu et al. 2022) dataset, where the foreground videos, segmentation masks, and source vidcos are available. Four well-known metrics, PSNR, SSIM (Wang et al.) 2004), CLIP (Radford al. 2021 and LPIPS (Zhang al., 2018) are used to measure performance in Tab. [ One can see that GenCompositor outperforms other harmonization methods across all metrics, showing its superiority over specialized methods in video harmonization. We also conduct user study in Sect. E) appendix.
5.3 COMPARISON WITH CONTROLLABLE GENERATION
Our method enables wajectory-controllee generation, where users can specify the motion trajectory and size of newly added elements in results. Considering our added element is dynamic video and there is no method using the same condition as ours, we compare with SOTA trajectory-contren
video generation and editing methods, Tora (Zhang et al. (025), Revideo (Mou et al. 2024a). and VACE (Jiang et al. (025).. Visual results are given in Fig. 7 where we report background videos, foreground videos, Tora generation results containing the red trajectory curves, Revideo editing re- sults, VACE generation, and our results. Note that Tora and VACE still require additional textual prompts as condition, Revideo and VACE have to edit the first frame as image condition, but Gen- Compositor requires neither of these priors. Although Tora and VACE could generate results that follow the trajectory, Tora cannot maintain the ID consistency of the added element and cannot strictly follow the user-specified trajectory. VACE gets object information only from the first-frame reference image, tends to mistake the object we wanted to inject and cannot predicts faithful dynam- ics. In contrast, GenCompositor could strictly follow the trajectories to generate composited videos,
8

---

Published as a conference paper at ICLR 2026
foreground video
background video
ReVideo
VACE
GenCompositon itor
Figure 7: Visual comparison of trajectory-connrollee video generation. All methods share the same trajectory, which is plotted as red line in the results of Tora. All compared methods need to receive the pre-edited Is--frame as condition image. Tora and Revideo generate temporally in- consistent foreground elements. VACE mistakes the element details we actually want to inject. Our method resolves these issues well without the preprocess of editing the first frame.
Table 3: Quantitative results of ablation study. The best results are highlighted in bold.
Settings
PSNR
SSIM
CLIP
LPIPS
Subject
Background
Motion
Aesthetic f
w/o fusion block
19,8940
0.8015
0.9341
0.1535
88.85%
92.21%
98.34%
48.85%
w/o BPBranch
40.0099
0.9378
0.9709
0.0432
88.77%
89.62%
97.25%
51.51%
w/o augmentation
39.8040
0.9295
0.9629
0.0520
88.00%
89.97%
98.30%
50.73%
w/o mask inflation
41.8553
0,9422
0.9701
0.0409
89.72%
91.62%
98.28%
50.87%
fullmodel
42.0010
0.9487
0.9713
0.0385
89.75%
93.43%
98.69%
52.00%
and the ID and motion of the element are inherited from foreground videos faithfully. We believe these advantages come from different task settings. Compared with generating a video from an image such as Tora, or inpainting based on only single reference such as VACE, we composite fore- ground video following the trajectories. This is inherently more conducive to inheriting the ID and detailed motion of foreground elements. Meanwhile, Revideo also aims to drag the added element in the first frame to move along given trajectory in subsequent frames, but its limited performance leads to non-robust results, elements may disappear in predicted frames as shown in Fig. 7
To conduct quantitative evaluations, we utilize two well-known generation quality metrics: FVD (Unterthiner et al. 2019) and KVD, and a common benchmark, i.e., VBench (Huang et al.) Zheng et al. 2025), to analyze the quality of generation from 4 dimensions: 1) Subject Con- sistency: consistency about subjects in the video. 2) Background Consistency: consistency about the video background. 3) Motion Smoothness: motion quality of the generated video. 4) Aesthetic Quality: subjective visual quality of the video. We believe these four metrics to be most relevant to task and generate 40 sets of videos using Tora, Revideo, VACE, and our method, respectively. As shown in Tab. [] our method achieves the best average scores across all six metrics. In addition, conduct user study in Sect. E of appendix compare intuitive quality of different methods.
2024
5.4 ABLATION STUDY
To analyze effectiveness of each component, we conduct ablation study with four settings. Specifi- cally, we attempt two potential designs of GenCompositor, one removes the background preservation branch (w/o BPBranch), the other uses cross- attention to inject foreground elements without the pro- posed DiT fusion block (w/o fusion block). We discuss their architecture details in Sect. G We also remove two designs. One does not inflate binary mask (w/o mask inflation), the other directly inputs original foreground videos for training, without luminance augmentation (w/o augmentation).

---

Published as a conference paper at ICLR 2026
foreground element
inflated nask
nanually
fusion block
w/o BPBranch
w/o augnentation
inflation
Fullnodel
Figure 8: Visual ablation results. Manually paste results own jagged artifacts. "w/o fusion block" cannot inject element faithfully. "w/o augmentation" and "w/o mask inflation" showcase jagged arti- facts at the edge of element. "w/o BPBranch" cannot realistically adjust added element. "fullmodel" setting performs best, which naturally fuses foreground elements with background video.
For visual comparison in Fig. D, we provide the foreground element and inflated mask in the first row. Following three rows show the results different settings. Where "manually paste". "w/o aug- mentation" and "w/o mask inflation" all exhibit obvious jagged artifacts the edges of foreground elements. We believe that, for "w/o augmentation", due to the powerful learning ability of network, it totally inherits and overfits the content of the foreground video without any adjustment. For "w/o mask inflation", as we provide pixel-aligned mask to model, it can only process the foreground element in this limited region and cannot adjust surrounding pixels fuse with the background, leaving artifacts at the edge. The other two settings address the border artifacts, but present other limitations. "w/o fusion block" cannot faithfully inject ID and dynamics of foreground elements, but successfully predicts the flame, which is semantically consistent with the foreground condition, indicating that cross-attention is good at injecting semantic information, but is not applicable in our task. Although "w/o BPBranch" also produces a realistic background, this end-to-end learning of both background video and added foreground elements increases training difficulty, limiting its performance. In its results, the foreground element is not perceptually consistent with main video.
For quantitative evaluation, we use four ablation settings, together with "fullmodel", to realize video harmonization and trajectory-controlle as Sect. 15.2 and Sect. 5.3 As shown in Tab. "fullmodel" outperforms ablation settings on all metrics. We believe this objectively demonstrates the significance of cach component in our method.
video generation, respectively. Related test data is the same
CONCLUSION
This paper introduces a novel video editing task, generative video compositing, which allows in- teractive video editing using dynamic visual elements. Specifically, we developed the first gen- erative method, GenCompositor, which is designed to address three main challenges of this task: maintaining content consistency before and after editing, injecting video elements, and facilitating user control. It comprises three main contributions. Firstly, lightweight background preserva- tion branch utilized inject tokens of background videos into the mainstream. Secondly, the foreground generation mainstream incorporates novel DiT fusion blocks to effectively fuse the ex- ternal video condition with the background latent. Finally, we revised novel position embedding, ERoPE, to force the model to add external elements to the desired positions in results with desired scales, adaptively. Notice that ERoPE points out a new effective way to utilize layout-unaligned video conditions for generative model without any additional computational cost. The first paired dataset, VideoComp, is also proposed in this paper. Comprehensive experiments demonstrate the effectiveness and practicality of our proposed method.
Acknowledgments. This work was financially supported in part by National Natural Science Foundation of China (62372016), Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology 122441212011 20000) Shenzhen Science and Technology Program (SYSPG2024121 173344000)))) and Outstanding Talents Training Fund in Shenzhen.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, local- ization, text reading, and beyond. arXiv preprint arXiv:2308. /2966, 2023.
Yuxuan Bian, Zhaoyang Zhang, Xuan Ju, Mingdeng Cao, Liangbin Xie, Ying Shan, and Qiang Xu. Videopainter: Any-length video inpainting and editing with woug-an--lay context control. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference, 2025.
Blackforestlabs. Flux. httpss//ssitaubb.cm/bbkkeokk fores labs/ flux 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rom- bach. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint 15Xiv2211..5112, 2023.
Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditctrl: Exploring attention control in multi-modal diffusion transformer for tuning-free multi-prompt longer video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Mas- actrl: Tuning- free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Duygu Ceylan, Chun-Hao P. Huang, and Niloy Mitra. Pix2video: Video editing using image diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023,
Xiuwen Chen, Li Fang, Long Ye, and Qin Zhang. Deep video harmonization by improving spatial-
temporal consistency. Machine Intelligence Research. 2024.
Yi Dong, Yuxi Wang, Zheng Fang, Wenqi Ouyang, Xianhui Lin, Zhiqi Shen, Peiran Ren, Xuansong Xie, and Qingming Huang. Movingcolor: Seamless fusion of fine-grained video color enhance- ment. In Proceedings of the ACM International Conference on Multimedia, 2024.
Patrick Esser, Johnathan Chiu, Parmida Atighehchian. Jonathan Granskog, and Anastasis Germani- dis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2023.
Xiao FU, Xian Liu, Xintao Wang, Sida Peng, Menghan Xia, Xiaoyu Shi, Ziyang Yuan, Pengfei Wan, Di ZHANG, and Dahua Lin. 3DTrajmaster: Mastering 3d trajectory for multi-entity motion in video generation. In Proceedings of the International Conference Learning Representations, 2025.
Jiaqi Guo, Lianli Gao, Junchen Zhu, JiaxinZhang, Siyang Li, and Jingkuan Song. Magievfx: Vi- sual effects synthesis in just minutes. In Proceedings of the ACM International Conference Multimedia, 2024a.
Zonghui Guo, Xinyu Han, Jie Zhang, Shiguang Shan, and Haiyong Zheng. Video harmonization with triplet spatio-temporal variation patterns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video difffusion models. In Proceedings of the Advances in Neural Information Processing Systems, 2022.
Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character anima- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
11

---

Published as a conference paper at ICLR 2026
Teng Hu, Zhentao Yu, Zhengguang Zhou, Sen Liang, Yuan Zhou, Qin Lin, and Qinglin Lu. Hun- yuancustom: multimodal-drive driven architecture for customized video generation. arXiv preprint 00Xiv:2500.04512. 2025.
Hao-Zhi Huang, Zhn Xu, Jun-Jiunn Lun-iiong Cai, Wei Liu, and Shi-Min Hu. Temporally coherent video harmonization using adversarial networks. IEEE Transactions on Image Processing, 2020.
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Champaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative mod- els. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Zeyinzi Jiang. Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in- one video creation and editing. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2025.
Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug- and-play image inpainting model with decomposed dual-branch diffusion. In Proceedings of the European Conference on Computer Vision, 2024.
Xuan Ju, Weicai Ye, Quande Liu, Qiulin Wang. Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, and Qiang Xu. Fulldit: Multi-task video generative foundation model with full attention. arXiv preprint arXiv:2503. 19907, 2025.
Zhanghan Ke, Chunyi Sun, Lei Zhu, Ke Xu, and Rynson W. H. Lau. Harmonizer: Learning to perform white-box image and video harmonization. In Proceedings of the European Conference on Computer Vision, 2022.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.
Lingen Li, Guangzhi Wang, Zhaoyang Zhang, Yaowei Li, Xiaoyu Li, Qi Dou, Jinwei Gu, Tianfan Xue, and Ying Shan. Tooncomposer: Streamlining cartoon production with generative post- keyframiig. arXiv preprint arXiv:2508. 10881, 2025a.
Linge Li. Zhaoyang Zhang, Yaowei Li, Jiale Xu, Wenbo Hu, Xiaoyu Li, Weihao Cheng, Jinwei Gu, Tianfan Xue, and Ying Shan. Nvcomposer: Boosting generative novel view synthesis with multiple sparse and unposed images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025b.
Weiqi Li, Shijie Zhao, Chong Mou, Xuhan Sheng, Zhenyu Zhang, Qian Wang, Junlin Li, Li Zhang, and Jian Zhang. Omnidrag: Enabling motion control for omnidirectional image-to-vide genera- tion. International Journal of Computer Vision, 2026.
Yaowei Li, Xintao Wang, Zhaoyang Zhang, Zhouxia Wang, Ziyang Yuan, Liangbin Xie, Ying Shan, and Yuexian Zou. Image conductor: Precision control for interactive video synthesis. In Proceed- ings of the AAAI Conference on Artificial Intelligence, 2025c.
Yuhang Li, Jincen Jiang, Xiaosong Yang, Youdong Ding, and Jian Jun Zhang. Harmony every- thing! masked autoencoders for video harmonization. In Proceedings of the ACM International Conference on Multimedia, 2024.
Risheng Liu, Zhiying Jiang, Shuzhou Yang, and Xin Fan. Twin adversarial contrastive learning for underwater image enhancement and beyond. IEEE Transactions on Image Processing, 2022.
Xinyu Liu, Ailing Zeng, Wei Xue, Harry Yang, Wenhan Luo, Qifeng Liu, and Yike Guo. Vfx creator: Animated visual effect generation with controllable diffusion transformer. arXiv preprint arXiv:2502..5599. 2025.
Xinyuan Lu, Shengyuan Huang, Li Niu, Wenyan Cong, and Liqing Zhang. Deep video harmoniza- tion with color mapping consistency. arXiv preprint arXiv:2205.0066 006687 2022.
12

---

Published as a conference paper at ICLR 2026
Zhiyuan Ma, Xinyue Liang, Rongyuan Wu, Xiangyu Zhu, Zhen Lei, and Lei Zhang. Progressive rendering distillation: Adapting stable diffusion for instant text-to-men tettto--eeshhteeeee generation without 3d data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Chong Mou, Mingdeng Cao, Xintao Wang. Zhaoyang Zhang, Ying Shan, and Jian Zhang. Revideo: Remake a video with motion and content control. In Proceedings of the Advances in Neural Information Processing Systems, 2024a.
Chong Mou, Xintao Wang. Liangbin Xie, Yanze Wu, Jian Zhang. Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024b.
Chenyang QI, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the International Conference on Machine Learning, 2021.
Nikhila Ravi, Valentin Gabeur, Suan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle. Chloe Rolland, Laura Gustafson, et al. SAM 2: Segment anything in images and videos. In Proceedings of the International Conference on Learning Representations, 2025.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint 10Xi::2401.11555 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference Computer Vision and Pallern Recognition, 2022.
Chaehun Shin, Heeseung Kim, Che Hyun Lee, Sang-gil Lee, and Sungroh Yoon. Edit-A-Video: Single video editing with object-aware consistency. In Proceedings of the Asian Conference on Machine Learning, 2024.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En- hanced transformer with rotary position embedding. Neurocomputing, 2024.
Jianchao Tan, Jose Echevarria, and Yotam Gingold. Palette-based color harmonization. IEEE Trans- actions Visualization and Computer Graphics, 2025.
Yuanpeng Tu, Hao Luo, Xi Chen, Sihui Ji, Xiang Bai, and Hengshuang Zhao. Videoanydoor: High- fidelity video object insertion with precise motion control. In Proceedings of the Special Interest Group Computer Graphics and Interactive Techniques Conference, 2025.
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. FVD: A new metric for video generation. Proceedings of the ICLR 2019 Workshop on DeepGenStruct, 2019.
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.203 203344 2025a.
Hanlin Wang. Hao Ouyang. Qiuyu Wang. Wen Wang, Ka Leong Cheng, Qifeng Chen, Yujun Shen, and Limin Wang. Levitor: 3d trajectory oriented image-to-video synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025b.
Qian Wang, Weiqi Li, Chong Mou, Xinhua Cheng, and Jian Zhang. 360dvd: Controllable panorama video generation with 360-degree video diffusion model. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Patlern Recognition, 2024a.
13

---

Published as a conference paper at ICLR 2026
Weihan Wang. Qingsong Lv. Wenmeng Yu, Wenyi Hong. Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Keqin Chen, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. Cogvim: Visual expert for pretrained language models. In Proceedings of the Advances in Neural Information Processing Systems, 2024b.
Wen Wang, Kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang. and Chunhua Shen. Zero- shot video editing using off off-the-shelf image diffusion models. arXiv preprint arXiv:2303. 17599, 17599, 2023.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004.
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionetrl: unified and flexible motion controller for video generation. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference, 2024c.
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Zeyu Xiao, Yurui Zhu, Xueyang Fu, and Zhiwei Xiong. Tsa2: Temporal segment adaptation and aggregation for video harmonization. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024.
Shuzhou Yang, Moxuan Ding, Yanmin Wu, Zihan Li, and Jian Zhang. Implicit neural representation for cooperative low-light image enbancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Shuzhou Yang, Chong Mou, Jiwen Yu, Yuhan Wang, Xiandong Meng, and Jian Zhang. Neural video fields editing. arXiv preprint arXiv:2312.08880 2024.
Shuzhou Yang, Xiaodong Cun, Xiaoyu Li, Yaowei Li, and Jian Zhang. 4dvd: Cascaded dense-view video diffusion model for high-quality 4d content generation. arXiv preprint arxiv:2508.04455 04467, 2025a.
Shuzhou Yang, Yu Wang, Haijie Li, Jiarui Meng, Yanmin Wu, Xiandong Meng, and Jian Zhang. Hy- brid fourier score distillation for efficient one image to 3d object generation. Visual Intelligence, 2025b.
Shuzhou Yang, Xuanyu Zhang, Yinhuai Wang, Jiwen Yu, Yuhan Wang, and Jian Zhang. Difflle: Diffusion-based domain calibration for weak supervised low-light image enhancement. Interna- tional Journal of Computer Vision, 2025c.
Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, Da Yin, Yuxuan. Yhauan,Zhang, Weihan Wang. Yean Cheng, Bin Xu, Xiaotao Gu, Yuxiao Dong, and Jie Tang. Cogvideox: Text-to-video diffusion models with an expert transformer. In Proceedings of the International Conference on Learning Repre- sentations, 2025d.
Danah Yatim, Rafail Fridman, Omer Bar-Tal, and Tali Dekel. Dynvfx: Augmenting real videos with dynamic content. In Proceedings of the SIGGRAPH Asia, 2025.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arxiv:2308.06721, 2023.
Mark Yu, Wenbo Hu, Jinbo Xing, and Ying Shan. Trajectorycrafter: Redirecting camera trajec- tory for monocular videos via diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision 2023.
14

---

Published as a conference paper at ICLR 2026
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference Computer Viston and Pattern Recognition, 2018.
Zhenghao Zhang, Junchao Liao, Menghao Li, ZuoZhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffusion transformer for video generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Lulu Gu, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.2175: 2025.
Xianpan Zhou Tiger200k: Manually curated high visual quality video dataset from uge platform. arXiv preprint arw.r:2204..551100 2025.
15

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
Related Work
Aillusion-basel Video Editing
Video Harmonization
3
Task Definition
4
4
Method
Input Conversion
Background Preservation Branch
Foreground Generation Mainstream
Extended Rotary Position Embedding
5
Experiments
8
Implementation Details
8
Comparison with Video Harmonization
Comparison with Controllable Generation
5.4
Ablation Study
6 Conclusion
10
A Stafement
17
B Dataset Construction
17
Data curation
18
8.2 Data filtering
18
B.3 Data type analysis
18
Generalization Ability
19
Implementation Details
19
User Study
20
Curve of Applying ERoPE
Ablation Architectures
22
Interactivity
22
More Visual Results
22
Discussion
23
Interactive Web App
24
16

---

Published as a conference paper at ICLR 2026
Label Curation
Source Video
Collected Data (409K)
Describe this video in detail, ensuring ... Focus transformations throughout the video, including
NULL
Human, Coffee,
The video features serene underwater scene where a grey nurse shark swims gracefully
[Prompt]
CogVLM
Identify the most prominent dynamic object in the video, such as Answer with set of nouns, separate commas. Return NULL if none.
Human, Coffee,
[Question]
Fish
Incomplete
Over-broken
X
QWWn
Complete
Grounded SAM2
Filter
Manual Screening
Foreground Video
Mask Video
VideoComp (61K)
(a) Data Curation
(b) Data Filtering
Figure 9: Dataset construction pipeline. We construct VideoComp dataset with two stages: data curation and data filtering. The former includes three steps: collection, labeling, and segmentation. Then, select high-quality subset based several rules.
A STATEMENT
Reproducibility statement. We have provided anonymous code in supplementary materials. Due to file size limitations, model weights not attached, but understand any details of our method in code itself. For reproducibility, we will open-source all contributions of this paper when it published, including code, model weights, and organized dataset. In addition, we provide complete description of data processing steps in Sect. The related code and the complete dataset will be open-sourced when this paper is published.
Ethics statement. This work does not involve human subjects, personal data, or any experiments that may raisc ethical coneerns. All datascts used curated from internal source vidcos with appropriate licenses. The proposed methodology is intended research purposes video editing and does not pose foresccable risks regarding fairness, privacy, or potential misuse. The authors have carefully reviewed adhered ICLR Code Ethics.
The Use of Large Language Models (LLMs). This paper did not involve the use of Large Language Models (LLMs) for research ideation, content generation, writing. All content presented in this submission created by the authors without significant contribution from LLMs. We confirm that we take full responsibility for accuracy and integrity the contents in this paper.
B DATASET CONSTRUCTION
Since there no existing dataset video compositing, we carefully build dataset called VideoComp this task. We present a scalable dataset construction pipeline that utilizes advanced models as tools for data construction (Wang et al. 2024b Bai et al. 2023 Ren et al. 2024). The proposed VideoComp dataset contains 61K groups of videos, each containing three video samples, i.e., the source video, the foreground video, and its corresponding mask video. The video is high-quality raw video and the other
extracted
17

---

Published as a conference paper at ICLR 2026
Category diversity
Domain coverage
Motion distribution
Amnd
carkk
melie
2436
JAS
canao
mnder
Lon motoo 2odd
Figure 10: Data type analysis. We analyze data characteristies including category diversity, domain coverage, and motion distribution, Each is represented pie chart.
from it. As shown in Fig. 9 our dataset construction process consists of two main steps, data curation and data filtering.
B.1 DATA CURATION
(I) We firstly collect about 240K cinematic HD videos that meet high aesthetic standards and large motion. Moreover, recently released Tige2200K (Zhou] 2025), which consists 169K videos, is also included our data source. For these total 409K videos, we propose an end-to-end workflow to process each of them as follows. (2) We distinguish the labels of prominent dynamic elements in the source video. We set up two questions call CogVLM (Wang al. 2024b) and QWen (Bai al. 2023) respectively, where the former a visual language model and latter is large language model. We input the video frames and then ask CLgVLM to get the detailed description of video. Based on this answer and our second question, QWen is used to identify prominent dynamic elements, or return NULL if there are none. (3) Based on the output label of QWen, employ Grounded SAM2 (Ren et al. 2024) to segment the elements in the video and save its foreground video and mask video. Note that the mask video shows the original motion trajectory. However, when saving the foreground video, we center the dynamic element in each frame, eliminating its global position and trajectory information. This approach allows us to control the trajectory of the resulting video totally based on the mask video, rather than relying on the position in the foreground video.
B.2 DATA FILTERING
To ensure a quality dataset construction, we filter out unsatisfactory cases according to several rules. As illustrated in right of Fig. [ our filtering principles are as follows: First, exclude cases where QWen NULL,, indicating that there is no significant object the video. Second, for videos containing multiple elements, we select only the element with the highest probability. Third, we exclude suboptimal cases where elements have incomplete or excessively fragmented structures. Finally, we manually filter out videos that are visually unappealing.
B.3 DATA TYPE ANALYSIS
To ease utilization. here showcase three pie chats to analyze our VideoComp from three perspectives re- spectively, including category diversity, domain coverage, and motion distribution. As shown in Fig. To
(1) Category diversity: we list categories of topics. Where half (49.4%) of the videos feature various animals, and human subjects account for almost quarter (23.0% 0%). Various vehicles account for 14.5%, including plane, car, and bus. 6.8% of videos record landscape and urban seene occupies 2.8%. The remaining 3.3% is about complex topics.
(2) Domain coverage: we analyze the types of scenes displayed in the video data. The majority of these nature outdoor pieces (64.65 (64.6%), which typically offer wider field of view and incorporate various real-world physical interactions and natural movements. This indirectly demonstrates the high quality VidcoComp. Indoor scenes account for 14.9% urban outdoor accounts for 7.0%, and sports venue taks about 5.0%. The remaining 4.2% is about underwater scenes, and the other 4.3% includes various other scenes.
(3) Motion distribution: also list the type of motion in the right end of Fig- Where extreme motion only occupies 9.4% and 1.9%. for fast and low motion, respectively. Over a quarter videos (25.7%) contain moderate subject motion, and almost half of videos (48.8%) actually have mixed motion, which include multiple subjects with different amplitudes of motion. These strongly support the validity of the content in the proposed VideoComp dataset. In additiion, our data also contains approximately 14.2% camera motion video, which makes the model practical in real application.
18

---

Published as a conference paper at ICLR 2026
source videos blank foreground mask videos
masked videos
results
Figure 11: Generalizability. After training for video compositing, our method seamlessly enables video inpainting and the removal of target objects from videos with blank foreground condition.
C GENERALIZATION ABILITY
In addition to the effective video compositing capability of the proposed GenCompositor, our method also demonstrates impressive generalizability in video inpainting and the removal of target objects from videos, which can be achieved by simply replacing the foreground condition with blank foreground video.
we employ blank foreground condition to remove target objects source videos through the trained GonConpositor. We first use SAM2 (Ravi et al. (0255) to obtain mask videos corresponding to the objects to be removed in source videos. Then, mentioned in Sect, 4.1 the same mask inflation operation is applied to get the inflated mask videos and masked videos. Given the blank foreground condition. GenCom- positor removes objects and paints the masked region well. This successful generalization study demonstrates the significance our proposed generative video compositing task, Le., this new video editing task inherently supports other vidco-related downstream tasks.
As shown Fig. I1
D IMPLEMENTATION DETAILS
We refer to CogVideox 12V-5B to design our model. Similarly other DIT models, our GenCompositor also consists main components, Transformer, VAE, and text encoder, where we inherit the pre-trained weights of VAE and text encoder of Cog VidexX in our model and do not train them. We only train Transformer here. Notice that as provide null text during training, although GenCompositor inherits text encoder, for inference, video compositing totally based the input videos and user-specified control, which is consistent with classical handmade process. The number of parameters each component is: VAE (215,583,907) text encoder (4.762,310,656) Transformer (5,872.247,936) totaling 10,850,142.499 Following previous work, we only consider parameter number of Transformer in latent diffusion model, and claim that GenCompositor is 6B model.
Our Diffusion Transformer contains branches, one background preservation branch containing 301,568,576 parameters, and one foreground generation mainstream with 5,570,679,366 parameters. Since original Cog CogVideoX-I2V-511 -VX-5B is designed for image-to-vided generation, we revised a novel module (DiT fusion block) to meet the characteristics of our task, video compositing. and train this new model from scratch. The foreground generation mainstream consists of 42 DiT fusion blocks.
As shown in Fig. in main paper, our new DiT model has three patchify modules, which aim patchify the features input vidcos encoded by the VAE encoder into tokens that can be processed by Transformer, where the features encoded by VAE encoder contains 16 feature channels. Notice that for the background preservation branch, its inputs the input channel number of its patchify is 32. For foreground generation mainstream, its inputs masked video, noise input, and foreground video condition. During training, the noise input the combination of random noise and masked video, similar to other diffusion models. For inference, the noise input is pure gaussian noise. Notice that noise input is concatenate with masked video in channel dimension. Hence the input channel number of blue patchify in Fig. 2 is 32, and the input channel number of green patchify for foreground video condition is 16.
mask video and masked video, which are concatenated on channel dimension. Hence,
19

---

Published as a conference paper at ICLR 2026
Video Harmonization
Trajectory-controllel Generation
Ours Hammnnizrr
TrdeeriipeCTaansfoome
Ours
Tora
Revideo
4.47%
13.95%
18.16%
14.47%
71.58%
77.37%
(a)
(b)
Figure 12: User study comparison. Our method receives the most user preference for both video harmonization(a) and trajectory-controllee generation(b).
loss
w/ EROPE_t
w/ EROPE_W
W/ ERoPE_h
w/o ERoPE
1.5
0.5
1k
2k
3k
4k
Figure 13: Loss curves of applying ERoPE. We apply ERoPE by extending along different optional dimensions, height, width, and timing. For comparison, we showcase the loss curve of ablation setting that applies the same RoPE on both masked video and foreground video, which marked in pink. One can see that the three extension directions have equal positive impact on performance.
E USER STUDY
We conduct user study for both video harmonization and trajectory-comtrolled video generation, compared with Harmonizer and VideoTTiple/TransSomees Tora and Revideo, respectively. For each task, provide 20 sets of visual comparisons and invite 19 professional volunteers select their most preferred results. shown in Fig. a2),), most users prefer the video harmonization capability of our method, while Harmonizer and TiderTiipeeTTransffomeet are equally favored. As to sageecooy--contrllle video generation. shown in Fig. [2), our method obviously outperforms other related algorithms, which is consistent with the visual and quantitative results provided in the main paper.
20

---

Published as a conference paper at ICLR 2026
raday talen
Patchifs
DMT Fosion Dlock
DxT Fusion Block
DaT Fusion Block
DET Fasion Block
VAVE Excader
VAE Decoder
F.RPP 244
x21
video
Paichify
Fie tolen
fg. video
Fg. Generation Mainstream
Zo
composited video
w/o Background
Preservation
Branch (BPB)
RoPE ch
Bg. Preservation Branch
Layer Neen
Chr www-aatemmmmm
Di1
Block
DIT
Block
Lisor
mask Video
Fg. token
VAL Encoder
DTT Block
DXT Block
D&T Block
masked video
Zo
composited video
fg. video
w/o fusion block
Figure 14: Architectures of ablation settings. We visualize architecture details of the two ablation settings, "w/o BPBranch" (upper) and "w/o fusion block" (lower). The former deletes the control branch, and the latter uses cross-attention to inject foreground condition instead of the proposed fusion block. Their control signal conversion the same as full model. So omit it and mainly showcase model designs.
foreground
inflated
manually
fusion block
fg. encoding
Figure 15: Ablation foreground CLIP encoding. Based on the ablation setting of "w/o fusion block", we further replace the VAE encoding the foreground dynamics with CLIP encoding (i.e. "wlo fusion block & w/ CLIP fg. encoding") to validate whether semantic feature is more suitable for cross-attention. But results demonstrate that our model cannot utilize such an abstract feature.
F LOss CURVE OF APPLYING EROPE
To study the impact of extended position embedding in different dimensions performance, visualize the training loss curves of four settings in Fig. 13 where we apply ERoPE by applying position embedding along three dimensions, f.e,, height, width, and timing, which are marked as brown curve (w/ ERoPE.h h),, blue curve (w/ ERoPE_w), and green curve (w/ ERoPE (),, respectively. For comparison, we also attach the loss curve of an ablation setting pink curve in Fig. 13 which is the training loss of using the same RoPE for both masked and foreground videos (w/o ERoPE). One
that the training loss curves of three settings that three settings are highly
ERoPE are obviously lower than w/o ERoPE. Meanwhile, the curves consistent with each other, i.c. the brown. blue, and green curves almost overlap. This comparison strongly prove the importance of our proposed ERoPE for utilizing unnal--n video conditions, and demonstrate the equality of these three extension directions. In the main paper, "w/ ERoPF_h" More importantly, believe this discovery inspires future work for utilizing layout-unaligned video conditions.
21

---

Published as a conference paper at ICLR 2026
foraground video
Foreground
background video
Background video
tnajectory
results
trajectory reselts
acttory
results
trajectory resllt
Figure 16: Generating with different user-provided trajectories. We apply two different trajec- tories to the same foreground-backgroon video pairs. We can that, given the same foreground and background videos with different trajectories. GenCompositor could generate different contents that strictly follow their corresponding trajectories.
G ABLATION ARCHITECTURES
As shown in Fig. we visualize the architecture details of two ablation settings, "wlo BPBranch" abd "wwo fusion block". mentioned the main paper. The former eliminates the background preservation branch, but still concatenates loreground condition with noisy tokens at the spatial-level and utilizes the proposed DiT fusion block to fusing them, combined with the EROPE. Whilst "w/o fusion block" remains the background branch, but replaces the DiT fusion block with normal DiT modules. To inject forcground control, it utilizes cross-attention as shown the lower Fig. 14
addition, considering that previous cross-attention based methods mainly employ semantic features as ex- ternal conditions and inject with cross-attention. For the ablation setting of "w/o fusion block" in the lower of Fig. 14 we also replace the VAE encoding for the foreground videos with CLIP encoding to validate whether semantic feature more suitable for coss-attention than low-level feature of VAE. The corresponding visual results are given in Fig. [5] where we attach the foreground element and inflated mask the first row. The second row shows a manually paste version as baseline reference, and results of "w/o fusion block" as ablation baseline. In the setting of "w/o fusion block & w/ CLIP fg. encoding", we use CLIP to encode foreground vidcos instead of VAE used in "wlo fusion block". One can that the main content of background is still preserved, but the foreground conditions cannot be injected, and even the abstract semantic information that could have been inherited by "w/o fusion block" cannot be preserved, directly generating noise. We believe this is because using such a high-level abstract information, such as semantics. to generate low-level and dense visual signals (i.e., video) is difficult. For example, using VAE features of foreground video can directly in- ject low-level messages of this dynamic condition, such its shape and texture, and naturally only affects part region in generated video. But using CLIP features is theoretically equal to input an abstract description text, which directly impacts entire vidco, increasing learning difficulty. Moreover, fusing the abstract features (Foreground video) extracted by CLIP with the low-level features (background video) extracted by VAE in a diffusion model is also challenging.
H INTERACTIVITY
In order validate the flexible interoperability of the proposed method, provide additional examples where the user specifies different trajectories and scale factors, respectively. In Fig. [6] given the foreground and background vidcos, we manually specify two different trajectories to GenCompositor for generation. The added elements exhibit different trajectories, which are highly consistent with the input trajectories. In Fig. 17 we provide three different scale factors in the same example. Obviously, the user-specified factors directly affect mask videos and control the size of elements in the final results.
I MORE VISUAL RESULTS
We provide more visual results in Fig. 18 to showease our superior video compositing capability. GenCom- positor enables seamless video elements injection according to the user-given interaction. We visualize the designated trajectories as right dots and lines in the first frame of background videos. One can see that Gen- Compositor precisely injects foreground elements into the user-given positions and harmonizes their color. On the other hand, the background environmental changes caused by the inserted objects (such as the shadows in
22

---

Published as a conference paper at ICLR 2026
background video
foreground element
results
inflated mask (scale factor
results
inflated mask (scale factor 2.0)
tssults
inflated (scale
Figure 17:
Generating with different user-provided scale factors. Our method produces mask videos that follow user-provided factors and control the size of added elements in final results.
Fg. video
video
reselts
video
be. video
roults
Figure 18: More visual results of CenCompositor. Our method enables seamless video elements injection and user interaction. On the hand, it precisely injects foreground elements into the user-given position and harmonizes their color. On the other hand, the background environmental changes caused by the inserted objects (such as the shadows in the red box) are also automatically predicted by our generative model, proving the superiority of generative video compositing.
the red box) also automatically predicted by generative model, proving the superiority of generative video compositing task.
DISCUSSION
As the first attempt in generative video compositing, this paper proposed practical video editing task, provided the first feasible solution, and curated the first usable training dataset. Intcrestingly, we found that the proposed new task, generative video compositing, naturally supports other related tasks such as video inpainting and video element removal. More importantly, for video compositing itself, the proposed GenCompositor allows users to specify arbitrary trajectories and object sizes changing with timing. It means that users even simulate 3D spatial effects, such hear-large-and--ar--mmm -malll... We provide some examples in supplementary video to demonstrate this function. In addition, some techniques this paper are novel and to be applied to other similar application scenarios. For example, the proposed ERoPE can fully utilize layout- unaligned video conditions, and experiments in this paper demonstrate that, for video signals, the optional three extension directions (e.g., width, height, and timing) are equal, said in Sect. F We believe this contribution also plays significant role in other video editing tasks.
the potential
23

---

Published as a conference paper at ICLR 2026
renolts vidko
Figure 19: Our performance in extreme background lightness. For the left, the freighter itself is not illuminated. The composited results shows a freighter illuminated from left right, with a warm, sunset-like glow, which is consistent with background content. For the right, the added aircraft itself has a high exposure intensity but background unexposed. In output, our model adaptively adjusts lightness of added element to make the entire result coordinated.
bg. video
The 1st fg. video
The 2nd results
The 1st composited results
The 2nd composited results
Our performance on multiple objects compositing. showcase the background video to be edited in the first row, and two element videos in the second row. As shown in the third row, on the left, we use red box to highlight the inserted airplane. On the right, we additionally add seagull; the airplane and the seagull are partially obscured.
Figure 20:
Future work. Although this paper enables vidco injection and realistic interaction, some challenging topics still exist in extreme conditions. For example, employ gamma correction, simple yet effective augmen- tation, to address most harmonious results by adaptively adjusting lighting and filter effects foreground elements to match challeng- ing background brightness. But it may not be robust enough for sophisticated extreme background lighting. Meanwhile, the added elements cannot undergo complex occlusion changes with environment. But the first attempt, GenCompositor actually has been able generate videos with impressive occlusion effects. As shown Fig. 20 we add an airplane case well. We believe more extreme background lighting can solved by replacing other luminance aug- mentation methods, and the more challenging occlusion can be achieved by adding depth-aware or 3D priors. Since ConConooiio already works for most scenarios, we Icave these issues for future work. We believe potential research direction in generative video compositing is enabling novel-view foreground prediction and new-oriented compositing, which may addressed by introducing 3D priors and we leave the future work.
cases well. As shown in Fig. T9 GenCompositor has been able generate
seagull to a single background video,
GenCompositor tackles this
K INTERACTIVE WEB APP
To case utilization of the proposed GenCompositor, we realize an interactive web demo in our code, which can be found in supplementary materials. Specifically, our interactive demo is shown in Fig 21 We firstly list the essential operating tips, and provide some available viden samples in the first box. Users can directly specify background and foreground videns from them, choose step 1, users segment foreground element from foreground video. In step users specify movement trajectory in the background video. Finally, in step 3, users can specify varying rescale parameters, which is defaulted as 0.4, to assign the scales of added element, and compositing final video through our model. Notice that also provide extended application in the box in the lower right corner, which inpaint video remove dynamic elements from the video. this function, code automatically selects the blank video as foreground, and removes the specified element from the video smoothly.
them and upload their own materials. In
need specify foreground video. Our
24

---

Published as a conference paper at ICLR 2026
GenCompositor Demo
Selec Fanggreed
select beckgreedd
Step Foreground Processing
Step Specify Trajectory in Background
Regls seiettaag
selctedpoiets
Angs spelttlagrapeeeyy
Clour selecteppoiii
AUL
malee tripetory
Step 3: Video Compositing
Coneria madk farrooo
Genoete vidke
Cencale campori led
Figure 21: Interactive interface of our web app. As an interactive video editing method, realize an interactive web demo to case the utilization of GenCompositor. Related code can be found in supplementary materials.
25


Reviews:
[{"review_id": "SNy99k7IGg", "reviewer": "Reviewer_sd3W", "summary": "This paper introduces a new video editing task called generative video compositing and proposes the first feasible solution, GenCompositor. Based on the Diffusion Transformer architecture, this method can automatically composite dynamic elements from a foreground video into a background video according to user-specified trajectories and dimensions.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work is the first to introduce the \"generative video compositing\" task, which aims to automatically composite foreground video elements into a background video using generative models while supporting user control over attributes such as trajectory and scale. \n2. A complete architecture is proposed, including a background preservation branch, DiT fusion blocks, and Extended Rotary Position Embedding (ERoPE), aiming to address the three main challenges: background consistency, foreground injection, and user control.", "weaknesses": "1. Appendix F shows that extending RoPE along height, width, or temporal dimensions yields nearly identical training loss curves, leading authors to conclude these three directions are equivalent. However, this dimension-agnostic behavior seems contradictory to the inherent spatial-temporal characteristics of videos. If ERoPE truly models spatial layout relationships, spatial extensions (height/width) should outperform temporal extension since they directly encode spatial proximity. Conversely, if it captures motion dynamics, temporal extension should be more effective given the causal and directional nature of video frames. The complete equivalence across all three dimensions suggests that ERoPE may simply function by expanding the position encoding to avoid feature conflicts, rather than genuinely modeling spatial-temporal interactions between foreground and background. Could the authors clarify this dimension-independence phenomenon and explain whether simpler alternatives (such as learnable position offsets for different video sources) might achieve comparable results?\n\n2. The practical usage requires SAM2 for foreground segmentation and optical flow algorithms for trajectory tracking, and these preprocessing steps may introduce errors and increase system complexity, yet the paper does not discuss how these errors propagate and accumulate to affect the final results.\n\n3. The paper only compares its method with two video harmonization approaches and two trajectory-controlled generation methods, but fails to include comparisons with recent state-of-the-art methods in related tasks such as video object insertion, which share similarities with generative video compositing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G4DOuAXp6X", "reviewer": "Reviewer_ThTu", "summary": "GenCompositor introduces generative video compositing, an automated task to inject dynamic foreground elements into background videos via user-specified trajectories/scales. Its core is a Diffusion Transformer pipeline with three key components: a lightweight background preservation branch ensuring input-output consistency, DiT fusion blocks with full self-attention for foreground-background token fusion, and Extended Rotary Position Embedding (ERoPE) addressing layout misalignment. Trained on the 61K-video VideoComp dataset, it outperforms video harmonization (e.g., Harmonizer) and trajectory-controlled generation (e.g., Tora) methods in PSNR, SSIM, and motion smoothness. Ablation studies validate the necessity of each component. GenCompositor enables seamless, interactive video editing with realistic foreground-background interactions.", "questions": "1. Sec.4.3 proposes a new token-wise concatenation method. However, what differentiates it from in-context learning approaches like VACE [Jiang2025 et al.]? Perhaps I missed some details, but I did not find a detailed explanation in the ablation study or supplementary materials.\n\n2. After reviewing the supplementary material on your data construction, I have a question: how do you ensure that the foreground data strictly adheres to the user-provided trajectories, especially when considering variations in angle and direction? This raises concerns about the strictness of the task’s generalizability.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The work addresses an underexplored yet challenging setting: interactive, adaptive injection of foreground identity and motion into target videos, enabling fine-grained control over size, trajectories, and other dynamic attributes.\n\n2. The showcased results demonstrate strong perceptual quality and temporal coherence, suggesting the method’s practical utility.\n\n3. The release of implementation details and code substantially improves reproducibility and facilitates future research.\n\n4. The construction of the 61K-pair VideoComp dataset represents a significant data-engineering contribution and a useful resource for benchmarking video composition.", "weaknesses": "1. In my opinion, the innovation of the module is more of an engineering improvement rather than a fundamental breakthrough. Specifically:  \n- For Sec.4.1, the proposed Input Conversion primarily involves operations like scaling and expanding the mask foreground object, which are quite common in existing methods.  \n- For Sec.4.2, the use of a black area to indicate the modified background and a binarized mask video to represent the masked area is also a common practice in prior work.  \n\n2. Although the RoPE alignment appears to be a promising idea for alignment, the authors did not provide sufficient elaboration on this design.  \n\n3. Additionally, it is suggested to revise Fig.3(A) to align with the paper's description by using a mask video instead of a foreground video, better demonstrating the unique application of ERoPE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vSIcUHzLZo", "reviewer": "Reviewer_irsQ", "summary": "The paper introduces GenCompositor, a diffusion-transformer-based framework for generative video compositing, enabling insertion, removal, and harmonization of dynamic foreground objects within videos. The method takes as input a masked background video, a mask video, and a foreground video, along with user-defined trajectory and scale controls. The model architecture follows a  MMDiT, similar to those used in FLUX, SD3, or HunyuanVideo, with full self-attention, and includes:\n- A background preservation branch to maintain scene consistency,\n- A DiT fusion block, which concatenates background and foreground tokens for fusion instead of using cross-attention, and\n- An Extended Rotary Position Embedding (ERoPE) to mitigate layout misalignment artifacts.\n\nFurthermore, a new dataset, VideoComp, containing 61K video triplets, is curated for training. The qualitative results are visually strong and demonstrate convincing object insertion and harmonization, though the quantitative metrics show only modest improvements over existing methods.", "questions": "- The results without the fusion block are much worse. Could this be due to the limited representational power of the VAE encoder? I would like to see additional results where a stronger feature extractor, such as DINO or CLIP, is used for the cross-attention variant, as this could provide more meaningful semantic conditioning.\n- The model depends strongly on mask and trajectory inputs. How flexible is it when these are imprecise or unavailable? Can it generalize to more unconstrained compositional setups?\n- How well does it handle multiple objects or occlusion? \n- The paper mentions luminance augmentation for harmonization. Is it sufficient for handling complex illumination changes, or does it fail in extreme lighting conditions?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- **Strong engineering effort:** The system is well-implemented and technically solid. The architecture is clean and the pipeline is carefully designed.\n- **Good visual quality:** Qualitative and video results are impressive, showing smooth motion and coherent integration of foreground and background.\n- **Clear writing and presentation:** The paper is well-organized, easy to follow, and supported by clear figures and a detailed video presentation.\n- **Practical relevance:** The task aligns well with real-world video editing workflows and could be useful for production or creative pipelines.", "weaknesses": "1. **Limited conceptual novelty**: \n    - The core components are incremental modifications of existing DiT architectures.\n    - The background preservation branch resembles ControlNet-style conditioning, which is also stated in the paper.\n    - ERoPE is a simple positional embedding shift; while useful, it is not conceptually innovative.\nOverall, the work feels like a strong system-level integration rather than a conceptual contribution.\n\n2. **Inadequate quantitative evaluation**:\n    - The paper reports only frame-level metrics (PSNR, SSIM, LPIPS, CLIP). These are not sufficient for video generation; video-level metrics such as FVD or KVD are missing.\n    - Quantitative improvements are relatively small, making it difficult to assess significance.\n\n3. **Weak baseline comparisons**:\n    - The compared baselines (Harmonizer, VideoTripletTransformer) are both video harmonization methods, not true generative compositing or conditional video generation systems.\n    - DynVFX can be used for comparisons. Although it cannot add directly the reference objects, it can add objects through text-prompts, which will also strengthen the paper's results.\n\n4. **Dataset concerns**:\n    - The construction of the new VideoComp dataset is clearly described, combining 409K source videos into 61K compositing triplets via a semi-automatic pipeline (SAM2 + CogVLM + Qwen). While this is a solid contribution, the paper provides limited analysis of dataset characteristics such as category diversity, motion distribution, or domain coverage, which limits understanding of how well the dataset covers diverse motion and scene types. Including a brief quantitative summary would strengthen this part.\n\n5. **Conceptual framing and scope**:\n    - The paper frames generative video compositing as a new task, but it heavily overlaps with existing controllable video generation and video inpainting setups.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NYji5Lsf8V", "reviewer": "Reviewer_7r6X", "summary": "This paper proposes GenCompositor, a generative video compositing method that can adaptively inject foreground video with user control into the background video. The proposed framework was developed based on the diffusion transformer pipeline, allowing users to control the size, trajectory of the foreground objects, and seamlessly preserve them in the background video. To validate the effectiveness of this proposed framework, the paper also curated a dataset called VideoComp. The experiments have been validated to compare against both trajectory-controlled video generation and non-trajectory-controlled video generation methods.", "questions": "1. The interaction between the added foreground object and the background is unclear. How this is related to the technical component in the proposed pipeline needs to be justified.\n\n2. The input conversion procedure is unclear and needs to be further explained. \n\n3. The relationship between layout-unaligned conditions and its proposed pipeline is not clearly justified. \n\n4. The role of the inflated mask and its generation process needs to be better explained.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, this paper proposes a new task, i.e., generative video compositing, which has not been well addressed before. This differentiates from existing works on video compositing as it utilizes the generative models to complete the task. From this perspective, the proposed task is similar to the existing one, but tackles the problem from a different technical perspective. \n\nIn addition to the task, the paper shows some strong points in curating the dataset required to evaluate on this task. If released, this should help further advance the research in this field. The paper offers a good motivation for why this problem is important and presents the components in the pipeline relatively well. Below, I will show the weak points that need particular attention.", "weaknesses": "The experimental visualization was not quite clear. For instance, by checking on Figure 1, what is the foreground object on the left top example of the figure? The depicted trajectory is also unclear in this case. It will be nice to have high-resolution and good-quality results. Similarly in Figure 7, the trajectory is not very clearly depicted. \n\nThe paper mentions the interaction between the added foreground object and the background, citing the explosion effect in Figure 1(a) as an example. This looks very interesting. However, I don't believe this paper presents sufficient technical details to explain why that happens. Usually, we will expect the added element to have a static presence. But this claim shows that the newly added element can adapt to the background content. This part is not supported by the technical details, at least not clearly presented. This is related to mask inflation, but not entirely. \n\nRegarding the input conversion, the user trajectory has been used to generate the mask video and masked video. The trajectory itself was not directly encoded. It will be nice to directly state what the conditions of \"c\" are in Eqn. (2). \n\nBased on the workflow of GenCompositor in Figure 2, the masked video and its corresponding mask video has been concatenated as the input. This indicates that the GenCompositor deals with the layout-aligned conditions. In Section 4.4, where ERoPE was discussed, the problem has been shifted to the layout-unaligned conditions. This creates some gaps in understanding, as the workflow in Figure 2 does not incorporate the layout-unaligned conditions. Will the layout-unaligned conditions change the workflow of the proposed GenCompositor? This creates some confusion here. \n\nThe role of the inflated mask could be better clarified. Initially, I was thinking that the mask regions were expanded to increase the size of the foreground object, if there is an imperfect foreground mask. A Gaussian filter was used to inflate the mask video. When looking at Figure 8, I have also noticed that the inflated mask was naturally generated as the foreground object changes. \n\nIn summary, the paper tackles an interesting task, and the presented pipeline is also feasible to address the proposed task. There are some technical contributions regarding the proposed pipeline, as well as the ERoPE part. The proposed dataset is also a good contribution. Therefore, the paper looks positive. However, the paper also shows the concerns listed above, which need to be addressed well. The corresponding answers to address these concerns would be critical for the final decision.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)