UniTransfer: Video Concept Transfer via Progressive Spatio-Temporal Decomposition
Authors: guojunlei
Rong Zhang
Tianhang Liu
Hong Li
Zhiyuan Ma
Chi Wang
Weiwei Xu
Venue: NeurIPS 2025 poster ()
Keywords: video concept transfer
video editing
controllable video generation
Primary area: general_machine_learning
TL;DR: Video Concept Transfer via Progressive Spatio-Temporal Decomposition

Abstract: Recent advancements in video generation models have enabled the creation of diverse and realistic videos, with promising applications in advertising and film production. However, as one of the essential tasks of video generation models, video concept transfer remains significantly challenging.
Existing methods generally model video as an entirety, leading to limited flexibility and precision when solely editing specific regions or concepts. To mitigate this dilemma, we propose a novel architecture UniTransfer, which introduces both spatial and diffusion timestep decomposition in a progressive paradigm, achieving precise and controllable video concept transfer. Specifically, in terms of spatial decomposition, we decouple videos into three key components: the foreground subject, the background, and the motion flow. Building upon this decomposed formulation, we further introduce a dual-to-single-stream DiT-based architecture for supporting fine-grained control over different components in the videos. We also introduce a self-supervised pretraining strategy based on random masking to enhance the decomposed representation learning from large-scale unlabeled video data. Inspired by the Chain-of-Thought reasoning paradigm, we further revisit the denoising diffusion process and propose a Chain-of-Prompt (CoP) mechanism to achieve the timestep decomposition. We decompose the denoising process into three stages of different granularity and leverage large language models (LLMs) for stage-specific instructions to guide the generation progressively. We also curate an animal-centric video dataset called OpenAnimal to facilitate the advancement and benchmarking of research in video concept transfer. 
Extensive experiments demonstrate that our method achieves high-quality and controllable video concept transfer across diverse reference images and scenes, surpassing existing baselines in both visual fidelity and editability.

UniTransfer: Video Concept Transfer via Progressive Spatial and Timestep Decomposition
Guojun Lei'; Rong Zhang"; Tianhang Liu', Hong Li*, Zhiyuan Ma' Chi Wang', Weiwei Xu
State Key Lab of CAD&CG, Zhejiang University, ZZejjaag Gongshang University, Huazhong University of Science and Technology, Beihang University,
Ningbo Global Innovation Center, Zhejiang University guojunleidzju. ju. edu, en, zhangrongōzjgau edu cn, link05020buaa edu, en, sianhangozju.edut cn, mzythChust.. edu. en, kangchi19960zzu..oo en, xww0cad. wasicadd.j...... zju.edu.co
[Web Page]
Tharpe Video
Tegpe Vitko
Rof Charchs
Ganeet
Characee Transfer
Garment Transfer
Target
Ref Barlaground
Tarpet Image
Background Transfer
Mobion Transfer
Figure 1: Results of our Transefrer. These qualitative results exhibit the superior performance of our approach in transferring various reference components, including characters, garments, backgrounds, and motions, to synthesize the new target videos.
Abstract
Recent advancements in video generation models have enabled the creation of diverse and realistic videos, with promising applications in advertising and film production. However, one of the essential tasks of video generation models, video concept transfer remains significantly challenging. Existing methods generally model video as an entirety, leading limited flexibility and precision when solely editing specific regions concepts. To mitigate this dilemma, we propose a novel architecture UniTransfer, which introduces both spatial and diffusion timestep decomposition in progressive paradigm, achieving precise and controllable video concept transfer. Specifically, in terms of spatial decomposition, we decouple videos into three key components: the
'Equal contributions.
Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

foreground subject, the background, and the motion flow. Building upon this decomposed formulation, we further introduce a tuuul-t-singie-strees DīT-based architecture for supporting fine-grained control over different components in the videos. We also introduce a self-supervised pretraining strategy based on random masking to enhance the decomposed representation learning from large-scale unlabeled video data. Inspired by the Chain-of-Thougee reasoning paradigm, we Shaain-ff-Prmpp
further revisit the denoising diffusion process and propose (CoP) mechanism to achieve the timestep decomposition. We decompose the denoising process into three stages of different granularity and leverage large language models (LLMs) for stage-specific instructions to guide the generation progressively. We also curate an mimall-centric video dataset called OpenAnimal to facilitate the advancement and benchmarking of research in video concept transfer. Extensive experiments demonstrate that method achieves high-quality and controllable video concept transfer across diverse reference images and scenes, surpassing existing baselines in both visual fidelity and editability. Web Page: shaonian. aerpiiiinnnititttt github. github.io//mmne www.:i/iranssneee----rr.
1 Introduction
Recent years, the rapid advancement of generative AI technologies [3 36 the community of video editing, opening up new potentials for diverse, fine-grained and user- controllable content manipulation tasks. Video Concept Transfer (VCT) is one of the most important downstream tasks in video editing, aiming to substitute various user-specified target concepts in video, such as objects, characters, backgrounds, or subject motions, to enable personalized content manipulation. Its applications span across diverse areas such as film production, game development, virtual reality, etc. drawing increasing attention from both academia and industry.
has greatly reformed
Despite its potential, achieving high-quality video concept transfer is still challenging as it requires seamless integration of different components, preservation of the identity of the target object, and visual fidelity of the generated videos. Some recent approaches rely on text-based guidance to control the transfer [10 41). However, the inherent ambiguity of natural language descriptions often results in imprecise manipulation of object attributes and behaviors, restricting their application scenarios. In contrast, image-guided methods offer more accurate way to encode the appearance and identity of target objects. For example, methods like AnimateAnyone221 [5] and MIMO [29] have demonstrated the ability to replace human subjects in videos with specific reference images. However, these existing approaches mainly focus on human transfer and struggle to generalize to broader editing tasks (25) involving arbitrary objects, backgrounds, garments or motion patterns. VideoSwap II] and V2V (20] rely on personalized modeling or image editing techniques to address this dilemma. But the ability of their foundation models limits their scalability and flexibility in complex videos.
This paper focuses on image-based video concept transfer, including animals, characters, backgrounds. and motions. This task inherently involves manipulations and integrations of different components within a video, which often exhibit substantial variation in terms of visual appearance, motion pattern, or semantic attributes. However, most existing approaches [23
20 28 generally model the video as a unified whole without considering the heterogeneous nature of its constituents, which may introduce undesired artifacts. MIMO (29) introduced spatial decomposed modeling to VCT by decomposing a video into three predefined components: human, scene, and occlusion. It helps improve the quality of video character replacement. However, MIMO only decomposes videos in the spatial dimension, which is not sufficient for high-quality video generation, as it takes all the timesteps in the denoising process equally. Motivated by ProSpect (53), diffusion models actually generate images in the progressive order of "layout - content - texture" ", and this work also exhibits that different stages in the diffusion models require guidance at different granularities.
In this work, we propose UniTransfer, Diffussin Transformer (DiT) based video concept transfer framework via progressive decomposition of both the spatial dimension and the denoising process. In the spatial dimension, we decompose videos into three core components: foreground, background, and motion dynamics, enabling our model to adapt to general concept transfer flexibly. To achieve this, we allow the model to learn the decomposition from coarse foreground masks to detailed ones. Specifically, we first introduce random masking-basse based superviuer pretraining strategy to
2

---

strengthen the decomposed representation learning, which enables the model to capture disentangled features without requiring fine-graine une-grained annotations. Then we design dual-to- tuull-to-ingll-stteemn Dit architecttreee to realize further decomposition with delicate semantic annotations. In this stage, individual branches are responsible for encoding different video components, and their features are later integrated into a unified representation through a single-stream network. This design enhances the model's capacity to manipulate different objects and maintain the temporal consistency.
In the denoising process, we decompose the timestep into coarse-grained, mid-grained, and fine- grained stages instead of modeling it equally. Inspired by the Chain-of Saain-of-Thougee Thought (CoT) strategy. we develop Chain-of- Maaii--o--Pronee Prompt (CoP) mechanism and leverage Large Language Models (LLMs) to produce hierarchical prompts at different granularities and utilize them to guide the generation, enabling progressive refinement from noise to detailed textures. The main contributions are summarized as follows:
We propose a DiT-based image- video concept transfer framework UniTransfer, which incorporates progressive spatial and timestep decomposition.
We introduce a self-supervised pretraining strategy based on randomized masking to enhance the disentangled representation learning and design a dual-to--inglee-ttean architecture to achieve spatial decomposition.
We further introduce an L.MMs-uuded chain-of chain-of-prompt prompt mechanism to achieve the timestep decomposition. This progressive prompting strategy guides the generation process with stage-specific instructions, improving the VCT generation quality.
We collect an animal-centric video dataset called OpenAnimal to facilitate the training and benchmarking of research in video concept transfer. Extensive experiments demonstrate that our method outperforms state-of-the-ar methods in various video concept transfer scenarios.
2
Related Work
Text-driven Video Editing, Video editing has witnessed remarkable advances in conditional content
synthesis and manipulation through diffusion-based architectures [27]. Recently, video editing
typically relies on textual prompts to control object attributes or behaviors [10 I. Video-P2P [23
32 achieves preservation of motion dynamics through attention modulation. RF-Edit 41 preserves structural integrity and temporal consistency by rectified flow ODE solving with reduced error. MotionDirector (55) requires separate training for each motion type. When adapting to new scenarios, it only supports text-guided scene transitions, which significantly limits its flexibility. VMC [18] also needs seperate training for different scenes. However, due to the inherent ambiguity and underspecification of natural language, such methods often struggle to achieve fine-grained and accurate control over video content.
Image-guided Video Editing, To overcome these limitations, researchers introduce additional reference images to guide the editing process of the video subject. MIMO [29] and MovieCharac- ter [34] decompose the video into elements such as foregrounds and preprocessed backgrounds, and perform character transfer through video composition techniques. Animate Anyone2 [15 proposes framework to animate characters while considering environmental affordances. Despite their effectiveness, they are designed for character video editing, which limits the application scenarios.
In contrast, VideoSwap 11 pioneers a more general approach to image-guided concept transfer through semantic correspondence learning, enabling more versatile and accurate video edits beyond character animation. Yet critically, its reliance on multi-image concept anchoring introduces semantic abstraction, achieving category-level consistency (c.g.. kitten, airplane) but failing to preserve instance- specific attributes. Any V2V (20] leverages arbitrary image editing tools [3 (2)42] 7 to modify the first frame of video and then propagates the modifications subsequent frames, enabling instance-based object-driven editing and identity manipulation. However, the two-stage pipeline heavily depends on the results of fff-the-shell image editing techniques, and the temporal consistency can not be guaranteed. To address these limitations, we propose novel framework that does not rely on external image editing tools. Instead, our method decomposes the video into three disentangled components: foreground, background, and motion. Integrating with a carefully-tailore network architecture in conjunction with large language model (L.L.M)-base chain of-tough reasoning mechanism, our method can achieve precise and flexible video concept transfer.
3

---

Spatial Decomposition
Coupled Training
Random Decompostiinn
Object Object-Spectic Decompositin
Random Mask
Uni Transfer
DIT
Training Process
Timestep Decomposition
Coarse Grain
Mid-Grain
Fine-Grain
dicse-up up shot of worznn standing against light colord, plain backgrroudd She pink shir haintyle
standling against light- colored plain background. Site wears t-shitt ind has short bob haiistyle.
A close up shot of woman standing against plain bockground
her chest. is looking at camern with a noutni expression.
Denoising Process
Figure 2: Our progressive spatial and timestep decomposition modeling.
Chain-of-Thought Prompting 19 aims to substantially improve the reasoning capabilities of large language models (LLMs) 66. provides framework for complex multi-step inference
through explicit intermediate reasoning steps. The CoT concept evolves into CoX [4 where
denotes swappable nodes (c.g. intermediates 19
56 40], augmentation [2
49 52).
feedback 46 24
2, g and models [4
5 45
for task-specific adaptation. We further propose
Chain-of-Promp! (CoP), which shifts the CoX paradigm to the iterative denoising steps of diffusion
models. This systematic approach decomposes the video generation into sequential coarse- to--to-fii- to-fine steps, where hierarchical prompt guidance progressively refines temporal features through successive diffusion iterations.
Method
In this section, present UniTransfer, DiT-based image-guided video concept transfer framework with progressive decomposed video modeling. Our goal is to generate high-quality videos with user-specified concepts in the referenced images, including objects, characters, animals, backgrounds, or motion dynamics. To achieve this, we decompose the video both in the spatial and the timestep dimension. The overview of the proposed framework is illustrated in Figure
3.1 Spatial Decomposition
Existing video generation methods typically treat the video as a holistic entity and attempt to model under the guidance of text prompts or reference images. Although such strategies are effective for general video synthesis, they fall short in the context of video concept transfer, which requires compositional control over different parts of the video, including foreground objects, background scenes, and motion dynamics. Encoding the entire video into single latent space makes it difficult independently manipulate these components during generation. a result, existing image-based based video object transfer methods are often limited to transferring only the main subject, instead of manipulating different components, including background appearance or motion dynamics.
To achieve more flexible and controllable video concept transfer, we propose to spatially disentangle the video generation process. Specifically, video can be decomposed into three distinct components: the foreground M, the background B, and the corresponding motion flow F. In general, the foreground component may consist of different objects (e.g., characters, animals, or objects). Under this setting, we redefine the video denoising process as follows:
(()) Ex M4, [|[e Eo (It, T,U,
(I)

---

DXT Bork XN
Fanggradd
VAE
NFPP
Taicall-
Encoder
Motion
Troor Repple
Froceler
Zass with trank rised
Text
Encoder
NPPV
1P3
N-90N
moore po0d
Decolrr
WE
Froodlrr
Bedegrond Irage
Groerttdd Vid0o
DIT Block KN
Spdtal Reedamm
DBT Block xN
Nele
Figure 3: The architecture of our Urinransfer.
where is the condition of text prompts, 4 (M, B, This decomposition enables treat each component independently and recombine them flexibly in the generation pipeline, laying the foundation for video concept transfer.
Building upon this decomposed formulation, we propose a progressive learning scheme that enables the model to effectively capture and utilize the decomposed components for video generation. To achieve this, we design two modules: (1) a random masking-bard baand self-supervised learning mechanism to model the coarse relationships between the foregrounds and the backgrounds without delicate semantic annotations. (2) a carefully designed sua--to-single stream architecture, UniTransfer, tailored to learn detailed interactions of three different components. In the following, will introduce them respectively.
3.1.1 The Architecture of UniTransfer
In this stage, we propose a DiT-based architecture named UniTransfer to decompose the video into three components: the foreground appearance, the background scene, and the motion dynamics to enable flexible and fine- ena-ggained control over VCT. Unlike previous approaches that treat the video as monolithic entity, UniTransfer is designed disentangle and recompose these components in controllable iiffusion-basee generation process.
In the training process, given the input video V. we first randomly sample a single frame to serve as reference image I. A pretrained semantic segmentation model is then utilized to predict a foreground mask M E {0, 13 H1 W which partitions the reference frame into a foreground region F = I @ M and background region B = I (1 B Boz(M)) where @ represents element-wise multiplication and Boo (M) means all the pixels the bounding box are setting to With the unaligned masks, we force the model to learn shape-agnostic interactions between the foreground and the background. Meanwhile, motion dynamics are extracted from the entire video using pretrained optical flow model RAFT (39] which captures the temporal dynamics independent of appearance content. UniTransfe takes F, B, along with the video description 7 as guidance and iteratively denoises from a randomly sampled Gaussian noise map to generate a new video V. Figure illustrates the framework of the network, which is dual-to-single-streeed paradigm consisting of three collaborative branches: foreground branch, a background branch, and fusion branch.
The foreground and background branches are built based the Cog VidVi architecture [48] Each of them is composed of an image VAE encoder to project and B into latent codes 2, and 2b- text encoder provide shared high-level text embedding 2,, and stack of N DiT blocks for temporal feature modeling. In the background branch, a random Gaussian noise vector 2, of the timestep / is concatenated with 2n and Zr, and the combined representation is passed through the DIT blocks hig for i = 1,..., N. In contrast, the foreground branch is treated as a conditional stream with no noise input. It adopts symmetric structure and produces intermediate feature maps for each DiT block hy. for i I,...,N, N. Afterwards, we design feature injection module that introduces foreground features into the background stream inspired by ControlNet [31]. Specifically, the foreground feature map is processed by zero-ine iro-iiniialized convolutional projection layer and then element-wist added to the corresponding layer in the background branch at each DiT block. allows the model inject spatially aligned foreground appearance into the generative path.
Moreover, the enhanced background features are combined with the motion dynamics the fusion branch, which consists of motion encoder to project the optical flow to a latent code 2o, and N
5

---

DiT blocks for further feature fusion. The flow encoder adopts four symmetrically arranged stages, aligning with the structure 3D VAE encoder [48). Inspired by [or,[54), we also design an adaptive norm layer before adding it to the dit block. To enable the model to handle the misalignments between the motion and the input images and improve the generalization ability, we inject random noise into the input optical flow. is fused with the background stream through element wise addition in each fusion DiT block h', for i = 1,... N.. The model outputs the predicted noise at given denoising timestep & as follows:
Eg = (1z @ 24) + ho(2 @ 26 @ 2,) + zol
(2)
where ZCone represents the zero-initialized convolution. @ and @ represent the concatenation operation and element-wise addition, respectively.
3.1.2 Self-Supervised Pre-Training via Random Masking
Learning robust video representations for concept-aware generation typically requires large-scale datasets with annotated semantic masks or motions. However, existing video datasets rarely provide sufficient annotations. Relying small-scale annotated datasets is not enough to directly learn spatially decoupled representations for video concept transfer. To address this limitation, we follow LMD [26] to introduce random masking strategy before training with fine-grained annotations. It leverages large-scale unlabeled video datasets for learning initial disentangled representations. In this stage, we only learn the foreground and background decomposition for coarse initialization. In this stage, the binary foreground mask M E {0, 1) HXW is generated through random masking to arbitrarily partition the reference image into two regions F and B. The mask is initialized as an all zel-zero matrix and is iteratively updated by randomly drawing rectangles of random size and position on it. The pixels inside each rectangle are set to 1, and the process continues until the foreground coverage exceeds 50% of the image area. This masking strategy ensures spatial diversity and balance between foreground and background regions. Then the reference frame is partitioned I 3 M and a background region Dilate represents the morphological dilation operation to enable the model to learn the boundary interactions. Then F and B are fed into the foreground branch and background branch, which inject the features into the denoising network independently. This allows the model to learn how to reconstruct coherent video content from partial and spatially isolated cues, without requiring ground- truth foreground-backgrouudd labels. Through extensive self-supervised training large-scale data, our model acquires strong prior knowledge, enabling efficient adaptation to downstream tasks via precise supervised fine-tuning.
into a foreground region
I (1 Dilate (M)) where
3.2 Timestep Decompostion
Revisiting the preliminary of diffusion models reveals another limitation in current video generation pipelines: the same conditioning prompt is applied across all timesteps during the denoising process. is difficult to generate grained texture corresponding to the grained prompts at the initial stage. Inspired from ProSpect [53), the role of each timestep varies significantly in the generation process. In the early stages of denoising, the model focuses primarily on recovering the global structure and semantic layout of the video. In contrast, the later stages are responsible for learning fine-grained details such as textures, colors, and subtle appearance attributes. Applying static, single-level prompt across all timesteps ignores this progression in modeling focus. For instance, using overly complex or fine-gu grainer textual prompts during carly timesteps may misguide the model, lcading to the omission of some semantic attributes or misalignments between the generated content and the input prompt.
To address this, we introduce timestep decomposition mechanism named Chain-of-Pronn Prompt (CoP) inspired by the Sfain-of-Thousee reasoning paradigm in large language models. Specifically, we decompose the denoising timesteps into three granularity levels levels-coarse, medium, and fine-and leverage large language model (LLM) to automatically generate three levels of prompts that progressively reflect the abstraction and detail required at different denoising stages. These prompts are then injected into the diffusion model stage-aware manner, ensuring that early timesteps receive high-level, structural guidance, while later steps benefit from detailed, appearance-level

---

Ref Character
Target Video
AnVV2V
Video-P2P
Animate Anyone
Champ
Uni Animate
Magic Animate
Ours
ROTY
OTY
TT
ROTY
Figure 4: Comparison of video character transfer. Note that our backgrounds are better preserved. control. Then the denoising loss can defined as follows:
c Eo (Zt: Teru, U1 13
t E te. T - 1]
( =
c Eo (2t, Tiidd U t) [2:
te tt,)
Eo (Ze, Tfiee U, ()
LE (0,y))
where Terss Tmids Tine are coarse-grained, mid miidggrineed and fine-grained text prompts, respectively, and y the end points of the corresponding stages. T is the total timestep of the diffusion process. In our paper, as the dataset provides detailed video descriptions, we utilize them as the fine ane-grained prompts. Then employ LLM Qwen-QWQ-322 QWQ-32B (38 summarize Tfine to Tersy Tmiid This decomposition strategy helps to align the complexity of text guidance with the focus the generation phase, improving both semantic consistency and visual fidelity in the final video output.
Experiments
To better demonstrate the strengths of our approach, conduct experiments validate foreground transfer, background transfer, and optical flow transfer across videos. The implementation details can be referred the appendix.
4.1 Animal-centrie Dataset
To promote the broader applicability of video editing models, we introduced a new dataset OpenAnimal, which focused on single-animal video sequences across a wide range of species and diverse motion patterns. While following a similar structure to human-centric datasets (e.g., Tok, UBC), OpenAnimal is specifically tailored for animal subjects with 10000 video clips.
4.2 Comparison and Analysis
As current state-of-the-art methods are primarily trained human datasets, they often struggle to generalize to non-human scenarios, such as animal motion transfer or object-level editing in nature-based scenes. For fair comparison, conducted qualitative and quantitative experiments UBC|50) and Tik [1] to evaluate the video character transfer performance. Besides, to further evaluate model's is ability generalize other concepts, also demonstrate qualitative results of animal transfer, cloth transfer, background transfer, and motion transfer.
4.2.1 Video Character Transfer
To evaluate the effectiveness our method video character transfer, we compare it with state-of- the-art character animation and video editing baselines, including Animate Anyone [4], Champ [57).
7

---

Table 1: Quantitative comparison of video quality for video character transfer.
TikTok
UBC
LPIPS!
PSNRT
F1D4
FVD.
LPIPS)
SSIMI
FDD
FVDJ
MRAA
0.513
25.31
0.425
134.23
634
0.589
23.32
0.537
89.13
438
DisCa
B.674
21.56
0.132
112.24
571
0.467
23.45
0.412
94.15
483
MagicAnimatie
0.259
89.23
428
48
Animatic Anyone
0.198
25.89
0.713
79.18
398
0.132
26.54
0.798
44.18
378
Champ
0.241
26.57
0.787
67.11
401
0.159
27.03
0.811
11.02
324
UniAnimate
0.191
24.34
0.724
58.19
378
0.127
27.09
0.798
43.24
334
0.152
26.78
46.74
345
27.12
0.814
39.73
Table 2: Video consistency quality comparison. SubC: Subject Consistency; BkgC: Background Consistency; MoS: Motion Smoothness; AesQ: Aesthetic Quality; DyaD: Dynamic Degree.
TikTok
UBC
SubC1
BkgCt
MrS1
Aes
Dyal2t
SubCr
BkgCt
MoSt
AesQt
Dyal?
Video-PIP
0,842
0.853
0.782
0.443
0.710
0.813
0.867
0.734
0.497
0.371
ControlVides
0.712
0.419
0.519
0.823
0.895
0.241
RF-Fditer
0.911
0.873
0.581
0.423
0.765
0.825
01.48
0.789
0.434
0.627
MotionClone
0.784
0,865
0.891
0.651
0.830
0.924
0.943
0.871
0.613
0.679
Cogg V
0.927
0.904
0,936
0.377
0.842
0911
0,917
0.911
0.663
0.902
Mott-VVidoo
0.923
0.917
0.962
0.593
0.837
0.923
0.879
0.957
0.612
0.829
0.945
0,931
0.664
0,911
UniAnimate 143 Any (20]. Video-P2P [23], etc. Qualitative results are presented in Figure 4 Given an input video, these existing approaches typically modify or replace objects based on textual or image- based prompts. As illustrated in Figure the text-guided transfer methods fail to transfer the reference image the video, while the image-guided baselines struggle to preserve the structural features maintain the appearance of the backgrounds. In contrast, our approach leverages reference image guidance and incorporates a progressive spatial and timestep decomposition, which aligns more naturally with real-world editing workflows. Our approach can achieve better preservation of the reference appearance, higher visual quality of the video, and greater inter-frame consistency than all baselines. These improvements are attributed to our novel decomposition strategy, which enables fine-grained control and more structured video generation.
Besides, we perform further quantitative evaluation from two perspectives: video quality and temporal consistency. As shown in Table T and Table[]] our method achieves superior performance across multiple metrics, including FID, LPIPS, and subject consistency, aesthetic quality, etc. from vbench [16. significantly outperforming the compared baselines. This demonstrates the robustness and generalization ability of our approach in complex video character transfer tasks.
4.2.2 Adaptation to Various Video Concept Transfer Tasks.
Our proposed decomposition- based modeling and random masking pre-training mechanism enable our framework to effectively isolate and manipulate individual visual factors within a video, which can generalize beyond conventional foreground transfer. In this section, we provide more diverse qualitative results of a wide range of video concept transfer tasks, including motion transfer, background transfer, animal transfer, and regional foreground transfer, such as clothing replacement. These results demonstrate the flexibility and adaptability of our approach in handling diverse transformations, which are typically challenging for conventional video editing methods.
Motion transfer. Our framework supports motion transfer by applying the motion dynamics extracted from driving video onto a reference image, similar to pioneer works[37]. To evaluate the effectiveness of our method on this task, we conduct comparisons with -tate-of-the-art motion- guided video generation approaches, including Any V2V 200 MotionClone [22]. Motion12V (37). V2V (20) and Mofa-Video [3I]. The results are shown in F As illustrated, Motion12V struggles with maintaining the video consistency as time progresses, often introducing noticeable artifacts or drift in later frames. V2V fails to preserve the appearance of the reference image. MotionClone primarily relies on text guidance for controlling video appearance, which limits its ability to precisely align the generated content with the reference image. Mofa-Video may generate blurry results. contrast, our method produces videos that not only maintain high visual fidelity but also exhibit coherent and realistic motion consistent with the driving video. This advantage stems from our explicit spatial decomposition and motion-conditioned generation design, which enables better disentanglement and control over appearance and temporal dynamics.
8

---

Ret Image
Target Video
Ref Motion
AnyV2V
MotionClone
Motion-12V
Mida-Vidoo Video
Ours
Figure Comparison video motion transfer.
Table 3: Ablation study.
Visual Quality
Video Smoothness
LPIPS 4
PSNR
SSIM
FID 4
FVI. 4
SubC
BkgC f
Early Motion Injection
0.234
25.16
0.743
56.79
374
0.871
0.736
w/o Flow Noise Injection
0.293
24.31
0.659
69.31
0.915
0.893
w/o Self-Supervised Pre-training
0.498
19.87
0.546
101.34
0.735
0.892
w/o dual-stream decomposition
0.341
23.12
0.694
78.13
423
0.894
0.639
w/o CoP Timestep Decomposition
0.192
25.87
0.712
51.32
357
0.881
0.924
Full Model
0.152
26.78
0.803
46.74
345
0.945
0.931
Foreground transfer. Our framework enables flexible foreground transfer, including part-level object replacement, such garment transfer. It poses significant challenges due to the need to selectively modify localized visual regions while preserving the subject's identity and overall video harmonization. This is particularly important in fashion, virtual try-on, and personalization applications. As shown in Figure 7 our model successfully replaces the clothing items in the target videos. with the referenced garments, without disrupting the consistency of facial features, body motion, or background. These results highlight the strength of our decomposition framework in enabling precise and high-quality object-level modifications within complex video generation tasks.
Background transfer. In most video editing approaches, modifications are typically limited to the main subject local elements. In contrast, our method supports background replacement by leveraging the spatial disentanglement. As illustrated in Figure 7 our framework enables users seamlessly replace the entire background of video. Our approach ensures that background changes-such as scene transitions from indoor to outdoor-are consistent across all frames, while the foreground subject remains temporally coherent and unaffected in terms of identity and motion.
4.3 Ablation Study.
To verify the effectiveness the components in our video generation pipeline, we designed several sets of ablation experiments on [ikT Specifically, we evaluate the following configurations: (1) Early Motion Injection: Instead of injecting motion in the intermediate network layers, we input motion information alongside the foreground and background features in the early layers, and apply iitention-based fusion within the denoising branch. (2) Feature Fusion (w/o dual- stream decomposition): We directly concatenate foreground and background features and feed them into the denoising module. (3) Without Self-Superviser Pre- Pre-training. (4) Without CoP Timestep Decomposition. (5) Without Flow Noise Injection. The ablation study results are shown Table From the first two rows, we can see the superiority of the design of our motion branch, where the added noise improves the generalization ability and the injection strategy provides more accurate guidance. The third row emphasizes that the self-supervised training is critical for enhancing the

---

decomposed representation learning. The last two rows illustrate that our modeling of the spatial and timestep decomposition both are essential for flexible and high-quality video concept manipulation.
Conclusion
In this work, we present novel framework for controllable video concept transfer by introducing progressive spatial and timestep decomposition modeling strategy. Unlike existing methods that treat the video as a holistic entity, our approach explicitly disentangles the video into foreground, background, and motion components, and further decomposes the denoising process into hierarchical stages via chain-of-prompt guidance. This design allows for more fine-graing sne-grained control over video synthesis, enabling diverse video concept transfer tasks, such as character transfer, motion transfer, background transfer, and garment-level editing. We also incorporate a self-supervised pretraining scheme based random masking to support robust learning under additional supervision, which effectively bootstraps spatial disentanglement using large-scale unlabeled video data. Besides, we curate a new dataset featuring animal subjects to advance research in video generation. Extensive experiments on different transfer tasks demonstrate that our method significantly outperforms state- of-the-art baselines in terms of visual quality, consistency, and controllability.
Acknowledgments and Disclosure of Funding
Weiwei Xu partially supported by NSFC (No. 92570206, 6o.62421003). Rong Zhang is partially supported by NSFC (No. 62302449) and Zhejiang Provincial Natural Science Foundation of China (No. LQ23F020009). Zhiyuan Ma supported by the National Natural Science Foundation of China (No. 62406161), China Postdoctoral Science Foundation (No. 2023M741950), and the Postdoctoral Fellowship Program of CPSF (No. G7BB0230347). This paper is partially supported by the Yongjiang Innovation Project No. 20257062.
References
[1] Rameen Abdal, Or Patashnik, Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Sergey Tulyakow, Daniel Cohen-Or, and Kfir Aberman. Dynamic concepts personalization from single videos. In Proceedings of the Special Interest Group Computer Graphics and Interactive Techniques Conference Conference Papers, pages 1-9, 2025.
[2] Rishabh Bhardwaj and Soujanya Poria. Red-tcaming large language models using chain of utterances for alignmet arXiv preprint 2066022 2023.
[3] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 18392-18402, 2023.
[4] Minghong Cai, Xiaodong Cun, Xiaoyu Li, Wenze Liu, Zhaoyang Zhang, Yong Zhang, Ying Shan, and Xiangyu Yue. Ditetrl: Exploring attention control in multi-modal diffusion transformer for tuning-free multi-prompt longer video generation. arXiv:2412. 18597, 2024.
[5] Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.0720. 2023.
[6] Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. survey on evaluation large language models. ACM transactions on intelligent systems and technology. 15(3):144 2024.
[7] Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 6593 6602, 2024.
[8] Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. Chain-of verificaaii reduces hallucination in large language models. arXiv preprint arXiv:2309.1144 11499,11111100 2023.
10

---

[9] Silin Gao, Jane Dwivedi- Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva. Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, and Tianlu Wang. Efficient tool with chain-of-abstraction 2024.
reasoning. arXiv preprint arXiv:2401.
[10] Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. In The Twelfth International Conference on Learning Representations, 2024.
[11] Yuchao Gu, Yipin Zhou, Bichen Wu, Licheng Yu, Jia-Wei Liu, Rui Zhao, Jay Zhangjie Wu, David Junhao Zhang. Mike Zheng Shou, and Kevin Tang. Videoswap: Customized video subject swapping with interactive semantic point correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 7621-7630, 2024.
[12] Shirley Anugrah Hayati, Tachee Jung, Tristan Bodding-Lort Sudipta Kar, Abhinav Sethy, Joo- noo-Kynng Kim, and Dongycop Kang. ioattiof-issttuctions: Compositional instruction tuning on large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24005-24013, 2025.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-68571 2020.
[14] Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. arXiv:231 1.1-117, 2023.
[15] Li Hu, Guangyuan Wang. Zhen Shen, Xin Gao, Dechao Meng, Lian Zhuo, Peng Zhang, Bang Zhang, and Liefeng Bo. Animate anyone 2: High-fidelity character image animation with environment affordance. arXiv preprint arXiv:2502.06144 2025.
[16] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprchensive benchmark suite for video generative models. 2024.
[17] Yasamin Jafarian and Hyun Sao Park. Learning high fidelity depths of dressed humans by watching social media dance videos. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), pages 12753-12762, June 2021.
[18] Hyeonho Jeong, Geon Yeong Park, and Jong Chul Ye. Vme: Video motion customization using temporal attention adaption for text-to-viden diffusion models. In Proceedings of the IEFE//VV Conference on Computer Vision and Pattern Recognition, pages 9212-9221, 2024.
[19] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22213,, 2022.
[20] Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to-video editing tasks. arXiv preprint arXiv:2403. 14468, 2024.
[21] Chengshu Li, Jacky Liang, Andy Zeng, Xinyun Chen, Karol Hausman, Dorsa Sadigh, Sergey Levine, Li Fei-Fei, Fei Xia, and Brian Tchter. Chain of code: Reasoning with language model-augmented code emulator. arXiv preprint arXiv:2312.04474. 2023.
[22] Pengyang Ling, Jiazi Bu, Pan Zhang, Xiaoyi Dong, Yuhang Zang, Tong Wu, Huaian Chen, Jiaqi Wang, and Yi Jin. Motionclone: Training- free motion cloning for controllable video generation. orXiv:2406.05338 2024.
[23] Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8599-8608, 2024.
[24] Zhili Liu, Yunhao Gou, Kai Chen, Lanqing Hong, Jiahui Gao, Mi, Zhang, Zhenguo Li, Xin Jiang, Qun Liu, et al. Mixture of insightful experts (mote): The synergy of thought chains and expert mixtures in self-alignment. arXiv preprint arXiv:2405.0055 2024.
[25] Zhiyuan Ma, Guoli Jia, and Bowen Zhou. Adapedit: Spatio-temporal guided adaptive editing algorithm for text-hased continuity-scnsitive image editing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4154-4161, 2024.
[26] Zhiyuan Ma. Zhihuan Yu, Jianjun Li, and Bowen Zhou. Lmd: faster image reconstruction with latent masking diffusion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4153- 2024.
11

---

[27] Zhiyuan Ma, Yuzhu Zhang, Guoli Jia, Liangliang Zhao, Yichao Ma, Mingjie Ma, Gaofeng Liu, Kaiyan Zhang, Ning Ding. Jianjun Li, et al. Efficient diflusion models: A comprehensive survey from principles to practices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[28] Zhiyuan Ma, Liangliang Zhao, Biqing Qi, and Bowen Zhou. Neural residual diffusion models for deep scalable vision generation. Advances in Neural Information Processing Systems, 37:1 17456-117480 2024.
[29] Yifang Men, Yuan Yao, Miaomiao Cui, and Liefeng Bo. Mimo: Controllable character video synthesis with spatial decomposed modeling. arXiv preprint arXiv:2409.1611 2024.
[30] Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang. Zhijie Chen, Xiang Li. Jian Yang. and Ying Tai. Openvid- Im: A large-scale high-quality dataset for text-to-video generation. arXiv:2407.02377. 2024.
[31] Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang, Ying Shan, and Yinqiang Zheng. Mofa-video: Controllable image animation via generative motion field adaptions in frozen image-to-video diffusion model. arXiv:2405.2022 2024.
[32] Hao Ouyang, Qiuyu Wang. Yuxi Xiao, Qingyan Bai, Juntao Zhang, Kecheng Zheng, Xiaowei Zhou, Qifeng Chen, and Yujun Shen. Codef: Content deformation ficlds for temporally consistent video processing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8089-800 9089-8099, 2024.
[33] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF international conference on computer vision, pages 4195-4205, 2023.
[34] Di Qiu, Zheng Chen, Rui Wang, Mingyuan Fan, Changqian Yu, Junshi Huang, and Xiang Wen. Moviecharacter: A tuning-free framework for controllable character video synthesis. arXiv preprint arXiv:2410.20974, 2024.
[35] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam Segment anything in images and videos. arXiv preprint arXiv:2408.00714 2024.
[36] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. 2022.
[37] Xiaoyu Shi, Zhaoyang Huang, Yun Wang, Weikang Bian, Dasong Li, Yi Zhang, Manyuan Zhang, Ka Chun Cheung. Simon See, Hongwei Qin, et al. Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling. In ACM SIGGRAPH 2024 Conference Papers, pages 1-11, 2024.
[38] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning. March 2025.
[39] Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision -ECCV 2020: 16th European Conference, Glasgow; UK, August 23-28, 2020, Proceedings, Part II, page 402-419, 2020.
[40] Hongru Wang, Rui Wang. Fei Mi, Yang Deng. Zezhong Wang, Bin Liang, Ruifeng Xu, and Kam-Fai Wong. Cue-cot: tfain-of-thought prompting for responding to in-depth dialogue questions with llms. arXiv preprint arXiv: 1192, 2023.
[41] Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing. arXiv preprint arXiv:2411. 144446466 2024.
[42] Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot -dentityppeeeevvig generation in seconds. arXiv preprint arXiv:2401.0755 2024.
[43] Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion models for consistent human image animation. arXiv preprint arXiv:2406.01 01 188, 2024.
[44] Yu Xia, Rui Wang. Xu Liu, Mingyan Li, Tong Yu, Xiang Chen, Julian McAuley, and Shuai Li. Beyond chain-of thought: survey of chain-of of paradigms for LLMs. Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend AI- Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, Proceedings of the 31st International Conference on Computational Linguistics, pages 10795- 10809, Abu Dhabi, UAE, January 2025. Association for Computational Linguistics.
12

---

[45] Ziyang Xiao. Dongxiang Zhang. Yangjun Wu, Lilin Xu, Yuan Jessica Wang, Xiongwei Han, Xiaojin Fu, Tao Zhong, Jia Zeng, Mingli Song, et al. Chain-of-experts: When Ilms meet complex operations research problems. The twelfth international conference on learning representations, 2023.
[46] Yutaro Yamada, Khyathi Chandu, Yuchen Lin, Jack Hessel, liker Yildirim, and Yejin Choi. L3go: Language agents with chain-of- oo--df---oouuuu sd-thoughts for generating unconventional objects. arXiv preprint arXiv:2402.0900 0995520... 2024.
[47] Tao Yang, Tianyuan Shi, Fanqi Wan, Xiaojun Quan, Qifan Wang, Bingzhe Wu, and Jiaxiang Wu. Psycot: psychological questionnaire as powerful thain-of-thougee for personality detection. arXiv preprint arXiv:2310.20256, 2023.
[48] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-vidoo diftusion models with an expert transformer.
2024.
[49] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran. Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[50] Polina Zablotskaia, Aliaksandr Siarohin, Bo Zhao, and Leonid Sigal. Dwnet: Dense warp based network pose-guide gose-guided human video generation. British Machine Vision Conference British Machine Vision Conference, Oct 2019.
[51] Lvmin Zhang and Maneesh Agrawala, Adding conditional control text-to-image diffusion models. In ICCV, 2023.
[52] Yuanhao Zhang, Yumeng Wang. Xiyuan Wang, Changyang He, Chenliang Huang. and Xiaojuan Ma. Coknowledge: Supporting assimilation of time-synced collective knowledge online science videos. arXiv preprint arxiv:2550..33767 2025.
[53] Yuxin Zhang. Weiming Dong, Fan Tang. Nisha Huang, Haibin Huang, Chongyang Ma. Tong- Yee Lee, Oliver Deussen, and Changsheng Xu. Prospect: Prompt spectrum for attribute-aware personalization of diffusion models. ACM Transactions on Graphics (TOG), 4266):244:11244:: 2023.
[54] Zhenghao Zhang, Junchao Liao, Menghao Li, Zuozhuo Dai, Bingxue Qiu, Siyu Zhu, Long Qin, and Weizhi Wang. Tora: Trajectory-orientee diffusion transformer for video generation, 2024.
[55] Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, and Mike 7heng Shou. Motiondirector: Motion customization of text-to-video diffusion models. In European Conference on Computer Vision, pages 273-290. Springer, 2024.
[56] Denny Zhou, Nathanacl Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang. Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205. 10625, 2022.
[57] Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu, Champ: Controllable and consistent human image animation with parametric guidance. In European Conference on Computer Vision, pages 145-162. Springer, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction accurately reflect our paper's contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitation of the method proposed is discussed in the supplemental
material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiaatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In the Section 3 and Section 4 have provided sufficient information for the experimental results reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
15

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instructions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We are happy to release our code the reproduction. Due to time constraints, we plan to organize and disclose the code upon completion of the paper submission process.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
public/guides/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccccuclccuuueeeeeeeeeeeee Cotee//ddeeeSbmmmcia ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
We have provided the training and test details in Section 4.1 of the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due to the significant computational resources required for multiple repeated execution of all experiments, we have not reported error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification:
We have reported the information on the computer resources in the Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///aaaarrr..ccppp//ccccccca
Answer: [Yes]
Justification: We have read the NeurIPS Code of Ethics and confirm that the research this article fully complies with its content.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper has discussed both potential positive social impacts and negative societal impacts of the work performed the supplemental material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited the original paper that provided the code and dataset used in the paper.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Video Diffusion Model Preliminary
Video diffusion models generalize the concept of image diffusion probabilistic models [13 to the temporal domain. Formally, let 2g E xfxAxxxx represent a video latent variable, where f denotes the number of frames, with h dimension, channels. The forward diffusion process is defined as a Markov chain that gradually corrupts the original video into Gaussian noise:
24 == V + vI O4C, <~N(0,,)
where E (1..... } indexes the diffusion timestep, ce controls the noise intensity, and e represents standard Gaussian noise. In the reverse process, denoising network is used to estimate the noise from 2 to zg, typically called Diffusion Model 0. The training objective minimizes the following loss function:
( = E,,, [||e éa (=e, C, t)11
(5)
where C represents conditioning information such as text prompts or reference images.
B Implementation Details
In this paper, we focus on image-guided video concept transfer, where the foregrounds, backgrounds, or motion dynamics can be manipulated. We perform experiments including character, animal, object, background and motion transfer to evaluate our approach. The weights of the model are initialized from pretrained Cog VideoX-5B 48 The experiments conducted on server equipped with 8x NVIDIA Tesla H100 80G GPUs. In our paper, we generate 49-frame videos at the resolution of 720 x 480.
Our training pipeline consists of two stages: a large-scale self-supervised pretraining phase and supervised fine-tuning phase. In the pre-training phase, we trained our model on the Open Vid dataset (30, large-scale collection of approximately 12 million videos covering a wide range of categories including humans, animals, vehicles, and diverse backgrounds. The training used learning rate of x 10 and ran for 200, 000 iterations. The Open dataset comprises approximately 12 million samples.
In the supervised training phase, we adapt the pretrained model to specifie video concept transfer scenarios. For the human-centric editing task, we fine-tune the model combination of the TikTok [17] and UBC Fashion [50] datasets, which include rich annotations for character-level transfer tasks. The model is finetuned for 10, 000 iterations with learning rate of 5 x 10 For the animal-centric task, we performed an additional 10, 000 fine-tuning iterations on OpenAnimal using the same learning rate.
C More Details about Self-Supervised Pretraining
In practice, obtaining large amounts of high-quality annotated data remains challenging. Although efficient segmentation tools like SAM2[35] are available, they still require extensive manual interactions, such as point prompts, bounding boxes, or object-specific filtering. To address this limitation and reduce the reliance on manual annotations, we introduce a self-supervised pretraining approach, the detailed algorithmic pipeline of which outlined below Algorithm and Figure
D More Details about CoP Guidance
The detailed algorithmic pipeline of our Chain-of-Promp (CoP) guidance is demonstrated in Algorithm 2
To assess the effectiveness of CoP, we conducted ablation experiments comparing with static prompts, including: (i) All Coarse Prompt: using a coarse prompt across all timesteps, and (ii) All Detailed Prompt: using detailed prompt across all timesteps. The results demonstrate that the all- coarse setting tends to produce results lacking fine mne-grained appearance fidelity and exhibiting texture degradation, while the all-detailed setting often introduces misguidance during early denoising steps, leading to distorted structures. In contrast, our CoP strategy, which progressively schedules coarse-to- fine prompts, strikes a balance between structural stability and appearance precision, achieving better performance. The quantitative evaluation can be referred to in Table
21

---

Figure 6: Self-supervised Pretraining.
E More Results
Our framework enables flexible foreground and background transfer, including part-level object replacement, such as garment transfer. The results of different transfer tasks are shown in Figure TO Figure [1. Figure 12 Figure
7
Our curated animal-centric dataset OpenAnimal
is demonstrated in Figure I3
Target Video
Target Video
RefAnimul
Ref Animtal
Animal Transfer
Animal Transfer
Target Video
Target Video
Ref Background
Gamment
Background Transfer
Garment Transfer
Figure 7: More UniTransfer results of different transfer tasks.
F
Limitation
Although our model can achieve subject transfer and background replacement in videos, there are still cases where the subject and background appear with artifacts. This issue might be due to current segmentation models cannot fully separate foreground and background elements, leading to imperfect composite results in the final output. In the future, plan to address this by leveraging large-scale models for enhanced video scene understanding, further improving the quality generated videos.
For scenes involving multiple moving entities, we can fuse their individual optical flows into composite representation and sequentially condition the network on each object through multiple inference passes. For interacting subjects, our method remains effective when the occlusion or overlap between them relatively small, as each subject's region can still be identified and modeled with reasonable independence. However, cases with large overlapping areas or complex physical interactions pose challenges to our models due ambiguity in motion attribution and visual entanglement. Joint modeling of interacting subjects is interesting direction for future work.
22

---

Algorithm 1: The pipeline of our self-supervised pretraining.
Input:
image: input image
coverage: target coverage ratio (default: 0.5)
min_block___ _izce_size:: minimum block size (default: 320)
mar_ caar_blocksiiee maximum block size (default: 640)
Output: foreground: processed image with coverage background: inverse masked image
Function random white blmaes coverage, if image is None then
size, max block maæ_block
size)
raise ValueError(" "Input image is empty"):
if image is grayscale then
resull + convert image to BGR color space; else
L result + copy of image;
(h, w) + height and width of result;
area h x wi; covered_aren 0- total
target_arees
coverage;
mask t- zero matrix of size (h, w);
result gray matrix (127.5) with same size as result; mar muar_iter-ao + 500;
while covered_aree _areeddrrraaae largel_areed and mar_iler 0 do max_iler + max__ler l;; 1;
block block_size size random integer between min block size and max block size; x + random integer between 0 and w block size; y random integer between and block block_sized
if maskly y + block_sicee size, x x [lock_size) contains any 1 then continue:
Special overlapping effect result_ -[y: y block_ size 5, x: I + block_ - 5] + resultfy resultly y block_sizee x. + block_sisi siooc_sszeeee + 5] [127.5, 127.5, 127.5]: maskly y block block_size,rr covered_aren covered_arend + block_sise biksssizee> block_size;
block_size _ize 5, x + block_sizee
size 5):
block_sized size]] +;;
if covered_aren _areeeddaaaaae > 1.1 x larget_area then break;
return foreground, background;
Table 4: CoP mechanism with different level prompts.
Visual Quality
Video Smoothness
LPIPS
PSNR f
SSIM 1
FID 4
FVD 4
SubC f
BkgC f
All Coarse Prompt
0.174
23.15
0.701
72.46
433
0.714
0.865
All Detailed Prompt
0.165
24.62
0.734
49.26
367
0.832
0.901
Full Model
0.152
26.78
0.803
46.74
345
0.945
0.931
G
Social Impact
Our research decouples video generation into foreground, background, and their corresponding motion. This technology will enhance the efficiency of video production, drive innovation in industries such as film and gaming, and enable more immersive entertainment and educational experiences. However, as this technology becomes more widespread, society may face ethical and legal challenges, including concerns over the authenticity of video content, characters, and backgrounds. Therefore, establishing appropriate regulatory frameworks ensure responsible use of this technology is critical task that demands our attention.
23

---

Algorithm 2: Chain-of-Prompt- Video Denoising
Input: initial initial_noisy__ pooisyyvieeo: Initial noisy video
pase_prompt: Original text description
total steps: Total denoising steps (default: 50)
T1, T2: Stage transition steps
Output: generated _iiieo: Final generated video
Function hierarchical wew.ooeinnnnnniiiiii
video, base_prompp, total fita__sspss__ 50)
// Phase partitioning (all steps) 71 +- 35;
// First phase // Second phase
T2 + 15;
I/ Generate hierarchical prompts using LLM
prompts QwQ32B GonccaoHHiorrcccccaPoootts/sssss poooppt____pp
// Returns: {'stagel' coarse, 'stage2' detailed, 'stage3': fine) current_viden t- ináttal_ iratial_noisy_vvide _onooyooiddeeo
for step t- total_steps to do
step 2 71 then
quidoncce_promye
prompls['stagel]]
// Coarse prompt
quidonce_weight + 1.5;
// Strong guidance
else if step 2 T2 then
guidonce_pprot proneptppppppttt + prompts('stage21]
// Detailed prompt
guidance_weight +- 2.0;
else
guidence_promept + prompts| stage3'] srogelss"stagee]]
// Fine prompt
guidance_weighe 1.0:
// Execute denoising step current current_vided video +
CenoiseStep(curreen video, guwdance_ppooee sereddnnneweight,.o step);
return petreett_vides
Ref Video
Ours
AnyVVV
Video påp
Control Video
Animate-Anyone
Champ
UniAnimate
Magic/knimace
Figure 8: More results compared with other methods.
24

---

Figure 9: More Visual Results (Left is reference video, middle is reference image, right is output).
Figure 10: More Visual Results. (Left reference video, middle is reference image, right is output)
25

---

PANI
PRACH
Paars
Figure 11: More Visual Results.(Le. (Left reference video, middle is reference image, right is output)
Figure 12: Animal Motion Transfer. (Left reference video, right is output)
26

---

Figure 13: OpenAnimals Datasets.
27


Reviews:
[{"review_id": "c0vUd4Gbvt", "reviewer": "Reviewer_FXM4", "summary": "This paper introduces UniTransfer, a novel architecture for precise and controllable video concept transfer. The core of UniTransfer lies in a progressive decomposition paradigm applied to both spatial and diffusion timestep dimensions. Spatially, it decouples a video into three key components: the foreground subject, the background, and the motion flow. This separation is managed by a dual-to-single-stream DiT-based architecture, enabling fine-grained control over each element.", "questions": "None", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength\n\nMulti-stage Training Strategy: The model is first pre-trained on a large-scale dataset using a random masking strategy to acquire extensive prior knowledge. During this process, the model's learning is progressively guided by gradually increasing the size of the masked regions.\n\nGranularity-aware Denoising: This work addresses the phenomenon in diffusion models where the denoising process first recovers structural information before progressively generating fine-grained textural details. To leverage this, a large language model is used to transform a single text prompt into three distinct, granularity-specific prompts. These are applied at different stages of the denoising process to focus the model's attention on the relevant aspects at each step.\n\nDecomposition and Fusion for Versatility: The training strategy involves decomposing the video into foreground, background, and motion components, which are then fused. This framework allows the model to adapt to a wide variety of transfer and combination tasks, demonstrating strong versatility. In contrast, other methods are often limited to editing or transferring only single, more isolated regions or objects\n\n\nWeakness\n\nIn the ablation study, was the comparison between the dual-stream and single-stream architectures conducted under the same computational cost? Similarly, was the comparison with early motion injection also performed under an equivalent computational budget?\n\nDoes the spatial branch use multi-frame noise? Why not use single-frame noise, given that this branch might only need to handle the spatial foreground-background relationship, while temporal dynamics are modeled by the motion branch? Modeling temporal features early on could potentially be misled by the text features.\n\nConsidering the results shown in Figure 4, Animate Anyone uses the pose sequence from the target video to drive a reference image. If we disregard the background and focus only on the foreground for comparison, the results in the second and third rows of Figure 4 suggest that the method proposed in this work is inferior to Animate Anyone in terms of preserving the person's appearance and identity.\n\nThis work uses optical flow as the motion condition. Does it account for the motion of the background? Generating consistent background motion would make the video more realistic. The framework could be further improved by considering the motion of multiple objects.", "was_revised": true, "final_justification": "After reading other reviews, I keep the positive rating.", "rebuttal": "(1) In the ablation study, was the comparison between the dual-stream and single-stream architectures conducted under the same computational cost? Similarly, was the comparison with early motion injection also performed under an equivalent computational budget?\n\n**Answer：**\nThank you for your thoughtful questions. Here is our response regarding the two points:\n\n- **Comparison between dual-stream and single-stream architectures.**\n  Our current experiments are built upon the pretrained CogVideoX-5B backbone. The single-stream model also maintains a 5B-scale parameter count, taking about $180$ seconds to generate a $720\\times480$ video with $49$ frames with an Nvidia-A100 GPU. However, the dual-stream architecture includes an additional foreground branch, which increases the total parameter count by approximately 30% and the inference time by about 19\\%. To ensure a fair comparison under an equivalent computational budget, we also reconstructed the dual-stream model using a smaller CogVideoX-2B backbone, resulting in a total model size of around 3B parameters. Despite the smaller size, this 3B dual-stream model outperforms the original 5B single-stream baseline in our evaluations, as shown in the following table. This result provides further evidence of the architectural effectiveness of our dual-stream design. We will include this additional comparison in the revised version and add more qualitative results to guarantee the fairness of the ablation setup.\n\n- **Comparison with early motion injection.**\n  The comparison with early motion injection was conducted under an equivalent computational budget, using the same base model and training settings. The only difference is the injection position of motion features, ensuring that the performance gains are due to the proposed motion-injection strategy, not model capacity or training cost differences.\n\n**Comparison of different base architectures.** SubC: Subject Consistency; BkgC: Background Consistency; MoS: Motion Smoothness; AesQ: Aesthetic Quality; DyaD: Dynamic Degree.\n|                       | SubC  | BkgC  |  MoS  | AesQ  | DyaD  |\n| :-------------------: | :---: | :---: | :---: | :---: | :---: |\n|   Single-stream-5B    | 0.927 | 0.904 | 0.926 | 0.557 | 0.842 |\n|  Ours-Dual-stream-3B  | 0.933 | 0.913 | 0.933 | 0.591 | 0.869 |\n| Ours-Dual-stream-6.5B | **0.945** | **0.931** | **0.962** | **0.651** | **0.903** |\n\n---\n\n(2) Does the spatial branch use multi-frame noise? Why not use single-frame noise, given that this branch might only need to handle the spatial foreground-background relationship, while temporal dynamics are modeled by the motion branch? Modeling temporal features early on could potentially be misled by the text features.\n\n**Answer:**\n\tOur model is built upon the CogVideoX architecture, and we follow its design paradigm by adopting multi-frame noise as input across all branches, including the spatial branch. While the motion branch primarily models temporal dynamics, the spatial branch in our architecture is mainly responsible for integrating foreground-background structures. However, the input motions may not always align perfectly with the foreground or background. In these cases, dynamics of the unaligned areas can be generated with the guidance of text prompts by introducing multi-frame noise in the spatial branch and enabling coarse temporal modeling during the early stage. This helps to suppress artifacts that may be caused by misaligned motion signals. We recognize the reviewer's concern that early modeling of temporal features could be misled by text prompts. In our work, during the initial denoising steps, only coarse textual guidance is provided. Finer-grained prompts are introduced progressively as the denoising proceeds. This staged guidance reduces the risk of misleading caused by inappropriate textual descriptions.\n\n---\n\n(3) Considering the results shown in Figure 4, Animate Anyone uses the pose sequence from the target video to drive a reference image. If we disregard the background and focus only on the foreground for comparison, the results in the second and third rows of Figure 4 suggest that the method proposed in this work is inferior to Animate Anyone in terms of preserving the person's appearance and identity.\n\n**Answer：**\n\tWe would like to clarify that our work focuses on video concept transfer, where the goal is to substitute the target concept in the video with a user-specified new concept, while preserving the remaining video content. AnimateAnyone is designed specifically for pose-driven human video generation from a single reference image. It assumes that the appearance and identity of the subject should remain fixed. While AnimateAnyone may appear to preserve the details better in certain examples, our solution significantly outperforms it in terms of overall object motion coherence and background consistency. Please refer to the videos on our anonymous webpage given in the paper for a holistic and objective comparison.\n\n\n---\n\n(4) This work uses optical flow as the motion condition. Does it account for the motion of the background? Generating consistent background motion would make the video more realistic. The framework could be further improved by considering the motion of multiple objects.\n\n**Answer：**\nThank you for your constructive suggestions regarding motion modeling for background and multiple objects.\n\n- **Background Motion Handling.**\n\n  We agree that generating consistent background motion can enhance video realism. Our current framework is capable of handling small-scale background motion—see the middle video in the second row of \"Human foreground transfer\" on our anonymous site (given in the paper), where global camera movements are reflected in generated videos. However, large-scale or complex background motion is challenging for our method. As the reviewer suggested, incorporating explicit background motion generation could further improve realism, and we consider this a promising direction for future work. We plan to explore learning or estimating background motion patterns jointly with foreground objects to achieve a more holistic motion representation.\n\n- **Multi-Object Motion Modeling.**\n  For scenarios involving multiple independently moving objects, we plan to fuse their individual optical flows into a composite representation and sequentially condition the network on each object through multiple inference passes. We will include corresponding results in the revised version of the paper."}, {"review_id": "Mdyr3DJ2Zo", "reviewer": "Reviewer_Lpcd", "summary": "The paper proposes UniTransfer, a novel architecture for precise and controllable video concept transfer. It spatially decomposes videos into foreground, background, and motion flow, using a dual-to-single-stream DiT-based architecture for fine-grained control. It also employs a \"Chain-of-Prompt\" mechanism for timestep decomposition, leveraging LLMs to guide the denoising process progressively. The paper also introduces the OpenAnimal dataset.", "questions": "- Is it reasonable to understand 'Sec. 3.1.2 Self-supervised Pre-training via Random Masking' as finetuning the vdm to learn inpainting prior?\n- Is the model trained from scratch or finetuned from a pretrained video diffusion model? If the model is trained from scratch, why couldn't the authors finetune from a pretrained video model?", "limitations": "- The paper omits discussing a highly relevant video concept personalization paper \"Dynamic Concepts Personalization from Single Videos\". What are the strength/limitation compared to this paper?\n- The paper omits discussing pioneering video motion transfer works, such as \"MotionDirector: Motion Customization of Text-to-Video Diffusion Models\", \"Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer\", and \"VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models\". Specifically, how does the method performs compared to MotionDirector?", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "[Strength]\n- The paper proposes a single integrated model to perform various concept transfer tasks.\n- The authors collect OpenAnimal dataset o facilitate the training and benchmarking of research community.\n\n[Weakness]\n- For character transfer or video motion transfer, the reference input images are quite spatially aligned with the target video (e.g., fig1 top left, fig 4, fig 5). Is it necessary to have spatially-aligned reference input image? How does UniTransfer perform given spatially-unaligned reference input image?\n- Regarding timestep decomposition, can the authors discuss why just providing a long, detailed prompt that contains all information of high-level, structural guidance, detailed, appearance-level description for all timesteps does not work? In other works, can the authors provide comparison experiments against this approach, which is similar to LLM-based prompt enhancing practice of recent vdms?", "was_revised": true, "final_justification": "The paper proposes an unified framework for various concept transfer tasks, in addition to a new benchmark dataset. I believe the contributions will be valued by the community. Moreover, the rebuttal answered all my remaining questions. Hence, I would like to keep my rating.", "rebuttal": "(1) For character transfer or video motion transfer, the reference input images are quite spatially aligned with the target video (e.g., fig1 top left, fig 4, fig 5). Is it necessary to have spatially-aligned reference input image? How does UniTransfer perform given spatially-unaligned reference input image?\n\n**Answer：**\n\tFor character or video motion transfer, our method can handle spatial misalignment between reference images and target videos (see videos on our anonymous website for examples). This is achieved by detecting the minimum bounding boxes of the main subjects in both the reference image and the first frame, then our model adjusts these boxes to align them through affine transformations.But significant misalignment truly poses a challenge for our method and the current research community.In such cases, a potential solution is to first perform pose alignment via pose transfer or image personalization methods as a pre-processing step. \n\n---\n\n(2) Regarding timestep decomposition, can the authors discuss why just providing a long, detailed prompt that contains all information of high-level, structural guidance, detailed, appearance-level description for all timesteps does not work? In other works, can the authors provide comparison experiments against this approach, which is similar to LLM-based prompt enhancing practice of recent vdms?\n\n**Answer**：\nOur timestep decomposition strategy is designed in terms of two aspects:\n\n- **Task Characteristic.** Our study specifically targets the concept transfer task, where the goal is to seamlessly merge two distinct images guided by different prompts. In this context, providing fine-grained textual prompts for either the foreground object or the background image may introduce misguidance to the model and hinder the effective feature fusion. \n  To overcome this limitation, we adopt a timestep decomposition strategy.\n- **Diffusion Model Characteristic.** The strategy is grounded in empirical and theoretical insights from prior diffusion literature *Prospect*, which observes that early timesteps in the denoising process are responsible for generating structural layouts, while later timesteps refine details such as texture and appearance. Based on this, we hypothesize that providing coarse, high-level prompts in early stages helps avoid conflicting guidance, thus stabilizing the structure generation process.\n  In contrast, providing a fully detailed prompt at every timestep can lead to confusion or prompt oversaturation, where certain aspects of the prompt may be neglected.\n\n**Cop mechanism On Different Methods.** SubC: Subject Consistency; BkgC: Background Consistency; MoS: Motion Smoothness; AesQ: Aesthetic Quality; DyaD: Dynamic Degree.\n\n|                                 |       |       | CogVideoX |       |       |      |       |       | MotionClone |       |       |\n| :-----------------------------: | ----- | :---: | :-------: | ----- | ----- | ---- | ----- | ----- | :---------: | ----- | ----- |\n|                                 | SubC  | BkgC  |    Mos    | AesQ  | DyaD  |      | SubC  | BkgC  |     MoS     | AesQ  | DyaD  |\n|        All Coarse Prompt        | 0.853 | 0.892 |   0.765   | 0.512 | 0.734 |      | 0.712 | 0.823 |    0.657    | 0.425 | 0.689 |\n|       All detailed Prompt       | 0.911 | 0.912 |   0.662   | 0.511 | 0.501 |      | 0.789 | 0.871 |    0.881    | 0.662 | 0.842 |\n| with CoP Timestep Decomposition | **0.917** | **0.921** |   **0.893**   | **0.559** | **0.893** |      | **0.836** | **0.889** |    **0.912**    | **0.683** | **0.864** |\n\nTo validate the design, we conducted two experiments:\n\n1.  ablation study of using the same detailed prompt across all timesteps. The results demonstrate that our method yields superior consistency and fidelity, particularly in subject consistency and background consistency.\n2. applying our chain-of-prompt strategy to recent VDM-based methods CogVideoX and MotionClone. We observed a notable performance improvement, indicating that CoP is also generalizable beyond our framework.\n   The quantitative results are demonstrated in the following tables. We will include these results and related discussions in the revised version of the paper.\n\n​                                                                  **CoP mechanism with different level prompts.**\n\n|                     |       |       | Visual Quality |       |      |      | Video Smoothness |       |\n| :-----------------: | ----- | ----- | :------------: | ----- | ---- | ---- | :--------------: | ----- |\n|                     | LPIPS | PSNR  |      SSIM      | FID   | FVD  |      |       SubC       | BkgC  |\n|  All Coarse Prompt  | 0.174 | 23.15 |     0.701      | 72.46 | 433  |      |      0.714       | 0.865 |\n| All Detailed Prompt | 0.165 | 24.62 |     0.734      | 49.26 | 367  |      |      0.832       | 0.901 |\n|     Full Model      | **0.152** | **26.78** |     **0.803**      | **46.74** | **345**  |      |      **0.945**       | **0.931** |\n\n---\n\n(3) Is it reasonable to understand 'Sec. 3.1.2 Self-supervised Pre-training via Random Masking' as finetuning the vdm to learn inpainting prior?\n\n**Answer**:  Yes, the random masking is utilized to finetune the vdm to learn inpainting prior to seamlessly fusing foreground and background without large-scale paired labels.\n\n---\n\n(4) Is the model trained from scratch or finetuned from a pretrained video diffusion model? If the model is trained from scratch, why couldn't the authors finetune from a pretrained video model?\n\n**Answer:** The weights of the model are initialized from a pretrained CogVideoX-5B, which can leverage its generation ability. We will clarify this in the revision.\n\n---\n\n(5) The paper omits discussing a highly relevant video concept personalization paper \"Dynamic Concepts Personalization from Single Videos\". What are the strength/limitation compared to this paper?\n\n**Answer:** \n\t*Dynamic Concepts Personalization* finetunes LoRA layers to learn an identity basis from videos that capture appearance while avoiding temporal interference, enabling realistic fusion effects through basis combination. Although it achieves realistic results, it requires separate training to fit for each scene and does not support target image input. In contrast, our feedforward approach allows for more efficient inference for all scenes. And we decouple the video generation into fore-ground image, background image and corresponding motions, making it more easier to transfer any of these parts. In addition, the two methods differ in focus: ours enables explicit subject transfer in video using a reference image, whereas theirs allows only limited object replacement via text prompt, making it limited application compared with us.\n\n---\n\n(6) The paper omits discussing pioneering video motion transfer works, such as \"MotionDirector: Motion Customization of Text-to-Video Diffusion Models\", \"Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer\", and \"VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models\". Specifically, how does the method performs compared to MotionDirector?\n\n**Answer:** \n\tThese three methods are all designed for text-guided video editing tasks, which lack the ability of pixel-level visual control. MotionDirector requires separate training for each motion type. When adapting to new scenarios, it only supports text-guided scene transitions, which significantly limits its flexibility. VMC also needs seperate training for different scenes.  Though 'Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer' is training free, when applied to motion-transfer, it only supports text guidance for new scene. In contrast, our method adopts both textural and visual guidance, enabling more precise control of the generated video. We will include the discussion in the related works of the revised version."}, {"review_id": "P0HZRc0q48", "reviewer": "Reviewer_djA4", "summary": "This paper introduces UniTransfer, an image-guided video generation framework that achieves controllable video concept transfer by progressively decomposing both the spatial content of a video by separating each clip into foreground, background and motion flow; and diffusion denoising timeline by applying a chain-of-prompt strategy. To reduce annotation effort, the model first trained on self-supervised random mask scheme , then fine-tuned with limited masks and optical flow. They also release a 10k-clip dataset of single-animal videos.", "questions": "- Please specify the timestep boundaries (T1, T2) and the prompt lengths produced by the LLM for each stage. How sensitive are results to (i) the choice of those boundaries and (ii) the quality or length of the LLM-generated coarse/mid prompts.\n- Report end-to-end inference time, peak GPU memory, and FLOPs for generating a 16-frame 512×512 video.\n- The current pipeline assumes one foreground mask. Can UniTransfer handle scenes with two interacting subjects (e.g. two people, or one animal-one person, or multiple subjects ) or must it be run sequentially per subject.\n- Only qualitative results are shown for these tasks. Could you report objective metrics versus any baselines where applicable?\n- Include a brief section (main or appendix) with representative failure modes (e.g. extreme perspective mismatch, inaccurate optical flow, or misleading LLM summaries) and describe potential remedies.", "limitations": "The paper lists a basic limitation (artifact‐prone foreground/background compositing) and notes potential misuse of synthetic videos, but some important points remain unaddressed:\n- Computational demand and reproducibility:\n- Single-foreground assumption.\n- Performance depends on accurate optical flow and LLM prompt quality.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper tackles with the long-standing challenge of flexible, high-fidelity video editing with a dual-decomposition strategy that is both technically sound and motivated.\n- By splitting generation along spatial (foreground, background, motion) and temporal (coarse-to-fine denoising) axes, the authors deliver a unified framework that handles character, garment, background and motion transfer in a single model.\n- The proposed dual-to-single-stream DiT architecture and Chain-of-Prompt guidance are rigorously described, backed by equations, ablations and intuitive figures, reflecting high quality and clarity.\n- The self-supervised random-mask pre-training further reduces annotation cost and is validated to be essential, adding practical value beyond raw performance.\n\nWeaknesses:\n- The framework depends on a three-branch diffusion transformer, a large language model for prompt decomposition, and an external optical-flow estimator, which raises the bar for reproducibility and deployment, diminishing practical quality despite strong results.\n- Some important scenarios remain unexplored: the method assumes a single foreground subject and largely static backgrounds, so its behaviour with multiple moving objects, heavy camera motion, or depth-layered scenes is unclear, limiting its claimed significance.\n- Quantitative evaluation is thorough for character replacement but only qualitative for garment and background transfer, leaving a gap in empirical evidence for those tasks and slightly weakening overall clarity of validation.\n- The paper also omits discussion of inference cost and LLM latency, and gives limited insight into failure modes of the Chain-of-Prompt mechanism.", "was_revised": true, "final_justification": "I thank the authors for their detailed rebuttal. After careful reflection on their response and the nature of the paper's contribution, I have decided to increase my score to 5.\nMy initial primary concern was the lack of quantitative evaluation for tasks like garment and background transfer. While this remains a valid point, I have reconsidered its weight relative to the paper's overall contribution. The core innovation of this work is the UniTransfer framework, a novel and technically sound architecture that demonstrates remarkable versatility across four distinct concept transfer tasks. The authors successfully show this flexibility qualitatively, which is the primary evidence for their main claim.\n\nFurthermore, the rebuttal effectively addressed my other concerns regarding computational cost, model limitations, and failure modes, demonstrating a high degree of transparency and rigor. Given this, I find that penalizing the paper for not conforming to evaluation metrics from more specialized domains may be overly rigid. The paper's contribution to general-purpose, controllable video generation is significant. The strengths in its novel architecture and demonstrated versatility now, in my view, outweigh the limitations in its evaluation strategy. Therefore, I now lean more towards acceptance.", "rebuttal": "(1) The framework depends on a three-branch diffusion transformer, a large language model for prompt decomposition, and an external optical-flow estimator, which raises the bar for reproducibility and deployment, diminishing practical quality despite strong results. \n\n**Answer**: \n\tThank you for your comments. To ensure reproducibility, the full codebase will be released to the community.\n\n---\n\n(2) Report end-to-end inference time, peak GPU memory, and FLOPs for generating a 16-frame 512×512 video.\n\n**Answer**: \n\tIn our paper, we generate 49-frame videos at the resolution of $720\\times480$. For a 16-frame $512\\times512$ video, the inference time is about 60 seconds, and the peak GPU memory is about 22G with 23.42 TFLOPS using Nvidia-A100 series GPU.\n\n---\n\n(3) Some important scenarios remain unexplored: the method assumes a single foreground subject and largely static backgrounds, so its behaviour with multiple moving objects, heavy camera motion, or depth-layered scenes is unclear, limiting its claimed significance. Can UniTransfer handle scenes with two interacting subjects (e.g. two people, or one animal-one person, or multiple subjects ) or must it be run sequentially per subject.\n\n**Answer**: \n\tFor scenes involving multiple moving entities, we can fuse their individual optical flows into a composite representation and sequentially condition the network on each object through multiple inference passes. For interacting subjects, our method remains effective when the occlusion or overlap between them is relatively small, as each subject’s region can still be identified and modeled with reasonable independence. However, cases with large overlapping areas or complex physical interactions pose challenges to our models due to ambiguity in motion attribution and visual entanglement.\nWe acknowledge this as a current limitation and joint modeling of interacting subjects is an interesting direction for future work.\n\n---\n\n(4) Quantitative evaluation is thorough for character replacement but only qualitative for garment and background transfer, leaving a gap in empirical evidence for those tasks and slightly weakening overall clarity of validation. Could you report objective metrics versus any baselines where applicable?\n\n**Answer**: \n​\t\tOur method is designed for the video concept transfer task in general scenarios. While we did not target garment or background transfer specifically, our method naturally generalizes to these tasks, as shown in the teaser of our paper. Direct quantitative comparison with prior methods is inherently unfair to both sides: those methods rely on small, narrow-domain datasets and incorporate task-specific structures and losses (e.g., structure preserving module and garment consistent losses), while our model is trained on large-scale, general-purpose data without such targeted structures and losses.\nAs a result, our approach supports broader generalization, though it does not explicitly enforce fine-grained consistency. Adding task-specific components could improve metric scores on these subtasks, but would compromise generality. Exploring such lightweight plugin-style extensions without compromising the generality of the core model is a promising future direction.\n\n---\n\n(5) The paper also omits discussion of inference cost and LLM latency, and gives limited insight into failure modes of the Chain-of-Prompt mechanism. Include a brief section (main or appendix) with representative failure modes (e.g. extreme perspective mismatch, inaccurate optical flow, or misleading LLM summaries) and describe potential remedies.\n\n**Answer**: \n\t\tWhile LLMs add some computation, the overhead is minimal, with prompt generation taking only 5 seconds out of 215 (∼2\\%) for a 49-frame $720\\times480$ video on an NVIDIA-A100 GPU, which is acceptable given the benefits.\nWe will include a brief section on representative failure modes (F) and potential remedies (R), as following:\n\n- **F: Extreme Perspective or Pose Mismatch.** R: Adopt a perspective/pose alignment via generative transfer or image personalization methods as a pre-processing step. \n\n- **F: Inaccurate Optical Flow.** R: Introduce auxiliary guidance from image features, depth estimation, semantics, and multi-frame correlations to enhance motion estimation in ambiguous regions.\n\n- **F: Misleading LLM Summaries.** To evaluate the impact of LLM stochasticity on CoP, we generated multiple prompt sequences using different random seeds for the same input. The resulting videos remained visually and structurally consistent, with no noticeable style shifts or artifacts, indicating that CoP offers a robust and near-deterministic control interface. This robustness likely stems from prompt stage sensitivity: while coarse-level prompts influence results more, they are typically short and simple, thus less affected by LLM variability. \n  Under extreme conditions, a potential remedy is to introduce an independent LLM-based verification agent to assess the plausibility of generated LLM summaries.\n\n- **F: Interacting Objects with Significant Overlap.** R: Auxiliary pose and depth information can be introduced alongside optical flow to better capture the spatial occlusion relationships between multiple objects..\n\n---\n\n(6) Please specify the timestep boundaries (T1, T2) and the prompt lengths produced by the LLM for each stage. How sensitive are results to (i) the choice of those boundaries and (ii) the quality or length of the LLM-generated coarse/mid prompts.\n\n**Answer**: \n- Since different tasks and denoising strategies may be better suited to different timestep boundaries, we adopt a generally applicable scheme resembling a three-way split as specified in Algorithm.2 of the supplementary material. To assess the sensitivity of the boundaries, we conduct ablation studies by varying either $T_1$ or $T_1$ individually. Experimental results reveal that  $T_1$ has a greater impact on the generated videos and is more sensitive than $T_2$, as coarse prompts are applied during the early timesteps and guide the overall generation process.\n  We will include this discussion and the corresponding results in the revision.\n- Our current multi-level prompts are generated with LLM based on semantic hierarchy. The reviewer’s suggestion to further control prompt length and quality is constructive, as it has the potential to enhance the analytical clarity and ensure consistency across tasks. In response, we plan to include comparative experiments with explicit constraints on prompt length and sentence structure (e.g., the number and arrangement of subject-verb-object and adjective) in the revision.\n\n---\n\n(7) Limitation: The paper lists a basic limitation (artifact‐prone foreground/background compositing) and notes potential misuse of synthetic videos, but some important points remain unaddressed: Computational demand and reproducibility/ Single-foreground assumption/ Performance depends on accurate optical flow and LLM prompt quality.\n\n**Answer**: \n\tWe will include detailed computational demand in the revised version and release the code to ensure reproducibility. In addition, we will include the discussion of the *single-foreground assumption* and *performance dependence*, to further explore the boundary of our method."}, {"review_id": "7O2ZFayDeg", "reviewer": "Reviewer_Cz3v", "summary": "This paper introduces UniTransfer, a video generation framework that supports controllable concept transfer (e.g., character, motion, garment, and background) using image-based references. Extensive experiments across TikTok, UBC, and a newly released OpenAnimal dataset show performance gains over baseline methods.", "questions": "1. What evidence supports that CoP improves generation beyond standard prompting?\n\nPlease compare with static prompts, or learnable embeddings. How robust are the LLM prompts under rephrasing or minor perturbations?\n\n2. How deterministic is CoP?\n\nGiven the stochastic nature of LLMs, how do different CoP generations affect the final result? Do prompt variations lead to style shifts or artifacts?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Clear motivation and modular design: Tackles a real problem in video generation—lack of flexible and fine-grained control.\n\n2. Methodologically comprehensive: Combines spatial disentanglement, motion conditioning, and progressive textual control into a unified pipeline.\n\n3. Solid empirical performance: Competitive results across several tasks (character transfer, background replacement, garment/motion transfer).\n\n4. Dataset contribution: OpenAnimal provides non-human evaluation data and enhances the paper’s experimental diversity.\n\n\nWeaknesses:\n\n1. The CoP mechanism is conceptually interesting but remains weakly justified. No baseline comparison (e.g., single prompt or learned embeddings), nor any analysis of prompt quality, semantic drift, or hallucination is provided.\n2. The use of LLMs for generating stage-specific prompts introduces non-determinism, inference overhead, and variability, which are not quantified or discussed.", "was_revised": true, "final_justification": "The authors have addressed my concerns by providing a complete rebuttal. I would be happy to raise my score.", "rebuttal": "(1) The CoP mechanism is conceptually interesting but remains weakly justified. No baseline comparison (e.g., single prompt or learned embeddings), nor any analysis of prompt quality, semantic drift, or hallucination is provided. Please compare with static prompts, or learnable embeddings. How robust are the LLM prompts under rephrasing or minor perturbations?\n\n**Answer:** \nThank you for your insightful suggestions regarding the CoP mechanism.\n\n- **Comparison with Static Prompts**\n\n  To assess the effectiveness of CoP, we conducted ablation experiments comparing it with static prompts, including:\n  (i) using coarse prompt across all timesteps, and (ii) using detailed prompt across all timesteps. The results demonstrate that the all-coarse setting tends to produce outputs that lack fine-grained appearance fidelity and exhibit texture degradation, while the all-detailed setting often introduces misguidance during early denoising steps, leading to distorted structure. \n  In contrast, our CoP strategy, which progressively schedules coarse-to-fine prompts, balances structural stability with appearance precision and achieves better performance both qualitatively and quantitatively.\n  The quantitative evaluation can be referred to in the following table.\n\n- **Robustness to Prompt Rephrasing**\n\n  We further evaluated the robustness of the LLM-generated prompts under rephrasing and textual perturbations. The results indicate that mid- and fine-level prompts are generally robust to rewording. In contrast, coarse-level prompts are relatively more sensitive to rephrasing as they are applied during the early timesteps and primarily guide the global structure. Rephrasing at this stage may lead to structural shifts or artifacts, since the deviations in the early steps tend to accumulate during the denoising process.\n\n**Cop mechanism On Different Methods.** SubC: Subject Consistency; BkgC: Background Consistency; MoS: Motion Smoothness; AesQ: Aesthetic Quality; DyaD: Dynamic Degree.\n\n|                                 |       |       | CogVideoX |       |       |      |       |       | MotionClone |       |       |\n| :-----------------------------: | ----- | :---: | --------- | ----- | ----- | ---- | ----- | ----- | ----------- | ----- | ----- |\n|                                 | SubC  | BkgC  | Mos       | AesQ  | DyaD  |      | SubC  | BkgC  | MoS         | AesQ  | DyaD  |\n|        All Coarse Prompt        | 0.853 | 0.892 | 0.765     | 0.512 | 0.734 |      | 0.712 | 0.823 | 0.657       | 0.425 | 0.689 |\n|       All detailed Prompt       | 0.911 | 0.912 | 0.662     | 0.511 | 0.501 |      | 0.789 | 0.871 | 0.881       | 0.662 | 0.842 |\n| with CoP Timestep Decomposition | **0.917** | **0.921** | **0.893**     | **0.559** | **0.893** |      | **0.836** | **0.889** | **0.912**       | **0.683** | **0.864** |\n\n​\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t**CoP mechanism with different level prompts.**\n\n|                     |       |       | Visual Quality |       |      |      | Video Smoothness |       |\n| :-----------------: | ----- | ----- | :------------: | ----- | ---- | ---- | :--------------: | ----- |\n|                     | LPIPS | PSNR  |      SSIM      | FID   | FVD  |      |       SubC       | BkgC  |\n|  All Coarse Prompt  | 0.174 | 23.15 |     0.701      | 72.46 | 433  |      |      0.714       | 0.865 |\n| All Detailed Prompt | 0.165 | 24.62 |     0.734      | 49.26 | 367  |      |      0.832       | 0.901 |\n|     Ours with CoP Timestep Decomposition      | **0.152** | **26.78** |     **0.803**      | **46.74** | **345**  |      |      **0.945**       | **0.931** |\n\n---\n\n(2) The use of LLMs for generating stage-specific prompts introduces non-determinism, inference overhead, and variability, which are not quantified or discussed. How deterministic is CoP? Given the stochastic nature of LLMs, how do different CoP generations affect the final result? Do prompt variations lead to style shifts or artifacts?\n\n**Answer:** \nThank you for raising important concerns regarding the use of LLMs in the Chain-of-Prompts (CoP) mechanism.\n\n- **Inference Overhead**\n\n  While it is true that incorporating LLMs introduces additional computation, we found the overall inference overhead to be minimal. For instance, generating a 49-frame video at a resolution of $720\\times480$ using an Nvidia-A100 GPU takes approximately 215 seconds, whereas LLM prompt generation only accounts for around 5 seconds, i.e., about 2\\% of the total runtime. This small cost is acceptable given its benefits.\n\n- **CoP Certainty**\n\n  To assess the certainty and impact of LLM stochasticity on CoP, experiments are conducted in which multiple CoP prompt sequences were generated using different random seeds for the same input video and reference image. But the resulting videos were visually and structurally consistent, and we did not observe significant style shifts or artifacts attributable to LLM-induced prompt changes. This supports the conclusion that CoP provides a robust and practically deterministic interface for guiding generation.\n\nA likely reason for this robustness may lie in the sensitivity to different prompt stages. As discussed earlier, mid- and fine-level prompts are generally robust to rewording, while the coarse-level prompts (applied in early denoising timesteps) have a greater influence on the results. But coarse-level prompts are typically short and simple (usually <10 words), making them less susceptible to LLM variability."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field. Most concerns are solved in the rebuttal and discussions. The paper finally received positive reviews with 2 Borderline Accepts and 2 Accepts. 

This paper proposes UniTransfer, a framework controllable video concept transfer. It spatially decomposes videos into foreground, background, and motion flow, using a dual-to-single-stream DiT architecture for fine-grained control. It employs a chain-of-prompt strategy for timestep decomposition, leveraging LLMs to guide the denoising process progressively. 

**strengths**
- the dual-decomposition strategy is technically sound and motivated, e.g., spatial disentanglement, motion conditioning, progressive textual control, etc.
- a unified framework that handles character, garment, background and motion transfer in a single model.
- comprehensive experimental results across several tasks

**weaknesses**
- cannot handle large-scale or complex background motion, complex interactions
- complicated framework (three-branch DiT, a LLM, and optical-flow estimator), which might introduce non-determinism and complexity 

According to the reviews, the reviewers acknowledge the paper's novel design of the decomposition strategy as well as the versatility of the unified framework. 
Following the rebuttal, all reviewers have provided positive ratings for the paper.
In the revision, the authors are suggested to incorporate the reviewers' feedback, such as the implementation details, the running time and memory usage, and the required experimental results.
Decision: Accept (poster)