Temporal In‑Context Fine‑Tuning with Temporal Reasoning for Versatile Control of Video Diffusion Models
Authors: Kinam Kim
Junha Hyung
Jaegul Choo
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion
Video Diffusion
Flow Matching
In context learning
Primary area: deep_learning
TL;DR: This paper introduces an efficient temporal in‑context fine‑tuning framework that enables pretrained video diffusion models to ingest diverse conditioning signals, granting precise and versatile control without any architectural changes.

Abstract: Recent advances in text-to-video diffusion models have enabled high-quality video synthesis, but controllable generation remains challenging—particularly under limited data and compute. Existing fine-tuning methods for conditional generation often rely on external encoders or architectural modifications, which demand large datasets and are typically restricted to spatially aligned conditioning, limiting flexibility and scalability. In this work, we introduce Temporal In-Context Fine-Tuning (TIC-FT), an efficient and versatile approach with temporal reasoning for adapting pretrained video diffusion models to diverse conditional generation tasks. Our key idea is to concatenate condition and target frames along the temporal axis and insert intermediate buffer frames with progressively increasing noise levels. These buffer frames enable smooth transitions, aligning the fine-tuning process with the pretrained model’s temporal dynamics. TIC-FT is architecture-agnostic and achieves strong performance with as few as 10–30 training samples. We validate our method across a range of tasks—including image-to-video and video-to-video generation—using large-scale base models such as CogVideoX-5B and Wan-14B. Extensive experiments show that TIC-FT outperforms existing baselines in both condition fidelity and visual quality, while remaining highly efficient in both training and inference.

Temporal In-Context Fine-Tuning with Temporal Reasoning for Versatile Control of Video Diffusion Models
Kinam Kim
Junha Hyung" KAIST Al
Jaegul Choo
(kinamplify, sharpeeee, Ckaist. acc kr
Abstract
Recent advances in text-to-video diffusion models have enabled high-quality video synthesis, but controllable generation remains caalllnginnn-aarticulaa under limited data and compute. Existing fine-tuning methods for conditional generation often rely on external encoders architectural modifications, which demand large datasets and are typically restricted to spatially aligned conditioning, limiting flexibility and scalability, In this work, we introduce Temporal In- m--tttext Fine- Tuning (TIC-FT), an efficient and versatile approach with temporal reasoning for adapting pretrained video diffusion models to diverse conditional generation tasks. Our key idea is to concatenate condition and target frames along the temporal axis and insert intermediate buffer frames with progressively increasing noise levels. These buffer frames enable smooth transitions, aligning the fine-tuning process with the pretrained model's 's temporal dynamics. TIC-FT architecture-agnostic and achieves strong performance with as few 10-30 training samples. We validate our method across a range of tasks-including including image-to-videe and video- to- video ceneration-usiie
large-scale base models such VogVideoX-5D and Wan 14B. Extensive experiments show that TIC-FT outperforms existing baselines in both condition fidelity and visual quality, while remaining highly efficient in both training and inference. For additional results, visit Ntttps://kinam0255: github. 1o/TFC-FT//
1 Introduction
Text-to-video generation models have advanced rapidly, reaching quality levels suitable for profes- sional applications [
4 6). Beyond basic generation, recent research has increasingly focused on leveraging pretrained models to enable more precise control and conditional guid- ance, addressing the growing demand for finer adjustments and more nuanced generation capa- bilities 7 10
2 3
15 [6].
Despite this progress, current fine-tuning approaches for conditioning video diffusion models face notable limitations. Many methods require large training datasets and introduce additional archi- tectural components, such as ControlNet 7 or other external modules, which impose substantial memory overhead. Moreover, the reliance on external encoders for conditioning often leads to the loss of fine-grained details during the encoding process. ControlNet-style methods [16] 17 [4] in particular, operate within rigid conditioning frameworks: they are primarily designed for spatially aligned conditions and require conditioning signals match the target video length. For example, when conditioning on a single image, common workarounds include replicating the image across the temporal dimension align with the video frames or embedding it global feature. These
indicates equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Prompt (CHARACTER) JoylE young long and choeks, wearing pale green blouse decorated grass pattenss, smilig...
Prompt: [CHABACTER)
onange spacia lage glasses, jetpack hovering bursting
Prompt: IMAGEE bob hairstyle
Prompt: IIMAGE)
transtioon
Prompt: [RACE) man his late 23s with smooth, clear siin, noany parted black hal,, and soft..
Prompt: (CONCITION: group of pactaged food produects...
Prampt.(ACeD-)) (VIDD1)) ekdetly chaf to while unilorn and hat rot's out pizza dough....
Drompt: [STYLE) paitting the style of vao gogh. Then
Figure Demonstration our method across diverse tasks, including character-to-viden virtual try-on, ad-video generation, object-to-motion toonification, and video style transfer.
approaches typically necessitate task-specific adaptations of the conditioning pipeline. Alternative fine-tuning strategies, such as IP-Adapter [18] and latent concatenation [10], encounter similar chal- lenges regarding flexibility and computational cost, as they modify or expand the pretrained model architectures.
In contrast, in-context learning (ICL) offers more efficient and versatile paradigm. ICL training-free and can be flexibly applied to user-defined tasks by providing examples directly within input context, eliminating the need additional parameter updates. While ICL. has shown strong success in large language models [19 20) its application image and video generation has primarily been explored autoregressive models [1 22]. with limited adaptation to diffusion models.
Efforts to implement ICL in diffusion models (23 15] have often relied on ControlNet-style training approaches, which contradict the core advantage of ICL: leveraging pretrained distributions without additional training. Departing slightly from the pure ICL paradigm, recent work has introduced in-context LoRA 8 related technique that enables consistent image generation by producing multiple images in a single forward pass arranged in grids, thereby facilitating information sharing across images. With minimal fine-tuning, this method achieves high-quality and highly consistent results, benefiting from the inherent in-context generation capabilities of pretrained text-to-image models, which are naturally suited for grid-based generation.
In contrast, video generation models possess far less of this capability. Although concurrent research has explored extending in-context LoRA to video generation 244, these models are poorly suited to producing grid-like outputs, making the approach significantly more training raining-intensive intensive and less effective. Furthermore, these methods are not inherently designed for conditional generation and often depend on training-fre free inpainting strategies 26]. which tend to degrade performance. They also lack flexibility handling mismatches between condition length and the number of target frames, as there are straightforward solutions for general cases. In the simple case of conditioning on a single image, the image must be redundantly replicated across all frames, resulting in substantial increases in memory usage and computational overhead.

---

In this paper, we propose a highly effective and versatile fine-tuning method for conditional video diffusion models: temporal in-context fine-tuning. Instead of spatially concatenating condition and target inputs, our approach aligns them cypporally-coccteenaan target frame(s) along the time axis-and fine-tunes the model using only minimal number of samples. This design leverages the inherent capability of pretrained video diffusion models to process temporally ordered inputs, enabling effective generation when condition and target frames are arranged sequentially.
condition frame(s) and
To ensure a smooth transition between the condition and target frames. we introduce buffer coines--iterreddiate frames with monotonically increasing noise levels that bridge the gap be- tween the clean condition frames and the fully noised target frames. These buffer frames facilitate smooth, natural fade-out transitions from condition to generated frames, preventing abrupt scene transitions and preserving consistency with the pretrained model's ' distribution. Combined with this design, our method enables fine-tuning with as few as 10-30 training samples. Additionally, our method preserves the original model architecture without introducing additional modules, thereby reducing VRAM requirements.
The proposed approach also allows the model leverage condition frames directly through unified 3D attention, avoiding the detail loss typically introduced by external encoders. Furthermore, it enables versatile conditional generation by eliminating the need for spatial alignment and accommodating a wide range of condition lengths from single images to full video sequences thereby supporting diverse video-to-video translations and image-to-vided generation tasks.
In summary, our main contributions are as follows:
We propose temporal in-context fine-tuning, a simple yet highly effective method for con- ditional video diffusion that minimizes the distribution mismatch between pretraining and fine-tuning, without requiring architectural modifications.
We demonstrate strong performance with minimal training data (10-30 samples), offering highly efficient fine-tuning strategy.
Our method enables versatile conditioning, supporting variable-length inputs and unifying diverse image- and video-conditioned generation tasks within a single framework.
We validate our method across wide range of tasks, including reference-to-vie videoo generation, motion transfer, keyframe interpolation, and style transfer with varying condition content and lengths.
Related work
Conditional Video Diffusson Models. Many conditional video generation methods 7 g 10
12
15 16 rely on auxiliary encoders (e.g., ControlNet (7)) or architectural modifications (e.g., Adaperr I8), which prevent full exploitation of the pretrained model's capabilities. These approaches typically require larger datasets, longer training. and incur significant memory overhead. Moreover, they are often limited to spatially aligned conditioning, making them less suitable for variable-length or misaligned condition-targe pairs.
L3
[4
In-Context Learning for Diffusion Models.
Inspired by its success in language models 19
20
in-context finetuning (IC-FT) has been explored in visual domains via grid-based generation g 24
but its extension to video is limited. Videos rarely follow grid layouts, and inference methods like SDEdit (25 degrade output quality. Moreover, these approaches assume strict condition-output alignment, making them unsuitable for flexible conditional video generation.
Diffusion with Heterogeneous Noise Levels. Recent works such FIFO-Diffusion (7) and Diffusson Forcing (8 demonstrate that diffusion models can effectively operate on sequences with varying noise levels across frames noise and motivating our use of buffer frames with progressively increasing noise.
sreens-challegging the conventional assumption of uniform
Building on these ideas, propose Temporal In-Context Fine- Fine-Tuning (TIC-FT)-a simple yet effective method that temporally concatenates condition and target frames, inserting buffer frames with increasing noise levels to smooth abrupt transitions in both scene content and noise levels. Unlike
3

---

ControlNet-stylee methods, TIC-FT is architecture-agnosic and naturally supports sariable-lenghe spatially misaligned condition-targer pairs.
Method
3.1 Preliminaries
We briefly review diffusion-baset texi-to-video generation. video with Fi RGB frames is X1:Fa
x A spatio-temporal encoder & maps it to latents z Z = (x E REXCxHXW with F S Fin. H < Hpxx. and W S Wpix, and decoder vi approximately inverts Φ. Latent frames are diffused by (z 1) = ( o, + o,e for t E {o..... T) and ~ (0, I), with predefined schedule (at, ot). 3. A 29 backbone predicts the noise and is trained with
[||= co(z ,t,
Lait B200) c.s.t
where latent frames and paired text condition is sampled from the dataset. Generation starts from (T7) NV I) and iteratively applies a sampler (tt-1) S(z(t), ,t,tt,, t, c; co) until (((), which e decodes to video.
3.2 Temporal concatenation
Overview We introduce overall pipeline of the proposed temporal in-context fine-tuning (TIC-FT) in this section. We first detail the temporal concatenation of condition and target latents- with buffer frames that ease the abrupt scene and noise-level transition monlowenn-followeed by the inference and training procedures formalized in Algorithms 2
Setup. The task to generate a sequence of target frames of length K, denoted as 2
[ (0) kl conditioned on set of input frame(s): z( [z] ....Z!
Our approach
concatenates the condtion and target frames along the temporal axis. A naive formulation simply places the clean condition frames directly before the noisy target frames:
(0)
E (+(++))xxHXX
1:L
LLIIL+K
condition
target
Here, (()) &L+ represents the target latent frames at denoising timestep L.
At inference time, we initialize with (0)) ( and iteratively denoise the concatenated frames with
(0) 2 (t-1)) 1 c3)
(3)
until reaching ()) = 2 At each denoising step. only the K target frames are denoised, while the condition frames are fixed to enforce consistency. The final output video corresponds the target
slice 1:
The flexibility of varying L allows this formulation to generalize across a wide range of conditional video generation tasks. When L 1, the problem becomes an image-to-video generation task: producing a full video sequence from single reference image together with a text description of the sequence.
3.3 Buffer frames
Unlike conventional anage-to-vided (I2V) approaches, where the condition acts as the first frame of the output, our setup also allows for discontinuous conditioning, broadening its applicability. For L > 1, the method naturally extends to tideo-to-video generation. A reference clip can perform video style transfer by transferring its appearance onto a new motion sequence. Likewise, providing an action snippet along with a query frame enables in-contexí action trunsfer, where the observed motion is adapted to novel scene. Supplying sparsely sampled frames supports keyframe inter- polation, allowing the model to smoothly generate intermediate transitions between distant frames.

---

Thus, simple temporal concatenation serves as a unified and highly versatile framework for diverse conditional video generation tasks.
However, this na've approach suboptimal for fully leveraging the capabilities of the pretrained video diffusion model. Aligning the finetuning task closely possible with the pretrained model's distribution is essential to achieve high efficiency-enabling strong performance with minimal data and computational resources. Thus, is desirable design the finetuning process around tasks the model is already proficient at.
Direct concatenation violates this principle in two key ways. First, in scenarios where the target frames do not naturally continue from the condition frames -ie., when there is an abrupt scene transition between the last condition frame and the first target frame- the model is forced to synthesize highly discontinuous content. Pretrained video diffusion models are typically trained on smoothly evolving sequences and lack the inherent capability handle such abrupt transitions, datasets with sudden scene changes are commonly filtered out during data curation. Second, diffusion models are not designed to denoise sequences containing frames with heterogeneous noise levels, as would occur when combining clean condition frames with noisy target frames during the sampling process.
We therefore introduce B intermediate buffer frames that perform temporal reasoning, whose noise levels ig linearly bridge and T:
z (f..D) = [2( (9)
[
b B
+1 T.
B
There can be different design choices for the buffer frames, and we empirically find that using the noised condition frames, (+) (((), yields a good performance. Then the full initial latent sequence becomes
(T) 1.t.
L+ELAB
(Fin)
||
(T)
(5)
butfer
target
3.4 Inference
Let T(z()) be noise level list corresponding the latent sequence ((); T RFXCXHXW
T)P. The initial noise level list at f. T is
T) = [0, fi,...,fpy ......T| E (O....,,TLLBBB
(6)
At any global timestep f, we define the noise levels as:
Txx [0, n (
. (B()), 1,...,1.
where rt(t) Fg F, < t, and (t) otherwise.
Our inference algorithm proceeds by iteratively identifying the frames currently at the maximal noise level f and applying the video diffusion sampler exclusively to those frames. This process continues from T down 0. The full inference procedure is detailed in Algorithm
3.5 Training
For each video-text pair (z( (0()) c)) D. the training proceeds follows. First, we randomly sample a global timestep ~ U111 ([,1,.... T) and Gaussian noise E ~N(00, I). Next, we construct the noised model input sequence (!) with the noise level defined in Eq. 7
The model then predicts the noise è = (z) f, c) for all frames. However, the loss is computed only over the target frames to avoid enforcing supervision for the buffer frames. Specifically, we minimize the mean squared eгror between the true noise and the predicted noise over the target
frame indices, defined C + E a gradient step computed from this loss. By excluding the buffer frames from the loss calculation, the network free predict whatever is most natural for these frames, thereby preventing spurious gradients that could shift the model away from the pretraining distribution. In practice, we observe that the buffer frames often evolve into smooth fade-out and fade-in transition between the condition and target frames. The full training procedure is summarized in Algorithm 2
leelllee - ||
The model parameters 0 updated via
5

---

Algorithm 1: TIC-FT inference
Input: Clean condition latents ") buffer noise levels 71.;; text prompt c; denoiser 60
Output: Denoised target latents z0)) B+
Generate buffer latents (f(1.))
(z FI.8);
// add noise
Sample target latents (T()) ~N(,
Concatenate (²²) +- (0)) 1 (11))) m
for = T to do
// global time descending // noise-level vector
t - )) + [i|4:01
A + S(z ci
return (0)
1,+ B4K
Algorithm 2: TIC-FT training
Input: Dataset D with tuples (z(0) ,)); huffer levels F1:8; noise schedule (ot, de)
Output: Fine-tuned parameters 8
foreach minibatch (z) (0) c) D do
foreach sample in minibatch do
Sample [1 ........,)) and ~ ((
(()) -- ( (t)):
20 (1) (0) + oiE;
(0) 2 (1
è (z c);
C t Li=++++ |e. è.1::
Update 0 using gradients of L:
Experiments
4.1 Overview
We evaluate our proposed method on two recent large-scale text-to-video generation models: Cog VideoX-5B and Wan-14B. Our experiments span diverse range of conditional generation tasks, including:
Image-to-Vides (I2V): e.g., character-to-videe generation, object-to-motion, virtual try-on,
ad-video generation.
Video-to-Vide (V2V): e.g., video style transfer, action transfer, toonification.
A key strength of TIC-FT is its ability to operate in the few-shot regime. We fine-tune models with as few as 10-30 training samples and fewer than 1,000 training steps requiring less than one hour of training time for Cog VideoX-5B on a single A100 GPU.
We both real and synthetic datasets for evaluation and demonstrations. Real datasets include SSv2 (30 and manually curated paired videos, while synthetic datasets are created using models such GPT-4o image generation [3]] and Sora 33 (c.g., translating real images into stylized videos). Each task is provided with 20 condition-target pairs. Additional details are provided in the Appendix.
We compare TIC-FT with three representative fine-tuning methods for conditional video generation. While Cog VigVideeXX555 and Wan-14B are among the most recent and powerful text-to-video diffusion models, most existing editing fine-tuning approaches have not been evaluated such large- scale backbones. To ensure meaningful comparisons, we reimplement the following representative baselines.
ControlNet 9 We include ControlNet as a baseline because large number recent methods built upon it extend its core architecture [16] 17 [4]. It is widely adopted framework that

---

XX
X
No Noiimage Conet
AAH
No Nathee
No Butte
Vo Mattee
Wi
Cond
Buffer
Thrget fromes
Turgie
(a) Zero-shot
(b) Trained
Figure 2: (a) Zero-shot comparison of our method (last row) with variants: without buffer frames and with an SDEdit-style style inpainting strategy ("Replace", second row). Buffer frames enable smoother transitions and better condition preservation. (b) Corresponding results after fine-tuning.
introduces an external reference network and zero-convolution layers to inject conditioning signals, enabling the model to preserve fine-grained visual details while integrating external guidance.
Fun-pose 10]. A simple yet widely adopted strategy is concatenate the condition and target latents, as seen in many recent methods (3 34 35 However, this approach requires architectural modifications and extensive retraining, which is infeasible in low-data regimes (e.g.. 20 samples). Since training such a model from scratch yields extremely poor results, direct comparison would be uninformative. Instead, we adopt Fun-pose- variant of Cog VideoX and Wan that has already been finetuned to accept reference videos effectivel giving it significant advantage.
SIC-FT-Replace 24 This method performs spatial in-context fine-tuning by training the model to predict videos arranged as spatial grids. At inference time, the ground-truth condition is noised and repeatedly injected into the condition grid slot at each denoising step, following SDEdit- style replacement strategy [25), while the remaining grid elements are progressively denoised. This approach represents recent trend in applying in-context fine-tuning techniques diffusion models.
4.2 Results
We conduct quantitative evaluations using VigeooX0 as the base model, focusing on two 12V tasks ootts--ojjcttto-ooooe and tharacter-to-vidco-- shown in Table T For V2V, we evaluate performance on style transfer task (real videos to animation), summarized fine-tuned using LoRA (rank 128) with 20 training samples over 6,000 steps, batch size of and a single NVIDIA H100 80GB GPU, Inference conducted with 50 denoising steps.
Table All models
To assess video quality comprehensively, we use three categories of evaluation metrics: VBench 36 GPT-4o [3I, and Perceptual similarity scores. VBench provides human-aligned assessments of temporal and spatial coherence, including subject consistency, background stability, and motion smoothness. GPT-4o leverages a multimodal large language model to rate acsthetic quality, structural fidelity, and semantic alignment with the prompt. Perceptual metrics quantify low- and high-level visual similarity between condition and target frames, including CLIP-I and CLIP-T (for image/text alignment), LPIPS and SSIM (for perceptual similarity), and DINO (for structural consistency). However, we omit Perceptual metrics when evaluating tasks like object-to-motion where different viewpoints may reduce similarity scores despite correct semantics.
Our model achieves strong performance even with limited training, showing competitive results after only 2,000 training steps-unlike other baselines that require significantly more optimization reach similar quality. Additional comparisons under this low-data, low-compute regime are presented in the Appendix. Despite being conditioned on reference frames, Fun-pose and ControlNet exhibit poor
7

---

Table 1: Comparison VBench, GPT-4o, and perceptual similarity metrics for I2V tasks.
VBench
GPT-4o
Perceptual similarity
Method
subject consislency
bockground consistency
motion
arsthetic quality
strictural similarity
CLIP-1
CLIP-T
LPIPS
SSLM
smvolfeess
similarity
ControlNet
2
0,9658
0,96101
0.9926
3.87
2.69
2.69
0.7349
0.2903
0.6535
0,3477
0.3427
Fun-pose
0,9508
0.3575
SIC-FT-Replice 8 P3
0,9513
0.9676
0.9921
4.10
2.42
2.95
0.7993
0.3064
0.6190
0.4555
0.4246
TXC-FI-Replace
0.9580
097222
0.9926
4.08
2.00
2.48
0.7925
0.3127
0.6163
0,423
0.421
TIC-FT (wia Buffer)
0.9474
0,9686
0.9892
4.05
3.05
3.53
0.7573
0.2986
0.6242
0,4058
0.4160
TIC-FT FT-FTTCA (2K)
0.9505
0.9696
0.9920
4.03
3.08
3.54
0.8066
0.3135
0.6162
0.4203
0.4240
TIC.PT (6K)
0.9672
0.9729
0.9930
4,13
3.14
3.63
0.8329
0,3143
0.4332
0.5917
0.5530
Table 2: Comparison on VBench, GPT-4o, and perceptual similarity metrics for V2V tasks.
VBench
GPT-4o
Perceptual similarity
Method
subjjcct consistenccy
background consisionc
motion sunedhnesy
arsiletic qualipy
strictural similartry
semuniic similarity
CLIPII
CLIP-T
LPIPS.
SSIM
DINO
ControlNet
0,9553
0.9545
0.9854
3.44
2.23
2.41
0.6221
0,2727
0.5431
0.3194
0.2839
Fun-pose
0,9679
4.24
0.3328
SIC-FT-Replace & 24
0.9609
0.9655
0.953
3.99
2.44
2.94
0.7368
0.3198
0.5998
0,2192
0.4025
TIC- TIC-FT-Ree Replace
0.9584
0,9696
0.8022
3.93
2.33
2.92
0.7305
0,3015
0.6373
0.2526
0.3673
TIC-FT (wlo Huffer)
0,9479
0.9571
09744
3.81
2.66
3:20
0.7471
0.3020
0.4687
0,3800
0.4429
(2K)
3.85
TIC-FT (6k)
0.9736
0.9743
0.9935
3.99
3.90
4.41
0.8794
0,3118
0.2251
0.6541
0.6745
condition fidelity. While their outputs appear visually plausible-as indicated by favorable VBench and GPT-4o scores they consistently underperform in Perceptual similarity metrics, highlighting lack of alignment with the conditioning input. This is especially problematic for ControlNet, which relies strict spatial alignment and thus struggles in tasks such character-to-video and object-to- motion, where viewpoint shifts are common. SIC-FT-Replaceffi 24 also performs suboptimally in 12V settings, as it requires replicating single frame across spatial grid leading to high memory usage and inefficient training. Furthermore, its reliance on SDEdit [25] -style sampling during inference degrades generation quality and weakens condition adherence.
supplement quantitative results with qualitative comparisons across 12V and V2V tasks in Figure 3 We also present additional scenarios innluding virtual try-on, ad-video generation, and action transfer-ard are illustrated in Figures and Tand
Overall, our proposed TIC-FT consistently outperforms prior methods across diverse tasks, with both quantitative metries and qualitative examples supporting its superior condition alignment and generation quality. More results and task-specific details are provided in the Appendix.
Caud
Coad
Ours
Fumppose
pose
ConretNrl
CouncolNel
SIC-FT
FT
Figure Qualitative comparison between our method and baseline approaches.

---

Table 3: Quantitative comparison of varying numbers of condition and buffer frames I2V task.
#Cond
CLIP1
LPIPS
1
DINO
#Buffer
Dynamic Degree
CLIP-11
LPMP
SSIM
0.8329
0.7493
0.5917
0.5530
0.72
0.7864
0.6123
0.4128
0.4237
0.8332
0.7390
0.5918
0.5531
0.73
0.7812
06112
0.4130
0.4259
0.8371
0.7360
0.6078
0.5606
0.77
0.7695
0.6121
0.4227
0.4170
0.8396
0.7346
0.7544
Pronjot (CIERRICTER) young
veartig a light ourir
Prompt: (CHARICTSR( amari chulefwolee lorge and short black dreswd pull-yillon cont looking
Prompt: (REFERENCE
anng being forning filning muz trancitinno
Prompt: (REFERENEE PIDEO poiper bellng taaring 24w (START FRAME).
(FM.:GG) A omiler with kands relard mour her clamiider, dheaseci abovbssieeeed welik Idlono, blue
Figure 4: Demonstration of our method on character-to-vider action transfer, and virtual try-on.
4.3 Ablation study
Zero-Shot Validation of Temporal Concatenation We validate the effectiveness of our temporal concatenation design with buffer frames by assessing its zero-shot performance. If the model successfully leverages the pretrained capabilities of video diffusion models, it should generate plausible outputs even without any additional training.
As shown in Figure a(a,, our method with buffer frames (last row) generates target frames that align well with the given condition-ddemonstaane demonstrating strong zero-shot performance. In contrast, removing the buffer frames leads to abrupt noise-level discontinuities between condition and target regions, causing the target frames to degrade and the condition information be poorly preserved. We also compare with zero-shot inpainting methods similar to SDEdit, denoted as "Replace" (second row), which similarly fails to propagate condition signals into the generated frames.
Furthermore, in Figure bb), we observe that strong zero-shot performance correlates with better results after fine-tuning. Our method with buffer frames consistently outperforms other variants: models trained without buffer frames begin with blurry target frames, and the "Replace" strategy fails to apply condition information effectively even after training.

---

Impact of Condition and Buffer Frames. We additionally perform quantitative analysis to in- vestigate how the numbers of condition and buffer frames affect performance. As shown in Table increasing condition frames slightly improves condition fidelity, while more buffer frames enhance video dynamics at the cost of condition adherence. Using one condition frame and three buffer frames achieves the best balance between visual quality, motion dynamics, and efficiency.
5 Conclusion and Limitation
Conclusion. Temporal In-Context Fine-Tuning (TIC-FT) offers an efficient framework for adapting pretrained text-to-video diffusion models diverse conditional video generation tasks that leverage contextual information from demonstrations. TIC-FT temporally concatenates condition and target frames with intermediate buffer frames better align with the pretrained model distribution. This design enables a unified and efficient framework for diverse conditional video generation tasks, consistently outperforming existing methods both condition fidelity and visual quality. TIC- FT achieves these gains without architectural modification and operates with significantly lower computational cost.
Future Directions. TIC-FT currently assumes paired condition -trrgt data per task. Generalizing to multi-task or zero-shot settings, where single TIC-FT model can adapt to heterogeneous tasks without retraining, represents exciting avenue for future research. Furthermore, while our method has been validated on both synthetic and real-world datasets (e.g.. SSv2), expanding datasets to more complex human motion and long-horizon dynamics will further strengthen its generalization ability.
Limitations. Unlike In-Context Learning (ICL) in Large Language Models (LLMs), which infers patterns through contextual reasoning without updating model parameters, TIC-FT requires actual fine- tuning during adaptation- similarr to In- In-Context Fine- tne-Tuning approaches used in image generative models [8] Exploring In-Context Learning (ICL) in video diffusion models could be interesting future direction.
Acknowledgments and Disclosure of Funding
This work was supported by Institute for Information & Communications Technology Planning & Evaluation (ITTP) grant funded by the Korea government (MSIT) NRS-2019-11190075, Artificial Intelligence Graduate School Program (KAIST)). the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2025-00555621), and i-Scream Media.
References
[1] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Macicj Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent vidco diffusion models to large datascts. arXiv preprint or arXhv:2311.J5127, 2023.
[2] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson. Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Lix-video: Realtime video latent diffusion. arXiv preprint axXiv22501.00033, 2024.
[3] Genmo Team. Mochi 1. httpp://ggithi: ctcpmuuub.omm/eeeeeee http://gnnoaaomoeeooo 2024.
[4] Zhuayi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.0661 06072, 2024.
Xingwu Sun, Yanfeng Chen. Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, et al. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint 2rXiv:2411.02266 2024.
[6] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 2025.
10

---

[7] Lvmin Zhang, Anyi Rao. and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836-3847, 2023.
[8] Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng. Yu Liu, and Jingren Zhou. In-context lora for diffusion transformers. arXiv preprint arXiv:2410.23775, 2024.
[9] Zhihao Hu and Dong Xu. Videocontrolnet: A motion-guided video-to-video translation framework by using diffusion model with controlnet. arXiv preprint vrXiv:2307.14077 2023.
[10] aige apps. VideoX-Fum A unified pipeline for vidco generation and editing. https://githut com/ Sess-apps/Viideoo Fun apr 2025. GitHub repository, commit <hash>
[11] Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. 12vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint ar Xiv: 1,0444,, 2023.
[12] David Junhao 7hang. Dongxu Li, Hung Le. Mike Zheng Shou, Caiming Xiong, and Doyen Sahoo. Moonshot: Towards controllable video generation and editing with multimodal conditions. arXiv preprint arXiv:2401.00 0010:2200.01127.. 2024.
[13] Xiang Wang. Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jingren Zhou. Videocomposer: Compositional video synthesis with motion controllability. Advances in Neural Information Processing Systems, 36:7594-766 1,, 2023.
[14] Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl-adapter: An efficient and versatile framework for adapting diverse controls to any diffusion model. arXiv preprint S4Kt::2004009667, 2024.
[15] Zhendong Wang. Yifan Jiang, Yadong Lu, Pengeheng He, Weizhu Chen, Zhangyang Wang, Mingyuan Zhou, et al. In-context learning unlocked for diffusion models. Advances in Neural Information Processing Systems, 36:8542-888 -26422 2023.
[16] Chong Mou, Mingdeng Cao. Xintao Wang, Zhaoyang Zhang. Ying Shan, and Jian Zhang. Revideo: Remake video with motion and content control. Advances in Neural Information Processing Systems, 37: 18481-18505, 2024.
[17] Cong Wang. Jiaxi Gu, Panwen Hu, Haoyu Zhao, Yuanfan Guo, Jianhua Han, Hang Xu, and Xiaodan Liang. Easycontrol: Transfer controlnet to video diffusion for controllable generation and interpolation. arXiv preprint arXiv:2408. 13005, 2024.
[18] Hu Ye, Jun Zhang. Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for toxt-to-image diffusion models. arXiv preprint arXiv:2308. 06721, 2023.
[19] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877: 877-1991, 2020.
[20] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning. arXiv preprint arXiv:2301.00233 2022.
[21] Wentao Zhang, Junliang Guo, Tianyu He, Li Zhao, Linli Xu, and Jiang Bian. Video in-context learning: Autoregressive transformers are zero-shot vidco imitators. In The Thirteenth International Conference on Learning Representations, 2025.
[22] Wentao Zhang, Junliang Guo, Tianyu He, Li Zhao, Linli Xu, and Jiang Bian. Video in-context learning. arXiv preprint arXiv:2407.0733 07706 2024.
[23] Jianzhi Liu, Junchen Zhu, Lianli Gao, Heng Tao Shen, and Jingkuan Song. Aicl: Action in-context learning for video diffusion model. arXiv preprint arXiv:2403.I 11535, 2024.
[24] Zhengcong Fei, Di Qiu, Debang Li, Changqian Yu, and Mingyuan Fan. Video diffusion transformers are in-context learners. arxiv preprint arXiv:2412. 2024.
[25] Chenlin Meng. Yutong He, Yang Song, Jiaming Song. Jiajun Wu, Jun-Ya Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint 2 2021.
[26] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffuusion probabilistic models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 11461-11471, 2022.
11

---

[27] Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. Fifo-diffusion: Generating infinite videos from text without training. arXiv preprint arXiv:2405. 114433....4.,... 2024.
[28] Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz. Russ Tedrake. and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems, 37:24081-241255 2024.
[29] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195-4205, 2023.
[30] Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim. Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag. et The" something something" video database for learning and evaluating visual common sense. In Proceedings of the IEEE international conference on computer vision, pages
2017.
[31] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21200 2024.
[32] Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang. Hanchi Sun, Jianfeng Gao, et al. Sora: A review background, technology, limitations, and opportunities of large vision models. arXiv preprint arXiv:2402. 17177, 2024.
[33] Yan Zeng, Guoqiang Wei. Jiani Zheng. Jiaxin Zou, Yang Wei, Yuchen Zhang. and Hang Li. Make pixels dance: High-dynamin video generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8850-8860, 2024.
[34] Dianbing Xi, Jiepeng Wang. Yuanzhi Liang, Xi Qiu. Yuchi Huo, Rui Wang, Chi Zhang, and Xuelong Li. Omnivdiff: Omni controllable video diffusion for generation and understanding. arXiv preprint arXiv:2504. 10825, 2025.
[35] Yong Zhong, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Chongxuan Li. Concat-id: Towards universal identity-preservnng video synthesis. arXiv preprint arXiv:2503. 150.. 2025.
[36] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vhench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEP/CVF Conference Computer Vision and Pattern Recognition, pages 21807-21818, 2024.
[37] Rasmus Jensen, Anders Dahl, George Vogiatzis. Engin Tola, and Henrik Aanæs. Large scale multi-view stereopsis evaluation. In Proceedings of the IEEE conference computer vision and pattern recognition. pages 406-413, 2014.
[38] Black Forest Labs. Flux. http::///ggaab.ooo./aaaaeeeeeeee 2024.
NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and socictal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
12

---

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] prroidddd proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly state that the paper proposes Temporal un-Context Fine- Tuning (TIC-FT) as simple yet effective method for conditional video generation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper includes limitation in the conclusion, noting that the method cannot handle long condition sequences (>10s) due to memory constraints, and identifies this direction for future work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
13

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
The paper does not present any theoretical results, assumptions, formal proofs. It focuses on a practical fine-tuning method for video diffusion models.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient details to reproduce the main experimental results, including the number of training samples, model backbones and experimental settings.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (c.g., in the case
14

---

of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e-g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will provide open access to the code and data used in the experiments via anonymized links in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips) cc/
cablic/guides httaa//ggiide///eeuuiiis..e
CoodeSuuuissiooooooiio for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurIPS code and data submission guidelines (https www.oooSeummmsssooooooloooo
/ //nips.co ccc tecips.cc//ublic//ge/ http:ppullllgdldee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper provides key experimental settings in Section 4, including model backbones, dataset types, number of training samples, batch size, number of steps, and GPU configurations.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper reports quantitative results using multiple metrics (e.g., VBench, GPT-4o, CLIP-I, LPIPS), but does not include error bars, confidence intervals, or statistical significance tests.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper specifies the compute resources used for experiments, including GPU types (NVIDIA A100 and H100), training steps (e.g., 2K, 6K), batch size (2), and training time.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
16

---

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httppe/e//arrpp...c//aaba///eaaa
Answer: [Yes]
Justification: The research adheres to the NeurIPS Code of Ethics. It does not involve human subjects, private or sensitive data, or ethically concerning applications. All experiments conducted on publicly available or synthetic datasets, and the paper preserves anonymity in compliance with submission guidelines. Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This paper discusses broader societal impacts.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper releases models and datasets and describes about risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
17

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper mentions the licenses and terms of use explicitly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi paperssithcodee thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new assets (code and training scripts) and provide documentation in the appendix and supplemental material.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve any crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with human subjects or crowdsourced data collection
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [NA]
Justification: The method and experiments presented in the paper do not involve the use large language models (LLMs) as an essential, original, or ntn-standard component.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https://neurips: http://arrrpp.ccc///o/eeeeeee//s for what should or should not be described.
19

---

A Technical Appendices and Supplementary Material
A.1 Training Details
All models are fine-tuned using an NVIDIA H100 GPU. Our method builds on the Cog CogVideoX- 5B backbone and is fine-tuned with LoRA (rank 128), resulting in approximately 130M trainable parameters. Training with 49 frames requires roughly 30GB of GPU memory. For ControlNet, we apply LoRA with the same rank, yielding a comparable parameter count of around 150M, and requiring approximately 60GB of GPU memory. For Fun- fun-pose, we use the official full fine-tuning setup, which consumes around 75GB of GPU memory.
A.2 Training Amount vs. Performance
This section demonstrates the training efficiency of our method compared to ControlNet. Figure presents performance curves for various metrics-including CLIP-T, CLIP-I, SSIM, DINO, and LPIPS -PIPS-plotted -lottedd against training time. Our method consistently outperforms ControlNet across all metries at equivalent training durations. Moreover, with the exception of CLIP-T, all metries show a clear upward trend, indicating continued improvement with more training. In contrast, ControlNet exhibits no such trend, suggesting that its training style tends to overfit and struggles to generalize under limited data regimes.
A.3 Ablation Study
We conduct ablation study on various buffer frame designs. Specifically, we compare our default uniformly increasing noise schedule-win with alternative strategies: (1) constant noise level for all buffer frames (denoted as Constant-t, where T = 100), and (2) -inaar-uuddratic schedules with concave or convex profiles. presennss both zero-shot and fine-tuned results for these configurations. While all variants produce reasonable target frames, we observe that the convex schedule and the constant-25 baseline exhibit poor condition alignment and noticeable artifacts in the zero-shot setting. After ane-tuning, all methods perform comparably, though our default setting with uniformly increasing noise remains preferred. Quantitative results after training are presented in Table[Jand and Table for the 12V and V2V tasks, respectively.
We also evaluate the effect of varying the number of buffer frames, ranging from to 5, denoted as Buffer-n Figure In the zero-shot setting, we observe that all configurations perform comparably overall; however, shorter buffers tend to produce noisier transitions, likely due to abrupt scene changes, Conversely, longer buffers show a tendency to weaken the influence of the condition. After fine-tuning, all variants produce similarly high-quality results.
A.4 Dataset
For the object-to-motion task, we use the DTU dataset [37] (License: Non-commercial research use only). For character-to-video. keyframe interpolation, and ad video generation tasks, we manually collected condition-videu pairs tailored to each task. For action transfer, we curate videos from SSv2 (30) (License: Research use only, con-cmmmerci..... In the video style transfer task, we first synthesize starting frames using FLUX. 1-dev (38) (License: Non-commercial License), and then generate paired videos using SoRA (32 (License: Proprictary: use governed by OpenAI terms of service), and Wan2.1 ( (License: Apache 2.0). Each task trained on 30 samples. All videas contain 49 frames at 10 frames per second (fps), resized to either 480x480 or 848x480 while preserving the original aspect ratio.
For evaluation and demonstration, we use image and video conditions that are not part of the training set. These include both manually collected images and synthesized ones generated using GPT- 4o, FLUX, and Sora. For the action transfer task, we use unseen video samples from SSv2 [30]. Quantitative evaluations are conducted on 100 samples. For image-based metries such as CLIP and LPIPS, scores are computed on per-frame basis and then averaged to obtain the final results.
Training and evaluation prompts are generated using GPT-4o. Each prompt is structured encompass the condition, buffer, and target frames, with condition and buffer frames denoted as [CONDITION] and target frames as [VIDE0]. Below is the full prompt used for the sample in the ablation study:
20

---

TIC-FT prompt
This animated clip demonstrates the transformation of static character illustration into a lively and expressive animated figure; [CONDITION] the condition image showcases a cheerful cartoon-style young buffalo with thick brown fur, curved yellow horns, and big, friendly smile. The character's 's wide eyes and upright posture are set against a warm orange background, giving it lively and playful presence. [VIDEO] the video animates the buffalo inside grand museum, where wears a red I-shirt and points excitedly at a large dinosaur skeleton behind glass. Its eyes wide with curiosity and its mouth open in awe, while elegant stone columns and soft lighting emphasize the sense of wonder and fascination with history.
A.5
Task Descriptions
We detail the construction of data and latent sequences for each conditional video generation task used in our experiments. All tasks are configured with a total of 13 latent frames, corresponding to 49 video frames. While this number can be adjusted based on application needs, we adopt the 13-frame setting throughout for implementation simplicity and consistency. The initial latent sequence comprises condition frames, intermediate buffer frames, and noised target frames. An exception is the action transfer task, where buffer frames are omitted, as the last condition frame serves as the starting frame of the target sequence. The specific configurations for each task are described below.
Image-to- Video This task aims to generate a full video conditioned on a single image. The video need not begin directly from the image's 's visual content; instead, the image may represent high-level concept such as character profile or an object viewed from the top, with the video depicting novel dynamics (e.g., rotating 360 view).
single reference image is replicated occupy the first latent frames, followed by 9 target frames.
Clean condition: frame (from the image)
Buffer: 3 frames (noised condition)
Target: 9 frames (pure noise)
We visualize the initial latent frames and their denoising process in Figure g
$11.
115
Toining
06
Tiaiing
Figure 5: Performance curves for CLIP-T, CLIP-I, SSIM. DINO, and LPIPS metrics plotted against training time. Our method consistently outperforms ControlNet across all metrics equivalent training durations.
21

---

Table 4: Ablation study of constant noise scheduling for buffer frames, evaluated on 12V tasks using VBench, GPT-4o, and perceptual/simiarity metrics.
VBench
GPT-4o
Perceptual similarity
Method
subject consistency
background consistency
motion smoothiness
aesthetic quality
structunal similarity
semantic similarity
CLIP-I
CLIP-T
LPIPS
SSIM
DINO
Ours
0,9672
0.9729
0,9930
4.13
3.14
3.63
0.8329
0.3143
0.4332
0.5917
0.5530
Constant-22
0.9516
0.9724
0,9920
4.09
2.81
3.45
0.7734
0.3062
0.6088
0.4240
0,4202
Constant-50
0,9509
0,9740
0,9915
4.05
3.01
3,51
0.7760
0.3010
0.6157
0.4188
0.4228
Constant-75
0.9511
0.9917
3.07
3.68
0.3003
0.6148
0,4250
0.4259
Table 5: Ablation study of constant noise scheduling for buffer frames, evaluated on V2V tasks using VBench, GPT-4o, and perceptual/similane metrics.
VBench
GPT-4o
Perceptual similarity
Method
subject consistency
background consistency
motion smoothhess
aesthetic quality
structunal similarity
semontic similarity
CLIP-I
CLIP-T
LPIPS
SSIM
DINO
Ours
0,9736
0.9743
0,9935
3.99
3,90
4.41
0,8794
0.3080
0.2298
0.6541
0.68%
Constant 25
0.9539
0.9652
0,9873
3.90
3,55
4.20
0,8037
0.3103
0.2744
0.5785
0.6083
Constant-50
0.9886
3.86
0.8460
0.3153
0.2364
0.6039
0.6528
Constant 0,9327
0.9552
0,9821
3.69
3.60
4.25
0.8330
0.3142
0.2797
0.5707
0.6368
Video Style Transfer This video-to-video task transforms the visual style of a source video into that of target domain (c.g., converting a realistic vidco into an animated version) while preserving motion and structure.
The first 7 latent frames are taken from source video and the remaining 6 from style-transferred version.
Clean condition: 4 frames (from the source video)
Buffer: 3 frames (noised condition)
Target: 6 frames (pure noise)
We visualize the initial latent frames and their denoising process in Figure g
In-Context Action Transfer This task generates a video that continues a novel scene using motion inferred from a source video. Given a reference action and the first frame new environment, the model synthesizes future frames that imitate the observed motion within the new context.
The first 6 latent frames are from reference action video, the 7th the first frame of novel scene, and the rest are the continuation.
Clean condition: frames (from the reference action video)
Query frame: 1 clean frame (from the novel scene)
Target: 6 frames (pure noise)
No buffer frames are used in this task, as the first frame of the target video is explicitly provided as part of the condition. We visualize the initial latent frames and their denoising process in Figure I0
Keyframe Interpolation This task fills intermediate frames between sparse keyframes to produce temporally coherent video. The goal is to ensure smooth transitions between given keyframes.
Four keyframes are replicated to fill the first latent frames, and the remaining are interpolated.
Clean condition: 4 frames (replicated keyframes)
Buffer: 3 frames (noised condition)
Target: frames (pure noise)
We visualize the initial latent frames and their denoising process in Figure 11
Multiple Image Conditions This task takes two distinet types of image conditions such as person and clothing, person and object-and generates target video that reflects the combination of
22

---

both. This setup is useful for applications like virtual try-on (VITON) or ad video synthesis, where two semantic entities must be jointly represented in motion.
The first 3 latent frames are derived from the first condition image, and the next from the second condition image.
Clean condition: 4 frames (3 from the first image, from the second)
Buffer: 3 frames (noised condition)
Target: 6 frames (pure noise)
Note that the number of condition sources is not limited to two; the framework supports arbitrary multi-condition setups. We visualize the initial latent frames and their denoising process in Figure 12
A.6 Broader Impacts and Misuse Discussion
Our TIC-FT method enables efficient adaptation of video diffusion models with minimal data. However, this ease of fine-tuning also introduces risks, particularly the potential misuse for creating deepfakes or misleading symthetic media. Clear usage policies and responsible deployment practices essential to mitigate societal risks.
23

---

Cond
Replace
Clean Cond
Ours
Constant-75 Constant- Constant-22 25
Concave
Convex
Cond
Buffer
Target frames
Cond
Buffer
Target frames
(a) Zero-shot
(b) Trained
Figure 6: Qualitative comparison of buffer frame designs in zero-shot and fine-tuned settings.
24

---

Buller-2
Outs
Buller-
Burice
Concone
Conoes
Cond
duffer
Target trames
(n) Zero-shot
Buffer
BLffer22
Chrr
Buffer 4
Bufitr 5
Concars
Coonvxx
Targel frumes
(b) Trained
Figure 7: Qualitative comparison of buffer frame designs in zero-shot and fine-tuned settings.
25

---

Denoos
Condition
Buffer
Target frames
Figure 8: Visual results for initial frames and their denoising process on image-to-video generation. Prompt: [Character] clear, high-resolution front-facing close-up of a cheerful cartoon-style wolf character, centered against...
Tageet
Figure 9: Visual results for initial frames and their denoising process on video style transfer task. Prompt: [VIDEOI]A A woman in tan cloak walks gracefully along forest path. Her hair flows gently with her movement, and the ...
26

---

Denoss
Condttin
Novel Scene
Target frames
Figure 10: Visual results for initial frames and their denoising process on in-context action transfer task. Prompt: IREFERENCE VIDEO) A white paper is folded in half by person wearing black sleeves dark indoor environment.
Figure 11: Visual results for initial frames and their denoising process on keyframe interpolation task.
Prompt: [VIDEOI] A cartoon woman with red hair and jeweled headpiece slowly tilts her head and changes facial expressions...
27

---

Denoise
Condition
Condition
Buffer
Target frames
Figure 12: Visual results for initial frames and their denoising process on virtual try-on task. Prompt: [IMAGE] A young woman with long black hair, wearing cream blouse, blue jeans, and black sandals, smiles with both...
28


Reviews:
[{"review_id": "cf5tuxe8qi", "reviewer": "Reviewer_QrUT", "summary": "The paper proposes a method, called temporal in-context learning (TIC), for conditional video generation, including image-to-video and video-to-video tasks. Essentially, the visual conditions are concatenated with the noisy latents along the time dimension, allowing the generated frames to be viewed as a natural extension of the conditional frame(s). Several buffer frames are introduced between them to smooth out the transition from input conditions to generated frames. This design spares the need to modify the model architecture and can be trained efficiently with just a few samples. Experimental results compared to a few baselines address the effectiveness of the method in consistency and transferability.", "questions": "See weaknesses", "limitations": "See weaknesses", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "(+) TIC-FT is a clever approach to enabling image/video conditions for video generation without requiring the introduction of a large number of new parameters. Viewing the generation as a continuation of the visual condition in the temporal dimension provides a new perspective and more flexible design for future work in this domain. \n\n(+) The design of buffer frames also makes sense in providing space for a transition process between conditions and generated frames. The noise schedule is carefully considered to make the transition smoother. \n\n(+) TIC-FT seems like an efficient fine-tuning strategy that works with just tens of training samples in limited GPU hours. \n\n(+) Experimental results demonstrate superiority in consistency and guidance following compared to ControlNet and Fun-pose. \n\n\nSome concerns about experiments:\n\n(-) Small-scale evaluation set. Each task is evaluated on only 20 condition-target pairs, which may not be very representative of showing the whole picture of the performance superiority. Are there larger scale benchmarks or more standard ones that can be used for evaluation?\n\n(-) Metrics: One drawback I could imagine from the temporal concatenation is the strong reliance of every generation frame on the input conditions (or condition shortcuts), which may lead to less dynamics in the output videos. VBench provides metrics beyond just consistency (including dynamic degree), but they are not reported in the tables. \n\n(-) Ablation study on the number of buffer frames. I like the design of buffer frames and that the noise schedule of them is compared carefully. But it seems to me that the paper should also come with certain takeaways or conclusions based on quantitative evaluations (beyond qualitative figures in the appendix) on how many buffer frames to choose. There are some interesting questions to ask: Is there a connection between the number of buffer frames and the number of frames to be generated? How about the connection with the number of condition images/frames? \n\nThe writing could be improved by making some of the claims more rigorous. See below points\n\n(-) I feel that the claim of \"requires no architectural changes\" is a bit off, as TIC-FT indeed requires additional LoRA parameters. The essential difference between TIC-FT and baselines, such as ControlNet, is probably the fact that the former introduces a much smaller number of new parameters, and these new modules are more general and not specifically designed for a certain type of visual condition. \n\n(-) I am unsure if the setting falls within the definition of in-context learning. Usually, the in-context demonstrations consist of several examples belonging to the same tasks. The V2V setting seems fine, but the image-to-video setting doesn't really sound like in-context learning because the input image is merely a condition used for generating the video instead of being a \"demonstration\" that falls into the same modality or task type as the output video frames.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns. The paper seems interesting and should be accepted from my perspective. Please incorporate the rebuttal content into the next version accordingly.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and detailed feedback. Your insightful comments have helped us identify key areas to clarify and improve. We have carefully addressed each concern below.\n***\n> (-) Small-scale evaluation set. Each task is evaluated on only 20 condition-target pairs, which may not be very representative of showing the whole picture of the performance superiority. Are there larger scale benchmarks or more standard ones that can be used for evaluation?\n\nWe would like to first clarify that the 20 condition-target pairs were **used for training** to demonstrate our method's few-shot efficiency. The quantitative evaluation reported in our initial submission was conducted using standard and diverse benchmarks **including VBench [1], GPT-4o, and Perceptual similarity scores**, with 100 unseen video samples per task. We did mention it in the manuscript, but will improve the camera-ready version for greater clarity!\n    \nTo further strengthen our evaluation and address the reviewer’s concern, we have benchmarked our method on Video-Bench [2], a newly proposed large-scale benchmark recently presented at CVPR 2025.  Video-Bench offers human-aligned evaluation metrics powered by multimodal LLMs, which demonstrate stronger correlation with human judgment compared to previous benchmarks such as VBench.\n    \nThe results on Video-Bench are presented below:\n    \n**Table: Evaluation on Video-Bench**\n    \n| method | aesthetic quality | imaging quality | motion effects | temporal consistency |\n| --- | --- | --- | --- | --- |\n| ControlNet | 3.60 | 3.58 | 2.70 | 4.75 |\n| Fun-pose | 3.94 | 4.42 | 3.05 | 4.82 |\n| TIC-FT-Replace | 3.92 | 4.29 | 3.07 | 4.80 |\n| TIC-FT (w/o Buffer) | 3.93 | 4.15 | 3.06 | 4.82 |\n| **TIC-FT** | **3.96** | **4.52** | **3.09** | **4.83** |\n\n    \nAs the tables show, our **TIC-FT** achieves competitive or state-of-the-art performance across most metrics on this large-scale, human-aligned benchmark (notably with less compute and better condition reflectance).\n    \nWe believe these additional results on a large-scale, standardized benchmark effectively demonstrate the generalizability and robust performance of our method. \n\n[1] Huang et al., Vbench: Comprehensive benchmark suite for video generative models\n\n[2] H. Han et al., “Video‑Bench: Human‑Aligned Video Generation Benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.\n***\n> (-) Metrics: One drawback I could imagine from the temporal concatenation is the strong reliance of every generation frame on the input conditions (or condition shortcuts), which may lead to less dynamics in the output videos. VBench provides metrics beyond just consistency (including dynamic degree), but they are not reported in the tables.\n\nWe thank the reviewer for raising this important point. We provide the VBench Dynamic Degree metrics in the tables below for both I2V and V2V tasks, and show that our method substantially improve dynamic degree compared to other methods. While pixel-aligned conditioning methods such as ControlNet are known to degrade dynamic degree, temporal concatenation method (Ours) is less susceptible to this issue. Additionally, our fine-tuning approach leverages the inherent data distribution more effectively than alternative methods, limiting such degradation.\n\n**Table: I2V Evaluation on VBench Dynamic Degree**\n| Method               | Dynamic Degree |\n|----------------------|----------------|\n| ControlNet           | 0.26           |\n| Fun-pose             | 0.64           |\n| **TIC-FT**               | **0.73**           |\n| *Training Data*        | *0.83*           |\n\n**Table: V2V Evaluation on VBench Dynamic Degree**\n| Method               | Dynamic Degree |\n|----------------------|----------------|\n| ControlNet           | 0.21           |\n| Fun-pose             | 0.69           |\n| **TIC-FT**           | **0.71**           |\n| *Training Data*        | *0.77*           |\n\nWe further report the Dynamic Degree Discrepancy, computed as the difference between the Dynamic Degree of the Training Data and that of each generated video. (The best model should have the smallest absolute value of the discrepancy). The results show our method closely matches dynamic degree of the training data.\n\n**Table: I2V Evaluation on VBench Dynamic Degree Discrepancy**\n\n| method | Dynamic Degree Discrepancy |\n| --- | --- |\n| ControlNet | 0.57  |\n| Fun-pose | 0.19  |\n| SIC-FT-Replace | 0.22 |\n| TIC-FT-Replace | **0.10** |\n| TIC-FT (w/o Buffer) | 0.14 |\n| **TIC-FT** | **0.10** |\n\n**Table: V2V Evaluation on VBench Dynamic Degree Discrepancy**\n\n| method | Dynamic Degree Discrepancy |\n| --- | --- |\n| ControlNet | 0.56 |\n| Fun-pose | 0.08 |\n| SIC-FT-Replace | -0.10 |\n| TIC-FT-Replace | -0.19 |\n| TIC-FT (w/o Buffer) | -0.17 |\n| **TIC-FT** | **0.06** |\n\n\n> (-) Ablation study on the number of buffer frames.\n\nWe have performed additional quantitative analysis to derive insights on how the number of condition and buffer frames impacts the final output.\n    \n**1. Impact of the Number of Condition Frames**\n\nWe found that increasing the number of condition frames in I2V tasks lead to a marginal but consistent improvement in reflecting the condition's visual details. As shown in the table below, most of the similarity metrics show a slight upward trend with more condition frames.\n    \n| # Condition | CLIP-I | LPIPS (↓) | SSIM | DINO |\n| --- | --- | --- | --- | --- |\n| 1 | 0.8329 | 0.7493 | 0.5917 | 0.5530 |\n| 3 | 0.8332 | 0.7390 | 0.5918 | 0.5531 |\n| 6 | 0.8371 | 0.7360 | 0.6078 | 0.5606 |\n| 9 | **0.8396** | **0.7346** | **0.6083** | **0.5643** |\n\n**2. Impact of the Number of Buffer Frames**\n\nWe observed that increasing the number of buffer frames enhances video dynamics in I2V tasks. This occurs because additional buffer frames create a temporal gap between the static input and target videos, leading to disentanglement between the two.\n       \n| # Buffer | Dynamic Degree |\n| --- | --- |\n| 3 | 0.73 |\n| 6 | 0.77 |\n| 9 | **0.82** |\n\nWe also observed that, as the model further disentangles the target frames from the condition frame, the influence of the condition on the generated target video tends to diminish, which is intuitive.\n\n| # Buffer | CLIP-I   | LPIPS  | SSIM   | DINO   |\n|----------|--------|--------|--------|--------|\n| 3        |**0.7812** | **0.6112** | **0.4130** | **0.4259** |\n| 6        |0.7695 | 0.6121 | 0.4127 | 0.4170 |\n| 9        | 0.7544 | 0.6232 | 0.3952 | 0.4152 |\n\nIn conclusion, increasing the number of condition frames results in stronger reflection of conditioning on the generated frames, though the improvement is marginal. Additionally, more buffer frames lead to disentanglement between condition and target frames. In I2V generation, since condition frames are static, additional buffer frames increase the dynamics of the generated video, while at the cost of reduced condition reflection.\n\nGiven that the influence of condition frame count is marginal, it is optimal to use the minimum number of condition frames, as additional frames require more memory and computation. We found that 1 condition frame is optimal for I2V tasks. For buffer frames, we determined that 3 buffer frames provide the optimal balance between memory usage, condition reflection, and dynamic degree.\n\n***\n> (-) I feel that the claim of \"requires no architectural changes\" is a bit off, as TIC-FT indeed requires additional LoRA parameters.\n\nWe agree with the reviewer that our original claim of \"no architectural changes\" is imprecise, as TIC-FT does introduce new LoRA parameters. We thank the reviewer for this point and will revise our manuscript with a more accurate description.\n\nOur original intention was to highlight that, while our method introduces new trainable LoRA parameters, it does not alter the model's high-level architecture or its computational graph. (LoRA modifies the behavior within existing operational blocks rather than adding new, parallel network branches.)\n\n(As the reviewer suggested) This preservation of the high-level architecture is what makes our TIC-FT framework inherently modality-agnostic and general-purpose, as no specialized modules are needed to handle different model architectures.\n\n\n***\n> (-) I am unsure if the setting falls within the definition of in-context learning.\n\nTo clarify, our paper does not claim to implement in-context learning (ICL) in the traditional sense as defined in the context of LLMs.\nInstead, we define our work as \"in-context finetuning\", a related but distinct paradigm adapted for diffusion models. It is an efficient fine-tuning technique inspired from ICL, but involves finetuning, that leverages spatial or—in our case—temporal concatenation to facilitate information sharing across a sequence. Our novelty lies in extending spatial In-Context finetuning used for image generation to video generation, with Temporal In-Context Fine-Tuning (TIC-FT), along with techniques like adding buffer frames.\n***\nWe sincerely appreciate your time and constructive feedback. We believe the additional results, clarifications, and analyses presented above effectively address your concerns and look forward to further discussion."}, {"review_id": "BrVJDCMQcf", "reviewer": "Reviewer_ngnG", "summary": "- Many existing methods for conditional video generation rely on large training datasets and introduce additional architectural modifications. To address these limitations, this paper proposes Temporal In-Context Fine-Tuning (TIC-FT), an efficient and versatile approach for adapting pretrained video diffusion models to diverse conditional generation tasks.\n- Extensive experiments show that TIC-FT outperforms existing baselines in both condition fidelity and visual quality.", "questions": "Have the pretrained video diffusion backbones been trained on any conditional generation tasks (i.e., sth like the downstream tasks of this paper) except for general video generation? If the former, how does this influence the effectiveness of TIC-FT?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper introduces a simple yet effective method for adapting pretrained video diffusion models to diverse conditional generation tasks.\n- TIC-FT shows strong empirical performance with small training data, offering a highly efficient fine-tuning strategy.\n- The proposed method avoids modifying the model architecture, making it easy for adoption.\n\n**Weaknesses**\n- L168–169 describe the inference procedure as “iteratively identifying the frames currently at the maximal noise level t and applying the video diffusion sampler exclusively to those frames”. It seems like the buffer frames need to be denoised iteratively, possibly one by one, because they have different noise levels? This seems to increase the inference time complexity? What's the inference time complexity of TIC-FT, and how does it compare to baseline methods?\n- It is unclear to me how the number of buffer frames is determined during training and inference.\n- It is unclear whether, in cases with multiple conditional frames, does the model only use the first conditional frame to construct the buffer frames? if so, how does the model use multiple conditional frames when constructing the buffer frames?", "was_revised": true, "final_justification": "I have read the reviews from others as well as the authors' responses, which addressed most of my concerns. I think the paper is good to be accepted to NeurIPS.", "rebuttal": "We sincerely thank the reviewer for the thoughtful evaluation and for highlighting both the strengths and the potential areas for clarification in our work. We address each of the concerns in detail and provide clarifications regarding the inference procedure, buffer frame configuration, and the handling of multiple conditional frames below.\n***\n> L168–169 describe the inference procedure as “iteratively identifying the frames currently at the maximal noise level t and applying the video diffusion sampler exclusively to those frames”. It seems like the buffer frames need to be denoised iteratively, possibly one by one, because they have different noise levels? This seems to increase the inference time complexity? What's the inference time complexity of TIC-FT, and how does it compare to baseline methods?\n\nWe would like to clarify that our TIC-FT denoising process remains **fully parallel across all frames** and does not introduce additional iterative denoising steps, thereby incurring no significant overhead. To provide a clearer picture of the denoising procedure, we offer the following step-by-step visualization.\n\n\n- **C:** Condition Frame (a clean frame at `t=0`)\n- **B1, B2, B3:** Buffer Frames with initial target noise levels (e.g., `t=25, t=50, t=75`)\n- **T:** Target Frames to be generated (starting at max noise, e.g., `t=100`)\n\n---\n\n### **Denoising Process of TIC-FT (Ours)**\n\nThis process follows the formal rules described in **Algorithm 1** and **Equation 7** of the paper.\n\n**Frame Structure:** `[ C | B1 | B2 | B3 | T | T | ... ]`\n\n### `t_global = 100` (Start)\n\n- **Frame Timesteps:** `[0, 25, 50, 75, 100, 100, ...]`\n- **Frames Being Denoised:** **Target Frames (T)**\n\n### `t_global = 99`\n\n- **Frame Timesteps:** `[0, 25, 50, 75, 99, 99, ...]`\n- **Frames Being Denoised:** **Target Frames (T)**\n\n... (This continues until `t_global` = 76) ...\n\n### `t_global = 75` \n\n- **Frame Timesteps:** `[0, 25, 50, 75, 75, 75, ...]`\n- **Frames Being Denoised:** **B3** + **Target Frames (T)**\n\n### `t_global = 74`\n\n- **Frame Timesteps:** `[0, 25, 50, 74, 74, 74, ...]`\n- **Frames Being Denoised:** **B3** + **Target Frames (T)**\n\n... (This continues until `t_global` = 51) ...\n\n### `t_global = 50`\n\n- **Frame Timesteps:** `[0, 25, 50, 50, 50, 50, ...]`\n- **Frames Being Denoised:** **B2** + **B3** + **Target Frames (T)**\n\n### `t_global = 49`\n\n- **Frame Timesteps:** `[0, 25, 49, 49, 49, 49, ...]`\n- **Frames Being Denoised:** **B2** + **B3** + **Target Frames (T)**\n\n... (This continues until `t_global` = 26) ...\n\n### `t_global = 25` \n\n- **Frame Timesteps:** `[0, 25, 25, 25, 25, 25, ...]`\n- **Frames Being Denoised:** **B1** + **B2** + **B3** + **Target Frames (T)**\n\n### `t_global = 24`\n\n- **Frame Timesteps:** `[0, 24, 24, 24, 24, 24, ...]`\n- **Frames Being Denoised:** **B1** + **B2** + **B3** + **Target Frames (T)**\n\n... (This process continues down to `t_global` = 1) ...\n\n### `t_global = 0` (End)\n\n- **Frame Timesteps:** `[0, 0, 0, 0, 0, 0, ...]`\n- **Result:** Final denoised sequence.\n\n---\n\nAs the visualizations illustrate, our method **requires no additional denoising steps**.  The only extra computation comes from additional condition and buffer frames, which is negligible when generating longer videos. For further clarity, visual examples of denoising are provided in Appendix Figures 8–12.\n\n***\n> It is unclear to me how the number of buffer frames is determined during training and inference.\n\nThe choice of using three (latent) buffer frames as the default in our main experiments was based on a careful balance between generation quality and computational overhead, informed by the ablation study presented in our appendix (Figure 7).\n\nOur ablation study on buffer frame count revealed the following trade-offs:\n\nToo few buffer frames (e.g., one): Results in noisier transitions due to abrupt changes between condition and target frames, causing artifacts and degraded quality in target frames.\n\nToo many buffer frames: Unnecessarily increases sequence length, leading to higher computational and memory costs, without further gain in generation quality. Additionally, excessive temporal gaps between target frames may reduce condition adherence.\n    \nWe also observed that increasing the number of buffer frames enhances video dynamics in I2V tasks. This occurs because additional buffer frames create a temporal gap between the static input and target videos, leading to disentanglement between the two.\n       \n| # Buffer | Dynamic Degree |\n| --- | --- |\n| 3 | 0.73 |\n| 6 | 0.77 |\n| 9 | **0.82** |\n\nIntuitively, as the model further disentangles the target frames from the condition frame, the influence of the condition on the generated target video tends to diminish:\n\n| # Buffer | CLIP-I   | LPIPS  | SSIM   | DINO   |\n|----------|--------|--------|--------|--------|\n| 3        |**0.7812** | **0.6112** | **0.4130** | **0.4259** |\n| 6        |0.7695 | 0.6121 | 0.4127 | 0.4170 |\n| 9        | 0.7544 | 0.6232 | 0.3952 | 0.4152 |\n\nIn conclusion, more buffer frames lead to disentanglement between condition and target frames. In I2V generation, since condition frames are static, additional buffer frames increase the dynamics of the generated video, while at the cost of reduced condition reflection.\nWe determined that 3 buffer frames provide the optimal balance between memory usage, condition reflection, and dynamic degree.\n\n***\n> It is unclear whether, in cases with multiple conditional frames, does the model only use the first conditional frame to construct the buffer frames? if so, how does the model use multiple conditional frames when constructing the buffer frames?\n\nWhen multiple conditions are used, the buffer frames are constructed using only the last conditional frame. This design ensures a smooth transition from the condition sequence to the target frames.\nThis process is illustrated in Figure 12 of the appendix.\n\n\n\n***\nOnce again, we appreciate the reviewer’s constructive feedback and critical observations. Your feedback will be used to help us refine our explanations. We look forward to further discussions."}, {"review_id": "FIUs7Qhk99", "reviewer": "Reviewer_9QKt", "summary": "This paper introduces Temporal In-Context Fine-Tuning (TIC-FT), a parameter-efficient method for adapting pretrained video diffusion models to diverse conditional generation tasks. Key innovations include:\n\n1. Temporal Context Construction:\n   - Concatenates condition frames (e.g., input images/videos) and target frames along the temporal axis\n   - Inserts noisy buffer frames with progressive noise levels between condition-target pairs to align with pretrained diffusion dynamics\n2. Training-Free Adaptation:\n   - Requires no architectural changes or external encoders\n   - Achieves strong performance with 10–30 training samples\n3. Versatile Control:\n   - Validated on image-to-video and video-to-video tasks using billion-scale models (CogVideoX-5B, Wan-14B)\n   - Maintains high condition fidelity and visual quality while reducing training/inference costs\n\nTIC-FT establishes temporal concatenation as an effective paradigm for lightweight control of video diffusion models.", "questions": "1. Baseline Relevance\n   - Q: Why omit video-specific control methods (e.g., ControlVideo, VidControl)? Does TIC-FT outperform them under identical compute?\n2. Temporal Artifact Analysis\n   - Q: Buffer frames reduce distribution shift but may cause motion flicker. Did you quantify temporal instability (e.g., FLD score)?\n3. Long-Sequence Scalability\n   - Q: The 10-second limit stems from memory constraints. Could sequence chunking or gradient checkpointing mitigate this?\n4. Real-World Generalization\n   - Q: Synthetic datasets (GPT-4o/Sora) may bias results. Does performance hold on complex real-world motions?", "limitations": "YES.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- Quality:\n  Rigorous validation on billion-scale models across 6+ tasks (style transfer/motion transfer/etc.). Few-shot efficiency is empirically verified.\n- Originality:\n  Novel temporal concatenation + noisy buffer frames mechanism elegantly aligns pretrained/fine-tuning distributions. Unifies diverse conditioning tasks without architectural changes.\n- Significance:\n  Democratizes control of large video models for low-resource users. Memory-efficient design.\n- Clarity:\n  Exceptional methodological description; clean task taxonomy.\n\nWeaknesses:\n\n- Quality:\n  - Baseline mismatch: ControlNet (image-centric) and Fun-pose (pre-finetuned) are suboptimal comparators. Missing video-specific adapters (e.g., TADA, VidControl).\n  - Limited temporal metrics: Relies on FVD/CLIP – neglects *motion consistency* (e.g., optical flow variance).\n- Significance:\n  - 10-second input limit restricts long-video applications.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thorough and encouraging evaluation of our work. We appreciate your recognition of our method’s originality, clarity, and practical significance, as well as your valuable suggestions regarding baselines, temporal metrics, and long-video applicability. We address the reviewer’s concerns point by point, specifically focusing on (1) baseline selection, (2) the absence of fine-grained motion metrics, and (3) the restriction related to input sequence length.\n***\n> Q: Why omit video-specific control methods (e.g., ControlVideo, VidControl)? Does TIC-FT outperform them under identical compute?\n\nWe want to clarify that ControlNet-based approached mentioned in the paper, **is the same method as ControlVideo**, and our method does outperform the baseline as presented in the paper, under less compute. \n(The ControlNet we used in our experiments is indeed a video-centric version adapted for a Video DiT architecture, functionally identical to the ControlVideo setup you suggested.)\n\nCurrent leading methods for conditional video generation primarily follow two main paradigms. The first is the ControlNet-based approach, where a parallel (and sometimes trainable) encoder branch is added to the main model to inject spatial or temporal conditions. The second is the concatenation-based approach, where conditioning information, such as a reference frame's latent representation, is directly concatenated with the input noise tensor during the diffusion process and often requires finetuning the whole model.\n\nTo ensure our comparison was representative of the current state-of-the-art, we selected baselines that reflect these two dominant strategies. ControlNet (ControlVideo) was chosen as a representative example of the first paradigm, while Fun-pose, a popular and efficient variant of the second category, was included as a strong representative.\n\nWe acknowledge that additional methods exist, but we did not identify any work that significantly deviates from these two paradigms or demonstrates substantially superior performance compared to these established approaches. So we believe this selection provides a fair and robust comparison against the primary methods currently employed in the field. However, we are open to discussion and would be happy to include additional baselines if you have specific suggestions!\n\n(We were unable to find TADA model you mentioned despite an extensive search. It would be grateful if you could provide a reference, and we would be happy to perform a comparison.)\n\nYou can also check out GPU Memory Usage Comparison:\n\n| Method | Training VRAM | Inference VRAM |\n| --- | --- | --- |\n| **Ours (TIC-FT)** | **30 GB** | **25 GB** |\n| ControlNet | 60 GB | 43 GB |\n| Fun-pose | 75 GB | 28 GB |\n\nAs the table demonstrates, our TIC-FT method operates at a significantly lower computational cost for both training and inference. Despite this high efficiency, TIC-FT achieves superior performance compared to these baselines, as detailed in Table 1 and Table 2 of our main paper.\n***\n> Q: Buffer frames reduce distribution shift but may cause motion flicker. Did you quantify temporal instability (e.g., FLD score)?\n\nWe appreciate the reviewer's focus on detailed temporal evaluation. As a quantitative measure, our main paper's Table 1 and Table 2 already include several VBench metrics that evaluate temporal qualities, such as `subject consistency`, `background consistency`, and `motion smoothness`.\n    \nIn addition to these metrics, we believe the qualitative results provide the most direct evidence of temporal stability. We kindly ask the reviewer to refer to our supplementary video examples. These videos demonstrate that our method produces temporally clean and smooth results, without introducing the flicker or instability the reviewer was concerned about.\n    \nTo more directly address the concern about motion and stability, we have conducted further evaluations using the new Video-Bench benchmark[1] (accepted at CVPR 2025), which provides human-aligned scores for `motion effects` and `temporal consistency`.\n    \nThe results on Video-Bench are presented below:\n    \n**Table: Evaluation on Video-Bench**\n    \n| method | aesthetic quality | imaging quality | motion effects | temporal consistency |\n| --- | --- | --- | --- | --- |\n| ControlNet | 3.60 | 3.58 | 2.70 | 4.75 |\n| Fun-pose | 3.94 | 4.42 | 3.05 | 4.82 |\n| TIC-FT-Replace | 3.92 | 4.29 | 3.07 | 4.80 |\n| TIC-FT (w/o Buffer) | 3.93 | 4.15 | 3.06 | 4.82 |\n| **TIC-FT** | **3.96** | **4.52** | **3.09** | **4.83** |\n    \nResults demonstrate the benefit of our design, and `temporal consistency` shows our method is robust from temporal artifacts.\n\n[1] H. Han et al., “Video‑Bench: Human‑Aligned Video Generation Benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.\n***\n> Q: The 10-second limit stems from memory constraints. Could sequence chunking or gradient checkpointing mitigate this?\n\nWe believe the limitation regarding long video generation is not specific to our method, but rather a common challenge and ongoing research area for all video generation models.\nRecent works have focused on longer video generation by employing more efficient attention mechanisms or autoregressive diffusion approaches. Gradient chunking, as you mentioned, would indeed help since the primary bottleneck is memory usage during chunk generation.\nImproving conditioning for longer videos represents an interesting direction for future research.\n\n***\n> Q: Synthetic datasets (GPT-4o/Sora) may bias results. Does performance hold on complex real-world motions?\n\nWe appreciate the reviewer's concern for evaluation on real-world data. To clarify, **our evaluation was not limited to synthetic data**. For the action transfer task, we curated videos directly from the SSv2 (Something-Something V2) dataset, which is composed entirely of real-world videos of humans performing various actions.\n    \nOur method demonstrated strong performance on this real-world dataset. The qualitative results for this task, showcasing TIC-FT's effectiveness on complex, real-world motion, can be found in Figure 10 in the Appendix.\nIf you are aware of any other real-world experiments that would enhance our evaluation, we would welcome your suggestions!\n***\nOnce again, we are grateful for your thoughtful and balanced feedback. Your comments have helped us further strengthen our work,  and we remain open to additional experiments and further discussion."}, {"review_id": "SFOspS7Two", "reviewer": "Reviewer_juYd", "summary": "This paper presents a novel paradigm for controllable video generation. Similar to in-context learning, this method concatenates conditionaframes to target frames and encourages the model to learn the in-context pattern in conditional frames. This method also introduces a buffer frame between target and condition frames to preserve the pretrained model's capability.", "questions": "See weakness.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Stength:\n\n1) The proposed method is novel and highly extensible. Compared to previous approaches such as ControlNet, it enables multiple applications through simple concatenation without introducing additional parameters.\n\n2) The paper's writing is clear. The method is straightforward and easy to follow.\n\n3) The experiments are sufficient and the performance is good.\n\nWeakness:\n\n(1) Positional Encoding. After concatenation of conditional , buffer and target frames, the total number of frames likely exceeds the original video length supported by the pretrained model. If the video length during fine-tuning differs from that of the pretrained model, it is unclear whether this would cause issues. It would be helpful to clarify how the positional encoding is handled for the longer frames.\n\n(2) Scaling. In this paper, each application requires a set of training data for fine-tuning. Is it possible to jointly fine-tuning on multiple tasks and  achieve zero-shot performance at inference time? \n\n(3) Precise Control. If fine-grained control over each frame (e.g., conditioning each frame on a depth map) is needed, is it feasible under the proposed in-context learning framework?", "was_revised": true, "final_justification": "The authors have addressed my concern in positional encoding, scaling and precise control. This paper is good to accept and I will keep my scores.", "rebuttal": "We sincerely thank the reviewer for the encouraging and thoughtful feedback. We appreciate your recognition of the clarity, extensibility, and effectiveness of our method. Below, we address your questions regarding positional encoding for longer sequences, the potential for multi-task joint fine-tuning, and the feasibility of fine-grained frame-level control under our in-context learning framework.\n***\n> (1) Positional Encoding. After concatenation of conditional , buffer and target frames, the total number of frames likely exceeds the original video length supported by the pretrained model. If the video length during fine-tuning differs from that of the pretrained model, it is unclear whether this would cause issues. It would be helpful to clarify how the positional encoding is handled for the longer frames.\n\nWe thank the reviewer for this important technical question.\nThis is generally not a concern because recent pretrained video models are trained on videos of diverse lengths and can generate longer sequences using efficient attention mechanisms and/or autoregressive approaches. Our method adds only a few buffer and condition frames, resulting in relatively small computational overhead.\n\nFurthermore, pretrained video models leverage Rotary Positional Encoding (RoPE), which effectively generalizes to sequence lengths beyond those seen during training. Since RoPE relies on relative rather than absolute positional information, it handles extended contexts without significant performance degradation.\n\nImportantly, within the scope of our experiments, the total concatenated sequence length (condition + buffer + target frames) remained well within the normal sequence lengths supported by the pretrained backbones, so we did not encounter any issues. Nevertheless, as long video generation remains a challenging problem with rapidly evolving research, we believe there are opportunities for further exploration in conditioning longer video generation.\n\n***\n> (2) Scaling. In this paper, each application requires a set of training data for fine-tuning. Is it possible to jointly fine-tuning on multiple tasks and achieve zero-shot performance at inference time?\n\nWhile not directly within the scope of our current research—which focuses on efficient fine-tuning with minimal data and computation—we believe this is definitely feasible.\nIn the image domain, the Kontext [1] model demonstrates how in-context fine-tuning can build a general image editing model capable of zero-shot performance. \n\nOur temporal in-context fine-tuning method works effectively because it leverages the pretrained distribution of video models well, suggesting it could quickly adapt to diverse samples when trained on large-scale diverse datasets.\nOne key difference would be that while our current approach requires paired videos (source/condition and target), scaling to multi-task zero-shot performance would likely require triplets of source, target, and text instructions to incorporate diversity and enable controllable generation.\nThis represents an exciting direction for future research.\n\n[1] Batifol, Stephen, et al. \"FLUX. 1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space.\" arXiv e-prints (2025): arXiv-2506.\n***\n> (3) Precise Control. If fine-grained control over each frame (e.g., conditioning each frame on a depth map) is needed, is it feasible under the proposed in-context learning framework?\n\nWe appreciate the reviewer's question about the scope of controllability.\nThe TIC-FT framework is conceptually flexible enough to support such scenarios and would fall under the video-to-video (V2V) category presented in our paper. For example, one could structure the input as temporally arranged sequences of `[condition frames (depth maps), target frames]` or `[condition frames (depth maps), buffer frames, target frames]`.\n\nHowever, we should clarify that TIC-FT's primary design goal is not for tasks requiring dense, pixel-aligned conditional inputs like per-frame depth control. For such applications, more direct approaches—such as channel-wise concatenation of conditions or ControlNet-style methods where pixel-aligned condition features are directly added to latent frames—would likely provide more straightforward learning pathways for the model.\n\n***\nOnce again, we are grateful for your valuable feedback. Your questions have prompted us to clarify important aspects of our design and explore directions for future extension. We hope our detailed responses have addressed your concerns and look forward to further discussions."}]

Author rebuttal:

Meta-review: Summary

This paper introduces Temporal In-Context Fine-Tuning (TIC-FT), a method for adapting pretrained video diffusion models to diverse conditional video generation tasks (e.g., image-to-video, video-to-video). The key innovation is to concatenate condition and target frames along the temporal axis, inserting buffer frames with progressively increasing noise levels to smooth the transition and align with the pretrained model’s temporal dynamics. TIC-FT requires no architectural changes (beyond LoRA parameters), is highly parameter- and compute-efficient, and achieves strong performance with as few as 10–30 training samples. The method is validated on large-scale models (CogVideoX-5B, Wan-14B) and a range of tasks, outperforming established baselines in condition fidelity, visual quality, and efficiency.

Strengths
- Novelty & Simplicity: The temporal concatenation and buffer frame mechanism is a creative, elegant solution for conditional video generation, enabling versatile control without architectural modifications.
- Efficiency: TIC-FT is highly parameter- and memory-efficient, requiring minimal training data and compute, making it accessible for low-resource users.
- Empirical Validation: Extensive experiments on multiple tasks and large-scale models demonstrate superior performance over strong baselines (ControlNet/ControlVideo, Fun-pose), including on human-aligned benchmarks (VBench).
- Clarity: The paper is well-written, with clear methodological exposition and thorough ablation studies.
- Extensibility: The approach generalizes across diverse conditional tasks and is compatible with various pretrained backbones.

Weaknesses
- Baseline Coverage: While ControlNet and Fun-pose are relevant baselines, some reviewers noted the absence of comparisons to other recent video-specific adapters (e.g., VidControl). Authors clarified ControlNet is functionally equivalent to ControlVideo
- Temporal Metrics: Initial evaluation focused on FVD/CLIP; reviewers requested more fine-grained motion metrics (e.g., dynamic degree, temporal consistency). Authors addressed this with additional results on VBench, showing competitive or SOTA performance.
- Long-Sequence Scalability: The method is currently limited to ~10-second videos due to memory constraints. While this is a common limitation, future work on longer sequences would be valuable.
- Precise Frame-Level Control: TIC-FT is less suited for dense, pixel-aligned conditioning (e.g., per-frame depth maps) compared to ControlNet-style methods.
- Definition of "In-Context": Some reviewers questioned the use of "in-context learning" terminology, as the method is more akin to in-context fine-tuning than classical ICL.


Overall, the paper presents a technically solid and positively received by all reviewers. The method is simple and effective, with strong empirical results and broad applicability. Most reviewer concerns were addressed convincingly in the rebuttal, including expanded evaluation, clarification of baselines, and detailed ablation studies. ACs concur with reviewer consensus. Authors must incorporate reviewer feedback in the final paper.
Decision: Accept (poster)