ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Sampler
Authors: Serin Yang
Taesung Kwon
Jong Chul Ye
Venue: ICLR 2025 Poster (2025)
Keywords: Keyframe interpolation
Stable video diffusion
Bidirectional diffusion sampling
Primary area: generative models

Abstract: Recent progress in large-scale text-to-video (T2V) and image-to-video (I2V) diffusion models has greatly enhanced video generation, especially in terms of keyframe interpolation. However, current image-to-video diffusion models, while powerful in generating videos from a single conditioning frame, need adaptation for two-frame (start \& end) conditioned generation, which is essential for effective bounded interpolation. Unfortunately, existing approaches that fuse temporally forward and backward paths in parallel often suffer from off-manifold issues, leading to artifacts or requiring multiple iterative re-noising steps. In this work, we introduce a novel, bidirectional sampling strategy to address these off-manifold issues without requiring extensive re-noising or fine-tuning. Our method employs sequential sampling along both forward and backward paths, conditioned on the start and end frames, respectively, ensuring more coherent and on-manifold generation of intermediate frames. Additionally, we incorporate advanced guidance techniques, CFG++ and DDS, to further enhance the interpolation process. By integrating these, our method achieves state-of-the-art performance, efficiently generating high-quality, smooth videos between keyframes. On a single 3090 GPU, our method can interpolate 25 frames at 1024$\times$576 resolution in just 195 seconds, establishing it as a leading solution for keyframe interpolation.
Project page: https://vibidsampler.github.io/

Published as a conference paper at ICLR 2025
VIBIDSAMPLER: ENHANCING VIDEO INTERPOLA- TION USING BIDIRECTIONAL DIFFUSION SAMPLER
Serin Yang': Taesung 2won Jong Chul Ye'
Kim Jacchul Graduate School of AI, KAIST DDett. of Bio & Brain Engineering, KAIST (yangsr, star kwon, jong. ye ye) yeeakaist. ac.kr
(w)
Start Image
Keyframe Interpolation
End Image
(b)
111
Chrr
SNRP Clura
I1Q
CUPS
Chura
Figure 1: Keyframe interpolation results using our ViBiDSampler. (a) The images in the first and last rows are keyframes, and the intermediate frames are generated using ViBiDSampler. (b) A comparison of results with three baseline methods-FILMM, TRF, and Generative Inbetweening (GI) demonstrates that these baselines exhibit artifacts or unnatural appearances. In contrast, our method generates clear and realistic frames.
ABSTRACT
Recent progress in large-scale text-to-video (T2V) and image-to-video (I2V) dif- fusion models has greatly enhanced video generation, especially terms keyframe interpolation. However, current image-to-video diffusion models, while powerful in generating videos from single conditioning frame, need adaptation for two-frame (start & end) conditioned generation. which essential for effec- tive bounded interpolation. Unfortunately. existing approaches that fuse tempo- rally forward and backward paths in parallel often suffer from off manifold is- sues, leading to artifacts requiring multiple iterative re-noising steps. In this work, introduce novel, bidirectional sampling strategy address these off- manifold issues without requiring extensive re-noising or fine-tuning. Our method
"Equal contribution.

---

Published as a conference paper at ICLR 2025
employs sequential sampling along both forward and backward paths, conditioned on the start and end frames, respectively, ensuring more coherent and on-manifold generation of intermediate frames. Additionally, we incorporate advanced guid- ance techniques, CFG++ and DDS, to further enhance the interpolation process, By integrating these, our method achieves state-of-the- art performance, efficiently generating high-quality, smooth videos between keyframes. On single 3090 GPU, our method can interpolate 25 frames at x024x576 resolution in just 195 seconds, establishing it as a leading solution for keyframe interpolation. Project page:
www.lerrrgituuutt.e github.
INTRODUCTION
Recent advancements in large-scale text-to-video (T2V) and image-to-video (12V) diffusion mod- els (Blattmann et al.] 2023a b Wu et al.) [023 Xing et al. 2023 Bar-Tal et al. 2024 have made it possible to generate high-quality videos that closely match given text or image conditions. Vari-
ous efforts have been made leverage the powerful generative capabilities of these video diffusion models, especially in the context of keyframe interpolation, to improve perceptual quality signifi- cantly. Specifically, diffusion-
keyframe interpolation (Voleti al., 2022 Danier et al.) [024] Huang et al. 2024, Feng et al., 2024 Wang et al. 2024) focuses on generating intermediate frames between two keyframes, aiming to create smooth and natural motion dynamics while preserving the keyframes' visual fidelity and appearance. Image-to-videe diffusion models are particularly well- suited for this task because they are designed to maintain the visual quality and consistency of the initial conditioning frame.
While image-to-vider diffusion models are designed for start-frame conditioned video generation, they need to be adapted for start and end frame conditioned video generation for keyframe inter- polation. One line of works (Feng et al., 2024] Wang et al.] 2024) addresses this issue by intro- ducing new sampling strategy that fuses the intermediate samples of the temporally forward path, conditioned on the start frame, and the temporally backward path, conditioned on the end frame. The fusing strategy ensures smooth and coherent frame generation in-between two keyframes using to-geo diffusion models in a training-free (Feng et al. 2024) or a lightweight fine-tuning manner (Wang al. 2024).
In the geometric view of diffusion models (Chung et al. 2022 the sampling process is typically described as iterative transitions M, + -M,,, T.... to the clean manifold Mo. From this perspective, fusing two intermediate sample points through linear interpolation on noisy manifold can lead to an undesirable off-manifold issue, where the generated samples deviate from the learned data distribution. TRF (Feng et al. (024) reported that this fusion strategy often results in undesired artifacts. To address these discrepancies, they apply multiple rounds of re-noising and denoising to the fused samples, which may help correct the off- manifold deviations.
moving from the noisy manifold Mr
Unlike the prior works, here introduce simple yet effective sampling strategy to address off- manifold issues. Specifically, at timestep t. we first denoise H, to obtain x;-1 along the tempo- rally forward path, conditioned the start frame (Tuur). Then, re-noise 2; back to æ, using stochastic noise. After that, we denoise a to get - along the temporally backward path, condi- tioned on the end frame (dena), where the notation indicates that the sample has been flipped along the time dimension. Unlike the fusing strategy, which computes two conditioned outputs in parallel and then fuses them, our bidirectional diffusion sampling strategy samples two conditioned outputs sequentially, which mitigates the offifoodfold issue.
Furthermore, we incorporate advanced on-manifold guidance techniques produce more reliable interpolation results. First, we employ the recently proposed CFG++ (Chung et al. (2024). which ad- dresses the off- off-manifold issues inherent Classifier-Free Guidance (CFG) (Ho Salimans (021). Second, we incorporate DDS guidance (Chung et al., 2023) to ensure proper alignment of the last frame of the generated samples with the given frames, as the ground-truth start and end frames are already provided. By combining bidirectional sampling with these guidance techniques, our method achieves stable, oftte-o-theeaal keyframe interpolation performance without requiring fine-tuning or multiple re-noising steps. Thanks to its efficient sampling strategy, method can interpolate be- tween two keyframes to generate a 25-frame video at 1024x576 resolution in just 195 seconds on a single 3090 GPU. Since our method is designed for high-quality and vivid video keyframe interpola-
2

---

Published as a conference paper at ICLR 2025
tion using bidirectional diffusion sampling, refer to it as Video Interpolation using Bldirectional Diffusion (ViBiD) Sampler.
RELATED WORKS
Video interpolation. Video interpolation is a task that generates the intermediate frames based on two bounding frames. Conventional interpolation methods have utilized convolutional neural networks (Kong et al.] 2022 Li al. 2023 Lu et al.) 2022 Huang et al. 2022 Zhang et al. 2023b Reda et al. 2019). which are typically trained in supervised manner to estimate the optical flows for synthesizing an intermediate frame. However, they primarily focus on minimizing I,₁ or Le distances between the output and target frames, emphasizing high PSNR values at the expense of perceptual quality. Furthermore, the train datasets generally consist of high frame rate videos, limiting the model's ' ability to learn extreme motion effectively.
ceruuiionbbased methods have been pro- posed (Danier et al. [2024 Huang al.) 2024; Voleti et al., 2022) to leverage the generative priors of diffusion models to produce high-quality perceptual intermediate frames. Although these meth- ods demonstrate improved perceptual performance, they still struggle with interpolating frames that contain significant motion. However, video keyframe interpolation methods that build on the robust performance of video diffusion models have been more successful in handling ambiguous and non- linear motion (Xing et al. 2023 Jain et al. (2024). largely due to the incorporation of the temporal
Diffusion-based methods and time reversal sampling.
attention layers in these models Blattmann et al.
2023a Ho et al. 2022 Chen et al. 2023 Zhang
et al. 2023a).
Recent advancements video diffusion models, particularly for image-to-video diffusion, have in- troduced new sampling techniques that leverage temporal and perceptual priors. These techniques reverse video frames parallel during inference and fuse bidirectional motion from both the tem- porally forward and backward directions. TRF (Feng al. (024) proposed method that combines forward and backward denoising processes, each conditioned on the start and end frames. Simi- larly, Generative Inbetweening [Wang al.) 2024) introduced method that extracts temporal self- attention maps and rotates them to sample reversed frames, enhancing video quality by fine-tuning diffusion models for reversed motion. However, these methods rely on fusion strategy that of- ten results in an off-manifold issue. Moreover, although methods such as multiple noise injections and model fine-tuning have been employed to address these challenges, they continue to exhibit off-manifold issues and substantially increase computational costs. In contrast, we introduce a sim- ple yet effective sampling strategy that eliminates the need for multiple noise injections or model fine-tuning.
3 VIDEO INTERPOLATION USING BIDIRECTIONAL DIFFUSION
Although our method applicable to general video diffusion models, we employ Stable Video Diffusion (SVD) (Blattmann et al. 2023a) as proof of concept in this paper. By introducing SVD, we aim to provide a clearer understanding of our approach. SVD is a latent video diffusion model employed in EDM-framework (Karras et al. (022) with micro-conditioning (Podell et al. 2023) on frame rate (fps). For the image-to-video model, SVD replaces text embeddings with the CLIP image embedding (Radford et al.] 202T) of the conditioning.
In EDM-framework, the denoiser D, computes the denoised estimate from the U-Net eg:
Do(2;; c) Cakip())æ; + (o) (Cin(a) )Rr; Croise (a), c)
(I)
where Cskip. Couts Cin. and Caoise are a-dependent preconditioning parameters and c is the condition. In practice, the denoiser Dg takes concatenated inputs [æ,, æx] return c-conditioned estimate and mull-conditioned estimate [mv.c. L, where Lu is then updated using w-scale classifier- guidance (CFG) (Ho& & Salimans) 2021):
$o.c < t0.0 w| øo.ol.
For sampling, SVD employs Euler-step to gradually denoise from Gaussian noise or to get Do!:
I:-1,c = 20.0 + Tt-1 (x: 20,c),
(3)
3

---

Published as a conference paper at ICLR 2025
Re-noise
XN
x=
Xt-L
Ar-1,
SVD
SVD
Facre
Saterk
Xp-1
Fatert
Caatt
Re-nolise
Time reversal
loed
Time reversal
reversal
Time reversal
x
t4,,0 cand
X Estort
X
SVD
SVD
(a) Time Reversal Fusion
(b) Bidirectional Sampling (Ours)
Figure 2: Comparison of denoising processes. (a) Time Reversal Fusion method and (b) bidirec- tional sampling (Ours).
where a is the denoised estimate from 2 and 0₂ is the discretized noise level for each timestep E [].
3.1 BIDIRECTIONAL SAMPLING
Prior approaches such as TRF (Feng al. 2024) and Generative Inbetweening (Wang et al. 2024) have employed a fusion strategy that linearly interpolates between samples from the temporally forward path, conditioned on the start frame (uum)) and the temporally backward path, conditioned on the end frame (Lend):
Or-1 (x: $0,cca))
- 1,0am 2.
(4)
0t
x 1,col = 1/1 aad
+
0c-1 (xi, #/o.oa):
(5)
Ft-1 11
(1 {)
where the notation indicates that the sample has been flipped along the time dimension, À denotes interpolation ratio, Cstar and Gend denotes the encoded latent condition of T and Tend, respectively. However, as the authors TRF (Feng et 2024) reported, the vanilla implementation of this fusion strategy suffers from random dynamics and unsmooth transitions. This occurs because linearly interpolating between two distinct sample points in the noisy manifold Mr can cause the deviation from the original manifold, illustrated in Fig. (a).
In this work, we aim to leverage the image-to-video diffusion model (SVD) for keyframe inter- polation tasks, eliminating the multiple noise injections or model fine-tuning. Notably, key innovation lies in the sequential sampling of the temporally forward path æ, and the temporally backward path of xt == flip(x;)) by integrating single re-noising step between them:
1111. $00.com
02-1 (x: @o.com))
(x;
Or
Lt T1-1cam
V 07 - of
(8)
r/11 = 200,cus +
01-1
(1r, @0,co))
(9)
Ft-1 (a/-1)'
(10)
Note that the amount of renoising is determined by the noise difference between M, and M,-1, which is crucial to avoid mode collapsing behavior from the forward path sampling. This approach effectively constrains the sampling process for bounded generation between the start frame (Jam) and the end frame (Tena). As depicted in Fig. (b), method seamlessly connects the temporally

---

Published as a conference paper at ICLR 2025
[a] Existing methods (Fusing; off the manifold)
(b) Ours (Bidirectional sampling; on the manifold)
(2) remoising
M;
Mr
(1 2) denoising
(3) denoising
(1:1) denoising
denoising
My-1
Mr-1
fusc
Forward
Forward
Backward
Backward
Figure 3: Comparison diffusion sampling paths. (a) Existing methods encounter off- ofnifoad issues due to the averaging of two sample points. (b) In contrast, our bidirectional sampling sequen- tially connects the temporally forward and backward paths, ensuring that the process remains within the manifold.
forward and backward paths that the sampling trajectory stays within the SVD manifold, resulting in smooth and coherent transitions throughout the interpolation process.
3.2 ADDITIONAL MANIFOLD GUIDANCES
We further employ recent advanced manifold guidance techniques to enhance the interpolation performance the bidirectional sampling. First, we introduce additional frame guidance us- ing DDS (Chung et al. 2023). Then, we replace traditional CFG (Ho & Salimans) [202)) with CFG++ (Chung et al. 2024) to mitigate the off-manifold issue of CFG in the original implementa- tion of SVD (Blattmann et al. 2023a).
Last frame guidance with DDS. We introduce additional frame guidance to achieve more accurate bounded generation. Specifically, employ DDS guidance to align the last frame with Cand in the temporally forward path and with Cstt the temporally backward path. DDS (Chung et al. 2023 synergistically combines the diffusion sampling and Krylov subspace methods (Liesen & Strakos 2013) such as the conjugate gradient (CG) method, guaranteeing the on-manifold solution of the tollowing optimization problem:
min {(x) ly A(=)
(11)
x #EM
where AA is the linear mapping, is the condition, and M represents the clean manifold of the diffusion sampling path.
Here, we present a novel insight that DDS, originally designed to address image inverse problems, can also effectively applied to bounded video generation. Specifically, we define A(z) == Bist as a last-frame extractor and as the target condition, which corresponds to Cend for the temporal forward path and Cstart for the temporal backward path in bounded video generation. Then we take DDS step from II denoised estimates (æ0.cum and $0.c...) with target condition (Cend and caar) to get the on-manilold solutions (0o.c,u and (0,c.)) the following optimization problems:
B0.cau
DDS( Cand)
arg min wE =E +X,
||Cend A(xx)
(12)
DDS( (om.
Cstan)
arg min #c
|| A(x))1
(13)
+K₁
where IC; the I-th order Krylov subspace, in which Krylov subspace methods seek an approximate solution. By leveraging this DDS framework, we effectively guide the sampling process toward a path conditioned by both the start and end frames, which is particularly effective for keyframe interpolation.
Better Image-Videe alignment with CFG++. Recent advances of CFG++ (Chung et al. 2024) tackles the inherent off-manifold issue in CFG (Ho & Salimans 2021). Specifically, CFG++ mit- igates this undesirable off-manifold issue using the unconditional score instead the conditional score in re-noising process of CFG. By using the unconditional score, CFG++ overcome the off manifold phenomena in CFG-generated samples, resulting in better text-image alignment for text-to-image generation tasks.
5

---

Published as a conference paper at ICLR 2025
While SVD replaces text embeddings with CLIP image embeddings, empirically found that CFG++ initially designed to enhance text alignment image diffusion models, also significantly improves image-to-video alignment in the video diffusion model. Specifically, after applying CFG+ into SVD sampling framework, the Euler-step of SVD (3) now reads:
Lt- = to.c + 0t-1 (x. Lo.0),
(14)
0t
where the last term in 3 is replaced by T. In practice, we apply DDS guidance before CFG++
update, #o.e in (4) should be replaced with To,e as in (22 13 We experimentally found that incorporating DDS and CFG++ guidance synergistically improves the interpolation performance of bidirectional sampling. The overall sampling method effectively steers the SVD sampling path perform keyframe interpolation in an on on-manifold manner, fully leveraging the generation capabili- ties of SVD. The detailed algorithm is provided in Algorithm The vanilla bidirectional sampling can be implemented by removing DDS guidance (orange) and replacing the CFG++ update (blue) with a traditional CFG update. The detailed algorithm of the vanilla bidirectional sampling is pro- vided in Appendix A
Algorithm ViBiDSampler
Require: TT (0, 1), Taaart Tnnd (ar )
1: Cstart: Cead f encode Lend)
2: for = do
3:
D0.com L0.0 f- Do(ai; oz, Cstan)
D EDM denoised estimate with Caat DDS guidance for end-frame matching
4: B0,com f DDS (DD, Cend)
5: Tt- 1.cra f Cam + 0t-1
(Ez 20,00
CFG++ update
6:
Lt,com
- I1-1,cmm
o o
Re-noise
7:
I L//Cau
Hiip(xr,cur)
D Time reverse EDM denoised estimate with Cend
8:
3 $b
$6.00 + Do
Do (z'2 i os, Cend)
9:
L.ccm DDDS(H'b Caa))
DDS guidance for start-frame matching
F/ f woo.om + (ar/,cun 16,z)
CFG++ update
11: Iz-1 t- flip(x((11)
D Time reverse
12: end for
13: return 200
4 EXPERIMENTAL RESULTS
4.1 EXPERIMENTAL SETTING
Dataset. The high-resolution (1080p) video datasets used for evaluation are sourced from the DAVIS dataset (Pont-Tuset et al. 2017) and the Pexels d For the DAVIS dataset, we pre- processed 100 videos into 100 video- keyf-ee pairs, with each video consisting of 25 frames. This dataset includes a wide range of large and varied motions, such as surfing, dancing, driving, and airplane flying. For the Pexels dataset, we collected 45 videos, primarily featuring scene motions, natural movements, directional animal movements, and sports actions. We used the first and last frames from each video as keyframes for our evaluation.
Implementation Details. For the sampling process, used the Euler scheduler with 25 timesteps for both forward and backward sampling. The motion bucket ID was fixed at 127, and the decoding frame number parameters followed the default settings from SVD. Since micro-condition fps is sensitive to the data, applied lower fps for cases with large motion and higher fps for cases with smaller motion. While both DDS and CFG++ generally improve the results, the choice between them depends on the specific use case. All evaluations were performed on single NVIDIA RTX 3090.
to 4 due memory limitations
NVIDIA RTX 3090 GPU. All other
4.2 COMPARATIVE STUDIES
We conducted comparative study with four different keyframe interpolation baselines, including FILM (Reda et al. 2019), conventional flow-based frame interpolation method, and three frame in- terpolation methods based on video diffusion models: TRF (Feng al. 2024). DynamiCrafter (Xing
https:/ www. www.ppp.eel... s.. com/

---

Published as a conference paper at ICLR 2025
lo
hu
120
122
124
FILM
TRF
Cinnerativa
Ours
1.2
136
120
125
FILM
TRF
Glnnrativve
Ours
1.0
120
122 125
FILM
TRF
Generalive
Ours
Figure 4: Qualitative evaluation compared to three baselines: FILM, TRE, and Generative Inbetweening. The start and end frames (Iu, Iz1) are used keyframes. While FILM encounters difficulties in capturing motion when there a significant discrepancy between the two keyframes, and TRF and Generative Inbetweening experience decline in perceptual quality due the blurring of objects within the image, our method successfully captures motion while maintaining high fidelity in the generated images.
7

---

Published as a conference paper at ICLR 2025
DAVIS
Pexels
Method
LPIPS 4
FID
FVD 1
LPIPS L
FID
FVD |
FILM
0.2697
40.241
833.80
0.0821
25.615
559.16
TRFE
0,3102
60.278
622.16
0.2222
80.618
880.97
DynamiCrafter
0.3274
46.854
538.36
0.1922
49.476
604.20
Generative Inbetweening
0.2823
36.273
490.34
0.1523
40.470
746.26
Ours (Vanilla) 0.3031
52.452
543.31
0.2074
63.241
717.37
Ours (Vanilla w/ CHG++)
0,2571
41,960
434.41
0.1524
41.347
478.35
Ours (Full)
0.2355
35.659
399.15
0.1366
37.341
452.34
Quantitative evaluation on DAVIS and Pexels datasets. We compared our method against four different baselines and conducted ablation studies to assess the impact of CFG++ and DDS. Ours (Vanilla) refers to the bidirectional sampling method utilizing traditional CFG update without DDS guidance. Ours Vanillla w/ CFG++) refers to the bidirectional sampling method with CFG++ update, also without DDS guidance. Bold and underline refer to the best and the second best, respectively.
Table 1:
4
20
122
12
TRF
Vanilla
+ CFG++
DDS
Ablation study on the effects of CFG++ and DDS. The inclusion of CFG++ and DDS results in improved perceptual quality in the generated frames.
Figure 5:
et al. )023), and Generative Inbetweening (Wang et al., 2024). We conducted these studies using the official implementations with default values, except for TRF, which has not been open-sourced yet.
Qualitative evaluation. As illustrated in Fig. 4 our model clearly outperforms the other methods in terms of motion consistency and identity preservation. Other baselines struggle accurately predict the motion between the two keyframes when there is significant difference in content. For exam- ple, in Fig. 4 the first frame shows the tip of airplane, while the last frame reveals the airplane's body. In this case, FILM fails to produce a linear motion path, instead showing the airplane's 's shape converging toward the middle frame from both end frames, resulting in the airplane's body being disconnected by the 18th frame. While TRF and Generative Inbetweening show sequential move- ment, the airplane's 's shape becomes distorted. In contrast, our method preserves the airplane's shape while effectively capturing its gradual motion. Furthermore, it can be observed from the second and third cases from Fig. that our method generates temporally coherent results while semantically adhering to the input frames. In TRF, the shapes of the robot and the dog become blurred due to the denoising paths deviating from the manifold during the fusion process, leading to artifacts the image. While Generative Inbetweening mitigated this off-manifold issue through temporal attention rotation and model fine-tuning, artifacts still persist. contrast, our method preserves the shapes of both the robot and the dog, generating frames with strong temporal consistency.
Quantitative evaluation. For quantitative evaluation, we used LPIPS (Zhang et al. 2018 and FID (Heusel et al. 2017) to assess the quality of the generated frames, and FVD VVVDDUntenn (Unterhiiner al. 2019)
Unofficial implementation: https:: http::///gttbb..ccm///igttaaaoap
Reversal
8

---

Published as a conference paper at ICLR 2025
Method
NFE
Train
Inference time (s)
Frame
Resolution
TRF
120
X
443
25
1024 x 576
DynamiCrafter
50
V
42
16
512 x 320
Generative Inbetweening
300
V
1,222
25
1024 x 576
Ours
50
X
195
25
1024 x 576
Table 2: A comprehensive comparison of our method with other diffusion-basee approaches.
l
0.6
0.8
Figure 6: Effect of CFG++ guidance scale. The rows, from top bottom, correspond to the CFG+ scales of 0.6, 0.8, and 1.0.
Metrics
0.6
0.8
1.0
LPIPS 525.36
424.03
399.15
FID &
52.5059
40.4968
35.6594
FVD 4
0.2697
0.2394
0.2355
Table Quantitative analysis on CFG++ guidance scale w. Effective results are obtained the scale of 1.0.
evaluate the overall quality of the generated videos. As shown in Table I our method surpasses the other baselines in terms of fidelity. Moreover, it achieves superior perceptual quality. particularly scenarios involving dynamic motions (DAVIS), indicating that our approach effectively addresses the issue of deviations from the diffusion manifold, resulting in improved video generation quality. For more comprehensive comparison, we present quantitative evaluations using PSNR and SSIM metrics in Appendix B.I Nonetheless, it is important note that these metries primarily evaluate pixel-wise accuracy, which does not always align with human perceptual preferences. Within the scope of video generative models that align better with human perceptual preferences, proposed method outperformed all baseline methods.
4.3 COMPUTATIONAL EFFICIENCY
We performed comparative studies on the computational cost of diffusion models, as presented in Table 2 In the training stage, DynamiCrafter undergoes additional training with large-scale image-to-video model for the frame interpolation task, while Generative Inbetweening also neces- sitates SVD model fine-tuning, both of which demands significant computational resources. During the inference stage. both TRF and Generative Inbetweening generate videos in 25 ~ 50 steps for each forward and backward direction, with additional noise injection steps that further increase the number of function evaluations (NFE) and inference time. However, our method does not require additional training or fine-tuning and completes the process in just 25 steps per direction, without requiring multiple re-noising.
4.4 ABLATION STUDIES
Bidirectional sampling and conditional guidance. The effectiveness of bidirectional sampling can be validated the vanilla version without any conditional guidance, such as CFG++ or DDS. As demonstrated in Table I vanilla model outperforms TRF across all three metries, supporting the claim that fusing time-reversed denoising paths leads to off manifold issues, which our method addresses through bidirectional sampling. In addition, with conditional guidance from CFG++ and

---

Published as a conference paper at ICLR 2025
Start
Keyfname Interpolation
End Image
Figure 7: Application to keyframe interpolation with various boundary conditions. The end image a is identical to the start image. End images b-c represent dynamic boundaries sampled from different time points.
DDS, we could achieve even better results and outperform DynamicCrafter and Generative Inbe- tweening which further train the image-to-video models. This is consistent with Fig. 5 which il- lustrates that frames generated by TRF exhibit blurry shapes of the golfer and unnecessary camera movement. In contrast, the body shape of the golfer and the golf club are progressively better pre- served as additional conditional guidance is incorporated.
CFG++ guidance scale. As shown in Fig. at higher CFG++ scales, the semantic information of the input frames is better preserved in the generated intermediate frames, resulting in improved fidelity. For instance, while the small person in the first input frame disappears in the intermediate frames at CFG++ scales of 0.6 and 0.8, the person remains visible across all the intermediate frames at scale of 1.0. Additionally, as the CFG++ scale decreases, the blurriness of the chairlift in the output frames gradually worsens. This aligns with the findings presented in Table B] The LPIPS, FID, and FVD values are lowest at a CFG++ scale of 1.0 and highest at a scale of 0.6, indicating that CFG++ contributes to improving the perceptual quality of the generated videos.
4.5 IDENTICAL AND DYNAMIC BOUNDS
Our method is applicable not only dynamic bounds, where the start and end frames are different, but also to static bounds, where the start and end frames are identical. As illustrated in Fig. 7 our method successfully generates temporally coherent videos with identical start and end images (a). For example, the wave line also consistently fluctuates with the progression of time. Furthermore, as seen in the fifth and sixth rows of Fig. ? our method effectively generates intermediate frames based on varying end frames (b and c). Given that the end images of the two rows differ, the resulting intermediate frames are generated accordingly.
CONCLUSION
We present Video Interpolation using Bidirectional Diffusion Sampler (ViBiDSampler), a novel approach for keyframe interpolation that leverages bidirectional sampling and advanced manifold guidance techniques to address off maniioldd issues inherent in time-reversal-fusion based methods. By performing denoising sequentially in both forward and backward directions and incorporating CFG++ and DDS guidance, ViBiDSampler offers reliable and efficient framework for generating high-quality, temporally coherent, and vivid video frames without requiring fine-tuning or repeated re-noising steps. Our method achieves ttate-of-the-art performance in keyframe interpolation, as evidenced by its ability to generate 25-frame video at high resolution in a short processing time.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work was partly supported by Institute for Information & communications Technology Tech- nology Planning & Evaluation(IITP) grant funded by the Korea govermeen(MiSTty(SS2000 1190075, Artificial Intelligence Graduate School Support Program(KAIST) and the National Re- search Foundation of Korea under Grant RS-2024-00336454.
REFERENCES
Omer Bar.Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Yuanzhen Li, Tomer Michaeli, et al. Lumiere: A space-time diffusion model for video generation. arXiv preprint arXiv:2401 240..1244,, 2024.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311. 2023a.
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22563-22575, 2023b.
Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma. Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Seine: Short-to-long video diffusion model for generative transition and prediction. In The Twelfth International Conference on Learning Representations, 2023.
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 35:25683-25696, 2022.
Hyungjin Chung, Suhyeon Lee, and Jong Chul Ye. Decomposed diffusion sampler for accelerating large-scale inverse problems. arXiv preprint arXiv:2303.05754 2023.
Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Lunifold-constraing classifier free guidance for diffusion models. arXiv preprint arXiv:2406.nm 08070, 2024.
Duolikun Danier, Fan Zhang, and David Bull. Ldmvfi: Video frame interpolation with latent dif- fusion models. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp- 1472-1480, 2024.
Haiwen Feng, Zheng Ding, Zhihao Xia. Simon Niklaus, Victoria Abrevaya, Michael Black, and Xuaner Zhang. Explorative inbetweening of time and space. arXiv preprint arXiv:2403. 1461 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Informaiion Processing Systems, 35:8633- 8646, 2022.
Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. European Conference on Computer Vision, pp- 624 642. Springer, 2022.
11

---

Published as a conference paper at ICLR 2025
Zhilin Huang, Yijie Yu, Ling Yang, Chujun Qin, Bing Zheng, Xiawu Zheng, Zikun Zhou, Yaowei Wang, and Wenming Yang. Motion- aware latent diffusion models for video frame interpolation. arXiv preprint ar arXhv:240.113534, 2024.
Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpola- tion with diffusion models. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7341-7351, 2024.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-26577,, 2022.
Lingtong Kong, Boyuan Jiang, Donghao Luo, Wenqing Chu, Xiaoming Huang, Ying Tai, Chengjie Wang, and Jie Yang. Ifrnet: Intermediate feature refine network for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2022.
Zhen Li, Zuo-Liang Zhu, Ling-Hao Han, Qibin Hou, Chun-Le Guo, and Ming-Ming Cheng. Amt: All-pairs multi-field transforms for efficient frame interpolation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9801-9810, 2023.
Jörg Liesen and Zdenek Strakos. Krylov subspace methods: principles and analysis. Numerical Mathematics and Scie, 2013.
Liying Lu, Ruizheng Wu, Huaijia Lin, Jiangbo Lu, and Jiaya Jia. Video frame interpolation with transformer. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 3532-35444 3542, 2022.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhor, Jonas Müller, Joc Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine- Hornnug, and Luc Van Gool. The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675. 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. Proceed- ings of the 38th International Conference on Machine Learning, pp.
2021.
Fitsum A Reda, Deqing Sun, Aysegul Dundar, Mohammad Shoeybi, Guilin Liu, Kevin J Shih, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Unsupervised video interpolation using cycle consistency. In Proceedings of the IEEE/CVF international conference on computer Vision, pp. 892-900, 2019.
Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for vidco generation. 2019.
Vikram Voleti, Alexia Iolicoeur-Marttneau, and Chris Pal. Mevd-masked conditional video diffusion for prediction, generation, and interpolation. Advances in neural information processing systems, 35:23371-23385, 2022.
Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Sameelmacher-Shhizeemme Aleksander Holynski, and Steven M Seitz. Generative inbetweening: Adapting image-to ti-ieo models for keyframe interpolation. arXiv preprint arXiv:2408. 15239, 2024.
Jay Zhangjie WU, Yixiao Ge, Xintao Wang. Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. auideo-video: One-shot tuning of image diffusion models for text-to-video generation. Proceedings of the IEEE/CVF International Conference on Computer Vision pp. 7623-7633, 2023.
12

---

Published as a conference paper at ICLR 2025
Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xintao Wang. Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. arXiv preprint arXiv:2310.12190, 2023.
David Junhao Zhang, Jay Zhangjie Wu, Jia-Wei Liu, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, and Mike Zheng Shou. Show-1: Marrying pixel and latent diffusion models for text-to- video generation. arXiv preprint arXiv:2309. 1591888,.,,88,,,, 2023a.
Guozhen Zhang, Yuhan Zhu, Haonan Wang. Youxin Chen, Gangshan Wu, and Limin Wang. Ex- tracting motion and appearance via inter-frame attention for efficient video frame interpolation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 5682-5692, 2023b.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pallern recognition, pp. 586-595, 2018.
13

---

Published as a conference paper at ICLR 2025
A ALGORITHM
Algorithm 2 ViBiDSampler (Vanilla)
Require: TT. Isaut: Lend; for)1=1
1: Cstart; Goad f- encode (Iaaut lend)
2: for t T do
3: $0.com $0 +- Do(a;; Tte Tr, Caan)
Di EDM denoised estimate with Caart
4: 11-1,cem &- BD..am
0t-1 (It
ae
5:
v o 02-Ic Dt,con t In-
> Re-noise
6:
Ftrom t ffip(xt..cm)
D Time reverse
7:
El $0.0 t Do Do(22 Cam
: oi, Cend)
EDM denoised estimate with Coad
8: T/-1 cow + Ci-1 F Coun
1// cau)
9: I:-1 f flip(x'--1)
> Time reverse
end for
11:
return 20
B ADDITIONAL RESULTS AND DISCUSSION
B.I QUANTITATIVE EVALUATION RESULTS
In addition the perceptual quality evaluations presented Table [ provide the PSNR and SSIM scores for our method and baseline models to offer a clearer understanding of our work. These metries were obtained by comparing the generated video frames with their corresponding ground truth frames. As shown in Table [4) FILM achieved the highest PSNR and SSIM scores among the evaluated video generative models, which can be attributed to its training strategy that employs PSNR- loss functions, such as the L1 loss calculated between the ground truth and output frames. However, within the scope of video generative models that align better with human perceptual preferences, the highest PSNR and SSIM scores on both the DAVIS and Pexels datasets. These results highlight model's 's ability to generate videos that are both visually realistic and exhibit superior fidelity.
proposed method outperformed all baseline models, recording
DAVIS
Pexels
Method
PSNR
SSIM f
PSNR 1
SSIM
FILM TRF
22.5521
0.5346
31.3986
0.8418
16.4078
0.4040
19.2670
0.5346
Generative Inbetweening Ours
16.0377
0.4255
20.8624
0.6334
17.5215
0.4387
21.4235
0.5939
Table 4: Quantitative evaluation DAVIS and Pexels datasets. We conducted comparative analysis of our method with FILM, TRF and Generative Inbetweening in terms of PSNR and SSIM. Bold and underline refer the best and the second best, respectively.
B.2 ABLATION STUDIES ON ADVANCED GUIDANCES
To emphasize the importance of bidirectional sampling, we conducted additional ablation studies on the guidance techniques, CFG++ and DDS, using the Pexels dataset. These studies involved evaluat- ing the results of the vanilla models and comparing them with the models enhanced by incorporating the guidance techniques.
As shown in Table 5 our vanilla bidirectional sampling consistently outperforms TRF across all metrics. This demonstrates that bidirectional sampling alone effectively addresses off-manifold is- sues, mitigating the artifacts commonly observed in TRF. Unlike Generative Inbetweening, which requires fine-tuning of the diffusion model, our method operates entirely in training-free manner.
14

---

Published as a conference paper at ICLR 2025
When comparing models enhanced with guidance techniques, method demonstrates superior performance compared to both TRF and Generative Inbetweening. Notably, TRF also shows im- provements with the addition of guidance techniques, as reflected in the quantitative metrics in Ta- and the qualitative results available on our anonymous Project pagel The results indicate more stable motion and better preservation of bounding frame information. However, its performance remains lower than that of our full method, highlighting the critical role of bidirectional sampling in addressing off-manifold issues and enabling the generation of more accurate and higher-quality videos. In addition, our proposed method exhibits significantly higher degree of improvement compared TRF. Since manifold guidance methods, such as CFG++ were originally designed for on-manifold sampling, our method can be synergistically enhanced with these guidance techniques for further improvement. In contrast, TRF faces challenges achieving this synergistic improve- ment due to the generation of off-manifoll samples, highlighting the differences in performance and effectiveness between the two approaches. In contrast, the integration of guidance techniques into Generative Inbetweening does not yield substantial improvements. This limited effect may be due to the interference between its fine-tuning process and rotated temporal attention, which could hinder the effectiveness of the guidance techniques.
We strongly encourage readers to visit our anonymous Website for a deeper understanding of the distinct roles of bidirectional sampling and guidance methods.
Vanilla
CFG++ & DDS
Method
LPIPS
FID L
FVD |
LPIPS |
FID
FVD |
TRF
0,2222
80.618
880.97
0.2010
52.738
778.69
Generative Inbelweening Ours
0.1523
40.470
746.26
0.1662
42.487
747.95
0.2074
63.241
717.37
0.1366
37.341
452.34
Table 5: Ablation study on CFG++ and DDS. Bold and underline refer to the best and the second best, respectively.
B.3 DISCUSSIONS ON THE ROLES OF CFG++ AND DDS
CFG++. CFG++ is a guidance technique designed to improve alignment between images and text model, the generated videos achieve better align- ment with the image conditions and maintain semantic consistency with the given frames, Istart. and Tend- This enhancement positively impacts the perceptual quality of the videos.
conditions. By applying CFG++ guidance
DDS. SVD, an image-to-video diffusion model, generates 25-frame videos based on an initial frame condition, without utilizing a final frame condition. To address the absence of last frame condition within the video interpolation framework, we incorporated DDS guidance. This approach ensures alignment of the last frame with Cand in the temporally forward path and Estart in the temporally backward path, thereby enhancing the temporal consistency of the generated videos.
B.4 ABLATION STUDY ON SVD VERSION.
Since we employed SVD-XT, which is the fine-tuned version of the original SVD, we checked the performance of our method with SVD. As shown in the Table we see the performance increment in better video generation models.
Method (Pexels)
FVD L
LPIPS
SVD
608.07
0.1577
SVD-XT (ours)
452.34
0.1366
Table 6: Ablation study on SVD version. Bold refer to the best.
B.5 FUTURE WORK AND DISCUSSTONS
Our proposed method demonstrates exceptional performance in generating intermediate frames from two bounding frames. However, as the model is based Stable Video Diffusion, which employs
15

---

Published as a conference paper at ICLR 2025
image embeddings conditions for cross-attention instead of textual prompts, have not sidered the incorporation of text conditioning. Nonetheless, when bidirectional sampling is applied to other image-to-vides video (12V) diffusion models, such as DynamiCrafterr we believe that it becomes feasible to guide actions based on textual conditions. Extending our method to support text-based control presents promising avenue for future research.
16

---

Published as a conference paper at ICLR 2025
B.6
ADDITIONAL EXPERIMENTAL RESULTS
Iu
1so
12a
FILM
TRF
Canaralive Intettwoening
Ours
hi
FILM
TRF
Generative
Ours
4/2
4,8
129
FILM
TRF
Generative
Ours
Figure 8: Additional comparison with baseline methods.
17

---

Published as a conference paper at ICLR 2025
42
1z0
121
FILM
TRF
Gensrativs
Ours
l12
h
124
FILM
TRF
Genaralive
Ours
112
l16
Iz0
125
FILM
TRF
Generative
Ours
Figure 9: Additional comparison with baseline methods
18

---

Published as a conference paper at ICLR 2025
ha
Izn
b
FILM
TRF
Generstive
Ours
1o
11
166
120
124
FILM
TRF
Clenarativ
Ours
h2
FILM
TRF
Clnnerativn
Ours
Figure 10: Additional comparison with baseline methods.
19


Reviews:
[{"review_id": "hGPC5jAbPL", "reviewer": "Reviewer_VMQD", "summary": "This work introduces a novel, bidirectional sampling strategy to address off-manifold issues without requiring extensive re-noising or fine-tuning. The proposed method employs sequential sampling along both forward and backward paths, conditioned on the start and end frames, respectively, ensuring more coherent and on-manifold generation of intermediate frames.", "questions": "- How to incorporate text conditioning into your framework so users can control the action in the generated frames?\n- The results and presentation are sound, except for some minor issues (see weaknesses). I will increase my score once the authors address my concerns.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Unlike previous fusing strategies, which compute two conditioned outputs in parallel and then fuse them, the proposed bidirectional diffusion sampling strategy samples two conditioned outputs sequentially, which mitigates the off-manifold issue.\n- The proposed method can further work with advanced on-manifold guidance techniques (CFG++, DDS) to produce more reliable interpolation results that have better alignment of the input frames.\n- This is a training-free method which can efficiently interpolate between two keyframes to generate a 25-frame video at 1024×576 resolution in just 195 seconds on a single 3090 GPU.", "weaknesses": "- The experimental results presented lack PSNR scores. While accurate recovery of intermediate frames may not be the primary objective of this study, and high PSNR values might not necessarily indicate superior human preferences, it is still beneficial to include the score to provide readers with a clearer understanding of the work’s potential.\n- The proposed bidirectional sampling addresses the off-manifold issue, but it comes with the potential for deviation from the start or end frame, depending on the order of sequential sampling. To mitigate this issue, the authors have incorporated previous guidance techniques. However, I would appreciate a more comprehensive video comparison in the ablation study, in addition to the provided static comparison on a single scene.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1XgWetSXgM", "reviewer": "Reviewer_wFk9", "summary": "This paper presented a bidirectional sampling strategy, sequential sampling along both forward and backward paths, for video interpolation. Additional manifold guidance such as DDS and CFG++ are further introduced to enhance the interpolation results. Experiments are conducted to evaluate the proposed ViBiDSampler.", "questions": "- The method is training-free. In general, training-based method can be more effective.\n- More ablation study is suggested to assess the effect of pre-trained video generation models. Will better video generation models result in better performance?\n- Discussion on the limitation and future work.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ A bidirectional sampling strategy, sequential sampling along both forward and backward paths, for video interpolation. \n+ Additional manifold guidance such as DDS and CFG++ are further introduced to enhance the interpolation results. \n+ Training-free, convenient for use.\n+ Experiments are conducted to evaluate the proposed ViBiDSampler.", "weaknesses": "- The method is training-free. In general, training-based method can be more effective.\n- More ablation study is suggested to assess the effect of pre-trained video generation models. Will better video generation models result in better performance?\n- Discussion on the limitation and future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "El4JNGY4xt", "reviewer": "Reviewer_hjqi", "summary": "This paper proposes a inference strategy to achieve frame interpolation w/o training. Its key idea is to use the pre-trained stable video diffusion model as a prior and complete the frame interpolation given the first and the last frame as a condition. The main improvement seems from the proposed bidirectional sampling combining with SOTA sampling approaches, i.e., DDS and CFG++.", "questions": "My main concerns are mainly as follows:\n\n1. The contribution of this paper seems marginal. It seems the main contribution comes from eq. (9). However, the improvement of such modification seems not that obvious comparing with using SOTA sampling approaches including DDS and CFG++ from Table. 1, i.e., only \"Ours (full)\" shows obvious improvement comparing with other baselines but if my understanding is correct, other baselines seem not using DDS and CFG++. The author should further verify the improvement of each technology used in the paper, i.e., bidirectional sampling, DDS and CFG++, e.g., providing more visualization as well as adding DDS and CFG++ to other baselines and show the improvement.\n\n2. The practical use the proposed approach seems not convincing. The proposed approach still requires 192s to generate 25 frames, which is at least two times longer than directly inferencing a video generation model with similar computational cost. Moreover, I also wonder how to keep the consistency between frames when the required length of the output video is beyond 25.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+The paper is easy to follow .\n+The visualization results seem impressive comparing with other baselines provided in the paper.", "weaknesses": "The contribution of this paper seems marginal. It seems the main contribution comes from eq. (9). However, the improvement of such modification seems not that obvious comparing with using SOTA sampling approaches including DDS and CFG++ from Table. 1, i.e., only \"Ours (full)\" shows obvious improvement comparing with other baselines but if my understanding is correct, other baselines seem not using DDS and CFG++.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AdYycEBZ6x", "reviewer": "Reviewer_hsxN", "summary": "This paper proposes ViBiDSampler, a training-free method utilizing pre-trained image-to-video diffusion models for keyframe interpolation. They aim to tackle the off-manifold problem of previous studies, which leads to artifacts or costly processes. They propose bidirectional sampling, with a set of denoising-renoising-denoising with alternating conditioning on start and end frames. This way, they manage to keep the generated results stay on the original manifold. The additionally integrate advanced guidance methods like CFG++ and DDS to reach state-of-the-art performance.", "questions": "1. To my understanding, CFG++ and DDS could possibly be applied to existing methods, such as Generative Inbetweening. If so, could the authors provide experiments on this, to ablate and show the importance of bidirectional sampling?\n\n2. In bidirectional sampling, in some aspects, I think it is possible for this to work, but in some aspects, I don't. For example, taking the example of Fig.3(b), there are three steps, (1) denoising (w/ $c_{start}$), (2) renoising, (3) denoising (w/ $c_{end}$). I am curious how the generation process goes through these steps. Maybe a visualization would be helpful, showing how the start / end conditions are really affecting in those steps. I am specifically curious of this process in that the (2) renoising step could possibly remove the information of $c_{start}$ injected from (1) denoising, and the conditioning could have possibly be injected only at (3), making the steps (1) and (2) redundant. A further analysis / clarification on this part, that the proposed method surely takes both start / end frame conditioning.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Strengths\n- Overall, the paper is easy to follow, with coherent and clear problem definition and solution.\n- Simple methodology. The proposed method, Bidirectional sampling seems to be easy to use.\n- Efficiency. The proposed method does not require additional training and is fast in inference.", "weaknesses": "1. Notations.\n- The notations are confusing. For instance, $x_t$ seems to be used as a variable and a function. \n- The definitions could be further clarified. \n- Lines 247 - 256 was kind of confusing at first, with many similar looking notations without predefinitions, and mixed use of notations as functions and variables.\n\n\n2. Limited contribution.\n\nAlthough the proposed method is intuitive, simple, and quantitatively effective, some uncertainties remain.\n- the 'Vanilla' setting at Table 1 and Figure 5, seems to show the effectiveness of bidirectional sampling solely, without guidance such as CFG++ and DDS. However, I am not fully convinced with its effectiveness, as Generative Inbetweening seems to outperform the vanilla version in Table 1. Also, in the qualitative results of Figure 5, the results of bidirectional sampling (vanilla) itself does not seem to be better than that of TRF.\n- The concern above makes me wonder, if the superior performance of the entire framework proposed comes from the good guidance methods, CFG++ and DDS.\n- In addition, CFG++ and DDS are manifold guidance methods proposed in other studies, and seems like this work simply took advantage of existing methods. I think it is hard to say that simply adoption of these guidance methods is a contribution.\n\nIn short, I am not fully convinced if the proposed bidirectional sampling itself is a powerful method compared to existing works, and suspect the superior performance could have come from the advanced manifold guidance methods, which, is hard to say to be a contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)