Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention
Authors: Dejia Xu
Yifan Jiang
Chen Huang
Liangchen Song
Thorsten Gernoth
Liangliang Cao
Zhangyang Wang
Hao Tang
Venue: ICML 2025 poster (2025)
Keywords: Video Generation
Camera Control
3D Consistency
Primary area: applications->computer_vision
TL;DR: We present Cavia, the first framework that enables users to generate multiple videos of the same scene with precise control over camera motion, while simultaneously preserving object motion.

Abstract: In recent years there have been remarkable breakthroughs in image-to-video generation. However, the 3D consistency and camera controllability of generated frames have remained unsolved. Recent studies have attempted to incorporate camera control into the generation process, but their results are often limited to simple trajectories or lack the ability to generate consistent videos from multiple distinct camera paths for the same scene. To address these limitations, we introduce Cavia, a novel framework for camera-controllable, multi-view video generation, capable of converting an input image into multiple spatiotemporally consistent videos. Our framework extends the spatial and temporal attention modules into view-integrated attention modules, improving both viewpoint and temporal consistency. This flexible design allows for joint training with diverse curated data sources, including scene-level static videos, object-level synthetic multi-view dynamic videos, and real-world monocular dynamic videos. To the best of our knowledge, Cavia is the first framework that enables users to generate multiple videos of the same scene with precise control over camera motion, while simultaneously preserving object motion. Extensive experiments demonstrate that Cavia surpasses state-of-the-art methods in terms of geometric consistency and perceptual quality.

Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention
Dejia Xu Xul Yifan Jiang?
Chen Huang?
Liangchen Song²
Thorsten Gernoth Liangliang Cao 2
Zhangyang Wang' Hao Tang²
Abstract
1. Introduction
The rapid development of diffusion models has enabled sig- nificant advancements in video generative models. Early efforts have explored various approaches, either training a video model from scratch or by fine-tuning pre-trained im- age generation models with additional temporal layers (Sta- bility, 2023; Wang et al., 2023a; Ho et al., 2022b; Singer et al., 2022: Ho al., 2022a; Nan al., 2024). The train- ing data of these video models typically consists of a rated mixture of image (Schuhmann et al., 2022) and video datasets (Bain et al., 2021; Wang et al., 2023b;a; Nan et al., 2024). While substantial progress has been made in improv- ing model architectures and refining training data, relatively little research has been conducted on the 3D consistency and camera controllability of generated videos.
In recent years, there have been remarkable break- throughs in image-to-video generation. How- ever, the 3D consistency and camera control- lability of generated frames have remained un- solved. Recent studies have attempted incorpo- rate camera control into the generation process, but their results are often limited to simple tra- jectories or lack the ability to generate consistent videos from multiple distinct camera paths for the same scene. To address these limitations, we introduce Cavia, novel framework for camera- controllable, multi-view video generation, capa- ble of converting an input image into multiple spatiotemporally consistent videos. Our frame- work extends the spatial and temporal attention modules into view-integrated attention modules, improving both viewpoint and temporal consis- tency. This flexible design allows for joint training with diverse curated data sources, including scene- level static vidcos, object-level synthetic multi- view dynamic videos, and real-world monocular dynamic videos. To the best of our knowledge, Cavia is the first framework that enables users to generate multiple videos of the same scene with precise control over camera motion, while simul- tancously preserving object motion. Extensive experiments demonstrate that Cavia surpasses state-of-the-art methods in terms of geometric consistency and perceptual quality. Project page: attps://irlds github.ttt.... wwt.tt.ii//aa.e......e io/Cavia/
To tackle this issue, several recent works (Wang al., 2023c; He et al., 2024; Bahmani et al., 2024; Xu al., 2024; Hou et al., 2024) have attempted to introduce camera controllability in video generation, aiming to ensure that generated frames adhere to viewpoint instructions, thereby improving 3D consistency. These works either enhance viewpoint control through better conditioning signals (Wang et al., 2023c; He et al., 2024; Bahmani et al., 2024) or by utilizing geometric priors, such epipolar constraints (Xu et al., 2024) or explicit 3D representations (Hou et al., 2024). However, despite these efforts, the generated videos often lack precise 3I) consistency or are restricted to static scenes with little to no object motion. Moreover, remains challenging for monocular video generators to produce multi-view consistent videos of the same scene from different camera trajectories.
Since independently sampling multiple sequences often re- sults in significantly inconsistent scenes, generating multiple video sequences simultaneously is desirable. However, this remains extremely challenging due to the scarcity of multi- view video data the wild, leading to multi-view genera- tions limited inconsistent near-static scenes synthetic objects. A concurrent work, CVD (Kuang et al., 2024), builds on multi-view static videos (Zhou et al., 2018) and aapping-augmented monocular videos (Bain et al., 2021), but can only generate videos with limited baselines, yield-
This work was performed while Dejia Xu interned at Apple. 'The University of Texas at Austin AApple. Correspondence to: Hao Tang <hao_ Lano_aangee @apple.com>.
Proceedings of the 12nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Cavia: Camera-controllable Multi-view Video Diffusion
ing inconsistent results when object motion is present. An- other concurrent work, Vivid-ZOO (Li et al., 2024a), lever- ages dynamic objects from the Objaverse (Deitke et al., 2023b) dataset and renders multi-view videos to train a video generator. However, due to limited data sources, their results are primarily object-centric frames from fixed view- points, lacking realistic backgrounds.
methods. Additionally, our flexible framework can operate on four views inference, offering improved view consistency and enabling 3D reconstruction of the generated frames.
2. Related Works
2.1. Camera Controllable Video Diffusion Models
To address these challenges, we propose Cavia, a novel framework that extends monocular video generator (Sta- bility, 2023) to generate multi-view consistent videos with precise camera control. We enhance the spatial and tem- poral attention modules to cross-view and cross-frame 3D attentions respectively, improving consistency across both viewpoints and frames. Our model architecture enables a novel joint training strategy that fully utilizes static, monoc- ular, and multi-view dynamic videos. Static videos (Zhou et al., 2018; Yu et al., 2023; Xia et al., 2024; Reizenstein et al., 2021; Deitke et al., 2023b;a) are converted to multi- view formats to ensure the geometric consistency in the generated frames. We then incorporate rendered synthetic multi-view videos of dynamic 3D objects (Liang et al., 2024; Jiang al., 2024; Li al., 2024c) to teach the model to generate reasonable object motion. To prevent overfitting on synthetic data, finetune the model pose-annotated monocular videos (Wang et al., 2023b; Nan et al., 2024) to enhance performance on complex scenes. Our framework synthesizes cross-view view and cross- framm consistent videos, and extensive evaluations on real and text-to-image gener- ated images show its applicability across challenging indoor, outdoor, object-centric, and large-scene cases. We systemat- ically measure the quality of the generated videos in terms of per-video and cross-view geometric consistency and per- ceptual quality. Our experiments demonstrate superior per- formance compared to previous methods, both qualitatively and quantitatively. Additionally, we show that our method can extrapolate to generate four views during inference and enable 3D reconstruction of the generated frames.
Recent advancements in video diffusion models have significantly benefited from scaling model architectures and leveraging extensive datasets (Bain et al., 2021; Wang et al., 2023b;a), leading to impressive capabilities in generating high-quality videos (Stability, 2023: Ho et al., 2022b; Singer et al., 2022; Ho et al., 2022a; OpenAI), While large foundational video diffusion models exist, our work focuses on enhancing camera control over video diffusion processes, a rapidly growing area of research. AnimateDiff (Guo et al., 2023) and Stable Video Diffusion (SVD) (Stability, 2023) employ individual camera LoRA (Hu et al., 2021) models for specific camera motions. MotionCtrl (Wang et al., 2023c) improves flexibility by introducing camera matrices, while CameraCtrl (He et al., 2024), CamCo (Xu et al., 2024), and VD3D (Bahmani et al., 2024) enhance the camera control accuracy by introducing Plücker coordinates to the video models via controlnet (Zhang & Agrawala, 2023). To further improve the geometric consistency, CamCo (Xu et al., 2024) applies epipolar constraints and CamTrol (Hou et al., 2024) incorporates 3D Gaussians (Kerbl et al., 2023). However, these methods focus on monocular video generation, limiting their ability to sample multiple consistent video sequences of the same scene from distinct camera paths. CVD (Kuang et al., 2024) extends CameraCtrl (He et al., 2024) for multi-view video generation, but their results are constrained to simple cam- era and object motion. ViVid-Zoo (Li et al., 2024a) extends MVDream (Shi et al., 2023b) for multi-view purposes but is limited to object-centric results with fixed viewpoints. In contrast, our work explores view-integrated attentions for more precise camera control over arbitrary viewpoints and introduces a joint training strategy leveraging data mixtures to improve novel-view performance in complex scenes.
Our main contributions can be summarized as follows,
We propose novel framework, Cavia, for generating multi-view videos with camera controllability. We in- troduce view-integrated attentions, namely crows-view and cross-frame 3D attentions, to enhance consistency across viewpoints and frames.
2.2. Multi-view Image Generation
Early approaches such as MVDiffusion (Tang et al., 2023) focused on generating multiview images in parallel by em- ploying correspondence-avee aware attention mechanisms, en- abling effective cross-view information interaction, par- ticularly for textured scene meshes. Recent approaches like Zero123++ (Shi et al., 2023a), Direct2.5 (Lu et al., 2024), Instant3D (Li et al., 2023), MVDream (Shi et al., 2023b), MVDiffusiont+ (Tang et al., 2024), CAT3D (Gao et al., 2024), and Wonder3D (Long et al., 2024) have in- troduced single-pass frameworks for multiview generation,
We introduce an effective joint training strategy that leverages a curated mixture of static, monocular dy- namic, and multi-view dynamic videos, ensuring ge- ometric consistency, high-quality object motion, and background preservation in the generated results.
Our experiments demonstrate superior geometric and perceptual quality in both monocular video genera- tion and cross-video consistency compared to baseline
2

---

Cavia: Camera-controllable Multi-view Video Diffusion
utilizing multiview self-attention to improve viewpoint con- sistency. Other works, such Sreamrr (Liu et al., 2023b), One-2-3-45 (Liu et al., 2024), Cascade-Zerol 123 (Chen et al., 2023) and ConsistNet (Yang et al., 2024a), in- corporate multiview features into 3D volumes to facilitate 3D-aware diffusion models (Liu et al., 2023a; Watson et al., 2022). Meanwhile, techniques such as Pose-Guided Diffu- sion (Tseng et al., 2023). Era3D (Li et al., 2024b), Epidiff (Huang et al., 2024), and SPAD (Kant et al., 2024) have in- tegrated epipolar-based features to facilitate enhanced view- point fusion within diffusion models. Finally, approaches like V3D (Chen et al., 2024b), IM-3D (Melas-Kyriazi et al., 2024), SV3D (Voleti et al., 2024) and Vivid-1-to-3 (Kwak et al., 2024) leverage priors from video diffusion models to achieve multiview generation with improved consistency. All these methods focus on generating static 3D objects or scenes, while our work introduces vivid object motion into multiview dynamic video generation in complex scenes.
tent video generation. An overview of our framework is provided in Fig. 1.
3.2. Camera Controllable Video Diffusion Model
Preliminaries Our model builds on pre-trained Stable Video Diffusion (SVD) (Stability, 2023). SVD extends Stable Diffusion 2. et al., 2022) by adding tem- poral convolution and attention layers, following the Vide- oLDM architecture (Blattmann et al., 2023). SVD is trained with a continuous-timg noise scheduler (Karras et al., 2022). In each iteration, the training data is perturbed by Gaus- sian noise n(t) ~ of and the diffusion model is tasked with estimating the clean data Thg Po- Let p(x; (f)) denote the marginal probability of noisy data x, = Xo + n(t), the iterative refinement process of dif- fusion model corresponds to the probability flow ordinary differential equation (ODE):
dx = -t((()oot)V logp(x; dt))
(1)
2.3. 4D Generation
Vx oo; refers to the score function, which is pa- rameterized by denoiser Da through Vx logp(x; a) R o) /o2. We follow the EDM-preconditioning framework (Karras et al., 2022; Stability, 2023) and param- eterize Do with neural network Fo as follows.
Recent efforts in 4D generation have explored various methods (Singer et al., 2023; Zhao et al., 2023; Bahmani et al., 2023; Zheng et al., 2023; Ling et al., 2023a) that use score distillation from video diffusion models to opti- mize dynamic NeRFs or 3D Gaussians for text- or image- conditioned scenes. Follow-up works (Jiang et al., 2023; Ren et al., 2023; Yin al., 2023; Ren et al., 2024; Zeng et al., 2024; Pan al., 2024) investigate video-to-4D generation, enabling controllable 4D) scene generation from monocular videos. More recent methods (Liang et al., 2024; Xie et al., 2024; Zhang et al., 2024) utilize video diffusion models to address the spatial-temporal consistency required for effi- cient 4D generation. However, these approaches primarily focus on object-centric generation and face challenges in producing realistic results with complex backgrounds. In contrast, our work emphasizes generating multi-view, 3D- consistent videos for complex scenes.
Do = Cskip + Cou Fo Po(CCnX; Cuoise)
(2)
During training, the network Fo is optimized using denois- ing score matching for Dg:
Da (xo + n o, cond) xx01
(3)
Camera Conditioning Although SVD is pre-trained on various high-quality video and image data, it does not na- tively support precise camera control instructions directly. To address this, we introduce camera conditioning to the model via Plücker coordinates (Jia, 2020), which is widely adopted as position embeddings in 360 unbounded light fields(Sitzmann et al., 2021). Plücker coordinates are de- fined as P (d', o d'), where x is the cross product and d' refers to the normalized ray direction d' = ||d]| Let camera extrinsic matrix be E = [R|T] and intrinsic matrix be K, the ray direction dz.y for 2D pixel located at (ax, y) is formulated d = R.K-K 1( ( + T. These spatial Plücker coordinates are concatenated channel-wise with the original latent inputs of SVD. We enlarge the convolution kernel of the first layer accordingly. The newly introduced matrices are zero-initialized to ensure training stability.
3. Method
3.1. Overview
d
Image-to-video generation takes a single image To as input and outputs a video sequence O1,... On, By introducing camera control, the model additionally takes in sequence of camera information Ci,... C.. which dictates the de- sired viewpoint changes for the output sequence. In the multi-view scenario, we extend each batch of the camera control signal and output video sequence to V sequences. In the following paragraphs, we present our proposed Cavia framework in detail. First, we outline the preliminaries of image-to- video diffusion and describe how camera control- lability is introduced in monocular video generation. Then, we elaborate on the model design for multi-view consis-
We utilize a relative camera coordinate system, where the first frame is positioned at the world origin with an identity matrix for rotation. The following frames are rotated accord- ingly. To stabilize training, we normalize the scale of the
3

---

Cavia: Camera-controllable Multi-view Video Diffusion
Conv3D
Cross- View Attention
Cross- Cross-Frama Frame Attention
Conv3D
Cross-Vies View Attention
Cross- Frame
Attention
View 1
Rearranged: (B w) C
Latent 2
Plücker C
Spatial Attention
View 2
Enc
Cross-View Attention
Rearranged: (B V) (F H C
Image lo
View 1
Cross- View
Cross-Froie Frame
Cross- View
Cross- Cross-Frame Frame Attention
Latent z
Conv3D
Attention
Attention
Conv3D
Attention
Temporal Attention
Plücker C
View
(a) Overall framework
(c) Cross-Frame Attention
Figure 1. An overview of Cavia is shown in (a). We introduce view-integrated attention modules, namely cross-view attentions and cross-frame attentions, which enforce viewpoint and temporal consistency the generated frames, respectively. As illustrated (b) and (c), our view- intterrterttdd attention incorporates additional feature dimensions into the attention mechanism, enhancing consistency across views and frames. The processed latent features are of shape (B F H W) where B, V. F, C refer to the batch size, the number of views, the length of frames, and the feature dimension, respectively.
training sequences to unit scale. This is implemented by resizing the maximum iistance-to-origin in the multi-view camera sequence 1.
of ( (B V W) C) our inflated attention operates on the rearranged shape (B (F w) C) integrating spatial features into the attention matrices. A visualization is provided in Fig. (cc).
Since our rearrange operation only alters the sequence length of the attention inputs without modifying the fea- ture dimensions, we can seamlessly inherit the pre-trained weights from the SVD backbone for our purpose. Thanks to this rearrange operation, our inflated temporal attention calculates the similarity of spatial-temporal features simultaneously, accommodating larger pixel displacements while maintaining temporal consistency.
Cross-frame Attention for Temporal Consistency Vanilla ID temporal attention in the SVD backbone is in- sufficient for modeling large pixel displacements when the viewpoint changes (Shi al., 2023b; Yang al., 2024b). In vanilla 1D temporal attention, attention matrices are cal- culated over the frame number dimension, and latent fea- tures only interact with features from the same spatial loca- tion across frames. This limits information flow between different teapial-temporal locations. While this might not be big issue for video generation with limited motion, viewpoint changes typically cause significant pixel displace- ments, which calls for better architecture for more efficient information propagation.
3.3. Consistent Multi-view Video Diffusion Model
Adding Plücker coordinates for camera control and introduc- ing improved temporal attention allows the video diffusion model to generate reasonably consistent monocular videos. However, for multi-view generation, a monocular video diffusion model that generates samples independently can- not ensure view consistency across multiple sequences. To address this, introduce novel design mechanisms and training strategies to extend the monocular video diffusion model to the multi-view generation task.
To overcome this issue, we inflate the original 1D tem- poral self-attention modules the SVD network into 3D fross-frame temporal attention modules, allowing for joint modeling spatial-temporal feature coherence. The in- flation operation can be achieved by rearranging the latent features before the attention matrix calculations. Consider the latent features of shape (B V F C H W) where B, V, F, C refer to the batch size, the number views, the length of frames, and the feature dimension, respectively, instead of employing attention mechanism on rearranged shape
Cross-view Attention for Multi-view consistency To im- prove cross- view consistency in multi-view videos, we aim to encourage information exchange during the generation

---

Cavia: Camera-controllable Multi-view Video Diffusion
process. Since our temporal cross-frame attention modules already handle intra-view feature connections within each video sequence, we focus exchanging inter-view signals through the spatial cross-view modules. Inspired by MV- Dream (Shi et al., 2023b), we introduce 3D cross-view atten- tion modules, inflated from the spatial self-attention blocks of SVD (Stability, 2023). Specifically, we rearrange the V views such that frames at each corresponding timesteps are concatenated before being sent into the attention modules. In detail, we rearrange the latent features from shape (B V F C H W) to ( (( F) (V H W) C) instead of ( (B V F) (H W) C). A visualization is provided in Fig. 1(b).
ments. For each set of renderings, we assign a shared ran- dom forward-facing starting point for all V views.
To avoid the model overfitting on synthetic images with sim- ple backgrounds, we include portion of data from monocu- lar in-the-wild videos (Wang et al., 2023b; Nan et al., 2024). Training multi-view camera control from monocular videos is extremely challenging. Although CamCo (Xu et al., 2024) and 4DiM (Watson et al., 2024) have explored joint training for monoeular camera-controllabbe video generation, these approaches are unsuitable for multi-view scenarios. The concurrent work CVD (Kuang et al., 2024) explored ho- mography warping to augment the monocular videos into pseudo-mule multi- videos, but the limited realism of these augmentations restricts their ability to generate complex camera and object motion.
Since only the second-to -o-nstt---asttt dimension, representing to- ken length, is extended while other dimensions remain un- changed, our inflated spatial attention can inherit the model weights from the monocular setting. This flexibility allows our model to leverage training data with varying numbers of views and facilitates extrapolation to additional views at inference. To handle multi-view generation, we introduce an additional view dimension to the input data. To maintain workflow simplicity, we absorb the view dimension into the batch dimension during processing of other blocks, ensuring flexibility in handling different numbers of views.
To overcome these issues, we choose to jointly train our model on monocular and multi-view videos to effectively utilize the abundant object motion information from all data sources. We annotate the monocular videos with camera poses using Particle-SfM (Zhao et al., 2022). Since in-the- wild monocular videos often contain noisy or unnatural content, we apply rigorous filtering pipeline remove unsuitable clips. These curated video clips, sourced from Intern Vid (Wang et al., 2023b) and Open Vid (Nan et al., 2024) datasets, provide rich object motion as well as com- plex backgrounds that mitigate the gap between scene-level static data and object-level dynamic data. We rearrange monocular videos as V samples so that all data items can be consumed by the model in the same way without hells and whistles. Thanks to our view-integrated atten- tion modules, which accommodate varying token lengths introduced by the varying view numbers V. the training pro- cess remains unaffected when our data items contain both monocular and multi-view videos.
4. Joint Training Strategy on Curated Data Mixtures
Thanks to the fie--integrated attention mechanism, which allows for inheriting the module weights with arbitrary view- point (V) numbers, our framework can leverage various data sources, including static, multi-view dynamic, and monoc- ular videos. This is hard to achieve in previous methods. In this section, we first illustrate our joint training strategy, followed by details on the curated data mixtures that enable this strategy.
4.2. Data Curation Workflows
4.1. Joint Training Strategy for Videos with Varying Views
We begin by training our model extensively on static video data sourced from various publicly available datasets. Wild- RGBD (Xia et al., 2024) includes nearly 20,000 RGB-D videos across 46 common object categories. MVImgNet (Yu et al., 2023) comprises 219,188 videos featuring objects from 238 classes. DL3DV-IOK (Ling et al., 2023b) pro- vides 7,000 long-duration videos captured in both indoor and outdoor environments. CO3Dv2 (Reizenstein et al., 2021) contains 34,000 turntable-like videos of rigid objects, crowd-sourced by nonexperts using cellphone cameras. Ob- javerse (Deitke et al., 2023b) and Objaverse-XX XLL (Deitke et al., 2023a) exhaustively crawl 10 million publicly avail- able 3D assets. From these, we filtered out low-quality assets, such as those with incorrect textures or overly sim- plistic geometry, yielding high-quality subset of 400,000 assets.
For videos capturing static scenes (Zhou et al., 2018: Yu et al., 2023; Xia et al., 2024; Reizenstein et al., 2021; Deitke et al., 2023b;a), we consider all frames to be tem- porally synchronized. An arbitrary subsequence of length (F-1) x V ++1 1 from the original video can be reformatted into a V V-view sequence with a shared starting frame and F' total frames per view. Static scenes also allow frame order reversal, providing additional augmentation opportunities. To overcome the static issue where camera-controlled video models are only able to generate static scenes as in (Wang et al., 2023c; He et al., 2024), we further prepare multi-view dynamic videos by rendering animatable objects from Ob- javerse (Liang et al., 2024; Jiang et al., 2024). We design random smooth trajectories with diverse elevation and az- imuth changes to avoid overfitting on simple camera move-
5

---

Cavia: Camera-controllable Multi-view Video Diffusion
Similar to Diffusion4D (Liang et al., 2024) and Ani- mate3D (Jiang et al., 2024), we filter the animatable objects from Objaverse's s Sketchfab subset. We exclude objects with excessive motion, which might result in partial ob- servations, as well as nearly static objects with minimal motion. This curation process helps us obtain 19,000 high- quality dynamic assets that can be rendered from arbitrary viewpoints and timesteps, facilitating multi-view video gen- eration. During each training iteration, augment the frames with randomly selected background colors.
era pose estimation is required for in-the-wild images. We configure the COLMAP following previous methods (Deng et al., 2022; Xu et al., 2024) for best few-view performance. A higher COLMAP error rate indicates poorer 3D consis- tency in the input images. Motivated by this, we report COLMAP errors as a measure of the 3D consistency of the frames. Each video retried up to five times to reduce ran- domness. We randomly sample 1,000 video sequences from RealEstate (Zhou et al., 2018) test set for evaluation. Since we have ground truth 3D scenes, we use the ground truth camera pose sequences as the viewpoint instruction of the video model and compare the generated frames against the ground truth images. Similar to prior works (He et al., 2024; Xu et al., 2024), we extract the estimated camera poses and calculate the relative translation and rotation dif- ferences. Specifically, given two camera pose sequences, we convert them to relative poses and align the first frames to world origin. We then measure the angular errors in translation and rotation. Unlike previous works (He et al., 2024; Xu et al., 2024) that calculate the Euclidean distance of translation vectors, we use angular error measurements to ensure the camera pose scales are normalized, addressing scale ambiguity. As shown in Tab. I, we calculate the area under the cumulative error curve (AUC) of frames whose rotation and translations are below certain thresholds (5', 10 20%). Our method significantly outperforms existing baselines.
To improve the model's ability to generate object mo- tion in the presence of complex backgrounds, we prepare monocular videos with camera pose annotations similar to CamCo (Xu et al., 2024). First, we use Particle-SfN (Zhao et al., 2022) estimate the camera poses for randomly sam- pled frames from videos from Intern Vid (Wang et al., 2023b) and Open Vid (Nan et al., 2024). Inspired by CO3D (Reizen- stein et al., 2021) and CamCo (Xu et al., 2024), we remove the videos where SfM fails to register all available frames or produces a point cloud with too few points or too many points. Fig. 4(a) shows the point count statisties. A point cloud with too few points indicates poor frame registration shared 3D representation, while too many points suggest mostly static scene, which is undesirable as we focus on ob- ject motion. Additionally, non-registered frames may indi- cate potential scene changes. We then apply a rigorous filter- ing pipeline to ensure the quality of the video samples used for training. This includes filtering based on aesthetic scores, optical character recognition (OCR), and camera motion classification using optical flow. Videos containing detected character regions are aggressively removed. Fig. 4(b) and (c) present statistics on aesthetic score and camera motion classification results. Videos with low aesthetic scores or those classified as having static camera motion are excluded from the training set. Ultimately, we construct a dataset of 393,000 monocular videos annotated with camera poses. We provide a summary of the data sources used in Fig. 5. More details and analysis are provided in the appendix.
Multi-view Consistency Alongside evaluating the indi- vidual monocular frame pose accuracy using COLMAP- based metrics, we further assess the cross-video consistency of the corresponding frames from generated multi-view videos. We randomly sample 1,000 videos, each with 27 frames, from Realte 10k (Zhou al., 2018) test set and convert each video into two-view sequence with 14 frames per view. The new camera pose sequences are generated by setting the 14th frame as the world origin and positioning the remaining frames relative to it. The scales of the scenes are normalized so that the maximum distance from the ori- gin is 1. Following CVD (Kuang et al.. 2024), we adopt SuperGlue (Sarlin et al., 2020) to find correspondences and estimate the camera poses between each time-aligned set of frames. SuperGlue not only measures angular errors in the rotation and translation but also computes the epipo- lar error of the matched correspondences, We similarly collect the AUC for frame pairs with rotation and transla- tion errors below specific thresholds (5 10", 20%). The epipolar errors for the estimated correspondences are sum- marized to the precision (P) and matching score (MS). As shown in Tab. 2, our method outperforms baselines greatly. The "Real10K" category means that the input images are taken from the corresponding RealEstate 10K test sequence, while the "General" means that the input images are taken from 1,000 randomly sampled images in the test split of our
5. Experiments
In this section, we present experimental results and analysis. Video comparisons are provided on the project webpage for optimal visual evaluation. It is important note that for all qualitative and quantitative evaluations, neither the input images nor the camera trajectories were used during model training.
5.1. Quantitative Comparisons
3D Consistency of Frames We evaluate the 3D consis- tency of the generated videos using COLMAP (Schönberger & Frahm, 2016; Schönberger et al., 2016). COLMAP is widely adopted for 3D reconstruction methods where cam-
6

---

Cavia: Camera-controllable Multi-view Video Diffusion
MotionCarl
CameraCiri
Ours
MotionCurl
CameraCur
Ouss
Figure 2. Qualitative comparisons for 2-view video generations. Each generation consists of two where each row represents sequence of generated frames, with columns showing frames at the same timestep. Neither the image nor the camera trajectories were used during model training. Red dotted lines annotated to highlight object motion. Video results included the project webpage for clearer comparisons.

---

Cavia: Camera-controllable Multi-view Video Diffusion
SVD
Motionaal
Ous
Referacc
SVD
MotionCri
CamnaaC
Ours
Refercnce
Figure Per-video qualitative comparisons. The frame in each reference set is the input image. Neither the image nor camera trajectories were seen during model training. Video results are provided on the project webpage for clearer qualitative comparisons.
8

---

Cavia: Camera-controllable Multi-view Video Diffusion
Table 1. Quantitative comparison for monocular geometry consistency RealEstate 10K
Rot. AUC
Trans. AUC
Methods
FID 4
FVD
COLMAP error.
(@5' /100 /20*)
(@5 /109 /209)
SVD 16.89
139.64
30.3%
14.4/ 35.3
0.2/ 1.0
MotionCtrl
21.09
119.06
55.0%
8.6 13.9 22.2
0.6/ 5.7
CameraCtrl
14.69
105.41
19.3%
/2.9 48.4
0.3 1.3 4.4
Ours
11.43
55.10
14.4%
22.9 / 34.5 50.1
5.1 12.7 24.6
Table 2. Quantitative comparison for 2-view video generation.
Rot. AUC
Trans. AUC f
Scenes
Methods
FID |
FVD
Prec. f
MS. 1
(@5% /209)
(@55/10' /209)
SVD
37.99
296.95
7.9 13.5 28.2
0.2
6.49
4.17
MotionCtrl
29.23
277.05
8.1/ 16.//6.5// 29.4
1.3/53/
11.45
5.90
Real
CameraCtrl
12.57
131.32
56.2
0.6
19.49
11.25
Ours
8.82
94.86
23.9 34.4/7.4/22
3.3 / 10.2 23.5
29.39
15.22
MotionCtrl
47.31
313.92
4../11.//111
0.7 2.4/2//
8.12
3.93
General
CameraCur
26.71
221.23
14.1 / 26.9 /33.2
0.5 1.7
15.13
7.35
Ours
26.12
173.70
19.7 / 32.7 48.4
0.8 2.8 8.7
33.10
19.96
monocular video dataset.
lucinations, introducing artifacts such as merging indoor and outdoor pixels or distorting input pixels to compensate for lack of generation ability. In Fig. 2, observe that MotionCtrl and CameraCtrl tend to generate static scenes without any object motion. Although their methods pro- duce realistic novel views, the synthesized objects remain static. contrast, our method generates vivid object motion while maintaining accurate camera control. We highlight the object motion in Fig. 2 using auxiliary red lines. We encourage readers view the project webpage for optimal visual comparisons.
Visual Quality To assess the frame perceptual quality, we evaluate visual quality using FID (Heusel et al., 2017) and FVD (Unterthiner et al., 2018). FID and FVD mea- sure the feature-space similarity of two sets of images and videos, respectively. In our case, they quantify the distribu- tion distance between the generated frame sequences and the ground-truth frames. We provide monocular evalua- tions in Tab. I and multi-view evaluations in Tab. 2. As shown in these tables, our proposed framework enjoys the best visual quality. For both the "Real1OK" and "General" categories, the ground-truth videos used to calculate these metrics are video sequences corresponding to the input frames. These video sequences are from the test set split of the datasets and seen during training.
5.3. Ablation Studies and Applications
Due the space limit, we refer readers the Appendix for ablation studies and applications of our framework. We provide detailed ablation studies in Sec. D on our proposed framework. Additionally, we explore the 3D reconstruction of our generated frames and four-view generation capabili- ties Sec. E. Videos are included in the project webpage for optimal qualitative comparison.
5.2. Qualtitative Comparison
We provide qualitative comparisons on RealEstate100 (Zhou al., 2018) scenes in Fig. 3 and text-to-image generated images in Fig. 2. As shown in Fig. 3, our method produces videos with precise camera control, whereas MotionCtrl tends to generate overly smooth trajectories that simplify the viewpoint instructions, and CameraCtrl suffers from severe distortions at novel viewpoints. For example, in the first case, the camera instruction involves multiple panning operations, first panning left and then panning right. Still, MotionCtrl only pans left, ignoring the rest of the instruc- tions. CameraCtrl" outputs, particularly in the first two cases, exhibit noticeable distortion, with the walls bending in the later frames. Additionally, in the third and fourth cases, where the camera trajectories cover a long distance, both MotionCtrl and CameraCtrl produce unrealistic hal-
6. Conclusion
In this paper, we propose Cavia, a novel framework for consistent multi-view camera-controllable video generation. Our framework incorporates cross-frame and cross-view attentions for effective camera controllability and view con- sistency. Our model benefits from joint training using static 3D scenes and objects, animatable objects, and in-the-wild monocular videos. Extensive experiments demonstrate the superiority of our method over previous works terms of geometric consistency and perceptual quality.

---

Cavia: Camera-controllable Multi-view Video Diffusion
Impact Statement
Chen, Y., Fang. J., Huang, Y., Yi, T., Zhang, X.,
Xie, I.., Wang, X., Dai, W., Xiong, H., and Tian, Q. Cascade-zerol23: One image highly consistent 3d with self self-promptee nearby views. arXiv preprint arXiv:2312.04424 2023.
The paper explores generating videos from image inputs. Similar to other large models trained on extensive datasets, our model may exhibit certain social biases. Such biases could reinforce stereotypes create misrepresentations in generated videos, thereby shaping public perceptions and potentially causing societal harm. Consequently, generative video models should be applied with caution.
Chen, Z.,, Wang. Y., Wang, F., Wang. Z., and Liu, H. V3d: Video diffusion models are effective 3d generators. arXiv preprint arXiv:2403.06733 2024b.
In this work, we aim to enhance the camera-contro) and motion synchronization capabilities of the video generation model. The base model is trained with NSFW content ex- cluded and is released alongside safety checker. During the fine-tuning stage, the data are sourced from publicly available datasets, with NSFW content similarly excluded. Additionally, the publicly available safety checker main- tained by the Diffusers library provides further protection against harmful use of the video generator.
Deitke, M., Liu, R., Wallingford, M., Ngo, H., Michel, O., Kusupati, A., Fan, A., Laforte, C., Voleti, V., Gadre, S. Y., et al. Objaverse-xl: xx:: A universe of 10m+ 3d objects, arXiv preprint arXiv:2307. 307666633,,, 2023a.
Deitke, M., Schwenk, D., Salvador, J., Weihs, L., Michel, O., VanderBilt, E., Schmidt, L.., Ehsani, K., Kembhavi, A., and Farhadi, A. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142- 13153, 2023b.
References
Deng, K., Liu, A., Zhu, J.-Y., and Ramanan, D. Depth- supervised nerf: Fewer views and faster training for free. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 12882-12891 2022.
Bahmani, S., Skorokhodov, I., Rong. V., Wetzstein, G., Guibas, L.,, Wonka, P., Tulyakov, S., Park, J., Tagliasac- chi, A., and Lindell, D. B. 4d-fy: Text-to-4d generation using hybrid score distillation sampling. arXiv preprint arXiv:2311.17984, 2023.
Gao, R., Holynski, A., Henzler, P., Brussee, A., Martin- Brualla, R., Srinivasan, P., Barron, J. T., and Poole, B. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint 20l.:2405.00344
Bahmani, S., Skorokhodov, I., Siarohin, A., Menapace, W., Qian, G., Vasilkovsky, M., Lee, H.-Y., Wang, C., Zou, J., Tagliasacchi, A., et al. Vd3d: Taming large video diffu- sion transformers for 3d camera control. arXiv preprint arXiv:2407.12781, 2024.
Guo, Y.. Yang. C., Rao, A., Wang, Y., Qiao, Y., Lin, D., and Dai, B. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725. 2023.
Bain, M., Nagrani, A., Varol, G., and Zisserman, A. Frozen in time: A joint video and image encoder for end-to-end retrieval. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1728-1738, 2021.
He, H., Xu, Y., Guo, Y., Wetzstein, G., Dai, B., Li, H., and Yang, C. Cameractrl: Enabling camera control for text- to-video generation. arXiv preprint arXiv:2404.02101,
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., and Kreis, K. Align your latents: High- resolution video synthesis with latent difffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22563-22573 2023.
2024.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Charatan, D., Li, S., Tagliasacchi, A., and Sitzmann, V. pixelsplat: 3d gaussian splats from image pairs for seal- able generalizable 3d reconstruction. arXiv preprint arXiv:2312. 2023.
Ho, J., Chan, W., Saharia, C., Whang. J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arxiv:2210.003003. 2022a.
Chen, T.SS., Siarohin, A., Menapace, W., Deyneka, E., Chao, H.-w., Jeon, B. E., Fang, Y., Lee, H.-Y., Ren, J., Yang, M.-H., et al. Panda-70m: Captioning 70m videos with multiple cross-modalits teachers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13320-13331, 2024a.
Ho, J.. Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-866 8646, 2022b.
10

---

Cavia: Camera-controllable Multi-view Video Diffusion
Hou, C., Wei, G., Zeng, Y., and Chen, Z. Training-free camera control for video generation. arXiv preprint AXiv:2406.101200 2024.
with diffusion model. arXiv preprint arXiv:2406.08659, 2024a.
Li, J., Tan, H., Zhang, Xu, Z., Luan, F., Xu, Y., Hong, Y., Sunkavalli, K., Shakhnarovich, G., and Bi, S. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. http:://ariu.org/a//331..... 2023.
Hu, J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv: N00:2/06.0966. 066006 2021.
Huang, Z.,, Wen, H., Dong, J., Wang, Y., Li, Y., Chen, X., Cao, Y.PP. Liang, D., Qiao, Y., Dai, B., et al. Epidiff: Enhancing multi-view synthesis via localized epipolar- constrained diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9784-9794, 2024.
Li, P., Liu, Y., Long, X., Zhang, F., Lin, C.,, Li, M., Qi, X., Zhang, S., Luo, W., Tan, P., et al. Era3d: High-resolution multiview diffusion using efficient row-wise attention. arXiv preprint arXiv:2405.11611 2024b.
Li, R., Zheng, C., Rupprecht, C.,, and Vedaldi, A. Puppet- master: Scaling interactive video generation as a mo- tion prior for part-level dynamics. arXiv preprint arXiv:2408 240.0641,,. 2024c.
Jia, Y.-B. Plücker coordinates for lines in the space. Problem Solver Techniques for Applied Computer Science, Com-S- 477/577 Course Handout, 3, 2020.
Liang, H., Yin, Y., Xu, D., Liang, H., Wang, Z., Platamiotis, K. N., Zhao, Y., and Wei, Y. Diffusion4d: Fast spatial- temporal consistent 4d generation via video diffusion models. arXiv preprint arXiv:2405.16645 2024.
Jiang, Y., Zhang, I.., Gao, J., Hu, W., and Yao, Y. Consis- tent4d: Consistent 360 {deeg dynamic object generation from monocular video. arXiv preprint arXiv:2311.0284,, 2023.
Ling, H., Kim, S. W., Torralba, A., Fidler, S., and Kreis, K. Align your gaussians: Text-to-4d with dynamic 3d gaussians and composed diffusion models, arXiv preprint arXiv:2312.13763 2023a.
Jiang, Y., Yu, C., Cao, C., Wang, F., Hu, W., and Gao, Animate3d: Animating any 3d model with multi-view video diffusion. arXiv preprint arXiv:2407.I 1338, 2024.
Kant, Y., Siarohin, A., Wu, Z., Vasilkovsky, M., Qian, G., Ren, J., Guler, R. A., Ghanem, B., Tulyakov, S., and Gilitschenski, I. Spad: Spatially aware multi-view dif- fusers. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10026- 10038, 2024.
Ling, L., Sheng, Y., Tu, Z., Zhao, W., Xin, C., Wan, K., Yu, L., Guo, Q. Yu, Z., Lu, Y., et al. D13dv-10k: A large- scale scene dataset for deep learning- based 3d vision. arXiv preprint arXiv:23 16256, 2023b.
Liu, M., Xu, C., Jin, H., Chen, I.., Varma T, M., Xu, 7.,, and Su, H. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization. Advances in Neural Information Processing Systems, 36, 2024.
Karras, T., Aittala, M., Aila, T., and Laine, Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35: 26565-26577, 2022.
Liu, R., Wu, R., Van Hoorick, B., Tokmakov, P., Zakharov, S., and Vondrick, C. Zero-1-to-3: Zero-shot one image to 3d object. arXiv preprint arXiv:2303. //000//////.... /328, 2023a.
Kerbl, B., Kopanas, G., Leimkühler, T., and Drettakis, G. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphics (ToG), 42(4):1-14, 2023.
Liu, Y., Lin, C., Zeng, Z., Long, X., Liu, L.,, Komura, T. and Wang, W. Syncdreamer: Generating multiview- consistent images from a single-view image. arXiv preprint arxiv:2309.0345. 2023b.
Kuang. Z., Cai, S., He, H., Xu, Y., Li, H., Guibas, L., and Wetzstein, G. Collaborative video diffusion: Consis- tent multi-video generation with camera control. arXiv preprint arXiv:2405. 17414, 2024.
Long, X., Guo, Y.C.., Lin, C., Liu, Y., Dou, Z., Liu, L., Ma, Y., Zhang, S.-H., Habermann, M., Theobalt, C., et al. Wonder3d: Single image to 3d using cross-domain dif- fusion. In Proceedings of the IEEF/VV Conference on Computer Vision and Pattern Recognition, pp. 9970- 9980, 2024.
Kwak, J.-g. Dong, E., Jin, Y., Ko, H., Mahajan, S., and Yi, K. M. Vivid-I-to-3: Novel view synthesis with video diffusion models. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Patlern Recognition, pp. 6775-6785. 2024.
the
Lu, Y., Zhang, J., Li, S., Fang, T., McKinnon, D., Tsin, Y., Quan, L., Cao, X., and Yao, Y. Direct2. 5: Diverse text-to-3d generation via multi-view 2.5 d diffusion. In
Li, B., Zheng, C., Zhu, W., Mai, J.. Zhang, B., Wonka, P., and Ghanem, B. Vivid-zoo: Multi view video generation

---

Cavia: Camera-controllable Multi-view Video Diffusion
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C.,
2024. Melas-Kyriazi, L., Laina, I., Rupprecht, C., Neverova, N., Vedaldi, A., Gafni, O., and Kokkinos, F. Im-3d: Iterative multiview diffusion and reconstruction for high-quality 3d generation. International Conference on Machine Learning, 2024, 2024.
Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08440 2022.
Shi, R., Chen, H., Zhang. 7.,, Liu, M., Xu, C., Wei, X., Chen, L., Zeng, C., and Su, H. 1ero123++: single image to consistent multi-view diffusion base model. arXiv preprint 1rXiv:2310.15/10 2023a.
Nan, K., Xie, R., Zhou, P., Fan, T., Yang, Z., Chen, Z., Li, X., Yang, J., and Tai, Y. Openvid- Im: A large-scale high-quality dataset for text-to-vides -ivdoo generation. arXiv preprint arXiv:2407.02377, 2024.
Shi, Y., Wang, P., Ye, J., Long, M., Li, K., and Yang. X. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023b.
OpenAL. Sora - openai.
htttps///opeai.coom index sora// (Accessed on 09/22/2024).
Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang. S., Hu, Q. Yang, H., Ashual, O., Gafni, O., al. Make-a- video: Text-to- generation without text-video data. arXiv preprint arXiv:2209. /4792, 2022.
Pan, Z., Yang, Z., Zhu, X., and Zhang, L. Fast dynamic 3d object generation from single-view video. arXiv preprint arXiv:2401.08742 2024.
Singer, U., Sheynin, S., Polyak, A., Ashual, O., Makarov, Kokkinos, F., Goyal, N., Vedaldi, A., Parikh, D., Johnson, J., al. Text-to 4d dynamic scene generation. arXiv preprint arXiv:2301. /280, 2023.
Reizenstein, J., Shapovalov, R., Henzler, P., Sbordone, L., Labatut, P., and Novotny, D. Common objects in 3d: Lcrge-scale learning and evaluation of real-life 3d cate- gory reconstruction. Proceedings of the IEEE/CVF international conference on computer vision, pp. 10901- 19911,0021,
Sitzmann, V., Rezchikov, S., Freeman, B., Tenenbaum, J., and Durand, Light field networks: Neural scene rep- resentations with single-evaluation rendering. Advances in Neural Information Processing Systems, 34:19313- 19325, 2021.
Ren, J., Pan, L., Tang, J., Zhang, C., Cao, A., Zeng, G., and Liu, Z. Dreamgaussian4d: Generative 4d gaussian splatting. arXiv preprint arXiv:2312.17142 2023.
Stability. Stable video diffusion: Scaling latent video diffu- sion models large datasets, 2023.
Ren, J., Xie, K., Mirzaei, A., Liang, H., Zeng, X., Kreis, K., Liu, Z., Torralba, A., Fidler, S., Kim, S. W., al. L4gm: Large 4d gaussian reconstruction model. arXiv preprint arXiv:2406. 10324, 2024.
Tang, S., Zhang, F., Chen, J., Wang, P., and Furukawa, Y. Mvdiffusion: Enabling holistic multi-view image genera- tion with correspondence aware diffusion. arXiv, 2023.
Rombach, R., Blattmann, A.. Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffuuson models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition. pp. 10684-10695, 2022.
Tang, S., Chen, J., Wang, D., Tang, C., Zhang, F., Fan, Y., Chandra, V., Furukawa, Y., and Ranjan, R. Mvdif- fusion++: A dense high-resolution multi-view diffusion model for single or sparse-vies view 3d object reconstruction. arXiv preprint arXiv:2402.12713. 2024.
Sarlin, P.-E., DeTone, D., Malisiewicz, T., and Rabinovich, A. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp- 4938-4947, 2020.
Tseng, H.-Y., Li, Q., Kim, C., Alsisan, S., Huang, J.-B., and Kopf, Consistent view synthesis with pose-guided diffusion models. In CVPR, 2023.
Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., and Gelly, S. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv: 1812.01717 2018.
Schönberger, J. L. and Frahm, J.-M. Structure-rrmmmmoo revisited. In Conference Computer Vision and Pattern Recognition (CVPR), 2016.
Voleti, V., Yao, C.-H., Boss, M., Letts, A., Pankratz, D., Tochilkin, D., Laforte, C., Rombach, R., and Jampani, V. Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion. arXiv preprint arXiv:2403. 2008 2024.
Schönberger, J. L., Zheng, E., Pollefeys, M., and Frahm, J.-.. Pixelwise view selection for unstructured multi- view stereo. In European Conference on Computer Vision (ECCV), 2016.
12

---

Cavia: Camera-controllable Multi-view Video Diffusion
Wang, W., Yang, H., Tuo, Z., He, H., Zhu, J., Fu, J., and Liu, J. Videofactory: Swap attention in spatiotemporal diffusions for text-to-video generation. 2023a.
large-scale dataset of multi-view images. In Proceedings the IEEE/CV Conference Computer Vision and Pattern Recognition, pp. 9150-9161, 2023.
Wang, Y., He, Y., Li, Y., Li, K., Yu, J., Ma, X., Chen, X., Wang, Y., Luo, P., Liu, Z., et al. Internvid: large- scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.0694 2023b.
Zeng, Y., Jiang, Y., Zhu, S., Lu, Y., Lin, Y., Zhu, H., Hu, W., Cao, X., and Yao, Y. Stag4d: Spatial-temporal anchored generative 4d gaussians. arXiv preprint arXiv: Stti::2003.4993333 14939, 2024.
Wang, Z., Yuan, Z., Wang, X., Chen, T., Xia, M., Luo, P. and Shan, Motionctrl: A unified and flexible motion controller for video generation. arXiv preprint arXiv:2312.036411 2023c.
Zhang. H., Chen, X., Wang, Y., Liu, X., Wang, Y., and Qiao, Y. 4diffusion: Multi-view video diffusion model for 4d generation. arXiv preprini arXiv:2405.20674 2024.
Zhang, L. and Agrawala, M. Adding conditional con- trol to text-to-image diffusion models. arXiv preprint arXiv:2302.055433 2023.
Watson, D., Chan, W., chi, A., and Norouzi, M. Novel view synthesis with dif- fusion models. arXiv preprint arXiv:2210.04628 2022.
R., Ho, J., Tagliasac-
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., and Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
Watson, D., Saxena, S., Li, L., Tagliasacchi, A., and Fleet, D. J. Controlling space and time with diffusion models. arXiv preprint arXiv:2407.07860, 2024.
Zhao, W., Liu, S., Guo, H., Wang, W., and Liu, Y.-J. Parti- clesfm: Exploiting dense point trajectories for localizing moving cameras in the wild. In European Conference on Computer Vision, pp. 523-542. Springer, 2022.
Xia, H., Fu, Y., Liu, S., and Wang, X. Rghd objects in the wild: Scaling real-world 3d object learning from rgb-d videos. arXiv preprint arXiv:2401. 12592, 2024.
Xie, Y. Yao, C.-H., Voleti, V., Jiang, H., and Jam- pani, V. Sv4d: Dynamic 3d content generation with multi-framm and multi-view consistency. arXiv preprint arXiv:2407.1 17470, 2024.
Zhao, Y., Yan, Z., Xie, E., Hong, L.,, Li, Z.,, and Lee, G. H. Animate124, Animating one image to 4d dynamic scene. arXiv preprint arXiv:231 1.t:2311..160... 14603, 2023.
Zheng, Y., Li, X., Nagano, K., Liu, S., Hilliges, O., and De Mello, S. A unified approach for text-and image-guided 4d scene generation. arXiv preprint arXiv:2311.165.. 2023.
Xu, D., Nie, w., Liu, C., Liu, S., Kautz, J., Wang, Z., and Vahdat, A. Camco: Camera-controllable 3d- consistent image-to-video generation. arXiv preprint arXiv:2406.025009 2024.
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., and Snavely, N. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv: 1xXiv:/800 1805.09817, 2018.
Yan, W. Zhang, Y., Abbeel, P., and Srinivas, A. Videogpt: Video generation using vae and transformers. arXiv preprint arXiv:2 104. 10157, 2021.
Yang, J., Cheng, 7.,, Duan, Y., Ji, P., and Li, H. Consistnet: Enforcing 3d consistency for multi-view images diffusion. In Proceedings of the IEEE/CVF Conference Com- puter Vision and Pattern Recognition, pp. 7079-7088, 2024a.
Yang. Z. Teng. J., Zheng, W., Ding, M. Huang, S., Xu, J. Yang, Y., Hong, W. Zhang, X., Feng, G., et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint 47X60:2408.06072, 2024b.
Yin, Y., Xu, D., Wang. Z.. Zhao, Y., and Wei, Y. 4dgen: Grounded 4d content generation with spatial-temporal consistency. arXiv preprint urXiv:2312./7223 2023.
Yu, X., Xu, M., Zhang, Y., Liu, H., Ye, C., Wu, Y., Yan, Z.,, Zhu, C., Xiong, Z.,, Liang, T., al. Mvimgnet: A
13

---

Cavia: Camera-controllable Multi-view Video Diffusion
120000
100000
250000
80000
200000
60000
Counts
Counts
60000
150000
40000
40000
100000
20000
20000
50000
statcc
dynamic
zoom in
pan right
zoum oul up
uitt down
pan left
0
0
20000
40000
60000
0
2
8
Points
Acsthetic
Motion
(a) Point Cloud Size
(b) A esthetic Score
(c) Camera Motion Classification
Figure Statistics of curated monocular video dataset. We provide visualizations of (a) point cloud size, (h) aesthetic score, and (c) camera motion classification result.
A. Additional Implementation Details
Our training divided into static stage and dynamic stage. Our static stage is trained for around 500k iterations and our dynamic stage trained for roughly 300k iterations. The effective batch size is 128 and the learning rate is le-4. Our video length is 14 frames for each view with the first frame shared across views. Our model is fine-tuned 256 x 256 spatial resolution from the SVD 1.0 checkpoint. The training data prepared by first center-cropping the original videos and then resizing each frame to the shape of 256 x 256. In the dynamic stage, 30% of iterations are used to train monocular videos. During static training, the strides of frames are randomly sampled in the range of [1, 8]. For monocular videos, the strides are sampled in the range [1, 2J. For dynamic multi-view object renderings, the strides are fixed to to use all rendered frames since we already introduced randomness frame rate during rendering. At inference time, the decoding chunk is 14 so all frames are decoded altogether. We sample 25 steps to obtain all our results.
B. Additional Data Curation Details
In this section, we provide additional details on our data processing and curation pipelines.
WildRGBD
MV WVllggett
Static 3D Objects Our static objects data comprises milti-view images rendered from the Objaverse (Deitke et al., 2023b) and Ahjaverse-XI.(Deinnss
20,000
Vid
219,000
et al., 2023a) dataset. Similar to InstantMesh, we use a filtered high- quality subset of the original dataset train our model. The filtering goal is remove objects that satisfy any of the following criteria: (i) objects without texture maps, (ii) objects with rendered images occupying less than 10% of the view from any angle, (iii) including multiple separate objects, (iv) objects with no caption information provided by the Cap3D dataset, and (v) low-quality objects. The objects determined based on the presence of tags such as "lowpoly" and its vari- ants (e.g., "low poly") in the metadata. By applying our filtering criteria, we curated approximately 400k high- quality instances from the initial pool of 800k objects in the Objaverse dataset.
RealEstate
CO3D
10,000
DESDV-IOK
7000
330000
000
400,000
OpeaVid
DifinsionDD
Objaverse
Figure 5. Sources our training vidcos. We visualize the numbers of raw vidcos from each source. During training, perform aug- mentations to the raw video sequences to avoid overfitting on certain camera motions.
classification of "low-quality" objuectts""ojecsss
For each 3D object, we use Blender's EEVEE renderer to render an 84 14-frame RGBA orbit at 512 x 512 resolution. we adaptively position the camera distance sufficient to ensure that the rendered object content makes good and consistent
14

---

Cavia: Camera-controllable Multi-view Video Diffusion
use of the image extents without being clipped in any view. For each frame, the azimuths can be irregularly spaced, and the elevation can vary per view. Specifically, the sequence of camera elevations for each orbit is obtained from a random weighted combination of sinusoids with different frequencies. The azimuth angles are sampled regularly, and then a small amount of noise is added to make them irregular. The elevation values are smoothed using simple convolution kernel and then clamped to a maximum elevation of 89 degrees.
Static 3D Scenes Our static scenes data are sourced from RealEstate 10k (Zhou et al., 2018), WildRGBD (Xia et al., 2024), MVImgNet (Yu et al., 2023), CO3Dv2 (Reizenstein et al., 2021), and DL3DV-IOK (Ling et al., 2023b). For RealEstate 10k, we use the train/test split released by PixelSplat (Charatan et al., 2023). During training, we sample every original frames to construct the training sequences. For DL3DV- 10K, we construct training sequences from the publicly available 7k subset. Since each video is very long for the 1L.3D--10 dataset, we offline randomly sample multiple sequences from a single ground truth video to obtain multiple training data items. For CO3Dv2, we remove the video sequences that contain whole-black images to avoid temporally inconsistent frames. For WildRGBD and MVImgNet we use all classes available and removed sequences whose lengths are not enough for two-view training (shorter than 27 frames).
Dynamic 3D Objects Our dynamic 3D objects are similarly rendered as the static 3D objects. The filtering pipelines remain mostly the same as the static objects, except that we introduce additional workflows to consider object motion. Inspired by previous works (Liang et al., 2024; Jiang et al., 2024; Li et al., 2024a) that employ animatable objects from Objaverse. We render multiple fixed-view videos to examine the motion quality of the objects. We utilize lpips (Zhang et al., 2018) to measure the similarity of nearby frames and consider an object to be static if lpips similarity is above a certain threshold. Additionally, we render the alpha masks of the object and use this as indicator of whether the object has moved out of the visible regions. Consequently, we remove objects with too large or sudden movements as well as objects with little-to-no motion. These filterings result in 19,000 objects. Our rendering strategy is also very similar to that of static 3D objects, introducing random elevation and azimuth changes to complicate the trajectories, except that we additionally introduce a random frame stride at rendering to augment the object motion. The stride is sampled individually for each object from the range [1, 3]. larger the stride leads to renderings with faster object motion.
Monocular Videos Our monocular video filtering pipeline involves filtering according to Particle-SfM output, OCR, aesthetic score, and camera motion. As mentioned in Sec. 4.2, we first attempt to annotate the camera poses for the video frames using Particle-SfM (Zhao et al., 2022). Take Intern (Wang et al., 2023h) an example, roughly 10 million video clips are processed and around 3 million samples are successfully processed by Particle-SIM For each video, we start from the first frame and randomly select a frame stride of 1 or 2. The total number of images sent to Particle-SfM is 32 images. Our point count filtering is empirically implemented as a cut-off at 1,000 points and 40,000 points. Point clouds with too few points are removed due to the concern that the frames are poorly registered. Point clouds with too many points are avoided because their limited object motion. This aggressive filtering results in around million samples for further processing. We then evaluate all the video clips using OCR detection algorithms and remove the samples whose detected text regions are larger then 10-4 of the image resolution (i.e. 6 pixels). This process results in 604,000 samples. The next step is filtering with aesthetic scores and videos with aesthetic score annotations smaller than 4 are removed. 467,000 videos are left after these filtering process. Finally, we employ camera motion classifier extended from the Open-Sora pipeline". The main motivation is that optical-flor fllw on consecutive frames can be summarized to global motion vector, assuming the most parts of the scene is moving in a uniform direction. Optical flow is first obtained using cv2 www.cllccc..... www.cppticclll. wwweccaaaaaaaaaaeeeeeee for each consecutive frame pairs. Then, the magnitudes and directions are calculated via cv2. cart ar. These magnitudes and directions are classified into 8 categories: static, zoom out, zoom in, pan left, tilt up, pan right, tilt down, and unknown. The results of the frame pairs are summarized to obtain the final result of each video clip. When a certain type appears more than 50%, the type for the whole video clip is determined directly. We aggressively classify video clip as static if any of its frame pairs is categorized into static or unknown. Finally, we obtain 355,000 clips that satisfy our needs. The process is similarly applied to Open Vid (Nan et al., 2024)'s ' Panda-70M subset (Chen et al., 2024a) and we obtained 38,000 clips. In summary, our monocular video dataset consists of 393,000 clips.
"https: //tittbbttttt////// com
15

---

Cavia: Camera-controllable Multi-view Video Diffusion
Table Ablation Studies each of our introduced modules. "w/o Plücker" refers replacing the Plücker coordinate conditioning with one- di-eesiioonl conditioning as in MotionCul. "w/o Cross-framee refers to replacing the ftass-frame attention with vanilla ID temporal attention. "w/o Cross-view refers replacing Cross-view attention with vanilla spatial attention. "Ours (Static)" means the model only trained static video datasets. "Ours (w/o Mono)" means that the model is fine-tuned on synthetic multi-view datasets, but is not trained with monocular video datasets. "Ours (Full)" means that model is trained on all available data sources.
Rot. AUC
Trans. AUC 1
Scenes
Methods
FID
FVD4
Prec. 1
MS.
@55% /10% /209)
@@5% (050/109 /209)
w/o Plücker
12.75
195.84
12.1/219/35.5
1.6/588/66.
14.74
10.02
w/o Cross-frame
17.04
154.54
21.4 21.4/34.8// 50.1
3.8/11.1/24,
25.67
12.70
Real10K
w/o Cross-view
9.45
106.82
22.8
36.7//
2.7/8.7
27.57
14.65
Ours
8.82
94.86
2 /7.4//2.9
3.3/ 23.5
29.39
15.22
w/o Cross-frame
71.39
249.02
9.8/ 19.1//23
0.5/
13.20
8.97
w/o Crews-view
30.89
246.68
42.9
1.2/ 12.2
17.58
9.59
General
Ours (Static)
27.20
185.58
15.9 44.1
1.4/4.6 12.9
21.75
12.04
Ours (w/o Mono)
35.79
243.05
15.0 42.6
0.3 1.3 /.2
18.55
10.78
Ours (Full)
26.12
173.70
19.7 32.7 48.4
0.8 / 2.8 8.7
33.10
19.96
Model
Plücker Coordinates
Cross-frame Attention
Cross-view Attention
w/o Plücker
w/o frass-frame
w/o Cross-view
Ours
Table 4. Illustration of model variants in ablation studies.
C. Evaluation Details
For MotionCtrl and CameraCtrl, we use the open-source checkpoints trained from SVD released by the authors. These check- points are designed for image-to-vides video tasks so we can have fair comparisons. We use "clean-fid" fid" and "common-metriee video-quality"t for obtaining FID and FVD, respectively. Our FVD results are reported in VideoGPT (Yan et al., 2021) format. Our COLMAP is configured following DSNeRF (Deng et al., 2022) and (Xu et al., 2024) to improve the few-view reconstruc- tion performance. Concretely speaking, we enable --Si wtMaMaaaiingg max__um_mas feature matching. To ensure that the SfM results best align with videos, we set - ------mmeRedee single camera 1 since most videos in our datasets consist of frames captured from single camera.
65536 to support robust
D. A blation Studies
In this section, we conduct extensive evaluations for ablation studies. We provide video comparisons on the project webpage. We provide thorough quantitative comparisons in Tab. 3 to illustrate the importance of our proposed components. In Tab. 4, illustrate the differences between model variants used for ablation studies. The models are evaluated using RealEstate 10K camera trajectories. For the "Real1OK" and "General" categories, the testing images are from our test set split of RealEstate IOK and Intern Vid, respectively. Our full model enjoys the best perceptual quality and geometric consistency.
We first examine the importance of Plücker coordinates conditioning and the cross- fram- attention modules. As shown in Fig. 6, model variants without cross cross-frame attention contains severe distortion artifacts, such as the bent walls. The model variant without Plücker coordinates results in simplified camera motion that ignores the complex camera viewpoint instructions.
We then evaluate the model variant without cross-view attention. As shown in Fig. 7, observe that removing the cros-view attention module results in multiple individual video samples that contain different object motions. For example, the penguin moves differently in the first case, and the wood sticks in the fire appear differently in the second case. This
phttpa://githii com www./aaaaaarr//eeaeeeeeooe fhttps: http:s:///tt //gbthub. com/ homm/JunyacHu/ccoo compooooooooooooooo////ooo http://cooooommroocceeeee www.restttttooemo sttpriissoooiieeuaaaaaaala
16

---

Cavia: Camera-controllable Multi-view Video Diffusion
behavior is not desirable because our goal is obtain multiple videos from different camera paths of the same scene.
Finally, examine the importance of monocular video joint training strategy. As shown in Fig. 8, we observe that when overfitting on dynamic objects from Objaverse, the generated results tend to contain frames with simplified backgrounds. This is mainly because. during the training, all data samples from Objaverse are implemented with single random color backgrounds. Our model benefits from joint training on monocular videos and preserves the ability to generate complex backgrounds when object motion is present.
E. Applications
In this section, we provide additional results on four-view inference and 3D reconstruction of our generated frames.
E.1. Advancing to Four Views at Inference
Our cross-view attention design enables us extrapolate more views straightforwardy inference time. This design is more efficient compared with the concurrent work CVD (Kuang al., 2024), which requires the enumeration of viewpoint pairs at inference time. We conduct side-by-side comparison for 4-view generation in Fig. 9. Our method enjoys better consistency and shows more realistic results than CVD (Kuang et al., 2024). In comparison, CVD tends to produce artifacts at border regions. For example, the structure of the wall (first case) and the window (second case) change when the viewpoint changes. The results from CVD are taken from their author's website. We provide video comparisons on the project webpage. We also provide more 4-view generation results from Cavia on the project webpage.
E.2. 3D Reconstruction Generated Frames
We further perform 3D reconstruction on our generated frames. We render our reconstructed 3D Gaussians from an elliptical trajectory consisting of 16 novel views. We provide side-by-side comparison with the concurrent work CVD (Kuang al., 2024) in Fig. 10. Compared with the results of CVD, frames more geometrically consistent and result in clearer 3D reconstruction and fewer floaters. For example, the results from CVD produce floaters on the cupboard regions and generate blurry artifacts for the wall and the TV due to inconsistencies. We provide video comparisons on the project webpage for clearer comparisons. We also provide additional 3D reconstruction results of Cavia's generated frames on the project webpage.
F. Limitations
Our framework has limited ability to generate large object motion, mainly due to the limitation of the base video generator SVD (Stability, 2023). We will explore better base models in future works. Moreover, our data curation pipelines assume a simple camera model using shared camera intrinsic across frames. While enabling easier data preparation, this limits our model from generalizing to complex camera intrinsic changes at inference time, which is widely adopted in cinematography. Additionally, for simplicity, our framework trained with normalized scales of scenes, which can be further improved potentially calibrated with metric scale. We will explore calibration techniques for better quality if gene-geeneal metric depth estimator becomes publicly available.
G. Additional Dynamic Results
We provide additional dynamic results in Fig. 11. The GIF version is provided on the project webpage.
H. Data Samples
We provide visualizations for the annotations from Particle-SIM in Fig. 12. The good one an example of high-quality pose estimation result, while the bad one is failure example.
We also provide examples of each data source in Fig. 13. Thanks to our joint training strategy, our model supports data samples with arbitrary viewpoint numbers. As a result, our model is jointly trained on static scenes, dynamic objects, and monocular videos.
17

---

Cavia: Camera-controllable Multi-view Video Diffusion
I. Motion Comparison with SVD
We provide side-by-side comparisons with our base model (SVD). As shown in Fig. 14, the frames generated by our method exhibit rich object motion that is comparable to. better than, that of SVD. For example, our penguin, giraffe, and bird examples are behaving realistically, while SVD presents limited object motion.
18

---

Cavia: Camera-controllable Multi-view Video Diffusion
Without Plücker
(b) Without Cross-frame Attention
(c) Full Model
(d) Reference
Figure 6. Ablation studies on Plücker coordinates and Cross-frame Attention. Vidco results provided on the project webpage for clearer qualitative comparisons.
19

---

Cavia: Camera-controllable Multi-view Video Diffusion
Without Cross-view Attention
(b) With Cross-view Attention
Figure 7. Ablation studies on Cross-view Attention. Video results are provided on the project webpage for clearer qualitative comparisons.
(a) Without Monocular Joint Training
(b) With Monocular Joint Training
Figure 8. Ablation studies on the joint training strategy on monocular videos. Video results are provided on the project webpage for clearer qualitative comparisons.
20

---

Cavia: Camera-controllable Multi-view Video Diffusion
(a) CVD
(b) Ours
Figure Four-view video comparison. The result CVD taken from their website. CVD tends to generate black border pixels, potentially due to its homography warping augmentations during training. In comparison, our method produces frames with better geometric consistency and perceptual quality.
21

---

Cavia: Camera-controllable Multi-view Video Diffusion
(a) CVD
(b) Ours
Figure 10. 3D Reconstruction comparison. We render the reconstructed 3D Gaussians from elliptical trajectory consisting of 16 novel views. The result CVD taken from their website. CVD's reconstruction results suffer from floaters and blurry artifacts due to the inconsistency in their generated frames. In comparison, our method produces sharper results with clearer visual quality.
22

---

Cavia: Camera-controllable Multi-view Video Diffusion
Figure 11. We provide additional dynamic results. The GIF version is provided on the project webpage.
23

---

Cavia: Camera-controllable Multi-view Video Diffusion
Good
Bad
Good
Bad
Figure 12. Examples of the annotations produced by Particle-SIM. We add black boxes to the original frames to avoid revealing the identity.
Static Scene
Dynamic Object
Monocular Video
Figure Examples our data samples from various sources.
24

---

Cavia: Camera-controllable Multi-view Video Diffusion
Ours
SVD
Ours
SVD
Ours
SVD
Ours
SVD
Figure 14. We compare the object motion against base model (SVD). The frames generated by our method exhibit rich object motion that is comparable to, if not better than, that of SVD. For example, our penguin, giraffe, and bird examples are behaving realistically, while SVD presents limited object motion.
25


Reviews:
[{"review_id": "W06Io2rySE", "reviewer": "Reviewer_RKXa", "summary": "This paper introduces a novel framework named Cavia for generating multi-view videos with camera controllability.\n\nThe primary contributions consist of two key components: 1) Cross-view and cross-frame 3D attention mechanisms designed to enhance consistency across different viewpoints and temporal frames. 2) A joint training strategy that effectively utilizes a carefully curated combination of static, monocular dynamic, and multi-view dynamic videos to ensure geometric consistency, realistic object motion, and background preservation.\n\nThe experimental results demonstrate superior geometric accuracy and perceptual quality compared to existing baseline methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer RKXa for their positive evaluation of the technical novelty and superior performance of our method. As suggested, we will enhance the presentation of our method and figures in the revised draft. Formal mathematical equations will be added to clarify the computation of the proposed attention modules, and the typos in the abstract will be corrected as recommended. Below, we provide detailed responses to the additional comments:\n\n**Q1. Additional quantitative ablation studies should be included in the main text.**\n\nA1. Due to space limitations, we initially placed the ablation studies in the supplementary material. However, we are happy to incorporate these studies into the main manuscript as suggested.\n\n**Q2. Figure 1(c) lacks textual annotations.**\n\nA2. Thank you for pointing this out. Your understanding is correct: V (View), F (Frame), H (Height), and W (Width) denote the respective dimensions. We used purple and orange to indicate corresponding blocks in Fig. 1(a). We apologize for any confusion and will update Fig. 1 with clear textual annotations and a legend to improve readability.\n\n**Q3. Figure 1(a) lacks sufficient detail. Specifically:(1) The inputs to the cross-attention mechanism are not clearly indicated. (2) The roles of the key, query, and value are not labeled or explained. (3) There are no formal equations provided to illustrate how the two types of cross-attention (cross-view and cross-frame) are calculated.**\n\nA3. Thank you for raising these important points. Our cross-view and cross-frame attention modules are both self-attention mechanisms, where the key, query, and value are derived from the same input features. We will clarify this in the revised draft and will also include formal equations to illustrate the computation of both cross-view and cross-frame attention modules.", "claims_and_evidence": "The claims presented in the paper are intuitively sound. However, the experimental results are insufficient to comprehensively validate the individual contributions of each component of the proposed method, including the cross-view 3D attention mechanism, the cross-frame 3D attention mechanism, and the joint training strategy.", "methods_and_evaluation_criteria": "Yes. The effectiveness of the proposed method is primarily validated using the evaluation metrics and benchmark datasets presented in Table 1 and Table 2.", "theoretical_claims": "There is no theoretical claim.", "experimental_designs_or_analyses": "Yes. There are no obvious issues in the experimental design and analysis.", "relation_to_broader_scientific_literature": "The key constributions are related to multi-view image generation such as V3D[1], IM-3D[2], SV3D[3] and Vivid-1-to-3[4]. \n\nThe primary distinction lies in that these methods focus on generating static 3D objects or scenes, whereas this work introduces vivid object motion into multi-view dynamic video generation within complex scenes.\n\n[1] V3d: Video diffusion models are effective 3d generators, arxiv 2024 \n\n[2] F. Im-3d: Iterative multiview diffusion and reconstruction for high-quality 3d generation, ICML 2024\n\n[3] Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion, arxiv 2024\n\n[4] Vivid-1-to-3: Novel view synthesis with video diffusion models, CVPR 2024", "essential_references_not_discussed": "There is no essential references not discussed.", "other_strengths_and_weaknesses": "Strengths:\n1. This paper proposes two noval cross-attention mechanisms, namely cross-view and cross-frame 3D attentions, to enhance the multi-view consistency of generated videos. The evaluation results presented in the paper appear promising.\n2. The supplementary materials are sufficient to demonstrate the effectiveness of the proposed method.\n\n\nWeaknesses:\n1. The presentation of the method requires improvement. Specifically, there is a lack of formal mathematical equations to clarify how the two types of cross-attention mechanisms are calculated.\n2. Additional quantitative ablation studies should be included in the main text to better illustrate the role and contribution of each component in the method’s design.", "other_comments_or_suggestions": "A List of Issues:\n\n- ​Abstract (Lines L033 and L036):\nThere is a repetition of the phrase \"To our best knowledge, ...\" in both lines. This redundancy should be addressed for conciseness and clarity.\n\n- ​Figure 1(c):\nAs I understand it, the attention mechanism represents four dimensions: V (View), F (Frame), H (Height), and W (Width). However, the figure lacks textual annotations to indicate each dimension, making it difficult to interpret. Additionally, the use of different colors is confusing, and there is no legend to explain their meanings. This information should be clearly labeled to improve readability.\n\n- ​Figure 1(a):\nThe figure is unclear and lacks sufficient detail. Specifically:\n1. The inputs to the cross-attention mechanism are not clearly indicated.\n2. The roles of the key, query, and value are not labeled or explained.\n3. There are no formal equations provided to illustrate how the two types of cross-attention (cross-view and cross-frame) are calculated.\n\n---post-rebuttal comments---\n\nI have no doubt about the contribution and technical merit of this work. The proposed Cavia framework addresses an interesting multi-view image-to-video generation under precise camera control. Regarding the discussion on viewpoint changes, I find the authors’ clarification reasonable. They provided evidence that the demonstrated camera motions are substantial and aligned with community standards.", "questions_for_authors": "Please see the weakness and suggestion part.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "61IUPRn8Tg", "reviewer": "Reviewer_KtKL", "summary": "This paper proposed a novel framework for camera-controlled multi-view video generation. \n\nBased on SVD, it proposes to use 3D attention in both frame attention and view attention to ensure spatio-temporal consistency.  In addition, it curated a mixed dataset from a lot of real and synthetic datasets for the model training. The experiments demonstrate the superior performance of the proposed methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer KtKL for the detailed comments and for recognizing our strong performance compared to existing works. Below are our detailed responses to the questions raised.\n\n**Q1. Baseline details are missing: 1) How did SVD achieve camera pose? 2) How are MotionCtrl and CameraCtrl implemented for 2-view video generation?**\n\nA1. Thank you for this insightful question. We clarify that the SVD results in our experiments are obtained from the vanilla SVD model without any modification; thus, SVD does not offer camera pose controllability. We include SVD's results solely to showcase the base model's motion generation capability, with additional visualizations provided in Fig. 6.\n\nFor 2-view comparisons involving MotionCtrl and CameraCtrl, we independently run each view with the corresponding camera poses, as these methods are originally designed for monocular video generation.\n\n**Q2. Speed and memory report on the 3D attention is needed.**\n\nA2. Indeed, the proposed 3D attention introduces additional computational overhead due to the extended sequence length. However, thanks to the 8$\\times$ compression of SVD's latent space, our attention operates on latent features sized $14\\times 32\\times 32$ for a $256 \\times 256$ video. As shown in the table below, the increased cost in speed and memory remains acceptable.\n\n| Cross-View Attention | Cross-frame Attention | Max sequence length in Attention | Inference Step | Training  | Train/Mem  |\n|------------|-------------|--------------------|-----------|-----------|------------|\n| No         | No          | 32x32              | 0.16s        | 1.68it/s  | 32.36G     |\n| Yes        | No          | 2x32x32            | 0.18s        | 1.66it/s  | 32.37G     |\n| No         | Yes         | 14x32x32           | 0.26s       | 1.62it/s  | 33.85G     |\n| Yes        | Yes         | 14x32x32           | 0.28s       | 1.52it/s  | 34.14G     |\n\n\n**Q3. Performance is limited for this highly challenging task. Some visual issues are: a. The motion magnitude in the results is still very small. b. Camera trajectories are simple. Not sure if the model can support 360-degree views.**\n\nA3. We wish to emphasize that the factors you prioritize most, such as object motion strength and camera trajectory complexity, are precisely the areas in which Cavia significantly outperforms prior works.\n\n- Object motion strength: SVD's results serve as a reference for motion generation capabilities, with additional visualizations in Fig. 6. More importantly, existing camera-controllable video generation methods, such as MotionCtrl and CameraCtrl, are limited to generating videos of static scenes. In contrast, Cavia retains the ability to generate object motion comparable to the base SVD model.\n- Camera trajectory complexity: Extensive experiments in Fig. 3 and Tab. 1 demonstrate Cavia's superior camera control compared to state-of-the-art methods. However, the current version does not support 360-degree view generation due to the limited context length of SVD (14 frames). We plan to explore longer video generation with more capable base models in future work.\n\n\n**Q4. Some concurrent works can be discussed: DimensionX (arXiv:2411.04928), CAT4D (arXiv:2411.18613),  SynCamMaster (arXiv:2412.07760), GenXD (arXiv:2411.02319).**\n\nA4. We will include and discuss these concurrent works in our revision.\n-  DimensionX: Utilizes separate S-director and T-director modules to generate time-frozen and viewpoint-frozen videos independently. Cavia, instead, generates multi-view videos simultaneously, ensuring better spatial-temporal consistency.\n- CAT4D: Converts monocular videos into 4D scenes. In contrast, Cavia generates multi-view videos from a single image input.\n- SynCamMaster: Generates multi-view videos but only supports fixed cameras, resulting in viewpoint-frozen videos. Cavia enables precise camera control for each frame of multiple generated videos.\n- GenXD: Employs a masked video diffusion framework for camera-controllable monocular video generation. Cavia, however, targets multi-view video generation, which is inherently more challenging.\n\nWhile these works are commendable, none address camera-controllable multi-view video generation. Cavia is the first framework to generate **multiple videos of the same scene** with **precise per-frame camera control**, while maintaining object motion quality comparable to SVD.\n\n**Q5. Will the authors release the model and the curated datasets?**\n\nA5. We are currently awaiting legal approval for the public release of the model and curated datasets. Meanwhile, we have provided sufficient details to facilitate the reproducibility of our model and dataset.", "claims_and_evidence": "The claims are supported by convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense.", "theoretical_claims": "No theoretical claim.", "experimental_designs_or_analyses": "I checked the experiments and ablative experiments. \nI found some baseline details are missing, e.g., \n1. When comparing to SVD in Table 1 and 2, how did the authors achieve camera pose controllability in SVD? \n2. When comparing to MotionCtrl and CameraCtrl in Table 2, how did the authors implement 2-view video generation? Just run each of them twice separately?", "relation_to_broader_scientific_literature": "This work contributes a lot to the multi-view video generation with camera control by proposing 3D attention for both spatial and temporal attention and mixed datasets for training.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n1. The idea of using 3D attention in both spatial and temporal attention is interesting\n2. The curated mixed datasets are beneficial for the following works\n\nWeakness:\n\n1. As discussed in **Experimental Designs Or Analyses**, it would be better if more baseline details were provided.\n2. Using 3D attention in the spatial and temporal module could increase the speed and memory requirement compared to baselines. A detailed speed analysis and memory requirement report are needed.\n3. In the provided results, I found the performance still limited, partially due to the highly challenging task. Some visual issues are: a. The motion magnitude in the results is still very small. b. Camera trajectories are simple. Not sure if the model can support 360-degree views.\n\n## update after rebuttal\nThanks for the rebuttal and the additional experiments. Overall, the primary concern remains the method’s ability to handle large viewpoint changes in dynamic scenes. While direct comparisons with ViewCrafter and SyncCamMaster could be informative, I understand that these works are concurrent or not officially published, so such comparisons are not required.\n\nTaking into account both the strengths and limitations of the paper, I will maintain my original score.", "other_comments_or_suggestions": "Some concurrent works can be discussed:\n\nSun, W., Chen, S., Liu, F., Chen, Z., Duan, Y., Zhang, J., & Wang, Y. (2024). Dimensionx: Create any 3d and 4d scenes from a single image with controllable video diffusion. *arXiv preprint arXiv:2411.04928*.\n\nWu, R., Gao, R., Poole, B., Trevithick, A., Zheng, C., Barron, J. T., & Holynski, A. (2024). Cat4d: Create anything in 4d with multi-view video diffusion models. *arXiv preprint arXiv:2411.18613*.\n\nBai, J., Xia, M., Wang, X., Yuan, Z., Fu, X., Liu, Z., ... & Zhang, D. (2024). SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpoints. *arXiv preprint arXiv:2412.07760*.\n\nZhao, Y., Lin, C. C., Lin, K., Yan, Z., Li, L., Yang, Z., ... & Wang, L. (2024). Genxd: Generating any 3d and 4d scenes. arXiv preprint arXiv:2411.02319.", "questions_for_authors": "Will the authors release the model and the curated datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1P2v1f49RW", "reviewer": "Reviewer_vi5f", "summary": "This paper introduced a multi-view video diffusion model enhanced by view-integrated attention, called Cavia. Specifically, Cavia used cross-view attention and cross-frame attention to ensure multi-view and temporal consistency respectively. This model design also enabled Cavia to train jointly with diverse datasets. This paper also included detailed data curation workflows, and ablation studies to support the re-implementation and effectiveness.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer vi5f for their valuable effort and for recognizing the strength of our experimental results. However, we respectfully disagree with the concerns regarding \"large viewpoint changes like SynCamMaster.\" We would like to clarify that **the concurrent work SynCamMaster is limited to generating fixed-viewpoint videos**, whereas our proposed method, Cavia, enables precise camera control for each individual frame under a multi-view video generation setting.\n\n**Q1. Why not include absolute pose error (APE) and relative pose error (RPE) metrics?**\n\nA1. APE and RPE combine translation and rotation errors into a single value. However, following common practices in camera-controllable video generation (e.g., CameraCtrl, Collaborative Video Diffusion), we evaluate translation and rotation errors separately. Our chosen metrics align with the spirit of RPE, as they also measure relative differences between camera matrices but allow for a more detailed assessment.\n\n**Q2. Should discuss and clarify the difference between syncammaster [ICLR2025]. It is still questionable whether Cavia can generate multi-view videos with large viewpoint changes like syncammaster.**\n\nA2. We appreciate the reviewer's suggestion and will cite and discuss SynCamMaster accordingly. However, SynCamMaster is designed for generating videos with **fixed camera viewpoints** and does not support dynamic viewpoint changes. In contrast, Cavia provides **precise per-frame camera control** in multi-video generation scenarios.\n\nThe table below (Tab. A) provides a detailed comparison: \n\n| Aspect | SynCamMaster | Cavia |\n|--------|--------------|-------|\n| **Task** | Text-to-video generation | Image-to-video generation |\n| **Control** | Generates multi-view videos with fixed cameras; each video is viewpoint-frozen | Supports precise camera control for each frame across multiple videos |\n| **Base Model** | Internal text-to-video model (KLing’s team, unpublished) | Uses SVD; achieves comparable object motion to SVD (see Fig. 6) |\n| **Method** | Focuses on static camera videos; trains only cross-view synchronization modules | Supports frame-wise camera control; Trains both cross-view and cross-frame attention modules to ensure spatial-temporal consistency with dynamic camera control |\n| **Data** | Both SynCamMaster and Cavia use data from 4D synthetic assets, 3D static scenes, and monocular videos; SynCamMaster collects static camera videos for monocular training | Employs a curated pipeline to collect high-quality monocular videos with accurate camera pose annotations, facilitating precise per-frame camera control |\n| **Joint Training Strategy** | Requires copying monocular video $v$ times and setting the same camera parameters across views | More compute-efficient; avoids data copying; flexible cross-view attention enables training/inferencing on arbitrary numbers of views |\n| **Viewpoint Changes** | Fixed viewpoint; cannot handle viewpoint changes | Allows precise viewpoint control for each frame in multi-video generation |\n\n\n**Q3. Limitation of Cavia to address multi-view video generation with large viewpoint changes.**\n\nA3. The current performance of Cavia is constrained by the limited context length of SVD (14 frames). However, Cavia's framework is general and scalable, enabling the generation of longer videos with larger viewpoint variations when paired with a stronger foundation model. It is worth emphasizing that SynCamMaster entirely lacks the ability to handle viewpoint changes. In contrast, Cavia is explicitly designed to address this limitation, directly responding to the reviewer's primary concern.", "claims_and_evidence": "Cavia claims that it could generate multi-view videos with precise camera control and object motion, but all results from the paper (especially for multi-view videos) suffer from small camera motion changes. It is still questionable whether Cavia can generate multi-view videos with large viewpoint changes like syncammaster.", "methods_and_evaluation_criteria": "Most experiments make sense. Some questions remain for the camera metric. The authors claimed that they normalize the camera pose scales in this paper, which is unlike previous work (CameraCtrl). However, the widely used camera metric (absolute pose error (APE), relative pose error (RPE)) should already contain the umeyama alignment to confirm the normalization of the camera pose. Why not include these metrics?", "theoretical_claims": "No theoretical claims are in this paper.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The key contributions are joint training and cross-view/cross-frame attention. However, the novelty of these components are constrained, which have been already proposed by syncammaster [ICLR2025].", "essential_references_not_discussed": "This paper did not discuss a very related work called syncammaster published in ICLR2025. Considering the publishing time of syncammaster is very close to the ICML deadline, the authors are under no obligation to compare to it. But this paper should also discuss and clarify the difference between syncammaster.", "other_strengths_and_weaknesses": "In my opinion, this paper should discuss and clarify their different contributions compared to syncammaster, which also includes similar view-attention, 3d-attention, and joint training on multi-view data, multi-view videos, and general videos.\nAnother concern is the limitation of Cavia to address multi-view video generation with large viewpoint changes.", "other_comments_or_suggestions": "Some words are repeated in the abstract, for example, \"to our best knowledge\" and \"to the best of our knowledge\".", "questions_for_authors": "The questions are mainly based on the camera pose metrics, a discussion between syncammaster, and the capacity of Cavia to address large viewpoint changes as mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VR1Wa0ExZk", "reviewer": "Reviewer_T9u5", "summary": "The authors propose a novel framework of image-to-multi-view video generation, with controllable cameras. To achieve this, the design a flexible multi-frame/multi-view attention module, allowing for joint training of static scene video, monocular video and multi-view dynamic scene video. Experiments on monocular and multi-view vide generation validate the 3D consistency and temporal consistency of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer T9u5 for their detailed comments and constructive suggestions. However, we respectfully disagree with the reviewer's concerns regarding our contribution. In response, we have added additional comparisons against concurrent works ViewCrafter and CVD, which are available on our anonymous webpage: https://cavia2025.github.io/. These new results clearly demonstrate that our proposed method, Cavia, significantly outperforms both ViewCrafter (Fig. A) and CVD (Fig. B). We address the reviewer's questions in detail below.\n\n**Q1. Multi-view experiments are mainly conducted on 2 views with small camera angle change. 4-view experiments’camera angle change is still not evident.**\n\nA1. We emphasize that camera control, the factor the reviewer prioritizes, is precisely where Cavia excels. Cavia is the first to enable multi-video generation with precise camera motion control while simultaneously preserving object motion. Concurrent works cited by the reviewer fail to achieve comparable quality. Specifically, ViewCrafter only generates static scenes, SynCamMaster is restricted to fixed-viewpoint videos, and CVD suffers from poor pixel quality with severe morphing artifacts.\n\n**Q2. 2-view video generation with limited camera change is not convincing. Does the training data also contains such small camera change? MotionCtrl and CamCtrl are not designed for multi-view video generation, so it is unfair. It is suggested to compare with SynCamMaster.**\n\nA2. As we illustrate above, SynCamMaster only supports video generation with fixed viewpoints, allowing no camera movement. In contrast, Cavia enables precise camera control while preserving object motion, a challenging and underexplored capability. While MotionCtrl and CameraCtrl are monocular methods, they are the closest relevant baselines as they also target precise camera control. Additionally, Cavia builds upon SVD, which supports video generation up to 14 frames, leading to naturally limited camera ranges. Extending this to larger ranges with a more powerful long context base video generation model is an exciting direction for future work.\n\nWe provide a detailed comparison with SynCamMaster in our response to reviewer vi5f’s Q2 (Tab. A). However, we note that SynCamMaster's official GitHub repository contains only boilerplate code, and the absence of essential components such as the model checkpoint and architecture makes it impossible to compare with SynCamMaster fairly.\n\n**Q3. The joint training strategy for static video is confusing. I don't know if the authors refers to V cameras move separately during F-1 frames, or they simply repeat the first frame to F-1 frames.**\n\nA3. We do not perform single-frame training. Our \"static\" videos refer to sequences where static scenes lack dynamic objects. During training, our Cavia framework consistently accepts $F$ frames, each with a distinct camera matrix.\n\n**Q4. For 3D consistency of monocular video generation, it is suggested to compare with ViewCrafter.**\n\nA4. We have included comparisons against ViewCrafter in Fig. A on https://cavia2025.github.io/#FigA . ViewCrafter primarily generates videos of static 3D scenes, likely due to its reliance on 3D point clouds. We evaluate ViewCrafter using test images and trajectories from the RealEstate10K dataset and observe that it often produces frames with noticeable color and lighting artifacts, as well as geometric distortions. In contrast, Cavia achieves visually pleasing results with accurate geometry. Furthermore, ViewCrafter requires approximately 4 minutes to generate a video sequence, whereas Cavia produces a set of 2-view videos in just 14 seconds.\n\n**Q5. It is suggested to move the comparison with CVD to the main text and conduct quantitative experiments under its evaluation pipeline.**\n\nA5. Thank you for the suggestion. We initially placed CVD’s results in the appendix due to space constraints but have now moved them to the main text. We have also provided additional comparisons using CVD’s recently released official codebase. The results can be found in Fig. B on https://cavia2025.github.io/#FigB . Our comparisons show that CVD suffers from severe morphing artifacts and unnatural object motion. It also fails to follow text prompt instructions and overlooks important details. In contrast, Cavia produces outputs with greater geometric consistency and more natural object motion.\n\nFor quantitative evaluation, we not only adopt SuperGlue, following CameraCtrl and CVD, but also incorporate COLMAP, a widely used tool in the 3D reconstruction community for estimating camera poses. COLMAP shares the same purpose as SuperGlue by measuring camera pose accuracy but is more effective in identifying morphing artifacts, as it focuses on global geometry rather than just local feature matching. Our extensive quantitative results in Tab. 1 and 2 demonstrate that Cavia significantly outperforms existing camera-control video generation methods.", "claims_and_evidence": "Not all.\n1. The authors claim they focus on multi-view video generation, however, most of the multi-view experiments are conducted on 2 views with small camera angle change, which is confusing. 4-view experiments are provided in supplementary material but only qualitatively, where camera angle change is still not evident.\n2. The joint training strategy for static video is confusing. The authors expand them with F-1 frames, but their architecture design allow for only one-frame training. I don't know if the authors refers to V cameras move separately during F-1 frames, or they simply repeat the first frame to F-1 frames.\n3. Lack of comparison with state-of-the-art methods for camera controllable video generation and multi-view video generation.", "methods_and_evaluation_criteria": "Not all, please see experiment part.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "1. monocular video generation: For 3D consistency, it is suggested to compare with ViewCrafter, following its evaluation pipeline.\n2. multi-view video generation: 2-view video generation with limited camera change is not convincing. Does the training data also contains such small camera change? Besides, the authors are encouraged to increase the number of views and conduct more experiments. Besides, the compared method, MotionCtrl and CamCtrl are not designed for multi-view video generation, so it is unfair. It is suggested to compare with SyncCamMaster.\n3. It is suggested to move the comparison with CVD to the main text and conduct quantitative experiments under its evaluation pipeline.", "relation_to_broader_scientific_literature": "N/A", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "I'm willing to increase my score if the authors address my concerns in response.", "questions_for_authors": "The training strategy of static scene videos could be explained more", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a multi-view video diffusion model enhanced by view-integrated attention mechanisms. By incorporating both cross-view and cross-frame attention, it ensures multi-view and temporal consistency, enabling joint training across diverse datasets. The paper includes detailed data curation steps and comprehensive ablation studies, making it both reproducible and technically solid.

The authors and reviewers engaged in a very highly active discussion. The final debate centered on whether the demonstrated camera motion was sufficient to support the paper’s claimed contributions. The authors provided a thorough and well-structured response. They clarified that Cavia actually achieves substantial camera motions, including both significant translations and rotations beyond 20 degrees, comparable to or exceeding several cited baselines. More importantly, they argued that Cavia differs fundamentally from some of the referenced methods by generating multi-view, dynamic videos in a single pass, without stitching or post-processing, thus preserving temporal and spatial coherence. They also emphasized that Cavia addresses a distinct and challenging task: multi-view image-to-video generation under precise per-frame camera control, which is not tackled by the cited works.

Overall, I find the authors’ rebuttal to be convincing. They address the concerns with specific comparisons and clarify misconceptions about both their method and others. Given the novelty of the task, the strength of the technical contributions, and the clarity of the response, I support acceptance of this paper.
Decision: Accept (poster)