Imagine360: Immersive 360 Video Generation from Perspective Anchor
Authors: Jing Tan
Shuai Yang
Tong Wu
Jingwen He
Yuwei Guo
Ziwei Liu
Dahua Lin
Venue: NeurIPS 2025 poster ()
Keywords: video generation
video outpainting
panorama video generation
360 video generation
Primary area: applications
TL;DR: Imagine360 creates high-quality, immersive 360 videos from perspective video anchors.

Abstract: $360^\circ$ videos offer a hyper-immersive experience that allows the viewers to explore a dynamic scene from full 360 degrees. 
To achieve more accessible and personalized content creation in $360^\circ$ video format, we seek to lift standard perspective videos into $360^\circ$ equirectangular videos. To this end, we introduce **Imagine360**, the first perspective-to-$360^\circ$ video generation framework that creates high-quality $360^\circ$ videos with rich and diverse motion patterns from video anchors.
Imagine360 learns fine-grained spherical visual and motion patterns from limited $360^\circ$ video data with several key designs. 
**1)** Firstly we adopt the dual-branch design, including a perspective and a panorama video denoising branch to provide local and global constraints for $360^\circ$ video generation, with motion module and spatial LoRA layers fine-tuned on $360^\circ$ videos.
**2)** Additionally, an antipodal mask is devised to capture long-range motion dependencies, enhancing the reversed camera motion between antipodal pixels across hemispheres.
**3)** To handle diverse perspective video inputs, we propose rotation-aware designs that adapt to varying video masking due to changing camera poses across frames.
**4)** Lastly, we introduce a new 360 video dataset featuring 10K high-quality, trimmed 360 video clips with structured motion to facilitate training.
Extensive experiments show Imagine360 achieves superior graphics quality and motion coherence with our curated dataset among state-of-the-art $360^\circ$ video generation methods. We believe Imagine360 holds promise for advancing personalized, immersive $360^\circ$ video creation.

Imagine360: Immersive 360 Video Generation from Perspective Anchor
Jing Tan
Shuai Yang?.
Tong Wust
Jingwen He
Yuwei Guo
Ziwei Liu
Dahua Lin
The Chinese University of Hong Kong
Shanghai Jiao Tong University
Stannford University
4S-Lab, Nanyang Technological University
Shanghai Artificial Intelligence Laboratory
Input View
Novel
Input View
Novel View
Novel View #2
Novel View #3
Novet View #2
Novet View #3
Figure 1: Overview of Imagine360. Imagine360 lifts standard perspective video into 360 video, enabling dynamic scene experience from full 360 degrees. Our approach achieves high-quality texture and plausible spherical motion patterns for both generated vidco (left) and in-the-wild vidco (right). Best viewed with Acrobat Reader for the animated 360 videos. More examples please visitlour webpage
Abstract
360 videos offer hyper-yimment dyper-inmersive experience that allows the viewers to explore a dynamic scene from full 360 degrees. To achieve more accessible and personalized content creation in 360 video format, we seek to standard perspective videos into 360 equirectangular videos. To this end, introduce Imagine360, the first perspective-to- to-300 video generation framework that creates high-quality 360 videos with rich and diverse motion patterns from video anchors. Imagine360 learns fine-grained spherical visual and motion patterns from limited 360 video data with several key designs. 1) Firstly adopt the dual-branch design, including perspective and a panorama video denoising branch to provide local and global constraints for 360" video generation, with motion module and spatial LoRA layers fine-tuned on 360" videos. 2) Additionally, an antipodal mask is devised to capture long-range motion dependencies, enhancing the reversed camera motion between antipodal pixels across hemispheres. 3) To handle diverse perspective video inputs, we propose rotation-ar aware designs that adapt varying video masking due
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

changing camera poses across frames. 4) Lastly, we introduce a new 360 video dataset featuring 10K high-quality, trimmed 360 video clips with structured motion to facilitate training. Extensive experiments show Imagine360 achieves superior graphics quality and motion coherence with our curated dataset among state-of- the-art 360" video generation methods with both real and generated videos. We believe Imagine360 holds promise for advancing personalized, immersive 360" video creation.
Introduction
Imagine embarking on a journey through the heart of a bustling city, serene beach, or cherished place of your own. It would be wonderful to record this 360-degree dynamic experience for future viewing. 360 video offers an interactive, immersive format that captures living, breathing world as if the viewer were part of the experience. With the rapid development of VR devices, the demand for 360 videos is increasing, leading to growing focus on 360 video generation research.
Recent advancements in 360 video generation focused on text-guided (34] and image-guided [8] models. While these methods produce plausible 360 videos, they require panoramic optical flow 33 or high-quality panoramic images [8]] as guidance, which are hard to obtain for users. In contrast, traditional perspective videos are more accessible, as they can be easily captured using smartphone cameras or generated by advanced video synthesis models. To enable more user-friendly and personalized 360 video creation, we propose a new task: perspective-to-366 video generation, which transforms standard video inputs into 360" equirectangular videos. Specifically, we take a perspective video with narrow FOV as the anchor video, project it to a 360 180 FOV video canvas, and synthesize the surrounding pixels.
One relevant task is video outpainting (2) 30 8), which aims fill in missing regions outside the edges of video frames in a larger canvas, typically in the perspective domain with fixed video masking across frames. Simply applying standard video outpainting methods does not achieve satisfactory results, as our perspective-to-660 video generation exhibits more challenges. First, due to the large domain gap between perspective and 360 videos, learning the spherical visual and motion patterns requires sophisticated design when trained on limited 360 video data. Second, as vidcos exhibit different camera poses across frames, after mapping the 360 canvas, the vidco mask would change significantly in shape, size, and location, requiring rotation-aware designs for robust generation.
To address these challenges, introduce Imagine360, the first framework to generate high- gigh-quaitty 360 videos from standard perspective video inputs. Inspired by the dual-domain concept [42 in text- to-image generation, our model employs dual-branch video outpainting Unets in the perspective and panoramic domains. These two branches jointly denoise the partially masked 360 spacetime canvas in the global scope and in each perspective window. A cross-domain attention module establishes interactions between dual- branch latents that map to the same position in the 360 sphere, enabling high-quality and plausible spherical video patterns. While positionally aligned latents offer local consistency, they fall short in accounting for a unique characteristic of panoramic videos, where each pixel undergoes reverse camera translation of its antipodal counterpart. Hence, we improve the gross-domain attention with an antipodal mask, to extend each pixel's 's receptive field from its local neighborhood to its antipodal region across the 360 sphere, making easier learn long-range panoramic motion dependencies. Another key challenge lies in the varying input camera poses. Handling general videos as anchors requires the framework to accommodate changing camera poses across frames. In practice, we incorporate rotation- designs, including rotation-aware data sampling in training and a camera pose estimation module in inference. Through explicit tracking of the rotating input video in the 360 spacetime canvas, model streamlines robust generation from customized video inputs. Effective training of our framework also relies on high-quality 360 video data. However, existing datasets are either small in scale [34] 37 38 or consist of large-scale, unfiltered web-scraped content that requires extensive cleaning 28 In this regard, we introduce YouTube360,, ready-to-tren to-trinn 360 video dataset comprising 10K curated clips from YouTube. Our dataset incorporates manual quality control and sophisticated data cleaning to select high-quality training segments with diverse and structured motion.
With these three key designs and our curated dataset, Imagine360 pioneers end-to-end, high-quality 360 video generation from perspective inputs. Extensive experiments show that our model achieves
2

---

state-of-ther -ft performance in both frame quality and motion consistency. As a bonus, our pipeline also demonstrates superior results in panoramic image outpainting. We believe Imagine360 can empower the 360 video generation community to create personalized, hyper-immersive experiences for real-world applications.
2
Related work
2.1 Video outpainting
Video Outpainting aims to fill in the missing regions at the edges of source videos. Compared to advanced image-level outpainting, video-level outpainting remains under-explored due to its challenges in maintaining both spatial and temporal fidelity and consistency. Recent video outpainting methods leverage diffusion to generate high-quality pixels in the missing regions. M3DDM [ proposes a frame-guided Masked 3D diffusion model and a coarse-to-fine inference strategy to tackle artifact accumulation in long video outpainting. MOTIA (30] employs per-case optimization strategy learn the data-specific patterns of source video for better outpainting quality, Follow-Your- Canvas
[ divides the canvas into multiple windows and achieves outpainting of different sizes and resolutions by merging each outpainted window. These methods focus on handling perspective video outpainting with fixed masking in each frame. Despite their appealing outpainting results, it remains difficult for them handle perspective-3600 video generation that requires high-quality outpainting in panoramic distribution and continuously changing video masks from varying camera poses. In contrast, our Imagine360 handles the perspective-to-666 video generation with global and local constraints from baanch diffusion and rotation- -wware designs to handle changing video masks, producing high-quality 360 video generations from perspective video anchors.
2.2 360 panorama generation
35 exploit GAN-based framework for panorama image genera- tion [9 27 OrniDre II proposes transformer-based framework for 360-degree outpainting and devises circular inference to obtain 360 close-loop continuity. Recently, diffusion-based meth- ods [33 17:
Early methods
31
5
15
21
16 26 43 have dominated image-level panorama generation. Due to the data scarcity of large panorama image datasets, methods [33 32) that directly fine-tune LDM to generate the panorama results in low-quality images with simple structures and sparse assets. PanFusion 43 introduces panorama and perspective branches to leverage the synergy from both global and local constraints for text-to-pannae panoranaora generation.
36 42 32,
Despite numerous efforts in image-level panorama generation, there are few works [4 [8; 20 focusing on panorama video generation. 360DVD 34 takes text prompts and additional panorama video optical flow as guidance and learns a 360-Adapter
standard T2V models to generate plausible 360 videos. 4K4DGen [8]] animates a static panoramic image at user-selected regions with 12V pre-trained prior in training-frree manner. These fine-tuning- based or training-fre frree approaches struggle to bridge the distribution gap between panoramic and perspective videos, resulting in simple natural perturbations, such as clouds moving and water running. In contrast, our Imagine360 benefits from the dual-branch video denoising structure with antipodal relation modeling and rotation- aware designs, resulting in more dynamic 360 videos with rich and structured motions.
3 Our approach
Given perspective vidco, we aim to generate equirectangular (ERP) 360 video that replicates its visual appearance and motion, and extends to the complete 360 x 180 field of view.
We propose Imagine360, the first perspective-to-3880 to- video generation framework, incorporating three key designs to synthesize high-quality panoramic videos, as illustrated in Fig. 2 First, to learn the spherical visual and motion patterns based on pre-trained perspective generative prior, we employ the dual-branch design, consisting of a panorama branch and perspective branch, to jointly denoise the 360 spacetime latent. Second, to obtain more fine-grained and plausible panoramic motion, we refine the cross-domain attention to highlight antipodal masking that captures long-range motion dependencies in antipodal directions. Lastly, to handle diverse video inputs with varying camera
3

---

Train
Infer
Inpie Perspective Video
PeriNGELoo
Pana MGE
Panorana Branch
Target Panoroma Video
Camera Pase Estnation
Pesture Eatrector
LLM Text Encoderr
Pusariinn slealo
Sampaing
EZP
Gros-daman SphericalAtto
Teeture
Perspective
Lighthouse
@ Text Encoder
cliff overLockiing
bie
viru
Perspective Btanch
Spatial Layer
Spatial LoRA
Motion Module
Figure 2: Pipeline of Imagine360. Given perspective anchor video, Imagine360 uses dual- bual-branch denoising structure across panorama and perspective domains, featuring cross-domain spherical attention with antipodal masking for modeling long-range antipodal motion, and rotation-aware designs to handle varying camera rotations.
poses, we propose rotation-awas aware training and inference designs to obtain robust generations from different video anchors.
3.1 Preliminary: 360 video format
360 video is video format that captures full horizontal 360 field of view (FOV) and vertical 180 FOV. Unlike standard perspective videos, which are limited fixed, narrow field of view, 360 video enables viewers to look in any direction at each frame location, creating an immersive experience. For encoding, storage, and playback, 360 video must be projected onto a 2D plane from the 3D sphere. In our work, we use 360 videos encoded by Equirectangular Projection (ERP).
To create perspective video frame from ERP video frame of resolution H x W, we simulate virtual pinhole camera looking in a specific direction defined by yaw (0), pitch (δ) and roll (v) Euler angles and FOV parameter. Each pixel on the image plane is computed with a direction vector Uiy [1, W 1,
1)]=.. tan 2 in 3D space and is rotated according to the desired viewing orientation T' Rs Rs Ro Vij given the Fuler angles. Then, these vectors are converted into spherical coordinates arctan 2(7 oh (",) = aresin (7,) and mapped to 2D
++ H for sampling. We denote this process as the Equirectangular-to- tispraannuuuuttt---pppccctiieee (E2P) mapping. The inverse mapping, denoted as P2E mapping. maps perspective video frame back to ERP frame. The mapping begins by computing 3D directional vector the spherical coordinates for each pixel in the ERP frame 2n (## t,
ERP coordinates (^+*) W. =
sin()))sinnd)) (i)) then rotate the vector inversely into H7 [cos(() cos(A), cos(Ø) sin(A),
the perspective camera's 's reference frame and projected onto the image plane. Pixels that fall within
the camera's 's FOV sample the perspective pixel value and write back to the ERP frame.
3.2
Video-conditioned 360 video generation
Our model is trained on triplets of 360 videos, text captions, and corresponding 360 video masks. Using E2P (Equirectangular-tor to-eesspcctivee mapping. the 360 video mask determines the anchor perspective video extracted from the full 360 video. To generate different training masks, we sample sets of Euler angles (01:T" 22:T 1:"T)) over T' frames via P2E Ceessectiveoo--Euuretttaggg mapping. where denotes yaw, & pitch, and & roll. Similar to previous inpainting models 23 2 our model takes as input a concatenation 360" video latent along the channel dimension. A frozen VAE encodes both the full and masked 360 video inputs into d-dimensional latent channels. Additionally, we project the masked region of 360" video into the perspective anchor video to provide fine-grained visual guidance. Following (2) the anchor video /1.7 is encoded into semantic features using the visual encoder from SAM [14] These features then processed by pre- pre-trained query- based Transformer, which distills high-level
the noisy 360 video latent, the video mask, and the masked

---

visual and motion cues into compact latent tokens. The visual tokens are concatenated with the text embeddings in the conditioning branch. To guide large-scale spacetime pixel generation, we incorporate IP attention [41], which decouples cross-attention for injecting text prompts and visual tokens in the U-Net layers. This enables our model to effectively propagate visual content and motion patterns across views and into unmasked regions of the spacetime panorama.
3.3 Dual-branch 360 video denoising
Dual-branch design. Extending a perspective video to 360 canvas requires careful model design due to the large differences between panoramic and perspective distributions. Mainstream methods either train a Latent Diffusion Model (LDM) to directly denoise panorama frames jointly denoise multiple perspective view projections. The former results in plain visuals and mild motion from limited 360 data, while the latter easily produces local, short-range motions in each view. Inspired by image-level generation 42]. we employ a dual-branch video denoising structure. It consists of a global panorama branch and local perspective branch based on U-Net structure, where each block consists of spatial layers initialized from SD weights and a motion module initialized from [2] weights.
The two branches share the same input: the panoramic branch latents of shape RT (2d 1) hx is projected into each view gets latent of shape I² 7 attention, the model gains holistic modeling of the 360 spacetime canvas for global consistency. In the perspective branch, the latent permutes and flattens the number of views with the batch size for self- attention to preserve the pre-trained generative power in the local window. each U-Net downsample block, the latents from both branches are aligned via cross-attention enhance the local-global synergy. The perspective latents 2} E R2''mxx Dxh//xx/x are reshaped to RT
x
20 perspective views according to the P2E icosahedron modeling [2 42), and
through projection. In the panoramic branch self-
then bidircctionally cross-attended with panorama latents 2 E xx where D is the channel dimension, h, w are the latent spatial dimension. Spherical positional encoding 42 is added to indicate the relative spherical location. A spherical attention mask the attention map enforces E2P and P2E mapping between the flattened latent sequence from the two branches. As illustrated in Fig. the directly mapped pixels via P2E mapping in the perspective branch are highlighted in orange. Gaussian blur is also applied to the spherical mask to enable neighboring activations in the attention.
Resource-friendl) fine-tune strategy. With only thousands of training panorama videos, we seek to fine-tune a limited proportion of parameters to yield good generation results. Thanks to the space-time disentangled design from AnimateDiff [0], we do not need to fine-tune the whole model to learn the distributions of panoramic videos. In practice, attention layers for both branches. As for the panoramic motion, we show in supp. that employing motion LoRA layers does not have sufficient influence the pretrained prior generate good panorama motion. Therefore, the whole motion module is fine-tuned for the panorama branch to learn the necessary prior for generating spherical motion patterns. Note that, fully exploit the perspective generative prior, we do not fine-tune the motion module in the perspective branch. Experimental results show that our resource-friendf] fine-tuning strategy achieves competitive results for 360 video generation despite the limited data and GPU resources. The training objectives in the two branches are:
add LoRA on spatial attention layers and IP
LE -BE(st")...ce J.C.T. [||e CE (2 (v)),, 12:)) [|2]
(1)
Lp Cp= -p=Eg(<)T)) N.
[|le- ep,9 eP, (2p, t, Te (v), 1:77)
(2)
where 1.::'' and e. denote the target equirectangular (E) and perspective (P) video and its predicted noise, y is the text prompt. To facilitate the synchronous exchange of information between two branches, we combine the two branches losses as the final training objective L LE + m EE Ltp.
3.4 Encouraging fine-grained panoramic patterns
The dual-branch design generates plausible spherical patterns for spatial appearance and motion, however, to achieve fine-grained panoramic videos, i.e., 360 close-loop continuity and reversed antipodal translation, we introduce additional advanced techniques, including circular padding and antipodal mask in cross-domain spherical mask.
5

---

Circular padding for close-loop continuity. The 360 closed-loop continuity ensures seamless alignment between the left-most and right-most edges of panoramic videos. To preserve this property and mitigate artifacts from local convolutions, we follow 42 39 by applying circular padding before each convolutional layer the U-Net blocks, then unpadding afterwards to restore the original resolution. For improved 360 continuity, we adopt circular padding in the optional video super-resolution [II] to process upsampled latents durine decodine.
Antipodal mask for reversed translational motion. One distinctive property panoramic motion is that antipodal pixels, i.e., those located on opposite sides of the 360 sphere, often ex- hibit reversed motion under camera translation. Specifically, if the camera moves forward rela- tive to the forward viewing direction, the scene observed from the backward viewing direction should appear to move away from the viewer. However, in both the panoramic and perspective branches, antipodal pixels are typically distant in the token sequence. Since attention modules ob-
Plant
Pano Pizels
Paed Leien
Aatn
Pers
Paels
rapping
Cross Cross-domain Spherical Attention
irapping
Figure Cross-domain Spherical Attention high- lights interaction for direct-mapped pixels (spherical mask) and antipodal-mappe pixels (antipodal mask) between panorama and perspective domains.
tend to focus on local neighborhoods. serve that motion in antipodal directions tends to be less evident compared to motion in nearby pixels of the anchor video. To address this, and to better emphasize antipodal relationships in 360 video generation, we introduce the antipodal mask: a new activation mask for the cross-domain spherical attention. As illustrated in Fig- [] we identify, for each latent in the perspective domain, its corresponding antipodal latent (shown in blue) in the panorama domain, and vice versa. These antipodal pairs are assigned higher activation values in the attention mask to encourage long-range attention across opposite directions. Additionally, we apply a Gaussian blur to the mask to softly include antipodal neighbors.
3.5 Rotation-robustness over general video input
Commonly, assume the video input be upright with fixed camera pose, but in-the-win perspective videos are of various rotation angles (@l-f 1/ and @I:T). ). The change of rotation angle significantly impacts the shape and location of the masking, as shown in Fig. [6] To obtain robust generation for in-the-wild video inputs, we propose aaatinn-wware data sampling in training and camera pose estimation during inference.
Rotation-aware data sampling. We create flexible masking for each frame with a rotation-aware data sampling strategy. For simplicity, we randomly sample (-20", 20"). 0 E (-45%, 4.55). and & E (-10", 10%) for each frame, then generate smooth angle sequences that are either monotonic or back-and-fort within the sampling range. Morcover, we add the sinusoidal positional embedding of the mask and the Euler angles to the conditioning tokens, i.c., the visual tokens of the anchor video and the text embedding, in order indicate the relative position of the mask and the global canvas. For mask positional embedding, we compute based on its maximum inscribed box coordinates. The positional embeddings concatenated along the channel dimension.
Camera pose estimation for inference. During inference, we employ MonST3R 44 to estimate the camera pose for each frame in the test video. The Euler angles are calculated from the extrinsic matrix at each frame. We do not normalize the Euler angle sequence according to the first frame, as sometimes the first frame does not map the center region of the 360 canvas. We create the 360 video mask and masked 360 video pixels based on the estimated Euler angles. The masked video pixels are encoded by VAE and concatenated with the 360 video mask and noise as inference input.
3.6 YouTube360 dataset
Fine-tuning for 360 motion generation requires substantial text- video paired data due to the domain gap between panoramic and perspective motions. Existing datasets are insufficient: WEB360 34 small and limited to simple landscape videos, while 360-1M 228. although large in scale, contains unfiltered low-quality videos with incomplete views and polar artifacts. Cleaning such data requires large time and computational costs.

---

Table I: Quantitative comparison on Vbench 12 flow-based Endpoint Error (EPE), OmniFVD 7 and user study results.
Vbench
User Study
Method
Guidance
EPE. OmniFVD vM
IQ ^
AQT
MS t
SC 1
GQ
SPt
TC 1
Animatediff+ -aaiieddiff.Lattttt.abbb..
12V
0.6248 0,5368 0,9866 0.8770 3,5393
285.9
1.2067 1.7588 1.4279
Follow- Your-Canvas
V2V
0.6257 0.521 0.9799 0.9122 3.1767
315.4
2.7692 2.1298 3.0385
360DVD Ours
T2V
0,5501 0.4359 0.9856 0.9356 3.1904
747.4
1.2067 1.7588 1.4279
V2V
0.7372 0.5722 0.9866 0.9649 2.5583
204.0
3.6827 3.7260 3.3942
To address this, we curate a high-quality 360 video dataset, YouTube360. from web videos. We collected 360-degree videos from YouTuue,, covering city tours, wildlife documentaries, and VR game captures. We prioritize long videos with diverse content, as they maintain a consistent format, offer varied scenes, and are easier to discard if polar artifacts are present. All videos are manually reviewed to remove sensitive content and low-quality samples with incomplete views or polar distortions. Then, the videos are converted to equirectangular format, resized to 512 x 1024, and segmented with Tet [25]. For slow-paced clips, apply 2x speed-up. To filter out static content, we compute optical flow using PanoFlow 24 and discard clips where fewer than 10% of frames exceed an average flow magnitude of 0.1. Captions are generated using VideoLLaMa-2 [6) The final dataset contains 9.558 five-second clips at 20 fps. Each annotated sample includes a caption, a You Tube video ID, and time interval. We provide annotations with documentation in the supplementary. Please refer to the supplementary material for additional details.
Experiments
4.1
Implementation details
Training settings. The spatial and motion modules are respectively initialized Stable Diffusion v2.1 and (2). Training is conducted on 8 NVIDIA A100 GPUs in 50k training steps, with the spatial LoRA rank and LoRA set 32 and 1.0. The training resolution H W is set to 256 x 512, the length of frames to 40, the batch size to I, and the learning rate to 1 x 10-5. For inference, generating video of 512 x 1024 resolution and 32 frames takes approximately 6 minutes on average, using a single NVIDIA A GPU with 39 GB VRAM.
Evaluation metrics. We collect test benchmark by randomly sampling videos from 360- 1M 28),
RealEstate OOK [5], and Cog Video (40) generations with prompts from GPT-4o 13, including both real videos and generated videos. We also randomly select 15 videos from the benchmark for the user study. Following previous practice, we employ Imaging Quality (IQ), Aesthetic Quality (AQ), Motion Smoothness (MS), and Subject Consistency (SC) metrics from VBench [12] to measure the graphics quality and motion consistency. To measure 360 motion correctness, we report Endpoint Error (EPE) on video optical flow (OF) with groundtruth 360 videos from 360-1M IM 28 The EPE metric is adapted from optical flow to evaluate motion accuracy. We compute optical flow for both the generated video Vgen and the ground-trurt Vg. using PanoFlow [24). and define EPE as the pixel-wise Euclidean distance between the two flows: EPE(V gen. Vgg) = V gen Vgul- Small EPE indicates the higher similarity of generated video OF with Groundtruth OF. To measure omnidirectional video quality, we report OmniFVD that extends from OmniFID 77 OmniFVD projects the 360 video into six cubemap views, computes FVD on each view, and reports the average as the final score.
Comparison methods. As the first perspective-to-360 video generation framework, it's infeasible to find a method that has the exact same input condition as ours. We compare with methods that produce 360 vidcos from various guidance. 360DVD (34] is the advanced text-guided 360 video generation method that takes text prompts and panorama video optical flow as input for 360 video generation. Here, we feed the same text prompt as ours with GT optical flow into 360DVD for comparison. Follow-Your Your-aavas 2 the state-of-the-an video outpainting method that employs based outpainting to handle outpainting of arbitrary size and resolution. Here, we project our perspective anchor video into the panorama canvas and feed the masked canvas into (2) for evaluation. AnimateDiff+LLe Latffff+aatttaaaaaaaaa comparison, we feed the first frame of our generations into AnimateDiff++ Lttettl 363 and take its animated vidcos for evaluation.
[10 takes image inputs and animates with the LatentL_ab366 LoRA. For
7

---

Frame 0
Frame 1/2
Frame
Frame -0
Frama 1/2
Frume
Oum
Liteett awwetaandioo
Folion NoAr Canaas
Figure 4: Qualitative comparisons on 360 video generations among sfatc-of-the-art methods. Imagine360 gen-
erales 360 video generation with superior visual quality and plausible panoramic patterns.
Table 2:
Quantitative ablation studies on dual-branch designs, antipodal mask, rotation- designs under Vhench, EPE, and OmniFVD metrics.
Vbench
Method
EPE |
OmniFVD
MS
SC
Ours(Full model) w/o persbranch panobranch w/o antipodal mask
0.7372
0,5722
0.9866
0.9649
2.5583
204.0
0.7269
0.5464
0.9783
0.9491
3.0264
681.6
0.7196
0.5081
0.9760
0.9307
3.5308
866.0
0,7321
0.5377
0.9737
0.9375
2.7169
257.6
rotation-awaar designs
0.7319
0.5518
0.9785
0.9441
2.7276
332.1
4.2 Quantitative comparisons
Tab. 1 provides the quantitative comparison with baseline models. Imagine360 outperforms the existing 360 video generation methods at both ground-level perspective views and overall 360 correctness. 360DVD often produces blurred videos, resulting in lower performance across all image quality metrics. Follow- Your-Canvas performs comparably to AnimateDiff in perspective image quality metrics (IQ, AQ); while it generates more aesthetically pleasing details, may suffer from distortion in projected perspective views. All methods achieve similar scores in perspective motion quality metrics (MS. SC). For the 360 motion metric EPE, ridco-conditionel generation methods outperform text- or image-based approaches, 360DVD achieves better EPE scores than AnimateDiff, as it leverages ground-trute optical flow as input.
We also involve a user study to examine the generated 360 videos. We invite 26 users with expertise in video and 3D generation assess the results across three dimensions: graphies quality, structure plausibility, and temporal coherence in both 360 videos and projected perspective views. Tab. reports the average in all three dimensions. Please refer to the supplementary material for more details regarding the setup and metrics. Our method also achieves superior panorama image outpainting performance, and we provide the comparison in the supplementary material.
ranking of all four methods, and method achieves the best performance
4.3 Qualitative comparisons
present the qualitative comparison between Imagine360 and other baseline models in Fig.
At each frame, we present projected perspective views (ńb 0º, 0 [0 9000000. 180", 270%) to examine the panoramic structure plausibility. Fig. g shows that Animate attnfl_aaatttabb333 and Follow- Nooww-You--Coe Canvas fail to achieve 360 close-loop continuity (orange box) and they produce mild-

---

Table 3: Additional Upgrading Experiments backbone and camera pose estimation.
Vhench
Method
EPE
OmniFVD
IQ f
AQT
MS f
SC
Ours
0.7372
0.5722
0,9866
0.9649
2.5583 2.7276 2.7385
204.0
TTT3R
0.7406
0,5731
0.9831
0.9659
332.1
WAN
0.7508
0.4896
0.9929
0.9639
268.6
Frama
why rosuiin
gra
desgrn
Ours|Dual Branch)
Perspective Panorama Branch
Figure 5: Qualitative ablation on the dual-branch de- sign shows reduced artifacts and improved 360 patterns with dual-branch video denoising.
Figure 6: Qualitative ablation the rotation- aware designs show plausible, consistent scene geometry with rotation designs.
scale motion, as observed in the change of the canoe location (left-side) and animal size (right-side). 360DVD produces more distorted patterns and blurred visual appearance in both cases. In contrast, our 160 achieves superior visual quality, and plausible, obvious motion in the generated 360 videos. We also provide side-by-side comparison videos in the supplements for clearer visualization.
Table 4:
Efficiency analysis with baseline methods under inference time and VRAM.
Table 5: Inference Runtime Breakdown of Imag- ine360.
Diim 36DDVD Follow-Your Your-Canvas
Ours
Model Load 10
Camera Est. Inference
Inference Time (6)
122.65
93.18 18.06
381.89 $1.89 27.99
182.76 02.76
Runtime (s)
13.29
2.1
47.88
182.76
VRAM (GB)
20.83
25.59
4.4 Efficiency Analysis
We provide runtime breakdown table (Tab. and efficiency comparison table (Tab. 4 in terms of runtime and VRAM consumption. We report the average runtime based on tests using 16-frame video clips.
From the efficiency comparison, observe that compared another V2V model Follow- Your- Canvas (FYC), our model is faster in inference speed and consumes less VRAM. The main reason that FYC requires multiple rounds of outpainting to ensure outpaint quality, and they also train and infer in float32. Thanks to our dual-bram structure, Imagine360 achieves good visual quality a single pass of inference. AnimateDiff and 360DVD have faster inference speeds, which is within expectation, as they are text-guided and image-guided guided frameworks. In addition, by employing multiple memory optimization techniques (bfloat 1,, vae slicing empty empty_cache, intermediate tensor cleanup, etc.), our dual-branch structure does not increase VRAM usage by lot.
In the runtime breakdown table, we observe that the external camera estimation module takes up considerable amount of time. Although optimizing the inference speed of off-the-shelf camera pose estimator beyond the scope of this work, we will seek upgrade this module with a faster and more robust camera estimator.
4.5 Ablative studies
Ablation on dual-branch design. Fig. shows that with limited training data, single panorama branch often proposes non-spherical patterns, and single perspective branch without global guidance produces messy boundary artifacts. Quantitative results Tab. also support the effectiveness of the dual-branch design. We also study the dual-branch fine-tuning strategy (see supp.), and find that compared fine-tuning the pano motion module, adding motion LoRA insufficient to generate good spherical pattern.

---

input video
without antipodal masking
with antipodal masking
w/o anti backview w/ anti backview
Figure 7: Qualitative ablation on the antipodal mask shows improved reversed motion in the antipodal view for forward forward-moving moving camera motion. Best viewed with Acrobat Reader for the motion.
Ablation on rotation-aware designs. As shown in Fig. 6, neglecting camera rotations in the input anchor video introduces artifacts in the scene geometry of the resulting 360 video. Without rotation- aware processing, the mountain appears to grow unnaturally taller (see yellow box). In contrast, our rotation-awar awaro design preserves consistent scene geometry. We provide a clearer video comparison in supp. to show the robustness to diverse rotations. Quantitative results in Tab, 2 also validates the benefits of rotation-aware designs.
Fig. 7 iillustrates the impact of the antipodal mask on antipodal motion. Thanks to the antipodal mask, as the camera moves forward in the input direction (center view), we can observe clear backward motion in the antipodal views (see red box). Quantitative result in Tab. 2 also proves the effectiveness of adding antipodal activations for general videos. We show the backward view for both cases: the result without antipodal mask exhibits distorted patterns at faraway locations, while with antipodal mask, the camera moves away from the scene in more plausible way. We further verify that this design does not compromise content quality for rotation-only videos. On filtered subset of 100 such videos from the test set, VBench IQ and AQ scores remain comparable: with the antipodal mask, IQ is 0.6688 and AQ is 0.5553; without it, IQ is 0.6621 and AQ is 0.5649,
Ablation on antipodal mask.
Analysis on camera estimation model. In default setting, pose estimation during inference, which provides accurate and stable results in most cases. While precise pose estimation remains challenging in rare corner cases, this is beyond our paper's scope. Nevertheless, as camera pose estimation continues to advance, MonST3R can he seamlessly replaced with more powerful models. For instance, replace MonST3R with more advanced TTT3R 3 and report the results in Tab. 3 Results with TTT3R improve on vbench IQ, AQ, and SC metrics, indicating more stable pose estimation ability of TTT3R over MonST3R.
MonST3R for camera
Analysis on backbone model. As more advanced video generation backbone emerges, we also upgrade backbone model from AnimateDiff [10] to Wan 229 with technical modifications from Unet to DiT. We find that this upgrade significantly improves the image quality and temporal consistency, especially solving the temporal flickering issue observed in some of our primary results. We report the results with Wan 29 in Tab. 3 and show qualitative results in the webpage as well as in the supplementary.
5 Conclusion
In this paper, we propose Imagine360, the first framework for video-conditioned 360 video generation with structured 360 motion. Our key contributions are: (1)
video denoising network with panorama and perspective branches for global-local constraints; (2) an antipodal mask in cross- domain spherical attention capture reverse translational camera motion; and (3) rotation-aware designs to handle diverse camera poses. We also introduce YouTuube600, a high-quality 360 video dataset with diverse, structured foreground motion. Experiments show that Imagine360 achieves superior video quality and panoramic motion realism.
Limitations and future work. During inference, Imagine360 leverages camera pose estimation model to obtain the input camera poses. While these methods 44 generally performs well, underestimate rapid movements involving large rotations due data bias, leading to noticeable distortions (see supp.) that limit the video fidelity in certain scenarios. As research in geometry estimation continues to progress, we plan to keep our model updated with more advanced techniques to alleviate this issue.
Acknowledgments and Disclosure of Funding
This work is supported by Shanghai Artificial Intelligence Laboratory.
10

---

References
[1] Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Diverse plausible 360-degree image outpainting for efficient 3deg hackground creation. CVPR, 2022.
[2] Qihua Chen, Yue Ma, Hongfa Wang, Junkun Yuan, Wenzhe Zhao, Qi Tian, Hongmei Wang. Shaobo
Min, Qifeng Chen, and Wei Liu. Follow-your-carvas, Higher-resolution vidco outpainting with extensive content generation. arXiv preprint arXiv:2449.00005 2024.
[3] Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. Tutr:: 3d reconstruction as
test-time training. arXiv preprint arXiv:2509, 2264009..6665,.... 2025.
[4] Zhaoxi Chen, Guangeong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation.
ACM Transactions Graphics (TOG), 1-6, 2022.
[5] Yen-Chi Cheng. Chieh Hubert Lin, Hsin- Ying Lee, Jian Ren, Sergey Tulyakov, and Ming- Yang.
Inout: Diverse image outpainting via GAN inversion. In CVPR. pages 11421-11430. IEEE, 2022.
[6] Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama Advancing spatial-temporal modeling and audio understanding in video-Ilms. arXiv preprint S5ti:2400.0747777 2024.
[7] Anders Christensen, Nooshin Mojab, Khushman Patel, Karan Ahuja, Zeynep Akata, Ole Winther, Mar Conzalez-Franco and Andrea Colaco. Geometry fidelity for spherical images. In European Conference on Computer Vision, pages 276-292. Springer 2024.
[8] Fanda Fan, Chaoxu Guo. Litong Gong. Biao Wang, Tiezheng Ge. Yuning Jiang, Chunjje Luo, and Jianfeng Zhan. Hierarchical masked 3d diffusion model for video outpainting. In Proceedings of the 31st ACM
International Conference on Multimedia, pages 7890-7900, 2023.
[9] Mengyang Feng. Jinlin Liu, Miaomiao Cui, and Xuansong Xie. Diffusion360: Seamless 360 degree
panoramic image generation based on diffusion models, 2023.
[10] Yuwei Guo, Ceyuan Yang. Anyi Rao, Zhengyang Liang. Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint 2rXiv:2307.04777 2023.
[11] Jingwen He, Tianfan Xue, Dongyang Liu, Xinqi Lin, Peng Gao, Dahua Lin, Yu Qiao, Wanli Ouyang, and Ziwei Liu. Venhancer: Generative space-time enhancement for video generation. arXiv preprint
arXiv:2407.07666
2024.
[12] Ziqi Huang. Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, pages 21807-21818. IEEE, 2024.
[13] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint xxXiv:2400.2127 2024.
[14] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg. Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4015-4026, 2023,
[15] Dilip Krishnan, Piotr Teterwak, Aaron Sarna, Aaron Maschinot, Ce Liu, David Belanger, and William T. Freeman. Boundless: Generative adversarial networks for image extension. In ICCV, pages 10520-10529 IEEE, 2019.
[16] Yuseung Lec, Kunho Kim, Hyunjin Kim, and Minhyuk Sung. Syncdiffusion: Coherent montage via
synchronized joint diffusions, 2023.
[17] Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for vision-and- language navigation. In NeurlPS. 2023.
[18] Renjie Li, Panwang Pan, Bangbang Yang. Dejia Xu, Shijie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, and Zhiwen Fan. 4k4dgen: Panoramic 4d generation 4k resolution. arXiv preprint arXiv:2406. 13527, 2024.
[19] Zhuqiang Lu, Kun Hu, Chaoyue Wang, Lei Bai, and and Zhiyong Wang. Autoregressive omni-aware outpainting for open-vocabulary 360 degree image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14219, 2024.
[20] Jingwei Ma, Erika Lu, Roni Paiss, Shiran Zada, Aleksander Holynski, Tali Dekel, Brian Curless, Michael Rubinstein, and Forrester Cole. Vidpanos: Generative panoramic videos from casual panning videos. In
SIGGRAPH Asia 2024 Conference Papers, December 2024.
[21] Changgyoon Oh, Wonjune Cho, Yujeong Chae, Dachee Park, Lin Wang, and Kuk-Jin -in Yoon. BIPS: bi bi-modal indoor panorama synthesis via residual depth- saddh adversarial learning. In ECCV (I6), volume 13676 of
Lecture Notes in Computer Science, pages 352-371. Springer, 2022.
[22] Manuel Rey-Area, Mingze Yuan. and Christian Richardt. 360monodepth: High-resolution 360deg monocular depth estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 3762-3772, 2022.
[23] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-rcsolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, pages 10684-10695. 2022.
[24] Hao Shi Yifan Zhou, Kailun Yang, Xiaoting Yin. Ze Wang, Yaozu Ye, Zhe Yin, Shi Meng, Peng Li, and Kaiwei Wang. Panoflow: Learning 360 optical flow for surrounding temporal understanding. IEEE
11

---

Transactions on Intelligent Transportation Systems, 24(5):5570-5585, 2023,
[25]
Tomás Součck and Jakub Lokoë. Transnet v2: An effective deep network architecture for fast shot transition
detection. arXiv preprint arXiv:2088.08 04838, 2020.
[26] Gabriela Ben Melech Stan, Diana Wofk, Scottie Fox, Alex Redden, Will Saxton, Jean Yu, Estelle Aflalo,
Shao-Yea Tseng, Fabio Nonato, Matthias Muller, and Vasudev Lal. Ldm3d: Latent difflusion model for 3d, 2023.
[27] Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa, Mvdiffusion: Enabling
holistic multi-view image generation with correspondence
diffusion, 2023,
[28] Matthew Wallingford, Anand Bhattad, Aditya Kusupati, Vivek Ramanujan, Matt Deitke, Aniruddha Kembhavi, Roozbeh Mottaghi, Wei- Wei-Chiu Ma, and Ali Farhadi. From an image to a scene: Learning to imagine the world from million 360 videos. Advances in Neural Information Processing Systems,
37:17743- 17760, 2024,
[29] Team Wan, Ang Wang. Baole Ai. Bin Wen. Chaojie Mao, Chen- Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang. et al. Wan: Open and advanced large-scale video generative models. arXiv preprint
arXiv:2503.203
2025,
[30] Fu-Yun Wang. Xiaoshi Wu, Zhaoyang Huang, Xiaoyu Shi, Dazhong Shen, Guanglu Song, Yu Liu, and Hongsheng Li. Be-your-ouupainter: Mastering video outpainting through input-specific adaptation. In European Conference on Computer Vision, pages 153-168. Springer. 2025.
[31] Guangcong Wang. Yinuo Yang. Chen Change Loy, and Ziwei Liu. Stylelight: Hdr panorama generation
for lighting estimation and editing, 2022.
[32] Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. Customizing 360-degree panoramas through
text-to-image diffusion models. In WACV, pages 4921 IEEE, 2024.
[33] Jionghao Wang. Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregistered nfov images. In ACM Multimedia, pages 6811 1-6811. ACM, 2023.
[34] Qian Wang. Weiqi Li, Chong Mou, Xinhua Cheng, and Jian Zhang. 360dvd: Controllable panorama video generation with 360-degree video diffusion model. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9913-6923. 2024.
[35] Songsong Wu, Hao Tang, Xiao- Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Scbe, and Yan Yan. Cross- view panorama image synthesis. JEEE Trans, Multim., 25:3546-36 15596-355,,20 2023.
[36] Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. Panodiffusion: 360-degree panorama outpainting via diffusion, 2024.
[37] Yutong Xu, Junhao Du, Jiahe Wang, Yuwei Ning, Sihan Zhou, and Vang Cao. Panonut360: A head and eye tracking dataset for panoramic video. In Proceedings of the 15th ACM Multimedia Systems Conference,
pages 319-325, 2024.
[38] Shilin Yan, Xiaohao Xu, Renrui Zhang, Lingyi Hong, Wenchao Chen, Wengiang Zhang, and Wei Zhang. Panovos: Bridging con-panoaamic and panoramic views with transformer for video segmentation. In European Conference on Computer Vision. pages 346 1663665. Springer. 2024.
[39] Shuai Yang. Jing Tan, Mengehen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layerpano3d: Layered panorama for hyper-immersive scene generation. arXiv preprint
arXiv:2408. 13252. 2024.
[40] Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuamming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvidcox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408. 06072, 2024.
[41] Hu Ye. Jun Zhang. Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.0677 2023.
[42] Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xiaoshui Huang, Dinh Phung. Wanli Ouyang, and Jianfei Cai. Taming stable difffusion for text to 360 panorama image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 6347-6357. 2024.
[43] Cheng Zhang. Qianyi Wu, Camilo Cruz Gambardella, Xiaoshui Huang, Dinh Phung. Wanli Ouyang, and Jianfci Cai. Taming stable diffusion for text to 360 panorama image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 6347-6357. 2024.
[44] Junyi Zhang, Charles Herrmann. Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion. arXiv preprint arxiv:2410.033 03820.0.3.2222.... 2024.
[45] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint 18Xiv:1805.0988/ 2018.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
This paper tackles perspective-tor -0-060 video generation via a dual-branch video denoising framework with rotation-awar designs and novel antipodal masking. Evaluation on 360 video quality and projected perspective video quality with user study demonstrates the effectiveness of the model.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: As discussed in supp.. MonST3R can underestimate large camera rotations which leads to distorted input video the canvas and hence distortions the generated results.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
13

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We discuss the model design and data collection process in Fig. 2 and Sec. 33.6] and in supplementary. We also provide the data annotations in the supplementary folder.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide YouTube ID, time intervals, and text captions a csv file in the supplementary material. We also provide the inference code of our proposed model in the supplementary.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips.co wwwpccppwllcee.eeececcceeeeeeece www.e//gg//eeeeeeeeeeeeeee httpggdddeeeeeebbbiiicacoco wttp///ooeeesssiiiooco wommissssoooiicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We explain all the training and test details in Sec. 4.1 and in the supplementary.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Our metrics use standard benchmarks for the tasks we evaluate. These metrics do not include statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We discuss in Sec. 4.1 that our model is trained on 8 80G A100 GPUs for 60 hours.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp////eaaaee..ccpppppa//ctcca
Answer: [Yes]
Justification:
This paper conforms to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential societal impacts in the supplementary.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
16

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: As described in the supplementary, the codes and dataset will be made available under CC BY 4.0. We also employ manual efforts filter videos with sensitive content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have properly cited all models and datasets we use in the paper. We also describe the licenses the supplementation.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssippsssie thcode cowc datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We describe the data curation and usage in See. and We also attach a readme file for the annotation file (.csv) in the supplementary.
and in supplementary pdf.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We provide the details of user study both in the experiment section and in the supplementary material.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We follow the standards of the country where user studies were performed and personal data analyzed.
Guidelines:
18

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question:
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: LLM is used exclusively for writing, editing, and data processing. See the method section and supplementary material.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
19


Reviews:
[{"review_id": "sbD6vMolGE", "reviewer": "Reviewer_6m1K", "summary": "This paper introduces a novel framework that converts standard videos into immersive 360° content. Key innovations include a dual-branch denoising architecture with spherical attention, an antipodal mask for motion modeling, and rotation-aware processing. Besides, They also introduce a new 360 videos dataset. Experiments demonstrate superior performance over baselines in metrics and user studies, with ablations validating each component.", "questions": "Questions:\n- When MonST3R’s camera pose estimation fails or is inaccurate, does it manifest as noticeable artifacts? Can you discuss possible solutions and illustrate the robustness of the system?\n- How does the inference speed compare to baseline efficiency and resource requirements?\n- Missing details of metric computation: EPE is originally computed in depth estimation. How is it used here to evaluate motion quality? Also, how exactly is the OmniFVD extended from OmniFID here?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed task of perspective-to-360° video generation is new, bridging accessibility with high-end content creation for immersive media. The method demonstrates generalization to in-the-wild videos, significantly boosting its practical utility.\n- Using a cross-domain spherical attention to model reversed motion patterns sounds new and is proved effective by ablations.\n- The rotation-aware modules for in-the-wild videos with different camera poses makes sense and enhances the generation robustness from looking at ablations.\n- Data contribution: provides high-quality 360 video datasets with highlight time intervals to the community.\n\nWeaknesses:\n- While inference speed is mentioned, no clear comparison to baseline efficiency or resource requirements is available.\n- Missing details of metric computation: EPE is originally computed in depth estimation. How is it used here to evaluate motion quality? Also, how exactly is the OmniFVD extended from OmniFID here?\n- The [41] and [42] both refer to the same citation in the reference section", "was_revised": true, "final_justification": "I appreciate the authors for the rebuttal, which has addressed my concerns. I will keep my score as positive.", "rebuttal": "# Responses to Reviewer 6m1K\n\nWe thank the reviewer for the positive feedback and constructive suggestions. We address the comments as below:\n\n**Q.1 No clear comparison to baseline on efficiency and resource requirement.**\n\nThanks for the advice. We add a runtime efficiency breakdown table below and an efficiency comparison table in terms of runtime and VRAM consumption. All experiments are conducted on a single NVIDIA A800 GPU. We report the average runtime based on tests using 16-frame video clips. For all methods listed in the comparison table, only inference time is measured. GPU memory usage is monitored using `nvidia-smi`.\n\nFrom the efficiency comparison, we observe that compared to another V2V model Follow-Your-Canvas (FYC), our model is faster in inference speed and consumes less VRAM. The main reason is that FYC requires multiple rounds of outpainting to ensure outpaint quality, and they also train and infer in `float32`. Thanks to our dual-branch structure, Imagine360 achieves good visual quality in a single pass of inference. \nAnimateDiff and 360DVD have faster inference speeds, which is within expectation, as they are text-guided and image-guided frameworks. In addition, by employing multiple memory optimization techniques (`bfloat16`,`vae slicing`,`cuda.empty_cache`, intermediate tensor cleanup, etc.), our dual-branch structure does not increase VRAM usage by a lot.\n\nIn the runtime breakdown table of our method, we observe that the external camera estimation module takes up a considerable amount of time. Although optimizing the inference speed of off-the-shelf camera pose estimator is beyond the scope of this work, we will seek to upgrade this module with a faster and more robust camera estimator. Both efficiency tables will be added to the final version with a detailed discussion.\n\n\n|               | Model Load | Input IO | Output IO | Camera Estimation | Inference |\n| ------------- | ---------- | -------- | --------- | ----------------- | --------- |\n| Runtime (sec) | 13.29      | 0.36     | 1.74      | 47.88             | 182.76    |\n\n\n|                         | AnimateDiff(I2V) | 360DVD (T2V) | Follow-Your-Canvas (V2V) | Ours (V2V) |\n| ----------------------- | ------------------------------ | ------------ | ------------------------ | ---------- |\n| Inference Runtime (sec) | 122.65                         | 93.18        | 381.89                   | 182.76     |\n| max VRAM (GB)           | 20.83                          | 18.06       | 27.99                    | 25.59      |\n\n**Q.2 Missing details of EPE and OmniFVD metric computation.**\n\nSorry for the confusion and thanks for pointing this out.\nThe EPE metric is adapted from optical flow to evaluate motion accuracy. We compute optical flow for both the generated video $V_{\\text{gen}}$ and the ground-truth $V_{\\text{gt}}$ using PanoFlow, and define EPE as the pixel-wise Euclidean distance between the two flows: $\\text{EPE}(V_{\\text{gen}}, V_{\\text{gt}}) = \\| V_{\\text{gen}} - V_{\\text{gt}} \\|$.\n\nFor OmniFVD, we follow OmniFID by projecting the 360 video into six cubemap views, compute FVD on each view, and report the average as the final OmniFVD score.\nWe will clarify these details in the final version.\n\n**Q.3 When MonST3R’s camera pose estimation fails or is inaccurate, does it manifest as noticeable artifacts? Can you discuss possible solutions and illustrate the robustness of the system?**\n\nThanks for the comment. Empirically, we find that MonST3R produces accurate and stable pose estimations in most test cases. In rare and challenging corner cases, estimating precise camera poses remains an open problem and is beyond the scope of this paper. Nevertheless, as camera pose estimation continues to advance, MonST3R can be seamlessly replaced with more powerful models, enabling our method to handle even more in-the-wild scenarios robustly.\n\n**Q.4 The [41] and [42] both refer to the same citation in the reference section.** \n\nThanks for pointing this out. We will revise the paper and fix this in the final version."}, {"review_id": "OW6OjWKwPH", "reviewer": "Reviewer_vJmB", "summary": "This paper presents Imagine360, a new method for generating 360° panoramic videos from standard perspective video inputs. The core contributions include a dual-branch denoising architecture (for perspective and panoramic views), an antipodal attention mask to model spherical motion consistency, and rotation-aware designs to handle diverse camera poses. The authors also introduce a new curated dataset, YouTube360, consisting of 10K high-quality 360° video clips. Extensive experiments (including comparisons against SOTA methods and ablation studies) demonstrate clear improvements in both perceptual quality and motion coherence.", "questions": "Please refer to the questions in Weaknesses. In addtion, \n\n- Somehow I think the generated videos are flickering if looking closely, a discussion on why this happens and how this could be improved would be beneficial\n\n- Why does this paper choose MonST3R for camera pose estimation? How robust it is and what if using more robust models like MegaSAM etc?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The task of perspective-to-360° video generation is both new and practically relevant. The paper clearly motivates the gap between current methods (which rely on panoramic guidance) and the proposed task. \n- The dual-branch video denoising framework, along with cross-domain attention and the antipodal mask, is a clever adaptation of recent generative modeling ideas to the spherical video domain. The use of antipodal attention for capturing global motion dependencies is well-motivated. The proposed rotation-aware training and inference makes the proposed method more robust on general inputs\n- The experimental results are extensive. In Table 1, the proposed method outperforms all baselines across nearly every metric. The ablations in Table 2 show that each proposed module contributes meaningfully to the performance of the full method. Qualitative comparisons are also provided to show the improvement of the proposed method.\n\n- New dataset. This paper introduces a new dataset YouTube360, which addresses the training data bottleneck in this domain. This dataset is a valuable resource, especially given the lack of clean, high-quality 360° training videos for generative models.\n\n### Weaknesses\n\n- The dual-branch design mostly follows PanFusion, which makes the contribution of this part a bit weak.\n- The dual-branch architecture and rotation-aware modules likely add computational overhead. A discussion or table showing runtime comparisons or inference speed would be valuable for understanding the practical trade-offs.\n- While the new dataset is helpful for training, it remains unclear how the model performs on in-the-wild or low-quality user-generated videos. A discussion or stress test would make the paper’s robustness claims more convincing.", "was_revised": true, "final_justification": "The rebuttal has addressed my previous questions and I would like to vote for accept.", "rebuttal": "# Response to Reviewer vJmB\nWe thank the reviewer for acknowledging our pipeline as *\"a clever adaptation of recent generative modeling ideas to the spherical video domain\"* and our data contribution as *\" addresses the training data bottleneck\", \"valuable\"*. The concerns and questions are addressed in detail as follows.\n\n**Q.1 The dual-branch design mostly follows PanFusion, which makes the contribution of this part a bit weak.** \n\nThanks for the comment. As detailed in Appendix C, we provide a side-by-side comparison between our work and PanFusion. Our work differs significantly in several key aspects, including the task domain, the overall pipeline design, and the introduction of temporal modeling. \n\nWhile we adopt a dual-branch denoising structure, our method goes beyond PanFusion through a cohesive integration of antipodal-aware attention, rotation-aware camera modeling. These designs are specifically tailored for the novel task of perspective-to-360° video generation, which has not been systematically explored before. The effectiveness of each component is validated through ablations (Table 2), and qualitative results demonstrate good generalization to in-the-wild scenarios. \n\nIn addition to the method, we also contribute a clean and high-quality 360 video dataset with temporal interval annotations, which we believe will be valuable for future research.\n\nWe will move the PanFusion comparison to the main paper to make the differences more prominent. We hope the reviewers and AC will consider the novelty of the task, the thoughtful framework design choices, and the data contribution when evaluating the overall contribution of our work.\n\n**Q.2 The dual-branch architecture and rotation-aware modules likely add computational overhead. Add a discussion or table showing runtime comparisons or inference speed for understanding the practical trade-offs.**\n\nThanks for the advice. We provide a runtime breakdown table and an efficiency comparison table in terms of runtime and VRAM consumption below. All experiments are conducted on a single NVIDIA A800 GPU. We report the average runtime based on tests using 16-frame video clips. GPU memory usage is monitored using `nvidia-smi`.\n\nIn the runtime breakdown table, we observe that the external camera estimation module from the rotation-aware designs indeed takes up a considerable amount of time. While optimizing the inference speed of the off-the-shelf camera pose estimator is beyond the scope of this work, we will seek to upgrade this module with a faster and more robust camera estimator. \n\nThe dual-branch architecture, on the other hand, eliminates the need for multi-round inpainting as required in Follow-Your-Canvas (FYC), thus reducing overall inference time. With memory optimization techniques applied, our method achieves a peak VRAM usage of 25.59 GB, which is lower than FYC’s 27.99 GB.\n\n|               | Model Load | Input IO | Output IO | Camera Estimation | Inference |\n| ------------- | ---------- | -------- | --------- | ----------------- | --------- |\n| Runtime (sec) | 13.29      | 0.36     | 1.74      | 47.88             | 182.76    |\n\n|                         | AnimateDiff (I2V) | 360DVD (T2V) | Follow-Your-Canvas (V2V) | Ours (V2V) |\n| ----------------------- | ------------------------------ | ------------ | ------------------------ | ---------- |\n| Inference Runtime (sec) | 122.65                         | 93.18        | 381.89                   | 182.76     |\n| max VRAM (GB)           | 20.83                          | 18.06       | 27.99                    | 25.59      |\n\n**Q.3 Add a discussion or stress test to strengthen the model's robustness claims.**\n\nThanks for the constructive suggestion. \nWe agree that demonstrating robustness under challenging, real-world cases is important. Our current quantitative evaluation (Table 1) already includes a wide range of diverse, in-the-wild scenarios, such as randomly sampled videos from RE10K and 360-1M, as well as videos generated by CogVideoX using random prompts from GPT-4o.\nTo further strengthen our robustness claims, we will curate a set of harder corner cases and include a dedicated stress test and analysis in the final version.\n\n**Q.4 A discussion on video flickering and how this could be improved.**\n\nThanks for the comment. The video flickering primarily stems from our backbone, AnimateDiff, which is known to produce temporal flickers in many cases. Despite this limitation, our method improves upon AnimateDiff in both visual quality and temporal consistency, as shown in Table 1.\nTo further address this issue, we plan to upgrade the backbone to a more advanced video generation model (Wan2.1/2.2), with technical modifications from UNet with DiT. This update is expected to significantly reduce flickering and improve overall temporal coherence. We will update our results with the upgraded backbone in the final version.\n\n**Q.5 Why choose MonST3R for camera pose estimation. How robust it is and what if using more robust models like MegaSAM etc?**\n\nThanks for the comment. \nWe chose MonST3R because it was one of the most robust camera pose estimation models when we primarily developed our method.\nEmpirically, we find that it produces accurate and stable pose estimations in most test cases. \nAs camera pose estimation continues to advance, MonST3R can be seamlessly replaced with more robust models, such as MegaSAM, helping our method to handle even more in-the-wild scenarios. We will upgrade MonST3R to MegaSAM and provide updated results and codebase in the final version."}, {"review_id": "xGAn0g1b0c", "reviewer": "Reviewer_opro", "summary": "This paper develops a pipeline for generating panoramic videos from a monocular perspective video. Specifically, the input perspective video is projected onto a panoramic canvas, where unobserved regions are outpainted by a finetuned dual-branch video inpainting model. This method comprises three key designs: the dual-branch video outpainting, antipodal mask, and rotation-aware design. The dual-batch design synergistically outpaints videos in panoramic and perspective domains, ensuring global and local high-quality visual and motion patterns. The antipodal mask is designed to enhance the motion connection of front-and-back regions. To align the pose changes of the input perspective video, the rotation-aware design is employed to augment the training data. Finally, experimental results demonstrate the superior performance of this proposed method over existing video outpainting and text2pano video generation methods.", "questions": "(1) Since the MonST3R is employed to estimate the camera poses during inference, why not directly use this model to predict camera trajectories from enormous monocular videos, and use these predicted camera poses to replace the rotation-aware design during training? By doing this, we can reduce the domain gap, and the predicted camera poses are more realistic. \n\n(2) Please clarify the contributions of this paper.", "limitations": "No", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n(1) This paper develops a pipeline that can convert an input monocular perspective video into a panoramic 360 video.\n(2) Antipodal mask explicitly connects regions with motion relevance.\n(3) This paper is well-organized and easy to follow.\n\nWeaknesses:\n(1) The technique contribution is limited. This paper is a simple combination of [1-3] and additionally adds a perspective video as input. As shown in Table 2, the most significant design is the dual-branch denoising outpainting, which has been proposed by [1].  And the rotation-aware design is essentially camera poses augmentation.\n\n(2) The antimodal mask design needs further verification. In Table 2, using the antimodal mask design brings minor improvements. Although Fig. 7 demonstrates the efficacy of the antimodal mask, only one case cannot exhibit the generalization capability. Moreover, I notice that the video in supp (a white SUV driving down a dirt road in a forested mountain valley) also has the problem of strange motion patterns (around the image boundary), which do not yield inverse motion effects. I suggest providing two perspective views, including both the input video and its antipodal views, for better comparisons.\n \n[1] Taming stable diffusion for text to 360 panorama image generation\n[2] 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model\n[3] AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning", "was_revised": true, "final_justification": "The technical contributions are weak, but I raised my score because this is a new task, from input monocular video to 360 video.", "rebuttal": "# Response to Reviewer opro\nThanks for the constructive comments and inspiring suggestions. We sincerely apologize for not including more qualitative examples due to the NeurIPS rebuttal policy, but we will add them in the final version of the paper. The comments are addressed in detail as follows.\n\n**Q.1 The technique contribution is limited. This paper is a simple combination of [1-3] and additionally adds a perspective video as input. As shown in Table 2, the most significant design is the dual-branch denoising outpainting, which has been proposed by [1]. And the rotation-aware design is essentially camera poses augmentation.**\n\nThanks for the comment. \nWe respectfully disagree with the claim that our method is a simple combination of [1–3] with an added perspective video input. These works address different tasks: [1] is text-to-image, [2] is text-to-360° video, and [3] is image-to-video. None of them solves the task of generating 360 videos from perspective video signals. Moreover, handling video-conditioned 360 video generation is nontrivial and requires careful modeling of dynamic input camera motion and 360 motion structure. To address these challenges, we introduce two new components: (1) the rotation-aware designs to improve robustness to changing camera trajectories, and (2) antipodal attention masking to model reversed camera motion in 360 views. As shown in Table 2, both modules lead to clear gains, especially in 360 motion metrics like EPE and OmniFVD.\n\nWhile we follow [1] in adopting a dual-branch denoising structure, our pipeline is a smart, cohesive integration of relevant generative components, tailored for perspective-to-360 video generation. The proposed modules are key to addressing the unique challenges of this new setup. Additionally, we contribute a clean and high-quality 360 video dataset with temporal interval annotations, which we believe will benefit the community.\n\nWe already included a side-by-side comparison with PanFusion in Appendix C, highlighting our differences in task domain, pipeline design, and temporal modeling. We will move this comparison to the main paper for a clearer comparison. We hope the reviewers and AC will consider the novelty of the task, the framework design, and the dataset contribution when evaluating our work.\n\n**Q.2 The antimodal mask design needs further verification. In Table 2, using the antimodal mask design brings minor improvements. Also, SUV case in supp. contain strange motion patterns. Suggest providing input and antipodal views for better comparisons.**\n\nThanks for the suggestion. \n\nWe would like to clarify that the antipodal mask is designed to address a unique challenge in 360 video generation, that antipodal regions should undergo reverse motion under camera translation. The purpose of this module is not to directly improve image quality or video smoothness, but rather to provide a geometry-aware bias that facilitates global motion modeling on the sphere. In Table 2, we see modest gains on VBench IQ and MS, which is as expected, since these metrics capture overall appearance and temporal smoothness. We also see **non-trivial gains on OmniFVD** (**257.6 → 204.0**) and **EPE** (**2.7169 → 2.5583**), which reflect 360 motion correctness, the intended benefits of antipodal modeling, indicating improved motion for 360 video generation.\n\nWe also acknowledge that antipodal motion remains a challenging issue. While our method does not fully solve it (SUV case), it provides clear improvements. This is supported by quantitative gains on large evaluation data (Table 2). We strongly agree that visualizing the input and antipodal views is a good idea, and we will include more qualitative video results in the final version to further illustrate the effectiveness of this module.\n\n\n**Q.3 Why not directly use MonST3R to predict camera trajectories from enormous monocular videos, and use these predicted camera poses to replace the rotation-aware design during training? By doing this, we can reduce the domain gap, and the predicted camera poses are more realistic.**\n\nThanks for the insightful suggestion. Our current approach employs random trajectory sampling during training to expose our model to diverse trajectory patterns and improve robustness. Sampling from a camera trajectory library collected by MonST3R on massive real videos seems a promising alternative, as it better reflects the real-world camera motion distribution and may reduce the domain gap. While collecting large-scale real videos and extracting camera trajectories from them is hard to accomplish with our current resources in a short time, we will continue to implement and verify this solution through experiments. The results will be reported in the final version."}, {"review_id": "HCXpVfLCwl", "reviewer": "Reviewer_jUcp", "summary": "This paper presents a novel method, Imagine360, to lift standard perspective video into 360° video. The authors first design a dual-branch denoising U-Net equipped with cross-domain spherical attention to capture both global context and local details. Additionally, they introduce an antipodal mask to model long-range motion dependencies. To handle diverse perspective video inputs, a rotation-aware data sampling strategy is proposed. Experimental results demonstrate the effectiveness of the proposed approach.", "questions": "Please refer to the Strengths And Weaknesses section.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper is well-written and easy to follow.\n\n2.\tThe use of a dual-branch U-Net to capture both global and local correlations is an interesting and effective design choice.\n\n3.\tThe demo is engaging and clearly demonstrates the practical applications of the proposed method.\n\nWeaknesses:\n\n1.\tThe visual quality of the generated videos is relatively poor. Although the proposed method achieves state-of-the-art performance compared to the baselines, the results still exhibit noticeable artifacts, such as blurring and temporal inconsistencies between adjacent frames. Given that recent video generation models (e.g., CogVideo and Wan-2.1) are capable of producing higher-quality outputs, it would be better if the authors could further explain these limitations.\n\n2.\tIt is unclear whether the baseline models were trained using the same dataset collected by the authors. If not, the comparisons may not be entirely fair, and this should be clarified.\n\n3.\tThe proposed method involves a relatively complex pipeline. It would be helpful if the authors provided a comparison of training and inference costs to better assess the model’s efficiency and practical applicability.", "was_revised": false, "final_justification": "", "rebuttal": "# Response for Reviewer jUcp\nThanks for the positive feedback and thoughtful suggestions. We will address the concerns and questions as below.\n\n**Q.1 The visual quality is relatively poor. Given that recent video generation models are capable of producing higher-quality outputs, it would be better if the authors could further explain these limitations.**\n\nThanks for the comment. The visual artifact primarily stems from our backbone, AnimateDiff, which is known to produce temporal flickers in many cases. Despite this limitation, our method improves upon AnimateDiff in both visual quality and temporal consistency, as shown in Table 1. To further address this issue, we plan to upgrade the backbone to a more advanced video generation model (Wan2.1/2.2), with technical modifications from UNet with DiT. This update is expected to significantly reduce visual artifacts and improve overall temporal coherence. We will update our results with the upgraded backbone in the final version.\n\n**Q.2 It is unclear whether the baseline models were trained using the same dataset collected by the authors. If not, the comparisons may not be entirely fair, and this should be clarified.**\n\nThank you for the thoughtful suggestion. The results of the baseline models reported in Table 1 are not trained on YouTube360. However, we have included the performance of 360DVD trained on YouTube360 in Table C of the supplementary material.\n\nWe agree that it is important to clarify the training data used for each model and to also report baselines trained on the same dataset. We add the results of baselines+LoRA trained on our data (marked with \\*) in the table below. Most of the metrics improve with our training data, especially EPE, which reflects 360 motion correctness. \nWe will move the results from Table C to the main paper, clarify the training data source in Table 1, and include additional results of other retrained baselines in the final version.\n\n|                   | IQ $\\uparrow$ | AQ $\\uparrow$ | MS $\\uparrow$ | SC$\\uparrow$ | EPE $\\downarrow$ | \n| ----------------- | ------------- | ------------- | ------------- | ------------ | ---------------- | \n| FYC               | 0.6978        | 0.6432        | 0.9771        | 0.9529       | 3.1767           | \n| Animatediff       | 0.7159        | 0.5840        | 0.9780        | 0.9338       | 3.5393           | \n| 360dvd            | 0.5537        | 0.4745        | 0.9701        | 0.9629       | 3.1904           | \n| FYC+LoRA*         | 0.7075        | 0.6345        | 0.9805        | 0.9656       | 2.9330           | \n| Animatediff+LoRA* | 0.6945        | 0.6019        | 0.9795        | 0.9237       | 3.0068           | \n| 360DVD*           | 0.6991        | 0.5381        | 0.9739        | 0.9696       | 3.1466           | \n| **Ours**          | **0.7487**    | **0.6439**    | **0.9806**    | **0.9710**   | **2.5583**       |\n\n\n**Q.3 Add comparison of training and inference costs to better assess the model’s efficiency and practical applicability.**\n\nThanks for the valuable advice. \nWe provide an inference comparison table in terms of runtime and VRAM below. The experiments are conducted on single NVIDIA A800 card. We report the average runtime based on tests using 16-frame video clips. The memory footprint is evaluated with `nvidia-smi`.\n\nFrom the **inference cost comparison**, we observe that compared to another V2V model Follow-Your-Canvas (FYC), our model is faster in inference speed and consumes less VRAM. The main reason is that FYC requires multiple round of outpainting to ensure outpaint quality, and they also train & infer in `float32`. Thanks to our dual-branch structure, Imagine360 achieves good visual quality in a single pass of inference. \nAnimateDiff and 360DVD have faster inference speeds, which is within expectation, as they are text-guided and image-guided frameworks. In addition, by employing multiple memory optimization techniques (`bfloat16`,`vae slicing`,`cuda.empty_cache`, intermediate tensor cleanup, etc.), results show the dual-branch structure incurs only a small and affordable increase in memory usage, while achieving better generation quality and fast inference speed compared to the baselines.\n\nIn terms of **training cost**, AnimateDiff is trained on WebVid10M dataset using 16 A100 GPUs for 5 epochs. Follow-Your-Canvas (FYC) is trained on Panda70M using 8 A800 GPUs for 50K steps. 360DVD is trained on WEB360 (which includes both RGB and optical flow videos) using a single GPU with ~50GB of VRAM for 100K steps.\nIn contrast, our model is trained on a significantly smaller dataset, containing only ~9K panoramic RGB videos, using 8 A100 GPUs for 20K steps.\nDespite requiring generally less training data and compute, our method achieves superior 360 video generation quality compared to all baselines, demonstrating both data and compute efficiency.\n\n|                         | AnimateDiff (I2V) | 360DVD (T2V) | Follow-Your-Canvas (V2V) | Ours (V2V) |\n| ----------------------- | ------------------------------ | ------------ | ------------------------ | ---------- |\n| Inference Runtime (sec) | 122.65                         | 93.18        | 381.89                   | 182.76     |\n| max VRAM (GB)           | 20.83                          | 18.06       | 27.99                    | 25.59      |\n\n\n**Q.4 Figure 3 overlaps with surrounding text, causing readability issues.**\n\nThanks for pointing this out! We will revise the paper and fix the formatting issues in the final version."}]

Author rebuttal:

Meta-review: All reviewers agree that the idea of perspective-to-360° video generation is novel and technical contribution and evaluation are mostly solid and . The AC recommends accepting the paper, but asks the authors to carefully address the concerns raised by the reviewers in the camera-ready version, including the computation and memory usage comparison, discussion on video flickering, new results etc.
Decision: Accept (poster)