CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation
Authors: Nikolai Kalischek
Michael Oechsle
Fabian Manhardt
Philipp Henzler
Konrad Schindler
Federico Tombari
Venue: ICLR 2025 Spotlight (2025)
Keywords: panorama generation
diffusion
multi-view
Primary area: generative models

Abstract: We introduce a novel method for generating 360° panoramas from text prompts or images. Our approach leverages recent advances in 3D generation by employing multi-view diffusion models to jointly synthesize the six faces of a cubemap. Unlike previous methods that rely on processing equirectangular projections or autoregressive generation, our method treats each face as a standard perspective image, simplifying the generation process and enabling the use of existing multi-view diffusion models. We demonstrate that these models can be adapted to produce high-quality cubemaps without requiring correspondence-aware attention layers. Our model allows for fine-grained text control, generates high resolution panorama images and generalizes well beyond its training set, whilst achieving state-of-the-art results, both qualitatively and quantitatively.

Published as a conference paper at ICLR 2025
CUBEDIFF: REPURPOSING DIFFUSION-BASED IMAGE MODELS FOR PANORAMA GENERATION
Nikolai Kalischek ETH Zürich, Google
Michael Oechsle Google
Fabian Manhardt Google
Philipp Henzler Google
Konrad Schindler
Federico Tombari Google
ETH Zürich
ABSTRACT
We introduce a novel method for generating 360 panoramas from text prompts images. Our approach leverages recent advances in 3D generation by employ- ing multi-view diffusion models to jointly synthesize the six faces of a cubemap. Unlike previous methods that rely on processing equirectangular projections autoregressive generation, our method treats each face as standard perspec- tive image, simplifying the generation process and enabling the of existing multi-view diffusion models. We demonstrate that these models can be adapted produce high-quality cubemaps without requiring correspondence-avaal atten- tion layers. Our model allows for fine-gra grained text control, generates high resolu- tion panorama images and generalizes well beyond its training set, whilst achiev- ing state-of-the-art results, both qualitatively and quantitatively. Project page: figittuu. .gg.tub.
Cube Diff
Indoors
Outdoors
Figure 1: CubeDiff leverages cubmaps represent 360 panoramas and denoises all faces together in single pass. In contrast to other works, Cubediff does not need to consider distortions, since it operatkes on common 90º FOV perspective images, maing possible directly utilize the internet- scale image prior of the underlying diffusion model.

---

Published as a conference paper at ICLR 2025
INTRODUCTION
Recent advances in diffusion-baset generative models have seen tremendous progress the last two years, enabling a wide range of applications from artistic expression and product design to per- sonalized content creation. Beyond generating realistic and diverse images based on texi-to-image models (Rombach et al. 2022; Saharia et al., (022). these models are now capable of more complex tasks such as 3D asset creation (e.g., (Kalischek et al. 2022 Wang al. [024] Mohammad Khalid
et al. 2022, Poole et al. (2022), ), estimating scene properties such as depth or semantics (Ke et al.,
2024
Baranchuk et al., 2021). illumination changes (Jin et al., 2024 Zhao et al. 2024 Zeng al,
2024
and generation of multi-view consistent images (Gao et al. 2024b) [Tang al. 2023).
The latter is particularly interesting in virtual reality, gaming and entertainment, where 3D consis- tency is crucial for fully immersive experiences and thus user satisfaction. However, synthesizing high-quality, visually coherent panoramas presents unique challenges. First, capturing sufficient panoramic data is tedious and costly, specialized cameras and/or additional processing needed to remove stitching artifacts. Consequently, models must be trained in a low-data regime making them prone to overfitting, this limiting their generalization capabilities. Exemplary, lot of models are restricted to indoor environments only (Wu et al., 2023 Song et al., 2023). Second, panoramas must fulfill additional constraints compared perspective images. Most notably, the image borders must align to allow seamless wrap-around around. But there are also more intricate, semantic constraints, e.g., the viewing frustum must cover the entire scene. Hence, when generating a panorama of bed- room, it must contain exactly one bed, at least one door, etc. On the other hand, outdoor panoramas should maintain realistic spatial relationships between elements.
To satisfy those requirements, prior work had to introduce complex additional model components (Gao et al. 2024a; Tang et al. 2023 Yang et al. 2024), or employ dedicated mechanisms such as autoregressive outpainting from perspective view (causing artifacts like content drift and the Janus effect ( (Wang et al. 2023), and circular padding to enforce consistent wrap-around (Feng et al. 2023
Wu et al. (023)).
We introduce a simple yet highly effective solution: we generate panoramas using fine-tuned multi- view diffusion model, following recent line of work (Gao et al., 2024b Tang al. 2023 Zhang
ct al. 2023b). This approach leverages the inherent properties of cubemaps, where a 360 X 180
panorama is represented by six perspective images the faces of a cube. This allows to fully cycle a pretrained text-to-image model, enabling generalization far beyond the limited training data. Contrary to existing methods, the architectural modifications
require ensure consistency be- tween cube faces are minimal: all attention layers are inflated by one additional dimension to enable crosstalk between the six faces. This simple modification, combined with fine-tuning on panorama data, achieves state-of-the-ar results with significant visual and semantic coherence. Additionally, the model allows for fine-ge ane-grained text control by training with specifce image-text pairs, easily generated by prompting an LLM to produce per face text descriptions.
Our key insight is that existing, generative image models can be easily extended generate high- resolution panoramas, by performing diffusion in cubemap space and adding attention mechanisms to other faces within the cubemap, see Figure]] The resulting model
enables consistent image generation across all cubemap faces and preserves the internet-scale image prior of the underlying diffusion model to generalize beyond the training panoramas; delivers state-of-the-art results panorama generation, both qualitatively and quantitatively, and outperforms previous methods in terms of visual fidelity and coherence: enables efficient high-resolution synthesis, benefiting from current and future advances in off- the-shelf image diffusion models; allows for novel fine-grained text control, enabling guide the generation with detailed textual descriptions.
RELATED WORK
Similar to 3D generative modelling, training data for panorama generation is scarce and much effort has been spent how repurpose standard perspective image priors for panoramas. The preva-
2

---

Published as a conference paper at ICLR 2025
lent approach has been to autoregressively outpaint panoramas, more recently multi-view diffusion models have attracted interest. We now discuss relevant works and differences to our approach.
2.1 PANORAMA GENERATION.
Most panorama generators operate in equirectangular projection, thus having deal with it severe nonlinear distortions (especially near the poles). Previous methods either autoregressively outpaint
the panorama (Gao et al., 2024a Lu et al. 2024 Wang al. 2023) or generate the entire equirect- angular image in one shot (Feng et al. 2023] Wu et al., 2023). They are commonly conditioned on
either single narrow field-of-view image (Akimoto al. 2022) or solely on a text prompt (Chen iet al. (2022). The state of the art are diffusion models. which have gradually replaced adversarial approaches (Akimoto al. 2022, Somanath & Kurz 2021). Feng et al. (2023) fine-tune latent diffusion model on a panorama dataset and apply circular blending strategy in the denoising and
decoding stages enforce consistent wrap-around. Similarly, Wu et al.) (2023) stitches the right part of the image to the left part in latent space in each denoising step. Such blending improves the results, but encumbers the inference step. In our method it is not required. [u et al. (2024) propose to autoregressively outpaint a panorama with complex architecture of submodules for panorama- aware visual guidance, NFoV guidance and panorama-aware geometric guidance. In Wang et al.) (022)), the authors extend the outpainting task to ingest multiple NFoV images of the same scene. two-stage network predicts their relative rotations, then a diffusion model with ControlNet (Zhang al. 2023a) outpaints the panorama based on the projected inputs. Recently (Voynov et al.) 2023) introduce a diffusion model with control over the rendering geometry, including panoramic outputs. Gao et al. 2024a) additionally incorporate a state space model to aggregate global information into cross-attention layers of the diffusion model, building up the panorama by inpainting empty regions. The present work demonstrates that, with the right representation, high-quality panoramas can be obtained without inflating the complexity and brittleness of the architecture. Related to panorama generation is the more modest strategy to alter existing panoramas by injecting user-defined style, in either equirectangular or cubemap projection (Yang et al.) 2024 Song al. 2023).
2.2 MULTI-VIEW DIFFUSION
Multi-view diffusion models offer a compelling alternative to equirectangular or autoregressive panorama generators. Zhang et al. (2023b) introduces compositional diffusion scheme that enables the generation of large-scale content, leveraging models trained on smaller constituent parts. That work is based on factor graphs, and demonstrates how the cubemap can be turned into factor graph in order to train a diffusion model conditioned on segmentation maps. The work most closely related ours is Tang al. (2023). It aims to generate cylindrical panoramas (i.e., 360 horizontal field of view, but restricted vertical view angle). The authors propose sophisticated correspondence-avaaa cross-attention between local neighborhoods of eight perspective feature maps spaced at 45 angles. Recently, Gao et al. (2024b) and (hi et al. (2023) discovered that expanded attention layers that con- nect not only features within an image but also across multiple images, are beneficial when handling multiple object-centric views. Our approach turns this setup inside-out and extends pretrained -it-tt--mmae (T2I) model in similar manner for panorama generation. We instead do not require camera pose or 3D information, due to the fixed viewing geometry of the cubemap.
PANORAMA REPRESENTATIONS
Panoramic images aim to capture a complete 360 180 view of a scene from fixed view point. There exist several different panorama representations in literature, each with its own advantages and drawbacks. This section briefly discusses the most prominent ones.
Spherical projection maps a 360 view onto sphere, preserving the geometric relationships between points in the scene. Points are generally defined using longitude and latitude. While con- ceptually intuitive, directly utilizing spherical representation for image processing is challenging due to difficulties in representing a sphere on a flat image plane, which often leads to distortions and non-uniform sampling densities in practical implementations.
3

---

Published as a conference paper at ICLR 2025
M10, () ~
~- (4(0 T?
(down prumps)
Eopet (177%
(front promps)
Reslaee MMD
synced
V2
MSE
V
Fash Atteriion
UNee
Redape (d7'
CharLayer
(a)
(b)
Figure 2: An overview of our training pipeline and panorama model. (a) We project all training panoramas onto cubmap and feed the faces to our frozen VAE encoder with synchronized Group- Norm to obtain the respective latents and enrich them with panorama-specific positional encodings for explicit spatial awareness. (b) We only train the inflated attention layers be cross-frame aware.
Equirectangular projection projects the spherical panorama onto a 2D rectangle. To this end, latitude and longitude coordinates on the sphere are mapped to vertical and horizontal coordinates on a rectangle. While widely used due to its simplicity, equirectangular projection suffers from significant distortions, especially near the poles where horizontal stretching becomes extreme. This distortion affects both visual quality and the performance of algorithms processing equirectangular panoramas, as most existing T2I models process images with NFoV images.
Cubemaps offer an alternative representation where a 360 view is projected onto the six faces of cube. Each face captures 90 field of view, resulting in six perspective images that can be seam- lessly stitched together. This representation avoids the polar distortions inherent to equirectangular projections, providing more uniform sampling, making it highly applicable to existing diffusion models trained on vast amount of perspective images. However, note that cubemaps introduce dis- continuities the edges of the cube faces, which needs be handled carefully.
4 METHOD
We introduce CubeDiff, a novel approach for generating high-quality, consistent panoramas using the cubemap representation. CubeDiff generates the six perspective views of a cubemap in parallel and context-aware manner, exploiting the strengths of pretrained T2I diffusion models. Below, we delve into the architectural choices that enable CubeDiff to achieve high-quality and consistent panoramas, while retaining strong generalization capabilities inherited from the pretrained model. Similar to Gao et al. (2024b). CubelDiff' comprises variational autoencoder ( (VAH) and latent diffusion model (LDM), mirroring the structure of conventional T2I diffusion models. However, we carefully adapt each component for effective multi-view panorama generation.
4.1 MODEL ARCHITECTURE
The latents produced by the VAE are used to fine-tune pretrained LDM operating on a 128x 1xxx1280 128x8 latent space, initialized with weights from model trained on large-scale image dataset. The pre- trained LDM consists of an architecture similar to Stable Diffusion (Rombach et al. 2022). which build with multiple convolutional, self-attention, and cross-attention layers. To enable cross-view awareness and maintain global consistency, we inflate all existing 2D attention layers, i.e. both self- attention and cross-atie attention for text conditioning. These layers, adapted from (Shi et al. 2023). extend the attention mechanism across all six cube faces, allowing the model to learn relationships and dependencies between different viewpoints. Inflating of layers can be easily conducted by ex- tending the token sequence length from 6 x (hw) x ( to b x (thw) I, e.g. for self-attention, where b is the batch size, hw the flattened spatial size and to more sophisticated attention layers (Tang et al. 2023 Huang et al. 2024). it in turn enables to retain the original pretrained attention weights, which reduces the risk of overfitting and thus greatly improves overall performance.
6 the cube length. While this is different
4

---

Published as a conference paper at ICLR 2025
The LDM receives two conditioning signals. We incorporate text embeddings, either one common prompt or one prompt for each face, and a single conditional view of the scene (w.lo.g. we assume the front face of the cube). During training, views to the noisy latents of the target views, providing the LDM with complete context informa- tion. Furthermore, we incorporate a binary mask channel into the latent representations. This mask distinguishes between conditioning views (provided as clean latents) and target views (subjected to noise injection during training). We show an overview of our model architecture in Figure 2
concatenate the VAE latents of the conditioning
4.2 SYNCHRONIZED GROUPNORM
Our VAE architecture incorporates synchronized group normalization, crucial element for achiev- ing consistent color tones across the generated panorama. Since our VAE processes the six faces of a cubemap as a batch of six individual images, standard group normalization can lead to subtle color inconsistencies among different views (c.f. fig. [6a). This occurs as feature statistics are computed and normalized independently for each image in the batch. Without synchronization, encoding and decoding a panorama results in noticeable shifts, particularly evident in the equirectangular pro- jection. Synchronized group normalization addresses this issue by jointly normalizing feature acti- vations across both spatial and inter-view dimensions. Consequently, synchronized group normal- ization contributes significantly to the generation of visually harmonious and coherent panoramas. Similar effects have been observed in (He et al.] 2023). We further discuss this in Section 5.6 and compare synchronized and unsynchronized results in Figure 6a
4.3 POSITIONAL ENCODING
To provide the LDM with explicit spatial awareness within the cubemap, we augment the latent representations with positional encodings derived from the 3D geometry of the cube. For each point on a cube face, we compute its corresponding UV coordinates on the unit cube, defined by:
arctan 2(x 2)
U arctan 2 V x2 + +²²),
where (x, y, 2) are the 3D coordinates of the point on the cube face, projected onto the unit cube. These UV coordinates are then normalized to [0, 1] and concatenated as two additional channels to the (noisy) latents. This positional encoding scheme provides the model with information about the spatial location of each latent patch within its respective cube face, facilitating the generation of panoramas with consistent geometry and object relationships across views.
4.4 OVERLAPPING PREDICTIONS
To further enhance the geometric and color consistency across cube faces, we introduce overlapping predictions during both training and generation. Instead of generating each face with 90º field of view VF, we enlarge the FoV by 2.5e on each side, resulting in an effective FoV of 95 per face. This means each generated face includes small overlap with its neighboring ones. This overlapping generation strategy serves two purposes. During training, it encourages the model to learn consistent representations across adjacent faces, the overlapping regions provide additional context and constraints. During panorama assembly, we discard these overlapping regions and only retain the central 908 portion of each generated face. This strategy effectively avoids the need for explicit blending operations at the cube face boundaries, which can sometimes introduce subtle artifacts. The overlaps can be seen at the boundaries of the cubemaps in Figure (e.g. e the duplicated fireplace in the right and back views) and in the appendix.
4.5 CLASSIFIER-FEEE FREE GUIDANCE
We employ classifier-free guidance (CFG) (Ho & Salimans 2022) on both the text and image con- ditions during training. Thereby, we randomly drop either the text prompt, the conditional image, or both. When the text prompt dropped, layers; when the conditional image is dropped, its corresponding tokens in the self-attention layers are masked out by setting them to negative infinity, effectively zeroing out their attention weights. This training procedure enables diverse panorama generation scenarios during inference. Users can provide both text and image conditions for maximum control and fidelity or drop both or either condition explore unconditional generation modes.
replaced with null tokens in the cross-attention
5

---

Published as a conference paper at ICLR 2025
5
EXPERIMENTS
This section details our experimental setup, followed by quantitative and qualitative evaluations. We compare the performance of CubeDiff against the state-of-the tete-of-theean art and ablate our design choices.
5.1 EVALUATION PROTOCOL
Training and inference setup We finetune model using Adam (Kingma & Ba 2014 and train for 30,000 iterations with batch size 64. The learning rate ramped up to 8 x 10 in the first 10,000 steps. During training. employ classifier-free guidance, dropping conditional signals 10% of the time. We find it important to not only drop the text condition in the cross-attention layers but also zero out the input condition in the self-attention layers. The diffusion model finetuned using v-prediction (Salimans & Ho 2022). We employ DDIM sampling (Song et al. 2020) with 50 steps during inference.
Datasets Training. We train on a mixture of indoor and outdoor environments by combining multiple publicly available sources, including Polyhaven (polyhaven.com accessed 09/2024). Hu- mus Perssso accessed 09/2024), Structured3D (Zheng et al.) 2020) and Pano360 Kocabas et al. 2021 giving total around 48000 panoramas for training. While Humus provides an explicit cubemap representations, all other datasets come with equirectangular panoramas. We thus first generate cubemaps from these panoramas using standard perspective projection, ensuring consistent overlap between adjacent faces. To further enable text- text-guided panorama generation, we infer textual descriptions for each panorama in the datasets using the publicly available Gemini model (Gemini Team Google 2023). We explore two captioning strategies: (1) generating a single caption for the entire panorama by providing Gemini with all six cube faces as input and (2) generating individual captions for each face independently, enabling fine-grann fine-grained text control.
Testing. We evaluate our method on the common Laval Indoor (Gardner et al., 2017) and Sun360 (Xiao et al. (2018) datasets. Laval Indoor consists of over 2100 high quality panorama captures of various indoor environments, Sun360 encompasses around 1000 panoramas including both - indoor and outdoor scenes. Note that we use those datasets only for evaluation, while Diffu- sion360 also uses Sun360 for training and OmniDreamer even leverages both datasets to train their models. Nonetheless, we decided to these datasets for the sake of faimess and due to the lack of any proper overlapping test datasets.
Metrics We use various metrics and modalities for evaluation including perceptual metrics, text alignment, and a user study.
Perceptual Metrics. We use the very common Fréchet Inception Distance (FID) (Heusel et al. 2017) metric to measure the similarity between the distribution of real and generated images in a feature space derived from pretrained Inception network. Lower FID) scores indicate greater similarity and, thus, higher image realism; We additionally report the CLIP-FID (Kykääänniemi et al. 2022) metric, replacing the Inception network with CLIP (Radford et al. 2021 to leverage its semantic understanding capabilities through a joint image-text embedding space. This metric captures thus both visual fidelity and text-image alignment; Finally, we employ the kernel inception distance (inkowii et al. 2018). Similar to FID, KID uses features from a pre- trained network, how- ever, quantifies the difference between real and generated data distributions using the maximum mean discrepancy rather than the Fréchet distance.
Text Alignment. To measure text alignment we refer to the common CLIP score (Hessel al.) 2021) The CLIP score computes the cosine similarity within the shared text-image embedding to measure the agreement between generated panoramas and their corresponding text prompts. Hence, higher CLIP score indicates stronger semantic agreement between image and text.
Competitors We compare CubeDiff to various state-of-the-art panorama generation methods. As for plain text to panorama generation, we employ Text2Light (Chen et al. 2022) and PanFu- sion (Zhang et al., 2024) to serve our main competitors. For single image conditioning, we spectively use OmniDreame (Lu et al. 2024) and PanoDiffusion Wu et al. (2023) as representatives for autoregressive and direct panorama generation based approaches. Finally, compare against

---

Published as a conference paper at ICLR 2025
Figure 3: Cubemaps and panoramas generated by CubeDiff with image and text condition. We depict diverse of generated panoramas including indoor, outdoors, bright and dark scenes. In all settings, CubeDiff produces high quality and realistic panoramas that align with the input image.
Diffusion360 (Feng et al. 2023) and MVDiffusion (Tang et al.) 2023) for text and image condition- ing based methods. Note that while Diffusion360 directly outputs panorama images, MVDiffusion instead employs multi-view diffusion models with a custom cross attention mechanism. Overall, the choice of baselines represents variety of different generation techniques, covering various different tasks. Please note that none of the existing methods besides MVDiffusion offers the possibility to condition specific parts of the panorama individual text prompts.
5.2 QUALITATIVE EVALUATION
In this section, we provide a qualitative evaluation of our method. We first present several conditional image generations of our method, before comparing CubeDiff against the state-of-the-art.
Conditional image generation. In Figure 3 show generated panoramas given text-iminn ext--imaee pairs as condition. We considered input conditions that cover broad range of scenes, such as outdoor and indoor scenes, bright and dark settings well texture rich and uniformly colored areas. Note that we do not show the text conditions due to limited space, however, we provide them in the appendix. We that our approach yields high quality results under these diverse input settings. We especially emphasize the level of detail and geometric consistency beyond the input image.
Qualitative comparison. For visual comparison against the state-of-the-arr, we show generated panoramas and their respective perspective projections in Figure 4 To this end, we sample random image and text pairs from the LAVAL Indoor dataset. We further group the methods according to their input modalities. Compared to the text-only approach Text2Light, our method is able to produce much more complex panoramas with better details and visual appeal. As for image-only approaches, we see that CubeDiff is capable of producing the most realistic panoramas. In particu- lar, while OmniDreamer suffers from blurry regions, PanoDiffusion is not able properly transfer
7

---

Published as a conference paper at ICLR 2025
Texr2l 1i22igght
Tateooly
PanPusion
OmniDramme
Imsgrwwnit
PinoDiffssio
MVDidtision
Image and Text
Deftuson100
Ourhing-mston
Figure 4: Qualitative comparison between CubeDiff and baselines on the LAVAL Indoor Dataset. Besides Text2Light, all panoramas are generated using the center face as input condi- tion and additional text prompts if applicable. For each sample show the panorama image as well as two projected images. Please zoom in to compare the different methods.
the input image appearance across the whole panorama. Finally, also for text and image condition- ing our method again produces the best results, especially in terms of geometry. For example, while MVDiffusion is indeed capable of generating high quality images, the method sometimes produces inaccurate geometries as, for example, some walls and hand rails exhibit bending artifacts after per- spective projection. Similarly, Diffusion360 occasionally suffers from implausible indoor layouts. To summarize, despite of using different input modalities, CubeDiff always generates high qual- ity panoramas, surpassing all other state-of-the-ar works in terms visual appeal and geometric consistency.
8

---

Published as a conference paper at ICLR 2025
LAVAL Indoor
SUN360
FID
KID (x10")L
Clip-FID L FAED 1
CS
FID.
KID (x10") Clip-FIDJ
FAED
CS
iigt
28.3
1.45
11.5
136.1
25.18
60.1
4.31
31.3
82.9
23.27
PanFusion
41.7
2.85
19.8
71.7
26.58
30.0
1.42
7.8
44.5
25.28
OmniDreumer
71.0
5.17
23.9
19.2
92.3
8.89
51.7
30.4
PunoDiffusion
58.6
4.08
26.6
106.8
52.9
3.51
28.9
98.0
OurSing
11.7
0.47
4.4
22.0
27.4
1.35
11.5
8.9
Diffusion600
33.1
2.07
16.9
23.7
26.38
45,4
3.73
18.5
12.6
22.89
OurStmpst 9.5
0.32
3.2
18.4
27.02
25.5
1.33
8.1
7.6
25,00
MVDiffusion 25.7
1.11
13.5
27.44
50.9
3.71
15.4
32.3
25.54
OurSingrmatted
10.0
0.35
4.1
21.2
30.17
24.1
1.33
7.0
5.7
28.14
Table I: Quantitative Evaluation the Laval Indoor and SUN360 dataset. We provide com- parison to various competitors and different input modalities. The first block of rows are text-only methods, the second image-only, the third image and single text description and the last block are image and multi-caption methods.
5.3 QUANTITATIVE EVALUATION
In this section, provide the results of quantitative evaluation on the Laval Indoor and the SUN360 dataset. We evaluate all methods on perceptual quality and consistency.
In Table we provide quantitative results for visual qual- ity. Our method outperforms all competitors significantly, regardless of input modalities. For example, we can report FID score of 9.47 on Laval Indoor, which is 270% rel- ative improvement compared to the second best performing method MVDIffusion, reporting score of 25.7. Compared to works that use only image or text as input conditioning, the gap even widens with Text2Light and PanoDiffusion re- spectively reporting FID of 28.3 and 58.6. This trend holds across all metrics. Interestingly, CubeDiff performs similarly across different input modalities, demonstrating its strong generalizibility.
However, the provided perceptual metrics can only evaluate the overall realism of the generated panoramas and are not capable of capturing consistency towards input. We next study the alignment to the input text prompt. To this end, we leverage the CLIP score measure how well the generated panoramas align with the text input. As can be seen in the table our method surpasses the state-of-the-an again by significant amount for all datasets and modalities, showing how precisely our model respects the textual input.
Figure 5: Fine-grained Text Con- trol. We show an example for fine- grained-text control of the back face. Our model is able to change details following the provided prompt. First, we add a golden globe above the fire- place; second, we place a picture above the fireplace; third, we leave the space above empty; last, we stead add a bookshelf above it.
5.4
USER STUDY
We conducted user study with two-alternttive forced choice (2AFC) survey to evaluate our panorama generation method. Each of the 28 participants was shown 30 pairs of generated panoramas alongside the original conditioning image and asked to select their preferred option based on quality, composition, style, and alignment with the condition image.
Our method outperformed competitors statistically (p < 0.1, binomial test). Specifically, 16.9%, 17.3%, and 19.5% of participants preferred our single-image, multi-image, and no-text variants, spectively. The no-text variant nearly matched the ground truth preference (19.9%), demonstrating method's 's ability to generate realistic and accurate panoramas. In contrast, OmniDreamer, Pan- Diiffusion, MVDiffusion, and Diffusion360 had significantly lower preference rates 1.7%. 5.3%, 7.0%, and 12.3%, respectively.

---

Published as a conference paper at ICLR 2025
uneynced CN
Sone CN
(a)
Figure 6: Ablation on synchronized GN and overlap prediction. (a) Top: Group normalization over the spatial dimension only. Bottom: Additional normalization over the frame dimension. (b) Top: Panoramas without overlapping cube faces. Bottom:
Panoramas with our standard 2.5 overlap.
5.5 FINE-GRAINED TEXT CONTROL
Different to all competitors, our method enables complete fine-grained and per-face text control. For example in Figure we show results for providing different text descriptions for the back face. We can always generate visually appealing results, regardless of the object we place above the fireplace.
5.6 ABLATIONS
Synchronized Group Norm (GN) Synchronized GN ensures consistency across cube faces by normalizing over both spatial and frame dimensions, as shown in Figure 6aa Without it, models often exhibit color inconsistencies and artifacts cube face boundaries. While metrics like FAED may not capture these subtle issues, synchronized GN significantly improves visual quality.
Overlapping predictions mitigate discontinuities cube face boundaries by introducing small overlaps, illustrated in Figure 6b) This ensures seamless transitions, with non-overlapping regions cropped for the final panorama. The approach leverages global context from full attention, eliminating visible seams without additional VAE finetuning.
Overlapping Prediction
CONCLUSION
This work introduces novel approach to panorama generation leveraging pretrained text-to-image diffusion models applied to cubemap representation. By enabling attention across the cube faces, method achieves state-of-the-art results in terms of visual fidelity and coherence, while requiring minimal architectural changes. This approach not only inherits the strengths of existing diffusion models, including nigh-resolution synthesis and generalization capabilities, but also unlocks fine- grained text control over the generated panorama. This opens up exciting new possibilities for creative applications and paves the way for future research in controllable panorama generation.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Diverse plausible 360-degree image out- painting for efficient 3DCG background creation, In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Dmitry Baranchuk, Ivan Rubachev, Andrey Voynov, Valentin Khrulkov, and Artem Babenko. Label- efficient semantic segmentation with diffusion models. arXiv preprint arXiv:2112.03/26, 2021.
Mikolaj Binikowsi,, Danica Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. arXiv preprint 1001:0100,,,, 2018.
Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2Light: Zero-shot text-driven HDR panorama generation. ACM Transactions on Graphics (TOG), 41(6), 2022.
Mengyang Feng, Jinlin Liu, Miaomiao Cui, and Xuansong Xie. Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models. arXiv preprint 1rxxix:2311.31144 2023.
Penglei Gao, Kai Yao, Tiandi Ye, Steven Wang, Yuan Yao, and Xiaofeng Wang. OPa-Ma: Text guided Mamba for 360-degree image out-painting. arXiv preprint arXiv:2407. 10923, 2024a.
Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin Birinn-,aaala,, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. CAT3D: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405. 10314, 2024b.
Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Chris- tian Gagné, and François Lalonde. Learning to predict indoor illumination from a single image. arXiv preprint arXivv11044 700900090 2017.
Gemini Team Google. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
Yingqing He, Shaoshu Yang, Haoxin Chen, Xiaodong Cun, Menghan Xia, Yong Zhang, Xintao Wang, Ran He, Qifeng Chen, and Ying Shan. ScaleCrafter: Tuning-free higher-resolution vi- sual generation with diffusion models. International Conference on Learning Representations (ICLR), 2023.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.
GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.
Jonathan Ho and Tim Salimans.
Classifier-free diffusion guidance.
arXiv preprint
arXiv:2207. 12598, 2022.
Zehuan Huang, Hao Wen, Junting Dong, Yaohui Wang. Yangguang Li, Xinyuan Chen, Yan-Pei Cao, Ding Liang, Yu Qiao, Bo Dai, et al. EpiDiff: Enhancing multi-view synthesis via local- ized epipolar-constrained diffusion. In IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. arXiv preprint arXiv:2406.07520, 2024.
Nikolai Kalischek, Torben Peters, Jan D Wegner, and Konrad Schindler. TetraDiffusion: Tetrahedral diffusion models for 3d shape generation. In European Conference Computer Vision (ECCV), 2022.
Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Metzger, Rodrigo Caye Daudt, and Konrad Schindler. Repurposing diffusion- image generators for monocular depth estimation. In IFFF/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2024.
11

---

Published as a conference paper at ICLR 2025
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 12 2014.
Muhammed Kocabas, Chun-Hao P. Huang, Joachim Tesch, Lea Müller, Otmar Hilliges, and Michael J. Black. SPEC: Seeing people in the wild with an estimated camera. In International Conference on Computer Vision (ICCV), 2021.
Tuomas Kynkäääniemii, Tero Karras, Miik Aittala, Timo Aila, and Jaakko Lehtinen. The role of ImageNet classes in Fréchet Inception Distance. arXiv preprint arXiv:2203.0602 2022.
Zhuqiang Lu, Kun Hu, Chaoyue Wang, Lei Bai, and Zhiyong Wang. Autoregressive omni-aware outpainting for open-vocabulary 360-degree image generation. In AAAI Conference Artificial Intelligence (AAAI), volume 38, 2024.
Nasir Mohammad Khalid, Tianhao Xie, Eugene Belilovsky, and Tiberiu Popa. CLIP-Mesh: Gener- ating textured meshes from text using pretrained image-text models. In SIGGRAPH Asia, 2022.
Emil Persson. Texture from Humus. http:://www..nnnsssssom/ddehehhs Textues accessed 09/2024.
polyhaven.com. HDRIs. hutps:/holyttaven.com//hiii accessed 09/2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. DreamFusion: Text-to-3d using 2d diffusion. arXiv preprint 2209: 14988, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning (ICML), 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- tion Processing Systems (NeurIPS), 2022.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.0051 2022.
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3d generation. arXiv preprint 2tti::2308.1611222 2023.
Gowri Somanath and Daniel Kurz. HDR environment map estimation for real-time augmented reality. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500/ 2020.
Liangchen Song, Liangliang Cao, Hongyu Xu, Kai Kang, Feng Tang, Junsong Yuan, and Yang Zhao. RoomDreamer: Text-driven 3d indoor scene synthesis with coherent geometry and texture. arXiv preprint ar Xiv:23335 11377, 2023.
Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. MVDiffu- sion: Enabling holistic multi-view image generation with correspondence-aware diffusion. arXiv preprint ar arXiv:2307.01097 2023.
Andrey Voynov, Amir Hertz, Moab Arar, Shlomi Fruchter, and Daniel Cohen-Or. Curved diffusion: A generative model with optical geometry control. arXiv preprint arxiv:2311.7760,, 2023.
Jionghao Wang, Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregistered NFoV images. arXiv preprint a30iv:2308. 14686, 2023.
12

---

Published as a conference paper at ICLR 2025
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifieDreamer: Bigh-fideling and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems (NeurIPS), 36, 2024.
Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. PanoDiffusion: 360-degree panorama outpainting via diffusion. In International Conference on Learning Representations (ICLR), 2023.
J. Xiao, K. A. Ehinger, A. Oliva, and A. Torralba. Recognizing scene viewpoint using panoramic place representation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Bangbang Yang. Wenqi Dong, Lin Ma, Wenbo Hu, Xiao Liu, Zhaopeng Cui, and Yuewen Ma. DreamSpace: Dreaming your room space with text-driven panoramic texture propagation. In IEEE Conference Virtual Reality and 3D User Interfaces (VR), 2024.
Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong. Hongzhi Wu, and Xin Tong. DiLightNet: Fine-grained lighting control for diffusion- image generation. In ACM SIGGRAPH, 2024.
Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xiaoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. Taming stable diffusion for text 360 ((deg) panorama image generation. arXiv preprint Atl::2004.0794455 2024.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In IEEE/CVF International Conference on Computer Vision, 2023a.
Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming- Yu Liu. DiftCollage: Parallel generation large content with diffusion models. In JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023b.
Xiaoming Zhao, Pratul P Srinivasan, Dor Verbin, Keunhong Park, Ricardo Martin Brualla, and Philipp Henzler. IllumiNeRF: 3d relighting without inverse rendering. arXiv preprint 2.Xv:22406.06527, 2024.
Jia Zheng, Junfei Zhang, Jing Li, Rui Tang. Shenghua Gao, and Zihan Zhou. Structured3d: A large photo-realistic dataset for structured 3d modeling. In European Conference on Computer Vision (ECCV), 2020.
13


Reviews:
[{"review_id": "WIE9rtoZMd", "reviewer": "Reviewer_MXUG", "summary": "This work introduces CubeDiff, which is a simple and efficient method for generating 360 panorama from image and text conditions. CubeDiff builds on pretrained a multi-view diffusion model and fine-tunes it to output cubemap representations that is later stitched together into a single panorama. Compared to previous approaches that employ complex cross-view attention mechanisms, CubeDiff simply inflates existing attention layers and adds positional embeddings to adapt to cubemap representation. Thanks to this simple change to multi-view diffusion model, CubeDiff achieves high-quality panorama generation while retaining the generalizability of the base diffusion model. The qualitative and quantitative comparisons with baselines demonstrate the CubeDiff achieves better quality panoramas, and also enabling fine-grained text control which was challenging in previous methods.", "questions": "The main questions on the method are in the Weaknesses section.\n\n* While fine-grained text control appears useful, I'm curious whether the conditioning can truly be applied on a \"per-face\" basis. Interactions between the pixels in different views may influence the content generated in each other. Could there be failure cases where changing the text condition for one view can lead to unintended changes in another view?\n\n* As described in Sec. 4.5, CubeDiff is trained with both dropped text and image conditions. In this case, is CubeDiff also capable of performing text-conditioned generation without an image input?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The proposed method is simple yet effective, according to the provided qualitative examples. Using cubemaps as a representation for panorama generation seems to be a valid choice, and building on a pretrained multi-view diffusion model is an adequate approach as it is proven to be effective at handling similar tasks requiring multi-view consistent generation.\n* The method shows notable training efficiency, as only 48k panoramas were used for training. It is convincing that it adapts well to out-of-distribution conditions, as shown by the outdoor generations in Fig. 1.\n* The fine-grainded text control of CubeDiff seems to be a clear advantage compared to previous approaches that use other types of panorama representatiions. From a user perspective, being able to provide certain controls for each of view would be more convenient with cubemaps, as it removes the need to consider the overlapping regions between neighboring views.", "weaknesses": "While the provided qualitative results of CubeDiff are convincing, several key aspects regarding the methodology and baseline comparisons are missing:\n\n* In Sec. 4.1, it is remains unclear how the \"inflated\" attention layer in CubeDiff is different from previous appoaches, such as the Correspondence-Aware Attention from MVDiffusion [1]. Specifically, it's unclear why this method reduces the risk of overfitting and results in greater generalization compared to approaches like MVDiffusion. This is particularly confusing since MVDream [2], from which CubeDiff adapts its layers, also incorporates an additional \"3D Self-Attention\" layer on top of a 2D diffusion model. Further clarification would be helpful to understand why one approach may lead to overfitting (as in MVDiffusion), while the other leads to generalization (as in CubeDiff).\n\n* Missing baseline: PanFusion [3] is an imporant baseline to consider for 360 panorama generation from text prompts. As it was published in CVPR 2024, it serves as a valid baseline for this work. To claim state-of-the-art performance on panorama generation, comparisons with PanFusion (e.g. in terms of image quality, generalization) would be required. Alternatively, it would be helpful to provide explanations on how CubeDiff differs from PanFusion and why a direct comparison may not be necessary.\n\n* I am curious about the details of the quantative evaluation procedure for the FID and KID metrics. Were the generated 360 panoramas directly used for measuring FID, or were projected perspective views used instead? Additionally, there exists a variant of FID, called FAED [4], specifically designed to assess the quality and realism of panoramas. Evaluating with such metrics could provide a more accurate reflection of the actual quality of the panorama generations.\n\n* I could not find ablation studies for the key components of the method, specifically regarding the effectiveness of synchronized group normalization and overlapped predictions. These are critical design choices for CubeDiff, yet it is difficult to assess their validity without quantative or qualitative comparisons showing the impact of these components.\n\nOverall, I agree that the cubemap representation could be an effective choice for panorama generation, and CubeDiff introduces a new approach of incorporating the cubemap representation. However, without the concerns about the key contribution addressed - particularly a clear explanation on why this approach should lead to enhanced generalization to out-of-distribution cases - it is difficult to assign a higher rating.\n\n[1] MVDiffusion: Enabling holistic multi-view image generation with correspondence-aware diffusion, Tang et al., NeurIPS 2023\n\n[2] MVDream: Multi-view Diffusion for 3D Generation, Shi et al., ICLR 2024\n\n[3] Taming Stable Diffusion for Text to 360 Panorama Image Generation, Zhang et al., CVPR 2024\n\n[4] Bips: Bi-modal indoor panorama synthesis via residual depth-aided adversarial learning, Oh et al., ECCV 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oWrkpNkgVf", "reviewer": "Reviewer_oYCw", "summary": "This paper proposes CubeDiff, a method that generates panoramas from text or single-image condition. The core of CubeDiff is to represent a panorama with cubemaps, 6 images representing the 6 faces of a cube that seamlessly capture the scene in an inside-out way. With this representation, CubeDiff adapts a pre-trained text-to-image diffusion model to generate the 6 cube-face images simultaneously. Similar to previous works like MVDream and Wonder3D, the proposed method extends the self-attention layer in the pre-trained denoising network to operate on multiple images for learning holistic multi-view consistency. Experiments on Laval Indoor and Sun360 datasets show that CubeDiff outperforms previous panorama generation methods (MVDiffusion, Diffusion360, etc,) across various generation settings (text-conditioned, image-condition, or both).", "questions": "Please refer to the Weaknesses section for the questions.\n\nIn general, the proposed CudeDiff is simple and obtains good experimental results. However, the paper does not provide necessary ablation studies for some detailed design choices, and it's also unclear if the comparisons against baselines are conducted in a fair setting. These two points are especially important given CudeDiff's limited technical novelty. I will consider updating the rating if these concerns can be properly addressed.\n\n\n-----\n\n**After rebuttal**: I increased the score to 6 as the authors provided a thorough study with different amounts of training panoramas, which helps demonstrate the method's effectiveness and generalization capability.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. CubeDiff's design choices are reasonable and backed up with a detailed discussion and comparison of different panorama representations. Extending the original self-attention over image patches of a single image to handle multiple views is a common strategy in the literature of multi-view diffusion models and is a natural choice here, considering the cube map representation. \n\n2. The paper provides a detailed discussion on different representations for the panorama in Sec.3. Together with the related works (Sec.2), choosing cube maps as the generation targets of the multi-view diffusion model is well-motivated. \n\n3. The experiments cover a rich set of conditioning settings, and CubeDiff demonstrates supreme performance both qualitatively and quantitatively. The anonymous website provides an interactive qualitative comparison, which helps better understand the quality of different approaches.", "weaknesses": "1. The experiment section of the paper lacks some critical ablation studies and is not complete: 1) Sec4.2 states the importance of the Synchronized GroupNorm, but there is no quantitative analysis for this design; 2) Sec4.4 claims the design of overlapping predictions between the nearby faces can reduce the generation artifacts, but again lack a corresponding ablation study.\n\n2. The overall technical contribution is relatively limited - there are no new designs from the modeling perspective. However, given the good empirical performance and the simplicity of the approach, this is not necessarily a vital weakness. Therefore, I hope the authors can address the first point above to complete the ablation studies for the core modeling components described in Sec.3 and Sec.4.\n\n3. It is unclear to me if the training data of the baselines (e.g., MVDiffusion and DIffusion360) is the same as the proposed CubeDiff. If they use different training data, would the comparisons be unfair? Ideally, the authors should re-train those diffusion-based baselines with the same training data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "70KYwLnxoO", "reviewer": "Reviewer_mPic", "summary": "The paper presents a new method for generating panoramic images. The key idea of the method is to utilize cubemap projections, instead of equirectangular projection. The method is relatively straightforward, but demonstrates good results.", "questions": "--", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- It is a simple to understand and implement method.\n- Paper is well written and easy to follow.\n- Results seem to be better than competitive methods.\n- New application of generating panorama based on multiple text-prompts.", "weaknesses": "- The claim of \"fully recycle pretrained text-to-image model, enabling generalization far beyond the limited training data\" is not well supported by results. The convincing results here would be, if the paper shows that the method can generate panoramas that are clearly outside of the panoramic dataset distribution. For example, stylistic panoramas, e.g. cartoons, oil painting or some fantasy concepts, such as \"Alice in wonderland\", etc.\n\n- Table. 1 is confusing. First of all there is no column $Ours_{text}$, while in Sec 4.5. It is said that models can operate in this mode. Second coloring in this Table is confusing. My suggestion would be to provide green/light green coloring within each modality: \"Text-only\", \"Image-only\", etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5QRcUMkFx9", "reviewer": "Reviewer_FHAm", "summary": "This paper presents a thoughtful approach to cubemap generation. The authors’ methodology is straightforward yet effective, addressing important challenges in cubemap generation, and achieves outstanding results compared to state-of-the-art.", "questions": "- Section 4.5 could be removed by briefly mentioning these details in the implementation section.\n- The implementation details could benefit from more specifics, such as the base model used.\n- Additional information on the VAE and UNet architectures would be helpful, particularly regarding the base models and the placement of GroupNorm in the VAE.\n- Why did the authors choose to concatenate positional encodings as additional channels? It’s more common to add positional embeddings directly to the latent representation along with timestep embeddings.\n- Is this work the first one to address panorama generation through cubemap diffusion?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This is a solid, simple yet effective work.  \n\nFirst, desipte the idea of using cubemap generation for panorama is not new,  the authors' solution through cubemap mv diffusion is simple but effective. This work offers an effective solution through overlapping predictions to address the wellknown issue of cubemap discontinuity.\n\nSecond, since each frame is generated individually (even with shared attention), tone shifts can occur across views. The authors address this by incorporating GroupNorm in VAE, which is a great application of existing techniques to solve this specific issue.\n\nLastly, their solution, while simple, performs significantly better than other alternatives, demonstrating that careful adjustments can yield meaningful improvements.", "weaknesses": "1. Lack of deeper analysis on how discontinuity is addressed: The proposed approach of overlapping predictions to handle discontinuity is interesting, but its effectiveness is not entirely clear. Overlapping predictions will bring new issues:  discontinuities within each view due to the six-view projection.  I recommend a more in-depth exploration:\n- Visualization of the overlapping predictions: How does each view appear before panorama merging?\n- Techniques for managing discontinuity within each view. Now each view might have severe boundary inside due to the discontinuity. Does the model handle this fine?  Is the achieved primarily due to VAE finetuning? How is shared attention helping here? Maybe attention visualization will benefit analysis.\n \n2. No ablation study on component effectiveness: An ablation study would strengthen the paper, particularly with results on:\n- The impact of GroupNorm\n- The impact of overlapping prediction", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)