SyncDreamer: Generating Multiview-consistent Images from a Single-view Image
Authors: Yuan Liu
Cheng Lin
Zijiao Zeng
Xiaoxiao Long
Lingjie Liu
Taku Komura
Wenping Wang
Venue: ICLR 2024 spotlight (2024)
Keywords: diffusion model; single-view reconstruction; 3D generation; generative models
Primary area: generative models
TL;DR: SyncDreamer is able to generate multiview-consistent images for single-view 3D reconstruction of arbitary objects.

Abstract: In this paper, we present a novel diffusion model called SyncDreamer that generates multiview-consistent images from a single-view image. Using pretrained large-scale 2D diffusion models, recent work Zero123 demonstrates the ability to generate plausible novel views from a single-view image of an object. However, maintaining consistency in geometry and colors for the generated images remains a challenge. To address this issue, we propose a synchronized multiview diffusion model that models the joint probability distribution of multiview images, enabling the generation of multiview-consistent images in a single reverse process. SyncDreamer synchronizes the intermediate states of all the generated images at every step of the reverse process through a 3D-aware feature attention mechanism that correlates the corresponding features across different views. Experiments show that SyncDreamer generates images with high consistency across different views, thus making it well-suited for various 3D generation tasks such as novel-view-synthesis, text-to-3D, and image-to-3D. Project page: https://liuyuan-pal.github.io/SyncDreamer/.

Published as a conference paper at ICLR 2024
SYNCDREAMER: GENERATING MULTIVIEW- CONSISTENT IMAGES FROM A SINGLE-VIEW IMAGE
Yuan Liu Cheng Lin
Zijiao Zeng²
Xiaoxiao Long"
Lingjie Liu
Taku Komura! Wenping Wang**
'hee university of Hong Kong
Teenent Games
Unniversity of Pennsylvania
*Texas A&M University
(yuanly, chlin, xxlong]@connectt. w....ku.... hk E.jiaoftencente
lingjie. liu@seas. upenn. edu
wenping@tamu edu
ABSTRACT
In this paper, present novel diffusion model called SyncDreamer that gen- erates aultiview-ccosiiten images from a single-view image. Using pretrained large-scale 2D) diffusion models, recent work 7ero123 (Liu et al. 2023b) demon- strates the ability to generate plausible novel views from a single- view image of an object. However, maintaining consistency in geometry and colors for the gener- ated images remains challenge. To address this issue, we propose synchronized multiview diffusion model that models the joint probability distribution of multi- view images, enabling the generation of multiview cenessccnnssteet images in single reverse process. SyncDreamer synchronizes the intermediate states of all the gen- erated images at every step of the reverse process through a 3D-aware feature attention mechanism that correlates the corresponding features across different views. Experiments show that SyncDreamer generates images with high consis- tency across different views, thus making it well-suited for various 3D generation tasks such as novel-view-synthesi. text-to-3D, and image-to-3D. Project page:
INTRODUCTION
Humans possess a remarkable ability to perceive 3D structures from a single image. When presented with image of object, humans easily imagine the other views of the object. Despite great progress Yao et al., 2018: Tewari et al. 20200] Wang et al. [021]] Mildenhall et al.] [2220] Xie et al.] 2022) brought by neural networks in computer vision graphics fields for extracting 3D information from images, generating multiview-consistern images from a single-view image of an object is still
a challenging problem due to the limited 3D information available in an image.
Recently, diffusion models (Rombach et al. 2022, Ho et al. 2020) have demonstrated huge success in 2D image generation, which unlocks new potential for 3D generation tasks, However, directly training a generalizable 3D diffusion model (Wang et al. 2023b Jun & Nichol 2023 Nichol et al., 2022 Müller et al., 2023) usually requires large amount of 3D data while existing 3D datasets are insufficient for capture the complexity of arbitrary 3D shapes. Therefore, recent methods (Poole al. 2023; Wang et al., 2023a d. Lin et al. 2023 Chen et al.) (023b) resort to distilling pretrained text-to-image diffusion models for creating 3D models from texts, which shows impressive results on this text-to-3D task. Some works (Tang et al. 2023a Melas-Kyriazi et al., 2023 Xu et al. 2022 Raj et al., 2023) extend such a distillation process to train neural radiance field Milleenaal al. 2020) (NeRF) for the image-to-3D task. In order to utilize pretrained text-to-image models, these methods have to perform textual inversion (Gal et al., 2022) find suitable text description of the input image. However, the distillation process along with the textual inversion usually takes a long time to generate single shape and requires tedious parameter tuning for satisfactory quality. Moreover, due to the abundance of specific details in an image, such as object category, appearance,
Equaal contribution. Corresponning Authors.

---

Published as a conference paper at ICLR 2024
Input Image
Generated Multiview-consitee Images
Mosh
Figure 1: SyncDreamer is able to generate multiview-consister images from a single-view input
image of arbitrary objects. The generated multiview images can be used for mesh reconstruction by reconstruction methods like NeuS (Wang al. 2021) without using SDS (Poole et al. 2023) loss.
and pose, it is challenging accurately represent an image using a single word embedding, which results in a decrease in the quality of 3D shapes reconstructed by the distillation method.
Instead of distillation, some recent works (Watson et al. 2022 Gu et al.
2023b Deng et al. 2023a
Zhou & Tulsiani 2023; Tseng et al. 2023 Yu al. 2023b] Chan et al.
2023 Tewari et al. 2023
Zhang al. 2023b Xiang et al. 2023) apply 2D diffusion models to directly generate multiview im- ages for the 3D reconstruction task. The key problem is how to maintain the multiview consistency when generating images of the same object. To improve the multiview consistency, these methods allow the diffusion model to condition on the input images (Zhou & Tulsiani 2023] Tseng et al.)
2023
Watson et al. 2022 Liu al., [023bb Yu al., 2023b), previously generated images (Tewari et al. 2023] Chan et al.] 2023) or renderings from a neural field (Gu et al. (0235).. Although some
impressive results are achieved for specific object categories from ShapeNet (Chang al.. 2015) or
Co3D (Reizenstein et al. ),, how to design a diffusion model to generate multiview-consitten
images for arbitrary objects still remains unsolved.
In this paper, we propose simple yet effective framework to generate multiview-consisten images for the single-view 3D reconstruction of arbitrary objects. The key idea is to extend the diffusion framework (Ho et al. 2020) to model the joint probability distribution of multiview images. We show that modeling the joint distribution can be achieved by introducing a synchronized multiview diffusion model. Specifically, for N target views to be generated, we construct N shared noise predictors respectively. The reverse diffusion process simultancously generates N images by N corresponding noise predictors, where information across different images is shared among noise predictors by attention layers on every denoising step. Thus, we name our framework SyncDreamer which synchronizes intermediate states of all noise predictors on every step in the reverse process.
SyncDreamer has the following characteristics that make it a competitive tool for lifting 2D single- view images to 3D. First, SyncDreamer retains strong generalization ability by initializing its weights from the pretrained Zero 123 (Liu et al.] 2023b) model which is finetuned from the Sta- ble Diffusion model (Rombach et al. 2022) on the Objaverse (Deitke et al.) (023b) dataset. Thus, SyncDreamer is able to reconstruct shapes from both photorealistic images and hand drawings as shown in Fig. T Second, SyneDremmer makes the single-view reconstruction easier than the distil- lation methods. Because the generated images are consistent in both geometry and appearance, we can simply run a vanilla NeRF (Mildenhall et al., 2020) or a vanilla NeuS (Wang et al., (02T) with- out using any special losses for reconstruction. Given the generated images, one can easily reckon the final reconstruction quality while it is hard for distillation methods know the output recon- struction quality beforehand. Third, SyncDreamer maintains creativity and diversity when inferring 3D information, which enables generating multiple reasonable objects from a given image as shown in Fig. In comparison, previous distillation methods only converge to one single shape.
We quantitatively compare SyncDreamer with baseline methods on the Google Scanned Ob- ject (Downs et al. 2022) dataset. The results show that, in comparison with baseline methods, SyncDreamer is able to generate more consistent images and reconstruct better shapes from input single-view images. We further demonstrate that SyneDremmer supports various styles of 2D input
2

---

Published as a conference paper at ICLR 2024
like cartoons, sketches, ink paintings, and oil paintings for generating consistent views and recon- structing 3D shapes, which verifies the effectiveness of SyneDreamer in lifting 2D images to 3D.
2
RELATED WORK
2.1 DIFFUSION MODELS
Diffusion models (Ho et al. 2020 Rombach et al. 2022 Croitoru al. 2023) have shown impres- sive results on 2D image generation. Concurrent work MVDiffusion TTngg et al. (023b) also adopts
the multiview diffusion formulation synthesize textures or panoramas with known geometry. We propose similar formulations in SyncDreamer but with unknown geometry. MultiDiffusion (Bar-Tal al. 2023) and SyncDiffusion (Lee al. 2023) correlate multiple diffusion models for different
regions of a 2D image. Many recent works (Nichol et al. 2022 Jun & Nichol
2023
Müller et al.
2023
Zhang et al.] 2023a Liu al., 2023d] Wang et al. 2023b Gupta et al. 2023
Cheng et al.,
2023
Karnewar et al. 2003bb Anciukevičius et al. 2023 Zeng et al. 2022 Erkoç et al.
[223]] Chen
et al.
2023a| Kim et al. 2023 Ntavelis et al., 2023] Gu et al. 2023a Karnewar et al.,
2023a) try
to repeat the success of diffusion models on the 3D generation task. However, the scarcity of 3D
data makes difficult to directly train diffusion models on 3D and the resulting generation quality is still much worse and less generalizable than the counterpart image generation models, though some works (Anciukevičius et al.) 2023; CCen et al. 2023a Karnewar et al. (023b) are trying to only use 2D images for training 3D diffusion models.
2.2
USING 2D DIFFUSION MODELS FOR 3D
Instead of directly learning a 3D diffusion model, many works resort to using high-quality 2D dif- fusion models (Rombach et al., 2022 Saharia et al. (022) for 3D tasks. Pioneer works DreamFu- sion (Poole al. 2003) and SJC (Wang et al. 2023a propose distill a 2D text-to-image generation
model to generate 3D shapes from texts. Follow-u up works (Chen et al. 2023b Wang et al. 2023d
Seo et al. 2023a Yu et al.] 2023a Lin al. 2023 Seo et | 2023b]
salicoglou et al. | 2023] (0023|Zhn Zhu
& Zhuang, 2023 Huang et al. 2023 Armandpour al.) 2023 Wu et 2023; Chen et al. 2023c)
improve such text-to-3D distillation methods in various aspects. Many works Tang et al. 2023a
Melas-Kyriaz et al.) 2023 Qian al. 2023 Xu et al.
2022 Raj al. 2023 Shen et al. 2023
also apply such a distillation pipeline in the single-view reconstruction task. Though some impres-
sive results are achieved, these methods usually require long time for textual inversion (Liu al.
2023a) and NeRF optimization and they do not guarantee to get satisfactory results.
Other works (Watson al.] 2022
Gu al. 2023b Deng al.
2023a|Zhou Zhou & Tulsiani 2023 Tssnn
2023a
cet al. 2023 Chan et al.] 2023 Yu et al. 2023b] Tewari et al.
2023
Yoo et al. 2023 Szymanowicz
al.
2023 Tang et al. 2023b Xiang et al. [023 Liu et al.
2023c
Lei al.
2022) directly apply
the 2D diffusion models to generate multiview images for 3D reconstruction. (Tssng el al.) 2023 Yu
al. 2023b) are conditioned on the input image by attention layers for novel-view synthesis in- door scenes. Our method also uses attention layers but is intended for object reconstruction. (Xiang al. 2023 Zhang et al., 2023b) resort to estimated depth maps to warp and inpaint for novel-view image generation, which strongly relies on the performance of the external single-view depth esti- mator. Two concurrent works (Chan et al.) [023] Tewari et al.] (003) generate new images in an autoregressive render-and-generate manner, which demonstrates good performances on specific ob- ject categories or scenes. In comparison, SyncDreamer is targeted to reconstruct arbitrary objects and generates all images in reverse process. The concurrent work Viewset Diffusion (Szy- manowicz et al., 2023) shares a similar idea to generate a set of images. The differences between SyncDre and Viewset Diffusion are that SyncDreamer does not require predicting radiance field like Viewset Diffusion but only uses attention to synchronize the states among views and Sync- Dreamer fixes the viewpoints of generated views for better convergence. Another concurrent work MVDream (Shi et al. 2023) also proposes multiview generation for the text-to-3D task while our work aims to reconstruct shapes from single-view images.
2.3 OTHER MINWLE-VIEW RECONSTRUCTION METHODS
Single-view reconstruction is a challenging ill ill-posed problem. Before the prosperity of generative models used in 3D reconstruction, there are many works (Tatarchenko al. 2019 Fu et al. 2021
Kato & Harada 2019 et al. 2020 Fahim et al. (022)) that reconstruct 3D shapes from single-vien view
images by regression Li et al. 2020) retrieval (Tatarchenko et al. 2019), which have difficulty in
generalizing to new categories. Recent NeRF-GAM methods (Niemeyer & Geiger 2021 Chan et al.]
2022
Gu et al. 2021 Schwarz et al. 2020
Gao al. 2022 Deng et al.
2023b learn generate
3

---

Published as a conference paper at ICLR 2024
NeRFs for specific categories like human cat faces. These NeRF-GANs achieve impressive results on single-view view image reconstruction but fail to generalize to arbitrary objects. Although some recent works also attempt to generalize NeRF-GAN to ImageNet (Skorokhodov et al. 2023), training NeRF-GANs for arbitrary objects still challenging.
2023 Sargent al.
METHOD
Given an imput view y of an object, our target is to generate multiview images the object. We assume that the object is located at the origin and is normalized inside cube of length 1. The target images are generated on fixed viewpoints looking at the object with azimuths evenly ranging from 0 to 360 and elevations of 30 To improve the multiview consistency of generated images, formulate this generation process as multiview diffusiio model. In the following, we begin with review of diffusion models (Sohl-Dickstein et 2015
Ho et al. 2020).
3.1 DIFFUSION
Diffusion models (Soh--Dickstein et al. 2015 Ho et al. 2020) aim to learn a probability model po(Xo) = po (Xu:r dxx where Xy the data and X1:T Xi....yy are latent variables. The joint distribution is characterized by Markov Chain (reverse process)
pe (x(oT) = p(xT) II (xt- (x:),
where p(xT) = (xT: 0, I) and po(x xx) = (x (,,,,)),, oi)). (x.)x.)) is trainable component while the variance of untrained time-dependent constants (Ho et al. 2020). The target is to learn the μl for the generation. To learn pe. a Markov chain called forward process is constructed
T
q(x1.T (2) = II (x-),
(2)
t=1
where (x, = (xxi v B,x,- B.I)) and 3, are all constants. DDPM (Ho et al. shows that by defining
2020
1
Be xx VI at co (xx,
no (x 1)
Va,
where a and ag are constants derived from Br and eg is a noise predictor, we can learn eg by Ex,xo.e [||e fie--ellf <a(vo vI a,e,t)l12)
where e is a random variable sampled from I..
3.2 MULTIVIEW DIFFUSION
Applying the vanilla DDPM model to generate novel-view images separately would lead to difficulty in maintaining multiview consistency across different views. To address this problem, we formulate the generation process
multiview diffusion model that correlates the generation of each view. Let
us denote the N images that we want generate on the predefined viewpoints as (x(²) ..... x (N) where suffix O means the time step O. We want to learn the joint distribution of all these views
X (N) ly). In the following discussion, all the probability functions are conditioned on the input view so we omit for simplicity.
po(x (I:N) ly) == pe (x1)
The forward process of the multiview diffusion model is a direct extension of the vanilla DDPM in Eq. where noises are added to every view independently by
((x(1xN) 12
(1:N)
(1:N),
II
{(x (1:N)
(1:N)
IIII
(n) (n) q(x)
x(-1),
(5)
where (xx (n) (x(")) A((( (n),
vI Br
x (n)
3.1.. Similarly, following Eq. the reverse process
is constructed as
T N
Po (x(x (1:N) = p(x
(1:N)
po (1:N) | (1-N)) p(x (1:M)
(x
pa
(x
(n)
(1:N))
(6)

---

Published as a conference paper at ICLR 2024
Synchronized Multiview Noise Predictor
Spatial volume (F,V,V.))
View frustum volume (F.D,H, J.D,H,M W)
Unproject
Interpolate
3D CNN
Target views (3.00
Teat attenioon
+dv (n)
Depth-wite attention
SyncDreamer Modules
Pretrained Zero 123
UNet
Input
Target view (n)
Target view 22-1
(n) Figure 2: The pipeline of synchronized multiview noise predictor to denoise the target view x)"
for one step. First, a spatial feature volume is constructed from all the noisy target views x; (I:N) Then, we construct a view frustum feature volume for xi [m) by interpolating the features of spatial feature volume. The input view y. current target view x(") and viewpoint difference (v(") fed into the backbone UNet initialized from Zerol23 (Liu et al. (2023b). On the intermediate feature maps of the UNet, new depth-wise attention layers are applied to extract features from the view frustum feature volume. Finally, the output of the UNet is used to denoise x((") to obtain x x
where po (x x
(1:00) = (V(x"") (n) (x(()))(((()))))) (1:0) .f),of1).
Note that the second equation in Eq.
holds because we assume diagonal variance matrix. However, the mean 1ig (n) of n-th view x
depends on the states of all the views x;
(1:N), Similar Eq. we define 14
(x) and the loss by
(1) (I:N)
6 (x! (1:N) .t)
Hg (x:
Va,
VI
(n) Eg (x
(1:N) ,
= B
1,20 (
(1: (I:N
(8)
where (1:N) is the standard Gaussian noise of size N x H x W added to all N views, (() is the noise added to the n-th view, and c(²) is the noise predictor on the n-th view.
(1:N) of the same object from Training procedure. In one training step, first obtain N images X-
the dataset. Then, sample timestep & and the noise (I:N) which is added all the images (1:N) to obtain x; (1:N), After that, we randomly select a view n and apply the corresponding noise Xg
(n) predictor Eg the selected view to predict the noise. Finally, the L2 distance between the sampled noise (n) and the predicted noise is computed the loss for the training.
Synchronized N-view noise predictor. The proposed multiview diffusion model can be regarded as N synchronized noise predictors {=(") I,... N). On each time step each noise predictor ((n) in charge of predicting noise on its corresponding view x; to get (m) Meanwhile, these noise predictors are synchronized because, on every denoising step, every noise predictor exchanges information with each other by correlating the states x; (I:N) of all the other views. In practical imple- mentation,
shared UNet for all N noise predictors and put the viewpoint difference between
the input view and the n-th target view (v(r), and the states x (1:N) of views as conditions to this shared noise predictor, i.c., (() (x(1 (1:N) t) ep (m) =x(x("); ;t, (v(n), x, (1:N)) The detailed computation of the viewpoint difference can be found in the supplementary material.
3.3 3D-AWARE PEATURE ATTENTION FOR DENOISING
In this section, we discuss how implement the synchronized noise predictor (x{"", (m) (n(") (I:N) y) by correlating the multiview features using 3D-aware attention
scheme. The overview is shown in Fig.
5

---

Published as a conference paper at ICLR 2024
Input View
Ours
Zero 123
RealFusion
Figure 3: Qualitative comparison with Zero 123 and RealFusion in multiview consistency.
Backbone UNet. Similar to previous works (Ho et al. 2020 Rombach et al. 2022). our noise predictor eo contains a UNet which takes a noisy image as input and then denoises the image. To ensure the generalization ability, we initialize the UNet from the pretrained weights of Zero123 (Liu al. 2023b) which is generalizable model with the ability to generate novel-view images from a given image of an object. 7ero 123 concatenates the input view with the noisy target view as the input to UNet. Then, to encode the viewpoint difference (v(a) in UNet, Z2ro123 reuses the text attention layers of Stable Diffusion to process the concatenation of Av(n) and the CLIP feature (Radford al. 202T) of the input image. We follow the same design Zero 123 and empirically freeze the UNet and the text attention layers when training SyncDreamer, Experiments to verify these choices are presented in Sec. 4.4
3D-aware feature attention. The remaining problem is how to correlate the states xt (I:N) of all the target views for the denoising of the current noisy target view x("). To enforce consistency among multiple generated views, it is desirable for the network to perceive the corresponding features in 3D space when generating the current image. To achieve this, we first construct 3D volume with V3 vertices and then project the vertices onto all the target views to obtain the features, The features from each target view are extracted by convolution layers and are concatenated to form a spatial feature volume. Next, 3D CNN is applied to the feature volume to capture and process spatial relationships. In order to denoise n-th target view, we construct a view frustum that is pixel-wise aligned with this view, whose features are obtained by interpolating the features from the spatial volume. Finally, on every intermediate feature map of the current view the UNet, we apply a new depth-wise attention layer to extract features from the pixel-wise aligned view-frustum feature volume along the depth dimension. The depth-wise attention is similar to the epipolar attention layers in Subail al. (2022): Zhou & Tulsiani (2023): Tseng et al. (2023): Yu et al. (2023b) as discussed in the supplementary material.
There two primary design considerations in this 3D-aware feature attention UNet. First, the spatial volume is constructed from all the target views and all the target views share the same spatial volume for denoising, which implies a global constraint that all target views are looking at the same object. Second, the added new attention layers only conduct attention along the depth dimension, which enforces local epipolar line constraint that the feature for specific location should be consistent with the corresponding features on the epipolar lines of other views.
Discussion.
EXPERIMENTS
4.1 EXPERIMENT PROTOCOL
Evaluation dataset. Following (Luu al. 2023b a). we adopt the Google Scanned Object (Downs) et al. (022) dataset as the evaluation dataset. To demonstrate the generalization ability to arbitrary objects, we randomly chose 30 objects ranging from daily objects to animals. For each object, render image with a size of 256 as the input view. We additionally evaluate some images collected from the Internet and the Wiki of Genshin Impact. More results are included in the supplementary materials.
Baselines. We adopt Zerol23 (Liu et al. 2023b), RealFusion Melas- Kyriazi et al. 2023),
Magic 123 (Qian et al. 2023), One-2-3-44 (Liu et al. 2023a), Point-E Nicholl et al. 2022) and Shap-

---

Published as a conference paper at ICLR 2024
Input View
Generated Instance A
Generated Instance B
Figure 4: Different plausible instances generated by SyncDremmer from the same input image.
(Jun& & Nichol 2023) baseline methods. Given input image of an object, Zero 23 (Liu al. 2023b) is able to generate novel-view images of the same object from different viewpoints. Zerol2 123 can also be incorporated with the SDS loss (Poole et al. 2023) for 3D reconstruction. We adopt the implementation of ThreeStudio (Guo et al.
2023) for reconstruction with 12ro which includes many optimization strategies to achieve better reconstruction quality than the original Zero123 im- plementation. RealFusion (Melas-Kyrazii et al. 2023) is based on Stable Diffusion (Rombach et al.) 2022) and the SDS loss for single-view reconstruction. Magic 123 (Qian et al., 2023) combines Zero 23 (Liu et al. 2023b) with RealFusion (Melas-Kyriazi et al. 2023) to further improve the re- construction quality. One-2-3-45 (Liu et al., (223a) directly regresses SDFs from the output images of Zerol23 and we use the official hugging face online demo (Face (023) to produce the results. Point-E (Nichol et al. 2022) and Shap-E (Jun & Nichol, 2023) are 3D generative models trained on large internal OpenAl 3D dataset, both of which are able to convert a single-view image into a point cloud or shape encoded in an MLP. For Point-E, we convert the generated point clouds to SDFs for shape reconstruction using the official models.
Metrics. We mainly focus on two tasks, novel view synthesis (NVS) and single view 3D recon- struction (SVR). On the NVS task, we adopt the commonly used metrics, i.e., PSNR, SSIM (Wang iet al., 2004) and LPIPS (Zhang et al., 2018). To further demonstrate the multiview consistency of the generated images, we also run the MVS algorithm COLMAP (Schönberger al., 2016) on the generated images and report the reconstructed point number. Because MVS algorithms rely on multiview consistency to find correspondences would lead to more reconstructed points. On the SVR task, report the commonly used Cham- fer Distances (CD) and Volume IoU between ground-truth shapes and reconstructed shapes. Since the shapes generated by Point-E (Nichol et al. (022) and Shap-E (Jun & Nichol, 2023) are defined in a different canonical coordinate system, we manually align the generated shapes of these two methods to the ground-truth shapes before computing these metrics. Considering randomness in the generation, we report the min, max, and average metrics on 8 objects in the supplementary material.
reconstruct 3D points, more consistent images
4.2 CONSISTENT NOVEL- SYNTHESTS
For this task, the quantitative results are shown in Table I and the qualitative results shown in Fig. 3 By applying a NeRF model distill the Stable Diffusion model (Poole let al., 2023, Rombach et al., (022). RealFu- sion Ceriaz-yrrazii al., 2023) shows strong multiview consistency producing more recon- structed points but unable to produce vi- sually plausible images as shown in Fig. 3 Zero123 (Liu et al.] 2023b) produces visually plausible images but the generated images are not multiview- www.viiww-oonssten.... Our method is able to generate images that not only are semantically consistent with the input image but also maintain multiview consistency in colors and geometry. Meanwhile, for the same input image, Our method can generate different plausible instances using different random seeds as shown in Fig. 4
Method
PSNR*
SSIM
LPIPS]
#Pointst
Realfusion
15.26
0.722
0.288
4010
Zerol23
18.93
0.779
0.166
95
Ours
20.05
0.798
0.146
1123
Table 1: The quantitative comparison in novel view synthesis. We report PSNR, SSIM, LPIPS and reconstructed point numbers by COLMAP on the GSO dataset.
7

---

Published as a conference paper at ICLR 2024
Input View
Ours
Zero
Magic
One-2-3-45
Point-E
Shap-E
Figure 5: Qualitative comparison of reconstruction from single view images with different methods.
4.3 SINGLE VIEW RECONSTRUCTION
We show the quantitative results in Table 2 and the qualitative comparison in Fig. Point-
Method
Chamfer Dist.
Volume IoU1
E (Nichol et al. 2022) and Shap-E Jun &
Nichol 2023) tend to produce incompleted meshes. Directly distilling Zero123 (Liu
Realfusion
0.0819
0.2741
Magic 123
0.0516
0.4528
al. allal, 2023bb generates shapes that are coarsely aligned with the input image, but the re- constructed surfaces are rough and not con- sistent with input images in detailed parts. Magic123 (Qian et al. 2023) produces much smoother meshes but heavily relies the timated depth values on the input view, which may lead incorrect results when the depth estimator is not robust. One-2-3-45 (Liu al. 2023a) reconstructs meshes from the snutiview-inconsistett outputs of Zerol23, 23, which is able to capture the general geometry but also loses details. comparison, our method achieves the best reconstruction quality with smooth sur- faces and detailed geometry.
One-2-3-45
0.0629
0.4086
Point-E
0,0426
0.2875
Shap-E
0.0436
0.3584
Zero123
0.0339
0.5035
Ours
0.0261
0.5421
Table 2: Quantitative comparison with baseline methods. We report Chamfer Distance and Vol- ume IoU on the GSO dataset.
4.4 DISCUSSIONS
In this section, we further conduct a of experiments to evaluate the effectiveness of our designs.
Generalization ability. To show the generalization ability, we evaluate SyncDremmer with 2D designs hand drawings like sketches, cartoons, and traditional Chinese ink paintings, which usually created manually by artists and exhibit differences in lighting effects and color space from real-world images. The results are shown in Fig. Despite the significant differences in lighting and shadow effects between these images and the real-world images, our algorithm is still able to perceive their reasonable 3D geometry and produce
consistent images.
Without 3D-aware feature attention. To show how the proposed 3D-aware feature attention im- proves multiview consistency, we discard the 3D-aware attention module in SyncDreamer and train this model on the same training set. This actually corresponds to finetuning a Zero 13 model with fixed viewpoints. As we can see Fig. 7 such a model still cannot produce images with strong consistency, which demonstrates the necessity of the 3D-aware attention module in generat- images.
ing multiview-conise
Initializing from Stable Diffusion instead of Zero123 123 (Liu et al. 2023b). An alternative strategy to initialize our model from Stable Diffusion (Rombach et al.) 2022). However, the results shown in Fig. 7 indicate that initializing from Stable Diffusion exhibits worse generalization ability than from Lerol23. Based on our observations, we find that the batch size plays an important role in enhancing the stability and efficacy of learning 3D priors from diverse dataset like Objaverse.
8

---

Published as a conference paper at ICLR 2024
Input design
multiview-consiitent Generated multiview consistent images
Mesh
Figure 6: Examples using SyncDreamer to generate 3D models from 2D designs..
Input
SynaDremmer
W/O 3D Attn
Init SD
Train UNet
Ablation studies to verify the designs of our method. "SyncDreamer" means full model. "W/O 3D Attn" means discarding the 3D-aware attention module in SyneDreamer, which ac- mallly results in Zero (Liu et al. (2033) finetuned fixed viewpoints the Objaverse (Deitke et af. 2023b) dataset. "Init SD" means initialize the Sync noise predictor from Stable Dif- fusion instead of 123.
Figure 7:
"Train UNet" means train the UNet instead of freezing
However, due to limited GPU memories, our batch size is 192 which smaller than the 1536 used by 1ero123. Finetuning Zerol23 enables SyncDreamer to utilize the 3D priors of Zero123.
Training UNet. During the training of SyneDremmer, another feasible solution is to not freeze the UNet and the related layers initialized from Zero123 but further finetune them together with the volume condition module. As shown in Fig. 77] the model without freezing these layers tends to predict the input object thin plate, especially when the input images 2D hand drawings. We speculate that this phenomenon is caused by overfitting, likely due to the numerous thin-plate objects within the Objaverse dataset and the fixed viewpoints employed during our training process.
Runtime. SyncDreamer uses about 40s to sample 64 images (4 instances) with 50 DDIM (Song al. 2020) sampling steps on 40G A100 GPU. Our runtime is slightly longer than Zerol23 because we need to construct the spatial feature volume on every step.
5 CONCLUSION
In this paper, we present SyncDreamer to generate multiview-consistent images from a single-view image. SyncDreamer adopts a synchronized multiview diffusion to model the joint probability dis- tribution of multiview images, which thus improves the multíview consistency. We design novel architecture that uses the Zero123 as the backbone and a new volume condition module to model cross-view dependency. Extensive experiments demonstrate that SyncDreamer not only efficiently generates multiview images with strong consistency, but also achieves improved reconstruction qual- ity compared to the baseline methods with excellent generalization various input styles.

---

Published as a conference paper at ICLR 2024
6 ACKNOWLEDGEMENT
This research is sponsored by the Innovation and Technology Commission of the HKSAR Govern- ment under the InnoHK initiative and Ref. T45-205/21-N of Hong Kong RGC. We sincerely thank Zhiyang Dou, Peng Wang, and Jiepeng Wang from AnySyn3D for discussions. This work is based on the computation resources from Tencent Taiji platform.
REFERENCES
Titas Anciukevičius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy Mitra, and Paul Guerrero. Renderdiffusion: Image diffusion for 3d reconstruction, inpainting and generation. In CVPR, 2023.
Mohammadreza Armandpour, Huangjie Zheng. Ali Sadeghian, Amir Sadeghian, and Mingyuan Zhou, Re-imagine the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond. arXiv preprint vrXiv:2304.04966 2023.
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. arXiv preprint arXiv:2302.081 13, 2023.
Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-ave aware 3d generative adversarial networks. In CVPR, 2022.
Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. ICCV, 2023.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.030122 2015.
Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single- stage diffusion nerf: unified approach to 3d generation and reconstruction. In ICCV, 2023a.
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. arXiv preprint arXiv:2303. 13733, 2023b.
Yiwen Chen, Chi Zhang, Xiaofeng Yang, Zhongang Cai, Gang Yu, Lei Yang. and Guosheng Lin. It3d: Improved text-to-3d generation with explicit view synthesis. arXiv preprint arXiv:2308. 11473, 2023c.
Yen-Chi Cheng. Hsin- Ying Lee, Sergey Tulyakov, Alexander G Schwing, and Liang- Yan Gui. Sd- fusion: Multimodal 3d shape completion, reconstruction, and generation. In CVPR, 2023.
Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusson models in vision: survey. T-PAMI, 2023.
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. arXiv preprint arXivv2307.05663 2023a.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: universe of anno- tated 3d objects. In CVPR, 2023b.
Congyue Deng, Chiyu Jiang, Charles Qi, Xinchen Yan, Yin Zhou, Leonidas Guibas. Dragomir Anguelov, et al. Nerdi: Single-view nerf synthesis with sagguage-guided diffusion as general image priors. In CVPR, 2023a.
Kangle Deng, Gengshan Yang, Deva Ramanan, and Jun- Yan Zhu. 3d-aware conditional image synthesis. In CVPR, 2023b.
10

---

Published as a conference paper at ICLR 2024
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B McHugh, and Vincent Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. In ICRA, 2022.
Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generat- ing implicit neural fields with weight-space diffusion. arXiv preprint arxtv:2303.17015,
Hugging Face. One-2-3-45.
One 2-3---3-- 45 2023.
George Fahim, Khalid Amin, and Sameh Zarif. Single-view 3d reconstruction: A survey of deep learning methods. Computers & Graphics, 94:164-190, 2021.
Kui Fu, Jiansheng Peng. Qiwen He, and Hanxiao Zhang. Single image 3d object reconstruction based on deep learning: A review. Multimedia Tools and Applications, 80:463-498, 2021.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01612 2022.
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. Get3d: A generative model of high quality 3d textured shapes learned from images. NeurlPS, 2022.
Jiatao Gu, Lingjie Liu, Peng Wang, and Christian Theobalt. Stylenerf: style-based 3d-aware generator for high-resolution image synthesis. In ICLR, 2021.
Jiatao Gu, Qingzhe Gao, Shuangfei Zhai, Baoquan Chen, Lingjie Liu, and Josh Susskind. Learning controllable 3d diffusion models from single-view images. arXiv preprint arXiv:2304.06700, 2023a.
Jiatao Gu, Alex Trevithick, Kai-En Lin, Joshua M Susskind, Christian Theobalt, Lingjie Liu, and Ravi Ramamoorthi. Nerfdiff: Single-image view synthesis with nerf-guided distillation from 3d-aware diffusion. In ICML, 2023b.
Yuan-Chen Guo. Instant neural surface reconstruction, 2022. http://giihbbboom//ennngggaaaatt
nsr-pl.
Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia- Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang, threestudio: A unified framework for 3d content generation. https://gitthee wwt.://oeeettetttieeieem pro--errreee///o threestudio 2023.
Anchit Gupta, Wenhan Xiong, Yixin Nic, lan Jones, and Barlas Oguz. 3dgen: Triplane latent diffusion for textured mesh generation. arXiv preprint arXiv:2303.05371, 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.
Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, and Lei Zhang. Dream- time: An improved optimization strategy for text-to-3d content creation. arXiv preprint arXiv:2306.1 /2422, 2023.
Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023.
Animesh Karnewar, Niloy J Mitra, Andrea Vedaldi, and David Novotny. Holofusion: Towards photo-realistic 3d generative modeling. In ICCV, 2023a.
Animesh Karnewar, Andrea Vedaldi, David Novotny, and Niloy J Mitra. Holodiffusion: Training 3d diffusion model using 2d images. In CVPR, 2023b.
Hiroharu Kato and Tatsuya Harada. Learning view priors for single-view 3d reconstruction. In CVPR, 2019.
11

---

Published as a conference paper at ICLR 2024
Seung Wook Kim, Bradley Brown, Kangxue Yin, Karsten Kreis, Katja Schwarz, Daiqing Li, Robin Rombach, Antonio Torralba, and Sanja Fidler. Neuralfield-Idm Scene generation with hierarchi- cal latent diffusion models. In CVPR, 2023.
Yuseung Lee, Kunho Kim, Hyunjin Kim, and Minhyuk Sung. Syncdiffusion: Coherent montage via synchronized joint diffusions. arXiv preprint arXiv:2336.05177, 2023.
Jiabao Lei, Jiapeng Tang, and Kui Jia. Generative scene synthesis via incremental view inpainting using rgbd diffusion models. In CVPR, 2022.
Xueting Li, Sifei Liu, Kihwan Kim, Shalini De Mello, Varun Jampani, Ming-Hsuan Yang, and Jan Kautz. Self-supervised single-view 3d reconstruction via semantic consistency. In ECCV, 2020.
Chen-Hsuan Lin, Jun Gao, Luming Tang. Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung- Yi Lin. Magic3d: High-resolution text-to-3d content creation. In CVPR, 2023.
Minghua Liu, Chao Xu, Haian Jin, Linghao Chen, Zexiang Xu, and Hao Su. One-2-3-45: Any single image to 3d mesh in 45 seconds without per-shape optimization. arXiv preprint arXiv:2306. 16928, 2023a.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-I-to-3; Zero-shot one image 3d object. ICCV, 2023b.
Xinhang Liu, Shiu-hong Kao, Jiaben Chen, Yu-Wing Tai, and Chi-Keung Tang. Deceptive-nerf: Enhancing nerf reconstruction using pseudo-observations from diffusion models. arXiv preprint arXiv:2305. 15171 2023c.
Zhen Liu, Yao Feng, Michael J Black, Derek Nowrouzezahrai, Liam Paull, and Weiyang Liu. Meshdiffusion: Score-based generative 3d mesh modeling. In ICLR, 2023d.
Xiaoxiao Long, Cheng Lin, Peng Wang, Taku Komura, and Wenping Wang. Sparseneus: Fast generalizable neural surface reconstruction from sparse views. In ECCV, 2022.
Luke Melas-Kyriazi Iro Laina, Christian Rupprecht, and Andrea Vedaldi. Realfusion: 360deg reconstruction of any object from a single image. In CVPR, 2023.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. In ECCV, 2020.
Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthias Nießner. Diffrf: Rendering-guiiee 3d radiance field diffusion. In CVPR, 2023.
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv preprint arXiv:2212.08751, 2022.
Michael Niemeyer and Andreas Geiger. Giraffe: Representing scenes as compositional generative neural feature fields. CVPR, 2021.
Evangelos Ntavelis, Aliaksandr Siarohin, Kyle Olszewski, Chaoyang Wang, Lue Van Gool, and Sergey Tulyakov. Autodecoding latent 3d diffusion models. arXiv preprint arXiv:2307.05445, 2023.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. In ICLR, 2023.
Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li. Hsin- Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, et al. Magic123: One image to high-quality object generation using both 2d and 3d diffusion priors. arXiv preprint arXiv:2306. / 17843, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML., 2021.
12

---

Published as a conference paper at ICLR 2024
Amit Raj. Srinivas Kaza, Ben Poole, Michael Niemeyer, Nataniel Ruiz, Ben Mildenhall, Shiran Zada, Kfir Aberman, Michael Rubinstein, Jonathan Barron, et al. Dreambooth3d: Subject-driven text-to-3d generation. arXiv preprint arXiv:2303.13550, 2023.
Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d cate- gory reconstruction. In CVPR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022.
Kyle Sargent, Jing Yu Koh, Han Zhang, Huiwen Chang, Charles Herrmann, Pratul Srinivasan, Jiajun Wu, and Deqing Sun. Vq3d: Learning a 3d-aware generative model on imagenet. arXiv preprint arXiv:2302.0668. 06833, 2023.
Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi-view stereo. In ECCV, 2016.
Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger. Graf: Generative radiance fields for 3d-aware image synthesis. NeurIPS, 2020.
Hoigi Sco, Hayeon Kim, Gwanghyun Kim, and Se Young Chun. Ditto-nerf: Irffusion-based itera- tive to omni-directional 3d model. arXiv preprint arXiv:2304.02822 02827, 2023a.
Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Jachoon Ko, Hyeonsu Kim, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d-consistency for robust text-to-3d generation. arXiv preprint arXiv:2303.009.37 2023b.
Qiuhong Shen, Xingyi Yang, and Xinchao Wang. Anything 3d: Towards single-view anything reconstruction in the wild. arXiv preprint arXiv:2334.10261, 2023.
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023.
Ivan Skorokhodov, Aliaksandr Siarohin, Yinghao Xu, Jian Ren, Hsin- Hsin-Ying Lee, Peter Wonka, and Sergey Tulyakov. 3d generation imagenet. arXiv preprint arXiv:2303.01416, 2023.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500 2020.
Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Generalizable patch-based neural rendering. In ECCV, 2022.
Stanislaw Szymanowicz, Christian Rupprecht, and Andrea Vedaldi. Viewset diffusion:(O-) image- conditioned 3d generative models from 2d data. arXiv preprint arXiv:2306.0788 2023.
Junshu Tang, Tengfei Wang, Bo Zhang, Ting Zhang, Ran Yi, Lizhuaan Ma, and Dong Chen. Make- it-3d: High-fidelity 3d creation from a single image with diffusion prior. In ICCV, 2023a.
Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. Mvdiffusion: En- abling holistic multi-view image generation with correspondence-awwes diffusion. arXiv preprint arXiv:2307.01097, 2023b.
Maxim Tatarchenko, Stephan Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks lear?? In CVPR, 2019.
13

---

Published as a conference paper at ICLR 2024
Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Be Brrullaa, Tomas Simon, Jason Saragih, Matthias Nießner, et al. State of the art on neural rendering. In Computer Graphics Forum, 2020.
Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Joshua B Tenenbaum, Frédo Durand, William T Freeman, and Vincent Sitzmann. Diffusion with forward models: Solving stochastic inverse problems without direct supervision. arXiv preprint arXiv:2306.11719, 2023.
Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari. Textmesh: Generation of realistic 3d meshes from text prompts. arXiv preprint 3tii::200.122399 2023.
Hung-Yo Yu Tseng, Qinbo Li, Changil Kim, Suhib Alsisan, Jia-Bin Huang, and Johannes Kopt. Con- sistent view synthesis with pose-guided diffusion models. In CVPR, 2023.
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In CVPR, 2023a.
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. NeurIPS, 2021.
Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, et al. Rodin: A generative model for sculpting 3d digital avatars using diffusion. In CVPR, 2023b.
Yiming Wang, Qin Han, Mare Habermann, Kostas Danillidis, Christian Theobalt, and Lingjie Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In ICCV, 2023c.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. arXiv preprint arXiv:2305. 16213, 2023d.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. 2004.
Daniel Watson, William Chan, Ricardo Martin Bruaala, Jonathan Ho, Andrea Tagliasacchi, and Mo- hammad Norouzi. Novel view synthesis with diffusion models. arXiv preprint arXiv:2210.04628, 2022.
Jinbo Wu, Xiaobo Gao, Xing Liu, Zhengyang Shen, Chen Zhao, Haocheng Feng, Jingtuo Liu, and Errui Ding. Hd-fusion: Detailed text-to-3d generation leveraging multiple noise estimation. arXiv preprint 1XX1:23307.1618. 2023.
Tong Wu, Jiaqi Wang, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, and Dahua Lin. Voxurf: Voxel-based efficient and accurate neural surface reconstruction. arXiv preprint arXiv:2208. 12697, 2022.
Jianfeng Xiang, Jiaolong Yang, Binbin Huang, and Xin Tong. 3d-aware image generation using 2d diffusion models. arXiv preprint arXiv:2303.17905 2023.
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual comput- ing and beyond. In Computer Graphics Forum, 2022.
Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, and Zhangyang Wang. Neurallift-360: Lifting an in-the-wild 2d photo to 3d object with 360 views. arXiv e-prints, pp. arXiv 2211, 2022.
Yao Yao, Zixin Luo, Shiwei Li. Tian Fang. and Long Quan. Mvsnet: Depth inference for unstruc- tured multi-view stereo. In ECCV, 2018.
Paul Yoo, Jiaxian Guo, Yutaka Matsuo, and Shixiang Shane Gu. Dreamsparse: Escaping from plato's cave with 2d frozen diffusion model given sparse views. CoRR, 2023.
14

---

Published as a conference paper at ICLR 2024
Chaohui Yu, Qiang Zhou, Jingliang Li, Zhe Zhang, Zhibin Wang, and Fan Wang. Points-to-3d Bridging the gap between sparse points and shape-controllabe text-to-3d generation. arXiv preprint arXiv:2307.13908 2023a.
Jason Yu, Fereshteh Forghani, Konstantinos G. Derpanis, and Marcus A. Brubaker. Long-term photometric consistent novel view synthesis with diffusion models. In 2023b
Xiaohui Zeng. Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, and Karsten Kreis. Lion: Latent point diffusion models for 3d shape generation. In NewrIPS, 2022.
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. In SIGGRAPH, 2023a.
Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. Text2nerf: Text-driven 3d scene generation with neural radiance fields. arXiv preprint arXiv:2305. 2023b.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, 2018.
Zhizhuo Zhou and Shubham Tulsiani. Sparsefusion: Distilling view-conditioned diffusion for 3d reconstruction. In CVPR, 2023.
Joseph Zhu and Peiye Zhuang. Hifa: High-fidelity text-to-3d with advanced diffusion guidance. arXiv preprint arXiv:2305.18766 2023.
A APPENDIX
A.1 IMPLEMENTATION DETAILS
We train SyneDreamer on the Objaverse (Deitke et al. 2023b) dataset which contains about 800k objects. We set the viewpoint number = 16. The spatial volume has the size 32 and the fie--fusttmm volume has the size of 32 32 48. We sample 48 depth planes for the view-frustum volume because the view may look into the volume from the diagonal direction. We chose these sizes because the latent feature map size of an image of 256 x 256 in the Stable Diffusion Rombach eet al. (2022) 32 x 32. The elevation of the target views set to 30 and the azimuth evenly distributes [0% 360%). Besides these target views, we also render 16 random views as input views on each object for training, which have the same azimuths but random elevations. We always assume that the azimuth of both the input view and the first target view is 0º. We train the SyneDreamer for 80k steps (~4 days) with 8 40G A100 GPUs using total batch size of 192. The learning rate is annealed from 5e-4 to 1e-5. The viewpoint difference is computed from the difference between the target view and the input view on their elevations and azimuths. Since we need an elevation of the input view to compute the viewpoint difference Ay(a), we use the rendering elevation in training while we roughly estimate an elevation angle as input in inference. Note that baseline methods RealFusion (Melas-Kyrial et al.) (023). Zerol23 (Liu et al. 2023b), and Magic123 (Qian et al.] 2 all require an estimated elevation angle as input in test time. It is also possible to adopt the elevation estimator in Liu et al. (2023a) to estimate the elevation angle of the input image. To obtain surface meshes, we predict the foreground masks of the generated images using CarveKi Then, train the vanilla NeuS (Wang et al. 1) for 2k steps to reconstruct the shape, which costs about 10 mins. On each step, we sample 4096 rays and sample 128 points on each ray for training. Both the mask loss and the rendering loss applied in training NeuS. The reconstruction process be further sped up by faster reconstruction methods (Wang et al., 2023c] Guo [022] Wu et al. 2022) or generalizable SDF predictors (Long et
2022
Liu al. 2023al with priors.
A.2 TEXT-TO-IMAGE-TO-33
By incorporating text2image models like Stable Diffusion (Rombach et al.) 2022) or Imagen (Saharia
ct al. 2022). SyncDreamer enables generating 3D models from text. Examples are shown in Fig. g Compared with existing text-to-3D distillation, our method gives more flexibility because users
15

---

Published as a conference paper at ICLR 2024
wooden statue
toy
muscles
lovely small
alien running
rocking horse
chair
Input text
Text to image
Generated images
Mesh
Figure 8: Examples of using SyncDreamer to generate 3D models from texts.
Input view
Good instance
Failure instance
Figure 9: Limitation on the generation quality. The generated instances from SyncDreamer are not always desirable. Sometimes, low-quality failure instances may be generated due the stochas- tic process of diffusion models.
can generate multiple images with their text2image models and select the desirable one to feed to SyncDreamer for 3D reconstruction.
A.3 LIMITATIONS AND FUTURE WORKS
Though SyncDreamer shows promising performances in generating multiview-consiseee images for 3D reconstruction, there are still limitations that the current framework does fully address. First, the generated images of SyncDreamer have fixed viewpoints, which limits some of its application scope when requiring images of other viewpoints. possible alternative is use the trained NeuS to render novel-view images, which achieves reasonable but little bit blurry results as shown in Fig, 13, Second, the generated images are not always plausible and we may need to generate mul- tiple instances with different seeds and select desirable instance for 3D reconstruction
shown in Fig. [ Especially, we notice that the generation quality is sensitive to the foreground object size in the image. The reason is that changing the foreground object size corresponds adjusting the perspective patterns of the input camera and affects how the model perceives the geometry the ob- ject. The training images of Syncereamer have predefined intrinsic matrix and all captured
16

---

Published as a conference paper at ICLR 2024
Input view
With depth-wise attention
Without depth-wise attention
Figure 10: Evaluation of depth-wise attention layers. The model without depth-wise attention layers has degenerated quality while the model with depth-wise attention produces better results.
predefined distance to the constructed volume, which makes the model adapt fixed perspective pattern. To further increase the quality, we may need to use a larger object dataset like Objaverse- XL (Deitke al. 2023a) and manually clean the dataset exclude some uncommon shapes like complex scene representation, textureless 3D models, and point clouds. Third, the current imple- mentation of SyneDreamer assumes perspective image input but many 2D designs drawn with orthogonal projections, which would lead to unnatural distortion the reconstructed geom- etry. Applying orthogonal projection in the volume construction SyneDremmer would alleviate this problem. Mcanwhile, we notice that generated textures are sometimes less detailed than the Zero123. The reason that the multiview generation more challenging, which not only needs to be consistent with the input image but also needs to be consistent with all other generated views. Thus, model may tend to generate large texture blocks with less detail, since it could more easily maintain multiview consistency.
A.4 DISCUSSION ON DEPTH- WISE ATTENTION LAYERS
We find that the depth-wise attention layers are important for generating high-quality multiview- consistent images. To show that, we design an alternative model that directly treats the view-frustun feature volume x W x convolutional layers to extract features on it and then add them to the intermediate feature maps of UNet. We find that the model without depth- wise attention layers produces degenerated images with undesirable shape distortions as shown in Fig. 10
F 2D feature map x W (D x F). Then, we apply 2D
A.5 GENERATING IMAGE ON OTHER VIEWPOINTS
To show the ability of SyncDreamer generate images of different viewpoints, train new SyncDreamer model but with different 16 viewpoints. The new viewpoints all have elevations of 0 and azimuth evenly distributed in the range [0 360%]. The generated images of this new model are shown in Fig.
A.6 ITERATIVE GENERATION
It is also possible re-generate novel view images from of the generated images of Sync- Dreamer. Two examples are shown in Fig. 12 In the figure, row shows the generated images of Synamrem and row 2 shows the re-generated images of SyncDreamer using one of first-row images as its input image. Though the regenerated images are still plausible, they reasonably differ from the original input view
A.7 NOVEL-VIEW RENDERINGS OF NEUS
Though SyncDreamer can only generate images on fixed viewpoints, we can render novel-view images from arbitrary viewpoints using the NeuS model trained the output of SyneDreamer,
17

---

Published as a conference paper at ICLR 2024
Input
Generated views
Input
Generated views
Figure 11: Generated images with 0 elevations by SyncDreamer.
Input image
Generated images
Figure 12: We use SyneDreamer to regenerate novel-view images from the outputs of SyncDreamer
For each object, Row is the original generation results while Row 2 one output image of Row 1 as input to regenerate novel-view images.
18

---

Published as a conference paper at ICLR 2024
Generation
Rendering
Generation
Rendering
Generation
Rendering
Figure 13: Odd rows show the image generated by SyncDreamer while even rows show the images rendered from arbitrary viewpoints using the NeuS model trained on the generated images.
shown in Fig. 13 However, since only 16 images are generated train the NeuS model, the renderings from NeuS are more blurry than the generated images of SyncDreamer.
A.8 FEWER GENERATED VIEWS FOR NEUS TRAINING
The NeuS reconstruction process can be accomplished with fewer views. as demonstrated in Fig. 14 Decreasing the number of views from 16 to 8 does have significant impact on the overall reconstruction quality. However, utilizing only 4 views results in steep decline in both surface reconstruction and novel-view-synthen viwt-synteesssss quality. Consequently, is possible to train a more efficient version of SyncDremmer to generate 8 views for the NeuS reconstruction without compromising the quality too much.
A.9 FASTER RECONSTRUCTION WITH BASH-GRID-BASEE NEUS
possible to use hash-grid-bane based NeuS to improve the reconstruction efficiency. Some qualita- tive reconstruction results are shown in Fig. 15 The mash-grid-based method takes about minutes which is less than half the time of the vanilla MLP-based NeuS (10min). Since hash-grid bosh--rrd-bbsedd SDF usually produces on the normals computed from the has- has-grid-based SDF.
noisy surfaces than MLP-based SDF, add additional smoothness losses
A.10 METRICS USING DIFFERENT GENERATION SEEDS
Due to the randomness the generation process, the computed metrics may differ we use different seeds for generation. To show this, randomly sample instances from the same input image of objects from the GSO dataset and compute the corresponding PSNR, SSIM, LPIPS, Chamfer Distance, and Volume IOU reported in Table
19

---

Published as a conference paper at ICLR 2024
views
views
16 views
Figure 14: Results of using fewer generated views SyneDremmer for NeuS reconstruction. Odd columns show the renderings of NeuS while even columns show the reconstructed surfaces of NeuS.
Input
MLP
Hash-grid
Input
MLP
Hash-grid
Figure 15: Surface reconstruction results using "MLP" NeuS and "Hash-grid" NeuS.
20

---

Published as a conference paper at ICLR 2024
Mario
S.Busl
S.Bus2
Shoe
S.Cups
Sofa
Hat
Turtle
Min
18.25
20.52
16.39
21.38
23.43
18.97
20.87
15.83
PSNRT
Max
18.74
20.70
16.67
21.70
24.48
19.53
21.08
16.33
Avg.
18.48
20.63
16.48
21.48
23.99
19.26
20.96
16.03
Mín
0.811
0.851
0.687
0.862
0.899
0.809
0.797
0.749
SSIM
Max
0.816
0.855
0.690
0.866
0.913
0.816
0.801
0.754
Avg.
0.813
0.853
0.688
0.864
0.906
0.812
0.799
0.751
Min 0.129
0.104
0.222
0.081
0.055
0.154
0.134
0.209
LPIPS
Max
0.135
0.108
0.229
0.084
0.087
0.157
0.136
0,223
Avg-
0.133
0.105
0.226
0.082
0.071
0.156
0.135
0.218
Min
0.0139
0.0076
0.0217
0.0167
0.0079
0.0237
0.0464
0.0225
CD1
Max
0.0194
0.0100
0.0236
0.0184
0.0138
0.0449
0.0511
0.0377
Avg.
0.0167
0.0087
0.0227
0.0172
0.0110
0.0312
0.0490
0.0301
Min 0.6604
0.8284
0.5247
0.4383
0.5966
0.3905
0.2614
0.6313
Vol. IOU1
Max 0.7336
0.8335
0.5731
0.4826
0.6873
0.5205
0.2919
0.7471
Avg. 0.6889
0.8309
0.5578
0,4575
0.6427
0.4729
0,2705
0.6864
Table 3: Statistical analysis of the generation randomness of SyncDreamer. We generate instances using SyncDremmer and compute the PSNR. SSIM, LPIPS, Chamfer Distance (CD), and Volume IOU (Vol. IOU). We list the minimum, maximum, and average values of these metrics.
A.11 DISCUSSION ON OTHER ATTENTION MECHANISM
There are several attention mechanisms similar to our depth-wise attention layers. MVDiffu- sion Tang et (2023b) utilizes correspondence-avee try. In SyncDreamer, the geometry is unknown so we cannot build such one-to-one correspondence for attention. An alternative way the epipolar attention layer in Suhail al. (2022); Zhou & Tul- siani (2023): Tseng et al. (2023) : Yu et al. (2023b) which constructs an epipolar line on every image and applies attention along the epipolar line. Epipolar line attention constructs epipolar lines on every image and applies attention along epipolar lines. Our depth-wise attention is very similar to epipolar line attention. If we project 3D point in the view frustum onto a neighboring view, we get a 2D sample point on the epipolar line. We notice that in epipolar line attention, we still need to main- tain a new tensor of size W x containing the epipolar features. This would cost as large GPU memory
attention layer based the known geome-
based attention. A concurrent work MVDream (Shi al. 2023) applies attention layers on all feature maps from multiview images, which also achieves promising results. However, applying such attention layer all the feature maps of 16 images in setting costs unaffordable GPU memory in training. Finding a suitable network design for multiview-consiiteer image generation would still be an interesting and challenging problem for future work.
A.12 DIAGRAM ON MULTIVIEW DIFFUSION
We provide a diagram in Fig. 16 visualize the derivation of the proposed multiview diffusion in Sec. 3.2 in the main paper.
21

---

Published as a conference paper at ICLR 2024
Forward
t-
D=0
n=]
n=2
n=3
11:4
n=7
n-0
n=]
pul (1W)
Reverse process
n=0
]=
2
n=3
n
n-6
n=0
n=1
n-2
n-3
n-4
n
n=6
n-7
Figure 16: An intuitive diagram illustrating the derivation of the forward and reverse processes of proposed multiview diffusion model. Better visualization quality with zooming in.
22


Reviews:
[{"review_id": "tQ5WDqc1ur", "reviewer": "Reviewer_Q4FP", "summary": "The authors proposed a synchronized multi-view image diffusion model to generate consistent multi-view images. Compared with the previous Zero123, the authors target at generating images at fixed camera views, and the denoising step of each view is conditioned on the noisy images of all views. A feature volume is introduced as an intermediate structure for aggregating information from multi-view noisy images. Experiments show that the generated multi-view images are significantly more consistent than previous methods.", "questions": "See above. Generally, I think the paper is of very good quality. I will raise my rating if my concerns can be addressed in the rebuttal.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- Consistent multi-view image generation with diffusion model has been a useful and heated research topic in the past months. The authors proposed a practical solution for the task. \n\n- Synchronized diffusion conditioned on multi-view information is new to me and is reasonable.\n\n- The visualization of the generated results is good, and experiments show that the multi-view consistency is much better than the Zero123 baseline.", "weaknesses": "I am unclear about several components in the proposed pipeline:\n\n- Multi-view image condition: is it possible to use the denoised multi-view images rather than the noisy images from the last step as a condition to the diffusion model? \n\n- 3D CNN: how is the 3D CNN trained? Is it pre-trained or trained with the attention module together in the finetuning?\n\n- 3D CNN: how will the 3D CNN affect the feature aggregation from multi-view and can we remove this part? It reminds me of the cost volume regularization in stereo/MVS networks. If the 3D CNN is used for volume denoising, again, how about directly using the denoised multi-view images from the last step as a condition of the diffusion model?\n\n- View frustum volume: will it be too heavy to pass a view frustum volume to the UNet as the multi-view condition? Also, how is W in the view frustum volume determined?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R5E4gUbrTs", "reviewer": "Reviewer_bdx1", "summary": "This paper proposes SyncDreamer, which can generate a set of consistent multiview images given a single image as the condition, and the generated images can be easily reconstructed to a mesh with a NeRF-based reconstruction method (e.g., NeuS). The proposed method extends Zero123 with several additional modules to achieve multiview consistency: i) a 3D feature volume to collect global features from all viewpoints and a corresponding 3D CNN to further process the feature volume; ii) a depth-wise attention module to inject the global features from the 3D volume into the UNet of each view. SyncDreamer produces decent multiview images and can handle input images with diverse styles. Experiments show that SyncDreamer outperforms previous works in terms of both image and mesh quality.", "questions": "The main questions are listed in the weaknesses section, and there are some minor questions: \n\n- The details of the mesh reconstruction process are missing. The authors mentioned in the caption of Fig.1 that NeuS is used to reconstruct the mesh from the generated images, but there are no relevant implementation details. How long does the NeuS training take, and does it always produce reasonable reconstruction results (as the generation results are random, sometimes wrong or inconsistent images can be generated)?\n\n- Some recent works, like [MVDiffusion](https://arxiv.org/abs/2307.01097) and  [MVDream](https://arxiv.org/abs/2308.16512), can also generate consistent multiview images while their methods directly exchange multiview feature inside UNet blocks. It would be beneficial if the authors could discuss the pros and cons of the proposed method and those relevant ones.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper presents a novel method that extends Zero123 to generate consistent multiview images. The designs of the 3D global volume and depth-wise attention are reasonable.  \n\n\n- The writing is clear and easy to follow, with nice figures to illustrate the core ideas of the method. \n\n\n- The experimental results show that SyncDreamer can generate high-quality multiview images and meshes given a single condition image, consistently outperforming the baseline approaches both quantitatively and qualitatively. \n\n\n- The supplementary material contains a local webpage showing many promising qualitative results, which demonstrates the capacity of the proposed method.", "weaknesses": "- The model seems to be limited to a fixed set of viewpoints based on the concatenation design for constructing the 3D feature volumes. In contrast, the initial diffusion model, Zero123, can generate novel-view images at any specified viewpoint. The experiments in the supplementary material also show that a new model needs to be trained to generate 16 images with a different elevation (i.e., 0$^\\circ$). Although the model successfully generates diverse 3D objects in the current setting (16 images at an elevation of 30$^\\circ$), this limitation still narrows the scope of application of the proposed method. Further analyses/discussions will be helpful: i) what will happen if the same model generates *another 16 images* with different elevations using one of the generated images as the condition? Will the model fail terribly, or some reasonable results can still be produced? ii) The authors might argue that a trained NeRF or NeuS could synthesize other viewpoints, but the quality of the rendered images may be lower than that of the ones generated by SyncDreamer, and some related results would be appreciated. \n\n\n\n- For quantitative evaluations, the settings are unclear. The generation process has randomness, so how are the generation results used in quantitative evaluations determined? Is it running multiple rounds and then taking the average metrics, or taking the best numbers out of multiple rounds? The concrete settings must be clarified, and it would be better to report the mean and standard deviation if multiple rounds of generation are executed for evaluation.\n\n\n\n- The ablation studies in Fig.7 seem to suggest that the proposed model components (3D feature volume + depth-wise attention) cannot learn proper 3D shape priors, and the method has to rely on a frozen Zero123 -- when the UNet weights are fine-tuned together with the new modules, the geometry of the generated images becomes broken. If this speculation is true, the proposed modules can only learn local consistency and must be plugged into a base model with 3D priors (e.g., Zero123) for generating plausible multiview images. This is not necessarily a flaw, but it indeed narrows down the scope of the paper, as the proposed method is more like an extension of Zero123 rather than a general solution for multiview image generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mul3nJImMu", "reviewer": "Reviewer_mZRr", "summary": "The paper presents a method for single-view novel view synthesis. It initializes from a pre-trained model Zero123, a finetuned StableDiffusion model on Objaverse for (relatively) consistent NVS. At each diffusion step, the UNet is conditioned on all target frames synchronously. This is achieved by constructing a feature volume from all the noisy intermediate target images at time step t, then projecting to the camera frustum of the desired target pose. Afterward, depth-wise attention is used to inject these features into the UNet to denoise a target view at time step t-1.", "questions": "None", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "* The task of single-view novel view synthesis is extremely challenging and well-motivated.\n* Evaluations were done with reasonable metrics and against SOTA methods, and a large margin of improvements can be observed. Cool renderings and additional applications are shown.\n* Through ablation studies were done to justify its design choices.\n* The method is novel and reasonable, the consistency achieved is significantly beyond prior methods.", "weaknesses": "* The biggest weakness of this paper is its writing. I feel the notations used in section 3.2 are very confusing. Following the standard notations in score-based generative model literature, for quite a while I thought the paper somehow extended the UNet so that it is able to take multiple noisy images as input, but in fact if I’m not mistaken, it still just takes a single noisy image as input, and the conditioning from the other noisy target views are just the injected features.\n* I failed to understand where the features used to create the volume came from. Is it extracted with another network? Please be more specific.\n* Please mention that the computation of viewpoint difference is discussed in the supplementary.\n* A small introduction to zero123 could be helpful, as many of the design choices are very specific to zero123 and require some prior knowledge of how zero123 works.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ex61sLnQHi", "reviewer": "Reviewer_hhwq", "summary": "This paper introduces a novel approach that suggests generating multiple views (16 views in the model) of images simultaneously using a diffusion process. The model is built upon the foundation of zero123. It puts forward a volume-based attention mechanism to facilitate communication among the various views. In terms of performance, it stands out with exceptional results, particularly excelling in multiview alignment. Remarkably, this paper marks a significant milestone as the first diffusion based approach (to the best of my knowledge) among all single-view 3D generation models that doesn't require SDS for 3D model creation.", "questions": "1. The paper chooses to use 16 views for its volume-based approach. It would be helpful to understand the rationale behind this specific choice. Is 16 views the minimal number required for effective multiview image reconstruction, or was this number selected for a different reason? Given the extra computational burden associated with more views, why not start with a smaller number, like 8 views, to assess the trade-off between computational efficiency and performance?\n\n2. I tried the model and found its sensitivity to the size of objects when using the code and released models. Is this sensitivity a result of the volume-based approach or if there are specific reasons behind this behavior. Understanding the limitations related to object size could provide insights into potential improvements or workarounds. Please see below for the attached images: if I scale the 'monkey' a little bit more, the generation could be very different. \n\n3. Epipolar-based attention mechanisms, as demonstrated in works like Generalizable Patch-Based Neural Rendering (ECCV 2022), are technically equivalent to volume-based attention. Did the authors consider or investigate this direction, and do they have any comparative results or insights regarding the choice of attention mechanism in SyncDreamer?\n\n---\n`1.jpg` and `2.jpg` show very different results when the input object (trim_size) is made slightly larger.\nPlease run the following python code to see the images.\n```\nimport base64\nfrom io import BytesIO\nim = Image.open(BytesIO(base64.b64decode(b'/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAGAAIADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD3+iiigAooooAKKK5nxr4xtfB+lLcSr5t1NlbaHn94wxnJ7DBoDc6as3UfEOj6TEJL/U7S3QkqDJKBkjqMV8/X3jvxLqc9zNLqc9vDMxb7PA5VEGMYHfGP15rnEgVn811BkbnJ5xTsaql3PrKOeGV2SOVHZQCyqwJGemakr5T0y+v9O85rC9ntZpQySPE5UsPQ16P8OPiVLFcW/h3X3zubZa3jtgdOEbPcnofeiwpU2lc9looopGYUUUUAFFFFABRRRQAUUUUAVdQ1C20qwlvbyTyreIAyOQTtGcZ4+teBfEvxHD4h8UZtJxNY2cflQMpyrMeWYfoM+1ez+OYPtPgfWIcOWe2YKExkt2HPvivmEb4lEU0bRurYIYYNHMr26mlNdSeVsxxj++QPwqYGp9Ht4rnSLq9lTzGthsRM9G7n9afJYsEmMLF/JUs5OApwMkDnP+NZ+1jdrsdCXUoq227kHrhqSYCYnaSroeGU4IPYitK7soP7KstUi3IzYilQ9DkEhqzoIpXMjRxSODJ/AhP8qcKsZK4mmtD6G8C+NLXxHotnHcXMQ1YKyTwjgll6tj0Iwfxrr68N+EelTw+K3vrqGeAeSyQqyYDk9SfQAD8c17lTjJS1TOaas7BRRRVEhRRRQAUUUUAFFFFAGR4ps7vUPDGoW1jKYrpoiY2AzkjnH44x+NfPdt40hcCPU7FJB0LooP5g19Nmvl/x/wCHE0zW726sUCWnnEPCBjyiT2H93+VY1aUKnxI1pSa2OisJNFvQ8umNCrv/AKyNfl3/AO8v9aWTR7WVubdJNuPlfqB9e9chc+DdVtEjubRluEZQ6NEdrYIz0quNa1uxV4JpJwWG394p3D6GuOWEmneEjpVRdTu7tNKS28vUXi8sMH2bscjoABVCfxpYWEYg0+zZlH3eNi/41zWmaHrOryiVbZ44j1mn4z+fJputaAbO9twLsSmRtgPTDZ/+vVQwUftu4Sqdjr/B+u6v4m8b6daxoqwwyieZUPRFOSST+H519AV5V8JPDsGjXmoyEGW5aNQZWXoMnIHpnjj2r1Wu2FOMFaKsctRty1CiiirMwooooAKKKKACiiigArjfiB4Rj1/QL17YLHeiIsrdA5XkZ9+OtdlXKeL/ABcuhW5hs/LlviD8rAkJxwT/AIVE5RiryKim3oeMZ123s7HUdJuPttq8KrLbg7l44ynf8vSun0qe5vLPzb2xEMgPC5DZFcV4Q8WJp8b6fqmVj8xmWUDhCSSQR6ZzXoUF/ZSxb4LmGRTz8rg1LubpNHKXGo+J76ZzHYfY7JSfkK5mce3YE1JY+GYNX8VaTpMivEWha4cBuQ+d2Cfw7Vs6l4g0/T7dpZrhDgcKhyzH0ArlPDHimdfGsWvzxn7PDuVYlxuKkEYyfrVQpuUriqSUUfQ+laTBpNsY4uXbmRz1Y1oVU0zU7TV7CK9sphLDIMgjqPYjsRVurOe9wooooAKKKKACiiigAoopskiRRtI7BUQFmJ7AUAY3ibxRZeFrAXl6HZM5ZY+WC9yB37V4+3iCy8WXlxeWqeVukJaFjkrk9ffNQfEDxG2r6s5G1rbYY3i35ZV9wOlcp4N0G+j1K71IrcLY2Fs83nKQqSt0SMsQR8xODjkY7da45x9vGyfodEPc1LWs+HTNfyT2eFkc7mjbgN6ke9c3On2KUrKGhfoR93NdnpesreWrLqcf2e6iPzHOAfcH0qHW9Ztb20NpFEJSesm3O36Z71lRqVoy9m1c2lKPLc5SKJrrHGI/U9604/8AR1WFUKgdCRjNCJtAZgAo5A/xqO/n1KOxMUUit50qyCJ+W4BGR6DB/SvTheGvc4qk+c6DQtb1HRrrz7C6eF+NwByrexHQ16l4G+Jf/CS6o2j31oIrwKzJLFykgXrkdVP6V4fbXbCNjIjRuo+ZW4IrpvhZqY0rxO2oTEiB0MUh9mOc/gQDVV5xSTYqcW7n0dRTY3WWNZEYMjAMpHQg06swCiiigAooooAK4P4m+J30PTLexjt2d78spcHAVRjOPfkY+ld5XDa+tzrOsTx20IaKyQh3Y4Vccn8f8KmaurFQV2eY+Fvh9qOs68FlV002Nw0tyRgOOu0epPT2r17WfBsNz4V/sLR3j0633h9uwsGwc4POeuDn2q/4ThMPh22zn59zjIxwWNY/xI1+80Hw6jWJ2TXMnk+aDygwSSPfinTjqrbhN7o+fNS+3aVrV3Z3BVzbTtEyMvynBx+Va6aKb62jvNP27ZF3eWxx+RrNuFilaW7vg0oT947Eks5z0z3ya0YfFNxaXjG5HmwHBTy4wpUY4AA4x7VhiVVi709zajOLVmRXdlPplg9zOFeUY2LuyEz3PrWLY3Ky3o+0uxaQ/fClsfgOcVf1fX7zUEdY7byrXGXDcs655HtTYrOKBFaDmN1DK3dhWmF9rvU3IreztojQ8d+HLnwvJpgluEnN/bEkxqdqbW4AOBnIarY8H6t4eksNQv48adcRLJ5sZzjK7ipU87h9K6vwfo8/jPw1c6JeXQa2tLyCdBLljGvzbgv1x0PHWvSfGFoJPDrbY9wgYNgDOFxg/oauquZNMim0mjH+GWvjVNImsmS4DWr/ALtpiTujPTB9vT6V3VctpVo+havFbsQYLmPCMq4G4YPPvz+tdTSjtYJ2voFFFFUSFFFFABTBDEocLGgDnL4UfMff1p9FAAAAAAMAdAK4L4tQQP4VhlmkZGS6RUOPlywI+b0Hv613tY3irw9D4o8O3WkzSGLzQCkgGdjA5Bx36U07bAfMWvE23lWBILuRJLg5AH8I/r+VXXRA7bRgKcD8KqeKPC+q+G9bezv1WR1AKSRNuVl7H1H0Ndr4A8HXfi9pLy98q206NirmNj5rtjoASQByOSPpRCaW71CUX0OSI+UjaCGBU5HY5/xqt4bm8+R9KlZVcFmhZzgAjqp/nXpnjP4Z32k2RvvDuy8giQtPBcn94AMkspGARjt1+teT2Gm32r33mW8aKZHGSr4C5745P5ZpVJre9gime7/B2/s59I1G1g+aaK43vIBwykYXB/4Cf8mvSq5LwB4Ki8GaRJC0wnvLhg08q5CnGdoAPYZ/WutoYCYpaKKACiiigAooooAKKKKACiiigD50+LmuQ6h4znt4Rbyx2saw+Zg7g4yWGQexOK9O+D9u0HgGF2jKGaeSQErjeM4B9xxXFeKfhb4j1bxdf3ltHE9tc3BdZZJVBCnHUdeP6V6/4d0n+wvDthpfmCQ2sIjLqu0MR1OKlQSdynJtWL1zbx3VrLbyjMcqFHHqCMGvkuWM6R4llQNHmyuWVXFvtY7WwDgEEdK+ua8V8bfC/WtS8V3upaZbwy2104k2iUKytgbshsd8nj1olFMIyaPYNNvodT0y2vrdw8M8ayKw7gjNWqw/B+kT6F4T0/Tbkr58KEPtORksT/WtyqJCiiigAooooASiiloASilooASilooASilooASlpkkscMbSSuqIoyWY4A/GuL1n4peH9LZo7Z31CUf8++NgPux/pmgaTex21LXj83xnu85j0eBVPTfOSf5Cov8AhdGo9tItf+/jf4UF+zkeyUV4rb/GbWE1BGvNOtTZZJdYlIkxg4AJbHXHOK9J8J+MdP8AF9rPLZRzRSW7BZYpgMjIyCCCQQefyosS4NbnQ0UUtBIlLSUtAHlHjjxX4lg8YvoGh3sEDNArqJQozkZOGIPPBri9L8S+LLjxK9hJr16NQ8wxnc4ECkDP3SOc89h2rufHNx4cbxhEl/JCl/bxoyMTtZTyRzWI3hHStV1aTU4dXkSeUDIRlxkAYI9+K8+ri1FuOzOmEVZM7bSW1OwTfcarcXkzD5jKfl/Begrbh14qcXEQI/vJ/hXN2OmXtvGqPqbTqOhZBn86v/Zcj5pGNcCxFaLupBKMWzq7a7gu498EquO+DyPqKmrgGto9NvP7Qt5GjuFBG/PUdwR3Fb9n4otr7wveatC6M1nHJ5yg8K6Lkjn8D+NephsUq3utWZlOFtUeUfFjxSdU8QDRraU/ZLHiUA8PL3z646fnXAh2eeO3hAMjcknoo9aoC6kuZpbr5pnZiSe7MT/jWpamPT7d553XzWG5z/QV1Nm0FZFmSCO3Xltz92PU1mXNzjhahtbqW+iadyfnYkD0FTfZS/Jq0gbvsJbI08LMwZpHISFfVvX6CvpjwNo9vong/TraDDO8Qkmk7vIwyxP48e2MV806KLm61tN2VtoMeWo6HJ6/WvfvAestNqWoaS8u5Y40mhX0B4b8M7fzpMznrE7qlpO9LSMRKWkpaAPIfiz4Ok1fXbPUYZVTdB5Tb14ypJHP/Av0rzTUND1bw9ALgTHyQcFomJC/UV9Na1pcOs6Rc2E+7bKhAZW2srdiCOhBrxvT/D+r2F5LZXt2b/TJFIJnA3qfTI6is5RT3NqbVjgofG3iDTsGC5aRR2JzV9fiT4nvNiRERc/MVUMT+dUfFOhNoF+Vjy9rLkxn0/2TW98PtDtdUtbiSYuk8cmAAegI4NQ8PSevKaeY99c8S3NmwuY2lRh99VwR9cVDoGuJo/hrxPZXpLQX1m6n5ckSEFVP055/D0roru6j024uLVzkwruJA6jGaxfhn4bHjv8A4SAX/mx2boqrKmMq5cNgZ6/KMfQ06dGEHeITkrHDWM0Qt9sW0Nt+UH1p2naPJdnz9QlYgc+Xnr9ao+ItEvPCPii80a4JL274RgOHQ8qw+o/rVqyvrpAACB6Bqpxa2CMlK1yxbOdjsiYUucADoKJ7qdYyIoyXI6VdS7vnGFjtj9a1dMlggnFxqs9uVXlYox396vmYzH03+2bSye9ksWjtFwN5HVj0r1v4M2k1w+p6xOzEkLAuR1/iPP5VyGueLdN1TRpNMtPmllZQuB0wQa9l8CaQdF8H2Ns67ZXXzpAeu5ucfgMD8Kad0ZSdonSUUlKKDESiiloAy/EcGo3XhzUINJkEeoSQssDFtuG+vavFLP4jWxt2g1ZWtr6IlJQRwWHB+h9q9/rldd+HXhbxJf8A27UtLV7kgBpI5GjL/wC9tIz9TSaLjKx4N4t8U2Gq2UdvbOJGVtxb0rJ8O+K00DWRdSea0LRbHWLByexwaveNvD2l2Hia6stDgMVjbnymeRmclx97kn1/lWHBocbdy59ScD8qUPe0RpKSS1PQYPEaeLIJNP0eAS6nqDsiI42lBjqx7ADJzXsvgvwrbeDvDcGlwEPIPnnmAx5kh6n6dAPYCvH/AITWsOj+MoXkYKJ4nhH+8cEfyxX0DVuLjuZSnzbHH+OfAFh4xihnYRxajb8RTlcgr3Rvb+RrxjxZ4L1HRIGFxYuioMrPGNyH/gQ/rivpekIBGCMg0gUmj4wjllZCQ3SoxfTRSjzRuTuQK90+LvgB7mJvEmiwRiWFCb6FBgyIP+Wg9WAzn1H0rxKBN0pIXcWU7R70WNFK53vw/wDC1rr3ieOa3G62CrJJkcKBjd+Z4H1r6PrgvhV4Vk8O+HftF0my5uwrbCMFEHTPuc5x9K72gibuwoFFFBAUtJRQAtFFFAHy/fTedrl3JcvJ5Ul07OHfO35jwTVFYhM+beUeWT8rCu/+LuiNZ61FqQBNtdrt6/ccdQB2yOfzry+1mNncGMH9y5yP9k+lbU7EyvY6G0F1ZzxTxXsgljYOh4IBByOK+ifDutQ6/osF7Gy+YVCzIv8ABJjkf57Yr5wjuEx8zYra0HxZqWiTsmlTE+djfHs3g4749aqootExufRVFeW+GPifd3HiG10fWLeMi8JWG5iG3D/3WHoeOa9SrCUXF2ZUZKSuhGUMpVgCCMEHvWTpvhbQdHk83T9Is7eTJPmJENwz1weoHtWvRSKCkpaKAEopaKAEooooAWiiigDwz4xaw13r66dHuKWMOXU9C7Ddn8tv615VLDcwS7H2tnBw3BGea9e+MOjxWGox6vFuzqCGOUE8b1UAEd+R/KuH1qxC6lIrL0Vf5CrUVKVl2HzNR1Objtb7UL2G3tgXl3YaPPTuMmvSNN8LeJWtlhjFlZRkfM68tXO2EFnpmmtqwucahFMP3bHhkx0rqX+IFzZ/uvs0AO0EMWPI9a83EVKqnyo6qaXLoYNxZy6fqdxBcTKxt5CodyBuweor3nwddTXvhSwnuH3uY9u/dneASAc/QV8u6xc3V9r889xlppyHA/3gCMfnX1holqbHQtPtGTY0NtHGy+hCgGu9VJTglI4vYxhNtF6iiigoKKKKACikpaAEpRSUtABTJjIIJDCFaUKdgY4BOOM0+igD5w13RfiBrupSz32n6nMqSGVY2B8tMf3RnHQY461majrdvqrJMwEFwo2yIxxzmvp6eeK2geeeVIooxud3YBVHqSa+cPiP/wAIvceLJJtO1GWWS6lDzOkYeCMEAEqwOW55OPf0pJ8r5h7qxjEDVFW1s0aeRztGxcjPpV6W5mhBtLqwWWaH5FLjBAHAzjrV3SLCLwvOurDUcQxDdCEODK2Oh7YOa9A8PS/b9HtLrULfbdvGGYyx4LejDI6Ec1M+Wbu9S6d1sM8C/D2wv0s/FGsp5926AxW2AIowpIUkd26deBXqlZ2iYGmIBGUUE4z/ABAnOR+daNUtiJbhRRRTEFFFFACUtIaWgBKWkpaACiiigDy74xW2qXdvp8Nutw1k27zFiyVZ+MbgPbpn3rjPD/wy1bXI0aa3Npbj5lnuFxkH0Xqa+hKKTjfcadjkNC+HGgaL5cjwG+uUXasl18wX/dT7q/zroZtIsbi5W4lg3uqhVBY7QB7dKvUUWQXYgUKoAAAAwAO1LRRTEFFFFABRRRQAUUUUAJRRRQAUUUUAFFc/4z8Rt4W8OyajHAZ5S6xRJjILN0z+Vee6b4m+KFy1zL9m0cI7DYJ3OEx127f656VEpxh8TKUWz2KiuO8P+JNfeRYvEGn2MK4Obi0nZue3yFf612COsiB0YMp6EURqRn8LE00LRRRViCiiigAooooAKKKKACiiigAoJwCT0FFFAHJeII7PxNYi2kL/AGcMHVkbDbh0YVVhtWtohGjAqBgZrzHWtH8UeFfFd9b6bdk6eZTLbw+YeI2OQBn06fhUi/ETUIG8q6tyki8EMK8ytSq813qdMI3Wh6JdSzRRkrEzkdlIqTw34hmi1BbO+jEEM52x72GQ/bp2PT8q84k+I7FeYxmub1Lxs11eKxjYEYw3pWdOFWMlJIv2aejPpDXfEWm+HLRbjUZygc7Y40Us7n0AH/6q831P4wXbMV0zS44l/v3LbmP4Dp+ZrivFnjB9a1M3Dnc4QIijpGvp/U+9c5C9zeFmRTtXlm6AfU17FjFQS3O5n+J/iqWVXS6ghA/gjhXB+ucmrEHxZ8Sw7vNFnNnpvhxj8iK8zmv44yVVzIw/u9Pzqu2p3G07YolHq5JplWR6q/xd1/zFciyRFOWURHDD3yc/lXUaT8bPDV6UjvfMtZCPmZR5iA/Uc/pXzt9peWb57lHDH/VgcfSpzZBLuIwKEYrklfftSbJcU9j680nW9N122e40y7S5iRtjMuRg9cEGtCvOPg3qRufC8+nvCEls5uXH8YfkE+4wR9AK9HpGTVnYKKKKBBRRRQBxHxGtBHp0OqIn7yA7XYdSvUD+f51474t1qzit4FMQN3IAcd8etfRmraZDq+mT2M5ISVcbgMkH1r5s8e2UNhrFxptw8N19nJW1ubaRXePAHySKOR7+lK12awlZWOZe9BTLbVbHQds9KrtcmS+iSJdxH6e9VDA27GDknNbuj6OBBdX093DHIiARW5J8x+Rk9Mcemc+1N2RSk3oNuYktLczStwOp6k1Faf2lrbraW0bRWwP3O7e5qe1vYb+bDJvwcAEfKv8A9eu20670zR7L5Nu8jLHuTVWKsZlp4ECoDcShT6LzTr7w3ptnaSth3YKSNx4zWtHry3ERlPAPQe1YGp6qbl/IQ/fOKBpdzAn8OeTaRTRNlmAY+1P8loY0lbqCM107KBAqdgKyNQjH2Zl9aGU0j0f4QXO3xDeQA4WW13Yz1KsO34mvZa+d/hffiHxtp3BbzQ0eAem5etfRFSc1TcKKKKCAooqO4nitbaW4mcJFEpd2PYDk0AYvjLVTpHhW+uUcLO0ZjhHcu3AA9+p/CvAG0Sx1W2a4jLxzrxIp6g+9dR4t8UyeJdS8yPfHZRDEMTH82Pua5CfURYztjrIm0/nXTGPKtTNu70MKayWGQpk5Brd8Nx2pvEF0RhzgbunFZEnmXEu4IeT34qG5mVFW3c4+bBcH1rnqtW5UbU99Tv8AxPo+nw6cb2zSONl67BjNeeQTSXk23cdoNTRa5dDT5tPuHZivy896XSbV44t7rgmlFNKzN76l93aOPap4qC2jeS43ntVllzU0EZRAQKoaLhchOay9Q86aG48iNnEMfmSFRnYuQMn2yR+dW5JSRgV6l8I/DkH9mXur3DLM13utmhZQVCAjIOeufT0pXCcrI4n4OadNd+NYLlFPk2ULSO2OBlSoH45P5V9E1naRoemaDatbaXZxWsTOXZUHUn3/AJelaFI5pO7CiiigQVV1Kzh1HS7qyuRmCeJo3GccEYNWq53xrrB0fw9K8cqpcSkJGGGd3OWH5Z/OpnJRi5MaV3Y8HvLM6T5kEjSQzR9Ype49Qe/4VzUl9LPeefFCXVEOQfqB/WvR/GOp6ZrOhKyMhuGACR/xhq4HT4Wms7/yD86ogjI/3qzp1nOnqaSguZaFKe4vSN7xiJfVjVFCk0uZgZUBBZckbhnpVryr2+mEciyM2cbcVp3Hhy6gSA24UOxxJv8A4e+fyqoq2iG7JXYzTbeF3kfmTa21XdcFlHAJHrjFaUhVF9KhWzuLBGkciROMsOPbpWho/h7VfFEl1DpqxGWCHzfLd8F+QMA9Aee/pWzTW4RqwaumZYlzIF7mrarKy4AwKktvDHiL7f5DaJfeep2lfIb+fT8a7HS/hz4h1AK00KWMRGd07fN/3yOfzxSK5kcRKVhADHJJ5r2P4Q3Rl0C+t9o2xXAYNnruUf4frSTfCHR59FNq91N9vLBhegfd9gmcY/X3rq/DPhqy8LaV9hs2kky2+SWTG52xjt29qCJTTVjZooopGQUUUUAFeM/Ey/1k30V19nMunohje2HJQ56n1P09K9mrlr3w1dTzoBJHJD5wYl2IbbkZzxyetTJXViotJnlOteENQtfBtr4hit5WLo5khb78CPja54z0Bz6bh71xfh0y2+pyWw6zxssZPI39V/WvdfH2leJLqVJ9LuZmswuxoLdijISMEnH3h/KvGJrW207TzdTzOssEwUwrgOCGHAPY49qSikuVIJTejZp2PjnTLf5dQ0mSOdeGaIBhn8cGn6hrdrqjwXunsDESySwSqAysMYOAc4PrXfx/C7wl4phh1i0vb1ra5USDy5EAY98/Lwc9R2NeYeOdFXwfrtvaWgIhQOqb+Sy5B5PfrVwtGSbFWj7SDjEq6o2pS4XaGg7LEOPx7muk+EN9LD8RjAzmNbm2ZXRv4mADfnwT+dca2uXTx7Iwie45P4Zrrvg/Zm78fx3DBn+zwSSlvQkbcn/vo1vVnGWkThwtGpTd5n0TRRRWB2hRRRQAUUUUAFFFFABRRRQAVx/jD4caL4wXzJ1a0vR0uoAAzezDo34812FFAHJeCPAkPgiG5ig1S8u0nIPlykBFPqFHf3ql8RPh4vjO0SW1uFt9RgO6NpB8j8Y2tjkduefpXdUUDufMMvwv8a294tt/YrSbjgSxzIU+uc8D64r174ZeArjwnb3F7qTqdRuVCGKM5WJAc4z3JNegUUCCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACigH/2Q==')))\nim.save('1.jpg', 'jpeg')\nim = Image.open(BytesIO(base64.b64decode(b'/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCAGAAIADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwD3+iiigBKKWigAxRRRQAYoxRVa+1Gy02Dzr66ht4+m6Vwo/WgCzSUK6uiupBVhkEdxS0AJRRRQAUUUUAGKWkpaAEpaSigBaKSigBaSiuV8a+L7Hw5ptzbvM6ahLas1sqgjcxyowe2DzQByfjr4oXGm6umn+H2R5baRlvGljym4cbfw56e1eV6lqeo+IL17zUrp5mY5Ck8D6DtVHe2yRmcs55ZmOSxPcn1zU8fCAe1M6IwSOg0Dx/4k0LSxBBefaI/MDkXGXK4Iyqk9FIGMflive/DXiG08T6JBqNowG4YljzkxPjlT9P1GDXy/C4QSIf4XxXY/DvxOnhfxHILl2Gn3ibZQq7sMM7WA/Ej8aGTKOl0fQtFRWtzFeWkNzCxaKZA6EjGQRkcGpaRiFFFFABS0lFABRRRQAUUUUAFeM/GuxubzVNMNnEryRwsHw/zAFuMjPTj+dez15D8b9Fm+zaf4hs28uW3Jt5mV8MVJyv1wd351Mr293cqO+p5rp2i3H9pRJqEJS2aQM7g8cDpntzU9zp6RC5PmiO4ExSC0Ckswzgc+/rVHT/FF7bqBcgTx9MsMH862E8W6cCjGCTchyoIBCn29K4ZSxCle33HUnGxl6zYG2FvLDG/m3fytCFzsZMA/rmpYNF1C6mAS2ZQBgFzin3fi52YpY243sScsMkk+1ZUmoa3e3P2eSa5DnA8pcr16DA9a0p+3cbOy9QfLc+o/Dk7XHh2wd1jVxCEYRtkArx/StSsrwzpI0Lw3YaaCCYIgGYLjLHknH1JrVrrV0tTke+gUUUUxBRQKWgBKWkooAKKKKAFrO1zSYta0qaxmClXHRhkVoU2SRIo2kkYKijJJPAFAHz2+k6roN5rWnW9hFdi1dG8iT+MOOMevANLpeiaZrlu9xP4fezlRtrRsWUZxnjpkfhV7x54tXSviCl/ZB57O5tFjnRjjcyMeV9CAR9cmrln4w0i8hLW9wAccpJ8rL+BrKPJL3os6VzNanOW97cQXQtdH8NCLdKI0lmQoSD/EQRnHBPWt7wp4eutV8S6jepbFwk+xZ2GFUrgHn1qjrHjy1s4H+ykXE+MKF5APua6T4TeLbeHTTpOqSrFcySGVJmOFkZuoJ7H+dXGGtyZuyPWUXZGq5zgAZ9aWl6jikqjAKKKKAClpKKACilooAKSlpKAFrxr4mfES4tdRg03TfKe2Ulp3V93mjpt/2SDn17V2/wAQfEzeHvD0rW5RruUbVUtyFPG7HpnivF9B8MSeN9XMNtG9umd00yruSH8+uRkAeuPQ1hUd3yW06mkF1ZZvdOg8RaWlwjExt80cg6of8exFcrf6FqFgMyW6yxZwJY+n4jtXQeIbeLw54jmg0eeextkk+zNDI25ZSgGXYHuxJ59MVUvry9u0SOfEcJ5BUcNXPTpVac1GLXKb88XHXcxIrXbgvjcOijoKv204CrjIDKGGRjIPQ0CMQoZpWVYxx161BZ2ckhNxGHlSPdIwj5/dAFmA56KOfpXpQaWxyyblud14V8f65puu6fpSsLyzuGWIQSkArlgMq3bGeh4r3qvkSzv2XX7aZuTEyvszjoc4r6m8P61HremrcAKkw4kjDZwfX6GonJe0cQt7qZq0UUUCCiiigAozS0YoATNLSYooA8r8daWuu+InlW4k8q3jEThQMd8g/if516Lo2kWWiaXDY2FusMKKOF6k9yT3PvU4sLTypIvs8eyR/MdcfebOcn8as1KjZ3Kcrqx87/FC7g1bxfcJDEkYhIhLqOZHHBY+/b6Cs3SrqzWF9JvQIkQ4V3frz1z2NVPElykGv6rPBIZEiu5EiZuu7eQM+4wT+FZ6wrJYQmQbj5fVuepNKpQVVWvYcKvIaPiGbTdTK2dltzCwKlOQQAeAfU8n8K0Ph/qP9jaq9nIim21FTau+wF4S/wAu9c/UZHQge1cxeILa33wrtdCHUr2Ib/Ct3S7yzi1PSr2WKSSGeZN0UX3sgjIH0q6dFUo8opVHN3PR/Bvw2tIfDNxPrunpLfi6lmgd+GAA2rnHUEruweORWh4E0YeHtYdTLJsvo2MfmDO/ByPm/wC+q9GIBGD0NV1sbZUgQRAi3/1WeSv40mru4J2VixRRRTJCiiigBaKKKACiiigQUUUUDPEvjD4ItbeT/hI7ScQNczKlzAQdrvg4cY6HA5/Osz4Y+DYfFP2yTU7lms7VViSGJtrFjzknGcAD8T9K2/jdrFzFcadpkJaOMIZ2cEjcTkY/T9a6j4QWH2TwNFcMrebdzPKxYYJGdo+owP1qbu+49LHF/E/wBb6Ho8eoaRcLBblvJniuMvkt0IbB29x+Iqt8JvB0Opal/aV3dBksHSWKGI5DP6kkdMgdK9X8eWcl94H1eGIDzRbmRM54KfMCMc544968k+C2pXMXjGazdmdLq3bfuOcFeQf5j8aG5cy10GkrHv1JS0VRIlFFFABS0lLQAUUUUAJRmlooAKKKKAM7U9B0rWTGdSsILkxghDIuSufSrsMMdvCkMMaxxRqFRFGAoHQCpKKAEZQylWAKkYII4NYuk+EtD0O/mvdM09LeeZdjlScYznABOByO1bdFAXEopaKAEooxRQAUUUUAFFFFABRRRQAUUuKKAEooyKglvbSB9kt1DG/915AD+tAE9FV49QspZBHHd27uTgKsqkn8M1YoAKKKKACiiigAooooAKKKKACiiigBssqQQvLKwWNFLMx6ADqa8N8UfFTVdTuJbTSHNlaDjzF/1rD1J/h/D869C+I2otDoEmnxOVe5Ri7D+FFGSPxwR+dfPsZHlNM3AdiRSjJNtdjanDqx7NNLIXlu5yxOSzSEmqty8CjfLczHP8R5z+dMmu1z8qlyOw6fnQtqxha5uMGQ/dHZR7VdzUrwaklldx3VrNcpcQuHjkB2lWHQ9K+rfCWq3GueE9M1O7VFnuYA7hAQM/jXyzEY3hw6g545FfUng50k8GaM0SsqfZIwFY5IwoHWkzGojbooopGQUUUUAFFFFABRRRQAUUUUAeS/FC6sE1xLbULtrUy2oEbKfvKSwP5c/mK83uNHsr2eNNO1a3lRVwsDttJ/HvXs3xQ8PaRqeiJqOpwOfsZwZ4s7okbvgdRnFeD6n4eSGy/tLS7tL2zBwzKfmT61zujJScoy3OiErxsaX/CLauh3C0jbHT94P0FRNoer3MgtntjDnrI5+UCsyx8S6tYKEhu3Kdlf5h+tdfpqeKNZjE0jw2kRGVMkR3P9Fz0+tZv6wtrFXRyGoafPpVyLeYfL1SQdGFfTXgZWXwLoocEN9lTgjB6V4DfrfX3iS38MXEcc80ssYWVCRjdjt24r6bt4I7W2it4l2xRIEQZzgAYFdFKU3G01qY1Gr6D6KKK0MwooooAKKKKACiiigAooooAgvbdbqymgYZDoRj1rx3UNJ07S5r6WO0S3aWMrOoG0HGeo6V7TXjnxzs7lBpeoWxkIYNDLGp4YDkHH5/nSk0tWaU3ZnDeD20qy1CeS/jQkLuhkZd20g84Hr7+1epW7QTwq9uyyK/Rl5zXz3Je6mHHkxmNuxIxXpnwY1i6sfE0tpq210vkAglbjy5B0A7fMDj1yBTXkVNnonhLwMbTxRfeJ9SXFzJ+7tYSP9UgABY+5xx6D68d9S0lBjuFFFFABRRRQAUUUUAFFFFABRRRQAV5T8ZBdudLVAogAkJOeS3H8v616tXnnxTu4WsINPLxrMVa4BbqFHB/r+QqZuKV5bDje+h4tbQJ9py+C2K1UzEyvGxVlIKspwQexFc7YXlu127fOkjfe3n09KmvtaeOcW9ogkk7nrXTGUUiJJtn0r4Q8QL4h0KKdmH2qL93cL/tDv9D1/wD1VvV8w+DfGWpeGPE8Nzcsj282IbiE8fISORjuOo/Ed6+ns5AIrF2voVZrcKKKKQBRRRQAUUUUAFFFFABRRRQAV4X8WoL4eKxcXULC0ZUS2kx8rKACwz65LcV7pXn3xikgj8E5cx+cLhDGGxu9Dt79+cU0wPDjaWsWplrpP3fJyOMGsvSbOS51eG2EhhimfBYfex9a29VuYFk2NkvtDHHvWSk0X2qNxI0YQ7iw6j6UVoK0mnqOnJ3SZ614Q8K6DY+LdPkeEySAsYzK5b59pxx09a9nrxHw/f8AhnT9esrptTvL6VZQqbwcKT8ucAds17dXNhlJQtJ3ZdW19AoooroMwooooAKKKKACiiigAooooAK5nxV4H03xc8El7NcwyQgqrQsOQexBBFdNR0GaAPn7x98LL7TZ477T5zPYKio7smXjOcfMF6jpyBWJ4C0Kym8Rxpdo08oUvArJlWlGCoYf3f8ACr/jrxtq3iy9n06DzItOSTMVui/OxXozEcn1x0rc+GngPXJb2z1bVGnt7KFhKiu5EkpHQY6hc+vaod2y1ojpr7RI9F1RZZILd5mIlDIMAnPP45r0pW3orDuM1natosGrmAyu6GFsgp1IPUVoRoI40jXO1QFGfQU4xswlLmSHUUUVRAUUUUAFFFFABRRRQAUUUYoAKKWigCEWtuJvOFvEJf7+wbvTrUtLRQAlFLikxQAUUUYoAKKMUUAFFFFABRRRQAUUUUAFFFFABRVe51CysmRbq7ggL52iWQLux6ZrBb4heFk1Q2DavAsmB+8J/dk+m/pQOzZ01FQwXltdAG3uIpgRuBjcNx68VNQIKKKKACiiigAooooAKKKKACiiigArjviN4ubwpoSNbMFvLlykTEA7ABlmx+Q/GuvlcRRPIQSEUsQPavnL4g6zLr3ip5b2IRx26BLeMkkbOu73JP8AL2pcyvYuEbs5a81ye/upLm58+5lc5aSRiSaga/tJcR3MciA9weR9KSS4EjkJ90d+1JI1vLFskQMPWq3NtBtrqNzp00r2V/OEKkKysVYZ68ivoz4a+Nl8WaJsuWgTULbCvGjcugAAfB98g/8A16+X5E+zuyoS0bcjPUV7p8CNBvrawvNZnEa2tz+7hUqC7bTy2eqjqMd/woexjI9jooopEBRRRQAUtJRQAUUUUAFFFFAAQCCDyD1r5+1q8tLDXbnR/ENjExt5mWKYplShOVJ9MqR7V9BVyPiW0ij1FbgRKGnT5n2/eI46/TFZ1KamtS6crM+d/EdiljefaLMKdPnOUMfKofT6elY5au/+I6Q6bHBPBCipcMY54wMB+OGx6j1rzmPaQCWJGOBTpc0VyyLlvoTxw+c4/u96+mPhQpX4fWQIwPMkxx1G81846dBcX17DaWkLSzysEjjUcsT2r6x8P6WNE8P2Gmg7jbwqjH1bv+ua0ZnI0qKKKRIUUUUAFFFFABRRRQAUUUUAFVr+xi1C0a3lyAeQy9VPqKs1xnxB8TNpGnrYWr7bu6U5YdUToSPc9B+NVGLk7ITdlc8U+JWqWsevzaFfJNOtmw23VuQockA9D6ZwfcGuID2R2rCbqQ9NrKq/rzXYanYxT2zsyjcoJBNYNhab5TJHA7qnLbFziqlRcXqylU5lc9q+HFz4W0S1t5GWztLi4/dLcTH94ZMZKlj0/TOK9YV1dQyMGUjIIOQa+S9Rtb+8083FpZTz2cThZZkX5UcjIB9OK9n+DOrTyeHm0m7lVnt/nhBPO1iSR7gH+dYNqEuS9x2bVz06iiirJCiiigAooooAKKKKACiiigAr5j8XeL7u88W313Pbo1t5pjTy33bVX5R/LP419KX93FYWE93O6pFChdmY4AxXzf4t8N28cUmp2EiIHJeSHPynPdfT6Vm8QqU0r7lqnzpmVdajFeQpHbOHab5VGcH8a6a3nsfDGkCDfGJm+aSV+rEjoBXAafZCYvglCeNw6itaLR4Fk82UvNJ/ekbNdNRVarXK7GXuQVj6A8G6bpl54BSGCMPDeiSSbcMEyMTnPuMAD6CsPwj4M1Dw54ntru8YzRvG6JxnYSAc/KMZ6jn1qb4QmYaRqCNJm3Wddkf91tvzH8fl/KvRqxlTSeuti4zdgooopiCiiigAooooAKKKKACiiigDl/Ht9PZeG28m0W682QI8T8ApySP0rwzxVbS2ml2c1tb38djeAtEJ1+RGBIKA9/Ue1e8+J5hJPYWGwlppCwOOOOP6msbx3e+HdE8JRaRrcc7w3MbJF5CBmVl/iGSMEE1HKnK7LTsjwLSpIo0w0ig+5p+q6o9rP9jixHNgFnk42g8jGfY1c8K+Hbe/8T6bbSS/u5LhCR5WQy5BPfp/jXqnxB+EcfiG8bVdF8mC8f8A18DfKsv+0COjfofat/btxtEz9mua7Oj+GWhWOkeEbe5tZhcT36ie4uA2d7dMD2XkfnXZVheDNGufD3hOw0u7kjeaBWBMZJUAsSBn2BxW7WaGFFFFABRRRQAUUUUAFFFFABRRRQAhRWYMVBZehI5FYnifwppniywW11GNsoSY5UOGQn0/T8q3KKAPH4PgzqFjq9pNa6+jWsMqufMiIcAMDgDJB/SvYKKKSVh3CiiimIKKKKACiiigAooooAKKKWgBKKXFJQAUUYpcUAJRRilxQAlFGKKACiijFABRS0XRQB//2Q==')))\nim.save('2.jpg', 'jpeg')\n```", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper addresses a crucial problem in the field by tackling the issue of multiview inconsistency in generated images. This is a significant challenge as inconsistent images often require additional techniques like SDS to derive 3D information. SyncDreamer's ability to generate multiview-consistent images directly addresses this problem, reducing the need for such supplementary methods.\n\n2. The proposed volume-based approach in SyncDreamer is a novel and innovative design. It effectively incorporates geometric regularization into the model, which enhances the generation of multiview-consistent images. The design maintains an elegant and straightforward implementation, making it a notable strength of the paper.\n\n3. The performance of SyncDreamer is highly impressive. It stands out as the first paper in the field to enable image-to-3D conversion without relying on techniques like SDS. This achievement is attributed to the model's ability to generate images with strong multiview consistency. SyncDreamer's capacity to produce high-quality and consistent results represents a significant advancement in 3D generation tasks, making it a remarkable contribution to the field.", "weaknesses": "1. One notable weakness of SyncDreamer is its computational cost, particularly the significant memory requirements associated with the volume-based approach. This could pose scalability challenges when aiming for high-resolution image generation, which is an important consideration for practical applications.\n\n2. A noticeable gap in the level of detail between images generated by Zero123 and SyncDreamer raises questions about the specifics of this discrepancy. Given that SyncDreamer leverages Zero123 as a pretrained model and maintains it frozen during training, it would be valuable to investigate whether this difference is influenced by the volume-based attention mechanism or other factors, providing a deeper understanding of the model's limitations.\n\n3. The fact that SyncDreamer is fine-tuned on Zero123 raises concerns about its generalizability and performance when trained from scratch, especially considering that both models use the same datasets. If SyncDreamer heavily relies on a strong backbone like Zero123 to achieve its results, its broader applicability might be limited, and it would be beneficial to explore how well it performs without the pretraining.\n\n4. The concatenation of features across all views in the volume, without a mechanism to handle occluded views, might lead to suboptimal performance, particularly in cases involving heavily occluded objects. A more refined approach to selectively attending to relevant views during convolutional operations could enhance the model's robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DOqxW8yKVU", "reviewer": "Reviewer_3H3E", "summary": "The paper presents a method that is able to generate multiview-consistent images. The images can be used to reconstruct 3D model with Neus.", "questions": "1. Why using stable diffusion initialization is worse than zero-1-to-3 initialization?\n2. Do you try Neus2 instead of Neus? Will Neus2 improve the reconstruction results in terms of time and accuracy?", "limitations": "", "rating": 8, "confidence": 1, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The motivation is strong and clear. The multi-view images generate by zero-1-to-3 is not consistent. The method injects 3D joint probability distribution of multiview image into zero-1-to3. The images are denoised in parallel for global awareness.\n2. The experiment is thorough and convincing. The generated images achieve better PSNR and more COLMAP points than zero-1-to-3. The generated images can be used for reconstruction.\n3. The presentation is clear. Good visualization and well-written sentences.", "weaknesses": "1. There are too many equations. It's better to use fewer equations and more intuitive sentences for method description for application papers.\n2. Why the generated image is not always plausible? What's the success rate for syncdreamer?\n3. No failure case visualization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)