Kati8/ICLR_2023_2025 huggingface dataset:
authors: Yuan Liu
authors: Cheng Lin
authors: Zijiao Zeng
authors: Xiaoxiao Long
authors: Lingjie Liu
authors: Taku Komura
reviews: 
Comments: The paper presents a method that is able to generate multiview-consistent images. The images can be used to reconstruct 3D model with Neus.
Strengths: 1. The motivation is strong and clear. The multi-view images generate by zero-1-to-3 is not consistent. The method injects 3D joint probability distribution of multiview image into zero-1-to3. The images are denoised in parallel for global awareness.
2. The experiment is thorough and convincing. The generated images achieve better PSNR and more COLMAP points than zero-1-to-3. The generated images can be used for reconstruction.
3. The presentation is clear. Good visualization and well-written sentences.
Weaknesses: 1. There are too many equations. It's better to use fewer equations and more intuitive sentences for method description for application papers.
2. Why the generated image is not always plausible? What's the success rate for syncdreamer?
3. No failure case visualization.
Recommendation: 8: accept, good paper
Reviewer confidence: 1: You are unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
Soundness/correctness: 4 excellent
Clarity: 3 good
reviews: 
Comments: The paper presents a method for single-view novel view synthesis. It initializes from a pre-trained model Zero123, a finetuned StableDiffusion model on Objaverse for (relatively) consistent NVS. At each diffusion step, the UNet is conditioned on all target frames synchronously. This is achieved by constructing a feature volume from all the noisy intermediate target images at time step t, then projecting to the camera frustum of the desired target pose. Afterward, depth-wise attention is used to inject these features into the UNet to denoise a target view at time step t-1.
Strengths: * The task of single-view novel view synthesis is extremely challenging and well-motivated.
* Evaluations were done with reasonable metrics and against SOTA methods, and a large margin of improvements can be observed. Cool renderings and additional applications are shown.
* Through ablation studies were done to justify its design choices.
* The method is novel and reasonable, the consistency achieved is significantly beyond prior methods.
Weaknesses: * The biggest weakness of this paper is its writing. I feel the notations used in section 3.2 are very confusing. Following the standard notations in score-based generative model literature, for quite a while I thought the paper somehow extended the UNet so that it is able to take multiple noisy images as input, but in fact if I’m not mistaken, it still just takes a single noisy image as input, and the conditioning from the other noisy target views are just the injected features.
* I failed to understand where the features used to create the volume came from. Is it extracted with another network? Please be more specific.
* Please mention that the computation of viewpoint difference is discussed in the supplementary.
* A small introduction to zero123 could be helpful, as many of the design choices are very specific to zero123 and require some prior knowledge of how zero123 works.
Recommendation: 8: accept, good paper
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 4 excellent
Clarity: 2 fair
reviews: 
Comments: The authors proposed a synchronized multi-view image diffusion model to generate consistent multi-view images. Compared with the previous Zero123, the authors target at generating images at fixed camera views, and the denoising step of each view is conditioned on the noisy images of all views. A feature volume is introduced as an intermediate structure for aggregating information from multi-view noisy images. Experiments show that the generated multi-view images are significantly more consistent than previous methods.
Strengths: - Consistent multi-view image generation with diffusion model has been a useful and heated research topic in the past months. The authors proposed a practical solution for the task. 

- Synchronized diffusion conditioned on multi-view information is new to me and is reasonable.

- The visualization of the generated results is good, and experiments show that the multi-view consistency is much better than the Zero123 baseline.
Weaknesses: I am unclear about several components in the proposed pipeline:

- Multi-view image condition: is it possible to use the denoised multi-view images rather than the noisy images from the last step as a condition to the diffusion model? 

- 3D CNN: how is the 3D CNN trained? Is it pre-trained or trained with the attention module together in the finetuning?

- 3D CNN: how will the 3D CNN affect the feature aggregation from multi-view and can we remove this part? It reminds me of the cost volume regularization in stereo/MVS networks. If the 3D CNN is used for volume denoising, again, how about directly using the denoised multi-view images from the last step as a condition of the diffusion model?

- View frustum volume: will it be too heavy to pass a view frustum volume to the UNet as the multi-view condition? Also, how is W in the view frustum volume determined?
Recommendation: 6: marginally above the acceptance threshold
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 3 good
Clarity: 4 excellent