Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: Existing neural fields-based methods fail to reconstruct high-quality surfaces for larger and complex scenes with sparse viewpoints. In this work, the authors inspect the issue as inherent ambiguity in RGB loss which provides insufficient constraints. Inspired by the area of monocular geometry prediction, this paper proposes MonoSDF, which explores the utility of depth and normal cues predicted by general-purpose monocular estimators. Experiments demonstrate the geometric monocular priors significantly improve the performance both for single and multi-object scenes.
strengths_and_weaknesses: Strengths:

-	The proposed method, which incorporates monocular predictions to ease geometry learning, is quite clean and easy to apply to any neural implicit method.



Weaknesses:

-	The contribution is weak and relatively incremental. The entire paper can be seen as a combination of two different parts: (1) adding monocular cues as additional supervision to improve MVS; (2) exploring different architecture choices for neural implicit representations.

-	For (1), although the idea is clean and easy to understand (as stated in strengths), it is not new in the scenario of learning neural fields. For example, several papers have already explored using predicted depth and other semantic features in learning NeRF. This work applies a very similar idea to surface reconstruction. However, the methodology did not make any changes (auxiliary loss terms) compared to the case of learning standard NeRF. I would expect different choices or additional analysis due to the task of surface reconstruction instead of simply applying the loss. For example, compared to NVS, will normal cues be more important? How the noise of the prediction affects the geometry quality?

-	For (2) part, it is a bit disconnected from the main story. It is always a nice contribution to conduct a systematic exploration of the best architecture for surface reconstruction. Although the findings from the paper about MLP vs. explicit grids are not surprising, the comparison itself can be a good topic. However, such experiments can be done at any settings with different loss functions.

questions: When conducting the comparison between MLP and dense grids, how is it possible to make the comparison fair? For example, MLP-based representation is known to be computationally heavy while having low storage requirements, while dense grids can consume very large memory. It is also hard to measure the “capacity” between the MLP-based model and grid (including multi-resolution grids). Simply comparing the number of learnable parameters might not be enough. Therefore, to conduct a complete exploration of different architecture choices, simple tables (like Table 2) might not be enough. For example, for each choice, a figure is required to see the performance vs.. model capacity vs. trade-offs (spatial, convergence speed).

How robust is the monocular cue supervision to the model? For example, how the results change if there are large errors in the depth or normal predictions

In Eq 13, are w and q estimated for each image separately? In Line 189, it mentioned, “per batch”. Does one batch contain multiple images?

limitations: The paper discussed the limitations and social impacts.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed an approach to address 3D reconstruction from multi-view images. The paper is built upon several milestone papers/techniques and the results presented are relatively better. It started off from signed distance function (SDF), and volume rendering of implicit surfaces. The proposal is the incorporation of two losses, i.e. depth and normal consistency estimated from individual images. The estimation, or "ground truth" for supervision, is from a pre-trained Omnidata model [14]. The paper also explored several architectures.
strengths_and_weaknesses: *Strength*
1. The paper is easy to read and the main idea is clearly delivered. In essence, the paper took an off-the-shelf depth estimator to serve as a strong prior. 
2. The proposed approach is robust across different numbers of images. The method can not only be applied to single objects, but also large scale scenes.

*Weakness*
1. The novelty of this paper is insignificant. It pieces together several techniques (SDF, VolSDF, Omnidata etc). And many of them are well established. I think the paper can potentially compensate this weakness by addressing the points below:
1.1 The exploration of the "monocular geometric cues" is also relatively weak. It is curious to see how different depth estimators may affect the improvement when incorporating the additional prior. 
1.2 It is also peculiar that this paper chose depth and normal as the only two monocular cues to experiment. The Omnidata is able to generate high quality ground truth for 19 more more tasks. 

2. The color aspect of the reconstruction is never demonstrated or explained. The results are mostly focused on the geometric reconstruction but not the color appearance or rendered 2D images.

3. The experiments for the architectures are confusing. The paper first showed results for comparing different architectures without monocular cues and arrived at the conclusion that the best model is Multi-Res. Fea. Grids. Then after adding the proposed monocular cues, the paper concluded that MLP is the best model. This behavior is not well explained.
questions: 1. I'm wondering what is the performance comparison between the proposed monocular priors vs self-supervised depth+normal estimation? 
limitations: The paper has addressed the limitations of the existing model it used for depth estimation. However, it does not address whether Omnidata is the best prior to use compared to other models. It also does not address other monocular cues to explore.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this work, the authors proposed a novel and powerful geometric representation using neural implicit function. Previous neural implicit functions are trained purely on RGB reconstruction loss and have difficulty in representing more complicated geometry. In this work, the authors try to address this problem from two directions: first, they propose a novel depth and normal cues that significantly improves the quality of the reconstruction. Secondly, they explored different representation functions, including dense SDF grid, simple MLP, feature grid + MLP and multi-resolution feature grid + MLP. Both of these changes significantly improve the quality of reconstructed geometry.
strengths_and_weaknesses: This is a high quality work. The idea of using depth and normal cues are simple and effective. The proposed multi-resolution feature grid + MLP representation is also novel and effectively improves the reconstruction quality over MLP solution. The experimental results are thorough  and the paper is well written and easy to follow.

I don’t find any particular negative point.
questions: I only have a few minor questions for the authors:

In Table 1, in addition to network type, it is also useful to report the number of parameters of each network. Normally, in neural representation, larger network parameters often lead to a better representation power, so this information is important to understand whether the improvement comes from network architecture itself, or simply a larger set of parameters. Moreover, it would be better to compare networks with similar numbers of parameters in Table 1.

The proposed neural representation also recovered the texture of the surface (color). I think it is also important to visualize them to understand the accuracy of texture recovery, even though they are not the main task of this work. It is hard for me to tell whether the network can also represent texture, or the color prediction network is simply to help to train the SDF.

I also have one minor suggestion to this work:

In Figure 3, 4, it would be better to show actual image view or ground truth rendered image, as it is hard to tell whether some small reconstructed geometry are correct or not.


limitations: No as far as I know.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper presents a framework to utilize monocular geometric cues to improve multi-view 3D reconstruction quality, efficiency, and scalability for neural implicit surface models. A systematic comparison and detailed analysis of design choices of neural implicit surface representations including vanilla MLP and grid-based approaches has been presented. Among these representations, a simple MLP architecture performs quite well, which demonstrates that MLPs are able to represent complex scenes.
strengths_and_weaknesses: I really like the idea proposed in this paper. To improve the reconstruction quality with sparse input, shape priors should be added. There are several ways to construct the priors. One solution is to construct the parametric model for some special types like face and body. This paper explores another way with the help of depth estimation from single image. Although the estimated depth and normal may contains noises or with wrong scales, the proposed method well handles these issues.
questions: One ablation study I want to see in the paper is the number of input multi-view images and monocular geometric cues. Specifically, the monocular geometric cues could help the reconstruction when the input images are sparse. If the input images are dense, the monocular geometric cues might influence the reconstruction quality due to the error contained in the monocular cues. I wonder where is the balance for the input number of images? For a reconstruction problem with different number of input images, how should I know whether the monocular geometric cues could help or not?
limitations: Except the question listed in the above, I don't have other concerns to this paper.
review_title: null

Meta-review: metareview: There was a range of reactions to this paper from borderline reject to strong accept.  Although several of the reviewers highlighted that the contribution could be viewed as incremental, it is clearly described, and robust across different types of scenes, and I concur with the three reviewers that give positive ratings.  Therefore I am accepting this paper.
confidence: Certain
Decision: Accept