Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an importance sampling scheme for the radiance evaluation of NeRF-style volumetric rendering. Importance sampling replaces the coarse network in NeRF with direct sampling of the high resolution density field. This promises a simpler algorithm without the need for a proxy loss on the coarse network, and improved quality/performance, as radiance samples are concentrated in regions of high opacity, while transparent regions are skipped.

The paper describes the relevant background on volumetric rendering and outlines how the piecewise constant ray integral can be recast, first as a monte-carlo estimate in ray space, and then as as an integral over the normalized opacity along the ray.

The paper describes in detail how the approximate opacity is computed with a piecewise approximation, and how samples from the distribution are drawn via inverse transform sampling. A complication of the paper is that in order to construct an approximation of the cumulative density function, the technique does require a relatively fine (256) sampling of the density field, so any performance improvement of importance sampling would be gained solely by saving radiance evaluations.

There is a insightful synthetic experiment that compares the proposed integration scheme to uniform sampling in ray space (with ablations for stratified sampling).

The paper then presents comparisons to NeRF and a voxel-based technique (DVGO). It appears from these experiments that both baselines (full NeRF with coarse model, and an unguided NeRF without hierarchical sampling) perform better than the proposed technique.


strength_and_weaknesses: The paper is well written and well motivated. The description of the technical contribution is clear, and Figures 1-3 are good visualizations of the technique. 

The outlined idea holds some promise, given that NeRFs coarse-then-fine scheme, despite its effectiveness, is not thoroughly analyzed in the original paper.

A drawback of the proposed technique is the need to finely sample the density field to create the approximation for inverse transform sampling. This seems to preclude major performance benefits that would otherwise be associated with importance sampling.

The experiments reported in Section 5.2 suggest that the proposed importance sampling scheme just doesn't work very well in its current implementation. The reconstruction quality is consistently lower than NeRF's, even if the coarse network of NeRF is disabled. (Could the authors clarify how many radiance evaluations the NeRF models in Table 1 and 2 perform?)

The comparison to DVGO also fails to show significant performance or quality improvements.

clarity,_quality,_novelty_and_reproducibility: The paper is well written. There are some typos ("improtance" in the Table 2 section) and some grammatical errors, but these don't hurt understanding of the paper. The contribution is not very novel, given that NeRF does point out the need for some form of importance sampling, and implements an effective ad-hoc importance sampling-style scheme.
summary_of_the_review: I'm leaning against accepting this paper. The empirical evaluation is unfortunately not very impressive. The technical content is solid, but only addresses a narrow aspect of the overall algorithm, and does not include enough new ideas to put the paper over the bar in light of the mixed experimental results.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper aims at avoiding computing radiance in less contributive parts by reparameterizing the sampling algorithm. This can help decrease the number of evaluations to MLPs. This sounds reasonable. But the experiments do not meet the authors’ expectations. First, decreasing the MLP evaluations should indeed improve efficiency a lot. But the results show only minor improvements on NeRF baseline. This reminds us of if the proposed approach itself slows down the efficiency. Second, DVGO is an explicit volume-based approach which is extremely efficient. I have to say the author have chosen a wrong baseline. Because the efficiency of explicit volume-based approach is not severely slow down by the great number of sampling points. Third, a big number of sampling points is vital to rendering quality. So, the rendering quality of this paper degrades. Fourth, the experiments cannot support the claim. Please refer to the weakness for more details.
strength_and_weaknesses: Strength:
1) The mathematical analysis looks great.

Weakness:
1)	The proposed method can lead to improvements on efficiency but degrades on quality. But the experiments presented in the paper are hard to support the claimed contribution. It would be fair if make experiments of evaluating two methods on the time used of reaching the same PSNR. Or vice versa, evaluating the produced PNSR with the same training time. 
2)	The improved efficiency is very slight compared with baseline approach. Normally, a smaller number of evaluations to MLPs should lead to great efficiency improvements. But in this paper, the efficiency improvement is little. The author should analyze the reason deeply, is this caused by the extra time used on solving Eq. 14 and 10?
3)	The reasons of piece wise constant has worse quality than piece wise linear are not analyzed.
4)	The reasons of different performance on three experiment setups (Fig. 3) are not analyzed.
5)	“Notably, in our experiments, the model trained with 64 samples achieved PSNR 34.33 even with 16 during evaluation stage.” I cannot find the 34.33 in paper.

clarity,_quality,_novelty_and_reproducibility: The originality is good. Clarity is fair. Quality is fair.
summary_of_the_review: The unimpressive experiment results mostly lead to a negative opinion to this paper. 
The claimed contribution on efficiency cannot be supported by the experiment. Some constraints (time used on reaching the same psnr, or evaluating the produced psnr using the same training time) have to be set for a fair comparison. And the rendering quality is also not good.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Volume rendering is a critically important algorithm in NeRF (neural radiance field) methods, which connects 3D real points with 2D image pixels. This paper aims at exploring an alternative rendering algorithm mainly focusing on the importance sampling for integration. A reparameterized Monte Carlo estimation method is proposed to enable end-to-end optimization, where fewer samples are needed for integration to save cost.
strength_and_weaknesses: **Strength**  
The topic of designing better sampling methods for volume rendering is interesting and valuable. This paper proposes a reparameterization method to enable end-to-end differentiable optimization, which is reasonable and has been evaluated in NeRF-provided benchmarks.

**Weaknesses**    
**The biggest concerns come from the experimental evaluation and comparisons.**  
* The proposed method shows a worse rendering quality while the speedup over the original NeRF and DVGO is not so significant. 
* Related works, e.g. NeRF-ID, DO-NeRF, AutoInt, TermiNeRF, should also be compared in experiments with regard to both rendering quality and speed. 
* Some highly related works are missing, including but not limited to [1, 2]. Please have a careful check.
* Experimental evaluation is not so comprehensive. Real scenes, e.g. LLFF ones, need to be included.

**Minor**:   
* The citation style does not follow the ICLR-required author-year ones.

[1] Fang J, Xie L, Wang X, et al. Neusample: Neural sample field for efficient view synthesis[J]. arXiv preprint arXiv:2111.15552, 2021.   
[2] Kurz A, Neff T, Lv Z, et al. AdaNeRF: Adaptive Sampling for Real-time Rendering of Neural Radiance Fields[J]. arXiv preprint arXiv:2207.10312, 2022.

clarity,_quality,_novelty_and_reproducibility: * **Clarity**: Most contents are clear and easy to follow.
* **Quality**: The proposed method has not yet been evaluated and studied on substantial benchmarks and settings (refer to weaknesses).
* **Novelty**: The proposed method is somewhat new but has some points in common with previous ones, e.g. density estimation.
* **Reproducibility**: Most implementation details are provided and a script is provided. It is likely to reproduce 
the method easily.
summary_of_the_review: The main concerns lie in experimental evaluation and comparisons. For its current state, no evident advantages over previous works are shown.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors proposed a rendering method for a neural radiance field. When marching on each light ray, the authors chose the sampling points using the idea of importance sampling; the samples are drawn in accordance with the contributions to the final rendered color. The method was designed so as to reduce the number of the radiance queries, which (they expected) would bring the computational advantages over the previous methods. They incorporated this sampling strategy into two baseline methods, NeRF and DIGO, and compared the performance with the original methods. 
strength_and_weaknesses: Strengths

- The idea of importance sampling is explained well in a logical manner. As far as I read, the theory and the related derivations are correct. Moreover, this idea was successfully implemented as a trainable module that can be incorporated into the existing neural rendering methods.

Weaknesses

- The authors failed to establish the effectiveness (computational advantages over the previous methods) of their proposal.  As shown in Tables 2 and 3, the authors' method achieved only a small amount (up to 25 percent) of reduction in computation time  compared to the baselines (NeRF and DVGO) at the cost of quality decline. The authors should notice the fact that the number of sampling points can be changed also for the original methods (NeRF and DVGO) so as to find a desirable trade-off between the computational cost and rendering quality. However, the authors used a fixed number of samples for the baselines,  and thus, I cannot see the advantage of the authors' proposal over the baseline methods. Moreover, the authors did not compare their proposed method against other speed-up methods for neural radiance fields. For example, DONeRF achieved a significant improvement in quality-speed trade-off: 15-78 times fewer computation and comparable rendering quality than/to the baseline NeRF. KiloNeRF achieved more than 2000 times speed up while maintaining the rendering quality. Compared to these works, the contribution of the present submission seems quite limited. 

- Insufficient consideration/evaluation for the method's design. The proposed method was designed to reduce the number of "radiance queries", but it still needs many "density queries" to obtain accurate opacity along the light ray. Moreover, the re-parameterization for the importance sampling introduced additional computational cost. Therefore, it seems that the authors' method cannot save the computation so much as the other speed-up methods. The authors should evaluate computational cost (e.g. in FLOP per pixel) involved in each of these steps (density queries, re-parameterization, and radiance queries) to see the problem clearer. Moreover, it seems that the authors sticked to the idea of using a single radiance field, rejecting the use of auxiliary/multiple networks. However, the use of auxiliary/multiple networks has practical advantages in terms of the speed-quality trade-off.
clarity,_quality,_novelty_and_reproducibility: I found no significant problems in these respects. The paper itself is well written and easy to follow. The proposed idea is clearly presented. I think the authors made a reasonable effort to include sufficient information for reproduction.

The author' idea, the use of accurate importance sampling for a neural radiance field, is somewhat new in the sense that  I do not know the same work that has been done/published before. However, as I mentioned in "Strength and Weaknesses", I do not think that this idea (in the current form) is reasonable and useful for advancing the technology in this field.
summary_of_the_review: As mentioned in "Strength and Weaknesses", I appreciate the logical explanation for the importance sampling for a neural radiance field. However, I think the authors' contribution is insufficient in the current form; they have not established the effectiveness of their proposal: computational advantages over the previous methods. Therefore, I recommend rejection for this submission.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work focuses on a novel proposed method for differentiable rendering, aiming for a computational speed-up.

According to my evaluation and the remarks of the reviewers, the main strengths of this paper are its clarity and motivation, as well as the novelty of the method. The main weakness is that it is not well demonstrated that this method outperforms existing ones. In particular, the trade-off in terms of rendering quality compared to the modest computational gain demonstrated in experiments.

As most reviewers said, this is more an evaluation of the current experimental setup, which is not complete enough, than of the method, and we think that a reworked version of this proposal - if it yields more convincing results - could be a very good contribution at another venue.
justification_for_why_not_higher_score: see meta-review above
justification_for_why_not_lower_score: see meta-review above
Decision: Reject