Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a method that uses a Neural Radiance Field representation as a state space for a reinforcement learning algorithm.  This approach is evaluated in tasks where multiple views of the scene are available, and where the 3D information present in the NeRF representation is useful for determining shape and pose of objects.  The paper explores settings where the NeRF represents the scene globally, and also where a compositional architecture allows representing each object with its own radiance field, and shows improved performance over several baselines.

## update after author response

Thank you to the authors for a diligent and thorough response. I have raised my score. 
strengths_and_weaknesses: ### Originality
The major contribution of the paper is using NeRFs as a representation specifically in RL, and although this is not a surprising combination of methods, it is the first paper I am aware of that uses RL on top of the NeRF representation.  The paper cites many similar approaches which use NeRFs in dynamics models which are then used for planning (though I believe the authors may want to additionally cite [Li, Li, Sitzmann, Agrawal, and Torralba. 3D Neural Scene Representations for Visuomotor Control. CoRL 2021]).  The substitution of RL for planning is a small contribution.

### Quality
The paper does well to address different methods for using NeRFs as a representation, both global and compositional, and has a thorough list of baselines for comparison.  The focus is on highlighting the advantages of using the 3D aware representations that come from a NeRF.  However, the tasks are specifically designed to increase the likelihood that this 3D information is useful, and discussion of how sensitive the approach is to those assumptions is not addressed.

### Clarity
The paper is well written and the method is described with enough background to give a self-contained understanding of the approach.  There are some issues and minor typos:
- Line 139 supevision -> supervision
- Line 188 then -> them
- In description of the CURL baseline, the text mentions carefully choosing views, but it is not clear how this is done.

### Significance
This paper presents a small but interesting contribution.  As the authors mention, it is unclear what realistic situations will actually provide multiple viewpoints of the type given in the simulations here, but this is a first glimpse to suggest that NeRFs can help.
questions: - I would have liked to see more discussion of the sensitivity of the RL performance to, say, the data available to train the NeRF model.  What sort of coverage of the scene by input views is required?
- I am surprised at the poor performance of the keypoint method.  I would guess that appropriately chosen 3D keypoint data should represent an upper bound of possible performance for the NeRF method.  What sort of limitation does the keypoint method have here that the NeRF method outperforms it?  In the door case, for example, what information besides the keypoints would the NeRF represent?   
limitations: The authors are responsible about pointing out the assumptions they make and limitations of the presented approach, and discuss these assumptions and limitations in an up-front way.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors proposed the first learned state representation through NeRF supervision for reinforcement learning purposes.
The proposed method shows that an offline trained latent-conditioned NeRF comes with a learned state representation that can be beneficial for training an RL policy online more sample efficiently.
The final results on three simulated environments are impressive compared to other learned representations and a key-points representation selected by human experts.
strengths_and_weaknesses: Pros:
* the first learned state representation through NeRF supervision for reinforcement learning purposes.
* the sample efficiency has greatly improved compared to other learned representations and a key-points representation selected by human experts.

Cons:
* Only simplified simulation environments are used rather than real-world environments rendered by NeRF.
questions: * Any thoughts on how the proposed method can be generalized to real-world environments?

limitations: I didn't find a clear section discussing the limitation.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper propose a Neural Radiance Fields-based RL (NeRF-RL) method, which makes use of supervision from NeRF to learn state representations describing the objects in the 3D scene. Several experiments involving robotic object manipulations were conducted to demonstrate the superior performance of NeRF-RL over existing methods.
strengths_and_weaknesses: ###########Strengths###########

The paper is well written, and motivated. I believe that the experiments are roughly solid and reproducible.

###########Weaknesses###########

The paper is lack of adequate analysis and discussion about the limitation of NeRF-RL. The experimental scenes are simple and clear, and have only a small number of objects. I concern about whether NeRF-RL can generalize well to complicated 3D scenes, such as VizDoom and Minecraft. 
questions: Q1. Does NeRF-RL limit to 3D scenes? Can NeRF-RL be used to solve 2D RL tasks such as Atari games? I think it is necessary to discuss whether and how NeRF-RL applies to general RL tasks.

Q2. NeRF-RL requires plenty of offline RL data to train the NeRF encoder before the policy training process. However, offline RL data are difficult to obtain for realistic 3D environments.

Q3. As mentioned before, all the experimental scenes are simple and clear. Can  NeRF-RL generalize well to complicated 3D scenes with a large number of objects? 

Q4. Does NeRF require more offline RL data and larger neural network to learn state representations than baseline methods?
limitations: See the questions.
review_title: null

Meta-review: metareview: The reviewers appreciated the detailed replies and the additional experiments. They are now all in favor of publishing the paper.
The paper proposes to use the NeRF representation on RL, the benefits of this combination is shown in a wide variety of experiments. The last open points by iBq4 and aWEL were well addressed by author replies and additional experiments.
confidence: Certain
Decision: Accept