Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: 
This paper presents a technique for improving a pre-trained object detection model by learning an exploration policy in a 3D environment, automatically labeling the experience collected by this policy, and then using this data to finetune the object detection model. This improves the object detection performance of the pretrained model, and using this finetuned model in an agent leads to higher success in object navigation tasks.

The exploration policy operates on a 3D voxel-based map representation. Each voxel contains one number per object category (e.g., chair, bed) denoting the confidence that an object of that type is present in that voxel. This representation is built by using a pre-trained MaskRCNN and using depth information to map back pixels in 2D space to 3D voxels. This voxel-level object confidence scores are summed over time to create the 3D (voxel-based) semantic map of the environment. To learn the exploration policy, they use a curiosity signal named Gainful Curiosity. This essentially counts the number of voxels with a confident object prediction. The policy consists of 2 levels with the high level Global policy proposing a new goal every 25 timesteps, and a low level Local policy that uses a path finding algorithm to navigate to the proposed goal. The global policy is trained to maximize the reward obtained from the Gainful Curiosity signal.

The experience collected by the exploration policy is then used to finetune the object detection model. To do this, they introduce a label propagation technique called 3DLabelProp. This essentially looks at a set of connected voxels with the same label and constructs the 2D bounding boxes/masks by mapping back from 3D map to 2D observations. Then they train the object detection model on observations labeled in this way.

They evaluate their technique on a simulated house environment and show that they can improve object detection/segmentation performance of a pre-trained MaskRCNN. They also show that this improved object detection model can then be used to achieve higher success in object goal navigation.

main_review: 
The presented technique is relatively straightforward and perhaps not very novel. However, the motivation is I think quite solid; it is obvious that the 3D spatio-temporal structure of the world provides valuable information to our perceptual systems and this works shows one simple way how this can be exploited to improve object detection/segmentation. I think overall the paper is technically sound and the evaluations look OK. It is also clearly written and easy to follow. 

I think the main issue with the paper is that the presented technique itself is quite simple. This is in itself not necessarily a problem of course. However, if the presented technique is simple, I'd expect much more evidence that even the technique itself is very simple, it has very wide applicability and results in quite significant improvements on a wide variety of environments. As it stands, I think the paper unfortunately fails to show this.

Please see my other comments below:
- Does the local policy (fast marching method) know about the 3D layout of the house (e.g., walls, doors etc.)?
- In the ablation study, what is random policy? Is this just picking random actions? Or picking goals randomly (positions from 3D map)? If it is the former, I think doing the latter (sample goals randomly and navigate to them) would be a better test of how much the gainful curiosity helps.
- For the object navigation experiment that shows you get higher success with finetuned MaskRCNN, isn't this pretty much expected? I feel like a better comparison could be to train a policy to navigate to objects from RGB-D data (or the 3D semantic map with a standard/pre-trained MaskRCNN). Since your technique uses RGB-D data to improve the perceptual model, it seems fairer to provide the same info to the baseline as well. 
- typo, line 133, "both the phases" -> "both phases" or "both of the phases"
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The proposed method aims to improve the performance of a pretrained perception model in an active learning way. The whole training framework consists of the Action phase and Perception phase, which are trained with gainful curiosity reward and pseudo labels projected from that of the 3D Map, separately. The experiments conducted on detection, segmentation, and object goal navigation tasks illuminate that they can achieve a stronger perception which benefits the embodied robotic task (Object Goal Navigation) in static scenes.
main_review: Strength：
- The motivation is intuitive and reasonable as well as and the results look great.
- It is well written and easy to understand the design of gainful curiosity reward and the way to get pseudo labels.
- The weaknesses of proposed methods are discussed and addressed by weak supervision.

Weakness: 
-  It is not a new concept to employ spatial-temporal consistency to generate pseudo labels for fine-tuning the perception module. It would be better to discuss the difference to related work, e.g., [1], [2].
-  If my understanding serves me right, the framework is carried out by two steps sequentially, instead of an interactive learning style, which seems more reasonable for me.
-  Missing some details, like the evaluation metric
-  Some typos or mistakes, like ‘to used to’ in Line160.

Detail comments:
-  It is not clear whether this method is applicable to all embodied robots. If it can, why not evaluate it on more tasks. If not, can it perform well on some common dataset for the segmentation task? Or does it have its own significant advantages for Object Goal Navigation? If there are some moving objects in the scene, can the method still work well?
-  Although the proposed method outperforms the baseline methods on some metrics, like AP50, according to the video, it seems that the proposed algorithm tends to cover a larger area (even some wrong areas) compared with Mask RCNN while predicting segmentation. Can you give an explanation? 
-  The Gainful reward designing seems intuitive, do you use the number of voxels in the 3D Map (value> score threshold) directly, without any processing?



References:

[1] Zhong, Fangwei, et al. "Detect-slam: Making object detection and slam mutually beneficial." WACV 2018.

[2] Wang, Kai, et al. "A unified framework for mutual improvement of slam and semantic segmentation." ICRA 2019.


limitations_and_societal_impact: The weaknesses of proposed methods are discussed and addressed by weak supervision.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes an approach to fine-tune and transfer an object detection and segmentation approach (Mask-RCNN) to a set of 3D textured scene models by label propagation and exploration. The object detector is pretrained on an annotated large-scale Internet image dataset (MS Coco). A "perception" component maps the class pixel labels of Mask R-CNN into the 3D scene through raycasting in the given map and classifying voxels in a volumetric 3D semantic map. A policy is trained to explore the environment for fine-tuning the Mask-RCNN detector.
main_review: Strengths:
- The paper proposes an interesting novel combination of 3D semantic mapping, label propagation and exploration using a goal policy network.
- The experiments demonstrate the approach on a benchmark dataset and show good results, improving over several state of the art baselines and ablations.

Weaknesses:
- A voxel-based semantic representation is accumulated by classifying the voxels according to the highest classification score of the pixels striding the voxel. It seems the instance segmentation of Mask R-CNN is not used and individual objects cannot be distinguished. This would make the method fail in scenes where multiple instances of a class (like chairs) are close and would be detected as one instance, since Mask R-CNN would receive wrong instance segmentation labels for training. 
- The method assumes noiseless depth and pose information which is unrealistic. How does the method perform if real depth images and a localization estimator need to be used ?
- Which optical flow method is used as a baseline? Please compare against a state-of-the-art method like PWC-Net.
- Please specify the memory and runtime requirements at test time for the Action model.
- It's not clear what the "Random Policy" is: does the agent execute a random move action, or navigate to random goal locations and take images there ? I think the latter would be clearly superior.
- When does each baseline method stop collecting images? Is the same number of images collected per baseline? Are the comparisons between methods fair ?

== Post-rebuttal rating ==

The author response well addressed most of my concerns. The paper contains limited but sufficient novelty over previous methods which perform active SLAM or simultaneously train the object detector during SLAM  by presenting an actor component which is trained for semantic exploration. I increase my score to borderline 6 "marginally above acceptance threshold".
limitations_and_societal_impact: Several limitations of the work are well discussed in Sec. 6. See paper weaknesses above for unaddressed shortcomings (instance segmentation not used).
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper introduces “SEAL”, a framework for self-supervised learning of an embodied exploration policy while simultaneously refining the agent’s perception network. The method relies on building a semantic 3d map propagating confident predictions from one viewpoint to another for visual refinement. The method is validated on the Gibson dataset in terms of perceptual accuracy and on the downstream task of object goal navigation.

main_review: Originality:

Paper builds on previous work and efficiently combines previously introduced concepts (intrinsic reward for exploration, self-supervised learning of object detector, training exploration policy using RL) into a holistic framework for self-supervised learning of embodied exploration and perceptual refinement. 
This area of research is very active and I believe the authors have cited the most relevant work. The most similar work I could find was “Move to See Better: Towards Self-Supervised Amodal Object Detection” which is currently under review and cited by the authors. The methodology of the two papers differs significantly since this paper relies on building a 3d map of the environment and learning an exploration policy.

Quality:

In general the paper is technically sound and the idea interesting since self-supervised learning would enable life-long learning for robots in the field. The problem however with the paper is that it relies on ground-truth depth and position. An ablation on the effects of using predicted depth and a noisy position sensor would demonstrate the applicability of SEAL for real-world robotics beyond a simulation. If the current method requires ground-truth pose and depth then this limitation should be discussed.
 
Clarity:

The paper is generally clear and easy to read. However the result section could be improved upon to be easier to read. It took me some time to understand why you had two AP50 scores. Writing that one is for object detection and is for instance segmentation rather than implying it would improve the text.
I would also clarify early in the text what self-supervision means in your context. I.e. given a supervised pre-trained model (MaskRCNN) you can learn an exploration policy using self-supervision. The perceptual model itself is learned using a mix of supervised and self-supervised training.

Significance:

The significance of this paper lies in the novel idea of a self-supervised learning framework for embodied exploration and navigation. As a researcher I find this concept very appealing and the results on the downstream task verifies that it could have practical usefulness. 

limitations_and_societal_impact: See my note regarding the limitations of the method since it relies on ground-truth pose and depth from the simulator.
review_title: null

Meta-review: This paper proposes a method which, like active vision, combines perception and control in a closed loop to improve perception (and also an agent policy). It leverages data from pre-trained models, which are bootstrapped, and proposes two different settings, generalization vs. specialization, which also allows training on the test environment.

The paper received 4 expert reviews and was initially on the fence. The reviewers initially mainly raised issues on lack of novelty, an requested additional experiments.

The authors provided a response, which was highly appreciated by most reviewers and solved several issues, in particular through a fruitful discussion on novelty. Additional experiments also help the paper. Some reviewers raised the score above borderline, and 3 out of the 4 reviewers were convinced on acceptance.

The AC followed the discussion, and while he does not find the paper particularly well-written, concurs that it has merits and recommends acceptance.

Decision: Accept (Poster)