Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper enables efficient multi-goal navigation through active perception.
The camera of the agent, which determines its limited field of view, is actively controlled to turn towards scene regions that are still unexplored.
Camera rotation is decoupled from agent movement, i.e. the agent can still move in directions other than the facing direction.
The camera control problem is framed as an MDP with rewards proportional to spatial exploration of the environment (measured in uncovered area, based on a maintained 2D occupancy map).
Camera actions are produced by a parametric policy optimized through PPO for the aforementioned MDP (with an explicit critic that approximates state values).
The policy is informed through a very strong heuristic for how the camera should turn, in the form of an extra objective term, but also seems to improve upon it (due to the refinement through the exploration objective from the described MDP).
It is demonstrated that actively controlling the camera leads to improved efficiency and success rate for the multi-goal navigation task on the MatterPort3D and Gibson data sets.
Both SLAM-based and end-to-end RL navigation baselines are augmented with the active camera policy in the experiments.
Generalization from MatterPort3D (used for training) to Gibson is demonstrated.

**Technical details**
The proposed scheme is used to augment existing navigation methods that are already tailored to the multi-goal navigation task.
The camera policy is recurrent and is conditioned on features from a top-down 2D map. 
It also takes in the hand-crafted heuristic camera control and the action produced by the movement policy as input (the latter is necesary only when a SLAM model is used, as camera and movement controllers are decoupled).
When an end-to-end RL model is considered, the camera and movement policy are modelled by the same recurrent neural network, jointly optimizing an MDP with rewards for both navigation and exploration.
strengths_and_weaknesses: **Strengths**
- The approach appears straightforward & functional. The design choices are pragmatic.
- The motivation is easy to understand and follow.
- The fact that camera and motion control needs to be considered jointly has been accounted for, albeit only for the end-to-end learned systems.
- Ablations of the method are appreciated, they indicate that the different modelling choices were necessary.
- It is nice to see the policy generalizes from one data set to the other.

**Weaknesses**
- The authors discuss the camera policy in the context of an MDP, but are the agent states actually observed? My impression was that the multi-goal navigation task is a partially-observed problem, where the observations are only RGB-D images. In that case the ideal decision problem would technically be a POMDP, and this should be discussed in the manuscript. 
- Optimizing the camera policy in isolation of the overarching movement control problem can be problematic, as the authors have also noted in the experimental section. This has been accounted for in the end-to-end learned experiments, where a joint control task is formulated. But for the SLAM-based controllers, I am not convinced conditioning on the movement controls is enough. Particularly if you consider the ideal POMDP formulation, the movement controller should be aware of the images that will be perceived in the future (due to camera control), as this will affect the quality of state estimation (which both policies should account for). What is the reason to keep the controllers separated in the SLAM case?
- In a POMDP the uncertainty of the state estimates normally plays a role, and I believe the agent states used to update the occupancy map are deterministic.
- The method is evaluated in only two environments in simulation, real-world experiments have remained out of scope.

Overall, I find that the proposed solution has value, the proposed idea is straightforward and seems to work as intended.
The solution is very specific in scope, tailored to the problem of actively controlling a camera for exploration, but on the other hand spatial exploration is an important topic and spatial agents are ubiquitous.
The other reservations I have are related to the missing discussion of how the method fits into the POMDP paradigm; I do not necessarily expect the method to change, but being explicit about its limitations would be appreciated.

questions: **Questions**
- The proposed method maintains a 2D occupancy map of the environment, which is necessary to compute the exploration heuristic. Where do the camera poses come from in the end-to-end learned RL experiments (for the SLAM-based navigation task I assume the SLAM estimates are used)? These poses are needed to update the occupancy map with the RGB-D observations.
- Have you considered quantifying the exact way in which the trained camera policy deviates from the heuristic?

limitations: I don't see any major negative societal impact. There is a short section on limitations right before the conclusion that mentions the lack of real-world experiments. It could be extended based on the remarks above.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper tackles the challenge of SLAM-based multi-object navigation for indoor
robots. This task is typically approached with an algorithmic or RL method which
drives a robot equipped with an RGB-D sensor through an unknown finite
environment. The robot's goal is to find a series of predefined objects under a
limited step budget. 

The paper makes the observation that prior approaches, such as Active Neural
SLAM, assume a fixed camera robot configuration with a limited forward-facing
FoV. The paper then proposes decoupling the robot motion from the camera yaw.
The robot can thus choose to, e.g., move in a direction while looking to the
side if it decides this provides more information. Two policies are learned,
one for navigation, one for looking around, with the navigation decision
feeding into the camera policy. The navigation policy is also responsible for
detecting the goal.

The paper focuses on the camera policy, which is designed to work both on top
of SLAM-based, as well as end-to-end navigation policies.

The method is trained on-policy using the photorealistic 3D environment
Matterport3D. The camera policy can generalize to the Gibson environment without
fine-tuning. In general, allowing the robot to move its camera independent of
its movement direction improves over baselines like not moving the camera,
moving it randomly, or moving it according to predefined patterns. This
improvement is consistent when evaluated on top of multiple navigation methods.
The paper also includes ablation studies for some of the key design decisions.  

strengths_and_weaknesses: ## Strengths
- [S1] The paper demonstrates cross-domain generalization by training on
  Matterport3D and successfully transferring the policies to the Gibson Env
  without any fine-tuning. 
- [S2] The proposed method is also shown to help improve both a SLAM-based, as
  well as a learning-based baseline.
- [S3] The paper defines an interesting new direction in active sensing.

## Weaknesses
- [W1] Clarity: It took me a while to realize that the method is actually fully
  compatible with both SLAM-based and end-to-end navigation systems. Would it be
  possible to add a diagram, maybe in the supmat if the main paper can't fit it,
  showing the specific information flow used when the end-to-end paradigm is
  used in navigation?
  - Either way, it would be nice to clarify this earlier in the paper, as it
    makes it easier to understand how the camera policy fits into the existing
    navigation pipeline.
- [W2] Multi-Camera Baseline: While I do understand the point brought up in
  Appendix A, I am not fully convinced by the argument.
  - A multi-pinhole-camera robot could make for an interesting comparison, in
    spite of the increased costs. A complex panoramic camera (with the
    associated geometric and hardware complexity) is not strictly required. 
  - An additional advantage of multiple fixed camera is eliminating the need for
    the moving parts of the revolute joint and its associated servo. 
  - Second, I wouldn't call the computational overhead of, say, four cameras
    "huge", since naive baselines like concatenating the images before feeding
    them to an end-to-end navigation policy would probably work pretty well
    already.
  - Multi-camera systems, albeit not depth-based, are already ubiquitous in mobile
    phones, drones, and (even non-autonomous) vehicles.
  - That being said, I still think it makes sense for this problem setting to
    exist, as coordinating between motion and active sensing is generally an
    important problem in robotics, which hasn't received as much attention as it
    should have. 


## Suggestions
- L194: "module SLAM-based" -> "modular, SLAM-based"?

questions: - [Q1] What is the actuation noise in the environment? If the policy always expects
  turning itself or its camera in exact 30° increments, suddenly rolling out in an
  environment with noisy actuation may cause serious problems.

limitations: Please see "Weaknesses".
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors propose to learn the active camera policy for multi-object robot navigation. Specifically, they show how to use camera rotations effectively to increase the information while in motion, and how to integrate the active camera policy with the navigation policy. Authors train active camera policy using Reinforcement Learning (RL) and carefully design a reward function that encourages robots to explore more areas. Through multiple baseline methods composed of SLAM-based, and end-to-end learning-based, the authors show that integrating active camera policy with the base navigation policy improves object navigation accuracy and efficiency on multiple metrics.
strengths_and_weaknesses: Strengths:
1. Paper is well written. The authors show clear motivation for the need for active cameras to aid exploration while navigating. They formulate the problem, propose a reinforcement-learning-based solution, and show the effectiveness of active cameras through extensive experiments.
2. Trained Active Camera Policy (EXPO) generalizes to other datasets than the one it’s been trained on (MatterPort3d → Gibson)
3. Authors show ablation study to show the usefulness of Active Camera Policy design components
4. Authors show improvements on 4 different baselines applied to 2 datasets. Baseline methods consist of SLAM-based, end-to-end learning-based methods, and hybrid methods.

Weakness:
1. Since the method is primarily trying to increase the scan horizon while moving, the authors should provide some details on the camera field of view. While I understand this is standard for the datasets like Mattterport3D, and Gibson, to be comprehensive it is necessary to include the field of view in the paper.
2. While an active camera is a great idea to scan an unexplored environment map while moving, the same can be achieved by fitting multiple cameras and ensuring we have a large field of view. Ideally, having multiple cameras covering a larger field of view will serve as an upper bound of what can be achieved.
3. I encourage authors to add more limitations of the method as described in the limitations review section

questions: 1. What is the egocentric map size and how did you choose it? How does the policy performance change with increasing / decreasing this map size? I believe increasing cropped egocentric map would increase % of an unexplored area of the input map, and it would be good to see how it affects the policy. 
2. For the experiment with an imperfect mapper, can the authors please also add what was the accuracy of such a trained mapper? 

limitations: The authors have mentioned very generic limitations. Few assumptions of the paper should be part of the limitations as well.
1. Along with testing the effectiveness of such a method in the real-world environment, it is also useful to mention the limitation that different types of robots might also have to experiment with environment representations. 
2. This active camera policy limits the use of a learning-based locomotion policy for high DoF robots such as Spot, and ANYMAL. Since the camera is not facing the robots' traversal path continuously.
review_title: null

Meta-review: metareview: This paper proposes to decouple the camera policy from the navigation policy in goal-driven navigation agents trained using RL, and builds upon the local and global mapping and planning approach by adding an additional recurrent network that takes as inputs global reconstructed maps, heuristic directions, and navigation actions, to predict camera left/none/right turn actions. Rewards for the camera policy come from a camera turning heuristic and from map exploration heuristic. The agent is evaluated on multi-goal navigation tasks and tested and transferred to Matterport 3D and Gibson. The authors conduct a large set of ablations and comparison b/w different mapping and planning, SLAM-based or deep RL methods.

Reviewers praised the clarity and motivation of the method (iN1f, pNpw, uqJh), the ablations (iN1f, uqJh), evaluation and improvement on a SLAM baseline (pNpw, uqJh) and the generalisation performance (iN1f, pNpw, uqJh).
Reviewers noted that camera policy was optimized irrespective of future navigation strategies, which could be an issue (iN1f), and a lack of discussion about POMDP formulations of the navigation policy and assumptions about perfect state estimation (iN1f), limitations to 2D motion (uqJh), and recommended that the authors emphasize that the method can work with pure SLAM mapping (pNpw). The authors added experiments demonstrating that a single active camera outperformed non-active multi-camera systems (pNpw, uqJh), experiments on different sizes of the egocentric map in the planner (uqJh) and experiments with actuator noise (pNpw).

Reviewers agree on high scores (6, 6, 7) and therefore I would recommend this paper for acceptance.

Thank you,
Sincerely,
Area Chair

confidence: Certain
Decision: Accept