Original: Learning Active Camera for Multi-Object Navigation
Revised: Learning Active Camera for Multi-Object Navigation

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: Abstract
Revised: Abstract

Original: Getting robots to navigate to multiple objects autonomously is essential yet difficultin robot applications.
Revised: Getting robots to navigate to multiple objects autonomously is essential yet difficultin robot applications.

Original: One of the key challenges is how to explore environmentsefficiently with camera sensors only.
Revised: One of the key challenges is how to explore environmentsefficiently with camera sensors only.

Original: Existing navigation methods mainly focus onfixed cameras and few attempts have been made to navigate with active cameras.
Revised: Existing navigation methods mainly focus onfixed cameras and few attempts have been made to navigate with active cameras.

Original: As a result, the agent may take a very long time to perceive the environmentdue to limited camera scope.
Revised: As a result, the agent may take a very long time to perceive the environmentdue to limited camera scope.

Original: In contrast, humans typically gain a larger fieldof view by looking around for a better perception of the environment.
Revised: In contrast, humans typically gain a larger fieldof view by looking around for a better perception of the environment.

Original: How tomake robots perceive the environment as efficiently as humans is a fundamentalproblem in robotics.
Revised: How tomake robots perceive the environment as efficiently as humans is a fundamentalproblem in robotics.

Original: In this paper, we consider navigating to multiple objects moreefficiently with active cameras.
Revised: In this paper, we consider navigating to multiple objects moreefficiently with active cameras.

Original: Specifically, we cast moving camera to a Markov
Revised: Specifically, we cast moving camera to a Markov

Original: Decision Process and reformulate the active camera problem as a reinforcementlearning problem.
Revised: Decision Process and reformulate the active camera problem as a reinforcementlearning problem.

Original: However, we have to address two new challenges: 1) how tolearn a good camera policy in complex environments and 2) how to coordinateit with the navigation policy.
Revised: However, we have to address two new challenges: 1) how tolearn a good camera policy in complex environments and 2) how to coordinateit with the navigation policy.

Original: To address these, we carefully design a rewardfunction to encourage the agent to explore more areas by moving camera actively.
Revised: To address these, we carefully design a rewardfunction to encourage the agent to explore more areas by moving camera actively.

Original: Moreover, we exploit human experience to infer a rule-based camera action toguide the learning process.
Revised: Moreover, we exploit human experience to infer a rule-based camera action toguide the learning process.

Original: Last, to better coordinate two kinds of policies, thecamera policy takes navigation actions into account when making camera movingdecisions.
Revised: Last, to better coordinate two kinds of policies, thecamera policy takes navigation actions into account when making camera movingdecisions.

Original: Experimental results show our camera policy consistently improves theperformance of multi-object navigation over four baselines on two datasets.
Revised: Experimental results show our camera policy consistently improves theperformance of multi-object navigation over four baselines on two datasets.

Original:  Introduction
Revised:  Introduction

Original: In the multi-object navigation task, an intelligent embodied agent needs to navigate to multiple goalobjects in a 3D environment.
Revised: In the multi-object navigation task, an intelligent embodied agent needs to navigate to multiple goalobjects in a 3D environment.

Original: Typically, no pre-computed map is available and the agent needs to use astream of egocentric observations to perceive the environment.
Revised: Typically, no pre-computed map is available and the agent needs to use astream of egocentric observations to perceive the environment.

Original: This navigation ability is the basis forindoor robots and embodied AI.
Revised: This navigation ability is the basis forindoor robots and embodied AI.

Original: Significant recent progress on this problem can be attributed to theavailability of large-scale visually rich 3D datasets [65, 7, 60, 63], developments in high-quality 3Dsimulators [65, 55, 40, 2] and research on deep memory-based architectures that combine geometryand semantics for learning representations of the 3D environment [8, 62, 52].
Revised: Significant recent progress on this problem can be attributed to theavailability of large-scale visually rich 3D datasets [65, 7, 60, 63], developments in high-quality 3Dsimulators [65, 55, 40, 2] and research on deep memory-based architectures that combine geometryand semantics for learning representations of the 3D environment [8, 62, 52].

Original: Despite these advances, how to efficiently perceive the environment and locate goal objects is still anunsolved problem.
Revised: Despite these advances, how to efficiently perceive the environment and locate goal objects is still anunsolved problem.

Original: Agents in current research [9, 12, 43] perceive the environment via an RGB-Dcamera.
Revised: Agents in current research [9, 12, 43] perceive the environment via an RGB-Dcamera.

Original: However, the camera is set to look forward and the range of its view is often limited.
Revised: However, the camera is set to look forward and the range of its view is often limited.

Original: As aresult, these agents can only perceive the environment in front of themselves within or slightly beyondthe view range [52].
Revised: As aresult, these agents can only perceive the environment in front of themselves within or slightly beyondthe view range [52].

Original: As shown at the bottom-right in Figure 1, the agent keeps looking forward andperceives redundant information which is nearly identical to the previous observations.
Revised: As shown at the bottom-right in Figure 1, the agent keeps looking forward andperceives redundant information which is nearly identical to the previous observations.

Original: In contrast,we human beings may be attracted by surrounding things so that we turn head actively to receivemore information when keeping walking straight.
Revised: In contrast,we human beings may be attracted by surrounding things so that we turn head actively to receivemore information when keeping walking straight.

Original: In this way, we can not only reach the position infront of ourselves efficiently but also get familiar with room arrangement and object location.
Revised: In this way, we can not only reach the position infront of ourselves efficiently but also get familiar with room arrangement and object location.

Original: [Figure]
Revised: [Figure]

Original: Motivated by the above observations, we design an agent which seeks to coordinate both camera andnavigation actions.
Revised: Motivated by the above observations, we design an agent which seeks to coordinate both camera andnavigation actions.

Original: The agent, called active-camera agent , employs two kinds of policies, namelythe navigation policy which determines where to go and a new camera policy which determines whereto look.
Revised: The agent, called active-camera agent , employs two kinds of policies, namelythe navigation policy which determines where to go and a new camera policy which determines whereto look.

Original: As shown at the top-right in Figure 1, the agent has found an object, i.e., blue Goal-1 , locatedat the end of a corridor.
Revised: As shown at the top-right in Figure 1, the agent has found an object, i.e., blue Goal-1 , locatedat the end of a corridor.

Original: Two kinds of policies cooperate and actuate the agent to move forward andturn its camera to the left.
Revised: Two kinds of policies cooperate and actuate the agent to move forward andturn its camera to the left.

Original: As a result, the agent walks closer to the Goal-1 and locates another goal,i.e., yellow Goal-2 , for the following navigation process.
Revised: As a result, the agent walks closer to the Goal-1 and locates another goal,i.e., yellow Goal-2 , for the following navigation process.

Original: However, it is nontrivial to learn a goodcamera policy because of sophisticated RGB-D observations and complex indoor layouts.
Revised: However, it is nontrivial to learn a goodcamera policy because of sophisticated RGB-D observations and complex indoor layouts.

Original: How toextract useful information from observations to judge which direction is worth being explored isdifficult.
Revised: How toextract useful information from observations to judge which direction is worth being explored isdifficult.

Original: Besides, the agent and the camera are moving simultaneously.
Revised: Besides, the agent and the camera are moving simultaneously.

Original: How to coordinate these twoactions for both the navigation and exploration processes is still unknown.
Revised: How to coordinate these twoactions for both the navigation and exploration processes is still unknown.

Original: In this paper, we propose an EXPloration-Oriented (EXPO) camera policy to determine cameraactions.
Revised: In this paper, we propose an EXPloration-Oriented (EXPO) camera policy to determine cameraactions.

Original: Specifically, to better understand the sophisticated RGB-D observations, we transform themto a top-down occupancy map.
Revised: Specifically, to better understand the sophisticated RGB-D observations, we transform themto a top-down occupancy map.

Original: Each pixel indicates whether it is an unexplored, free or occupiedarea.
Revised: Each pixel indicates whether it is an unexplored, free or occupiedarea.

Original: Such a map helps to simplify unstructured RGB-D observations and provides the necessaryinformation, e.g., explored areas and room layouts, for determining camera actions.
Revised: Such a map helps to simplify unstructured RGB-D observations and provides the necessaryinformation, e.g., explored areas and room layouts, for determining camera actions.

Original: Besides, toreduce the learning difficulty, we propose a heuristic module to infer an expected camera directionaccording to heuristic rules.
Revised: Besides, toreduce the learning difficulty, we propose a heuristic module to infer an expected camera directionaccording to heuristic rules.

Original: We consider the heuristic module an expert and exploit it to guide thelearning process.
Revised: We consider the heuristic module an expert and exploit it to guide thelearning process.

Original: We then feed three types of information, i.e., the progressively built map, theexpected camera direction, and the upcoming navigation action, into a neural network to predict acamera action.
Revised: We then feed three types of information, i.e., the progressively built map, theexpected camera direction, and the upcoming navigation action, into a neural network to predict acamera action.

Original: The neural network considers both the heuristic rules and the navigation intention.
Revised: The neural network considers both the heuristic rules and the navigation intention.

Original: Weuse a reinforcement learning algorithm to train the neural network by awarding camera actions thatmaximize the exploration area.
Revised: Weuse a reinforcement learning algorithm to train the neural network by awarding camera actions thatmaximize the exploration area.

Original: We incorporate the EXPO camera policy with existing navigationpolicies [52, 62] and train them in an end-to-end manner for the multi-object navigation task.
Revised: We incorporate the EXPO camera policy with existing navigationpolicies [52, 62] and train them in an end-to-end manner for the multi-object navigation task.

Original: Extensive experiments on two benchmark demonstrate the effectiveness of the proposed methods.
Revised: Extensive experiments on two benchmark demonstrate the effectiveness of the proposed methods.

Original: Our main contributions are in threefold:
Revised: Our main contributions are in threefold:

Original: • Unlike existing agents that are set to look forward, we propose a navigation paradigm that
Revised: • Unlike existing agents that are set to look forward, we propose a navigation paradigm that

Original: an agent coordinates camera and navigation actions for efficiently perceiving environmentsto solve the multi-object navigation task.
Revised: an agent coordinates camera and navigation actions for efficiently perceiving environmentsto solve the multi-object navigation task.

Original: • We propose to learn an EXPO camera policy to determine camera actions.
Revised: • We propose to learn an EXPO camera policy to determine camera actions.

Original: The camera
Revised: The camera

Original: policy leverages heuristic rules to reduce the learning difficulty and takes into account
Revised: policy leverages heuristic rules to reduce the learning difficulty and takes into account

Original: the
Revised: the

Original: navigation intention to coordinate with the navigation policy.
Revised: navigation intention to coordinate with the navigation policy.

Original: Such a camera policy can be
Revised: Such a camera policy can be

Original: incorporated with most existing navigation methods.
Revised: incorporated with most existing navigation methods.

Original: • Extensive experiments demonstrate consistent improvement over four navigation methods
Revised: • Extensive experiments demonstrate consistent improvement over four navigation methods

Original: on MatterPort3D [7] and Gibson [65] datasets.
Revised: on MatterPort3D [7] and Gibson [65] datasets.

Original: More critically, the camera policy alsoexhibits promising transferability to unseen scenes.
Revised: More critically, the camera policy alsoexhibits promising transferability to unseen scenes.

Original:  Related Work
Revised:  Related Work

Original: Visual indoor navigation.
Revised: Visual indoor navigation.

Original: According to the types of goal, visual navigation task can be categorized into different classes such as PointGoal [27, 9] task where the goal is a given coordinate,
Revised: According to the types of goal, visual navigation task can be categorized into different classes such as PointGoal [27, 9] task where the goal is a given coordinate,

Original: ObjectGoal [27, 62] task where the agent needs to navigate to an object given by language and
Revised: ObjectGoal [27, 62] task where the agent needs to navigate to an object given by language and

Original: ImageGoal [12, 68] task where the object is specified by an image.
Revised: ImageGoal [12, 68] task where the object is specified by an image.

Original: The location of the goal is notexplicitly known in the last two categories of above tasks and the agent is expected to explore theenvironment to find the goals.
Revised: The location of the goal is notexplicitly known in the last two categories of above tasks and the agent is expected to explore theenvironment to find the goals.

Original: Classical approaches [26, 61, 28, 57, 39] solve the navigation problemvia path planning [11, 28] on a constructed map, which usually requires handcraft design.
Revised: Classical approaches [26, 61, 28, 57, 39] solve the navigation problemvia path planning [11, 28] on a constructed map, which usually requires handcraft design.

Original: Recentworks [68, 47, 67, 59, 62, 15, 58, 44, 50, 32] aim to use learning-based methods in an end-to-endmanner to learn a policy for navigation.
Revised: Recentworks [68, 47, 67, 59, 62, 15, 58, 44, 50, 32] aim to use learning-based methods in an end-to-endmanner to learn a policy for navigation.

Original: Other works [9, 12, 52, 37, 41, 5, 42, 18, 46, 17, 25, 54, 22]combine the classic and learning-based methods, which use a learned SLAM module with a spatialmap or topological map.
Revised: Other works [9, 12, 52, 37, 41, 5, 42, 18, 46, 17, 25, 54, 22]combine the classic and learning-based methods, which use a learned SLAM module with a spatialmap or topological map.

Original: We propose a camera policy, which seeks to move the camera actively andcan be incorporated with existing navigation methods, to improve the navigation performance.
Revised: We propose a camera policy, which seeks to move the camera actively andcan be incorporated with existing navigation methods, to improve the navigation performance.

Original: Exploration for navigation.
Revised: Exploration for navigation.

Original: Common methods explore the environment based on heuristics likethe frontier-based exploration algorithm [66, 29, 23], which chooses frontier point between exploredand unexplored areas as the exploration goal.
Revised: Common methods explore the environment based on heuristics likethe frontier-based exploration algorithm [66, 29, 23], which chooses frontier point between exploredand unexplored areas as the exploration goal.

Original: Recent works tackle the exploration problem vialearning [9, 20, 6, 21, 51, 30, 38, 48], which allows the agent to incorporate knowledge frompreviously seen environments and generalize to novel environments.
Revised: Recent works tackle the exploration problem vialearning [9, 20, 6, 21, 51, 30, 38, 48], which allows the agent to incorporate knowledge frompreviously seen environments and generalize to novel environments.

Original: Specially, Chen et al.
Revised: Specially, Chen et al.

Original: [20]and Jayaraman et al.
Revised: [20]and Jayaraman et al.

Original: [34] use end-to-end reinforcement learning policy to maximize an externalreward ( i.e., exploration area).
Revised: [34] use end-to-end reinforcement learning policy to maximize an externalreward ( i.e., exploration area).

Original: Burda et al.
Revised: Burda et al.

Original: [6] and Dean et al.
Revised: [6] and Dean et al.

Original: [21] consider intrinsic rewards such ascuriosity for efficient exploration, which performs better when external rewards are sparse.
Revised: [21] consider intrinsic rewards such ascuriosity for efficient exploration, which performs better when external rewards are sparse.

Original: Chaplotet al.
Revised: Chaplotet al.

Original: [9], Ramakrishnan [52] and Chen et al.
Revised: [9], Ramakrishnan [52] and Chen et al.

Original: [16] infer an exploration goal by a learned policy andnavigate to it using path planner, which avoids sample complexity problem in end-to-end training.
Revised: [16] infer an exploration goal by a learned policy andnavigate to it using path planner, which avoids sample complexity problem in end-to-end training.

Original: Elhafsi et al.
Revised: Elhafsi et al.

Original: [24] predict the map out of view range for better path planning.
Revised: [24] predict the map out of view range for better path planning.

Original: Unlike existing methods,we try to actively control the camera direction for efficient exploration.
Revised: Unlike existing methods,we try to actively control the camera direction for efficient exploration.

Original: Active perception.
Revised: Active perception.

Original: Active perception [3] aims to change the state parameters of sensors according tointelligent control strategies and gain more information about the environment.
Revised: Active perception [3] aims to change the state parameters of sensors according tointelligent control strategies and gain more information about the environment.

Original: Common methodsguide the view selection using information-theoretic approaches, such as mutual information [36, 13,14 , 31].
Revised: Common methodsguide the view selection using information-theoretic approaches, such as mutual information [36, 13,14 , 31].

Original: Recent work applies this strategy on different tasks such as object recognition [35, 1, 33],object localization [10, 68, 45, 49] and scene completion [34, 53].
Revised: Recent work applies this strategy on different tasks such as object recognition [35, 1, 33],object localization [10, 68, 45, 49] and scene completion [34, 53].

Original: We refer readers to [19] and [4]for a detailed review.
Revised: We refer readers to [19] and [4]for a detailed review.

Original: In this paper, we propose a camera control strategy for active camera moving tohelp multi-object navigation.
Revised: In this paper, we propose a camera control strategy for active camera moving tohelp multi-object navigation.

Original:  Multi-object Navigation with Active Camera
Revised:  Multi-object Navigation with Active Camera

Original: 3.1 Problem Formulation
Revised: 3.1 Problem Formulation

Original: Considering an agent equipped with one RGB-D camera in a novel environment, multi-objectnavigation task asks the agent to explore the environment to find multiple goal objects and thennavigate to them.
Revised: Considering an agent equipped with one RGB-D camera in a novel environment, multi-objectnavigation task asks the agent to explore the environment to find multiple goal objects and thennavigate to them.

Original: During these processes, existing methods [62] design a navigation policy toprocess the observations o from the environment and infer a navigation action a n ( i.e., FORWARD ,
Revised: During these processes, existing methods [62] design a navigation policy toprocess the observations o from the environment and infer a navigation action a n ( i.e., FORWARD ,

Original: TURN - LEFT , TURN - RIGHT , and FOUND ).
Revised: TURN - LEFT , TURN - RIGHT , and FOUND ).

Original: The navigation action is responsible for moving towardgoal objects and indicates whether agents have found goals.
Revised: The navigation action is responsible for moving towardgoal objects and indicates whether agents have found goals.

Original: However, these methods do not considermoving camera direction actively during the navigation process.
Revised: However, these methods do not considermoving camera direction actively during the navigation process.

Original: Thus, agents can only perceive thefield along their navigation direction, which causes low efficiency of finding goal objects [52].
Revised: Thus, agents can only perceive thefield along their navigation direction, which causes low efficiency of finding goal objects [52].

Original: To resolve the above problem, we propose a new navigationparadigm that consists of a navigationpolicy π n p¨q anda camera policy π c p¨q .
Revised: To resolve the above problem, we reformulate both the navigation and camera moving process to a Markov Decision Process (MDP).

Revised: Formally, the MDP can be defined as a tuple M “ p S , A , P , R q ,where S is a finite set of states, A is a finite set of actions, P : S ˆ A Ñ S is the state transitiondistribution, R :

Original: We reformulate both the  navigation  andcamera movingprocess to a Partially Observable Markov Decision Process (POMDP), where the observation oreceived by the agent does not fully specify the state of the environment.
Revised: S ˆ A Ñ R is the reward function. Moreover, a policy π p¨q determines an actiongiven the current state. In this paper, we propose a new navigation paradigm that consists of anavigation policy π n p¨q and a camera policy π c p¨q . In our context, we denote the state for bothtwo MDPs by the observation of current environment.

Original: At each time step, for agiven state estimated from o , these two policies predict a navigation action a n „ π n p¨q togetherwith a camera action a c „ π c p¨q for active camera moving.
Revised: At each time step, for a given state, thesetwo policies predict a navigation action a n „ π n p¨q together with a camera action a c „ π c p¨q foractive camera moving.

Original: The action space A for a n is similarto navigation action space.
Revised: The action space A for a n is similar to navigation action space.

Original: The possible camera action includes { TURN - CAMERA - LEFT , TURNCAMERA - RIGHT , and KEEP }, where KEEP indicates the camera direction remains unchanged.
Revised: The possiblecamera action includes { TURN - CAMERA - LEFT , TURN - CAMERA - RIGHT , and KEEP }, where KEEPindicates the camera direction remains unchanged.

Original: Afterperforming the action, the policies will  receive a reward r whose details can be found in Section3.3.
Revised: After performing the action, the policies will Observation [Figure] receive a reward r whose details can be found in Section 3.3.

Original: We call the agent using this paradigm active-camera agent and the general scheme is shown in Figure 2.[Figure]
Revised: We call the agent using this paradigmactive-camera agent and the general scheme is shown in Figure 2.

Original: 3.2 Learning to Determine Camera Action
Revised: 3.2 Learning to Determine Camera Action

Original: Learning a camera policy for determining camera actions is challenging because 1) it is difficult tounderstand complex room structure from RGB-D observations; 2) the position of agents is changingall the time so that camera policies must coordinate with navigation policies.
Revised: Learning a camera policy for determining camera actions is challenging because 1) it is difficult tounderstand complex room structure from RGB-D observations; 2) the position of agents is changingall the time so that camera policies must coordinate with navigation policies.

Original: In this paper, we designan EXPO camera policy which consists of three components, i.e., a mapper, a heuristic module, and apolicy network.
Revised: In this paper, we designan EXPO camera policy which consists of three components, i.e., a mapper, a heuristic module, and apolicy network.

Original: The mapper transforms the RGB-D image to an occupancy top-down map, whichreveals room layout and location of unexplored areas straightforwardly.
Revised: The mapper transforms the RGB-D image to an occupancy top-down map, whichreveals room layout and location of unexplored areas straightforwardly.

Original: The heuristic module infersa heuristic direction being worth exploring.
Revised: The heuristic module infersa heuristic direction being worth exploring.

Original: The inferred direction can be served as a reference forcamera policy and reduce the learning difficulty.
Revised: The inferred direction can be served as a reference forcamera policy and reduce the learning difficulty.

Original: Then, a policy network predicts camera actionsconsidering the encoded features of the map, heuristic direction, and upcoming navigation action.
Revised: Then, a policy network predicts camera actionsconsidering the encoded features of the map, heuristic direction, and upcoming navigation action.

Original: The upcoming navigation action informs the policy network of the next location of agents.
Revised: The upcoming navigation action informs the policy network of the next location of agents.

Original: Theoverview of the camera policy is shown in Figure 2.
Revised: Theoverview of the camera policy is shown in Figure 2.

Original: Next, we introduce each component in detail.
Revised: Next, we introduce each component in detail.

Original: Mapper.
Revised: Mapper.

Original: We follow existing work [52] to build an occupancy map by a mapper.
Revised: We follow existing work [52] to build an occupancy map by a mapper.

Original: Each map pixelhas three possible values, i.e., 0 , 1 , 2 , indicating an unexplored, free or occupied area, respectively.
Revised: Each map pixelhas three possible values, i.e., 0 , 1 , 2 , indicating an unexplored, free or occupied area, respectively.

Original: Specifically, the mapper takes RGB-D as input and predicts a local map M l .
Revised: Specifically, the mapper takes RGB-D as input and predicts a local map M l .

Original: This local map representsa small area in front of the camera.
Revised: This local map representsa small area in front of the camera.

Original: Then we integrate the local map into a global map M g via aregistration function R p¨q , i.e., M g “ R p M l , M g , q q , where q is the camera’s pose representing itsposition and orientation.
Revised: Then we integrate the local map into a global map M g via aregistration function R p¨q , i.e., M g “ R p M l , M g , q q , where q is the camera’s pose representing itsposition and orientation.

Original: The global map covers the whole explored environment.
Revised: The global map covers the whole explored environment.

Original: Heuristic module.
Revised: Heuristic module.

Original: We seek to find a heuristic direction indicating which area is worth being exploredby using heuristic rules.
Revised: We seek to find a heuristic direction indicating which area is worth being exploredby using heuristic rules.

Original: The principle is to find a direction where the camera sight is not blocked byoccupancy ( e.g., wall).
Revised: The principle is to find a direction where the camera sight is not blocked byoccupancy ( e.g., wall).

Original: Following this principle, we draw K points P “ t p i u
Revised: Following this principle, we draw K points P “ t p i u

Original: Ki “ 1 uniformly aroundthe agent.
Revised: Ki “ 1 uniformly aroundthe agent.

Original: Each point is located at angle θ i from the current navigation direction and r meters awayfrom the agent.
Revised: Each point is located at angle θ i from the current navigation direction and r meters awayfrom the agent.

Original: We calculate geodesic distance D
Revised: We calculate geodesic distance D

Original: “ t D p p
Revised: “ t D p p

Original: i qu Ki “ 1 from agent to these points on theprogressively built global map M g via A* path planning algorithm D p¨q [28].
Revised: i qu Ki “ 1 from agent to these points on theprogressively built global map M g via A* path planning algorithm D p¨q [28].

Original: During path planning,we consider both the free and unexplored areas navigable.
Revised: During path planning,we consider both the free and unexplored areas navigable.

Original: If the geodesic distance d i is smaller thana threshold γ p γ ą r q and the point p i locates at unexplored areas in the global map, we denote thesepoints explorable because there is no obstacle locating between the agent and point p i .
Revised: If the geodesic distance d i is smaller thana threshold γ p γ ą r q and the point p i locates at unexplored areas in the global map, we denote thesepoints explorable because there is no obstacle locating between the agent and point p i .

Original: Moving thecamera to these explorable directions allows the agent to see unexplored parts of environment.
Revised: Moving thecamera to these explorable directions allows the agent to see unexplored parts of environment.

Original: Ifthere exist such explorable points, we select one point closest to the camera direction and take itsangle θ i as the heuristic direction θ ˚ .
Revised: Ifthere exist such explorable points, we select one point closest to the camera direction and take itsangle θ i as the heuristic direction θ ˚ .

Original: Otherwise, we set θ ˚ “ 0 , i.e., facing the navigation direction.
Revised: Otherwise, we set θ ˚ “ 0 , i.e., facing the navigation direction.

Original: Policy network.
Revised: Policy network.

Original: With the progressively built map and inferred heuristic direction, we use a recurrentnetwork to aggregate this information at each time step.
Revised: With the progressively built map and inferred heuristic direction, we use a recurrentnetwork to aggregate this information at each time step.

Original: The network takes as input map features,heuristic direction features and navigation action features.
Revised: The network takes as input map features,heuristic direction features and navigation action features.

Original: Specifically, a convolutional neural networkis exploited to extract map features from a egocentric map which is cropped from the global map.
Revised: Specifically, a convolutional neural networkis exploited to extract map features from a egocentric map which is cropped from the global map.

Original: The other two types of features are encoded from the heuristic direction θ ˚ and navigation actiona n , respectively by using a learned embedding layer.
Revised: The other two types of features are encoded from the heuristic direction θ ˚ and navigation action Let local M o M

Original: The output state features from the recurrentnetwork are fed into an actor-critic network.
Revised: R p M l , M g , q q , q is agent’s pose. Crop an map M Obtain points P u meters away from agent’s position Θ

Revised: Calculate geodesic distance from the to points D

Revised: Select set

Revised: Obtain direction θ

Original: The
Revised: S returns the to camera Sample Compute Update

Revised: 17: end while a n , respectively by using a learned embedding layer.

Revised: The output state features from the recurrentnetwork are fed into an actor-critic network.

Revised: The actor predicts a camera action and the critic predictsa value of the agent’s current state.

Original: 3.3 Reward Function for Camera Policy
Revised: 3.3 Reward Function for Camera Policy

Original: We expect the camera policy to help agents explore environments more efficiently.
Revised: We expect the camera policy to help agents explore environments more efficiently.

Original: We achieve thisobjective in two ways.
Revised: We achieve thisobjective in two ways.

Original: On the one hand, we encourage the camera policy to follow the heuristicrules by awarding actions that move cameras toward heuristic direction.
Revised: On the one hand, we encourage the camera policy to follow the heuristicrules by awarding actions that move cameras toward heuristic direction.

Original: Such heuristic reward isdefined as the reduction of angle s P r 0 ˝ , 180 ˝ s between camera direction and heuristic directioni.e.
Revised: Such heuristic reward isdefined as the reduction of angle s P r 0 ˝ , 180 ˝ s between camera direction and heuristic directioni.e.

Original: , r heuristic “ s t ´ 1 ´ s t .
Revised: , r heuristic “ s t ´ 1 ´ s t .

Original: On the other hand, because it is impractical to design a heuristic rule tocover all situations, we explicitly encourage the camera policy to explore more areas by an areareward r area “ c t ´ c t ´ 1 , which indicates the increase of explored area c in successive time steps.
Revised: On the other hand, because it is impractical to design a heuristic rule tocover all situations, we explicitly encourage the camera policy to explore more areas by an areareward r area “ c t ´ c t ´ 1 , which indicates the increase of explored area c in successive time steps.

Original: The camera agent makes its own decision according to recommended direction and occupancy mapinformation.
Revised: The camera agent makes its own decision according to recommended direction and occupancy mapinformation.

Original: In addition, to avoid constant camera moving and reduce energy consumption, we expectagents to execute camera motor for camera moving as less frequently as possible by introducing aturn-penalty reward r turn-penalty “ 1 r turn-camera s , where 1 r turn-camera s equals to 1 when the agentactuates camera motor to move a camera and otherwise equals to 0.
Revised: In addition, to avoid constant camera moving and reduce energy consumption, we expectagents to execute camera motor for camera moving as less frequently as possible by introducing aturn-penalty reward r turn-penalty “ 1 r turn-camera s , where 1 r turn-camera s equals to 1 when the agentactuates camera motor to move a camera and otherwise equals to 0.

Original: The detail about camera motorexecution will be described in Section 3.4.
Revised: The detail about camera motorexecution will be described in Section 3.4.

Original: To sum up, the reward for camera policy is as follow:
Revised: To sum up, the reward for camera policy is as follow:

Original: [Equation]
Revised: [Equation]

Original: where α and β are two scaling hyper-parameters.
Revised: where α and β are two scaling hyper-parameters.

Original: We train the camera policy to maximize suchexploration reward using PPO [56].
Revised: We train the camera policy to maximize suchexploration reward using PPO [56].

Original: In this way, the EXPO camera policy is encouraged to take intoaccount heuristic rule-based decisions and to explore more areas by executing the camera motor asless frequently as possible.
Revised: In this way, the EXPO camera policy is encouraged to take intoaccount heuristic rule-based decisions and to explore more areas by executing the camera motor asless frequently as possible.

Original: 3.4 Combination of Navigation and Camera Policies
Revised: 3.4 Combination of Navigation and Camera Policies

Original: Navigation and camera actions execution.
Revised: Navigation and camera actions execution.

Original: We assume the camera is attached on a mobile robotplatform using a revolute joint.
Revised: We assume the camera is attached on a mobile robotplatform using a revolute joint.

Original: The robot can move to a location using a wheel actuator and turnthe camera using a camera motor simultaneously.
Revised: The robot can move to a location using a wheel actuator and turnthe camera using a camera motor simultaneously.

Original: The action space in this paper is based on thereal-world coordinate.
Revised: The action space in this paper is based on thereal-world coordinate.

Original: The rotation angles for navigation and camera actions are set to the same.
Revised: The rotation angles for navigation and camera actions are set to the same.

Original: Thus, if both navigation and camera actions tend to move to the same direction, e.g., turning theplatform and camera to the left, we only need to actuate the robot wheel and do not need to actuatethe camera motor.
Revised: Thus, if both navigation and camera actions tend to move to the same direction, e

Original: Incorporating camera policy into existing navigation methods.
Revised: Incorporating camera policy into existing navigation methods.

Original: Existing navigation methods canbe mainly categorized into two types, namely module SLAM-based and end-to-end learning-based.
Revised: Existing navigation methods canbe mainly categorized into two types, namely module SLAM-based and end-to-end learning-based.

Original: For the first type of navigation method, we use the existing navigation policy [52] to infer a navigationaction, which is then fed to the proposed camera policy to infer a camera action.
Revised: For the first type of navigation method, we use the existing navigation policy [52] to infer a navigationaction, which is then fed to the proposed camera policy to infer a camera action.

Original: The paradigmis shown in Figure 2.
Revised: The paradigm isshown in Figure 2.

Original: In this case, it is worth mentioning that the camera policy coordinates with the navigation policy by conditioning the output of the navigation policy.

Original: The navigation policydecides navigation action by exploiting a progressively built map, which is built from historicalcamera policy output.

Original: For the end-to-end learning-based navigation method, the existing navigationpolicy [62] typically contains a same recurrent network as our camera policy and uses it to predictnavigation actions.
Revised: For the end-to-end learning-based navigation method, the existing navigationpolicy [62] typically contains a same recurrent network as our camera policy and uses it to predictnavigation actions.

Original: To better coordinate two kinds of policies, we use one recurrent network to predictboth navigation action and camera action.
Revised: To better coordinate two kinds of policies, we use one recurrent network to predictboth navigation action and camera action.

Original: Specifically, we feed this recurrent network both camerapolicy input ( i.e., map and heuristic direction) and navigation policy input (typically RGB-D images).
Revised: Specifically, we feed this recurrent network both camerapolicy input ( i.e., map and heuristic direction) and navigation policy input (typically RGB-D images).

Original: Unlike the paradigm in Figure 2, the recurrent network does not need the features of navigation actionanymore because it predicts both types of actions at the same time (the paradigm is shown in thesupplementary material).
Revised: Unlike the paradigm in Figure 2, the recurrent network does not need the features of navigationaction anymore because it predicts both types of actions at the same time.

Original: We use the summation of exploration reward in Equation 1 and navigationreward in existing methods (typically the distance reduced to goals) to evaluate the goodness ofpredicted navigation-camera actions.
Revised: We use the summation ofexploration reward in Equation 1 and navigation reward in existing methods (typically the distancereduced to goals) to evaluate the goodness of predicted navigation-camera actions.

Original:  Experiments
Revised:  Experiments4.1 Experimental Setup

Original: 4.1 Experimental Setup

Original: Task details.
Revised: Task details.

Original: We follow MultiON [62] to place multiple goal objects in the environment randomly.
Revised: We follow MultiON [62] to place multiple goal objects in the environment randomly.

Original: The objects are embodied with cylinders in different colors.
Revised: The objects are embodied with cylinders in different colors.

Original: In this way, We are free to decide thelocation and the number of goal objects to adjust the task difficulty.
Revised: In this way, We are free to decide thelocation and the number of goal objects to adjust the task difficulty.

Original: The agent succeeds in an episodeif it calls FOUND action within a 1.5 meters radius of all objects in order.
Revised: The agent succeeds in an episodeif it calls FOUND action within a 1.5 meters radius of all objects in order.

Original: We call FOUND automaticallywhen the agent is near the current goal object because we focus on evaluating the effectiveness ofthe agent finding objects and navigating to them.
Revised: We call FOUND automaticallywhen the agent is near the current goal object because we focus on evaluating the effectiveness ofthe agent finding objects and navigating to them.

Original: By default, we place three goal objects and denoteit 3-ON task.
Revised: By default, we place three goal objects and denoteit 3-ON task.

Original: We also show results of 2-ON and 1-ON in the supplementary material.
Revised: We also show results of 2-ON and 1-ON in the supplementary material.

Original: We performexperiments on two photorealistic 3D indoor environments, i.e., Matterport3D [7] and Gibson [65].
Revised: We performexperiments on two photorealistic 3D indoor environments, i.e., Matterport3D [7] and Gibson [65].

Original: We follow the existing work [62] to evaluate the navigation success rate using Success and Progressmetrics.
Revised: We follow the existing work [62] to evaluate the navigation success rate using Success and Progressmetrics.

Original: Success indicates the percentage of successfully finished episodes, while Progress indicatesthe percentage of objects being successfully found.
Revised: Success indicates the percentage of successfully finished episodes, while Progress indicatesthe percentage of objects being successfully found.

Original: We also evaluate the navigation efficiency using
Revised: We also evaluate the navigation efficiency using

Original: SPL and PPL , which are short for Success weighted by Path Length and Progress weighted by Path
Revised: SPL and PPL , which are short for Success weighted by Path Length and Progress weighted by Path

Original: Length, respectively.
Revised: Length, respectively.

Original: The weight is proportional to the navigation efficiency and is defined as d { ¯ d ,where d is the length of the ground-truth path and ¯ d is the path length traveled by an agent.
Revised: The weight is proportional to the navigation efficiency and is defined as d { ¯ d ,where d is the length of the ground-truth path and ¯ d is the path length traveled by an agent.

Original: Implementation details.
Revised: Implementation details.

Original: For navigation and camera actions, a FORWARD action moves the agentforward by 0.25 meters and a TURN action turns by 30 ˝ .
Revised: For navigation and camera actions, a FORWARD action moves the agentforward by 0.25 meters and a TURN action turns by 30 ˝ .

Original: The maximum episode time step is 500because an agent with a global ground-truth map ( i.e., oracle agent) finishes more than 97% episodeswithin 500 steps.
Revised: The maximum episode time step is 500because an agent with a global ground-truth map ( i.e., oracle agent) finishes more than 97% episodeswithin 500 steps.

Original: Our camera policy tries to narrow the performance gap between such an oracle agentand the agent with a progressively built map.
Revised: Our camera policy tries to narrow the performance gap between such an oracle agentand the agent with a progressively built map.

Original: We set K “ 8 , r “ 2 . 4 , γ “ r ˆ 1 . 2 “ 2 . 88 in heuristicmodule empirically.
Revised: We set K “ 8 , r “ 2 . 4 , γ “ r ˆ 1 . 2 “ 2 . 88 in heuristicmodule empirically.

Original: We use a mapper that outputs the ground-truth of occupancy anticipation [52],because how to train a good mapper is orthogonal to our work.
Revised: We use a mapper that outputs the ground-truth of occupancy anticipation [52],because how to train a good mapper is orthogonal to our work.

Original: Reward scaling factors α and βare set to 10 and 1 respectively such that three reward terms are in the same order of magnitude atinitialization.
Revised: Reward scaling factors α and βare set to 10 and 1 respectively such that three reward terms are in the same order of magnitude atinitialization.

Original: We evaluate models for five runs using a same set of random seeds and report the meanresults and standard deviation.
Revised: We evaluate models for five runs using a same set of random seeds and report the meanresults and standard deviation.

Original: More details are shown in the supplementary material.
Revised: More details are shown in the supplementary material.

Original: [Table]
Revised: 4.2 Baselines

Original: Mapping + FBE [66]: This modular SLAM-based navigation method breaks the problem intomapping and path planning.
Revised: Mapping + FBE [66]: This modular SLAM-based navigation method breaks the problem intomapping and path planning.

Original: We use depth projection [20] for mapping and use the frontier-boundaryexploration (FBE) method [66] to select an exploration point.
Revised: We use depth projection [20] for mapping and use the frontier-boundaryexploration (FBE) method [66] to select an exploration point.

Original: Once the built map covers goalobjects, the agent navigates to them by taking deterministic actions [8] along the path planned by A*algorithm [28].
Revised: Once the built map covers goalobjects, the agent navigates to them by taking deterministic actions [8] along the path planned by A*algorithm [28].

Original: OccAnt [52]: This baseline is the same as the previous one except that we replace the depth projectionwith an occupancy anticipation neural network [52].
Revised: OccAnt [52]: This baseline is the same as the previous one except that we replace the depth projectionwith an occupancy anticipation neural network [52].

Original: Such a network can infer the occupancy statebeyond the visible regions.
Revised: Such a network can infer the occupancy statebeyond the visible regions.

Original: We assume the neural network is well-trained so that we use the groundtruth occupancy state within the field of view for experiments.
Revised: We assume the neural network is well-trained so that we use the groundtruth occupancy state within the field of view for experiments.

Original: DD-PPO [64]: This end-to-end learning-based baseline performs navigation using RL algorithm.
Revised: DD-PPO [64]: This end-to-end learning-based baseline performs navigation using RL algorithm.

Original: Itconsists of a recurrent policy network, which takes as input RGB-D images, goal object categoriesand previous actions for predicting navigation actions.
Revised: Itconsists of a recurrent policy network, which takes as input RGB-D images, goal object categoriesand previous actions for predicting navigation actions.

Original:  MultiON [62]: This is the variant of DD-PPO, with a progressively built object map as an extrainput.
Revised: [Table] MultiON [62]: This is the variant of DD-PPO, with a progressively built object map as an extrainput.

Original: Each cell of the object map is a one-hot vector indicating the existence of the goal objects.
Revised: Each cell of the object map is a one-hot vector indicating the existence of the goal objects.

Original: Westore an object on the map once it is within the field of view of agents.
Revised: Westore an object on the map once it is within the field of view of agents.

Original: We encode the egocentriccropped object map and feed it to the policy network.
Revised: We encode the egocentriccropped object map and feed it to the policy network.

Original: This baseline is the same as OracleEgoMapvariant presented by [62].
Revised: This baseline is the same as OracleEgoMapvariant presented by [62].

Original: 4.3 Multi-object Navigation Results
Revised: 4.3 Multi-object Navigation Results

Original: Results on Matterport3D dataset.
Revised: Results on Matterport3D dataset.

Original: In Table 1, the agent with our EXPO camera policy performsbetter on multi-object navigation task upon four baselines.
Revised: In Table 1, the agent with our EXPO camera policy performsbetter on multi-object navigation task upon four baselines.

Original: Specifically, our agent increases Successand Progress metrics for a large margin, indicating incorporating our camera policy helps agentssuccessfully navigate to more goal objects.
Revised: Specifically, our agent increases Successand Progress metrics for a large margin, indicating incorporating our camera policy helps agentssuccessfully navigate to more goal objects.

Original: We attribute the improvement to a better explorationability of our agent.
Revised: We attribute the improvement to a better explorationability of our agent.

Original: With such ability, the agent finds more goal objects in a limited time step andthen navigates to them.
Revised: With such ability, the agent finds more goal objects in a limited time step andthen navigates to them.

Original: Besides, the improvement on SPL and PPL indicates our agent navigatesto goal objects along a shorter path.
Revised: Besides, the improvement on SPL and PPL indicates our agent navigatesto goal objects along a shorter path.

Original: The agent does not need to walk inside all rooms.
Revised: The agent does not need to walk inside all rooms.

Original: Instead, theactively moving camera allows the agent to perceive what is inside a room when it passes by.
Revised: Instead, theactively moving camera allows the agent to perceive what is inside a room when it passes by.

Original: Theabove results show our agent navigates to goal objects more efficiently in a higher success rate.
Revised: Theabove results show our agent navigates to goal objects more efficiently in a higher success rate.

Original: Weencourage readers to see the visualization in Figure 4 and watch supplementary videos.
Revised: Weencourage readers to see the visualization in Figure 4 and watch supplementary videos.

Original: The camera action provides more movement freedom for the agent.
Revised: The camera action provides more movement freedom for the agent.

Original: We are interested in the questionthat whether the improvement comes from simply extending the action space.
Revised: We are interested in the questionthat whether the improvement comes from simply extending the action space.

Original: To this end, we removeour camera policy.
Revised: To this end, we removeour camera policy.

Original: The agent determines camera actions naively.
Revised: The agent determines camera actions naively.

Original: Specifically, for SLAM-basedbaselines, the agent chooses a camera action randomly.
Revised: Specifically, for SLAM-basedbaselines, the agent chooses a camera action randomly.

Original: For learning-based baselines, the agent learnsto predict both camera and navigation actions using the original navigation input and rewards.
Revised: For learning-based baselines, the agent learnsto predict both camera and navigation actions using the original navigation input and rewards.

Original: In
Revised: In

Original: Table 1, using these naive camera actions brings little improvement or even negative influence.
Revised: Table 1, using these naive camera actions brings little improvement or even negative influence.

Original: Thisis not surprised because it is nontrivial to coordinate the camera and navigation actions.
Revised: Thisis not surprised because it is nontrivial to coordinate the camera and navigation actions.

Original: Also, a largeraction space may increase the learning difficulty.
Revised: Also, a largeraction space may increase the learning difficulty.

Original: These results further suggest the importance of theproposed camera policy for determining a reasonable camera action.
Revised: These results further suggest the importance of theproposed camera policy for determining a reasonable camera action.

Original: Transferability of camera policy to Gibson dataset.
Revised: Transferability of camera policy to Gibson dataset.

Original: We evaluate the transferability of the learnedcamera policy on Gibson dataset.
Revised: We evaluate the transferability of the learnedcamera policy on Gibson dataset.

Original: In Table 1 , a similar trend is observed on all baselines, i.e., usingnaive camera actions does not help for navigation while our EXPO camera policy performs better thanbaselines.
Revised: In Table 1 , a similar trend is observed on all baselines, i.e., usingnaive camera actions does not help for navigation while our EXPO camera policy performs better thanbaselines.

Original: It is worth noting that the EXPO camera policy is trained on Matterport3D and has not beenfine-tuned on Gibson dataset.
Revised: It is worth noting that the EXPO camera policy is trained on Matterport3D and has not beenfine-tuned on Gibson dataset.

Original: There are significant differences between these two datasets in scenestyle and layout distribution [7, 65].
Revised: There are significant differences between these two datasets in scenestyle and layout distribution [7, 65].

Original: The consistent improvement demonstrates that our active-cameraagent has learned general exploration and navigation skills for the multi-object navigation task.
Revised: The consistent improvement demonstrates that our active-cameraagent has learned general exploration and navigation skills for the multi-object navigation task.

Original: Italso shows the possibility of transferring the agent to a real-world scene.
Revised: Italso shows the possibility of transferring the agent to a real-world scene.

Original: 4.4 Further Analysis
Revised: 4.4 Further Analysis

Original: Rule-based camera actions vs. learned camera policy.
Revised: Rule-based camera actions vs. learned camera policy.

Original: In contrast to learning a neural network,one may use handcrafted rules to decide camera actions.
Revised: In contrast to learning a neural network,one may use handcrafted rules to decide camera actions.

Original: We compare the learned camera policywith three types of rule-based camera actions, i.e., 1) selecting a random camera action; 2) forcingthe agent to look forward and swing within 90 ˝ around the navigation direction; 3)
Revised: We compare the learned camera policy

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: with three types of rule-based camera actions, i.e., 1) selecting a random camera action; 2) forcingthe agent to look forward and swing within 90 ˝ around the navigation direction; 3) following theheuristic direction inferred from the heuristic module.

Original: The results upon a SLAM-based baseline, i.e., OccAnt, are shown in Table 3.
Revised: The results upon a SLAM-based baseline, i.e., OccAnt, are shown in Table 3.

Original: Exploiting randomcamera actions drops the performance because the agent often looks backward and captures redundantuseless information.
Revised: Exploiting randomcamera actions drops the performance because the agent often looks backward and captures redundantuseless information.

Original: The other two types of rule-based camera actions improve the performanceslightly.
Revised: The other two types of rule-based camera actions improve the performanceslightly.

Original: These camera actions help the agent build a map covering more areas.
Revised: These camera actions help the agent build a map covering more areas.

Original: Consequently, the
Revised: Consequently, the

Original: SLAM-based navigation policy can plan a better path for navigation using path planning algorithm.
Revised: SLAM-based navigation policy can plan a better path for navigation using path planning algorithm.

Original: However, it is hard for us to design a robust rule covering all situations.
Revised: However, it is hard for us to design a robust rule covering all situations.

Original: For example, the swingcamera actions may miss some explorable areas because the agent has passed by these areas before thecamera swing to the direction pointing to them.
Revised: For example, the swingcamera actions may miss some explorable areas because the agent has passed by these areas before thecamera swing to the direction pointing to them.

Original: Also, there exist false positive areas in the occupancymap ( e.g., free space behind a table is predicted as obstacles).
Revised: Also, there exist false positive areas in the occupancymap ( e.g., free space behind a table is predicted as obstacles).

Original: These areas may mislead the heuristicmodule to consider an unexplored area as an explored one.
Revised: These areas may mislead the heuristicmodule to consider an unexplored area as an explored one.

Original: Compared with these camera actions, ourlearned camera policy brings a more significant improvement.
Revised: Compared with these camera actions, ourlearned camera policy brings a more significant improvement.

Original: We attribute the improvement to theexploration reward in Equation 1.
Revised: We attribute the improvement to theexploration reward in Equation 1.

Original: With such a reward, the agent is encouraged to take into accountnot only the handcraft rules but also the noisy occupancy map to predict a better camera action.
Revised: With such a reward, the agent is encouraged to take into accountnot only the handcraft rules but also the noisy occupancy map to predict a better camera action.

Original: As for learning-based baseline, i.e., MultiON, using these three types of rule-based camera actionssignificantly drops the performance.
Revised: As for learning-based baseline, i.e., MultiON, using these three types of rule-based camera actionssignificantly drops the performance.

Original: In these experiments, it is worth noting that we have fed previouscamera action to inform the policy which direction the observation is taken from.
Revised: In these experiments, it is worth noting that we have fed previouscamera action to inform the policy which direction the observation is taken from.

Original: We suspectthe poor performance is because the camera movement decided by rules is uncontrollable by thenavigation policy.
Revised: We suspectthe poor performance is because the camera movement decided by rules is uncontrollable by thenavigation policy.

Original: As a result, the navigation policy can not get desired observations for predictingnext navigation action.
Revised: As a result, the navigation policy can not get desired observations for predictingnext navigation action.

Original: In contrast, the learned camera policy, which is trained together with thenavigation policy, allows the agent to determine how to move its camera by itself.
Revised: In contrast, the learned camera policy, which is trained together with thenavigation policy, allows the agent to determine how to move its camera by itself.

Original: The above resultsfurther demonstrate the importance of the learned EXPO camera policy.
Revised: The above resultsfurther demonstrate the importance of the learned EXPO camera policy.

Original: Ablation study on camera policy inputs and rewards.
Revised: Ablation study on camera policy inputs and rewards.

Original: We conduct this ablation study by removingone of inputs and rewards of a camera policy upon OccAnt baseline on MatterPort3D dataset.
Revised: We conduct this ablation study by removingone of inputs and rewards of a camera policy upon OccAnt baseline on MatterPort3D dataset.

Original: In
Revised: In

Original: Table 2, removing any input or reward will drop the performance.
Revised: Table 2, removing any input or reward will drop the performance.

Original: We note that human knowledge( i.e., heuristic input and reward) is important for the camera policy.
Revised: We note that human knowledge( i.e., heuristic input and reward) is important for the camera policy.

Original: Awarding the policy to follow thisknowledge can be considered as a form of regularization and guidance for learning.
Revised: Awarding the policy to follow thisknowledge can be considered as a form of regularization and guidance for learning.

Original: The egocentricoccupancy map input and area reward are also critical.
Revised: The egocentricoccupancy map input and area reward are also critical.

Original: They encourage the agent to explore moreareas.
Revised: They encourage the agent to explore moreareas.

Original: With a better exploration ability, the agent can find goal objects and navigate to them moreefficiently.
Revised: With a better exploration ability, the agent can find goal objects and navigate to them moreefficiently.

Original: The turn-penalty reward has little influence on the navigation performance.
Revised: The turn-penalty reward has little influence on the navigation performance.

Original: However,it helps to reduce the frequency of actuating camera motor described in Section 3.4.
Revised: However,it helps to reduce the frequency of actuating camera motor described in Section 3.4.

Original: Experimentalresults show that without this reward, agents actuate the camera motor for 12.41% of total time steps.
Revised: Experimentalresults show that without this reward, agents actuate the camera motor for 12.41% of total time steps.

Original: Adding turn-penalty reward decreases the number to 5.70%.
Revised: Adding turn-penalty reward decreases the number to 5.70%.

Original: Does camera policy work with imperfect mapper?
Revised: Does camera policy work with imperfect mapper?

Original: Our EXPO camera policy obtains indoorlayout information mainly from the progressively built map.
Revised: Our EXPO camera policy obtains indoorlayout information mainly from the progressively built map.

Original: In this subsection, we would like toevaluate whether the proposed camera policy works with noisy map.
Revised: In this subsection, we would like toevaluate whether the proposed camera policy works with noisy map.

Original: To this end, we use a learnedneural network [52] to predict the occupancy map from RGB-D images.
Revised: To this end, we use a learnedneural network [52] to predict the occupancy map from RGB-D images.

Original: The predicted map containsmany false positive points ( e.g., predicting the free space as occupancy) and false negative points( e.g., predicting a wall as free space).
Revised: The predicted map containsmany false positive points ( e.g., predicting the free space as occupancy) and false negative points( e.g., predicting a wall as free space).

Original: These noisy points may mislead the camera policy to makea wrong camera action.
Revised: These noisy points may mislead the camera policy to makea wrong camera action.

Original: Experimental results on OccAnt baseline show that incorporating camerapolicy brings consistent improvement.
Revised: Experimental results on OccAnt baseline show that incorporating camerapolicy brings consistent improvement.

Original: Specifically, SPL and PPL increase from 13.6 to 18.1 andfrom 20.5 to 25.4, respectively.
Revised: Specifically, SPL and PPL increase from 13.6 to 18.1 andfrom 20.5 to 25.4, respectively.

Original: Success and Progress increase from 18.39 to 24.0 and from 28.4 to34.2, respectively.
Revised: Success and Progress increase from 18.39 to 24.0 and from 28.4 to

Original: These results demonstrate that the proposed EXPO camera policy is robust to themap noise and can be deployed in a robot with no need of a ground-truth map.

Original: Does improvement come from extra information?

Original: Compared with baselines, our active-cameraagent leverages extra information ( i.e., occupancy map, heuristic direction and exploration reward)

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: t
Revised: t

Original: [Figure]
Revised: [Figure]

Original: t= t= t= for determining camera actions.
Revised: t= t= t= 34.2, respectively.

Revised: These results demonstrate that the proposed EXPO camera policy is robust to themap noise and can be deployed in a robot with no need of a ground-truth map.

Revised: Does improvement come from extra information?

Revised: Compared with baselines, our active-cameraagent leverages extra information ( i.e., occupancy map, heuristic direction and exploration reward)for determining camera actions.

Original: For a fair comparison, we add this extra information into MultiONto build an enhanced baseline.
Revised: For a fair comparison, we add this extra information into MultiONto build an enhanced baseline.

Original: Results on MatterPort3D dataset demonstrate that the enhancedbaseline performs slightly better than MultiON baseline but worse than our active-camera agent.
Revised: Results on MatterPort3D dataset demonstrate that the enhancedbaseline performs slightly better than MultiON baseline but worse than our active-camera agent.

Original: Specifically, SPL, PPL, Success and Progress are 35.1, 45.9, 48.0, and 63.2, respectively.
Revised: Specifically, SPL, PPL, Success and Progress are 35.1, 45.9, 48.0, and 63.2, respectively.

Original: We suspectthe improvement of the enhanced baseline comes from the fact that heuristic direction provideslocation information about unexplored areas.
Revised: We suspectthe improvement of the enhanced baseline comes from the fact that heuristic direction provideslocation information about unexplored areas.

Original: Also, the extra rewards encourage agents to explorethese areas.
Revised: Also, the extra rewards encourage agents to explorethese areas.

Original: However, due to the limited action space, the agent in enhanced baseline can notcoordinate their camera and navigation actions well, which limits the performance.
Revised: However, due to the limited action space, the agent in enhanced baseline can notcoordinate their camera and navigation actions well, which limits the performance.

Original: Exploration performance.
Revised: Exploration performance.

Original: One of critical abilities for the multi-object navigation task is exploringthe environment efficiently to locate all goal objects.
Revised: One of critical abilities for the multi-object navigation task is exploringthe environment efficiently to locate all goal objects.

Original: To evaluate the exploration ability, we placethe agent in a novel environment with three goal objects located in different places.
Revised: To evaluate the exploration ability, we placethe agent in a novel environment with three goal objects located in different places.

Original: We follow FBEmethod [66] to explore the environment.
Revised: We follow FBEmethod [66] to explore the environment.

Original: Given a limited time step budget, we evaluate successrate of finding all three goal objects and explored area in Figures 3.
Revised: Given a limited time step budget, we evaluate successrate of finding all three goal objects and explored area in Figures 3.

Original: Compared with the baselinesthat the agent is always looking forward or moving its camera randomly, the agent with our EXPOcamera policy finds more goal objects and explores more areas.
Revised: Compared with the baselinesthat the agent is always looking forward or moving its camera randomly, the agent with our EXPOcamera policy finds more goal objects and explores more areas.

Original: These results have the same trend ofnavigation performance in Table 1, suggesting that the learned EXPO camera policy helps to explorethe environment more efficiently and consequently boosts the multi-object navigation performance.
Revised: These results have the same trend ofnavigation performance in Table 1, suggesting that the learned EXPO camera policy helps to explorethe environment more efficiently and consequently boosts the multi-object navigation performance.

Original: Visualization.
Revised: Visualization.

Original: In Figure 4, both baseline ( i.e., OccAnt) and our agent are navigating to Goal-1 atthe beginning.
Revised: In Figure 4, both baseline ( i.e., OccAnt) and our agent are navigating to Goal-1 atthe beginning.

Original: During this process, our agent moves its camera actively at time step t “ 16 andt “ 26 , finding Goal-2 and Goal-3 respectively.
Revised: During this process, our agent moves its camera actively at time step t “ 16 andt “ 26 , finding Goal-2 and Goal-3 respectively.

Original: Knowing the location of goal objects, our agentplans a shortest path for navigation.
Revised: Knowing the location of goal objects, our agentplans a shortest path for navigation.

Original: In contrast, the baseline agent goes straight to Goal-1 , with thecamera looking forward constantly.
Revised: In contrast, the baseline agent goes straight to Goal-1 , with thecamera looking forward constantly.

Original: Consequently, after it navigates to Goal-1 , it cannot find othergoal objects and has to waste time exploring the environment again.
Revised: Consequently, after it navigates to Goal-1 , it cannot find othergoal objects and has to waste time exploring the environment again.

Original: Failure case analysis can befound in the supplementary material.
Revised: Failure case analysis can befound in the supplementary material.

Original:  Discussion
Revised:  Discussion

Original: Limitations and future work.
Revised: Limitations and future work.

Original: Although our active camera agent shows promising performancein photo-realistic simulated data, it has not been thoroughly evaluated in real world, which is aninteresting future work.
Revised: Although our active camera agent shows promising performancein photo-realistic simulated data, it has not been thoroughly evaluated in real world, which is aninteresting future work.

Original: Before thoughtful evaluation, directing adapting it to real environmentsmay cause accidents such as breaking somethings and hitting pedestrians.
Revised: Before through evaluation, directing adapting it to real environments maycause accidents such as breaking somethings and hitting pedestrian.

Original: Besides, even thoughwe have evaluated the transferability of camera policy among different datasets (transferring from

Original: MatterPort3D to Gibson) in our paper, evaluating the robustness of the camera policy among differentrobot types and different embodied tasks is also a valuable research direction.

Original: Conclusion.
Revised: Conclusion.

Original: In order to solve the uncoordinated camera-navigation actions problem of existingagents, we propose a navigation paradigm in which agents can dynamically move their camera forperceiving environments more efficiently.
Revised: In order to solve the uncoordinated camera-navigation actions problem of existingagents, we propose a navigation paradigm in which agents can dynamically move their camera forperceiving environments more efficiently.

Original: Such exploration ability is important for multi-objectnavigation.
Revised: Such exploration ability is important for multi-objectnavigation.

Original: To determine the camera actions, we learn a camera policy via reinforcement learningby awarding it to explore more areas.
Revised: To determine the camera actions, we learn a camera policy via reinforcement learningby awarding it to explore more areas.

Original: Also, we use heuristic rules to guide the learning process andreduce the learning difficulty.
Revised: Also, we use heuristic rules to guide the learning process andreduce the learning difficulty.

Original: The proposed camera policy can be incorporated into most existingnavigation methods.
Revised: The proposed camera policy can be incorporated into most existingnavigation methods.

Original: Experimental results show that our camera policy consistently improvesthemulti-object navigation performance of multiple existing methods on two benchmark datasets.
Revised: Experimental results show that our camera policy consistently improves

Original: Checklist
Revised: Checklist

Revised: [Table]

Original: For all authors...
Revised: For all authors...

Revised: [Table]

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] Please see Section
Revised: [Yes] Please see Section

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] Please seeSection 5(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [Yes] Please seeSection 5(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]
Revised: [Yes]2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [N/A]3.
Revised: [N/A]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?

Original: [No] We will publish
Revised: [No] We will publish

Original: our code upon acceptance.
Revised: our code upon acceptance.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes] We have included the training details in the supplemental material.(c)
Revised: [Yes] We have included the training details in the supplemental material.

Original:  Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We have run evaluation using 5 seeds and reported errorbars.
Revised: [Yes] We have run evaluation using 5 seeds and reported errorbars.(d)

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] We have included the details of the computational resources in the supplemental material.
Revised: [Yes] We have included the details of thecomputational resources in the supplemental material.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] We have added the license in the supplemental material.
Revised: [Yes] We have added the license in thesupplemental material.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No]
Revised: [No]

Revised: [Table]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: [Table]

Original: using/curating?
Revised: using/curating?

Original: [No]
Revised: [No]

Revised: [Table]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: [Table]

Original: information or offensive content?
Revised: information or offensive content?

Original: [No]
Revised: [No]

Revised: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original:  (a) Did you include the full text of instructions given to participants and screenshots, if 
Revised: [Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Revised: [Table]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: [Table]

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Revised: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: [Table]

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Revised: [Table]