Original: Learning Active Camera for Multi-Object Navigation
Revised: Learning Active Camera for Multi-Object Navigation

Original: Abstract
Revised: Abstract

Original: Getting robots to navigate to multiple objects autonomously is essential yet difficult in robot applications.
Revised: Getting robots to navigate to multiple objects autonomously is essential yet difficult in robot applications.

Original: One of the key challenges is how to explore environments efficiently with camera sensors only.
Revised: One of the key challenges is how to explore environments efficiently with camera sensors only.

Original: Existing navigation methods mainly focus on fixed cameras and few attempts have been made to navigate with active cameras.
Revised: Existing navigation methods mainly focus on fixed cameras and few attempts have been made to navigate with active cameras.

Original: As a result, the agent may take a very long time to perceive the environment due to limited camera scope.
Revised: As a result, the agent may take a very long time to perceive the environment due to limited camera scope.

Original: In contrast, humans typically gain a larger field of view by looking around for a better perception of the environment.
Revised: In contrast, humans typically gain a larger field of view by looking around for a better perception of the environment.

Original: How to make robots perceive the environment as efficiently as humans is a fundamental problem in robotics.
Revised: How to make robots perceive the environment as efficiently as humans is a fundamental problem in robotics.

Original: In this paper, we consider navigating to multiple objects more efficiently with active cameras.
Revised: In this paper, we consider navigating to multiple objects more efficiently with active cameras.

Original: Specifically, we cast moving camera to a Markov
Revised: Specifically, we cast moving camera to a Markov

Original: Decision Process and reformulate the active camera problem as a reinforcement learning problem.
Revised: Decision Process and reformulate the active camera problem as a reinforcement learning problem.

Original: However, we have to address two new challenges: 1) how to learn a good camera policy in complex environments and 2) how to coordinate it with the navigation policy.
Revised: However, we have to address two new challenges: 1) how to learn a good camera policy in complex environments and 2) how to coordinate it with the navigation policy.

Original: To address these, we carefully design a reward function to encourage the agent to explore more areas by moving camera actively.
Revised: To address these, we carefully design a reward function to encourage the agent to explore more areas by moving camera actively.

Original: Moreover, we exploit human experience to infer a rule-based camera action to guide the learning process.
Revised: Moreover, we exploit human experience to infer a rule-based camera action to guide the learning process.

Original: Last, to better coordinate two kinds of policies, the camera policy takes navigation actions into account when making camera moving decisions.
Revised: Last, to better coordinate two kinds of policies, the camera policy takes navigation actions into account when making camera moving decisions.

Original: Experimental results show our camera policy consistently improves the performance of multi-object navigation over four baselines on two datasets.
Revised: Experimental results show our camera policy consistently improves the performance of multi-object navigation over four baselines on two datasets.

Original:  Introduction
Revised:  Introduction

Original: In the multi-object navigation task, an intelligent embodied agent needs to navigate to multiple goal objects in a 3D environment.
Revised: In the multi-object navigation task, an intelligent embodied agent needs to navigate to multiple goal objects in a 3D environment.

Original: Typically, no pre-computed map is available and the agent needs to use a stream of egocentric observations to perceive the environment.
Revised: Typically, no pre-computed map is available and the agent needs to use a stream of egocentric observations to perceive the environment.

Original: This navigation ability is the basis for indoor robots and embodied AI.
Revised: This navigation ability is the basis for indoor robots and embodied AI.

Original: Significant recent progress on this problem can be attributed to the availability of large-scale visually rich 3D datasets [72, 7, 67, 70, 32, 29], developments in high-quality 3D simulators [72, 62, 47, 2, 30], and research on deep memorybased architectures that combine geometry and semantics for learning representations of the 3D environment [8, 69, 59, 19, 31].
Revised: Significant recent progress on this problem can be attributed to the availability of large-scale visually rich 3D datasets [71, 7, 66, 69, 32, 29], developments in high-quality 3D simulators [71, 61, 46, 2, 30], and research on deep memorybased architectures that combine geometry and semantics for learning representations of the 3D environment [8, 68, 58, 19, 31].

Original: Despite these advances, how to efficiently perceive the environment and locate goal objects is still an unsolved problem.
Revised: Despite these advances, how to efficiently perceive the environment and locate goal objects is still an unsolved problem.

Original: Agents in current research [9, 12, 50] perceive the environment via an RGB-D camera.
Revised: Agents in current research [9, 12, 49] perceive the environment via an RGB-D camera.

Original: However, the camera is set to look forward and the range of its view is often limited.
Revised: However, the camera is set to look forward and the range of its view is often limited.

Original: As a
Revised: As a

Original: [Figure]
Revised: [Figure]

Original: result, these agents can only perceive the environment in front of themselves within or slightly beyond the view range [59].
Revised: result, these agents can only perceive the environment in front of themselves within or slightly beyond the view range [58].

Original: As shown at the bottom-right in Figure 1, the agent keeps looking forward and perceives redundant information which is nearly identical to the previous observations.
Revised: As shown at the bottom-right in Figure 1, the agent keeps looking forward and perceives redundant information which is nearly identical to the previous observations.

Original: In contrast, we human beings may be attracted by surrounding things so that we turn our heads actively to receive more information when keeping walking straight.
Revised: In contrast, we human beings may be attracted by surrounding things so that we turn our heads actively to receive more information when keeping walking straight.

Original: In this way, we can not only reach the position in front of ourselves efficiently but also get familiar with room arrangement and object location.
Revised: In this way, we can not only reach the position in front of ourselves efficiently but also get familiar with room arrangement and object location.

Original: Motivated by the above observations, we design an agent which seeks to coordinate both camera and navigation actions.
Revised: Motivated by the above observations, we design an agent which seeks to coordinate both camera and navigation actions.

Original: The agent, called active-camera agent , employs two kinds of policies, namely the navigation policy which determines where to go and a new camera policy which determines where to look.
Revised: The agent, called active-camera agent , employs two kinds of policies, namely the navigation policy which determines where to go and a new camera policy which determines where to look.

Original: As shown at the top-right in Figure 1, the agent has found an object, i.e., blue Goal-1 , located at the end of a corridor.
Revised: As shown at the top-right in Figure 1, the agent has found an object, i.e., blue Goal-1 , located at the end of a corridor.

Original: Two kinds of policies cooperate and actuate the agent to move forward and turn its camera to the left.
Revised: Two kinds of policies cooperate and actuate the agent to move forward and turn its camera to the left.

Original: As a result, the agent walks closer to the Goal-1 and locates another goal, i.e., yellow Goal-2 , for the following navigation process.
Revised: As a result, the agent walks closer to the Goal-1 and locates another goal, i.e., yellow Goal-2 , for the following navigation process.

Original: However, it is nontrivial to learn a good camera policy because of sophisticated RGB-D observations and complex indoor layouts.
Revised: However, it is nontrivial to learn a good camera policy because of sophisticated RGB-D observations and complex indoor layouts.

Original: How to extract useful information from observations to judge which direction is worth being explored is difficult.
Revised: How to extract useful information from observations to judge which direction is worth being explored is difficult.

Original: Besides, the agent and the camera are moving simultaneously.
Revised: Besides, the agent and the camera are moving simultaneously.

Original: How to coordinate these two actions for both the navigation and exploration processes is still unknown.
Revised: How to coordinate these two actions for both the navigation and exploration processes is still unknown.

Original: In this paper, we propose an EXPloration-Oriented (EXPO) camera policy to determine camera actions.
Revised: In this paper, we propose an EXPloration-Oriented (EXPO) camera policy to determine camera actions.

Original: Specifically, to better understand the sophisticated RGB-D observations, we transform them to a top-down occupancy map.
Revised: Specifically, to better understand the sophisticated RGB-D observations, we transform them to a top-down occupancy map.

Original: Each pixel indicates whether it is an unexplored, free or occupied area.
Revised: Each pixel indicates whether it is an unexplored, free or occupied area.

Original: Such a map helps to simplify unstructured RGB-D observations and provides the necessary information, e.g., explored areas and room layouts, for determining camera actions.
Revised: Such a map helps to simplify unstructured RGB-D observations and provides the necessary information, e.g., explored areas and room layouts, for determining camera actions.

Original: Besides, to reduce the learning difficulty, we propose a heuristic module to infer an expected camera direction according to heuristic rules.
Revised: Besides, to reduce the learning difficulty, we propose a heuristic module to infer an expected camera direction according to heuristic rules.

Original: We consider the heuristic module an expert and exploit it to guide the learning process.
Revised: We consider the heuristic module an expert and exploit it to guide the learning process.

Original: We then feed three types of information, i.e., the progressively built map, the expected camera direction, and the upcoming navigation action, into a neural network to predict a camera action.
Revised: We then feed three types of information, i.e., the progressively built map, the expected camera direction, and the upcoming navigation action, into a neural network to predict a camera action.

Original: The neural network considers both the heuristic rules and the navigation intention.
Revised: The neural network considers both the heuristic rules and the navigation intention.

Original: We use a reinforcement learning algorithm to train the neural network by awarding camera actions that maximize the exploration area.
Revised: We use a reinforcement learning algorithm to train the neural network by awarding camera actions that maximize the exploration area.

Original: We incorporate the EXPO camera policy with existing navigation policies [59, 69] and train them in an end-to-end manner for the multi-object navigation task.
Revised: We incorporate the EXPO camera policy with existing navigation policies [58, 68] and train them in an end-to-end manner for the multi-object navigation task.

Original: Extensive experiments on two benchmarks demonstrate the effectiveness of the proposed methods.
Revised: Extensive experiments on two benchmarks demonstrate the effectiveness of the proposed methods.

Original: Our main contributions are threefold: 1) Unlike existing agents that are set to look forward, we propose a navigation paradigm that an agent coordinates camera and navigation actions for efficiently perceiving environments to solve the multi-object navigation task.
Revised: Our main contributions are threefold: 1) Unlike existing agents that are set to look forward, we propose a navigation paradigm that an agent coordinates camera and navigation actions for efficiently perceiving environments to solve the multi-object navigation task.

Original: We propose to learn an EXPO camera policy to determine camera actions.
Revised: We propose to learn an EXPO camera policy to determine camera actions.

Original: The camera policy leverages heuristic rules to reduce the learning difficulty and takes into account the navigation intention to coordinate with the navigation policy.
Revised: The camera policy leverages heuristic rules to reduce the learning difficulty and takes into account the navigation intention to coordinate with the navigation policy.

Original: Such a camera policy can be incorporated with most existing navigation methods.
Revised: Such a camera policy can be incorporated with most existing navigation methods.

Original: 3) Extensive experiments demonstrate consistent improvement over four navigation methods on MatterPort3D [7] and Gibson [72] datasets.
Revised: 3) Extensive experiments demonstrate consistent improvement over four navigation methods on MatterPort3D [7] and Gibson [71] datasets.

Original: More critically, the camera policy also exhibits promising transferability to unseen scenes.
Revised: More critically, the camera policy also exhibits promising transferability to unseen scenes.

Original:  Related Work
Revised:  Related Work

Original: Visual indoor navigation.
Revised: Visual indoor navigation.

Original: According to the types of goal, visual navigation tasks can be categorized into different classes such as PointGoal [33, 9] task where the goal is a given coordinate,
Revised: According to the types of goal, visual navigation tasks can be categorized into different classes such as PointGoal [33, 9] task where the goal is a given coordinate,

Original: ObjectGoal [33, 69] task where the agent needs to navigate to an object given by language and
Revised: ObjectGoal [33, 68] task where the agent needs to navigate to an object given by language and

Original: ImageGoal [12, 75] task where the object is specified by an image.
Revised: ImageGoal [12, 74] task where the object is specified by an image.

Original: The location of the goal is not explicitly known in the last two categories of the above tasks and the agent is expected to explore the environment to find the goals.
Revised: The location of the goal is not explicitly known in the last two categories of the above tasks and the agent is expected to explore the environment to find the goals.

Original: Classical approaches [28, 68, 34, 64, 46] solve the navigation problem via path planning [11, 34] on a constructed map, which usually requires handcraft design.
Revised: Classical approaches [28, 67, 34, 63, 45] solve the navigation problem via path planning [11, 34] on a constructed map, which usually requires handcraft design.

Original: Recent works [75, 54, 74, 66, 69, 15, 65, 51, 57, 39] aim to use learning-based methods in an end-to-end manner to learn a policy for navigation.
Revised: Recent works [74, 53, 73, 65, 68, 15, 64, 50, 56, 38] aim to use learning-based methods in an end-to-end manner to learn a policy for navigation.

Original: Other works [9, 12, 59, 44, 48, 5, 49, 18, 53, 17, 27, 61, 23] combine the classic and learning-based methods, which use a learned SLAM module with a spatial map or topological map.
Revised: Other works [9, 12, 58, 43, 47, 5, 48, 18, 52, 17, 27, 60, 23] combine the classic and learning-based methods, which use a learned SLAM module with a spatial map or topological map.

Original: We propose a camera policy, which seeks to move the camera actively and can be incorporated with existing navigation methods, to improve the navigation performance.
Revised: We propose a camera policy, which seeks to move the camera actively and can be incorporated with existing navigation methods, to improve the navigation performance.

Original: Exploration for navigation.
Revised: Exploration for navigation.

Original: Common methods explore the environment based on heuristics like the frontier-based exploration algorithm [73, 35, 24], which chooses a frontier point between explored and unexplored areas as the exploration goal.
Revised: Common methods explore the environment based on heuristics like the frontier-based exploration algorithm [72, 35, 24], which chooses a frontier point between explored and unexplored areas as the exploration goal.

Original: Recent works tackle the exploration problem via learning [9, 21, 6, 25, 22, 58, 36, 45, 55], which allows the agent to incorporate knowledge from previously seen environments and generalize to novel environments.
Revised: Recent works tackle the exploration problem via learning [9, 21, 6, 25, 22, 57, 36, 44, 54], which allows the agent to incorporate knowledge from previously seen environments and generalize to novel environments.

Original: Specially, Chen et al.
Revised: Specially, Chen et al.

Original: [21] and Jayaraman et al.
Revised: [21] and Jayaraman et al.

Original: [41] use end-to-end reinforcement learning policy to maximize an external reward ( i.e., exploration area).
Revised: [40] use end-to-end reinforcement learning policy to maximize an external reward ( i.e., exploration area).

Original: Burda et al.
Revised: Burda et al.

Original: [6] and Dean et al.
Revised: [6] and Dean et al.

Original: [22] consider intrinsic rewards such as curiosity for efficient exploration, which performs better when external rewards are sparse.
Revised: [22] consider intrinsic rewards such as curiosity for efficient exploration, which performs better when external rewards are sparse.

Original: Chaplot et al.
Revised: Chaplot et al.

Original: [9], Ramakrishnan [59] and Chen et al.
Revised: [9], Ramakrishnan [58] and Chen et al.

Original: [16] infer an exploration goal by a learned policy and navigate to it using path planner, which avoids sample complexity problem in end-to-end training.
Revised: [16] infer an exploration goal by a learned policy and navigate to it using path planner, which avoids sample complexity problem in end-to-end training.

Original: Elhafsi et al.
Revised: Elhafsi et al.

Original: [26] predict the map out of view range for better path planning.
Revised: [26] predict the map out of view range for better path planning.

Original: Unlike existing methods, we try to actively control the camera direction for efficient exploration.
Revised: Unlike existing methods, we try to actively control the camera direction for efficient exploration.

Original: Active perception.
Revised: Active perception.

Original: Active perception [3] aims to change the state parameters of sensors according to intelligent control strategies and gain more information about the environment.
Revised: Active perception [3] aims to change the state parameters of sensors according to intelligent control strategies and gain more information about the environment.

Original: Common methods guide the view selection using information-theoretic approaches, such as mutual information [43, 13, 14, 38].
Revised: Common methods guide the view selection using information-theoretic approaches, such as mutual information [42, 13, 14, 37].

Original: Recent work applies this strategy on different tasks such as object recognition [42, 1, 40], object localization [10, 75, 52, 56] and scene completion [41, 60].
Revised: Recent work applies this strategy on different tasks such as object recognition [41, 1, 39], object localization [10, 74, 51, 55] and scene completion [40, 59].

Original: We refer readers to [20] and [4] for a detailed review.
Revised: We refer readers to [20] and [4] for a detailed review.

Original: In this paper, we propose a camera control strategy for active camera moving to help multi-object navigation.
Revised: In this paper, we propose a camera control strategy for active camera moving to help multi-object navigation.

Original:  Multi-object Navigation with Active Camera 3.1 Problem Formulation
Revised:  Multi-object Navigation with Active Camera 3.1 Problem Formulation

Original: Considering an agent equipped with one RGB-D camera in a novel environment, the multi-object navigation task asks the agent to explore the environment to find multiple goal objects and then navigate to them.
Revised: Considering an agent equipped with one RGB-D camera in a novel environment, the multi-object navigation task asks the agent to explore the environment to find multiple goal objects and then navigate to them.

Original: During these processes, existing methods [69] design a navigation policy to process the observations o from the environment and infer a navigation action a n ( i.e., FORWARD ,
Revised: During these processes, existing methods [68] design a navigation policy to process the observations o from the environment and infer a navigation action a n ( i.e., FORWARD ,

Original: TURN - LEFT , TURN - RIGHT , and FOUND ).
Revised: TURN - LEFT , TURN - RIGHT , and FOUND ).

Original: The navigation action is responsible for moving toward goal objects and indicates whether agents have found goals.
Revised: The navigation action is responsible for moving toward goal objects and indicates whether agents have found goals.

Original: However, these methods do not consider moving the camera direction actively during the navigation process.
Revised: However, these methods do not consider moving the camera direction actively during the navigation process.

Original: Thus, agents can only perceive the field along their navigation direction, which causes low efficiency in finding goal objects [59].
Revised: Thus, agents can only perceive the field along their navigation direction, which causes low efficiency in finding goal objects [58].

Original: To resolve the above problem, we propose a new navigation paradigm that consists of a navigation policy π n p¨q and a camera policy π c p¨q .
Revised: To resolve the above problem, we propose a new navigation paradigm that consists of a navigation policy π n p¨q and a camera policy π c p¨q .

Original: We reformulate both the navigation and camera moving process to a Partially Observable Markov Decision Process, where the observation o received by the agent does not fully specify the state of the environment.
Revised: We reformulate both the navigation and camera moving process to a Partially Observable Markov Decision Process, where the observation o received by the agent does not fully specify the state of the environment.

Original: At each time step, for a given state estimated from o , these two policies predict a navigation action a n „ π n p¨q together with a camera action a c „ π c p¨q for active camera moving.
Revised: At each time step, for a given state estimated from o , these two policies predict a navigation action a n „ π n p¨q together with a camera action a c „ π c p¨q for active camera moving.

Original: The action space A for a n is similar to the navigation action space.
Revised: The action space A for a n is similar to the navigation action space.

Original: The possible camera action includes { TURN - CAMERA
Revised: The possible camera action includes { TURN - CAMERA

Original: , TURN
Revised: , TURN

Original: CAMERA
Revised: CAMERA

Original: RIGHT , and KEEP }, where KEEP indicates the camera direction remains unchanged.
Revised: RIGHT , and KEEP }, where KEEP indicates the camera direction remains unchanged.

Original: After performing the action, the policies will receive a reward r whose details can be found in Section 3.3.
Revised: After performing the action, the policies will receive a reward r whose details can be found in Section 3.3.

Original: We call the agent using this paradigm active-camera agent and the general scheme is shown in Figure 2.
Revised: We call the agent using this paradigm active-camera agent and the general scheme is shown in Figure 2.

Original: [Figure]
Revised: [Figure]

Original: 3.2 Learning to Determine Camera Action
Revised: 3.2 Learning to Determine Camera Action

Original: Learning a camera policy for determining camera actions is challenging because 1) it is difficult to understand complex room structure from RGB-D observations; 2) the position of agents is changing all the time so camera policies must coordinate with navigation policies.
Revised: Learning a camera policy for determining camera actions is challenging because 1) it is difficult to understand complex room structure from RGB-D observations; 2) the position of agents is changing all the time so camera policies must coordinate with navigation policies.

Original: In this paper, we design an EXPO camera policy that consists of three components, i.e., a mapper, a heuristic module, and a policy network.
Revised: In this paper, we design an EXPO camera policy that consists of three components, i.e., a mapper, a heuristic module, and a policy network.

Original: The mapper transforms the RGB-D image to an occupancy top-down map, which reveals the room layout and location of unexplored areas straightforwardly.
Revised: The mapper transforms the RGB-D image to an occupancy top-down map, which reveals the room layout and location of unexplored areas straightforwardly.

Original: The heuristic module infers a heuristic direction that is worth exploring.
Revised: The heuristic module infers a heuristic direction that is worth exploring.

Original: The inferred direction can be served as a reference for camera policy and reduce the learning difficulty.
Revised: The inferred direction can be served as a reference for camera policy and reduce the learning difficulty.

Original: Then, a policy network predicts camera actions considering the encoded features of the map, heuristic direction, and upcoming navigation action.
Revised: Then, a policy network predicts camera actions considering the encoded features of the map, heuristic direction, and upcoming navigation action.

Original: The upcoming navigation action informs the policy network of the next location of agents.
Revised: The upcoming navigation action informs the policy network of the next location of agents.

Original: The overview of the camera policy is shown in Figure 2.
Revised: The overview of the camera policy is shown in Figure 2.

Original: Next, we introduce each component in detail.
Revised: Next, we introduce each component in detail.

Original: Mapper.
Revised: Mapper.

Original: We follow existing work [59] to build an occupancy map by a mapper.
Revised: We follow existing work [58] to build an occupancy map by a mapper.

Original: Each map pixel has three possible values, i.e., 0 , 1 , 2 , indicating an unexplored, free or occupied area, respectively.
Revised: Each map pixel has three possible values, i.e., 0 , 1 , 2 , indicating an unexplored, free or occupied area, respectively.

Original: Specifically, the mapper takes RGB-D as input and predicts a local map M l .
Revised: Specifically, the mapper takes RGB-D as input and predicts a local map M l .

Original: This local map represents a small area in front of the camera.
Revised: This local map represents a small area in front of the camera.

Original: Then we integrate the local map into a global map M g via a registration function R p¨q , i.e., M g “ R p M l , M g , q q , where q is the camera’s pose representing its position and orientation.
Revised: Then we integrate the local map into a global map M g via a registration function R p¨q , i.e., M g “ R p M l , M g , q q , where q is the camera’s pose representing its position and orientation.

Original: The global map covers the whole explored environment.
Revised: The global map covers the whole explored environment.

Original: Heuristic module.
Revised: Heuristic module.

Original: We seek to find a heuristic direction indicating which area is worth being explored by using heuristic rules.
Revised: We seek to find a heuristic direction indicating which area is worth being explored by using heuristic rules.

Original: The principle is to find a direction where the camera sight is not blocked by occupancy ( e.g., wall).
Revised: The principle is to find a direction where the camera sight is not blocked by occupancy ( e.g., wall).

Original: Following this principle, we draw K points P “ t p i u
Revised: Following this principle, we draw K points P “ t p i u

Original: Ki “ 1 uniformly around the agent.
Revised: Ki “ 1 uniformly around the agent.

Original: Each point is located at angle θ i from the current navigation direction and r meters away from the agent.
Revised: Each point is located at angle θ i from the current navigation direction and r meters away from the agent.

Original: We calculate geodesic distance D
Revised: We calculate geodesic distance D

Original: “ t D p p
Revised: “ t D p p

Original: i qu Ki “ 1 from agent to these points on the progressively built global map M g via A* path planning algorithm D p¨q [34].
Revised: i qu Ki “ 1 from agent to these points on the progressively built global map M g via A* path planning algorithm D p¨q [34].

Original: During path planning, we consider both the free and unexplored areas navigable.
Revised: During path planning, we consider both the free and unexplored areas navigable.

Original: If the geodesic distance d i is smaller than a threshold γ p γ ą r q and the point p i locates at unexplored areas in the global map, we denote these points explorable because there is no obstacle locating between the agent and point p i .
Revised: If the geodesic distance d i is smaller than a threshold γ p γ ą r q and the point p i locates at unexplored areas in the global map, we denote these points explorable because there is no obstacle locating between the agent and point p i .

Original: Moving the camera to these explorable directions allows the agent to see unexplored parts of the environment.
Revised: Moving the camera to these explorable directions allows the agent to see unexplored parts of the environment.

Original: If there exist such explorable points, we select one point closest to the camera direction and take its angle θ i as the heuristic direction θ ˚ .
Revised: If there exist such explorable points, we select one point closest to the camera direction and take its angle θ i as the heuristic direction θ ˚ .

Original: Otherwise, we set θ ˚ “ 0 , i.e., facing the navigation direction.
Revised: Otherwise, we set θ ˚ “ 0 , i.e., facing the navigation direction.

Original: Policy network.
Revised: Policy network.

Original: With the progressively built map and inferred heuristic direction, we use a recurrent network to aggregate this information at each time step.
Revised: With the progressively built map and inferred heuristic direction, we use a recurrent network to aggregate this information at each time step.

Original: The network takes as input map features, heuristic direction features, and navigation action features.
Revised: The network takes as input map features, heuristic direction features, and navigation action features.

Original: Specifically, a convolutional neural network is exploited to extract map features from an egocentric map which is cropped from the global map.
Revised: Specifically, a convolutional neural network is exploited to extract map features from an egocentric map which is cropped from the global map.

Original: The other two types of features are encoded from the heuristic direction θ ˚ and navigation action a n , respectively by using a learned embedding layer.
Revised: The other two types of features are encoded from the heuristic direction θ ˚ and navigation action a n , respectively by using a learned embedding layer.

Original: The output state features from the recurrent network are fed into an actor-critic network.
Revised: The output state features from the recurrent network are fed into an actor-critic network.

Original: The actor predicts a camera action and the critic predicts the value of the agent’s current state.
Revised: The actor predicts a camera action and the critic predicts the value of the agent’s current state.

Original: p M
Revised: p M

Original: M g q q pose
Revised: M g q q pose

Original: 3.3 Reward Function for Camera Policy
Revised: 3.3 Reward Function for Camera Policy

Original: We expect the camera policy to help agents explore environments more efficiently.
Revised: We expect the camera policy to help agents explore environments more efficiently.

Original: We achieve this objective in two ways.
Revised: We achieve this objective in two ways.

Original: On the one hand, we encourage the camera policy to follow the heuristic rules by awarding actions that move cameras toward the heuristic direction.
Revised: On the one hand, we encourage the camera policy to follow the heuristic rules by awarding actions that move cameras toward the heuristic direction.

Original: This heuristic reward is defined as the reduction of s in successive time steps, i.e., r heuristic “ s t
Revised: This heuristic reward is defined as the reduction of s in successive time steps, i.e., r heuristic “ s t

Original: ´ 1 ´ s t .
Revised: ´ 1 ´ s t .

Original: The s P r 0 ˝ , 180 ˝ s represents the angle between the camera direction and the heuristic direction.
Revised: The s P r 0 ˝ , 180 ˝ s represents the angle between the camera direction and the heuristic direction.

Original: On the other hand, because it is impractical to design a heuristic rule to cover all situations, we explicitly encourage the camera policy to explore more areas by an area reward r area “ c t ´ c t ´ 1 , which indicates the increase of explored area c in successive time steps.
Revised: On the other hand, because it is impractical to design a heuristic rule to cover all situations, we explicitly encourage the camera policy to explore more areas by an area reward r area “ c t ´ c t ´ 1 , which indicates the increase of explored area c in successive time steps.

Original: The camera agent makes its own decision according to recommended direction and occupancy map information.
Revised: The camera agent makes its own decision according to recommended direction and occupancy map information.

Original: In addition, to avoid constant camera moving and reduce energy consumption, we expect agents to execute camera motor for camera moving as less frequently as possible by introducing a turn-penalty reward r turn-penalty “ 1 r turn-camera s , where 1 r turn-camera s equals to 1 when the agent actuates camera motor to move a camera and otherwise equals to 0.
Revised: In addition, to avoid constant camera moving and reduce energy consumption, we expect agents to execute camera motor for camera moving as less frequently as possible by introducing a turn-penalty reward r turn-penalty “ 1 r turn-camera s , where 1 r turn-camera s equals to 1 when the agent actuates camera motor to move a camera and otherwise equals to 0.

Original: The detail about camera motor execution will be described in Section 3.4.
Revised: The detail about camera motor execution will be described in Section 3.4.

Original: To sum up, the reward for camera policy is as follows:
Revised: To sum up, the reward for camera policy is as follows:

Original: [Equation]
Revised: [Equation]

Original: where α and β are two scaling hyper-parameters.
Revised: where α and β are two scaling hyper-parameters.

Original: We train the camera policy to maximize such exploration reward using PPO [63].
Revised: We train the camera policy to maximize such exploration reward using PPO [62].

Original: In this way, the EXPO camera policy is encouraged to take into account heuristic rule-based decisions and to explore more areas by executing the camera motor as less frequently as possible.
Revised: In this way, the EXPO camera policy is encouraged to take into account heuristic rule-based decisions and to explore more areas by executing the camera motor as less frequently as possible.

Original: 3.4 Combination of Navigation and Camera Policies
Revised: 3.4 Combination of Navigation and Camera Policies

Original: Navigation and camera actions execution.
Revised: Navigation and camera actions execution.

Original: We assume the camera is attached to a mobile robot platform using a revolute joint.
Revised: We assume the camera is attached to a mobile robot platform using a revolute joint.

Original: The robot can move to a location using a wheel actuator and turn the camera using a camera motor simultaneously.
Revised: The robot can move to a location using a wheel actuator and turn the camera using a camera motor simultaneously.

Original: The action space in this paper is based on real-world coordinates.
Revised: The action space in this paper is based on real-world coordinates.

Original: The rotation angles for navigation and camera actions are set to the same.
Revised: The rotation angles for navigation and camera actions are set to the same.

Original: Thus, if both navigation and camera actions tend to move to the same direction, e.g., turning the platform and camera to the left, we only need to actuate the robot wheel and do not need to actuate the camera motor.
Revised: Thus, if both navigation and camera actions tend to move to the same direction, e.g., turning the platform and camera to the left, we only need to actuate the robot wheel and do not need to actuate the camera motor.

Original: Incorporating camera policy into existing navigation methods.
Revised: Incorporating camera policy into existing navigation methods.

Original: Existing navigation methods can be mainly categorized into two types, namely modular SLAM-based [59, 73] and end-to-end learning-based [69, 71].
Revised: Existing navigation methods can be mainly categorized into two types, namely modular SLAM-based [58, 72] and end-to-end learning-based [68, 70].

Original: For the first type of navigation method, we use the existing navigation policy to infer a navigation action, which is then fed to the proposed camera policy to infer a camera action.
Revised: For the first type of navigation method, we use the existing navigation policy to infer a navigation action, which is then fed to the proposed camera policy to infer a camera action.

Original: The paradigm is shown in Figure 2.
Revised: The paradigm is shown in Figure 2.

Original: In this case, it is worth mentioning that the camera policy coordinates with the navigation policy by conditioning the output of the navigation policy.
Revised: In this case, it is worth mentioning that the camera policy coordinates with the navigation policy by conditioning the output of the navigation policy.

Original: The navigation policy decides navigation action by exploiting a progressively built map, which is built from historical camera policy output.
Revised: The navigation policy decides navigation action by exploiting a progressively built map, which is built from historical camera policy output.

Original: For the end-to-end learning-based navigation method, the existing navigation policy typically contains the same recurrent network as our camera policy and uses it to predict navigation action.
Revised: For the end-to-end learning-based navigation method, the existing navigation policy typically contains the same recurrent network as our camera policy and uses it to predict navigation action.

Original: To better coordinate two kinds of policies, we use one recurrent network to predict both navigation action and camera action.
Revised: To better coordinate two kinds of policies, we use one recurrent network to predict both navigation action and camera action.

Original: Specifically, we feed this recurrent network both camera policy input ( i.e., map and heuristic direction) and navigation policy input (typically RGB-D images).
Revised: Specifically, we feed this recurrent network both camera policy input ( i.e., map and heuristic direction) and navigation policy input (typically RGB-D images).

Original: Unlike the paradigm in Figure 2, the recurrent network in this case does not need the features of navigation action anymore because it predicts both types of actions at the same time.
Revised: Unlike the paradigm in Figure 2, the recurrent network in this case does not need the features of navigation action anymore because it predicts both types of actions at the same time.

Original: We use the summation of exploration reward in Equation 1 and navigation reward in existing methods (typically the distance reduced to goals) to evaluate the goodness of predicted navigation-camera actions.
Revised: We use the summation of exploration reward in Equation 1 and navigation reward in existing methods (typically the distance reduced to goals) to evaluate the goodness of predicted navigation-camera actions.

Original: The paradigm is shown in Figure A in Supplementary Materials.
Revised: The paradigm is shown in Figure ?? in Supplementary Materials.

Original:  Experiments 4.1 Experimental Setup
Revised:  Experiments 4.1 Experimental Setup

Original: Task details.
Revised: Task details.

Original: We follow MultiON [69] to place multiple goal objects in the environment randomly.
Revised: We follow MultiON [68] to place multiple goal objects in the environment randomly.

Original: The objects are embodied with cylinders in different colors.
Revised: The objects are embodied with cylinders in different colors.

Original: In this way, We are free to decide the location and the number of goal objects to adjust the task difficulty.
Revised: In this way, We are free to decide the location and the number of goal objects to adjust the task difficulty.

Original: The agent succeeds in an episode if it calls FOUND action within a 1.5 meters radius of all objects in order.
Revised: The agent succeeds in an episode if it calls FOUND action within a 1.5 meters radius of all objects in order.

Original: We call FOUND automatically when the agent is near the current goal object because we focus on evaluating the effectiveness of the agent finding objects and navigating to them.
Revised: We call FOUND automatically when the agent is near the current goal object because we focus on evaluating the effectiveness of the agent finding objects and navigating to them.

Original: By default, we place three goal objects and denote it 3-ON task.
Revised: By default, we place three goal objects and denote it 3-ON task.

Original: We also show results of 2-ON and 1-ON in Supplementary Materials.
Revised: We also show results of 2-ON and 1-ON in Supplementary Materials.

Original: We perform experiments on two photorealistic 3D indoor environments, i.e., Matterport3D [7] and
Revised: We perform experiments on two photorealistic 3D indoor environments, i.e., Matterport3D [7] and

Original: Gibson [72].
Revised: Gibson [71].

Original: We follow the existing work [69] to evaluate the navigation success rate using Success and Progress metrics.
Revised: We follow the existing work [68] to evaluate the navigation success rate using Success and Progress metrics.

Original: Success indicates the percentage of successfully finished episodes, while Progress indicates the percentage of objects being successfully found.
Revised: Success indicates the percentage of successfully finished episodes, while Progress indicates the percentage of objects being successfully found.

Original: We also evaluate the navigation efficiency using
Revised: We also evaluate the navigation efficiency using

Original: SPL and PPL , which are short for Success weighted by Path Length and Progress weighted by Path
Revised: SPL and PPL , which are short for Success weighted by Path Length and Progress weighted by Path

Original: Length, respectively.
Revised: Length, respectively.

Original: The weight is proportional to the navigation efficiency and is defined as d { ¯ d , where d is the length of the ground-truth path and ¯ d is the path length traveled by an agent.
Revised: The weight is proportional to the navigation efficiency and is defined as d { ¯ d , where d is the length of the ground-truth path and ¯ d is the path length traveled by an agent.

Original: Implementation details.
Revised: Implementation details.

Original: For navigation and camera actions, a FORWARD action moves the agent forward by 0.25 meters and a TURN action turns by 30 ˝ .
Revised: For navigation and camera actions, a FORWARD action moves the agent forward by 0.25 meters and a TURN action turns by 30 ˝ .

Original: The maximum episode time step is 500 because an agent with a global ground-truth map ( i.e., oracle agent) finishes more than 97% of episodes within 500 steps.
Revised: The maximum episode time step is 500 because an agent with a global ground-truth map ( i.e., oracle agent) finishes more than 97% of episodes within 500 steps.

Original: Our camera policy tries to narrow the performance gap between such an oracle agent and the agent with a progressively built map.
Revised: Our camera policy tries to narrow the performance gap between such an oracle agent and the agent with a progressively built map.

Original: We set K “ 8 , r “ 2 . 4 , γ “ r ˆ 1 . 2 “ 2 . 88 in heuristic module empirically.
Revised: We set K “ 8 , r “ 2 . 4 , γ “ r ˆ 1 . 2 “ 2 . 88 in heuristic module empirically.

Original: We use a mapper that outputs the ground-truth of occupancy anticipation [59], because how to train a good mapper is orthogonal to our work.
Revised: We use a mapper that outputs the ground-truth of occupancy anticipation [58], because how to train a good mapper is orthogonal to our work.

Original: Reward scaling factors α and β are set to 10 and 1 respectively such that three reward terms are in the same order of magnitude at initialization.
Revised: Reward scaling factors α and β are set to 10 and 1 respectively such that three reward terms are in the same order of magnitude at initialization.

Original: We evaluate models for five runs using the same set of random seeds and report the mean results and standard deviation.
Revised: We evaluate models for five runs using the same set of random seeds and report the mean results and standard deviation.

Original: More details are shown in Supplementary
Revised: More details are shown in Supplementary

Original: Materials.
Revised: Materials.

Original: 4.2 Baselines
Revised: 4.2 Baselines

Original: Mapping + FBE [73]: This SLAM-based navigation method breaks the problem into mapping and path planning.
Revised: Mapping + FBE [72]: This SLAM-based navigation method breaks the problem into mapping and path planning.

Original: We use depth projection [21] for mapping and frontier-boundary-exploration (FBE) method [73] to select an exploration point.
Revised: We use depth projection [21] for mapping and frontier-boundary-exploration (FBE) method [72] to select an exploration point.

Original: Once the built map covers goal objects, the agent navigates to them by taking deterministic actions [8] along the path planned by A* algorithm [34].
Revised: Once the built map covers goal objects, the agent navigates to them by taking deterministic actions [8] along the path planned by A* algorithm [34].

Original: OccAnt [59]: This baseline is the same as the previous one except that we replace the depth projection with an occupancy anticipation neural network [59].
Revised: OccAnt [58]: This baseline is the same as the previous one except that we replace the depth projection with an occupancy anticipation neural network [58].

Original: Such a network can infer the occupancy state beyond the visible regions.
Revised: Such a network can infer the occupancy state beyond the visible regions.

Original: We assume the neural network is well-trained so that we use the groundtruth occupancy state within the field of view for experiments.
Revised: We assume the neural network is well-trained so that we use the groundtruth occupancy state within the field of view for experiments.

Original: [Table]
Revised: [Table]

Original: DD-PPO [71]: This end-to-end learning-based baseline performs navigation using RL algorithm.
Revised: DD-PPO [70]: This end-to-end learning-based baseline performs navigation using RL algorithm.

Original: It consists of a recurrent policy network, which takes as input RGB-D images, goal object categories and previous actions for predicting navigation action.
Revised: It consists of a recurrent policy network, which takes as input RGB-D images, goal object categories and previous actions for predicting navigation action.

Original: MultiON [69]: This is the variant of DD-PPO, with a progressively built object map as an extra input.
Revised: MultiON [68]: This is the variant of DD-PPO, with a progressively built object map as an extra input.

Original: Each cell of the object map is a one-hot vector indicating the existence of the goal objects.
Revised: Each cell of the object map is a one-hot vector indicating the existence of the goal objects.

Original: We store an object on the map once it is within the field of view of agents.
Revised: We store an object on the map once it is within the field of view of agents.

Original: We encode the egocentric cropped object map and feed it to the policy network.
Revised: We encode the egocentric cropped object map and feed it to the policy network.

Original: This baseline is the same as OracleEgoMap variant presented by [69].
Revised: This baseline is the same as OracleEgoMap variant presented by [68].

Original: 4.3 Multi-object Navigation Results
Revised: 4.3 Multi-object Navigation Results

Original: Results on Matterport3D dataset.
Revised: Results on Matterport3D dataset.

Original: In Table 1, the agent with our EXPO camera policy performs better on multi-object navigation task upon four baselines.
Revised: In Table 1, the agent with our EXPO camera policy performs better on multi-object navigation task upon four baselines.

Original: Specifically, our agent increases Success and Progress metrics for a large margin, indicating incorporating our camera policy helps agents successfully navigate to more goal objects.
Revised: Specifically, our agent increases Success and Progress metrics for a large margin, indicating incorporating our camera policy helps agents successfully navigate to more goal objects.

Original: We attribute the improvement to a better exploration ability of our agent.
Revised: We attribute the improvement to a better exploration ability of our agent.

Original: With such ability, the agent finds more goal objects in a limited time step and then navigates to them.
Revised: With such ability, the agent finds more goal objects in a limited time step and then navigates to them.

Original: Besides, the improvement on SPL and PPL indicates our agent navigates to goal objects along a shorter path.
Revised: Besides, the improvement on SPL and PPL indicates our agent navigates to goal objects along a shorter path.

Original: The agent does not need to walk inside all rooms.
Revised: The agent does not need to walk inside all rooms.

Original: Instead, the actively moving camera allows the agent to perceive what is inside a room when it passes by.
Revised: Instead, the actively moving camera allows the agent to perceive what is inside a room when it passes by.

Original: The above results show our agent navigates to goal objects more efficiently with a higher success rate.
Revised: The above results show our agent navigates to goal objects more efficiently with a higher success rate.

Original: We encourage readers to see the visualization in Figure 4 and watch supplementary videos.
Revised: We encourage readers to see the visualization in Figure 4 and watch supplementary videos.

Original: The camera action provides more movement freedom for the agent.
Revised: The camera action provides more movement freedom for the agent.

Original: We are interested in the question that whether the improvement comes from simply extending the action space.
Revised: We are interested in the question that whether the improvement comes from simply extending the action space.

Original: To this end, we remove our camera policy.
Revised: To this end, we remove our camera policy.

Original: The agent determines camera actions naively.
Revised: The agent determines camera actions naively.

Original: Specifically, for SLAM-based baselines, the agent chooses a camera action randomly.
Revised: Specifically, for SLAM-based baselines, the agent chooses a camera action randomly.

Original: For learning-based baselines, the agent learns to predict both camera and navigation actions using the original navigation input and rewards.
Revised: For learning-based baselines, the agent learns to predict both camera and navigation actions using the original navigation input and rewards.

Original: In
Revised: In

Original: Table 1, using these naive camera actions brings little improvement or even negative influence.
Revised: Table 1, using these naive camera actions brings little improvement or even negative influence.

Original: This is not surprising because it is nontrivial to coordinate the camera and navigation actions.
Revised: This is not surprising because it is nontrivial to coordinate the camera and navigation actions.

Original: Also, a larger action space may increase the learning difficulty.
Revised: Also, a larger action space may increase the learning difficulty.

Original: These results further suggest the importance of the proposed camera policy for determining a reasonable camera action.
Revised: These results further suggest the importance of the proposed camera policy for determining a reasonable camera action.

Original: Transferability of camera policy to Gibson dataset.
Revised: Transferability of camera policy to Gibson dataset.

Original: We evaluate the transferability of the learned camera policy on Gibson dataset.
Revised: We evaluate the transferability of the learned camera policy on Gibson dataset.

Original: In Table 1, a similar trend is observed on all baselines, i.e., using naive camera actions does not help for navigation while our EXPO camera policy performs better than baselines.
Revised: In Table 1, a similar trend is observed on all baselines, i.e., using naive camera actions does not help for navigation while our EXPO camera policy performs better than baselines.

Original: It is worth noting that the EXPO camera policy is trained on Matterport3D and has not been fine-tuned on Gibson dataset.
Revised: It is worth noting that the EXPO camera policy is trained on Matterport3D and has not been fine-tuned on Gibson dataset.

Original: There are significant differences between these two datasets in scene style and layout distribution [7, 72].
Revised: There are significant differences between these two datasets in scene style and layout distribution [7, 71].

Original: The consistent improvement demonstrates that our active-camera agent has learned general exploration and navigation skills for the multi-object navigation task.
Revised: The consistent improvement demonstrates that our active-camera agent has learned general exploration and navigation skills for the multi-object navigation task.

Original: It also shows the possibility of transferring the agent to a real-world scene.
Revised: It also shows the possibility of transferring the agent to a real-world scene.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4.4 Further Analysis
Revised: 4.4 Further Analysis

Original: Rule-based camera actions vs. learned camera policy.
Revised: Rule-based camera actions vs. learned camera policy.

Original: In contrast to learning a neural network, one may use handcrafted rules to decide camera actions.
Revised: In contrast to learning a neural network, one may use handcrafted rules to decide camera actions.

Original: We compare the learned camera policy with three types of rule-based camera actions, i.e., 1) selecting a random camera action; 2) forcing the agent to look forward and swing within 90 ˝ around the navigation direction; 3) following the heuristic direction inferred from the heuristic module.
Revised: We compare the learned camera policy with three types of rule-based camera actions, i.e., 1) selecting a random camera action; 2) forcing the agent to look forward and swing within 90 ˝ around the navigation direction; 3) following the heuristic direction inferred from the heuristic module.

Original: The results upon a SLAM-based baseline, i.e., OccAnt, are shown in Table 3.
Revised: The results upon a SLAM-based baseline, i.e., OccAnt, are shown in Table 3.

Original: Exploiting random camera actions drops the performance because the agent often looks backward and captures redundant useless information.
Revised: Exploiting random camera actions drops the performance because the agent often looks backward and captures redundant useless information.

Original: The other two types of rule-based camera actions improve the performance slightly.
Revised: The other two types of rule-based camera actions improve the performance slightly.

Original: These camera actions help the agent build a map covering more areas.
Revised: These camera actions help the agent build a map covering more areas.

Original: Consequently, the
Revised: Consequently, the

Original: SLAM-based navigation policy can plan a better path for navigation using a path-planning algorithm.
Revised: SLAM-based navigation policy can plan a better path for navigation using a path-planning algorithm.

Original: However, it is hard for us to design a robust rule covering all situations.
Revised: However, it is hard for us to design a robust rule covering all situations.

Original: For example, the swing camera actions may miss some explorable areas because the agent has passed by these areas before the camera swing to the direction pointing to them.
Revised: For example, the swing camera actions may miss some explorable areas because the agent has passed by these areas before the camera swing to the direction pointing to them.

Original: Also, there exist false positive areas in the occupancy map ( e.g., free space behind a table is predicted as an obstacle).
Revised: Also, there exist false positive areas in the occupancy map ( e.g., free space behind a table is predicted as an obstacle).

Original: These areas may mislead the heuristic module to consider an unexplored area as an explored one.
Revised: These areas may mislead the heuristic module to consider an unexplored area as an explored one.

Original: Compared with these camera actions, our learned camera policy brings a more significant improvement.
Revised: Compared with these camera actions, our learned camera policy brings a more significant improvement.

Original: We attribute the improvement to the exploration reward in Equation 1.
Revised: We attribute the improvement to the exploration reward in Equation 1.

Original: With such a reward, the agent is encouraged to take into account not only the handcraft rules but also the noisy occupancy map to predict a better camera action.
Revised: With such a reward, the agent is encouraged to take into account not only the handcraft rules but also the noisy occupancy map to predict a better camera action.

Original: As for learning-based baseline, i.e., MultiON, using these three types of rule-based camera actions significantly drops the performance.
Revised: As for learning-based baseline, i.e., MultiON, using these three types of rule-based camera actions significantly drops the performance.

Original: In these experiments, it is worth noting that we have fed previous camera action to inform the policy in which direction the observation is taken from.
Revised: In these experiments, it is worth noting that we have fed previous camera action to inform the policy in which direction the observation is taken from.

Original: We suspect the poor performance is because the camera movement decided by rules is uncontrollable by the navigation policy.
Revised: We suspect the poor performance is because the camera movement decided by rules is uncontrollable by the navigation policy.

Original: As a result, the navigation policy can not get desired observations for predicting the next navigation action.
Revised: As a result, the navigation policy can not get desired observations for predicting the next navigation action.

Original: In contrast, the learned camera policy, which is trained together with the navigation policy, allows the agent to determine how to move its camera by itself.
Revised: In contrast, the learned camera policy, which is trained together with the navigation policy, allows the agent to determine how to move its camera by itself.

Original: The above results further demonstrate the importance of the learned EXPO camera policy.
Revised: The above results further demonstrate the importance of the learned EXPO camera policy.

Original: Ablation study on camera policy inputs and rewards.
Revised: Ablation study on camera policy inputs and rewards.

Original: We conduct this ablation study by removing one of the inputs and rewards of a camera policy upon OccAnt baseline on MatterPort3D dataset.
Revised: We conduct this ablation study by removing one of the inputs and rewards of a camera policy upon OccAnt baseline on MatterPort3D dataset.

Original: In
Revised: In

Original: Table 2, removing any input or reward will drop the performance.
Revised: Table 2, removing any input or reward will drop the performance.

Original: We note that human knowledge ( i.e., heuristic input and reward) is important for the camera policy.
Revised: We note that human knowledge ( i.e., heuristic input and reward) is important for the camera policy.

Original: Awarding the policy to follow this knowledge can be considered as a form of regularization and guidance for learning.
Revised: Awarding the policy to follow this knowledge can be considered as a form of regularization and guidance for learning.

Original: The egocentric occupancy map input and area reward are also critical.
Revised: The egocentric occupancy map input and area reward are also critical.

Original: They encourage the agent to explore more areas.
Revised: They encourage the agent to explore more areas.

Original: With a better exploration ability, the agent can find goal objects and navigate to them more efficiently.
Revised: With a better exploration ability, the agent can find goal objects and navigate to them more efficiently.

Original: The turn-penalty reward has little influence on navigation performance.
Revised: The turn-penalty reward has little influence on navigation performance.

Original: However, it helps to reduce the frequency of actuating camera motor described in Section 3.4.
Revised: However, it helps to reduce the frequency of actuating camera motor described in Section 3.4.

Original: Experimental results show that without this reward, agents actuate the camera motor for 12.41% of total time steps.
Revised: Experimental results show that without this reward, agents actuate the camera motor for 12.41% of total time steps.

Original: Adding turn-penalty reward decreases the number to 5.70%.
Revised: Adding turn-penalty reward decreases the number to 5.70%.

Original: Does camera policy work with imperfect mapper?
Revised: Does camera policy work with imperfect mapper?

Original: Our EXPO camera policy obtains indoor layout information mainly from the progressively built map.
Revised: Our EXPO camera policy obtains indoor layout information mainly from the progressively built map.

Original: In this subsection, we would like to evaluate whether the proposed camera policy works with a noisy map.
Revised: In this subsection, we would like to evaluate whether the proposed camera policy works with a noisy map.

Original: To this end, we use a learned neural network [59] to predict the occupancy map from RGB-D images.
Revised: To this end, we use a learned neural network [58] to predict the occupancy map from RGB-D images.

Original: The predicted map contains many false positive points ( e.g., predicting the free space as occupancy) and false negative points ( e.g.,
Revised: The predicted map contains many false positive points ( e.g., predicting the free space as occupancy) and false negative points ( e.g.,

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: t= t= t= predicting a wall as free space).
Revised: t= t= t= predicting a wall as free space).

Original: These noisy points may mislead the camera policy to make a wrong camera action.
Revised: These noisy points may mislead the camera policy to make a wrong camera action.

Original: Experimental results on the OccAnt baseline show that incorporating camera policy brings consistent improvement.
Revised: Experimental results on the OccAnt baseline show that incorporating camera policy brings consistent improvement.

Original: Specifically, SPL and PPL increase from 13.6 to 18.1 and from 20.5 to 25.4, respectively.
Revised: Specifically, SPL and PPL increase from 13.6 to 18.1 and from 20.5 to 25.4, respectively.

Original: Success and Progress increase from 18.39 to 24.0 and from 28.4 to 34.2, respectively.
Revised: Success and Progress increase from 18.39 to 24.0 and from 28.4 to 34.2, respectively.

Original: These results demonstrate that the proposed EXPO camera policy is robust to the map noise and can be deployed in a robot with no need for a ground-truth map.
Revised: These results demonstrate that the proposed EXPO camera policy is robust to the map noise and can be deployed in a robot with no need for a ground-truth map.

Original: Does improvement come from extra information?
Revised: Does improvement come from extra information?

Original: Compared with baselines, our active-camera agent leverages extra information ( i e., occupancy map, heuristic direction and exploration reward) for determining camera actions.
Revised: Compared with baselines, our active-camera agent leverages extra information ( i e., occupancy map, heuristic direction and exploration reward) for determining camera actions.

Original: For a fair comparison, we add this extra information into MultiON to build an enhanced baseline.
Revised: For a fair comparison, we add this extra information into MultiON to build an enhanced baseline.

Original: Results on MatterPort3D dataset demonstrate that the enhanced baseline performs slightly better than MultiON baseline but worse than our active-camera agent.
Revised: Results on MatterPort3D dataset demonstrate that the enhanced baseline performs slightly better than MultiON baseline but worse than our active-camera agent.

Original: Specifically, SPL, PPL, Success and Progress are 35.1, 45.9, 48.0, and 63.2, respectively.
Revised: Specifically, SPL, PPL, Success and Progress are 35.1, 45.9, 48.0, and 63.2, respectively.

Original: We suspect the improvement of the enhanced baseline comes from the fact that heuristic direction provides location information about unexplored areas.
Revised: We suspect the improvement of the enhanced baseline comes from the fact that heuristic direction provides location information about unexplored areas.

Original: Also, the extra rewards encourage agents to explore these areas.
Revised: Also, the extra rewards encourage agents to explore these areas.

Original: However, due to the limited action space, the agent in enhanced baseline can not coordinate their camera and navigation actions well, which limits the performance.
Revised: However, due to the limited action space, the agent in enhanced baseline can not coordinate their camera and navigation actions well, which limits the performance.

Original: Exploration performance.
Revised: Exploration performance.

Original: One of the critical abilities for the multi-object navigation task is exploring the environment efficiently to locate all goal objects.
Revised: One of the critical abilities for the multi-object navigation task is exploring the environment efficiently to locate all goal objects.

Original: To evaluate the exploration ability, we place the agent in a novel environment with three goal objects located in different places.
Revised: To evaluate the exploration ability, we place the agent in a novel environment with three goal objects located in different places.

Original: We follow FBE method [73] to explore the environment.
Revised: We follow FBE method [72] to explore the environment.

Original: Given a limited time step budget, we evaluate the success rate of finding all three goal objects and explored areas in Figures 3.
Revised: Given a limited time step budget, we evaluate the success rate of finding all three goal objects and explored areas in Figures 3.

Original: Compared with the baselines that the agent is always looking forward or moving its camera randomly, the agent with our
Revised: Compared with the baselines that the agent is always looking forward or moving its camera randomly, the agent with our

Original: EXPO camera policy finds more goal objects and explores more areas.
Revised: EXPO camera policy finds more goal objects and explores more areas.

Original: These results have the same trend of navigation performance in Table 1, suggesting that our EXPO camera policy helps to explore the environment more efficiently and consequently boosts the multi-object navigation performance.
Revised: These results have the same trend of navigation performance in Table 1, suggesting that our EXPO camera policy helps to explore the environment more efficiently and consequently boosts the multi-object navigation performance.

Original: Visualization.
Revised: Visualization.

Original: In Figure 4, both baseline ( i.e., OccAnt) and our agent are navigating to Goal-1 at the beginning.
Revised: In Figure 4, both baseline ( i.e., OccAnt) and our agent are navigating to Goal-1 at the beginning.

Original: During this process, our agent moves its camera actively at time step t “ 16 and t “ 26 , finding Goal-2 and Goal-3 respectively.
Revised: During this process, our agent moves its camera actively at time step t “ 16 and t “ 26 , finding Goal-2 and Goal-3 respectively.

Original: Knowing the location of goal objects, our agent plans the shortest path for navigation.
Revised: Knowing the location of goal objects, our agent plans the shortest path for navigation.

Original: In contrast, the baseline agent goes straight to Goal-1 , with the camera looking forward constantly.
Revised: In contrast, the baseline agent goes straight to Goal-1 , with the camera looking forward constantly.

Original: Consequently, after it navigates to Goal-1 , it cannot find other goal objects and has to waste time exploring the environment again.
Revised: Consequently, after it navigates to Goal-1 , it cannot find other goal objects and has to waste time exploring the environment again.

Original: Failure case analysis can be found in Supplementary Materials.
Revised: Failure case analysis can be found in Supplementary Materials.

Original:  Conclusion
Revised:  Conclusion

Original: In order to solve the uncoordinated camera-navigation actions problem of existing agents, we propose a navigation paradigm in which agents can dynamically move their cameras for perceiving environments more efficiently.
Revised: In order to solve the uncoordinated camera-navigation actions problem of existing agents, we propose a navigation paradigm in which agents can dynamically move their cameras for perceiving environments more efficiently.

Original: Such exploration ability is important for multi-object navigation.
Revised: Such exploration ability is important for multi-object navigation.

Original: To determine the camera actions, we learn a camera policy via reinforcement learning by awarding it to explore more areas.
Revised: To determine the camera actions, we learn a camera policy via reinforcement learning by awarding it to explore more areas.

Original: Also, we use heuristic rules to guide the learning process and reduce the learning difficulty.
Revised: Also, we use heuristic rules to guide the learning process and reduce the learning difficulty.

Original: The proposed camera policy can be incorporated into most existing navigation methods.
Revised: The proposed camera policy can be incorporated into most existing navigation methods.

Original: Experimental results show that our camera policy consistently improves the multi-object navigation performance of multiple existing methods on two benchmark datasets.
Revised: Experimental results show that our camera policy consistently improves the multi-object navigation performance of multiple existing methods on two benchmark datasets.

Original: Acknowledgments
Revised: Acknowledgments

Original: This work was partially supported by National Key R&D Program of China (No. 2020AAA0106900),
Revised: This work was partially supported by National Key R&D Program of China (No. 2020AAA0106900),

Original: National Natural Science Foundation of China (No. 62072190 and No. 62006137), Program for
Revised: National Natural Science Foundation of China (No. 62072190 and No. 62006137), Program for

Original: Guangdong Introducing Innovative and Enterpreneurial Teams (No. 2017ZT07X183)Supplementary Materials for “Learning
Revised: Guangdong Introducing Innovative and Enterpreneurial Teams (No. 2017ZT07X183)

Original: Active Camera for Multi-Object Navigation”

Original: In the supplementary, we provide more implementation details and more experimental results of our

Original: EXPO camera policy.

Original: We organize the supplementary as follows.

Original: • In Section A, we provide the differences of active cameras from panoramic cameras.

Original: • In Section B, we provide more discussion on how to coordinate two types of policies.

Original: • In Section C, we provide more architecture details on our EXPO camera policy.

Original: • In Section D, we provide more details on updating the top-down map.

Original: • In Section E, we provide more experimental details, including training settings, environment details, and evaluation metrics.

Original: • In Section F, we provide experimental results on multi-object navigation with different number of goal objects.

Original: • In Section G, we provide experimental results to explain the choice of step budget for evaluation.

Original: • In Section H, we provide more discussions on oracle information used in experiments.

Original: • In Section I, we provide analysis on failure cases.

Original: • In Section J, we discuss potential future researches and social impact.

Original: A Differences of active cameras from panoramic cameras

Original: In this paper, we propose an active camera to coordinate camera and navigation actions for perceiving a novel environment more efficiently.

Original: An alternative is to use panoramic cameras for capturing panoramic images.

Original: However, panoramic images are often in a very large size and contain a lot of redundant information, requiring a huge amount of resources to analyze.

Original: It may not be practical to equip panoramic cameras on mobile platforms.

Original: In this case, designing an active camera would be a more practical way to capture necessary environmental information required for navigation at a much cheaper cost.

Original: An agent with multiple fixed pinhole cameras could make for an interesting comparison with our active-camera agent.

Original: To compare our active-camera agent with the multi-fixed-camera agent, we equip an agent with four fixed pinhole cameras facing different directions ( i.e., front, left, right, and back).

Original: Specifically, we concatenate the features of these four images and feed them to an end-to-end navigation baseline ( i.e., MultiON).

Original: We train this baseline for 30 million frames, which is the same as our active-camera agent.

Original: In Table A, the multi-fixed-camera agent performs better than the single-fixed-camera agent but still worse than our single-active-camera agent.

Original: We speculate this is because the end-to-end learned neural network is hard to map this larger amount of information (four

Original: RGBD images) to correct actions.

Original: We note that the multi-fixed-camera results in Table A are different from the results reported in Openreview during the rebuttal phase.

Original: This is because we train the agent using different numbers of parallel threads.

Original: We found that using less threads for training significantly decreases performance.

Original: The possible reason is that the model may be unstable during training if capturing training data from only a few environments.

Original: In Table A, all variants use the same number of parallel threads for training ( i.e., 36).

Original: [Table]

Original: We also have conducted multi-fixed-camera experiments on the SLAM-based method.

Original: The information captured by multiple cameras helps agents build a map efficiently.

Original: This map provides more information for path-planning.

Original: The results are shown in Table

Original: B.

Original: The multi-fixed-camera agent outperforms our active camera agent.

Original: Because the SLAM-based multi-fixed-camera agent does not need to learn a neural network to map RBGD observation to navigation action, it will not suffer from learning difficulty problems as in e2e-based agents.

Original: It is worth noting that although using a single camera, our active camera policy helps the agent achieve comparative performance compared with an agent with four fixed cameras.

Original: [Table]

Original: B More discussion on the coordination between two types of policies

Original: When incorporating camera policy with an end-to-end navigation policy, we use one policy network ( i.e., Navigation-Camera Joint Policy) to jointly predict both camera and navigation actions, as described in Section 3.4 in the main paper.

Original: The paradigm is shown in Figure A.

Original: We feed this policy network both camera policy inputs ( i.e., map and heuristic direction) and navigation policy inputs ( i.e., RGB-D images, target, and previous action).

Original: In this way, the policy network better coordinates two types of actions.

Original: We have tried a variant that uses two separate policy networks to predict camera and navigation actions, respectively, which is similar to the paradigm in Figure 2 in the main paper.

Original: In Table C, this separated variant performs worse than our joint policy and even worse than the baseline without an active camera.

Original: We suspect this is because the navigation policy can not control camera direction to perceive desired observations for deciding navigation action.

Original: As a result, an agent performs poorly in navigation.

Original: These observations are consistent with the experimental results in

Original: Table 3 in the main paper.

Original: In Table 3, the agent performs worse when using rule-based camera actions because these camera actions are uncontrollable by the navigation policy either.

Original: [Table]

Original: When incorporating a camera policy with a SLAM-based navigation policy, two types of policies decide actions separately as shown in Figure 2 in the paper.

Original: Although the agent does not decide two types of actions jointly, two types of policies cooperate with each other from the following two aspects.

Original: The camera policy cooperates with navigation action by taking it as input when deciding camera action.

Original: The navigation policy cooperates with camera action by an off-the-shelf path-planning algorithm ( e.g., if the agent finds a better path toward goal objects after executing a camera action, the navigation policy will change the navigation action accordingly).

Original: C More architecture details on camera policy

Original: The general scheme of our active-camera agents, i.e., , incorporating a camera policy with a modular

Original: SLAM-based [59, 73] navigation policy or an end-to-end learning-based [69, 71] navigation policy, are shown in Figure 2 and Figure A, respectively.

Original: We feed map features, heuristic direction features and navigation features to a policy network to predict camera action.

Original: These features are extracted from three encoders, respectively.

Original: Next, we describe the architecture details of these encoders and the policy network.

Original: Observation

Original: [Figure]

Original: The map encoder consists of three convolution layers followed by a fully connected layer.

Original: We use

Original: ReLU as an activation function after each layer.

Original: The setting of three convolution layers (kernel size, stride) are p 4 ˆ 4 , 2 q , p 3 ˆ 3 , 2 q and p 2 ˆ 2 , 1 q , respectively.

Original: The fully connected layer outputs 512-dimension map features v m .

Original: The heuristic direction encoder is a learned embedding layer, transforming the input heuristic action into 32-dimension features v h .

Original: As for navigation encoder, its architecture depends on what kind of navigation policy we incorporate the camera policy with.

Original: If we use a SLAM-based navigation policy that output an navigation action, the navigation encoder is a learned embedding layer which transforms the action into 32-dimension features v n .

Original: If we use an end-to-end navigation policy, the encoder of this navigation policy [69, 71] would become our navigation encoder, transforming navigation input (typically RGB-D images) to latent features.

Original: With the features from these three encoders, we concatenate them to build the input v “ r v m , v h , v n s for the policy network.

Original: The policy network consist of a one-layer gate recurrent unit (GRU), followed by two individual fully connected layers corresponding for predicting camera action and values of the agent’s current state, respectively.

Original: D Updating top-down map by a registration function R p¨q

Original: As described in Section 3.2 in the paper, we use a registration function R p¨q to update the global top-down occupancy map M g , i.e., M g “ R p M l , M g , q q , where M l is a local map and q is the relative pose change of the agent from the first time step.

Original: After that, we crop an egocentric map from the global as camera policy input.

Original: We describe the details of this registration function below.

Original: All these three types of maps are with the same resolution.

Original: Each pixel in a map represents 0 .

Original: 08 ˆ 0 .

Original: 08 m 2 area in the real world.

Original: The size of local map, global map and egocentric map are 61 ˆ 61 , 3000 ˆ 3000 , and 125 ˆ 125 , respectively.

Original: These maps represent 4 . 88 ˆ 4 .

Original: 88 m 2 area in front of the agent, 240 ˆ 240 m 2 area in the real-world, and 10 ˆ 10 m 2 area around the agent, respectively.

Original: The global map is with a larger size and covers the whole environment.

Original: In contrast, the local map represents only a small area, which is projected from the visual observation within a 79 ˝ field-of-view in front of an agent.

Original: To merge these two maps, we initialize a blank map with the same size as the global map.

Original: We put the local map in the center.

Original: Then, we use affine transformation [37] on this map based on the pose q .

Original: This affine transformation transforms the local to its location on the global map.

Original: After that, we aggregate the transformed map with the global map by a max-pooling operation.

Original: E More experimental details

Original: More implementation details.

Original: We use PyTorch to implement our active-camera agent.

Original: We train it using 36 parallel threads on 3 Nvidia Titan X GPUs.

Original: Each thread contains one of the scenes from the training set.

Original: The agent is trained using PPO with 4 mini-batches and 2 epochs in each PPO update.

Original: We use Adam optimizer with a learning rate of 2.5e-4, a discount factor of γ “ 0 . 99 , an entropy coefficient of 0.001, a value loss coefficient of 0.5.

Original: We train the proposed camera policy together with the SLAM-based and learning-based methods for 2 and 30 million frames, respectively.

Original: More details on environments.

Original: We use Habitat simulator 2 [62] along with two large-scale photorealistic 3D indoor environments ( i.e., Matterport3D [7] and Gibson [72]).

Original: Both environments consist various of scenes such as house, church, and factory.

Original: For Matterport3D, we use the dataset proposed in MultiON [69] for the multi-object navigation task.

Original: In this dataset, 79 scenes are divided into two disjoint parts, 61 scenes for training and 18 scenes for testing.

Original: For Gibson, we evaluate the transferability of our camera policy.

Original: As the MultiON paper does not provide a corresponding multi-object dataset, we use the test scene split provided by Active Neural SLAM[9] and follow the episode generation rule in MultiON ( i.e., the geodesic distance between two successive goals is in the range of 2m and 20m) to generate 861 multi-object episodes within 14 different scenes for testing.

Original: For each episode, we ensure that all objects and start positions are on the same floor.

Original: More details on metrics.

Original: We follow MultiON [69] to evaluate multi-object navigation in terms of success rate and navigation efficiency.

Original: A good agent should successfully navigate to all goal objects through the shortest path.

Original: The details are described below.

Original: • Success : ratio of successfully navigating to all goal objects and calling FOUND in a correct order within an allowed time step budget.

Original: • Progress : the fraction of goal objects being successfully found.

Original: • SPL : Success weighted by Path Length.

Original: Concretely, SPL “ s ˆ d { max p d, ¯ d q , where s indicates the value of Success metric, d is the shortest geodesic distance from the starting point to all objects in order, ¯ d indicates the geodesic distance traveled by the agent.

Original: • PPL : Progress weighted by Path Length.

Original: This is an extended version of SPL based on progress.

Original: Concretely, PPL “ p ˆ d 1 { max p d 1 , ¯ d 1 q , where p indicates the value of Progress metric, d 1 and ¯ d 1 is similar to d and ¯ d in SPL metric but only account for the path through all founded objects.

Original: F More results on 1-ON, 2-ON and 3-ON episodes

Original: In order to evaluate the efficiency of our camera policy for the navigation tasks with different difficulties, except for 3-ON episodes ( i

Original: [Table]

Original: e., navigating to 3 different objects in an environment), we follow MultiON [69] to evaluate on 2-ON and 1-ON episodes on both Matterport3D and Gibson datasets.

Original: Performance on Matterport3D dataset.

Original: In Table D our camera policy consistently improves the navigation performance based on different baselines.

Original: These results suggest that actively moving camera following our camera policy helps agents explore the environment and locate objects more efficiently.

Original: These abilities benefit both the single object navigation task and the multi-object navigation task.

Original: [Table]

Original: [Table]

Original: MultiON

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: Policy

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: PPO

Original: [Table]

Original: Policy

Original: [Table]

Original: Policy

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: 2 https://aihabitat.org/

Original: Transferability results to Gibson dataset.

Original: In Table E, we show the Transferability results to the

Original: Gibson dataset.

Original: Most of the results share the same trend as the results on the Matterport3D dataset, i.e., our EXPO camera policy improves the object navigation performance in terms of Success, Progress,

Original: SPL

Original: [Table]

Original: and PPL based on different baselines.

Original: These results demonstrate that our method can generalize well to different domains.

Original: This allows us to train the agent on the collected photorealistic datasets and then easily deploy it in the real-world environment

Original: [Table]

Original: [Table]

Original: MultiON 68.

Original: 77.3 +Naive Camera Policy 69.

Original: 75.2 + Ours Camera Policy 73.

Original: DD-PPO 53.

Original: 54.8 +Naive Camera Policy 56.

Original: 54.5 + Ours Camera Policy 57.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: G Selection of time step budget for evaluation

Original: [Figure]

Original: Exploring the environment and locating objects efficiently are important abilities for multiobject navigation.

Original: We propose EXPO camera policy to help agents coordinate their camera and navigation actions for exploring the environment more efficiently.

Original: To better evaluate the efficiency, we evaluate the navigation success rate given a limited time step budget.

Original: If given an infinite time step budget for an agent, it is trivial for the agent to transverse the entire environment and then go to goal objects.

Original: To determine the value of time step budget, we consider an oracle agent with a ground-truth occupancy and object map.

Original: In other words, the oracle agent knows the position of objects and thus it does not need to explore the environment and find them.

Original: We evaluate how many time steps the oracle agent need for finishing the multi-object navigation task.

Original: In Figure B, the oracle agent successfully finishes more than 97% episodes using 500 time steps and the success rate remains the same when the time step is greater than 500.

Original: According to these results, we select 500 as the time step budget for evaluation because 500 time steps are sufficient for an oracle agent to navigate to all goal objects.

Original: We hope our EXPO camera policy helps the agents without map knowledge narrow the performance gap to the oracle agent.

Original: H More discussions on oracle information used in experiments

Original: In our experiments, to indicate whether an agent has reached goal objects, we use an oracle FOUND action ( i.e., automatically execute FOUND action once the agent reaches goal objects).

Original: We argue that this simplification is reasonable because the agent can easily judge whether a goal object appears near it with the help of a well-trained semantic segmentation model and a depth image.

Original: After simplifying this problem by using the oracle FOUND action, we are allowed to focus more on evaluating the exploration ability of active-camera agents.

Original: I Failure cases analysis

Original: We analyze failure cases where our active-camera agent takes more time for reaching goal objects compared with baselines.

Original: We observe that most of these cases are because our agent selects the wrong way for exploration.

Original: This is inevitable for all agents because goal objects are placed randomly in our experiments.

Original: We believe this problem can be solved if we have more information about the object’s location ( e.g., finding a specific object like food, which is likely located in the kitchen).

Original: We can involve some constraints to guide our agent to explore relevant areas actively.

Original: We leave this in future works.

Original: Even so, as seen in Figure 3 in the paper, given the same time step, our agent explores more areas and finds all objects more quickly, which demonstrates its superior exploration ability.

Original: We strongly recommend readers watch the supplementary video for more examples.

Original: J Potential future researches and social impact

Original: In this paper, we present an active-camera agent.

Original: This agent can explore the environment and finish multi-object navigation task more efficiently by dynamically moving RGB-D camera sensors.

Original: In the real world, we may equip robots with different sensors, such as microphones and cameras.

Original: How to coordinate the movement of these sensors is worth exploring.

Original: Besides, we have assumed perfect camera pose localization in our experiments.

Original: However, there exist actuation and sensor noise in the real world.

Original: Leveraging a neural network [9] to solve this problem is an interesting research direction.

Original: Besides, even though we have evaluated the transferability of camera policy among different datasets (transferring from MatterPort3D to Gibson) in our paper, evaluating the robustness of the camera policy among different robot types and different embodied tasks is also a valuable research direction.

Original: In the future, these agents may be able to help people deliver parcels and even take care of patients.

Original: On the other hand, an imperfect robotic agent may cause accidents such as breaking some things and hitting pedestrians.

Original: A possible way to avoid these accidents is to first develop such a robotic agent in a simulator environment and then deploy it into a controlled real environment for evaluation.