Original: V ISUAL S EMANTIC N AVIGATION USING S CENE P RIORS
Revised: V ISUAL S EMANTIC N AVIGATION USING S CENE P RIORS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: [Figure]
Revised: [Figure]

Original: A BSTRACT
Revised: A BSTRACT

Original: How do humans navigate to target objects in novel scenes?
Revised: How do humans navigate to target objects in novel scenes?

Original: Do we use the semantic/functional priors we have built over years to efﬁciently search and navigate?
Revised: Do we use the semantic/functional priors we have built over years to efﬁciently search and navigate?

Original: For example, to search for mugs, we search cabinets near the coffee machine and for fruits we try the fridge.
Revised: For example, to search for mugs, we search cabinets near the coffee machine and for fruits we try the fridge.

Original: In this work, we focus on incorporating semantic priors in the task of semantic navigation.
Revised: In this work, we focus on incorporating semantic priors in the task of semantic navigation.

Original: We propose to use Graph Convolutional Networks for incorporating the prior knowledge into a deep reinforcement learning framework.
Revised: We propose to use Graph Convolutional Networks for incorporating the prior knowledge into a deep reinforcement learning framework.

Original: The agent uses the features from the knowledge graph to predict the actions.
Revised: The agent uses the features from the knowledge graph to predict the actions.

Original: For evaluation, we use the AI2-THOR framework.
Revised: For evaluation, we use the AI2-THOR framework.

Original: Our experiments show how semantic knowledge improves performance signiﬁcantly.
Revised: Our experiments show how semantic knowledge improves performance signiﬁcantly.

Original: More importantly, we show improvement in generalization to unseen scenes and/or objects.
Revised: More importantly, we show improvement in generalization to unseen scenes and/or objects.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Consider the kitchen scene shown in Figure 1(a) and the task of ﬁnding an object such as a mug.
Revised: Consider the kitchen scene shown in Figure 1(a) and the task of ﬁnding an object such as a mug.

Original: Even though we have never seen this particular kitchen before and no mug is visible in the scene, we can still infer the likely locations to ﬁnd the mug and create an exploration plan accordingly.
Revised: Even though we have never seen this particular kitchen before and no mug is visible in the scene, we can still infer the likely locations to ﬁnd the mug and create an exploration plan accordingly.

Original: For example, in Figure 1(a), we can infer that since there is a coffee machine, the mug is most likely in the cabinet near the coffee machine.
Revised: For example, in Figure 1(a), we can infer that since there is a coffee machine, the mug is most likely in the cabinet near the coffee machine.

Original: How do we do that?
Revised: How do we do that?

Original: We infer that mugs are usually used for coffee.
Revised: We infer that mugs are usually used for coffee.

Original: And since there is a coffee machine, the mug is likely to be near the machine due to functional efﬁciency.
Revised: And since there is a coffee machine, the mug is likely to be near the machine due to functional efﬁciency.

Original: We argue that humans use strong priors about the functional and semantic structure of the world to develop such efﬁcient navigation strategies.
Revised: We argue that humans use strong priors about the functional and semantic structure of the world to develop such efﬁcient navigation strategies.

Original: And how do we learn such functional/semantic priors?
Revised: And how do we learn such functional/semantic priors?

Original: Our prior experience and exploration of tens of kitchens help us to learn these priors.
Revised: Our prior experience and exploration of tens of kitchens help us to learn these priors.

Original: But these priors are not just used for navigating to known objects but also to completely unknown and unseen objects.
Revised: But these priors are not just used for navigating to known objects but also to completely unknown and unseen objects.

Original: For example, let us assume you have never seen a mango before and someone gives you a task of ﬁnding a mango in a new kitchen you have never seen before (let’s say Figure 1(b)).
Revised: For example, let us assume you have never seen a mango before and someone gives you a task of ﬁnding a mango in a new kitchen you have never seen before (let’s say Figure 1(b)).

Original: How would you do it?
Revised: How would you do it?

Original: Assuming you have searched for fruits like apples and grapes before, and you know mango is also a fruit; so a similar exploration strategy might apply.
Revised: Assuming you have searched for fruits like apples and grapes before, and you know mango is also a fruit; so a similar exploration strategy might apply.

Original: Therefore, in Figure 1(b), you are more likely to navigate to the fridge to search for a mango.
Revised: Therefore, in Figure 1(b), you are more likely to navigate to the fridge to search for a mango.

Original: Therefore, we use the semantic/functional priors to navigate to unseen objects as well.
Revised: Therefore, we use the semantic/functional priors to navigate to unseen objects as well.

Original: Inspired by these observations, in this paper, we explore how to exploit semantic priors for the task of semantic and goal-oriented navigation.
Revised: Inspired by these observations, in this paper, we explore how to exploit semantic priors for the task of semantic and goal-oriented navigation.

Original: In our visual navigation task, the environment, the path to the target, the target location, or the exact appearance of the target object can be unknown.
Revised: In our visual navigation task, the environment, the path to the target, the target location, or the exact appearance of the target object can be unknown.

Original: The prior knowledge about the semantic/functional structure of the world helps to improve the navigation.
Revised: The prior knowledge about the semantic/functional structure of the world helps to improve the navigation.

Original: We propose to use Graph Convolutional Networks (GCNs) (Kipf & Welling, 2017) to incorporate the prior knowledge into a Deep Reinforcement Learning framework.
Revised: We propose to use Graph Convolutional Networks (GCNs) (Kipf & Welling, 2017) to incorporate the prior knowledge into a Deep Reinforcement Learning framework.

Original: The knowledge of the agent is encoded in a graph.
Revised: The knowledge of the agent is encoded in a graph.

Original: GCNs allow arbitrary structured graphs to be encoded in an efﬁcient way.
Revised: GCNs allow arbitrary structured graphs to be encoded in an efﬁcient way.

Original: The knowledge is updated according to the current observation of the agent, which is speciﬁc to the current environment, and the knowledge at the previous time step or the prior knowledge.
Revised: The knowledge is updated according to the current observation of the agent, which is speciﬁc to the current environment, and the knowledge at the previous time step or the prior knowledge.

Original: The prior knowledge is obtained from large-scale datasets designed for scene understanding.
Revised: The prior knowledge is obtained from large-scale datasets designed for scene understanding.

Original: Our model is based on the actor-critic model (Mnih et al., 2016) that is augmented by the knowledge graph and object visibility information.
Revised: Our model is based on the actor-critic model (Mnih et al., 2016) that is augmented by the knowledge graph and object visibility information.

Original: To evaluate our model, we use the AI2-THOR framework (Kolve et al., 2017), which provides near photo-realistic customizable environments.
Revised: To evaluate our model, we use the AI2-THOR framework (Kolve et al., 2017), which provides near photo-realistic customizable environments.

Original: The agent can take navigation actions in these environments and observe the changes as a result of those actions.
Revised: The agent can take navigation actions in these environments and observe the changes as a result of those actions.

Original: AI2-THOR includes various objects that can be arranged in many different conﬁgurations.
Revised: AI2-THOR includes various objects that can be arranged in many different conﬁgurations.

Original: The agent location is randomized as well at each episode of training or testing.
Revised: The agent location is randomized as well at each episode of training or testing.

Original: Our experiments show that the semantic prior improves the performance of the baseline RL models signiﬁcantly.
Revised: Our experiments show that the semantic prior improves the performance of the baseline RL models signiﬁcantly.

Original: Furthermore, we show the results of the model on the challenging setting where the scene and/or the object are new to the agent.
Revised: Furthermore, we show the results of the model on the challenging setting where the scene and/or the object are new to the agent.

Original: Our contributions are summarized as follows: (1) We integrate a deep reinforcement learning model with knowledge graphs.
Revised: Our contributions are summarized as follows: (1) We integrate a deep reinforcement learning model with knowledge graphs.

Original: This allows the agent to encode any form of knowledge that can be represented by graph structures.
Revised: This allows the agent to encode any form of knowledge that can be represented by graph structures.

Original: (2) We show that semantic prior knowledge can signiﬁcantly improve the navigation performance.
Revised: (2) We show that semantic prior knowledge can signiﬁcantly improve the navigation performance.

Original: By considering the prior information and the semantics of the target objects, we improve generalization to unseen environments and novel target objects.
Revised: By considering the prior information and the semantics of the target objects, we improve generalization to unseen environments and novel target objects.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Semantic and goal-oriented navigation is one of the most prominent tasks that intelligent species perform in their daily life.
Revised: Semantic and goal-oriented navigation is one of the most prominent tasks that intelligent species perform in their daily life.

Original: There are several challenges involved in visual navigation.
Revised: There are several challenges involved in visual navigation.

Original: First, the environment might be unknown to the agent.
Revised: First, the environment might be unknown to the agent.

Original: In this situation, the agent requires to explore the environment to have a better understanding of that environment.
Revised: In this situation, the agent requires to explore the environment to have a better understanding of that environment.

Original: The second challenge is about the visibility of target objects.
Revised: The second challenge is about the visibility of target objects.

Original: The target object might not be visible when the agent starts the navigation or it might go out of the ﬁeld of view during navigation.
Revised: The target object might not be visible when the agent starts the navigation or it might go out of the ﬁeld of view during navigation.

Original: Hence, the agent needs to learn an efﬁcient search strategy to ﬁnd the target object.
Revised: Hence, the agent needs to learn an efﬁcient search strategy to ﬁnd the target object.

Original: The third challenge is related to planning.
Revised: The third challenge is related to planning.

Original: The object might be visible but planning a reasonable path towards the object is another issue that the agent needs to deal with.
Revised: The object might be visible but planning a reasonable path towards the object is another issue that the agent needs to deal with.

Original: There have been several efforts in the past to tackle these challenges which we describe below.
Revised: There have been several efforts in the past to tackle these challenges which we describe below.

Original: Geometry-based navigation.
Revised: Geometry-based navigation.

Original: Navigation methods can be divided into two main categories of geometry-based and learning-based.
Revised: Navigation methods can be divided into two main categories of geometry-based and learning-based.

Original: Most of the traditional navigation approaches fall into the former category, where it is assumed that either the map of the environment is known a priori, e.g.,
Revised: Most of the traditional navigation approaches fall into the former category, where it is assumed that either the map of the environment is known a priori, e.g.,

Original: Matthies & Shafer (1987); Borenstein & Koren (1991); Meng & Kak (1993); Kim & Nevatia (1999) or a map is built on the ﬂy e.g., Thrun (1998); Feder et al. (1999); Jones & Soatto (2011); Siagian et al.
Revised: Matthies & Shafer (1987); Borenstein & Koren (1991); Meng & Kak (1993); Kim & Nevatia (1999) or a map is built on the ﬂy e.g., Thrun (1998); Feder et al. (1999); Jones & Soatto (2011); Siagian et al.

Original: Our work is different from these approaches since we do not rely on a map for our navigation and we leverage semantic prior knowledge to reduce the required exploration time.
Revised: Our work is different from these approaches since we do not rely on a map for our navigation and we leverage semantic prior knowledge to reduce the required exploration time.

Original: Learning-based navigation.
Revised: Learning-based navigation.

Original: Recent success of deep learning and reinforcement learning has made learning-based navigation approaches more popular.
Revised: Recent success of deep learning and reinforcement learning has made learning-based navigation approaches more popular.

Original: Zhu et al.
Revised: Zhu et al.

Original: (2017) propose a deep RL-based navigation approach, where they provide the picture of the target object.
Revised: (2017) propose a deep RL-based navigation approach, where they provide the picture of the target object.

Original: In contrast, we only provide semantic labels to the agent, so we can show generalization to unseen scenes.
Revised: In contrast, we only provide semantic labels to the agent, so we can show generalization to unseen scenes.

Original: Gupta et al.
Revised: Gupta et al.

Original: (2017) propose a mapper and planner to output navigation actions.
Revised: (2017) propose a mapper and planner to output navigation actions.

Original: Mirowski et al.
Revised: Mirowski et al.

Original: (2017) also propose a navigation framework that optimizes a loss for auxiliary tasks such as depth prediction and loop closure classiﬁcation.
Revised: (2017) also propose a navigation framework that optimizes a loss for auxiliary tasks such as depth prediction and loop closure classiﬁcation.

Original: Sadeghi & Levine (2017) propose an RL-based approach for collision avoidance.
Revised: Sadeghi & Levine (2017) propose an RL-based approach for collision avoidance.

Original: Brahmbhatt & Hays (2017) explore a CNN-based approach for navigating in cities using local observations of the streets.
Revised: Brahmbhatt & Hays (2017) explore a CNN-based approach for navigating in cities using local observations of the streets.

Original: Wu & Tian (2017) combine deep RL with curriculum learning in a ﬁrst-person shooting game setting.
Revised: Wu & Tian (2017) combine deep RL with curriculum learning in a ﬁrst-person shooting game setting.

Original: Savinov et al.
Revised: Savinov et al.

Original: (2018) introduce a topological landmark-based memory for navigation.
Revised: (2018) introduce a topological landmark-based memory for navigation.

Original: Kahn et al.
Revised: Kahn et al.

Original: (2018) propose a method based on model-free and model-based
Revised: (2018) propose a method based on model-free and model-based

Original: RL to learn navigation policies using a few samples.
Revised: RL to learn navigation policies using a few samples.

Original: Mousavian et al.
Revised: Mousavian et al.

Original: (2018) use object detection and semantic segmentation to better navigate in unseen environments.
Revised: (2018) use object detection and semantic segmentation to better navigate in unseen environments.

Original: Chen et al.
Revised: Chen et al.

Original: (2015) directly map the input image to an action in an autonomous driving setting.
Revised: (2015) directly map the input image to an action in an autonomous driving setting.

Original: There is also a large body of work that address visually grounded navigation instructions e.g., Anderson et al.
Revised: There is also a large body of work that address visually grounded navigation instructions e.g., Anderson et al.

Original: (2018b); Chaplot et al.
Revised: (2018b); Chaplot et al.

Original: ; Hermann et al.
Revised: ; Hermann et al.

Original: Yu et al.
Revised: Yu et al.

Original: ; Misra et al.
Revised: ; Misra et al.

Original: Mei et al. (2016).
Revised: Mei et al. (2016).

Original: In contrast to all these approaches, we incorporate semantic and functional priors to improve navigation performance and better generalize to unseen scenes and objects.
Revised: In contrast to all these approaches, we incorporate semantic and functional priors to improve navigation performance and better generalize to unseen scenes and objects.

Original: Context and scene prior.
Revised: Context and scene prior.

Original: Contextual reasoning has been studied extensively in the computer vision literature (Torralba et al., 2003; Hoiem et al., 2005; Rabinovich et al., 2005; Divvala et al., 2009;
Revised: Contextual reasoning has been studied extensively in the computer vision literature (Torralba et al., 2003; Hoiem et al., 2005; Rabinovich et al., 2005; Divvala et al., 2009;

Original: Desai et al., 2009; Marszalek et al., 2009; Malisiewicz & Efros, 2009; Mottaghi et al., 2014; Zhu et al., 2015; Shrivastava & Gupta, 2016).
Revised: Desai et al., 2009; Marszalek et al., 2009; Malisiewicz & Efros, 2009; Mottaghi et al., 2014; Zhu et al., 2015; Shrivastava & Gupta, 2016).

Original: However, contextual information is mainly used for static settings such as object detection, semantic segmentation or action recognition.
Revised: However, contextual information is mainly used for static settings such as object detection, semantic segmentation or action recognition.

Original: We use contextual reasoning for an interactive navigation task, where the agent updates its belief based on the current observation and the prior knowledge as it moves in the environment.
Revised: We use contextual reasoning for an interactive navigation task, where the agent updates its belief based on the current observation and the prior knowledge as it moves in the environment.

Original: Object relationships have been used for tasks such as image retrieval (Johnson et al., 2015), visual relation detection (Zhang et al., 2017), referring expressions (Nagaraja et al., 2016;
Revised: Object relationships have been used for tasks such as image retrieval (Johnson et al., 2015), visual relation detection (Zhang et al., 2017), referring expressions (Nagaraja et al., 2016;

Original: Hu et al., 2017), and visual question answering (Johnson et al., 2017).
Revised: Hu et al., 2017), and visual question answering (Johnson et al., 2017).

Original: Knowledge graphs.
Revised: Knowledge graphs.

Original: There are recent works that use knowledge graphs for computer vision problems.
Revised: There are recent works that use knowledge graphs for computer vision problems.

Original: A knowledge graph is used by Marino et al.
Revised: A knowledge graph is used by Marino et al.

Original: (2017) for image classiﬁcation, by Li et al.
Revised: (2017) for image classiﬁcation, by Li et al.

Original: (2017) for situation recognition and by Wang et al.
Revised: (2017) for situation recognition and by Wang et al.

Original: (2018) for zero-shot recognition.
Revised: (2018) for zero-shot recognition.

Original: We use knowledge graphs in an RL setting for the interactive task of visual navigation.
Revised: We use knowledge graphs in an RL setting for the interactive task of visual navigation.

Original: Reasoning about unknown environments or objects.
Revised: Reasoning about unknown environments or objects.

Original: Various works have explored zero-shot reasoning in the context of reinforcement learning.
Revised: Various works have explored zero-shot reasoning in the context of reinforcement learning.

Original: Yu et al.
Revised: Yu et al.

Original: (2018) address the problem of learning language in a 2D maze, where they can handle unseen word combinations or new sentences that contain unseen words.
Revised: (2018) address the problem of learning language in a 2D maze, where they can handle unseen word combinations or new sentences that contain unseen words.

Original: Harrison et al.
Revised: Harrison et al.

Original: ; Higgins et al.
Revised: ; Higgins et al.

Original: (2017) study zero-shot policy transfer in the scenarios that the dynamics or the states of the target domain is different from those of the source domain.
Revised: (2017) study zero-shot policy transfer in the scenarios that the dynamics or the states of the target domain is different from those of the source domain.

Original: Pathak et al.
Revised: Pathak et al.

Original: (2018) propose a zero-shot imitation learning approach where the expert demonstration for a particular task is never seen.
Revised: (2018) propose a zero-shot imitation learning approach where the expert demonstration for a particular task is never seen.

Original: Oh et al.
Revised: Oh et al.

Original: (2017) address generalization of RL to unseen instructions and longer instructions.
Revised: (2017) address generalization of RL to unseen instructions and longer instructions.

Original: Our problem is different since we address navigation to novel objects or navigating in unseen scenes using scene priors .
Revised: Our problem is different since we address navigation to novel objects or navigating in unseen scenes using scene priors .

Original:  V ISUAL S EMANTIC N AVIGATION
Revised:  V ISUAL S EMANTIC N AVIGATION

Original: In this section, we ﬁrst deﬁne the task of visual semantic navigation.
Revised: In this section, we ﬁrst deﬁne the task of visual semantic navigation.

Original: We then describe the formulation using deep reinforcement learning and the baseline model for the task.
Revised: We then describe the formulation using deep reinforcement learning and the baseline model for the task.

Original: 3. T ASK D EFINITION
Revised: 3. T ASK D EFINITION

Original: Our goal is to navigate from a random starting location in a scene to a speciﬁed target object category given only egocentric RGB perception of the agent.
Revised: Our goal is to navigate from a random starting location in a scene to a speciﬁed target object category given only egocentric RGB perception of the agent.

Original: The target object category is speciﬁed by a semantic label, thus we call our task visual semantic navigation.
Revised: The target object category is speciﬁed by a semantic label, thus we call our task visual semantic navigation.

Original: The task is considered successful if an instance of the target object category is visible.
Revised: The task is considered successful if an instance of the target object category is visible.

Original: By “visible”, we mean the target object is in the ﬁeld of view and within a threshold of distance.
Revised: By “visible”, we mean the target object is in the ﬁeld of view and within a threshold of distance.

Original: 3. T HE B ASELINE M ODEL
Revised: 3. T HE B ASELINE M ODEL

Original: We formulate the visual semantic navigation using a deep reinforcement learning framework.
Revised: We formulate the visual semantic navigation using a deep reinforcement learning framework.

Original: Given a semantic task objective g , the agent perceives a state s t (i.e., the egocentric RGB image from the current location and orientation) at the time step t and samples an action a t from the set of possible actions A according to its policy π .
Revised: Given a semantic task objective g , the agent perceives a state s t (i.e., the egocentric RGB image from the current location and orientation) at the time step t and samples an action a t from the set of possible actions A according to its policy π .

Original: We approximate the policy by a deep policy network π ( · ; θ ) : a
Revised: We approximate the policy by a deep policy network π ( · ; θ ) : a

Original: [Equation]
Revised: [Equation]

Original: [Equation] 
Revised: [Equation] [Figure]

Revised: n e x t t o

Original: [Figure]
Revised: [Figure]

Revised: u, v, and θ and two φ ( · ; u ) and ψ ( · ; v ) to map

Original: Visual network.
Revised: Visual network.

Original: As illustrated in Figure 2 (top), the visual network takes 224 × 224 RGB images as input and generates a 512-d feature vector as output.
Revised: As illustrated in Figure 2 (top), the visual network takes 224 × 224 RGB images as input and generates a 512-d feature vector as output.

Original: The backbone of the visual branch is ResNet50 (He et al., 2016) pre-trained on ImageNet.
Revised: The backbone of the visual branch is ResNet50 (He et al., 2016) pre-trained on ImageNet.

Original: Speciﬁcally, we extract the 2048-d feature after the global average pooling of ResNet-50.
Revised: Speciﬁcally, we extract the 2048-d feature after the global average pooling of ResNet-50.

Original: To account for the history of the actions taken by the agent, we concatenate the features of the current frame and three past observations, which results in a 8192-d feature vector.
Revised: To account for the history of the actions taken by the agent, we concatenate the features of the current frame and three past observations, which results in a 8192-d feature vector.

Original: We then add a fully connected layer and a ReLU layer to map the concatenated image feature into the 512-d visual-semantic feature.
Revised: We then add a fully connected layer and a ReLU layer to map the concatenated image feature into the 512-d visual-semantic feature.

Original: Semantic network.
Revised: Semantic network.

Original: The semantic task objective is described by an object category, e.g., Microwave or Television .
Revised: The semantic task objective is described by an object category, e.g., Microwave or Television .

Original: We use fastText (Joulin et al., 2016) to compute a 100-d embedding for each word.
Revised: We use fastText (Joulin et al., 2016) to compute a 100-d embedding for each word.

Original: Then we map the word embedding into a 512-d feature by a fully connected layer and ReLU, as illustrated in Figure 2 (middle).
Revised: Then we map the word embedding into a 512-d feature by a fully connected layer and ReLU, as illustrated in Figure 2 (middle).

Original: Actor-Critic policy network.
Revised: Actor-Critic policy network.

Original: We employ the Asynchronous Advantage Actor-Critic (A3C) (Mnih et al., 2016) model to predict the policy at each time step.
Revised: We employ the Asynchronous Advantage Actor-Critic (A3C) (Mnih et al., 2016) model to predict the policy at each time step.

Original: The input of our A3C model is the joint representation of the current state and the semantic task objective, which is a 1024-d feature vector made by concatenating the outputs of the visual network and the semantic network.
Revised: The input of our A3C model is the joint representation of the current state and the semantic task objective, which is a 1024-d feature vector made by concatenating the outputs of the visual network and the semantic network.

Original: The A3C model generates two outputs, i.e., the policy and the value.
Revised: The A3C model generates two outputs, i.e., the policy and the value.

Original: We sample the action from the predicted policy.
Revised: We sample the action from the predicted policy.

Original: Our implementation of the A3C model consists of three layers: the input, the hidden layer, and the outputs.
Revised: Our implementation of the A3C model consists of three layers: the input, the hidden layer, and the outputs.

Original: The hidden layer is a fully connected layer followed by the ReLU activation layer which maps the fused input into a 512-d latent space.
Revised: The hidden layer is a fully connected layer followed by the ReLU activation layer which maps the fused input into a 512-d latent space.

Original: Then the |A| dimensional policy and the value are generated by two branches of network, as shown in Figure 2.
Revised: Then the |A| dimensional policy and the value are generated by two branches of network, as shown in Figure 2.

Original: Unlike previous work (e.g., Zhu et al.
Revised: Unlike previous work (e.g., Zhu et al.

Original: (2017)) which uses different policy networks for different scenes, we use a single policy network for different scene examples.
Revised: (2017)) which uses different policy networks for different scenes, we use a single policy network for different scene examples.

Original: This makes our model more compact and generalizable.
Revised: This makes our model more compact and generalizable.

Original: Reward.
Revised: Reward.

Original: We consider a reward to minimize the trajectory length to the targets: If any object instance from the target object category is reached within a certain number of steps, the agent receives a large positive reward 10.0.
Revised: We consider a reward to minimize the trajectory length to the targets: If any object instance from the target object category is reached within a certain number of steps, the agent receives a large positive reward 10.0.

Original: Otherwise, we penalize each step with a small negative reward -0.01.
Revised: Otherwise, we penalize each step with a small negative reward -0.01.

Original: The design of the reward function is also affected by the types of actions A .
Revised: The design of the reward function is also affected by the types of actions A .

Original: In our experiments, we ablate two sets of actions A with or without the stop action.
Revised: In our experiments, we ablate two sets of actions A with or without the stop action.

Original: In the setting without the stop action, the agent will receive the positive reward if the environment notiﬁes it when it reaches the target, which also ends an episode of training.
Revised: In the setting without the stop action, the agent will receive the positive reward if the environment notiﬁes it when it reaches the target, which also ends an episode of training.

Original: In the setting with the stop action, the episode is terminated when the stop action is executed, and the positive reward will be provided only if the agent is within the threshold of distance from the target (1 meter in our experiments) and facing the target.
Revised: In the setting with the stop action, the episode is terminated when the stop action is executed, and the positive reward will be provided only if the agent is within the threshold of distance from the target (1 meter in our experiments) and facing the target.

Original: This makes the task much more challenging.
Revised: This makes the task much more challenging.

Original: [Figure]
Revised: [Figure]

Original: n e x t t o n e x t t o / o n
Revised: n e x t t o n e x t t o / o n

Original: TV
Revised: TV

Original: Table
Revised: Table

Original: [Figure]
Revised: [Figure]

Original: Painting n e x t t
Revised: Painting n e x t t

Original: o
Revised: o

Original: o n
Revised: o n

Original:  G ENERALIZATION WITH G RAPH C ONVOLUTIONAL N ETWORKS
Revised:  G ENERALIZATION WITH G RAPH C ONVOLUTIONAL N ETWORKS

Original: Our goal in this paper is to incorporate semantic knowledge into a Reinforcement Learning framework.
Revised: Our goal in this paper is to incorporate semantic knowledge into a Reinforcement Learning framework.

Original: To this end, we incorporate semantic knowledge in the form of graph representation and use
Revised: To this end, we incorporate semantic knowledge in the form of graph representation and use

Original: Graph Convolutional Networks (GCNs) (Kipf & Welling, 2017) to compute relational features on the graph.
Revised: Graph Convolutional Networks (GCNs) (Kipf & Welling, 2017) to compute relational features on the graph.

Original: GCNs allow us to incorporate prior knowledge and dynamically update it as the agent receives information speciﬁc to the current environment.
Revised: GCNs allow us to incorporate prior knowledge and dynamically update it as the agent receives information speciﬁc to the current environment.

Original: We ﬁrst brieﬂy describe how we build a semantic knowledge graph to represent the priors.
Revised: We ﬁrst brieﬂy describe how we build a semantic knowledge graph to represent the priors.

Original: We then provide the background for GCNs.
Revised: We then provide the background for GCNs.

Original: Finally, we delve into the details of how we incorporate GCNs for the task of visual semantic navigation and how it helps generalization to unseen scenes and novel object categories.
Revised: Finally, we delve into the details of how we incorporate GCNs for the task of visual semantic navigation and how it helps generalization to unseen scenes and novel object categories.

Original: 4. K NOWLEDGE G RAPH C ONSTRUCTION
Revised: 4. K NOWLEDGE G RAPH C ONSTRUCTION

Original: Our knowledge graph for visual navigation provides two main advantages: (1) It encodes spatial relationships between different object categories.
Revised: Our knowledge graph for visual navigation provides two main advantages: (1) It encodes spatial relationships between different object categories.

Original: (2) It provides the spatial and visual relationships between the known objects and novel categories in cases that we have not seen any visual examples of the novel categories.
Revised: (2) It provides the spatial and visual relationships between the known objects and novel categories in cases that we have not seen any visual examples of the novel categories.

Original: We denote our knowledge graph by G =
Revised: We denote our knowledge graph by G =

Original: ( V, E ) , where V and E denote the nodes and the edges between nodes, respectively.
Revised: ( V, E ) , where V and E denote the nodes and the edges between nodes, respectively.

Original: Speciﬁcally, each node v ∈ V denotes an object category, and each edge e ∈ E denotes a relationship between a pair of object categories.
Revised: Speciﬁcally, each node v ∈ V denotes an object category, and each edge e ∈ E denotes a relationship between a pair of object categories.

Original: We use the Visual Genome (Krishna et al., 2017) dataset as a source to build the knowledge graph.
Revised: We use the Visual Genome (Krishna et al., 2017) dataset as a source to build the knowledge graph.

Original: Visual Genome consists of over 100K natural images.
Revised: Visual Genome consists of over 100K natural images.

Original: Each image is annotated with objects, attributes and the relationships between objects.
Revised: Each image is annotated with objects, attributes and the relationships between objects.

Original: Since there is no predeﬁned object category list, the annotators are free to label any objects in the image, which results in very diverse object categories.
Revised: Since there is no predeﬁned object category list, the annotators are free to label any objects in the image, which results in very diverse object categories.

Original: In our experiments, we build the knowledge graph by including all object categories that appear in the AI2-THOR environment.
Revised: In our experiments, we build the knowledge graph by including all object categories that appear in the AI2-THOR environment.

Original: Each object category is represented as a node in the graph.
Revised: Each object category is represented as a node in the graph.

Original: We count the occurrence of object-to-object relationships in the Visual Genome dataset.
Revised: We count the occurrence of object-to-object relationships in the Visual Genome dataset.

Original: Two nodes are connected with an edge only when the occurrence frequency of any relationship is more than three.
Revised: Two nodes are connected with an edge only when the occurrence frequency of any relationship is more than three.

Original: Some examples of the mined relationships are shown in Figure 3.
Revised: Some examples of the mined relationships are shown in Figure 3.

Original: 4. I NCORPORATING S EMANTIC K NOWLEDGE INTO A CTOR -C RITIC M ODEL
Revised: 4. I NCORPORATING S EMANTIC K NOWLEDGE INTO A CTOR -C RITIC M ODEL

Original: The baseline policy model decides the action using the current state and target object features.
Revised: The baseline policy model decides the action using the current state and target object features.

Original: However, we want the policy network to incorporate semantic knowledge of the world when planning the actions.
Revised: However, we want the policy network to incorporate semantic knowledge of the world when planning the actions.

Original: How do we represent the semantic knowledge?
Revised: How do we represent the semantic knowledge?

Original: More importantly, how do we extract semantic knowledge in the context of the current environment and state?
Revised: More importantly, how do we extract semantic knowledge in the context of the current environment and state?

Original: Our core idea is that the graph structure represents how the information propagates between different nodes.
Revised: Our core idea is that the graph structure represents how the information propagates between different nodes.

Original: We initialize each node based on the current state (input scene image) and then perform information propagation to compute a semantic knowledge vector that is passed as another feature vector to the policy function.
Revised: We initialize each node based on the current state (input scene image) and then perform information propagation to compute a semantic knowledge vector that is passed as another feature vector to the policy function.

Original: For information propagation, we use the recently proposed Graph
Revised: For information propagation, we use the recently proposed Graph

Original: Convolutional Network (GCN) (Kipf & Welling, 2017).
Revised: Convolutional Network (GCN) (Kipf & Welling, 2017).

Original: [Figure]
Revised: [Figure]

Original: c l a ss s c
Revised: c l a ss s c

Original: o r e
Revised: o r e

Original: R e s N e t
Revised: R e s N e t

Original: [Figure]
Revised: [Figure]

Original: 4.2. G RAPH C ONVOLUTIONAL N ETWORK (GCN)
Revised: 4.2. G RAPH C ONVOLUTIONAL N ETWORK (GCN)

Original: The GCNs are the extension of the Convolution Neural Networks to graph structures, where the goal is to learn a function representation for a given graph G =
Revised: The GCNs are the extension of the Convolution Neural Networks to graph structures, where the goal is to learn a function representation for a given graph G =

Original: ( V, E ) .
Revised: ( V, E ) .

Original: The input to each node v is a feature vector x v .
Revised: The input to each node v is a feature vector x v .

Original: We summarize the inputs of all nodes as a matrix X =
Revised: We summarize the inputs of all nodes as a matrix X =

Original: [ x 1 , · · · , x | V | ] ∈ R | V |× D , where D denotes the dimension of the input feature.
Revised: [ x 1 , · · · , x | V | ] ∈ R | V |× D , where D denotes the dimension of the input feature.

Original: The graph structure is represented as a binary adjacency matrix A .
Revised: The graph structure is represented as a binary adjacency matrix A .

Original: We perform normalization on A following (Kipf & Welling, 2017) and obtain (cid:98) A .
Revised: We perform normalization on A following (Kipf & Welling, 2017) and obtain (cid:98) A .

Original: The GCN outputs a node-level representation Z =
Revised: The GCN outputs a node-level representation Z =

Original: [ z 1 , · · · , z | V | ] ∈ R | V |× F .
Revised: [ z 1 , · · · , z | V | ] ∈ R | V |× F .

Original: Let f ( · ) denote the
Revised: Let f ( · ) denote the

Original: ReLU activation function, we have
Revised: ReLU activation function, we have

Original: [Equation]
Revised: [Equation]

Original: with H (0) =
Revised: with H (0) =

Original: X and H ( L )
Revised: X and H ( L )

Original: =
Revised: =

Original: Z , where W ( l ) is the parameter for the l -th layer and L is the number of GCN layers.
Revised: Z , where W ( l ) is the parameter for the l -th layer and L is the number of GCN layers.

Original: 4.2. GCN FOR N AVIGATION
Revised: 4.2. GCN FOR N AVIGATION

Original: In our visual semantic navigation task, the input of each node is designed as a joint representation of both the semantic cues (e.g., the word embedding) and the visual cues (e.g., the image classiﬁcation score depending on the current state s t ).
Revised: In our visual semantic navigation task, the input of each node is designed as a joint representation of both the semantic cues (e.g., the word embedding) and the visual cues (e.g., the image classiﬁcation score depending on the current state s t ).

Original: Speciﬁcally, the word embedding is generated by fastText (Joulin et al., 2016) and the classiﬁcation score is generated by a ResNet-50 (He et al., 2016) pretrained on the 1000-class ImageNet dataset.
Revised: Speciﬁcally, the word embedding is generated by fastText (Joulin et al., 2016) and the classiﬁcation score is generated by a ResNet-50 (He et al., 2016) pretrained on the 1000-class ImageNet dataset.

Original: Note that the classiﬁcation score is obtained based on the frame of the current state and we have different word embeddings for different graph nodes.
Revised: Note that the classiﬁcation score is obtained based on the frame of the current state and we have different word embeddings for different graph nodes.

Original: These two representations are ﬁrst mapped to 512-d features by two different fully connected layers respectively.
Revised: These two representations are ﬁrst mapped to 512-d features by two different fully connected layers respectively.

Original: We then concatenate these two features and form a 1024-d joint representation for each graph node.
Revised: We then concatenate these two features and form a 1024-d joint representation for each graph node.

Original: As illustrated in Figure 4, we use three layers of GCN, the ﬁrst two layers output 1024 dimensional latent features, the last layer outputs a single value for each node which results in a | V | dimensional feature vector.
Revised: As illustrated in Figure 4, we use three layers of GCN, the ﬁrst two layers output 1024 dimensional latent features, the last layer outputs a single value for each node which results in a | V | dimensional feature vector.

Original: This feature vector is basically an encoding of semantic prior in the context of the current scene and environment.
Revised: This feature vector is basically an encoding of semantic prior in the context of the current scene and environment.

Original: Finally, we map this feature vector into the 512-d feature embedding and concatenate it with the features generated from the visual and semantic branches (1024-d embedding), which results in a 1536-d feature vector.
Revised: Finally, we map this feature vector into the 512-d feature embedding and concatenate it with the features generated from the visual and semantic branches (1024-d embedding), which results in a 1536-d feature vector.

Original: As illustrated in Figure 2, the joint feature is further fed into the policy network for policy prediction.
Revised: As illustrated in Figure 2, the joint feature is further fed into the policy network for policy prediction.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we provide the results of navigation using GCNs.
Revised: In this section, we provide the results of navigation using GCNs.

Original: We evaluate our model in scenarios where the scenes are unseen and/or the target objects are novel to the agent.
Revised: We evaluate our model in scenarios where the scenes are unseen and/or the target objects are novel to the agent.

Original: We also provide ablation results that show the knowledge graph is useful.
Revised: We also provide ablation results that show the knowledge graph is useful.

Original: 5. E VALUATION F RAMEWORK
Revised: 5. E VALUATION F RAMEWORK

Original: We evaluate our method in the interactive environments of AI2-THOR (Kolve et al., 2017).
Revised: We evaluate our method in the interactive environments of AI2-THOR (Kolve et al., 2017).

Original: AI2THOR provides 120 scenes covering four different room categories: kitchens , living rooms , bedrooms , and bathrooms .
Revised: AI2THOR provides 120 scenes covering four different room categories: kitchens , living rooms , bedrooms , and bathrooms .

Original: Each room category consists of 30 rooms with diverse appearance and conﬁgurations.
Revised: Each room category consists of 30 rooms with diverse appearance and conﬁgurations.

Original: We randomly split the scenes into three splits for each room category, i.e., 20 training rooms, 5 validation rooms, and 5 testing rooms.
Revised: We randomly split the scenes into three splits for each room category, i.e., 20 training rooms, 5 validation rooms, and 5 testing rooms.

Original: [Table]
Revised: [Table]

Original: There are 87 object categories within AI2-THOR that are common among the scenes.
Revised: There are 87 object categories within AI2-THOR that are common among the scenes.

Original: However, some of the objects are not visible without interaction.
Revised: However, some of the objects are not visible without interaction.

Original: For example, spoons were not visible since they always appeared in closed drawers during random initialization of the scenes so we did not use spoon among our categories.
Revised: For example, spoons were not visible since they always appeared in closed drawers during random initialization of the scenes so we did not use spoon among our categories.

Original: Therefore, we have | V | = 53 categories based on their visibility at random initialization of the scenes.
Revised: Therefore, we have | V | = 53 categories based on their visibility at random initialization of the scenes.

Original: To test the generalization ability of our method on novel objects, we split the 53 object categories into known and novel sets.
Revised: To test the generalization ability of our method on novel objects, we split the 53 object categories into known and novel sets.

Original: Only the known set of object categories are used in training.
Revised: Only the known set of object categories are used in training.

Original: The full split of object categories is shown in Appendix A. We only use navigation commands of AI2-THOR for our experiments.
Revised: The full split of object categories is shown in Appendix A. We only use navigation commands of AI2-THOR for our experiments.

Original: These actions include: move forward , move back , rotate right , rotate left , and stop .
Revised: These actions include: move forward , move back , rotate right , rotate left , and stop .

Original: We evaluate the models based on two metrics: Success Rate and the Success weighted by Path
Revised: We evaluate the models based on two metrics: Success Rate and the Success weighted by Path

Original: Length (SPL) metric recently proposed by Anderson et al.
Revised: Length (SPL) metric recently proposed by Anderson et al.

Original: (2018a).
Revised: (2018a).

Original: Success Rate is deﬁned as the ratio of the number of times the agent successfully navigates to the target and the total number of episodes.
Revised: Success Rate is deﬁned as the ratio of the number of times the agent successfully navigates to the target and the total number of episodes.

Original: SPL is a better metric which is a function considering both Success Rate and the path length to reach the goal from the starting point.
Revised: SPL is a better metric which is a function considering both Success Rate and the path length to reach the goal from the starting point.

Original: It is deﬁned as 1 N (cid:80) Ni =1 S
Revised: It is deﬁned as 1 N (cid:80) Ni =1 S

Original: i
Revised: i

Original: L i max( P i ,L i ) , where N is the number of episodes, S
Revised: L i max( P i ,L i ) , where N is the number of episodes, S

Original: i is a binary indicator of success in episode i , P i represents path length and L i is the shortest path distance (provided by the environment for evaluation) in episode i .
Revised: i is a binary indicator of success in episode i , P i represents path length and L i is the shortest path distance (provided by the environment for evaluation) in episode i .

Original: 5. R ESULTS
Revised: 5. R ESULTS

Original: We train each of the models three times with different random initializations.
Revised: We train each of the models three times with different random initializations.

Original: We show the training curves in Appendix B, where we plot the curves with error bands representing the standard deviation.
Revised: We show the training curves in Appendix B, where we plot the curves with error bands representing the standard deviation.

Original: The curves show that our proposed model converges in fewer training episodes compared to baseline and achieves better Success Rate as well as SPL , which shows the effectiveness of scene priors.
Revised: The curves show that our proposed model converges in fewer training episodes compared to baseline and achieves better Success Rate as well as SPL , which shows the effectiveness of scene priors.

Original: For evaluation, we run 250 episodes for each scene, where the initial location and orientation of the agent is randomized.
Revised: For evaluation, we run 250 episodes for each scene, where the initial location and orientation of the agent is randomized.

Original: The target object is randomly sampled for each episode.
Revised: The target object is randomly sampled for each episode.

Original: We select the models which perform best on the validation set for all methods and evaluate them on the test set.
Revised: We select the models which perform best on the validation set for all methods and evaluate them on the test set.

Original: We compare the performance of the following models: (1) Random walk , which is the simplest baseline for navigation.
Revised: We compare the performance of the following models: (1) Random walk , which is the simplest baseline for navigation.

Original: The agent randomly samples an action from the action space at each step.
Revised: The agent randomly samples an action from the action space at each step.

Original: (2) A3C (Mnih et al., 2016) , which refers to the baseline model presented in Section 3.2.
Revised: (2) A3C (Mnih et al., 2016) , which refers to the baseline model presented in Section 3.2.

Original: It is a state-of-the-art deep reinforcement learning model.
Revised: It is a state-of-the-art deep reinforcement learning model.

Original: (3) Ours , which is our proposed model.
Revised: (3) Ours , which is our proposed model.

Original: Each node of the ﬁrst layer of GCNs is fed by a joint representation of the word embedding and the image classiﬁcation scores extracted by ResNet-50, which depends on the current observed image.
Revised: Each node of the ﬁrst layer of GCNs is fed by a joint representation of the word embedding and the image classiﬁcation scores extracted by ResNet-50, which depends on the current observed image.

Original: We analyze the generalization ability of our method for unseen scenes and novel objects.
Revised: We analyze the generalization ability of our method for unseen scenes and novel objects.

Original: Specifically, there are three experimental settings: 1) test on seen scenes with novel object categories as the navigation target; 2) test on unseen scenes with known object categories; and 3) test on unseen scenes with novel object categories.
Revised: Specifically, there are three experimental settings: 1) test on seen scenes with novel object categories as the navigation target; 2) test on unseen scenes with known object categories; and 3) test on unseen scenes with novel object categories.

Original: Table 1 shows the results for these different settings.
Revised: Table 1 shows the results for these different settings.

Original: In addition to the above settings, we also provide the results for seen scenes and known objects in the ﬁrst row of the table.
Revised: In addition to the above settings, we also provide the results for seen scenes and known objects in the ﬁrst row of the table.

Original: Note that most previous work (e.g., Zhu et al.
Revised: Note that most previous work (e.g., Zhu et al.

Original: (2017)) assume the environment notiﬁes the agent when it reaches the target, and the agent does not have any idea if it has reached the target or not.
Revised: (2017)) assume the environment notiﬁes the agent when it reaches the target, and the agent does not have any idea if it has reached the target or not.

Original: In contrast, we consider the stop action and expect the agent to issue this action when it reaches the target.
Revised: In contrast, we consider the stop action and expect the agent to issue this action when it reaches the target.

Original: As mentioned in Section 3.2, this makes the learning challenging.
Revised: As mentioned in Section 3.2, this makes the learning challenging.

Original: In Table 2, we report the results for the simpler case where we remove the “stop” action from the list of actions.
Revised: In Table 2, we report the results for the simpler case where we remove the “stop” action from the list of actions.

Original: [Table]
Revised: [Table]

Original: Our method that incorporates the knowledge graph outperforms the baselines in terms of both success rate and SPL.
Revised: Our method that incorporates the knowledge graph outperforms the baselines in terms of both success rate and SPL.

Original: We observe a higher performance for the case that we do not use a stop action (Table 2), which is expected.
Revised: We observe a higher performance for the case that we do not use a stop action (Table 2), which is expected.

Original: The scenario in which both scenes and target objects are novel is quite challenging, and the performance degrades drastically for both A3C and our method.
Revised: The scenario in which both scenes and target objects are novel is quite challenging, and the performance degrades drastically for both A3C and our method.

Original: However, the performance is signiﬁcantly better than random.
Revised: However, the performance is signiﬁcantly better than random.

Original: The bathroom scenes are typically small so there is not much difference between the performance of our method and the baseline.
Revised: The bathroom scenes are typically small so there is not much difference between the performance of our method and the baseline.

Original: Note that more than half of the object categories are not among ImageNet categories.
Revised: Note that more than half of the object categories are not among ImageNet categories.

Original: Also, note that “Unseen scenes, Novel objects” is not necessarily the hardest case.
Revised: Also, note that “Unseen scenes, Novel objects” is not necessarily the hardest case.

Original: For instance, in “Seen scenes, Novel objects”, the appearance of the object and the mapping between the name and the object appearance are still unknown.
Revised: For instance, in “Seen scenes, Novel objects”, the appearance of the object and the mapping between the name and the object appearance are still unknown.

Original: We also observe overﬁtting to known scenes and objects (refer to “Seen scenes,
Revised: We also observe overﬁtting to known scenes and objects (refer to “Seen scenes,

Original: Known objects”).
Revised: Known objects”).

Original: So the results of different cases are not directly comparable, and it depends on the structure of the scenes and the conﬁguration of objects.
Revised: So the results of different cases are not directly comparable, and it depends on the structure of the scenes and the conﬁguration of objects.

Original: We show some qualitative examples in
Revised: We show some qualitative examples in

Original: Appendix D, and the implementation details are provided in Appendix C.
Revised: Appendix D, and the implementation details are provided in Appendix C.

Revised: Generalization Across Scene Types.

Revised: We evaluate generalization across scene types as well.

Revised: The idea is that we train the model on one scene type and evaluate it on a different scene type.

Revised: The result is close to random in the scenario with the termination action.

Original:  Ablations on Knowledge Graph.
Revised: This is expected since there are very few common objects among different scene categories. The result for the simpler case of without the termination action is shown in Table 3. [Table] Ablations on Knowledge Graph.

Original: We perform evaluations on how the performance is affected by changing the knowledge graph in our model.
Revised: We perform evaluations on how the performance is affected by changing the knowledge graph in our model.

Original: The experiment is performed with the kitchen scenes without the “stop” action.
Revised: The experiment is performed with the kitchen scenes without the “stop” action.

Original: We ﬁrst remove different fractions of object nodes or relations from the graph and re-train the models.
Revised: We ﬁrst remove different fractions of object nodes or relations from the graph and re-train the models.

Original: As shown in Table 3, the SPL performance drops as more information is removed from the knowledge graph.
Revised: As shown in Table 4, the SPL performance drops as more information is removed from the knowledge graph.

Original: We also train our model with a fully-connected graph which leads to the SPL of 32 . 5 and the model with a random graph leads to the SPL of 30 . 1 ± 0 .
Revised: We also train our model with a fully-connected graph which leads to the SPL of 32 . 5 and the model with a random graph leads to the SPL of 30 . 1 ± 0 .

Original: 6 (we repeated this experiment three times).
Revised: 6 (we repeated this experiment three times).

Original: The performance of these two cases is worse than the performance of the model with a proper knowledge graph ( 38 . 5 ).
Revised: The performance of these two cases is worse than the performance of the model with a proper knowledge graph ( 38 . 5 ).

Original: [Table]
Revised: [Table]

Original: We have tried using the edge types (“on”, “next to”, etc.), but the results is not better than the case that we ignore the edge types.
Revised: We have tried using the edge types (“on”, “next to”, etc.), but the results is not better than the case that we ignore the edge types.

Original: That is probably due to the lack of training data for each type separately.
Revised: That is probably due to the lack of training data for each type separately.

Original: We have also tried training only one model for all scene categories, but the performance is lower.
Revised: We have also tried training only one model for all scene categories, but the performance is lower.

Original: Computation Cost.
Revised: Computation Cost.

Original: It is worth mentioning that the GCN module in our model increases only 0 .
Revised: It is worth mentioning that the GCN module in our model increases only 0 .

Original: GFLOPs computation compared to the baseline A3C ( ∼ 4 GFLOPs), which is marginal.
Revised: GFLOPs computation compared to the baseline A3C ( ∼ 4 GFLOPs), which is marginal.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We propose an approach to integrate semantic and functional priors with a deep reinforcement learning model for the task of navigation.
Revised: We propose an approach to integrate semantic and functional priors with a deep reinforcement learning model for the task of navigation.

Original: We use Graph Convolutional Networks to encode the prior knowledge and to update the knowledge according to the observations from the current scene.
Revised: We use Graph Convolutional Networks to encode the prior knowledge and to update the knowledge according to the observations from the current scene.

Original: Our experiments show that prior knowledge improves generalization to unseen scenes and targets.
Revised: Our experiments show that prior knowledge improves generalization to unseen scenes and targets.

Original: The current formulation of the problem does not include a long-term memory so in the future we plan to integrate memory to learn more complex exploration strategies.
Revised: The current formulation of the problem does not include a long-term memory so in the future we plan to integrate memory to learn more complex exploration strategies.

Original: Incorporating higher-order relationships between objects and scenes is another future direction that we consider.
Revised: Incorporating higher-order relationships between objects and scenes is another future direction that we consider.

Original: A PPENDIX A N AVIGATION T ARGETS
Revised: A PPENDIX A N AVIGATION T ARGETS

Original: In Table 4, we show the object categories that are used as our navigation targets.
Revised: In Table 5, we show the object categories that are used as our navigation targets.

Original: The split of train and test categories is provided as well.
Revised: The split of train and test categories is provided as well.

Original: Room type
Revised: Room type

Original: Train objects
Revised: Train objects

Original: Test objects
Revised: Test objects

Original: Kitchen
Revised: Kitchen

Original: HousePlant, StoveKnob, Sink, TableTop, Potato, Bread,
Revised: HousePlant, StoveKnob, Sink, TableTop, Potato, Bread,

Original: Tomato, Knife, Cabinet, Fridge, Container, ButterKnife,
Revised: Tomato, Knife, Cabinet, Fridge, Container, ButterKnife,

Original: Lettuce, Pan, Bowl, CoffeeMachine, StoveBurner, Plate
Revised: Lettuce, Pan, Bowl, CoffeeMachine, StoveBurner, Plate

Original: Mug, Apple,
Revised: Mug, Apple,

Original: Microwave,
Revised: Microwave,

Original: Toaster
Revised: Toaster

Original: Living room
Revised: Living room

Original: Television, HousePlant, Chair, TableTop, Box, Cloth,
Revised: Television, HousePlant, Chair, TableTop, Box, Cloth,

Original: Newspaper, KeyChain, WateringCan, PaintingHanger
Revised: Newspaper, KeyChain, WateringCan, PaintingHanger

Original: Painting,
Revised: Painting,

Original: Statue
Revised: Statue

Original: Bedroom
Revised: Bedroom

Original: Painting, HousePlant, CellPhone, LightSwitch, Candle,
Revised: Painting, HousePlant, CellPhone, LightSwitch, Candle,

Original: TableTop, Bed, Lamp, Statue, Book, CreditCard, KeyChain, Bowl, Pen, Box, Pencil, Blinds, Laptop, AlarmClock
Revised: TableTop, Bed, Lamp, Statue, Book, CreditCard, KeyChain, Bowl, Pen, Box, Pencil, Blinds, Laptop, AlarmClock

Original: Television,
Revised: Television,

Original: Mirror,Cabinet
Revised: Mirror,Cabinet

Original: Bathroom
Revised: Bathroom

Original: SprayBottle, Painting, Candle, LightSwitch, Sink, Cabinet, TowelHolder, Watch, ToiletPaper, ShowerDoor,
Revised: SprayBottle, Painting, Candle, LightSwitch, Sink, Cabinet, TowelHolder, Watch, ToiletPaper, ShowerDoor,

Original: SoapBottle
Revised: SoapBottle

Original: SoapBar,
Revised: SoapBar,

Original: Towel
Revised: Towel

Original: A PPENDIX B T RAINING C URVES
Revised: A PPENDIX B T RAINING C URVES

Original: We show the training curves in Figure 5.
Revised: We show the training curves in Figure 5.

Original: We compare our method with the baseline A3C. All the models are trained 3 times with different initializations.
Revised: We compare our method with the baseline A3C. All the models are trained 3 times with different initializations.

Original: We compute the model performance with
Revised: We compute the model performance with

Original: Success Rate and SPL every 10 million iterations during training.
Revised: Success Rate and SPL every 10 million iterations during training.

Original: We use the error band to represent the standard deviation.
Revised: We use the error band to represent the standard deviation.

Original: The curves show our model converges faster than the A3C baseline and obtain better performance in both metrics, which indicates the effectiveness of the scene priors.
Revised: The curves show our model converges faster than the A3C baseline and obtain better performance in both metrics, which indicates the effectiveness of the scene priors.

Original: A PPENDIX C I MPLEMENTATION D ETAILS
Revised: A PPENDIX C I MPLEMENTATION D ETAILS

Original: Our method is implemented in Tensorﬂow (Abadi et al., 2015) and the actor-critic policy network is trained with a single NVIDIA GeForce GTX Titan X GPU with 20 threads for 10 million frames for experiments without stop action, and for 25 million frames for experiments with stop action.
Revised: Our method is implemented in Tensorﬂow (Abadi et al., 2015) and the actor-critic policy network is trained with a single NVIDIA GeForce GTX Titan X GPU with 20 threads for 10 million frames for experiments without stop action, and for 25 million frames for experiments with stop action.

Original: The initial learning rate is set empirically as 7 e − 4 , and is decreased linearly as the training progresses.
Revised: The initial learning rate is set empirically as 7 e − 4 , and is decreased linearly as the training progresses.

Original: The network parameters are optimized by the RMSProp optimizer (Tieleman & Hinton).
Revised: The network parameters are optimized by the RMSProp optimizer (Tieleman & Hinton).

Original: The maximum number of steps is set to 100 for kitchen , bedroom and bathroom , and to 200 for living room due to the larger exploration space.
Revised: The maximum number of steps is set to 100 for kitchen , bedroom and bathroom , and to 200 for living room due to the larger exploration space.

Original: Since there is almost no overlap between object categories within different room types, we train separate models for each room type.
Revised: Since there is almost no overlap between object categories within different room types, we train separate models for each room type.

Original: A PPENDIX D Q UALITATIVE R ESULTS
Revised: A PPENDIX D Q UALITATIVE R ESULTS

Original: [Figure]
Revised: [Figure]

Original: K i t c h e n ( T o a s t e r )
Revised: K i t c h e n ( T o a s t e r )

Original: [Figure]
Revised: [Figure]

Original: L i v i n g r oo m ( P a i n t i n g )
Revised: L i v i n g r oo m ( P a i n t i n g )

Original: [Figure]
Revised: [Figure]

Original: oo
Revised: oo

Original: [Figure]
Revised: [Figure]

Original: w
Revised: w

Original: [Figure]
Revised: [Figure]

Original: l )
Revised: l )