Original: I NTERACTIVE G ROUNDED L ANGUAGE A CQUISITION AND G ENERALIZATION IN 2D E NVIRONMENT
Revised: I NTERACTIVE G ROUNDED L ANGUAGE A CQUISITION AND G ENERALIZATION IN 2D E NVIRONMENT

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We build a virtual agent for learning language in a 2D maze-like environment.
Revised: We build a virtual agent for learning language in a 2D maze-like environment.

Original: The agent sees surrounding environment images, listens to a virtual teacher, and takes actions to receive rewards.
Revised: The agent sees surrounding environment images, listens to a virtual teacher, and takes actions to receive rewards.

Original: It interactively learns the teacher’s language from scratch based on two language use cases: sentence-directed navigation and question answering.
Revised: It interactively learns the teacher’s language from scratch based on two language use cases: sentence-directed navigation and question answering.

Original: It learns simultaneously the visual representations of the environment, the language, and the action control.
Revised: It learns simultaneously the visual representations of the environment, the language, and the action control.

Original: By disentangling language grounding from other computational routines and sharing a concept detection function between language grounding and prediction, the agent reliably extrapolates to interpret navigation commands that may contain not only new word combinations but also new words.
Revised: By disentangling language grounding from other computational routines and sharing a concept detection function between language grounding and prediction, the agent reliably extrapolates to interpret navigation commands that may contain not only new word combinations but also new words.

Original: These new words are transferred from question answering.
Revised: These new words are transferred from question answering.

Original: This language ability is trained and evaluated on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words.
Revised: This language ability is trained and evaluated on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words.

Original: The proposed model signiﬁcantly outperforms four comparison methods on interpreting zero-shot sentences.
Revised: The proposed model signiﬁcantly outperforms four comparison methods on interpreting zero-shot sentences.

Original: We additionally demonstrate human-interpretable intermediate outputs of the model.
Revised: We additionally demonstrate human-interpretable intermediate outputs of the model.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Some empiricists argue that language may be learned based on its usage (Tomasello, 2003).
Revised: Some empiricists argue that language may be learned based on its usage (Tomasello, 2003).

Original: Skinner (1957) suggests that the successful use of a word reinforces the understanding of its meaning as well as the probability of it being used again in the future.
Revised: Skinner (1957) suggests that the successful use of a word reinforces the understanding of its meaning as well as the probability of it being used again in the future.

Original: Bruner (1985) emphasizes the role of social interaction in helping a child develop the language, and posits the importance of the feedback and reinforcement from the parents during the process.
Revised: Bruner (1985) emphasizes the role of social interaction in helping a child develop the language, and posits the importance of the feedback and reinforcement from the parents during the process.

Original: This paper takes a positive view of the above behaviorism and tries to explore some of the ideas by instantiating them in a 2D virtual environment where interactive language acquisition happens.
Revised: This paper takes a positive view of the above behaviorism and tries to explore some of the ideas by instantiating them in a 2D virtual environment where interactive language acquisition happens.

Original: This interactive setting contrasts to a common learning setting in that instead of from static labeled data, language is learned from dynamic interactions with an environment.
Revised: This interactive setting contrasts to a common learning setting in that instead of from static labeled data, language is learned from dynamic interactions with an environment.

Original: Language acquisition can go beyond mapping language as input patterns to output labels for merely obtaining high rewards or accomplishing tasks.
Revised: Language acquisition can go beyond mapping language as input patterns to output labels for merely obtaining high rewards or accomplishing tasks.

Original: We take a step further to require the language to be grounded (Harnad, 1990).
Revised: We take a step further to require the language to be grounded (Harnad, 1990).

Original: Speciﬁcally, we consult the paradigm of procedural semantics (Woods, 2007) which advocates that words, as abstract procedures, should be capable to pick out referents.
Revised: Speciﬁcally, we consult the paradigm of procedural semantics (Woods, 2007) which advocates that words, as abstract procedures, should be capable to pick out referents.

Original: We will attempt to explicitly link words to environment concepts instead of treating the whole model as a black box.
Revised: We will attempt to explicitly link words to environment concepts instead of treating the whole model as a black box.

Original: Such capability also implies that depending on the interactions with an environment, words would have particular meanings in a particular context and some content words in the usual case might not even have meanings in our case.
Revised: Such capability also implies that depending on the interactions with an environment, words would have particular meanings in a particular context and some content words in the usual case might not even have meanings in our case.

Original: As a result, the goal of this paper is to acquire “in-context” word meanings regardless of their completeness in all scenarios.
Revised: As a result, the goal of this paper is to acquire “in-context” word meanings regardless of their completeness in all scenarios.

Original: It has been argued that a child’s exposure to adult language provides inadequate evidence for language learning (Chomsky, 1991), but some induction mechanism might exist to bridge this gap (K. Landauer & T. Dumais, 1997).
Revised: It has been argued that a child’s exposure to adult language provides inadequate evidence for language learning (Chomsky, 1991), but some induction mechanism might exist to bridge this gap (K. Landauer & T. Dumais, 1997).

Original: This characteristics is critical for any AI system to learn inﬁnite sentences from ﬁnite training data.
Revised: This characteristics is critical for any AI system to learn inﬁnite sentences from ﬁnite training data.

Original: Such language generalization problem is specially addressed in our problem setting.
Revised: Such language generalization problem is specially addressed in our problem setting.

Original: After training, we want the agent to extrapolate to interpret zero-shot sentences that contain:
Revised: After training, we want the agent to extrapolate to interpret zero-shot sentences that contain:

Original: 1) new combinations of previously seen words for the same use case, or 2) new words learned from other use cases by different network modules.
Revised: 1) new combinations of previously seen words for the same use case, or 2) new words learned from other use cases by different network modules.

Original: Training
Revised: Training

Original: Testing
Revised: Testing

Original: [Figure]
Revised: [Figure]

Original: In the following, we will call the former ZS1 sentences and the latter ZS2 sentences.
Revised: In the following, we will call the former ZS1 sentences and the latter ZS2 sentences.

Original: Note that so far the zero-shot problems, addressed by most recent work (Hermann et al., 2017; Chaplot et al., 2017) of interactive language learning, belong to the category of ZS1.
Revised: Note that so far the zero-shot problems, addressed by most recent work (Hermann et al., 2017; Chaplot et al., 2017) of interactive language learning, belong to the category of ZS1.

Original: In contrast, a reliable interpretation of ZS2 sentences, which is essentially a transfer learning (Pan & Yang, 2010) problem, will be a major contribution of this work.
Revised: In contrast, a reliable interpretation of ZS2 sentences, which is essentially a transfer learning (Pan & Yang, 2010) problem, will be a major contribution of this work.

Original: We created a 2D maze-like environment called XWORLD (Figure 1), as a testbed for interactive grounded language acquisition and generalization.
Revised: We created a 2D maze-like environment called XWORLD (Figure 1), as a testbed for interactive grounded language acquisition and generalization.

Original: In this environment, a virtual agent has two language use cases: navigation (NAV) and question answering (QA).
Revised: In this environment, a virtual agent has two language use cases: navigation (NAV) and question answering (QA).

Original: For NAV, the agent need to navigate to correct places indicated by language commands from a virtual teacher.
Revised: For NAV, the agent need to navigate to correct places indicated by language commands from a virtual teacher.

Original: For QA, the agent has to correctly generate single-word answers according to the teacher’s questions.
Revised: For QA, the agent has to correctly generate single-word answers according to the teacher’s questions.

Original: NAV tests the language comprehension ability while QA additionally tests the language prediction ability.
Revised: NAV tests the language comprehension ability while QA additionally tests the language prediction ability.

Original: They happen simultaneously: when the agent is navigating, the teacher might ask questions regarding its current interaction with the environment.
Revised: They happen simultaneously: when the agent is navigating, the teacher might ask questions regarding its current interaction with the environment.

Original: Once the agent reaches the target or the time is up, the current session ends and a new one is randomly generated according to our conﬁguration (Appendix A).
Revised: Once the agent reaches the target or the time is up, the current session ends and a new one is randomly generated according to our conﬁguration (Appendix A).

Original: The ZS2 sentences deﬁned in our setting require word meanings to be transferred from answers to navigation commands, or more generally from language prediction to grounding .
Revised: The ZS2 sentences deﬁned in our setting require word meanings to be transferred from answers to navigation commands, or more generally from language prediction to grounding .

Original: This is achieved by establishing an explicit link between grounding and prediction via a common concept detection function, which constitutes the major novelty of our model.
Revised: This is achieved by establishing an explicit link between grounding and prediction via a common concept detection function, which constitutes the major novelty of our model.

Original: With this transferring ability, the agent is able to navigate to a freshly taught object without the need of retraining the NAV pipeline.
Revised: With this transferring ability, the agent is able to navigate to a freshly taught object without the need of retraining the NAV pipeline.

Original: It is worthwhile emphasizing that this seemingly “simple” environment in fact poses great challenge for language acquisition and generalization, because:
Revised: It is worthwhile emphasizing that this seemingly “simple” environment in fact poses great challenge for language acquisition and generalization, because:

Original: ˝
Revised: ˝

Original: The state space is huge.
Revised: The state space is huge.

Original: Even for a 7 ˆ 7 map with 15 wall blocks and 5 objects selected from 119 distinct classes, there are already octillions ( 10 27 ) of possible different conﬁgurations, not to mention the intra-class variance of object instances (see Figure 11 in the Appendix).
Revised: Even for a 7 ˆ 7 map with 15 wall blocks and 5 objects selected from 119 distinct classes, there are already octillions ( 10 27 ) of possible different conﬁgurations, not to mention the intra-class variance of object instances (see Figure 11 in the Appendix).

Original: For two conﬁgurations that only differ in one block, their successful navigation paths could be completely different.
Revised: For two conﬁgurations that only differ in one block, their successful navigation paths could be completely different.

Original: This requires an accurate perception of the environment.
Revised: This requires an accurate perception of the environment.

Original: Moreover, the conﬁgurations are constantly changing from session to session, and from training to testing.
Revised: Moreover, the conﬁgurations are constantly changing from session to session, and from training to testing.

Original: Particularly, the target changes across sessions regarding both its location and appearance.
Revised: Particularly, the target changes across sessions regarding both its location and appearance.

Original: ˝
Revised: ˝

Original: The goal space implied by the language is huge.
Revised: The goal space implied by the language is huge.

Original: For a vocabulary size as small as 185, the total number of distinct commands that can be said by the teacher conﬁrming to our deﬁned grammar is already over half a million.
Revised: For a vocabulary size as small as 185, the total number of distinct commands that can be said by the teacher conﬁrming to our deﬁned grammar is already over half a million.

Original: Two sentences that differ only in one word could imply completely different goals.
Revised: Two sentences that differ only in one word could imply completely different goals.

Original: This requires an accurate grounding of language.
Revised: This requires an accurate grounding of language.

Original: ˝
Revised: ˝

Original: The environment demands a strong language generalization ability from the agent.
Revised: The environment demands a strong language generalization ability from the agent.

Original: The agent has to learn to interpret zero-shot sentences that might be up to 13 words.
Revised: The agent has to learn to interpret zero-shot sentences that might be up to 13 words.

Original: It has to “plug” the meaning of a new word or word combination into a familiar sentential context while trying to still make sense of the unfamiliar whole.
Revised: It has to “plug” the meaning of a new word or word combination into a familiar sentential context while trying to still make sense of the unfamiliar whole.

Original: Recent work (Hermann et al., 2017; Chaplot et al., 2017) addresses ZS1 (for short sentences with several words) but not ZS2 sentences, which is a key difference between the challenge of our learning problem and theirs.
Revised: Recent work (Hermann et al., 2017; Chaplot et al., 2017) addresses ZS1 (for short sentences with several words) but not ZS2 sentences, which is a key difference between the challenge of our learning problem and theirs.

Original: We describe an end-to-end model for the agent to interactively acquire language from scratch and generalize to unfamiliar sentences.
Revised: We describe an end-to-end model for the agent to interactively acquire language from scratch and generalize to unfamiliar sentences.

Original: Here “scratch” means that the model does not hold any assumption on the semantics or syntax: each sentence is simply a sequence of tokens with each token being equally meaningless in the beginning of learning.
Revised: Here “scratch” means that the model does not hold any assumption on the semantics or syntax: each sentence is simply a sequence of tokens with each token being equally meaningless in the beginning of learning.

Original: This is unlike some early pioneering system ( e.g. , SHRDLU (Winograd, 1972)) that hard coded the syntax or semantics to link language to a simulated world, which potentially has the scalability issue.
Revised: This is unlike some early pioneering system ( e.g. , SHRDLU (Winograd, 1972)) that hard coded the syntax or semantics to link language to a simulated world, which potentially has the scalability issue.

Original: The interaction has two aspects: one is with the teacher ( i.e. , language and rewards) and the other is with the objects in the environment ( e.g. , stepping on objects or hitting walls).
Revised: The interaction has two aspects: one is with the teacher ( i.e. , language and rewards) and the other is with the objects in the environment ( e.g. , stepping on objects or hitting walls).

Original: The model takes as input RGB images, sentences, and rewards.
Revised: The model takes as input RGB images, sentences, and rewards.

Original: It learns simultaneously the visual representations of the environment, the language, and the action control.
Revised: It learns simultaneously the visual representations of the environment, the language, and the action control.

Original: We evaluate our model on randomly generated XWORLD maps with random agent positions, on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words.
Revised: We evaluate our model on randomly generated XWORLD maps with random agent positions, on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words.

Original: Detailed analysis of the trained model shows that the language is explicitly grounded such that words are capable to pick out referents in the environment.
Revised: Detailed analysis of the trained model shows that the language is explicitly grounded such that words are capable to pick out referents in the environment.

Original: We specially test the generalization ability of the agent for handling zero-shot sentences.
Revised: We specially test the generalization ability of the agent for handling zero-shot sentences.

Original: The average navigation success rates are 84.3% for ZS1 and 85.2% for ZS2 when only half of the commands is used for training, comparable to the rate of 90.5% in a normal language setting.
Revised: The average navigation success rates are 84.3% for ZS1 and 85.2% for ZS2 when only half of the commands is used for training, comparable to the rate of 90.5% in a normal language setting.

Original:  M ODEL
Revised:  M ODEL

Original: Our model incorporates two objectives: the ﬁrst is to maximize the cumulative reward of NAV and the second is to minimize the classiﬁcation cost of QA.
Revised: Our model incorporates two objectives: the ﬁrst is to maximize the cumulative reward of NAV and the second is to minimize the classiﬁcation cost of QA.

Original: For the former, we follow the standard reinforcement learning (RL) paradigm: the agent has to learn the action at every step from reward signals.
Revised: For the former, we follow the standard reinforcement learning (RL) paradigm: the agent has to learn the action at every step from reward signals.

Original: It employs the Actor-Critic (AC) algorithm (Sutton & Barto, 1998) to learn the control policy.
Revised: It employs the Actor-Critic (AC) algorithm (Sutton & Barto, 1998) to learn the control policy.

Original: For the latter, we adopt the standard supervised setting of Visual QA (Antol et al., 2015): the groundtruth answer is provided by the teacher during training.
Revised: For the latter, we adopt the standard supervised setting of Visual QA (Antol et al., 2015): the groundtruth answer is provided by the teacher during training.

Original: The training cost is formulated as the multiclass cross entropy.
Revised: The training cost is formulated as the multiclass cross entropy.

Original: 2. M OTIVATION
Revised: 2. M OTIVATION

Original: The model takes two streams of inputs: images and sentences.
Revised: The model takes two streams of inputs: images and sentences.

Original: The key is how to model the language grounding problem.
Revised: The key is how to model the language grounding problem.

Original: That is, the agent has to link (either implicitly or explicitly) language concepts to environment entities in order for it to correctly take an action by understanding the command in the current visual context.
Revised: That is, the agent has to link (either implicitly or explicitly) language concepts to environment entities in order for it to correctly take an action by understanding the command in the current visual context.

Original: A straightforward idea would be to encode the sentence s with an RNN and encode the perceived image e with a CNN, after which the two encoded representations are mixed together.
Revised: A straightforward idea would be to encode the sentence s with an RNN and encode the perceived image e with a CNN, after which the two encoded representations are mixed together.

Original: Speciﬁcally, let the multimodal module be M , the action module be A , and the prediction module be P , this idea can be formulated as:
Revised: Speciﬁcally, let the multimodal module be M , the action module be A , and the prediction module be P , this idea can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: Hermann et al.
Revised: Hermann et al.

Original: (2017); Chaplot et al.
Revised: (2017); Chaplot et al.

Original: ; Misra et al.
Revised: ; Misra et al.

Original: (2017) all employ similar paradigms as such.
Revised: (2017) all employ similar paradigms as such.

Original: In their implementations, M is either vector concatenation or element-wise product.
Revised: In their implementations, M is either vector concatenation or element-wise product.

Original: In this paradigm, for any particular word in the sentence, its actual interaction with the image could happen anywhere starting from M all the way to the end, right before a label is output.
Revised: In this paradigm, for any particular word in the sentence, its actual interaction with the image could happen anywhere starting from M all the way to the end, right before a label is output.

Original: This is due to the fact that the RNN folds the string of words into a compact embedding which then goes through the subsequent blackbox computations.
Revised: This is due to the fact that the RNN folds the string of words into a compact embedding which then goes through the subsequent blackbox computations.

Original: Therefore, language grounding and other computational routines are entangled.
Revised: Therefore, language grounding and other computational routines are entangled.

Original: Because of this, we say that this paradigm has an implicit language grounding strategy.
Revised: Because of this, we say that this paradigm has an implicit language grounding strategy.

Original: Such poses a great challenge for processing a ZS2 sentence because it is almost unpredictable about how a new word learned from language prediction would perform in the complex entanglement involved.
Revised: Such poses a great challenge for processing a ZS2 sentence because it is almost unpredictable about how a new word learned from language prediction would perform in the complex entanglement involved.

Original: Thus a careful inspection on the grounding process is needed.
Revised: Thus a careful inspection on the grounding process is needed.

Original: 2. A PPROACH
Revised: 2. A PPROACH

Original: The main idea of our approach is to disentangle language grounding from other computations in a model.
Revised: The main idea of our approach is to disentangle language grounding from other computations in a model.

Original: This disentanglement makes it possible for us to explicitly deﬁne language grounding around a core function that is also used by language prediction.
Revised: This disentanglement makes it possible for us to explicitly deﬁne language grounding around a core function that is also used by language prediction.

Original: Speciﬁcally, both grounding and prediction are cast as concept detection problems, where each word (embedding) is treated as a detector.
Revised: Speciﬁcally, both grounding and prediction are cast as concept detection problems, where each word (embedding) is treated as a detector.

Original: This opens up the possibility of transferring word meanings from the latter to the former.
Revised: This opens up the possibility of transferring word meanings from the latter to the former.

Original: 2.2. E XPLICIT GROUNDING
Revised: 2.2. E XPLICIT GROUNDING

Original: We begin with our deﬁnition of “grounding”.
Revised: We begin with our deﬁnition of “grounding”.

Original: Given a sentence s and an image representation h , we say that s is grounded in h as z if
Revised: Given a sentence s and an image representation h , we say that s is grounded in h as z if

Original: 1) h consists of M entities, and 2)
Revised: 1) h consists of M entities, and 2)

Original: z P t 0 , 1 u M with
Revised: z P t 0 , 1 u M with

Original: each entry z m representing a binary selection
Revised: each entry z m representing a binary selection

Original: of
Revised: of

Original: the m -th entity of
Revised: the m -th entity of

Original: h
Revised: h

Original: Thus z is a combinatorial selection over h .
Revised: Thus z is a combinatorial selection over h .

Original: An entity could be any subset of image features.
Revised: An entity could be any subset of image features.

Original: For example, if h is a collection of object regions, then an entity could be one particular region.
Revised: For example, if h is a collection of object regions, then an entity could be one particular region.

Original: In our case, h P R N ˆ D is a spatially ﬂattened feature cube (originally in 3D, now the 2D spatial domain collapsed into 1D for notation simplicity), where D is the number of channels and N is the number of feature vectors in the spatial domain.
Revised: In our case, h P R N ˆ D is a spatially ﬂattened feature cube (originally in 3D, now the 2D spatial domain collapsed into 1D for notation simplicity), where D is the number of channels and N is the number of feature vectors in the spatial domain.

Original: We adopt three deﬁnitions for an entity:
Revised: We adopt three deﬁnitions for an entity:

Original: 1) a feature vector at a particular image location, 2) a particular feature map along the channel dimension, and 3) a scalar feature at the intersection of a feature vector and a feature map
Revised: 1) a feature vector at a particular image location, 2) a particular feature map along the channel dimension, and 3) a scalar feature at the intersection of a feature vector and a feature map

Original: Their grounding results are denoted as Φ x p s, h q “ x P t 0 , 1 u N , Φ α p s, h q “ α P t 0 , 1 u D , and Φ p s, h q “ z P t 0 , 1 u N ˆ D , respectively.
Revised: Their grounding results are denoted as Φ x p s, h q “ x P t 0 , 1 u N , Φ α p s, h q “ α P t 0 , 1 u D , and Φ p s, h q “ z P t 0 , 1 u N ˆ D , respectively.

Original: We assume that Φ x and Φ α are independent:
Revised: We assume that Φ x and Φ α are independent:

Original: [Equation]
Revised: [Equation]

Original: With this deﬁnition, we propose a new paradigm for our problem:
Revised: With this deﬁnition, we propose a new paradigm for our problem:

Original: [Equation]
Revised: [Equation]

Original: where M A , M P , A , and P all rely on grounded results but not on other sentence representations.
Revised: where M A , M P , A , and P all rely on grounded results but not on other sentence representations.

Original: This explicit grounding strategy disentangles language grounding from subsequent computations such as obstacle detection, path planning, action making, and feature classiﬁcation, which all should be inherently language-independent routines.
Revised: This explicit grounding strategy disentangles language grounding from subsequent computations such as obstacle detection, path planning, action making, and feature classiﬁcation, which all should be inherently language-independent routines.

Original: Also, it abstracts away irrelevant input signals from the execution of the tasks, which potentially improves the generalization to similar sensor inputs.
Revised: Also, it abstracts away irrelevant input signals from the execution of the tasks, which potentially improves the generalization to similar sensor inputs.

Original: To make the model fully differentiable, in the following we relax the deﬁnition of grounding so that x P r 0 , 1 s N , α P r 0 , 1 s D , and z P r 0 , 1 s N ˆ D .
Revised: To make the model fully differentiable, in the following we relax the deﬁnition of grounding so that x P r 0 , 1 s N , α P r 0 , 1 s D , and z P r 0 , 1 s N ˆ D .

Original: The attention map x is responsible for image spatial attention.
Revised: The attention map x is responsible for image spatial attention.

Original: The channel mask α is responsible for selecting image feature maps, and is thus assumed to be independent on the speciﬁc h , i.e. , Φ α p s, h q “ Φ α p s q .
Revised: The channel mask α is responsible for selecting image feature maps, and is thus assumed to be independent on the speciﬁc h , i.e. , Φ α p s, h q “ Φ α p s q .

Original: The attention cube z is always an outer product of the two.
Revised: The attention cube z is always an outer product of the two.

Original: Similar attention mechanism has only been explored partially so far.
Revised: Similar attention mechanism has only been explored partially so far.

Original: For example, Xu et al.
Revised: For example, Xu et al.

Original: (2015) employed only x for image captioning.
Revised: (2015) employed only x for image captioning.

Original: Chaplot et al.
Revised: Chaplot et al.

Original: (2017) employed only α for 3D navigation.
Revised: (2017) employed only α for 3D navigation.

Original: Nevertheless, none attempted to establish a link between language grounding and prediction, which we will do next.
Revised: Nevertheless, none attempted to establish a link between language grounding and prediction, which we will do next.

Original: 2.2. C ONCEPT DETECTION
Revised: 2.2. C ONCEPT DETECTION

Original: With language grounding being disentangled, now we relate it to language prediction.
Revised: With language grounding being disentangled, now we relate it to language prediction.

Original: This relation is a common concept detection function.
Revised: This relation is a common concept detection function.

Original: We assume that every word in a vocabulary, as a concept, is detectable against entities of type (1).
Revised: We assume that every word in a vocabulary, as a concept, is detectable against entities of type (1).

Original: For meaningful detection of spatial-relation words that are irrelevant to image content, we incorporate parametric feature maps into h to learn spatial features.
Revised: For meaningful detection of spatial-relation words that are irrelevant to image content, we incorporate parametric feature maps into h to learn spatial features.

Original: The concept detection operates by sliding over the spatial domain of the feature cube h , which can be written as a function φ :
Revised: The concept detection operates by sliding over the spatial domain of the feature cube h , which can be written as a function φ :

Original: [Equation]
Revised: [Equation]

Original: where χ P R N is a detection score map and u is a word embedding vector.
Revised: where χ P R N is a detection score map and u is a word embedding vector.

Original: This function scores the embedding u against each feature vector of h , modulated by α that selects which feature maps to
Revised: This function scores the embedding u against each feature vector of h , modulated by α that selects which feature maps to

Original: [Figure]
Revised: [Figure]

Original: use for the scoring.
Revised: use for the scoring.

Original: Intuitively, each score on χ indicates the detection response of the feature vector in that location.
Revised: Intuitively, each score on χ indicates the detection response of the feature vector in that location.

Original: A higher score represents a higher detection response.
Revised: A higher score represents a higher detection response.

Original: While there are many potential forms for φ , we implement it as
Revised: While there are many potential forms for φ , we implement it as

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where ˝ is the element-wise product.
Revised: where ˝ is the element-wise product.

Original: In order to do so, we have word embedding u P R D where D is equal to the number of channels of h .
Revised: In order to do so, we have word embedding u P R D where D is equal to the number of channels of h .

Original: 2.2. P REDICTION BY CONCEPT DETECTION
Revised: 2.2. P REDICTION BY CONCEPT DETECTION

Original: Based on the detection function, M P outputs a score vector κ P R K over the entire lexicon
Revised: Based on the detection function, M P outputs a score vector κ P R K over the entire lexicon

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where u k is the k -th entry of the word embedding table.
Revised: where u k is the k -th entry of the word embedding table.

Original: The score κ k is the result of weighting the scores on the map χ k with x .
Revised: The score κ k is the result of weighting the scores on the map χ k with x .

Original: It represents the correctness of the k -th lexical entry as the answer to the question s .
Revised: It represents the correctness of the k -th lexical entry as the answer to the question s .

Original: To predict an answer
Revised: To predict an answer

Original: [Equation]
Revised: [Equation]

Original: Note that the role of α in the prediction is to select which feature maps are relevant to the question s .
Revised: Note that the role of α in the prediction is to select which feature maps are relevant to the question s .

Original: Otherwise it would be confusing for the agent about what to predict ( e.g. , whether to predict a color or an object name).
Revised: Otherwise it would be confusing for the agent about what to predict ( e.g. , whether to predict a color or an object name).

Original: By using α , we expect that different feature maps encode different image attributes (see an example in the caption of Figure 2).
Revised: By using α , we expect that different feature maps encode different image attributes (see an example in the caption of Figure 2).

Original: More analysis on α is done in section 4.6.
Revised: More analysis on α is done in section 4.6.

Original: 2.2. G ROUNDING BY CONCEPT DETECTION
Revised: 2.2. G ROUNDING BY CONCEPT DETECTION

Original: To deﬁne Φ , we deﬁne Φ α and Φ x separately given their independence.
Revised: To deﬁne Φ , we deﬁne Φ α and Φ x separately given their independence.

Original: By assumption, the channel mask α is meant to be determined solely from s , namely, which features to use should only depend on the sentence itself, but not on the feature cube h .
Revised: By assumption, the channel mask α is meant to be determined solely from s , namely, which features to use should only depend on the sentence itself, but not on the feature cube h .

Original: Thus it is computed as
Revised: Thus it is computed as

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: where the RNN returns an average state of processing s , followed by an MLP with the Sigmoid activation.
Revised: where the RNN returns an average state of processing s , followed by an MLP with the Sigmoid activation.

Original: Basically, we assume no word-image interaction during the computation of α .
Revised: Basically, we assume no word-image interaction during the computation of α .

Original: Then we want Φ x to be built on the detection function φ .
Revised: Then we want Φ x to be built on the detection function φ .

Original: One can expect that Φ x computes a series of score maps χ of individual words and merges them into x .
Revised: One can expect that Φ x computes a series of score maps χ of individual words and merges them into x .

Original: Suppose that s consists of L words t w l u with w l “ u k for some k .
Revised: Suppose that s consists of L words t w l u with w l “ u k for some k .

Original: Let τ p s q be a sequence of indices t l i u with 0 ď l i ă L .
Revised: Let τ p s q be a sequence of indices t l i u with 0 ď l i ă L .

Original: This sequence function τ decides which words of the sentence are selected and organized in what order.
Revised: This sequence function τ decides which words of the sentence are selected and organized in what order.

Original: We deﬁne Φ x p s, h q as
Revised: We deﬁne Φ x p s, h q as

Original: [Equation]
Revised: [Equation]

Original: where 1 P t 0 , 1 u D is a vector of ones, meaning that it selects all the feature maps for detecting w l i .
Revised: where 1 P t 0 , 1 u D is a vector of ones, meaning that it selects all the feature maps for detecting w l i .

Original: Υ is an aggregation function that combines the sequence of score maps
Revised: Υ is an aggregation function that combines the sequence of score maps

Original: χ l i of individual words.
Revised: χ l i of individual words.

Original: As such, φ makes it possible to transfer new words from Eq.
Revised: As such, φ makes it possible to transfer new words from Eq.

Original: 4 to Eq.
Revised: 4 to Eq.

Original: 5 during test time.
Revised: 5 during test time.

Original: If we were provided with an oracle that outputs a parsing tree for any sentence, we could set τ and Υ according to the tree semantics.
Revised: If we were provided with an oracle that outputs a parsing tree for any sentence, we could set τ and Υ according to the tree semantics.

Original: Neural Module Networks (NMNs) (Andreas et al., 2016a;b;
Revised: Neural Module Networks (NMNs) (Andreas et al., 2016a;b;

Original: Hu et al., 2017) rely on such tree for language grounding.
Revised: Hu et al., 2017) rely on such tree for language grounding.

Original: They generate a network of modules where each module corresponds to a tree node.
Revised: They generate a network of modules where each module corresponds to a tree node.

Original: However, labeled trees are needed for training.
Revised: However, labeled trees are needed for training.

Original: Below we propose to learn τ and Υ based on word attention (Bahdanau et al., 2015) to bypass the need of labeled structured data.
Revised: Below we propose to learn τ and Υ based on word attention (Bahdanau et al., 2015) to bypass the need of labeled structured data.

Original: We start with feeding a sentence s “ t w l u of length L to a Bidirectional RNN (Schuster & Paliwal, 1997).
Revised: We start with feeding a sentence s “ t w l u of length L to a Bidirectional RNN (Schuster & Paliwal, 1997).

Original: It outputs two things: a compact sentence embedding m and a sequence of L word context vectors w l .
Revised: It outputs two things: a compact sentence embedding m and a sequence of L word context vectors w l .

Original: Each w l summarizes the sentential pattern around the word.
Revised: Each w l summarizes the sentential pattern around the word.

Original: We then employ a meta controller called interpreter .
Revised: We then employ a meta controller called interpreter .

Original: For the i -th interpretation step, the interpreter computes the word attention as:
Revised: For the i -th interpretation step, the interpreter computes the word attention as:

Original: [Equation]
Revised: [Equation]

Original: where S cos is cosine similarity and GRU is the Gated Recurrent Unit (Cho et al., 2014).
Revised: where S cos is cosine similarity and GRU is the Gated Recurrent Unit (Cho et al., 2014).

Original: We set p 0 to the sentence embedding m .
Revised: We set p 0 to the sentence embedding m .

Original: Essentially, Eq. 6 uses soft word attention to approximate τ .
Revised: Essentially, Eq. 6 uses soft word attention to approximate τ .

Original: After this, the attended word s i is fed to the detection function φ .
Revised: After this, the attended word s i is fed to the detection function φ .

Original: The interpreter aggregates its score map by:
Revised: The interpreter aggregates its score map by:

Original: [Equation]
Revised: [Equation]

Original: where ˚ denotes 2D convolution, σ is Sigmoid, and ρ i is a scalar.
Revised: where ˚ denotes 2D convolution, σ is Sigmoid, and ρ i is a scalar.

Original: W ρ and b ρ are parameters to be learned.
Revised: W ρ and b ρ are parameters to be learned.

Original: Finally, the interpreter outputs x
Revised: Finally, the interpreter outputs x

Original: I as Φ x p s, h q , where I is the predeﬁned maximum step.
Revised: I as Φ x p s, h q , where I is the predeﬁned maximum step.

Original: [Equation]
Revised: [Equation]

Original: L
Revised: L

Original: G r ound i ng “Move to the red apple.”
Revised: G r ound i ng “Move to the red apple.”

Original: “Say the color of avocado.”
Revised: “Say the color of avocado.”

Original: F V
Revised: F V

Original: i s i on
Revised: i s i on

Original: P
Revised: P

Original: P r e d i c ti on
Revised: P r e d i c ti on

Original: A A c ti on 𝑠 𝜙 𝑒 𝑐 𝑎 𝑣 ℎ ℎ 𝑧 (x, 𝛼) 𝜅 𝑞 𝐌𝐌
Revised: A A c ti on 𝑠 𝜙 𝑒 𝑐 𝑎 𝑣 ℎ ℎ 𝑧 (x, 𝛼) 𝜅 𝑞 𝐌𝐌

Original: M u lti m od a l
Revised: M u lti m od a l

Original: NAV:
Revised: NAV:

Original: QA:
Revised: QA:

Original: Note that in the above we formulate the map transform as a 2D convolution.
Revised: Note that in the above we formulate the map transform as a 2D convolution.

Original: This operation enables the agent to reason about spatial relations.
Revised: This operation enables the agent to reason about spatial relations.

Original: Recall that each attention map x is ego-centric.
Revised: Recall that each attention map x is ego-centric.

Original: When the agent need to attend to a region speciﬁed by a spatial relation referring to an object, it can translate the object attention with the attention map of the spatial-relation word (Figure 3).
Revised: When the agent need to attend to a region speciﬁed by a spatial relation referring to an object, it can translate the object attention with the attention map of the spatial-relation word (Figure 3).

Original: For this reason, we set y 0 as a one-hot map where the map center is one, to represent the identity translation.
Revised: For this reason, we set y 0 as a one-hot map where the map center is one, to represent the identity translation.

Original: In summary, the overall architecture of our model is shown in Figure 4.
Revised: In summary, the overall architecture of our model is shown in Figure 4.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Our XWORLD environment is similar to the 2D block world MaseBase (Sukhbaatar et al., 2016).
Revised: Our XWORLD environment is similar to the 2D block world MaseBase (Sukhbaatar et al., 2016).

Original: However, we emphasize the problem of grounded language acquisition and generalization, while they do not.
Revised: However, we emphasize the problem of grounded language acquisition and generalization, while they do not.

Original: There have been several 3D environments such as ViZDoom (Kempka et al., 2016),
Revised: There have been several 3D environments such as ViZDoom (Kempka et al., 2016),

Original: DeepMind Lab (Beattie et al., 2016), and Malmo (Johnson et al., 2016).
Revised: DeepMind Lab (Beattie et al., 2016), and Malmo (Johnson et al., 2016).

Original: Still, they are intended for visual perception and control, with less or no language.
Revised: Still, they are intended for visual perception and control, with less or no language.

Original: Our problem setting draws inspirations from the AI roadmap delineated by Mikolov et al.
Revised: Our problem setting draws inspirations from the AI roadmap delineated by Mikolov et al.

Original: Like theirs, we have a teacher in the environment that assigns tasks and rewards to the agent, potentially with a curriculum.
Revised: Like theirs, we have a teacher in the environment that assigns tasks and rewards to the agent, potentially with a curriculum.

Original: Unlike their proposal of using only the linguistic channel, we have multiple perceptual modalities, the fusion of which is believed to be the basis of meaning (Kiela et al., 2016).
Revised: Unlike their proposal of using only the linguistic channel, we have multiple perceptual modalities, the fusion of which is believed to be the basis of meaning (Kiela et al., 2016).

Original: Contemporary to our work, several end-to-end systems also address language grounding problems in a simulated world with deep RL.
Revised: Contemporary to our work, several end-to-end systems also address language grounding problems in a simulated world with deep RL.

Original: Misra et al.
Revised: Misra et al.

Original: (2017) maps instructions and visual observations to actions of manipulating blocks on a 2D plane.
Revised: (2017) maps instructions and visual observations to actions of manipulating blocks on a 2D plane.

Original: Hermann et al.
Revised: Hermann et al.

Original: (2017); Chaplot et al.
Revised: (2017); Chaplot et al.

Original: (2017) learn to navigate in 3D under instructions, and both evaluate ZS1 generalization abilities.
Revised: (2017) learn to navigate in 3D under instructions, and both evaluate ZS1 generalization abilities.

Original: Despite short of the vision challenges in their environments, we have a much larger space of zero-shot sentences and additionally require ZS2 generalization which is in fact a transfer learning (Pan & Yang, 2010) problem.
Revised: Despite short of the vision challenges in their environments, we have a much larger space of zero-shot sentences and additionally require ZS2 generalization which is in fact a transfer learning (Pan & Yang, 2010) problem.

Original: Other recent work (Andreas et al., 2017; Oh et al., 2017) on zero-shot multitask learning treats language tokens as (parsed) labels for identifying skills.
Revised: Other recent work (Andreas et al., 2017; Oh et al., 2017) on zero-shot multitask learning treats language tokens as (parsed) labels for identifying skills.

Original: Their zero-shot settings are not intended for language understanding.
Revised: Their zero-shot settings are not intended for language understanding.

Original: Our learning problem is related to some recent work on grounding language in images and videos (Yu & Siskind, 2013; Rohrbach et al., 2016; Gao et al., 2016).
Revised: Our learning problem is related to some recent work on grounding language in images and videos (Yu & Siskind, 2013; Rohrbach et al., 2016; Gao et al., 2016).

Original: The NAV task is relevant to robotics navigation under language command (Chen & Mooney, 2011; Tellex et al., 2011; Barrett et al., 2015).
Revised: The NAV task is relevant to robotics navigation under language command (Chen & Mooney, 2011; Tellex et al., 2011; Barrett et al., 2015).

Original: The QA task is relevant to image question answering (VQA)
Revised: The QA task is relevant to image question answering (VQA)

Original: (Antol et al., 2015; Ren et al., 2015; Gao et al., 2015; Lu et al., 2016;
Revised: (Antol et al., 2015; Ren et al., 2015; Gao et al., 2015; Lu et al., 2016;

Original: Yang et al., 2016).
Revised: Yang et al., 2016).

Original: The interactive setting of learning to accomplish tasks is similar to that of learning to play video games from pixels (Mnih et al., 2015).
Revised: The interactive setting of learning to accomplish tasks is similar to that of learning to play video games from pixels (Mnih et al., 2015).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We design a variety of experiments to evaluate the agent’s language acquisition and generalization ability.
Revised: We design a variety of experiments to evaluate the agent’s language acquisition and generalization ability.

Original: Our model is ﬁrst compared with several methods to demonstrate the challenges in XWORLD .
Revised: Our model is ﬁrst compared with several methods to demonstrate the challenges in XWORLD .

Original: We then evaluate the agent’s language generalization ability in a zero-shot setting.
Revised: We then evaluate the agent’s language generalization ability in a zero-shot setting.

Original: We also visualize and analyze the intermediate outputs of the model and demonstrate interpretable results.
Revised: We also visualize and analyze the intermediate outputs of the model and demonstrate interpretable results.

Original: Finally, we conclude this section with some preliminary thoughts on how to scale our model to a 3D world.
Revised: Finally, we conclude this section with some preliminary thoughts on how to scale our model to a 3D world.

Original: 4. G ENERAL SETUP
Revised: 4. G ENERAL SETUP

Original: For all the experiments, both the sentences and the environment change from session to session, and from training to testing.
Revised: For all the experiments, both the sentences and the environment change from session to session, and from training to testing.

Original: The sentences are drawn conﬁrming to the teacher’s grammar.
Revised: The sentences are drawn conﬁrming to the teacher’s grammar.

Original: There are three types of language data: NAV command, QA question, and QA answer, which are illustrated in the ﬁgure below:
Revised: There are three types of language data: NAV command, QA question, and QA answer, which are illustrated in the ﬁgure below:

Original: [Equation]
Revised: [Equation]

Original: QA
Revised: QA

Original: answer
Revised: answer

Original: [Equation]
Revised: [Equation]

Original: XWORLD sentences QA
Revised: XWORLD sentences QA

Original: question
Revised: question

Original: [Equation]
Revised: [Equation]

Original: “Please arrive at the location between the cat and the dog.”
Revised: “Please arrive at the location between the cat and the dog.”

Original: “What is the object that is the west of the orange?”
Revised: “What is the object that is the west of the orange?”

Original: “Dragon”
Revised: “Dragon”

Original: In total, there are „ 570k NAV commands, „ 1m QA questions, and 136 QA answers (all the content words including 119 objects, 9 spatial relations, and 8 colors).
Revised: In total, there are „ 570k NAV commands, „ 1m QA questions, and 136 QA answers (all the content words including 119 objects, 9 spatial relations, and 8 colors).

Original: The environment conﬁgurations are randomly generated from octillions of possibilities of a 7 ˆ 7 map, conﬁrming to some high-level speciﬁcations such as the numbers of objects and wall blocks.
Revised: The environment conﬁgurations are randomly generated from octillions of possibilities of a 7 ˆ 7 map, conﬁrming to some high-level speciﬁcations such as the numbers of objects and wall blocks.

Original: Our model is evaluated on four types of navigation commands:
Revised: Our model is evaluated on four types of navigation commands:

Original: [Equation]
Revised: [Equation]

Original: Navigate to an object
Revised: Navigate to an object

Original: [Equation]
Revised: [Equation]

Original: Navigate to an object with a speciﬁc color
Revised: Navigate to an object with a speciﬁc color

Original: [Equation]
Revised: [Equation]

Original: Navigate to a location near an object
Revised: Navigate to a location near an object

Original: [Equation]
Revised: [Equation]

Original: Navigate to a location between two objects
Revised: Navigate to a location between two objects

Original: [Equation]
Revised: [Equation]

Original: For the detailed experiment settings, we refer the reader to Appendix A.
Revised: For the detailed experiment settings, we refer the reader to Appendix A.

Original: 4. C OMPARISON METHODS
Revised: 4. C OMPARISON METHODS

Original: The four comparison methods are described below:
Revised: The four comparison methods are described below:

Original: ContextualAttention [CA]
Revised: ContextualAttention [CA]

Original: A variant of our model that replaces the interpreter with a contextual attention model in L .
Revised: A variant of our model that replaces the interpreter with a contextual attention model in L .

Original: The attention model uses a gated RNN to convert a sentence to a ﬁlter which is then convolved with the feature cube h to obtain the attention map x .
Revised: The attention model uses a gated RNN to convert a sentence to a ﬁlter which is then convolved with the feature cube h to obtain the attention map x .

Original: The ﬁlter covers the 3 ˆneighborhood of each feature vector in the spatial domain.
Revised: The ﬁlter covers the 3 ˆneighborhood of each feature vector in the spatial domain.

Original: The channel mask α is still computed as before.
Revised: The channel mask α is still computed as before.

Original: The rest of the model is unchanged.
Revised: The rest of the model is unchanged.

Original: StackedAttentionNet [SAN] An adaptation of Yang et al.
Revised: StackedAttentionNet [SAN] An adaptation of Yang et al.

Original: (2016) which was originally proposed for
Revised: (2016) which was originally proposed for

Original: VQA.
Revised: VQA.

Original: We replace our interpreter with their stacked attention model in L to obtain the attention map x .
Revised: We replace our interpreter with their stacked attention model in L to obtain the attention map x .

Original: The channel mask α is still computed as before.
Revised: The channel mask α is still computed as before.

Original: Instead of employing a pretrained CNN as they did, we train a CNN from scratch to accommodate to the XWORLD environment.
Revised: Instead of employing a pretrained CNN as they did, we train a CNN from scratch to accommodate to the XWORLD environment.

Original: The CNN is conﬁgured as the one employed by our model.
Revised: The CNN is conﬁgured as the one employed by our model.

Original: The rest of our model is unchanged.
Revised: The rest of our model is unchanged.

Original: VIS-LSTM
Revised: VIS-LSTM

Original: VL] An adaptation of Ren et al.
Revised: VL] An adaptation of Ren et al.

Original: (2015) which was originally proposed for VQA.
Revised: (2015) which was originally proposed for VQA.

Original: We ﬂatten h and project it to the word embedding space R D .
Revised: We ﬂatten h and project it to the word embedding space R D .

Original: Then it is appended to the input sentence s as the ﬁrst word.
Revised: Then it is appended to the input sentence s as the ﬁrst word.

Original: The augmented sentence goes through an LSTM whose last state is used for both
Revised: The augmented sentence goes through an LSTM whose last state is used for both

Original: NAV and QA (Figure 12 Appendix C).
Revised: NAV and QA (Figure 12 Appendix C).

Original: ConcatEmbed [CE] An adaptation of Mao et al. (2015) which was originally proposed for image captioning.
Revised: ConcatEmbed [CE] An adaptation of Mao et al. (2015) which was originally proposed for image captioning.

Original: It instantiates L as a vanilla LSTM which outputs a sentence embedding.
Revised: It instantiates L as a vanilla LSTM which outputs a sentence embedding.

Original: Then h is
Revised: Then h is

Original: Number of training examples ( × 10 6 ) −−−−−
Revised: Number of training examples ( × 10 6 ) −−−−−

Original: R e w a r d
Revised: R e w a r d

Original: Ours
Revised: Ours

Original: CA
Revised: CA

Original: SAN
Revised: SAN

Original: CE
Revised: CE

Original: VL 0.0 0.2 0.4 0.6 0.8 1.
Revised: VL 0.0 0.2 0.4 0.6 0.8 1.

Original: n a v _ o b
Revised: n a v _ o b

Original: j n a v _ c o l _ o b j n a v _ n r _ o b j n a v _ b w _ o b j o v e r a ll
Revised: j n a v _ c o l _ o b j n a v _ n r _ o b j n a v _ b w _ o b j o v e r a ll

Original: [Figure]
Revised: [Figure]

Original: projected and concatenated with the embedding.
Revised: projected and concatenated with the embedding.

Original: The concatenated vector is used for both NAV and
Revised: The concatenated vector is used for both NAV and

Original: QA (Figure 13 Appendix C).
Revised: QA (Figure 13 Appendix C).

Original: This concatenation mechanism is also employed by Hermann et al.
Revised: This concatenation mechanism is also employed by Hermann et al.

Original: (2017); Misra et al.
Revised: (2017); Misra et al.

Original: In the following experiments, we train all ﬁve approaches (four comparison methods and our model) with a small learning rate of 1 ˆ 10 ´ 5 and a batch size of 16, for a maximum of 200k mini-batches.
Revised: In the following experiments, we train all ﬁve approaches (four comparison methods and our model) with a small learning rate of 1 ˆ 10 ´ 5 and a batch size of 16, for a maximum of 200k mini-batches.

Original: Additional training details are described in Appendix B. After training, we test each approach on 50k sessions of navigation.
Revised: Additional training details are described in Appendix B. After training, we test each approach on 50k sessions of navigation.

Original: We compute the average success rate of navigation where a success is deﬁned as reaching the correct location before the time out of a session.
Revised: We compute the average success rate of navigation where a success is deﬁned as reaching the correct location before the time out of a session.

Original: 4. B ASIC EVALUATION
Revised: 4. B ASIC EVALUATION

Original: In this experiment, the training and testing NAV commands are sampled from the same distribution over the entire space 1 .
Revised: In this experiment, the training and testing NAV commands are sampled from the same distribution over the entire space 1 .

Original: We call it the normal language setting.
Revised: We call it the normal language setting.

Original: The training reward curves and testing results are shown in Figure 5.
Revised: The training reward curves and testing results are shown in Figure 5.

Original: VL and CE have quite low rewards without convergences.
Revised: VL and CE have quite low rewards without convergences.

Original: These two approaches do not use the spatial attention x , and thus always attend to an entire image with no focus.
Revised: These two approaches do not use the spatial attention x , and thus always attend to an entire image with no focus.

Original: The region of a target location is tiny compared to the whole ego-centric image (a ratio of 1 : p
Revised: The region of a target location is tiny compared to the whole ego-centric image (a ratio of 1 : p

Original: 7 ˆ 2 ´ 1 q 2 “ 1 : 169 ).
Revised: 7 ˆ 2 ´ 1 q 2 “ 1 : 169 ).

Original: It is possible that this difﬁculty drives the models towards overﬁtting QA without learning useful representations for NAV.
Revised: It is possible that this difﬁculty drives the models towards overﬁtting QA without learning useful representations for NAV.

Original: Both CA and SAN obtain rewards and success rates slightly worse than ours.
Revised: Both CA and SAN obtain rewards and success rates slightly worse than ours.

Original: This suggests that in a normal language setting, existing attention models are good enough for understanding previously seen sentences.
Revised: This suggests that in a normal language setting, existing attention models are good enough for understanding previously seen sentences.

Original: However, their language generalization abilities remain to be seen.
Revised: However, their language generalization abilities remain to be seen.

Original: [Figure]
Revised: [Figure]

Original: 4. L ANGUAGE GENERALIZATION
Revised: 4. L ANGUAGE GENERALIZATION

Original: Our more important question is whether the agent has the ability of interpreting zero-shot sentences.
Revised: Our more important question is whether the agent has the ability of interpreting zero-shot sentences.

Original: For comparison, we use CA and SAN from the previous section, as they achieved good performance in the normal language setting.
Revised: For comparison, we use CA and SAN from the previous section, as they achieved good performance in the normal language setting.

Original: For a zero-shot setting, we setup two language conditions:
Revised: For a zero-shot setting, we setup two language conditions:

Original: NewWordCombination [ZS1] Some word pairs are excluded from both the NAV commands and the
Revised: NewWordCombination [ZS1] Some word pairs are excluded from both the NAV commands and the

Original: QA questions.
Revised: QA questions.

Original: We consider three types of unordered combinations of the content words: ( object , spatial relation ), ( object , color ), and ( object , object ).
Revised: We consider three types of unordered combinations of the content words: ( object , spatial relation ), ( object , color ), and ( object , object ).

Original: We randomly hold out X % of word pairs during training and only use them in NAV testing.
Revised: We randomly hold out X % of word pairs during training and only use them in NAV testing.

Original: NewWord [ZS2] Some single words are excluded from both the NAV commands and the QA questions but can appear in the QA answers.
Revised: NewWord [ZS2] Some single words are excluded from both the NAV commands and the QA questions but can appear in the QA answers.

Original: We randomly hold out X % of the object words during training and only use them in NAV testing.
Revised: We randomly hold out X % of the object words during training and only use them in NAV testing.

Original: We vary the value of X % (12.5%, 20.0%, 50.0%, 66.7%, and 90.0%) in both conditions to test how sensitive the generalization is to the amount of held-out data.
Revised: We vary the value of X % (12.5%, 20.0%, 50.0%, 66.7%, and 90.0%) in both conditions to test how sensitive the generalization is to the amount of held-out data.

Original: The results are shown in Figure 6.
Revised: The results are shown in Figure 6.

Original: We make three observations from the results.
Revised: We make three observations from the results.

Original: First, nav bw obj is an outlier command type in which: 1) CA almost consistently outperforms our model on both ZS1 and ZS2, and 2) on ZS2, it is much less sensitive to the increase of X compared to the other three command types.
Revised: First, nav bw obj is an outlier command type in which: 1) CA almost consistently outperforms our model on both ZS1 and ZS2, and 2) on ZS2, it is much less sensitive to the increase of X compared to the other three command types.

Original: Deep analysis
Revised: Deep analysis

Original: [Figure]
Revised: [Figure]

Original: reveals that for nav bw obj , CA learns to cheat by looking for a context that contains the special pattern of object pairs in the image without having to recognize them ﬁrst.
Revised: reveals that for nav bw obj , CA learns to cheat by looking for a context that contains the special pattern of object pairs in the image without having to recognize them ﬁrst.

Original: Because this pattern is relatively rare and the chance of the existence of multiple such pairs in one image is slim, they can achieve a high success rate by doing so.
Revised: Because this pattern is relatively rare and the chance of the existence of multiple such pairs in one image is slim, they can achieve a high success rate by doing so.

Original: This suggests that neural networks tend to exploit data usually in an unexpected way to achieve tasks if no constraints are imposed (Kottur et al., 2017).
Revised: This suggests that neural networks tend to exploit data usually in an unexpected way to achieve tasks if no constraints are imposed (Kottur et al., 2017).

Original: Second, ZS1 sentences are much easier to interpret than ZS2 sentences.
Revised: Second, ZS1 sentences are much easier to interpret than ZS2 sentences.

Original: Neural networks seem to inherently have this capability to some extent.
Revised: Neural networks seem to inherently have this capability to some extent.

Original: This is consistent with what have been observed in some previous work (Hermann et al., 2017; Chaplot et al., 2017) that addresses the generalization ability on new word combinations.
Revised: This is consistent with what have been observed in some previous work (Hermann et al., 2017; Chaplot et al., 2017) that addresses the generalization ability on new word combinations.

Original: Third, ZS2 sentences are difﬁcult for CA and SAN .
Revised: Third, ZS2 sentences are difﬁcult for CA and SAN .

Original: Even with a held-out portion as small as 12.5%, their success rates drop at most 80% compared to the normal language setting.
Revised: Even with a held-out portion as small as 12.5%, their success rates drop at most 80% compared to the normal language setting.

Original: In contrast, our model tends to maintain the same success rate until X “ 90 . 0 .
Revised: In contrast, our model tends to maintain the same success rate until X “ 90 . 0 .

Original: Impressively, it achieves an average rate of 59.5% even when the number of new object words is 9 times that of seen object words!
Revised: Impressively, it achieves an average rate of 59.5% even when the number of new object words is 9 times that of seen object words!

Original: To sum up, our model exhibits a strong generalization ability on both ZS1 and ZS2 sentences, the latter of which pose a great challenge for traditional language grounding models.
Revised: To sum up, our model exhibits a strong generalization ability on both ZS1 and ZS2 sentences, the latter of which pose a great challenge for traditional language grounding models.

Original: Although we use a particular environment setup for evaluation in this work, the promising results imply the potential of scaling to an even larger vocabulary and grammar with a much larger language space.
Revised: Although we use a particular environment setup for evaluation in this work, the promising results imply the potential of scaling to an even larger vocabulary and grammar with a much larger language space.

Original: 4. R OBUSTNESS TO NOISE
Revised: 4. R OBUSTNESS TO NOISE

Original: We add noise to the test NAV commands to evaluate the robustness of our model.
Revised: We add noise to the test NAV commands to evaluate the robustness of our model.

Original: Given a sentence of length L , we insert t ηL u noise words in it at random locations, where η is the noise level selected as 0.3, 0.5, or 0.8.
Revised: Given a sentence of length L , we insert t ηL u noise words in it at random locations, where η is the noise level selected as 0.3, 0.5, or 0.8.

Original: Each noise word has an embedding vector of which each entry is independently sampled from the standard normal distribution.
Revised: Each noise word has an embedding vector of which each entry is independently sampled from the standard normal distribution.

Original: Note that these noise words do not appear in training; they only appear in testing.
Revised: Note that these noise words do not appear in training; they only appear in testing.

Original: We directly use our trained model from section 4.3 for testing.
Revised: We directly use our trained model from section 4.3 for testing.

Original: The navigation success rates under noise-added commands are shown in Figure 7.
Revised: The navigation success rates under noise-added commands are shown in Figure 7.

Original: It can be seen that the model correctly executed the commands for a reasonable portion of test sessions despite the noise.
Revised: It can be seen that the model correctly executed the commands for a reasonable portion of test sessions despite the noise.

Original: This shows its potential of scaling to test scenarios where the commands are possibly not well formed.
Revised: This shows its potential of scaling to test scenarios where the commands are possibly not well formed.

Original: 4. V ISUALIZATION AND ANALYSIS
Revised: 4. V ISUALIZATION AND ANALYSIS

Original: In this section, we visualize some intermediate results of our trained model.
Revised: In this section, we visualize some intermediate results of our trained model.

Original: Channel mask α .
Revised: Channel mask α .

Original: We inspect the channel mask Φ α p s q which allows the model to select certain feature maps from a feature cube h and predict an answer to the question s .
Revised: We inspect the channel mask Φ α p s q which allows the model to select certain feature maps from a feature cube h and predict an answer to the question s .

Original: We randomly sample 10k QA questions and compute α for each of them using the grounding module L .
Revised: We randomly sample 10k QA questions and compute α for each of them using the grounding module L .

Original: Because in our experiments each answer must be a content word, we can divide the 10k questions into 135 groups 2 , where each group corresponds to a distinct content word.
Revised: Because in our experiments each answer must be a content word, we can divide the 10k questions into 135 groups 2 , where each group corresponds to a distinct content word.

Original: Then we compute an Euclidean distance matrix D where entry
Revised: Then we compute an Euclidean distance matrix D where entry

Original: D i,j is the average distance between the α of a question from the i -th group and that from the j -th group (Figure 8).
Revised: D i,j is the average distance between the α of a question from the i -th group and that from the j -th group (Figure 8).

Original: It is clear that there are three topics (object, color, and spatial relation) in the matrix.
Revised: It is clear that there are three topics (object, color, and spatial relation) in the matrix.

Original: Distances within the same topic are much smaller than across topics.
Revised: Distances within the same topic are much smaller than across topics.

Original: This demonstrates that with the channel mask, the model is able to look at different subsets of features for questions of different topics, while using the same subset of features for questions of the same
Revised: This demonstrates that with the channel mask, the model is able to look at different subsets of features for questions of different topics, while using the same subset of features for questions of the same

Original: [Table]
Revised: [Table]

Original: a c o c k p e n g u i n p e pp e r p i g p i n e a pp l e p l un g e r p o t a t o p u m p k i n r a bb i t
Revised: a c o c k p e n g u i n p e pp e r p i g p i n e a pp l e p l un g e r p o t a t o p u m p k i n r a bb i t

Original: r a c oo n r a t r h i n o c e r o s r oo s t e r s e a h
Revised: r a c oo n r a t r h i n o c e r o s r oo s t e r s e a h

Original: o r s e s e a s h e ll s e a u r c h i n s h r i m p s n a il s n a k e s o f a s p i d e r s q u a r e s q u
Revised: o r s e s e a s h e ll s e a u r c h i n s h r i m p s n a il s n a k e s o f a s p i d e r s q u a r e s q u

Original: i rr e l s t
Revised: i rr e l s t

Original: a i r s s t a r s t r a w b e rr y t
Revised: a i r s s t a r s t r a w b e rr y t

Original: i g e r t o il e t t o m a t o t r i a n g l e t u r t l e v a c uu m w a r d r o b e w a s h i n g m a c h i n e w a t e r m e l o n w h a l e w h e a t z e b r a e a s t n o r t h n o r t
Revised: i g e r t o il e t t o m a t o t r i a n g l e t u r t l e v a c uu m w a r d r o b e w a s h i n g m a c h i n e w a t e r m e l o n w h a l e w h e a t z e b r a e a s t n o r t h n o r t

Original: h
Revised: h

Original: e a s t n o r t
Revised: e a s t n o r t

Original: h
Revised: h

Original: w e s t s o u t
Revised: w e s t s o u t

Original: h s o u t
Revised: h s o u t

Original: h e a s t s o u t h
Revised: h e a s t s o u t h

Original: w e s t w e s t
Revised: w e s t w e s t

Original: [Figure]
Revised: [Figure]

Original: topic.
Revised: topic.

Original: It also implies that the feature cube h is learned to organize feature maps according to image attributes.
Revised: It also implies that the feature cube h is learned to organize feature maps according to image attributes.

Original: Word context & attention.
Revised: Word context & attention.

Original: To intuitively demonstrate how the interpreter works, we visualize the word context vectors w l in Eq.
Revised: To intuitively demonstrate how the interpreter works, we visualize the word context vectors w l in Eq.

Original: 6 for a total of 20k word locations l (10k from QA questions and the other 10k from NAV commands).
Revised: 6 for a total of 20k word locations l (10k from QA questions and the other 10k from NAV commands).

Original: Each word context vector is projected down to a space ofdimensions using PCA (Jolliffe, 1986), after which we use t-SNE (van der Maaten & Hinton, 2008;
Revised: Each word context vector is projected down to a space ofdimensions using PCA (Jolliffe, 1986), after which we use t-SNE (van der Maaten & Hinton, 2008;

Original: Ulyanov, 2016) to further reduce the dimensionality to 2.
Revised: Ulyanov, 2016) to further reduce the dimensionality to 2.

Original: The t-SNE method uses a perplexity ofand a learning rate of 200, and runs for 1k iterations.
Revised: The t-SNE method uses a perplexity ofand a learning rate of 200, and runs for 1k iterations.

Original: The visualization of the 20k points is shown in Figure 9 (a).
Revised: The visualization of the 20k points is shown in Figure 9 (a).

Original: Recall that in Eq.
Revised: Recall that in Eq.

Original: 6 the word attention is computed by comparing the interpreter state p
Revised: 6 the word attention is computed by comparing the interpreter state p

Original: i ´ 1 with word context vectors w l .
Revised: i ´ 1 with word context vectors w l .

Original: In order to select content words to generate meaningful score maps via φ , the interpreter is expected to differentiate them from the remaining grammatical words based on the contexts.
Revised: In order to select content words to generate meaningful score maps via φ , the interpreter is expected to differentiate them from the remaining grammatical words based on the contexts.

Original: This hypothesis is supported by the above visualization in which the context vectors of content words (in blue, green, and red) and the those of grammatical words (in black) are mostly separated.
Revised: This hypothesis is supported by the above visualization in which the context vectors of content words (in blue, green, and red) and the those of grammatical words (in black) are mostly separated.

Original: Figure 9 (b) shows some example questions whose word attentions are[Figure]
Revised: Figure 9 (b) shows some example questions whose word attentions are

Original: spatial relation object other

Original: Where is green apple located ?
Revised: Where is green apple located ?

Original: What is between lion and hedgehog ?
Revised: What is between lion and hedgehog ?

Original: The grid in east to cherry ?
Revised: The grid in east to cherry ?

Original: The location of the fish is ?
Revised: The location of the fish is ?

Original: What is in blue ?
Revised: What is in blue ?

Original: low high
Revised: low high

Original: [Figure]
Revised: [Figure]

Original: computed from the word context vectors.
Revised: computed from the word context vectors.

Original: It can beseen that the content words can be successfully selected bythe interpreter.
Revised: It can be that the content words can be successfully selected by

Original: Attention map x .
Revised: Attention map x .

Original: Finally, we visualize the computation of the attention map Φ x p s, h q .
Revised: Finally, we visualize the computation of the attention map Φ x p s, h q .

Original: In each example, the intermediate results ( x i and ρ i in Eq. 7) of three interpretation steps are shown.
Revised: In each example, the intermediate results ( x i and ρ i in Eq. 7) of three interpretation steps are shown.

Original: (Figure 10).
Revised: (Figure 10).

Original: The environment maps perfectly detect all possible entities and thus provide a guide for the agent to navigate successfully without hitting walls or confounding targets.
Revised: The environment maps perfectly detect all possible entities and thus provide a guide for the agent to navigate successfully without hitting walls or confounding targets.

Original: 4. H OW DOES IT ADAPT TO 3D?
Revised: 4. H OW DOES IT ADAPT TO 3D?

Original: We discuss the possibility of adapting our model to build an agent with similar language abilities in a 3D world ( e.g. , Beattie et al.
Revised: We discuss the possibility of adapting our model to build an agent with similar language abilities in a 3D world ( e.g. , Beattie et al.

Original: ; Johnson et al.
Revised: ; Johnson et al.

Original: This is our goal for the future, but here we would like to share some preliminary thoughts.
Revised: This is our goal for the future, but here we would like to share some preliminary thoughts.

Original: Generally speaking, a 3D environment will mainly pose a greater challenge for visual perception but hardly for language understanding.
Revised: Generally speaking, a 3D environment will mainly pose a greater challenge for visual perception but hardly for language understanding.

Original: The key element of our model is the attention cube z that is intended for an explicit grounding of language, which includes the channel mask Φ α and the interpreter Φ x .
Revised: The key element of our model is the attention cube z that is intended for an explicit grounding of language, which includes the channel mask Φ α and the interpreter Φ x .

Original: The channel mask only depends on the sentence itself, and thus is expected to work regardless of the world dimensionality.
Revised: The channel mask only depends on the sentence itself, and thus is expected to work regardless of the world dimensionality.

Original: The interpreter depends on a sequence of score maps χ whose computation φ for now is deﬁned as multiplying a word embedding with the feature cube (Eq. 3).
Revised: The interpreter depends on a sequence of score maps χ whose computation φ for now is deﬁned as multiplying a word embedding with the feature cube (Eq. 3).

Original: A more sophisticated deﬁnition of φ will be needed to detect objects in a 3D environment.
Revised: A more sophisticated deﬁnition of φ will be needed to detect objects in a 3D environment.

Original: Additionally, the interpreter models spatial attention transform as a 2D convolution (Eq. 7).
Revised: Additionally, the interpreter models spatial attention transform as a 2D convolution (Eq. 7).

Original: This assumption is no longer valid in 3D, and a better transformation method that accounts for perspective distortion is required.
Revised: This assumption is no longer valid in 3D, and a better transformation method that accounts for perspective distortion is required.

Original: Lastly, the surrounding environment is only partially observable to a 3D agent.
Revised: Lastly, the surrounding environment is only partially observable to a 3D agent.

Original: Working memory such as LSTM added to the action module A will be important for navigation in such case.
Revised: Working memory such as LSTM added to the action module A will be important for navigation in such case.

Original: Despite these changes to be made, we believe that the general explicit grounding strategy and the common detection function shared by language grounding and prediction have shed some light on the adaptation.
Revised: Despite these changes to be made, we believe that the general explicit grounding strategy and the common detection function shared by language grounding and prediction have shed some light on the adaptation.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have presented an end-to-end model for a virtual agent to acquire language from a 2D environment in an interactive and grounded manner, through the visual and linguistic perception channels.
Revised: We have presented an end-to-end model for a virtual agent to acquire language from a 2D environment in an interactive and grounded manner, through the visual and linguistic perception channels.

Original: After learning, the agent is able to extrapolate to interpret zero-shot sentences that contain new word combinations or even new words.
Revised: After learning, the agent is able to extrapolate to interpret zero-shot sentences that contain new word combinations or even new words.

Original: Such generalization ability is supported by an explicit grounding strategy that disentangles language grounding from subsequent language-independent computations.
Revised: Such generalization ability is supported by an explicit grounding strategy that disentangles language grounding from subsequent language-independent computations.

Original: It also depends on sharing a detection function between language grounding and language prediction as the core computation.
Revised: It also depends on sharing a detection function between language grounding and language prediction as the core computation.

Original: This function enables word meanings to transfer from prediction to grounding during test time.
Revised: This function enables word meanings to transfer from prediction to grounding during test time.

Original: Promising language acquisition and generalization results have been
Revised: Promising language acquisition and generalization results have been

Original: “Northwest near the chair is your goal.”
Revised: “Northwest near the chair is your goal.”

Original: 𝜌 " = 0.
Revised: 𝜌 " = 0.

Original: 𝜌 ’ = 0.
Revised: 𝜌 ’ = 0.

Original: “Go to the grid between rabbit and lion.”
Revised: “Go to the grid between rabbit and lion.”

Original: 𝜌 " = 0.
Revised: 𝜌 " = 0.

Original: 𝜌 ’ = 0.
Revised: 𝜌 ’ = 0.

Original: “Please reach the gray horse.”
Revised: “Please reach the gray horse.”

Original: 𝜌 " = 0.
Revised: 𝜌 " = 0.

Original: 𝜌 ’ = 0.
Revised: 𝜌 ’ = 0.

Original: 𝜌 " = 0.
Revised: 𝜌 " = 0.

Original: 𝜌 ’ = 0.
Revised: 𝜌 ’ = 0.

Original: “Identify the block in east of the grape.”
Revised: “Identify the block in east of the grape.”

Original: 𝑥 ,- 𝑔 ,
Revised: 𝑥 ,- 𝑔 ,

Original: [Figure]
Revised: [Figure]

Original: shown in the 2D XWORLD environment.
Revised: shown in the 2D XWORLD environment.

Original: We hope that this work can shed some light on acquiring and generalizing language in a similar way in 3D environments.
Revised: We hope that this work can shed some light on acquiring and generalizing language in a similar way in 3D environments.

Original: A PPENDICES A XWORLD SETUP
Revised: A PPENDICES A XWORLD SETUP

Original: The XWORLD environment is conﬁgured with 7 ˆ 7 grid maps.
Revised: The XWORLD environment is conﬁgured with 7 ˆ 7 grid maps.

Original: On each map, the open space for the agent has a size ranging from 3 ˆ 3 to 7 ˆ 7 .
Revised: On each map, the open space for the agent has a size ranging from 3 ˆ 3 to 7 ˆ 7 .

Original: Smaller open spaces are setup for curriculum learning (Appendix D), but not for testing.
Revised: Smaller open spaces are setup for curriculum learning (Appendix D), but not for testing.

Original: To keep the size of the environment image ﬁxed, we pad the map with wall blocks if needed.
Revised: To keep the size of the environment image ﬁxed, we pad the map with wall blocks if needed.

Original: The agent has four navigation actions in total: left , right , up , and down .
Revised: The agent has four navigation actions in total: left , right , up , and down .

Original: In each session,
Revised: In each session,

Original: I The maximum number of time steps is a constant computed as four times the map dimension ( 7 ˆ 4 “ 28 ).
Revised: I The maximum number of time steps is a constant computed as four times the map dimension ( 7 ˆ 4 “ 28 ).

Original: That is, the agent only has 28 steps to reach a target before time out.
Revised: That is, the agent only has 28 steps to reach a target before time out.

Original: II The number of objects on the map ranges from 1 to 5.
Revised: II The number of objects on the map ranges from 1 to 5.

Original: III
Revised: III

Original: The number of wall blocks on the map ranges from 0 to 15.
Revised: The number of wall blocks on the map ranges from 0 to 15.

Original: IV
Revised: IV

Original: The positive reward when the agent reaches the correct location is 1 . 0 .
Revised: The positive reward when the agent reaches the correct location is 1 . 0 .

Original: V The negative rewards for hitting walls and for stepping on non-target objects are ´ 0 .
Revised: V The negative rewards for hitting walls and for stepping on non-target objects are ´ 0 .

Original: 2 and ´ 1 . 0 , respectively.
Revised: 2 and ´ 1 . 0 , respectively.

Original: [Table]
Revised: [Table]

Original: VI The time penalty of each step is ´ 0 .
Revised: VI The time penalty of each step is ´ 0 .

Original: The teacher has a vocabulary size of 185.
Revised: The teacher has a vocabulary size of 185.

Original: There are 9 spatial relations, 8 colors, 119 distinct object classes, and 50 grammatical words.
Revised: There are 9 spatial relations, 8 colors, 119 distinct object classes, and 50 grammatical words.

Original: Every object class contains 3 different instances.
Revised: Every object class contains 3 different instances.

Original: All object instances are shown in Figure 11.
Revised: All object instances are shown in Figure 11.

Original: Every time the environment is reset, a number of object classes are randomly sampled and an object instance is randomly sampled for each class.
Revised: Every time the environment is reset, a number of object classes are randomly sampled and an object instance is randomly sampled for each class.

Original: There are in total 16 types of sentences the teacher can speak, including 4 types of NAV commands and 12 types of QA questions.
Revised: There are in total 16 types of sentences the teacher can speak, including 4 types of NAV commands and 12 types of QA questions.

Original: Each sentence type has several non-recursive templates, and corresponds to a concrete type of tasks the agent must learn to accomplish.
Revised: Each sentence type has several non-recursive templates, and corresponds to a concrete type of tasks the agent must learn to accomplish.

Original: In total there are 1,639,015 distinct sentences with 567,579 for NAV and 1,071,436 for QA.
Revised: In total there are 1,639,015 distinct sentences with 567,579 for NAV and 1,071,436 for QA.

Original: The sentence length varies between 2 and 13.
Revised: The sentence length varies between 2 and 13.

Original: The object, spatial-relation, and color words of the teacher’s language are listed in Table 1.
Revised: The object, spatial-relation, and color words of the teacher’s language are listed in Table 1.

Original: These are the content words that can be grounded in XWORLD .
Revised: These are the content words that can be grounded in XWORLD .

Original: All the others are grammatical words which serve to support sentence structures.
Revised: All the others are grammatical words which serve to support sentence structures.

Original: Note that the differentiation between the content and the grammatical words is automatically learned by the agent according to the tasks.
Revised: Note that the differentiation between the content and the grammatical words is automatically learned by the agent according to the tasks.

Original: Every word is equally represented by an entry in the word embedding table.
Revised: Every word is equally represented by an entry in the word embedding table.

Original: The sentence types that the teacher can speak are listed in Table 2.
Revised: The sentence types that the teacher can speak are listed in Table 2.

Original: Each type has a triggering condition about when the teacher says that type of sentences.
Revised: Each type has a triggering condition about when the teacher says that type of sentences.

Original: Besides the conditions shown, an extra condition for NAV commands is that the target must be reachable from the current agent location.
Revised: Besides the conditions shown, an extra condition for NAV commands is that the target must be reachable from the current agent location.

Original: An extra condition for color-related questions is that the object color must be one of the eight deﬁned colors.
Revised: An extra condition for color-related questions is that the object color must be one of the eight deﬁned colors.

Original: If at any time step there are multiple types triggered, we randomly sample one for NAV and another for QA.
Revised: If at any time step there are multiple types triggered, we randomly sample one for NAV and another for QA.

Original: After a sentence type is sampled, we generate a sentence according to the corresponding sentence templates.
Revised: After a sentence type is sampled, we generate a sentence according to the corresponding sentence templates.

Original: B I MPLEMENTATION DETAILS
Revised: B I MPLEMENTATION DETAILS

Original: The environment image e is a 156 ˆ 156 ego-centric RGB image.
Revised: The environment image e is a 156 ˆ 156 ego-centric RGB image.

Original: The CNN in F has four convolutional layers: p 3 , 3 , 64 q , p 2 , 2 , 64 q , p 2 , 2 , 256 q , p 1 , 1 , 256 q , where p x, y, z q represents a layer conﬁguration of z kernels of size x applied at stride width y .
Revised: The CNN in F has four convolutional layers: p 3 , 3 , 64 q , p 2 , 2 , 64 q , p 2 , 2 , 256 q , p 1 , 1 , 256 q , where p x, y, z q represents a layer conﬁguration of z kernels of size x applied at stride width y .

Original: All the four layers are ReLU activated.
Revised: All the four layers are ReLU activated.

Original: To enable the agent to reason about spatial-relation words ( e.g. , “north”), we append an additional parametric cube to the convolutional output to obtain h .
Revised: To enable the agent to reason about spatial-relation words ( e.g. , “north”), we append an additional parametric cube to the convolutional output to obtain h .

Original: This parametric cube has the same number of channels with the CNN output, and it is initialized with zero mean and zero standard deviation.
Revised: This parametric cube has the same number of channels with the CNN output, and it is initialized with zero mean and zero standard deviation.

Original: The word embedding table is initialized with zero mean and unit standard deviation.
Revised: The word embedding table is initialized with zero mean and unit standard deviation.

Original: All the gated
Revised: All the gated

Original: RNNs (including the bidirectional RNN) in L have 128 units.
Revised: RNNs (including the bidirectional RNN) in L have 128 units.

Original: All the layers in L , unless otherwise stated, use Tanh as the activation function.
Revised: All the layers in L , unless otherwise stated, use Tanh as the activation function.

Original: [Table]
Revised: [Table]

Original: rec loc2col
Revised: rec loc2col

Original: [Table]
Revised: [Table]

Original: rec
Revised: rec

Original: [Table]
Revised: [Table]

Original: rec loc
Revised: rec loc

Original: obj2obj
Revised: obj2obj

Original: [Table]
Revised: [Table]

Original: loc
Revised: loc

Original: [Table]
Revised: [Table]

Original: obj2loc
Revised: obj2loc

Original: [Table] [Table]
Revised: [Table] [Table]

Original: The
Revised: The

Original: [Table]
Revised: [Table]

Original: is near the block which is between the two reference objects.
Revised: is near the block which is between the two reference objects.

Original: rec bw obj2col
Revised: rec bw obj2col

Original: [Table]
Revised: [Table]

Original: For NAV, x is used as the target to reach on the image plane.
Revised: For NAV, x is used as the target to reach on the image plane.

Original: However, knowing this alone does not sufﬁce.
Revised: However, knowing this alone does not sufﬁce.

Original: The agent also has to be aware of walls and possibly confounding targets (other objects) in the environment.
Revised: The agent also has to be aware of walls and possibly confounding targets (other objects) in the environment.

Original: Toward this end, M A further computes an environment map g “ σ p hf q where f P R D is a parameter vector to be learned and σ is Sigmoid.
Revised: Toward this end, M A further computes an environment map g “ σ p hf q where f P R D is a parameter vector to be learned and σ is Sigmoid.

Original: We expect that g detects any blocks informative for navigation.
Revised: We expect that g detects any blocks informative for navigation.

Original: Note that g is unrelated to the speciﬁc command; it solely depends on the current environment.
Revised: Note that g is unrelated to the speciﬁc command; it solely depends on the current environment.

Original: After stacking x and g together, M A feeds them to another CNN followed by an MLP to obtain q .
Revised: After stacking x and g together, M A feeds them to another CNN followed by an MLP to obtain q .

Original: The CNN has two convolutional layers p 3 , 1 , 64 q and p 3 , 1 , 4 q , both with paddings of 1 .
Revised: The CNN has two convolutional layers p 3 , 1 , 64 q and p 3 , 1 , 4 q , both with paddings of 1 .

Original: It is followed by a three-layer MLP where each layer has 512 units and is ReLU activated.
Revised: It is followed by a three-layer MLP where each layer has 512 units and is ReLU activated.

Original: The action module A contains a two-layer MLP of which the ﬁrst layer has 512 ReLU activated units and the second layer is a Softmax whose output dimension is equal to the number of actions.
Revised: The action module A contains a two-layer MLP of which the ﬁrst layer has 512 ReLU activated units and the second layer is a Softmax whose output dimension is equal to the number of actions.

Original: We use Adagrad (Duchi et al., 2011) with a learning rate of 10 ´ 5 for Stochastic Gradient Descent (SGD).
Revised: We use Adagrad (Duchi et al., 2011) with a learning rate of 10 ´ 5 for Stochastic Gradient Descent (SGD).

Original: The reward discount factor is set to 0 .
Revised: The reward discount factor is set to 0 .

Original: All the parameters have a default weight decay of 10 ´ 4 ˆ 16 .
Revised: All the parameters have a default weight decay of 10 ´ 4 ˆ 16 .

Original: For each layer, its parameters have zero mean and a standard deviation of 1 { ?
Revised: For each layer, its parameters have zero mean and a standard deviation of 1 { ?

Original: K , where K is the number of parameters of that layer.
Revised: K , where K is the number of parameters of that layer.

Original: The agent has 1m exploration steps in total, and the exploration rate λ decreases linearly from 1 to 0 .
Revised: The agent has 1m exploration steps in total, and the exploration rate λ decreases linearly from 1 to 0 .

Original: At each time step, the agent takes an action a with a probability of
Revised: At each time step, the agent takes an action a with a probability of

Original: [Equation]
Revised: [Equation]

Original: q where π is the current policy.
Revised: q where π is the current policy.

Original: We set the maximum interpretation step I “ 3 .
Revised: We set the maximum interpretation step I “ 3 .

Original: The whole model is trained end to end.
Revised: The whole model is trained end to end.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: C B ASELINE DETAILS
Revised: C B ASELINE DETAILS

Original: Some additional implementation details of the baselines in section 4.3 are described below.
Revised: Some additional implementation details of the baselines in section 4.3 are described below.

Original: [CA] Its RNN has 512 units.
Revised: [CA] Its RNN has 512 units.

Original: [VL] Its CNN has four convolutional layers p 3 , 2 , 64 q , p 3 , 2 , 64 q , p 3 , 2 , 128 q , and p 3 , 1 , 128 q .
Revised: [VL] Its CNN has four convolutional layers p 3 , 2 , 64 q , p 3 , 2 , 64 q , p 3 , 2 , 128 q , and p 3 , 1 , 128 q .

Original: This is followed by an FC layer of size 512 , which projects the feature cube to the word embedding space.
Revised: This is followed by an FC layer of size 512 , which projects the feature cube to the word embedding space.

Original: The RNN has 512 units.
Revised: The RNN has 512 units.

Original: For either QA or NAV, the RNN’s last state goes through a three-layer
Revised: For either QA or NAV, the RNN’s last state goes through a three-layer

Original: MLP of which each layer has 512 units (Figure 12).
Revised: MLP of which each layer has 512 units (Figure 12).

Original: [CE] It has the same layer-size conﬁguration with VL (Figure 13).
Revised: [CE] It has the same layer-size conﬁguration with VL (Figure 13).

Original: [SAN] Its RNN has 256 units.
Revised: [SAN] Its RNN has 256 units.

Original: Following the original approach (Yang et al., 2016), we use two attention layers.
Revised: Following the original approach (Yang et al., 2016), we use two attention layers.

Original: All the layers of the above baselines are ReLU activated.
Revised: All the layers of the above baselines are ReLU activated.

Original: D E XPERIENCE REPLAY AND CURRICULUM LEARNING
Revised: D E XPERIENCE REPLAY AND CURRICULUM LEARNING

Original: We employ Experience Replay (Mnih et al., 2015) for training both NAV and QA, for all the approaches.
Revised: We employ Experience Replay (Mnih et al., 2015) for training both NAV and QA, for all the approaches.

Original: The environment inputs, rewards, and the actions taken by the agent in the most recent 10k time steps are stored in a replay buffer.
Revised: The environment inputs, rewards, and the actions taken by the agent in the most recent 10k time steps are stored in a replay buffer.

Original: During training, each time two mini-batches of the same size are sampled from the buffer, one for NAV and the other for QA.
Revised: During training, each time two mini-batches of the same size are sampled from the buffer, one for NAV and the other for QA.

Original: The mini-batch for NAV is sampled from the replay buffer using the Rank-based Sampler (Schaul et al., 2016) which has proven to increase the training efﬁciency by prioritizing rare experiences during sampling.
Revised: The mini-batch for NAV is sampled from the replay buffer using the Rank-based Sampler (Schaul et al., 2016) which has proven to increase the training efﬁciency by prioritizing rare experiences during sampling.

Original: We exploit Curriculum Learning (Bengio et al., 2009) to gradually increase the environment complexity to help the agent learn.
Revised: We exploit Curriculum Learning (Bengio et al., 2009) to gradually increase the environment complexity to help the agent learn.

Original: We increase the following quantities in proportional to min p 1 , G 1 { G q , where G 1 is the number of sessions trained so far and G is the total number of curriculum sessions:
Revised: We increase the following quantities in proportional to min p 1 , G 1 { G q , where G 1 is the number of sessions trained so far and G is the total number of curriculum sessions:

Original: I The size of the open space on the environment map.
Revised: I The size of the open space on the environment map.

Original: II The number of objects in the environment.
Revised: II The number of objects in the environment.

Original: III
Revised: III

Original: The number of wall blocks.
Revised: The number of wall blocks.

Original: IV
Revised: IV

Original: The number of object classes that can be sampled from.
Revised: The number of object classes that can be sampled from.

Original: V
Revised: V

Original: [Table]
Revised: [Table]

Original: NAV
Revised: NAV

Original: [Table]
Revised: [Table]

Original: QA question.
Revised: QA question.

Original: We found that this curriculum is important for an efﬁcient learning.
Revised: We found that this curriculum is important for an efﬁcient learning.

Original: In the experiments, we set
Revised: In the experiments, we set

Original: G “ 25 k during training while do not have any curriculum during test ( i.e. , test with maximal difﬁculty).
Revised: G “ 25 k during training while do not have any curriculum during test ( i.e. , test with maximal difﬁculty).