Original: S IMULATING A CTION D YNAMICS WITH N EURAL P ROCESS N ETWORKS
Revised: S IMULATING A CTION D YNAMICS WITH N EURAL P ROCESS N ETWORKS

Original: A BSTRACT
Revised: A BSTRACT

Original: Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated.
Revised: Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated.

Original: In this work, we introduce Neural
Revised: In this work, we introduce Neural

Original: Process Networks to understand procedural text through (neural) simulation of action dynamics.
Revised: Process Networks to understand procedural text through (neural) simulation of action dynamics.

Original: Our model complements existing memory architectures with dynamic entity tracking by explicitly modeling actions as state transformers.
Revised: Our model complements existing memory architectures with dynamic entity tracking by explicitly modeling actions as state transformers.

Original: The model updates the states of the entities by executing learned action operators.
Revised: The model updates the states of the entities by executing learned action operators.

Original: Empirical results demonstrate that our proposed model can reason about the unstated causal effects of actions, allowing it to provide more accurate contextual information for understanding and generating procedural text, all while offering more interpretable internal representations than existing alternatives.
Revised: Empirical results demonstrate that our proposed model can reason about the unstated causal effects of actions, allowing it to provide more accurate contextual information for understanding and generating procedural text, all while offering more interpretable internal representations than existing alternatives.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Understanding procedural text such as instructions or stories requires anticipating the implicit causal effects of actions on entities.
Revised: Understanding procedural text such as instructions or stories requires anticipating the implicit causal effects of actions on entities.

Original: For example, given instructions such as “ add blueberries to the mufﬁn mix, then bake for one half hour,” an intelligent agent must be able to anticipate a number of entailed facts (e.g., the blueberries are now in the oven; their “temperature” will increase).
Revised: For example, given instructions such as “ add blueberries to the mufﬁn mix, then bake for one half hour,” an intelligent agent must be able to anticipate a number of entailed facts (e.g., the blueberries are now in the oven; their “temperature” will increase).

Original: While this common sense reasoning is trivial for humans, most natural language understanding algorithms do not have the capacity to reason about causal effects not mentioned directly in the surface strings (Levy et al., 2015; Jia & Liang, 2017; Lucy & Gauthier, 2017).
Revised: While this common sense reasoning is trivial for humans, most natural language understanding algorithms do not have the capacity to reason about causal effects not mentioned directly in the surface strings (Levy et al., 2015; Jia & Liang, 2017; Lucy & Gauthier, 2017).

Original: + f wash f cut “Wash and cut the tomatoes”
Revised: + f wash f cut “Wash and cut the tomatoes”

Original: G ( ) f _ ,
Revised: G ( ) f _ ,

Original: [Figure]
Revised: [Figure]

Original: e tomato
Revised: e tomato

Original: Later
Revised: Later

Original: In this paper, we introduce Neural Process Networks, a procedural language understanding system that tracks common sense attributes through neural simulation of action dynamics.
Revised: In this paper, we introduce Neural Process Networks, a procedural language understanding system that tracks common sense attributes through neural simulation of action dynamics.

Original: Our network models interpretation of natural language instructions as a process of actions and their cumulative effects on entities.
Revised: Our network models interpretation of natural language instructions as a process of actions and their cumulative effects on entities.

Original: More concretely, reading one sentence at a time, our model attentively selects what actions to execute on which entities, and remembers the state changes induced with a recurrent memory structure.
Revised: More concretely, reading one sentence at a time, our model attentively selects what actions to execute on which entities, and remembers the state changes induced with a recurrent memory structure.

Original: In
Revised: In

Original: Figure 1, for example, our model indexes the “tomato” embedding, selects the “wash” and “cut” functions and performs a computation that changes the “tomato” embedding so that it can reason about attributes such as its “ SHAPE ” and “ CLEANLINESS ”.
Revised: Figure 1, for example, our model indexes the “tomato” embedding, selects the “wash” and “cut” functions and performs a computation that changes the “tomato” embedding so that it can reason about attributes such as its “ SHAPE ” and “ CLEANLINESS ”.

Original: Our model contributes to a recent line of research that aims to model aspects of world state changes, such as language models and machine readers with explicit entity representations (Henaff et al., 2016;
Revised: Our model contributes to a recent line of research that aims to model aspects of world state changes, such as language models and machine readers with explicit entity representations (Henaff et al., 2016;

Original: Yang et al., 2016; Ji et al., 2017), as well as other more general purpose memory network variants (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016).
Revised: Yang et al., 2016; Ji et al., 2017), as well as other more general purpose memory network variants (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016).

Original: This worldcentric modeling of procedural language (i.e., understanding by simulation) abstracts away from the surface strings, complementing text-centric modeling of language, which focuses on syntactic and semantic labeling of surface words (i.e., understanding by labeling).
Revised: This worldcentric modeling of procedural language (i.e., understanding by simulation) abstracts away from the surface strings, complementing text-centric modeling of language, which focuses on syntactic and semantic labeling of surface words (i.e., understanding by labeling).

Original: Unlike previous approaches, however, our model also learns explicit action representations as functional operators (See Figure 1).
Revised: Unlike previous approaches, however, our model also learns explicit action representations as functional operators (See Figure 1).

Original: While representations of action semantics could be acquired through an embodied agent that can see and interact with the world (Oh et al., 2015), we propose to learn these representations from text.
Revised: While representations of action semantics could be acquired through an embodied agent that can see and interact with the world (Oh et al., 2015), we propose to learn these representations from text.

Original: In particular, we require the model to be able to explain the causal effects of actions by predicting natural language attributes about entities such as “ LOCATION ” and “ TEMPERATURE ”.
Revised: In particular, we require the model to be able to explain the causal effects of actions by predicting natural language attributes about entities such as “ LOCATION ” and “ TEMPERATURE ”.

Original: The model adjusts its representations of actions based on errors it makes in predicting the resultant state changes to attributes.
Revised: The model adjusts its representations of actions based on errors it makes in predicting the resultant state changes to attributes.

Original: This textual simulation allows us to model aspects of action causality that are not readily available in existing simulation environments.
Revised: This textual simulation allows us to model aspects of action causality that are not readily available in existing simulation environments.

Original: Indeed, most virtual environments offer limited aspects of the world – with a primary focus on spatial relations (Oh et al., 2015; Chiappa et al., 2017; Wahlstrom et al., 2015).
Revised: Indeed, most virtual environments offer limited aspects of the world – with a primary focus on spatial relations (Oh et al., 2015; Chiappa et al., 2017; Wahlstrom et al., 2015).

Original: They leave out various other dimensions of the world states that are implied by diverse everyday actions such as “dissolve” (change of “ COMPOSITION ”) and “wash” (change of “ CLEANLINESS ”).
Revised: They leave out various other dimensions of the world states that are implied by diverse everyday actions such as “dissolve” (change of “ COMPOSITION ”) and “wash” (change of “ CLEANLINESS ”).

Original: Empirical results demonstrate that parametrizing explicit action embeddings provides an inductive bias that allows the neural process network to learn more informative context representations for understanding and generating natural language procedural text.
Revised: Empirical results demonstrate that parametrizing explicit action embeddings provides an inductive bias that allows the neural process network to learn more informative context representations for understanding and generating natural language procedural text.

Original: In addition, our model offers more interpretable internal representations and can reason about the unstated causal effects of actions explained through natural language descriptors.
Revised: In addition, our model offers more interpretable internal representations and can reason about the unstated causal effects of actions explained through natural language descriptors.

Original: Finally, we include a new dataset with ﬁne-grained annotations on state changes, to be shared publicly, to encourage future research in this direction.
Revised: Finally, we include a new dataset with ﬁne-grained annotations on state changes, to be shared publicly, to encourage future research in this direction.

Original:  N EURAL P ROCESS N ETWORK
Revised:  N EURAL P ROCESS N ETWORK

Original: The neural process network is an interpreter that reads in natural language sentences, one at a time, and simulates the process of actions being applied to relevant entities through learned representations of actions and entities.
Revised: The neural process network is an interpreter that reads in natural language sentences, one at a time, and simulates the process of actions being applied to relevant entities through learned representations of actions and entities.

Original: 2. O VERVIEW AND N OTATION
Revised: 2. O VERVIEW AND N OTATION

Original: The main component of the neural process network is the simulation module ( § 2.5), a recurrent unit whose internals simulate the effects of actions being applied to entities.
Revised: The main component of the neural process network is the simulation module ( § 2.5), a recurrent unit whose internals simulate the effects of actions being applied to entities.

Original: A set of V actions is known a priori and an embedding is initialized for each one, F = { f 1 , ...
Revised: A set of V actions is known a priori and an embedding is initialized for each one, F = { f 1 , ...

Original: f V } .
Revised: f V } .

Original: Similarly, a set of
Revised: Similarly, a set of

Original: I entities is known and an embedding is initialized for each one: E = { e 1 , ...
Revised: I entities is known and an embedding is initialized for each one: E = { e 1 , ...

Original: e I } .
Revised: e I } .

Original: Each e i can be considered to encode information about state attributes of that entity, which can be extracted by a set of state predictors ( § 2.6).
Revised: Each e i can be considered to encode information about state attributes of that entity, which can be extracted by a set of state predictors ( § 2.6).

Original: As the model reads text, it “applies” action embeddings to the entity vectors, thereby changing the state information encoded about the entities.
Revised: As the model reads text, it “applies” action embeddings to the entity vectors, thereby changing the state information encoded about the entities.

Original: For any document d , an initial list of entities I d is known and E d = { e i | i ∈ I d } ⊂ E entity state embeddings are initialized.
Revised: For any document d , an initial list of entities I d is known and E d = { e i | i ∈ I d } ⊂ E entity state embeddings are initialized.

Original: As the neural process network reads a sentence from the document, it selects a subset of both F ( § 2.3) and E d ( § 2.4) based on the actions performed and entities affected in the sentence.
Revised: As the neural process network reads a sentence from the document, it selects a subset of both F ( § 2.3) and E d ( § 2.4) based on the actions performed and entities affected in the sentence.

Original: The entity state embeddings are changed by the action and the new embeddings are used to predict end states for a set of state changes ( § 2.6).
Revised: The entity state embeddings are changed by the action and the new embeddings are used to predict end states for a set of state changes ( § 2.6).

Original: The prediction error for end states is backpropagated to the action embeddings, learning action representations that model the simulation of desired causal effects on entities.
Revised: The prediction error for end states is backpropagated to the action embeddings, learning action representations that model the simulation of desired causal effects on entities.

Original: This process is broken down into ﬁve modules below.
Revised: This process is broken down into ﬁve modules below.

Original: Unless explicitly deﬁned, all W and b variables are parametrized linear projections and biases.
Revised: Unless explicitly deﬁned, all W and b variables are parametrized linear projections and biases.

Original: We use the notation { e i } t when referring to the values of the entity embeddings before processing sentence s t .
Revised: We use the notation { e i } t when referring to the values of the entity embeddings before processing sentence s t .

Original: 2. S ENTENCE E NCODER
Revised: 2. S ENTENCE E NCODER

Original: Given a sentence s t , a Gated Recurrent Unit (Cho et al., 2014) encodes each word and outputs its last hidden vector as a sentence encoding h t (Sutskever et al., 2014).
Revised: Given a sentence s t , a Gated Recurrent Unit (Cho et al., 2014) encodes each word and outputs its last hidden vector as a sentence encoding h t (Sutskever et al., 2014).

Original: 2. A CTION S ELECTOR
Revised: 2. A CTION S ELECTOR

Original: Given h t from the sentence encoder, the action selector (bottom left in Fig. 2) contextually determines which action(s) from F to execute.
Revised: Given h t from the sentence encoder, the action selector (bottom left in Fig. 2) contextually determines which action(s) from F to execute.

Original: For example, if the input sentence is “wash and cut
Revised: For example, if the input sentence is “wash and cut

Original: [Figure]
Revised: [Figure]

Original: GRU GRU
Revised: GRU GRU

Original: [Figure]
Revised: [Figure]

Original: GRU
Revised: GRU

Original: Cook in pan .
Revised: Cook in pan .

Original: f _
Revised: f _

Original: Simulation Module to selectors ) ( e _ h T
Revised: Simulation Module to selectors ) ( e _ h T

Original: E n t
Revised: E n t

Original: i t y S e l ec
Revised: i t y S e l ec

Original: [Figure]
Revised: [Figure]

Original: e garlic e beef e tomato e salt e pepper e onion ℰ ×
Revised: e garlic e beef e tomato e salt e pepper e onion ℰ ×

Original: Recurrent Attention (Eq. 3)
Revised: Recurrent Attention (Eq. 3)

Original: Sequence Attention (Eq. 2) h t h t
Revised: Sequence Attention (Eq. 2) h t h t

Original: Applicator
Revised: Applicator

Original: A c t i o n S e l ec t
Revised: A c t i o n S e l ec t

Original: o r
Revised: o r

Original: Entity Updater (Eq. 7)
Revised: Entity Updater (Eq. 7)

Original: M L P 𝓕 … … … …
Revised: M L P 𝓕 … … … …

Original: f roast f cut f wash f cook f
Revised: f roast f cut f wash f cook f

Original: [Figure]
Revised: [Figure]

Original: f cook f put + 𝓕 w p ×
Revised: f cook f put + 𝓕 w p ×

Original: GRU beef
Revised: GRU beef

Original: [Figure]
Revised: [Figure]

Original: } k t
Revised: } k t

Original: [Figure]
Revised: [Figure]

Original: P
Revised: P

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: s
Revised: s

Original: Location hot
Revised: Location hot

Original: Cooked? <
Revised: Cooked? <

Original: NO_CHANGE> k t pan
Revised: NO_CHANGE> k t pan

Original: Temp? …
Revised: Temp? …

Original: Clean? cooked beets” , both f wash and f cut must be selected.
Revised: Clean? cooked beets” , both f wash and f cut must be selected.

Original: To account for multiple actions, we make a soft selection over F , yielding a weighted sum of the selected action embeddings ¯ f t :
Revised: To account for multiple actions, we make a soft selection over F , yielding a weighted sum of the selected action embeddings ¯ f t :

Original: [Equation]
Revised: [Equation]

Original: where MLP is a parametrized feed-forward network with a sigmoid activation and w p ∈ R V is the attention distribution over V possible actions ( § 3.1).
Revised: where MLP is a parametrized feed-forward network with a sigmoid activation and w p ∈ R V is the attention distribution over V possible actions ( § 3.1).

Original: We compose the action embedding by taking the weighted average of the selected actions.
Revised: We compose the action embedding by taking the weighted average of the selected actions.

Original: E NTITY S ELECTOR
Revised: E NTITY S ELECTOR

Original: Sentence Attention Given h t from the sentence encoder, the entity selector chooses relevant entities using a soft attention mechanism:
Revised: Sentence Attention Given h t from the sentence encoder, the entity selector chooses relevant entities using a soft attention mechanism:

Original: [Equation]
Revised: [Equation]

Original: where W 2 is a bilinear mapping, e i 0 is a unique key for each entity ( § 2.5), and d i is the attention weight for entity embedding e i .
Revised: where W 2 is a bilinear mapping, e i 0 is a unique key for each entity ( § 2.5), and d i is the attention weight for entity embedding e i .

Original: For example, in “wash and cut beets and carrots” , the model should select e beet and e carrot .
Revised: For example, in “wash and cut beets and carrots” , the model should select e beet and e carrot .

Original: Recurrent Attention While sentence attention would sufﬁce if entities were always explicitly mentioned, natural language often elides arguments or uses referent pronouns.
Revised: Recurrent Attention While sentence attention would sufﬁce if entities were always explicitly mentioned, natural language often elides arguments or uses referent pronouns.

Original: As such, the module must be able to consider entities mentioned in previous sentences.
Revised: As such, the module must be able to consider entities mentioned in previous sentences.

Original: Using ˜ h t , the model computes a soft choice over whether to choose affected entities from this step’s attention d i or the previous step’s attention distribution.
Revised: Using ˜ h t , the model computes a soft choice over whether to choose affected entities from this step’s attention d i or the previous step’s attention distribution.

Original: [Equation]
Revised: [Equation]

Original: where c ∈ R 3 is the choice distribution, a i t − 1 is the previous sentence’s attention weight for each entity, a i t is the ﬁnal attention for each entity, and 0 is a vector of zeroes (providing the option to not change any entity).
Revised: where c ∈ R 3 is the choice distribution, a i t − 1 is the previous sentence’s attention weight for each entity, a i t is the ﬁnal attention for each entity, and 0 is a vector of zeroes (providing the option to not change any entity).

Original: Prior entity attentions can propagate forward for multiple steps.
Revised: Prior entity attentions can propagate forward for multiple steps.

Original: 2. S IMULATION M ODULE
Revised: 2. S IMULATION M ODULE

Original: Entity Memory A unique state embedding e i is initialized for every entity i in the document.
Revised: Entity Memory A unique state embedding e i is initialized for every entity i in the document.

Original: A unique key to index each embedding e i 0 is set as the initial value of the embedding (Henaff et al., 2016; Miller et al., 2016).
Revised: A unique key to index each embedding e i 0 is set as the initial value of the embedding (Henaff et al., 2016; Miller et al., 2016).

Original: After the model reads s t , it modiﬁes { e i } t to reﬂect changes inﬂuenced by actions.
Revised: After the model reads s t , it modiﬁes { e i } t to reﬂect changes inﬂuenced by actions.

Original: At every time step, the entity memory receives the attention weights from the entity selector, normalizes them and computes a weighted average of the relevant entity state embeddings:
Revised: At every time step, the entity memory receives the attention weights from the entity selector, normalizes them and computes a weighted average of the relevant entity state embeddings:

Original: [Equation]
Revised: [Equation]

Original: Applicator Given the action summary embedding ¯ f t and the entity summary embedding ¯ e t , the applicator (middle right in Fig. 2) applies the selected actions to the selected entities, and outputs the new proposal entity embedding k t .
Revised: Applicator Given the action summary embedding ¯ f t and the entity summary embedding ¯ e t , the applicator (middle right in Fig. 2) applies the selected actions to the selected entities, and outputs the new proposal entity embedding k t .

Original: [Equation]
Revised: [Equation]

Original: where W 4 is a third order tensor projection.
Revised: where W 4 is a third order tensor projection.

Original: The vector k t is the new representation of the entity ¯ e t after the applicator simulates the action being applied to it.
Revised: The vector k t is the new representation of the entity ¯ e t after the applicator simulates the action being applied to it.

Original: Entity Updater
Revised: Entity Updater

Original: The entity updater interpolates the new proposal entity embedding k t and the set of current entity embeddings { e i } t :
Revised: The entity updater interpolates the new proposal entity embedding k t and the set of current entity embeddings { e i } t :

Original: [Equation]
Revised: [Equation]

Original: yielding an updated set of entity embeddings { e i } t +1 .
Revised: yielding an updated set of entity embeddings { e i } t +1 .

Original: Each embedding is updated proportional to its entity’s unnormalized attention a i , allowing the model to completely overwrite the state embedding for any entity.
Revised: Each embedding is updated proportional to its entity’s unnormalized attention a i , allowing the model to completely overwrite the state embedding for any entity.

Original: For example, in the sentence “mix the ﬂour and water,” the embeddings for e flour and e water must both be overwritten by k t because they no longer exist outside of this new composition.
Revised: For example, in the sentence “mix the ﬂour and water,” the embeddings for e flour and e water must both be overwritten by k t because they no longer exist outside of this new composition.

Original: 2. S TATE P REDICTORS
Revised: 2. S TATE P REDICTORS

Original: Given the new proposal entity embedding k t , the state predictor (bottom right in Fig. 2) predicts changes to the resulting entity embedding k t along the following six dimensions: location, cookedness, temperature, composition, shape, and cleanliness.
Revised: Given the new proposal entity embedding k t , the state predictor (bottom right in Fig. 2) predicts changes to the resulting entity embedding k t along the following six dimensions: location, cookedness, temperature, composition, shape, and cleanliness.

Original: Discrete multi-class classiﬁers, one for each dimension, take in k t and predict a unique end state for their corresponding state change type:
Revised: Discrete multi-class classiﬁers, one for each dimension, take in k t and predict a unique end state for their corresponding state change type:

Original: [Equation]
Revised: [Equation]

Original: For location changes, which require contextual information to predict the end state, k t is concatenated with the original sentence representation h t to predict the ﬁnal state.
Revised: For location changes, which require contextual information to predict the end state, k t is concatenated with the original sentence representation h t to predict the ﬁnal state.

Original:  T RAINING 3. S TATE C HANGE K NOWLEDGE
Revised:  T RAINING 3. S TATE C HANGE K NOWLEDGE

Original: In this work we focus on physical action verbs in cooking recipes.
Revised: In this work we focus on physical action verbs in cooking recipes.

Original: We manually collect a set of 384 actions such as cut, bake, boil, arrange , and place , organizing their causal effects along the following predeﬁned dimensions: LOCATION , COOKEDNESS , TEMPERATURE , SHAPE , CLEANLINESS and COMPOSITION .
Revised: We manually collect a set of 384 actions such as cut, bake, boil, arrange , and place , organizing their causal effects along the following predeﬁned dimensions: LOCATION , COOKEDNESS , TEMPERATURE , SHAPE , CLEANLINESS and COMPOSITION .

Original: The textual simulation operated by the model induces state changes along these dimensions by applying actions functions from the above set of 384.
Revised: The textual simulation operated by the model induces state changes along these dimensions by applying actions functions from the above set of 384.

Original: For example, cut entails a change in SHAPE , while bake entails a change in TEMPERATURE , COOKEDNESS , and even LOCATION .
Revised: For example, cut entails a change in SHAPE , while bake entails a change in TEMPERATURE , COOKEDNESS , and even LOCATION .

Original: We annotate the state changes each action induces, as well as the end state of the action, using Amazon Mechanical Turk.
Revised: We annotate the state changes each action induces, as well as the end state of the action, using Amazon Mechanical Turk.

Original: The set of possible end states for a state change can range fromfor binary state changes to more than 200 (See Appendix C for details).
Revised: The set of possible end states for a state change can range fromfor binary state changes to more than 200 (See Appendix C for details).

Original: Table 1 provides examples of annotations in this action lexicon.
Revised: Table 1 provides examples of annotations in this action lexicon.

Original: [Table]
Revised: [Table]

Original: D ATASET
Revised: D ATASET

Original: For learning and evaluation, we use a subset of the Now You’re Cooking dataset (Kiddon et al., 2016).
Revised: For learning and evaluation, we use a subset of the Now You’re Cooking dataset (Kiddon et al., 2016).

Original: We chose 65816 recipes for training, 175 recipes for development, and 700 recipes for testing.
Revised: We chose 65816 recipes for training, 175 recipes for development, and 700 recipes for testing.

Original: For the development and test sets, crowdsourced workers densely annotate actions, entities and state changes that occur in each sentence so that we can tune hyperparameters and evaluate on gold evaluation sets.
Revised: For the development and test sets, crowdsourced workers densely annotate actions, entities and state changes that occur in each sentence so that we can tune hyperparameters and evaluate on gold evaluation sets.

Original: Annotation details are provided in Appendix C.3.
Revised: Annotation details are provided in Appendix C.3.

Original: 3. C OMPONENT - WISE T RAINING
Revised: 3. C OMPONENT - WISE T RAINING

Original: The neural process network is trained by jointly optimizing multiple losses for the action selector, entity selector, and state change predictors.
Revised: The neural process network is trained by jointly optimizing multiple losses for the action selector, entity selector, and state change predictors.

Original: Importantly, our training scheme uses weak supervision because dense annotations are prohibitively expensive to acquire at a very large scale.
Revised: Importantly, our training scheme uses weak supervision because dense annotations are prohibitively expensive to acquire at a very large scale.

Original: Thus, we heuristically extract verb mentions from each recipe step and assign a state change label based on the state changes induced by that action ( § 3.1).
Revised: Thus, we heuristically extract verb mentions from each recipe step and assign a state change label based on the state changes induced by that action ( § 3.1).

Original: Entities are extracted similarly based on string matching between the instructions and the ingredient list.
Revised: Entities are extracted similarly based on string matching between the instructions and the ingredient list.

Original: We use the following losses for training:
Revised: We use the following losses for training:

Original: Action Selection Loss Using noisy supervision, the action selector is trained to minimize the cross-entropy loss for each possible action, allowing multiple actions to be chosen at each step if multiple actions are mentioned in a sentence.
Revised: Action Selection Loss Using noisy supervision, the action selector is trained to minimize the cross-entropy loss for each possible action, allowing multiple actions to be chosen at each step if multiple actions are mentioned in a sentence.

Original: The MLP in the action selector (Eq. 1) is pretrained.
Revised: The MLP in the action selector (Eq. 1) is pretrained.

Original: Entity Selection Loss Similarly, to train the attentive entity selector, we minimize the binary cross-entropy loss of predicting whether each entity is affected in the sentence.
Revised: Entity Selection Loss Similarly, to train the attentive entity selector, we minimize the binary cross-entropy loss of predicting whether each entity is affected in the sentence.

Original: State Change Loss For each state change predictor, we minimize the negative loglikelihood of predicting the correct end state for each state change.
Revised: State Change Loss For each state change predictor, we minimize the negative loglikelihood of predicting the correct end state for each state change.

Original: Coverage Loss
Revised: Coverage Loss

Original: An underlying assumption in many narratives is that all entities that are mentioned should be important to the narrative.
Revised: An underlying assumption in many narratives is that all entities that are mentioned should be important to the narrative.

Original: We add a loss term that penalizes narratives whose combined attention weights for each entity does not sum to more than 1.
Revised: We add a loss term that penalizes narratives whose combined attention weights for each entity does not sum to more than 1.

Original: [Equation]
Revised: [Equation]

Original: where a i t is the attention weight for a particular entity at sentence t and I d is the number of entities in a document.
Revised: where a i t is the attention weight for a particular entity at sentence t and I d is the number of entities in a document.

Original: (cid:80)
Revised: (cid:80)

Original: St =1 a i t is upper bounded by 1.
Revised: St =1 a i t is upper bounded by 1.

Original: This is similar to the coverage penalty used in neural machine translation (Tu et al., 2016).
Revised: This is similar to the coverage penalty used in neural machine translation (Tu et al., 2016).

Original:  E XPERIMENTAL S ETUP
Revised:  E XPERIMENTAL S ETUP

Original: We evaluate our model on a set of intrinsic tasks centered around tracking entities and state changes in recipes to show that the model can simulate preliminary dynamics of the recipe task.
Revised: We evaluate our model on a set of intrinsic tasks centered around tracking entities and state changes in recipes to show that the model can simulate preliminary dynamics of the recipe task.

Original: Additionally, we provide a qualitative analysis of the internal components of our model.
Revised: Additionally, we provide a qualitative analysis of the internal components of our model.

Original: Finally, we evaluate the quality of the states encoded by our model on the extrinsic task of generating future steps in a recipe.
Revised: Finally, we evaluate the quality of the states encoded by our model on the extrinsic task of generating future steps in a recipe.

Original: 4. I NTRINSIC E VALUATION - T RACKING
Revised: 4. I NTRINSIC E VALUATION - T RACKING

Original: In the tracking task, we evaluate the model’s ability to identify which entities are selected and what changes have been made to them in every step.
Revised: In the tracking task, we evaluate the model’s ability to identify which entities are selected and what changes have been made to them in every step.

Original: We break the tracking task into two separate evalua-
Revised: We break the tracking task into two separate evalua-

Original: [Table]
Revised: [Table]

Original: tions, entity selection and end state prediction, and also investigate whether the model learns internal representations that approximate recipe dynamics.
Revised: tions, entity selection and end state prediction, and also investigate whether the model learns internal representations that approximate recipe dynamics.

Original: Metrics In the entity selection test, we report the F1 score of choosing the correct entities in any step.
Revised: Metrics In the entity selection test, we report the F1 score of choosing the correct entities in any step.

Original: A selected entity is deﬁned as one whose attention weight a i is greater than 50% ( § 2.4).
Revised: A selected entity is deﬁned as one whose attention weight a i is greater than 50% ( § 2.4).

Original: Because entities may be harder to predict when they have been combined with other entities (e.g., the mixture may have a new name), we also report the recall for selecting combined (CR) and uncombined (UR) entities.
Revised: Because entities may be harder to predict when they have been combined with other entities (e.g., the mixture may have a new name), we also report the recall for selecting combined (CR) and uncombined (UR) entities.

Original: In the end state prediction test, we report how often the model correctly predicts the state change performed in a recipe step and the resultant end state.
Revised: In the end state prediction test, we report how often the model correctly predicts the state change performed in a recipe step and the resultant end state.

Original: This score is then scaled by the accuracy of predicting which entities were changed in that same step.
Revised: This score is then scaled by the accuracy of predicting which entities were changed in that same step.

Original: We report the average F1 and accuracy across the six state change types.
Revised: We report the average F1 and accuracy across the six state change types.

Original: Baselines We compare our models against two baselines.
Revised: Baselines We compare our models against two baselines.

Original: First, we built a GRU model that is trained to predict entities and state changes independently.
Revised: First, we built a GRU model that is trained to predict entities and state changes independently.

Original: This can be viewed as a bare minimum network with no action representations or recurrent entity memory.
Revised: This can be viewed as a bare minimum network with no action representations or recurrent entity memory.

Original: The second baseline is a Recurrent Entity Network (Henaff et al., 2016) with changes to ﬁt our task.
Revised: The second baseline is a Recurrent Entity Network (Henaff et al., 2016) with changes to ﬁt our task.

Original: First, the model can tie memory cells to a subset of the full list of entities so that it only considers entities that are present in a particular recipe.
Revised: First, the model can tie memory cells to a subset of the full list of entities so that it only considers entities that are present in a particular recipe.

Original: Second, the entity distribution for writing to the memory cells is re-used when we query the memory cells.
Revised: Second, the entity distribution for writing to the memory cells is re-used when we query the memory cells.

Original: The normalized weighted average of the entity cells is used as the input to the state predictors.
Revised: The normalized weighted average of the entity cells is used as the input to the state predictors.

Original: The unnormalized attention when writing to each cell is used to predict selected entities.
Revised: The unnormalized attention when writing to each cell is used to predict selected entities.

Original: Both baselines are trained with entity selection and state change losses ( § 3.3).
Revised: Both baselines are trained with entity selection and state change losses ( § 3.3).

Original: Ablations We report results on six ablations.
Revised: Ablations We report results on six ablations.

Original: First, we remove the recurrent attention (Eq. 3).
Revised: First, we remove the recurrent attention (Eq. 3).

Original: The model only predicts entities using the current encoder hidden state.
Revised: The model only predicts entities using the current encoder hidden state.

Original: In the second ablation, the model is trained with no coverage penalty (Eq. 9).
Revised: In the second ablation, the model is trained with no coverage penalty (Eq. 9).

Original: The third ablation prunes the connection from the action selector w p to the entity selector (Eq. 2).
Revised: The third ablation prunes the connection from the action selector w p to the entity selector (Eq. 2).

Original: We also explore not pretraining the action selector.
Revised: We also explore not pretraining the action selector.

Original: Finally, we look at two ablations where we intialize the action embeddings with vectors from a skipgram model.
Revised: Finally, we look at two ablations where we intialize the action embeddings with vectors from a skipgram model.

Original: In the ﬁrst, the model operates normally, and in the second, we do not allow gradients to backpropagate to the action embeddings, updating only the mapping tensor W 4 instead (Eq. 6).
Revised: In the ﬁrst, the model operates normally, and in the second, we do not allow gradients to backpropagate to the action embeddings, updating only the mapping tensor W 4 instead (Eq. 6).

Original: 4. E XTRINSIC E VALUATION - G ENERATION
Revised: 4. E XTRINSIC E VALUATION - G ENERATION

Original: The generation task tests whether our system can produce the next step in a recipe based on the previous steps that have been performed.
Revised: The generation task tests whether our system can produce the next step in a recipe based on the previous steps that have been performed.

Original: The model is provided all of the previous steps as context and must generate the following sentence.
Revised: The model is provided all of the previous steps as context.

Original: Metrics We report the combined BLEU score and ROUGE score of the generated sequence relative to the reference sequence.
Revised: Metrics We report the combined BLEU score and ROUGE score of the generated sequence relative to the reference sequence.

Original: Each candidate sequence has one reference sentence.
Revised: Each candidate sequence has one reference sentence.

Original: Both metrics are computed at the corpus-level.
Revised: Both metrics are computed at the corpus-level.

Original: Also reported are “VF1”, the F1 score for the overlap of the actions performed in the reference sequence and the verbs mentioned in the generated sequence, and “SF1”, the F1 score for the overlap of end states annotated in the reference sequence and predicted by the generated sequences.
Revised: Also reported are “VF1”, the F1 score for the overlap of the actions performed in the reference sequence and the verbs mentioned in the generated sequence, and “SF1”, the F1 score for the overlap of end states annotated in the reference sequence and predicted by the generated sequences.

Original: End states for the generated sequences are extracted using the lexicon from Section 3.1 based on the actions performed in the sentence.
Revised: End states for the generated sequences are extracted using the lexicon from Section 3.1 based on the actions performed in the sentence.

Original: Baselines For the generation task, we use three baselines: a seq2seq model with no attention (Sutskever et al., 2014), an attentive seq2seq model (Bahdanau et al., 2014), and the neural checklist model (Kiddon et al., 2016). Implementation details for baselines can be found in Appendix B.
Revised: Setup To apply our model to the task of recipe step generation, we input the context sentences through the neural process network and record the entity state vectors once the entire context has

Original: [Table]
Revised: [Table]

Original: Setup To apply our model to the task of recipe step generation, we input the context sentences through the neural process network and record the entity state vectors once the entire context has been read ( § 2.5).
Revised: been read ( § 2.5).

Original: These vectors can be viewed as a snapshot of the current state of the entities once the preceding context has been simulated inside the neural process network.
Revised: These vectors can be viewed as a snapshot of the current state of the entities once the preceding context has been simulated inside the neural process network.

Original: We encode these vectors using a bidirectional GRU (Cho et al., 2014) and take the ﬁnal time step hidden state e I .
Revised: We encode these vectors using a bidirectional GRU (Cho et al., 2014) and take the ﬁnal time step hidden state e I .

Original: A different GRU encodes the context words in the same way (yielding h T ) and the ﬁrst hidden state input to the decoder is computed using the projection function:
Revised: A different GRU encodes the context words in the same way (yielding h T ) and the ﬁrst hidden state input to the decoder is computed using the projection function:

Original: [Equation]
Revised: [Equation]

Original: (10) where ◦ is the Hadamard product between the two encoder outputs.
Revised: where ◦ is the Hadamard product between the two encoder outputs.

Original: All models are trained by minimizing the negative loglikelihood of predicting the next word for the full sequence.
Revised: All models are trained by minimizing the negative loglikelihood of predicting the next word for the full sequence.

Original: Implementation details can be found in Appendix A.
Revised: Implementation details can be found in Appendix A.

Revised: Baselines For the generation task, we use three baselines: a seq2seq model with no attention (Sutskever et al., 2014), an attentive seq2seq model (Bahdanau et al., 2014), and a similar variant as our NPN generator, except where the entity states have been computed by the Recurrent Entity

Revised: Network (EntNet) baseline ( § 4.1).

Revised: Implementation details for baselines can be found in Appendix B.

Original:  E XPERIMENTAL R ESULTS 5. I NTRINSIC E VALUATIONS
Revised:  E XPERIMENTAL R ESULTS 5. I NTRINSIC E VALUATIONS

Original: Entity Selection As shown in Table 2, our full model outperforms all baselines at selecting entities, with an F1 score of 55.39%.
Revised: Entity Selection As shown in Table 8, our full model outperforms all baselines at selecting entities, with an F1 score of 55.39%.

Original: The ablation study shows that the recurrent attention, coverage loss, action connections and action selector pretraining improve performance.
Revised: The ablation study shows that the recurrent attention, coverage loss, action connections and action selector pretraining improve performance.

Original: Our success at predicting entities extends to both uncomposed entities, which are still in their raw forms (e.g., melt the butter → butter), and composed entities, in which all of the entities that make up a composition must be selected.
Revised: Our success at predicting entities extends to both uncomposed entities, which are still in their raw forms (e.g., melt the butter → butter), and composed entities, in which all of the entities that make up a composition must be selected.

Original: For example, in a Cooking lasagna recipe, if the ﬁnal step involves baking the prepared lasagna, the model must select all the entities that make up the lasagna (e.g., lasagna sheets, beef, tomato sauce).
Revised: For example, in a Cooking lasagna recipe, if the ﬁnal step involves baking the prepared lasagna, the model must select all the entities that make up the lasagna (e.g., lasagna sheets, beef, tomato sauce).

Original: In Table 3, we provide examples of our model’s ability to handle complex cases suchas compositional entities (Ex. 1, 3), and elided arguments over long time windows (Ex. 2). We also provide examples where the model fails to select the correct entities because it does not identify the
Revised: In Table 3, we provide examples of our model’s ability to handle complex cases such

Original: [Table]
Revised: [Table]

Original: Nearest Neighbor Actions cut slice, split, snap, slash, carve, slit, chop boil cook, microwave, fry, steam, simmer add sprinkle, mix, reduce, splash, stir, dust wash rinse, scrub, refresh, soak, wipe, scale mash spread, puree, squeeze, liquefy, blend place ease, put, lace, arrange, leave rinse wash, refresh, soak, wipe, scrub, clean warm reheat, ignite, heat, light, crisp, preheat steam microwave, crisp, boil, parboil, heat sprinkle top, pat, add, dip, salt, season grease coat, rub, dribble, spray, smear, line
Revised: cut slice, split, snap, slash, carve, slit, chop boil cook, microwave, fry, steam, simmer add sprinkle, mix, reduce, splash, stir, dust wash rinse, scrub, refresh, soak, [Table] , scale mash spread, puree, squeeze, liquefy, blend place ease [Table]

Revised: lace

Original: [Table]
Revised: [Table]

Revised: arrange

Revised: [Table]

Revised: leave

Revised: [Table]

Revised: wash, refresh, soak, wipe, scrub, clean warm reheat, ignite, heat, light, crisp, preheat

Revised: [Table]

Revised: microwave, crisp, boil, parboil, heat sprinkle

Revised: [Table]

Revised: season

Revised: [Table]

Revised: coat,

Revised: [Table]

Revised: smear

Revised: [Table]

Original:  mapping between a reference construct such as “pizza” (Ex. 4) or “dough” (Ex. 5) and the set of entities that composes it, showcasing the difﬁculty of selecting the full set for a composed entity.
Revised: line [Table] as compositional entities (Ex. 1, 3), and elided arguments over long time windows (Ex. 2). We also provide examples where the model fails to select the correct entities because it does not identify the mapping between a reference construct such as “pizza” (Ex. 4) or “dough” (Ex. 5) and the set of entities that composes it, showcasing the difﬁculty of selecting the full set for a composed entity.

Original: State Change Tracking In Table 2, we show that our full model outperforms competitive baselines such as Recurrent Entity Networks (Henaff et al., 2016) and jointly trained GRUs.
Revised: State Change Tracking In Table 8, we show that our full model outperforms competitive baselines such as Recurrent Entity Networks (Henaff et al., 2016) and jointly trained GRUs.

Original: While the ablation without the coverage loss shows higher accuracy, we attribute this to the fact that it predicts a smaller number of total state changes.
Revised: While the ablation without the coverage loss shows higher accuracy, we attribute this to the fact that it predicts a smaller number of total state changes.

Original: Interestingly, initializing action embeddings with skipgram vectors and locking their values shows relatively high performance, indicating the potential gains in using powerful pretrained representations to represent actions.
Revised: Interestingly, initializing action embeddings with skipgram vectors and locking their values shows relatively high performance, indicating the potential gains in using powerful pretrained representations to represent actions.

Original: Action Embeddings
Revised: Action Embeddings

Original: In our model, each action is assigned its own embedding, but many actions induce similar changes in the physical world (e.g.,“cut” and “slice”).
Revised: In our model, each action is assigned its own embedding, but many actions induce similar changes in the physical world (e.g.,“cut” and “slice”).

Original: After training, we compute the pairwise cosine similarity between each pair of action embeddings.
Revised: After training, we compute the pairwise cosine similarity between each pair of action embeddings.

Original: In Table 4, we see that actions that perform similar functions are neighbors in embedding space, indicating the model has captured certain semantic properties of these actions.
Revised: In Table 4, we see that actions that perform similar functions are neighbors in embedding space, indicating the model has captured certain semantic properties of these actions.

Original: Learning action representations through the state changes they induce has allowed the model to cluster actions by their transformation functions.
Revised: Learning action representations through the state changes they induce has allowed the model to cluster actions by their transformation functions.

Original: Entity Compositions When individual entities are combined into new constructs, our model averages their state embeddings (Eq. 5), applies an action embedding to them (Eq. 6), and writes them to memory (Eq. 7).
Revised: Entity Compositions When individual entities are combined into new constructs, our model averages their state embeddings (Eq. 5), applies an action embedding to them (Eq. 6), and writes them to memory (Eq. 7).

Original: The state embeddings of entities that are combined should be overwritten by the same new embedding.
Revised: The state embeddings of entities that are combined should be overwritten by the same new embedding.

Original: In Figure 3, we present the percentage increase in cosine similarity for state embeddings of entities that are combined in a sentence (blue) as opposed to the percentage increase for those that are not (red bars).
Revised: In Figure 3, we present the percentage increase in cosine similarity for state embeddings of entities that are combined in a sentence (blue) as opposed to the percentage increase for those that are not (red bars).

Original: While the soft attention mechanism for entity selection allows similarities to leak between entity embeddings, our system is generally able to model the compositionality patterns that result from entities being combined into new constructs.
Revised: While the soft attention mechanism for entity selection allows similarities to leak between entity embeddings, our system is generally able to model the compositionality patterns that result from entities being combined into new constructs.

Original: 5. E XTRINSIC E VALUATIONS
Revised: 5. E XTRINSIC E VALUATIONS

Original: Recipe Step Generation Our results in Table 5 indicate that sequences generated using the neural process network entity states as additional input yield higher scores than competitive baselines.
Revised: Recipe Step Generation Our results in Table 5 indicate that sequences generated using the neural process network entity states as additional input yield higher scores than competitive baselines.

Original: The entity states allow the model to predict next steps conditioned on a representation of the world being simulated by the neural process network.
Revised: The entity states allow the model to predict next steps conditioned on a representation of the world being simulated by the neural process network.

Original: Additionally, the higher VF1 and SF1 scores indicate that the model is indeed using the extra information to better predict the actions that should follow the context provided.
Revised: Additionally, the higher VF1 and SF1 scores indicate that the model is indeed using the extra information to better predict the actions that should follow the context provided.

Original: Examples from the dev set are provided in Table 6,showing that the NPN generator can use information about ingredient states to reason about the most likely next step. The ﬁrst example is particularly interesting as it contrasts the generation from the NPN-aware model,
Revised: Example generations for each baselines from the dev set are provided in Table 6,

Original: [Table]
Revised: [Table]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Recent studies in machine comprehension have used a neural memory component to store a running representation of processed text (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016).
Revised: Recent studies in machine comprehension have used a neural memory component to store a running representation of processed text (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016).

Original: While these approaches map text to memory vectors using standard neural encoder approaches, our model, in contrast, directly interprets text in terms of the effects actions induce in entities, providing an inductive bias for learning how to represent stored memories.
Revised: While these approaches map text to memory vectors using standard neural encoder approaches, our model, in contrast, directly interprets text in terms of the effects actions induce in entities, providing an inductive bias for learning how to represent stored memories.

Original: More recent work in machine comprehension also sought to couple the memory representation with tracking entity states (Henaff et al., 2016).
Revised: More recent work in machine comprehension also sought to couple the memory representation with tracking entity states (Henaff et al., 2016).

Original: Our work seeks to provide a relatively more structured representation of domain-speciﬁc action knowledge to provide an inductive bias to the reasoning process.
Revised: Our work seeks to provide a relatively more structured representation of domain-speciﬁc action knowledge to provide an inductive bias to the reasoning process.

Original: Neural Programmers (Neelakantan et al., 2015; 2016) have also used functions to simulate reasoning, by building a model to select rows in a database and applying operation on those selected rows.
Revised: Neural Programmers (Neelakantan et al., 2015; 2016) have also used functions to simulate reasoning, by building a model to select rows in a database and applying operation on those selected rows.

Original: While their work explicitly deﬁned the effect of a number of operations for those rows, we provide a framework for learning representations for a more expansive set of actions, allowing the model to learn representations for how actions change the state space.
Revised: While their work explicitly deﬁned the effect of a number of operations for those rows, we provide a framework for learning representations for a more expansive set of actions, allowing the model to learn representations for how actions change the state space.

Original: Works on instructional language studied the task of building discrete graph representations of recipes using probabilistic models (Kiddon et al., 2015; Mori et al., 2014; 2012).
Revised: Works on instructional language studied the task of building discrete graph representations of recipes using probabilistic models (Kiddon et al., 2015; Mori et al., 2014; 2012).

Original: We propose a complementary new model by integrating action and entity relations into the neural network architecture and also address the additional challenge of tracking the state changes of the entities.
Revised: We propose a complementary new model by integrating action and entity relations into the neural network architecture and also address the additional challenge of tracking the state changes of the entities.

Original: Additional work in tracking states with visual or multimodal context has focused on 1) building graph representations for how entities change in goal-oriented domains (Gao et al., 2016; Liu et al., 2016; Si et al., 2011) or 2) tracking visual state changes based on decisions taken by agents in environment simulators such as videos or games (Chiappa et al., 2017;
Revised: Additional work in tracking states with visual or multimodal context has focused on 1) building graph representations for how entities change in goal-oriented domains (Gao et al., 2016; Liu et al., 2016; Si et al., 2011) or 2) tracking visual state changes based on decisions taken by agents in environment simulators such as videos or games (Chiappa et al., 2017;

Original: Wahlstrom et al., 2015;
Revised: Wahlstrom et al., 2015;

Original: Oh et al., 2015).
Revised: Oh et al., 2015).

Original: Our work, in contrast, models state changes in embedding space using only text-based signals to map real-world actions to algebraic transformations.
Revised: Our work, in contrast, models state changes in embedding space using only text-based signals to map real-world actions to algebraic transformations.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We introduced the Neural Process Network for modeling a process of actions and their causal effects on entities by learning action transformations that change entity state representations.
Revised: We introduced the Neural Process Network for modeling a process of actions and their causal effects on entities by learning action transformations that change entity state representations.

Original: The model maintains a recurrent memory structure to track entity states and is trained to predict the state changes that entities undergo.
Revised: The model maintains a recurrent memory structure to track entity states and is trained to predict the state changes that entities undergo.

Original: Empirical results demonstrate that our model can learn the causal effects of action semantics in the cooking domain and track the dynamic state changes of entities, showing advantages over competitive baselines.
Revised: Empirical results demonstrate that our model can learn the causal effects of action semantics in the cooking domain and track the dynamic state changes of entities, showing advantages over competitive baselines.

Original: A CKNOWLEDGMENTS
Revised: A CKNOWLEDGMENTS

Revised: We thank Yonatan Bisk, Peter Clark, Bhavana Dalvi, Niket Tandon, and Yuyin Sun for helpful discussions at various stages of this work.

Original: This research was supported in part by NSF (IIS-1524371, IIS-1714566, NRI-1525251), DARPA under the CwC program through the ARO (W911NF-15-1-0543), Samsung Research, and gifts by Google and Facebook.
Revised: This research was supported in part by NSF (IIS-1524371, IIS-1714566, NRI-1525251), DARPA under the CwC program through the ARO (W911NF-15-10543), Samsung Research, and gifts by Google and Facebook.

Original: A T RAINING D ETAILS OF OUR F ULL M ODEL AND A BLATIONS A. T RACKING M ODELS
Revised: A T RAINING D ETAILS OF OUR F ULL M ODEL AND A BLATIONS A. T RACKING M ODELS

Original: The hidden size of the instruction encoder is 100, the embedding sizes of action functions and entities are 30.
Revised: The hidden size of the instruction encoder is 100, the embedding sizes of action functions and entities are 30.

Original: We use dropout with a rate of 0.3 before any non-recurrent fully connected layers
Revised: We use dropout with a rate of 0.3 before any non-recurrent fully connected layers

Original: Srivastava et al.
Revised: Srivastava et al.

Original: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of .001 and decay by a factor of 0.1 if we see no improvement on validation loss over a single epoch.
Revised: We use the Adam optimizer (Kingma & Ba, 2014) with a learning rate of .001 and decay by a factor of 0.1 if we see no improvement on validation loss over three epochs.

Original: We stop training early if the development loss does not decrease for ﬁve epochs.
Revised: We stop training early if the development loss does not decrease for ﬁve epochs.

Original: The batch size is 64.
Revised: The batch size is 64.

Original: We use two instruction encoders, one for the entity selector, and one for the action selector.
Revised: We use two instruction encoders, one for the entity selector, and one for the action selector.

Original: Word embeddings and entity embeddings are initialized with skipgram embeddings (Mikolov et al., 2013a;b) using a word2vec model trained on the training set.
Revised: Word embeddings and entity embeddings are initialized with skipgram embeddings (Mikolov et al., 2013a;b) using a word2vec model trained on the training set.

Original: We use a vocabulary size of 7358 for words, and 2996 for entities.
Revised: We use a vocabulary size of 7358 for words, and 2996 for entities.

Original: Gradients with respect to the coverage loss (Eq. 9) are only backpropagated in steps where no entity is annotated as being selected.
Revised: Gradients with respect to the coverage loss (Eq. 9) are only backpropagated in steps where no entity is annotated as being selected.

Revised: To account for the false negatives in the training data due to the heuristic generation of the labels, gradients with respect to the entity selection loss are zeroed when no entity label is present.

Original: A. G ENERATION M ODEL
Revised: A. G ENERATION M ODEL

Original: The hidden size of the context encoder is 200.
Revised: The hidden size of the context encoder is 200.

Original: The hidden size of the state vector encoder is 200.
Revised: The hidden size of the state vector encoder is 200.

Original: State vectors have dimensionality 30 (the same as in the neural process network).
Revised: State vectors have dimensionality 30 (the same as in the neural process network).

Original: Dropout of 0. is used during training in the decoder.
Revised: Dropout of 0. is used during training in the decoder.

Original: The context and state representations are projected jointly using an element-wise product followed by a linear projection Kim et al.
Revised: The context and state representations are projected jointly using an element-wise product followed by a linear projection Kim et al.

Original: Both encoders and the decoder are single layer.
Revised: Both encoders and the decoder are single layer.

Original: The learning rate is 0.0003 initially and is halved every 5 epochs.
Revised: The learning rate is 0.0003 initially and is halved every 5 epochs.

Original: The model is trained with the Adam optimizer.
Revised: The model is trained with the Adam optimizer.

Original: B T RAINING D ETAILS OF B ASELINES B. T RACKING B ASELINES Joint 
Revised: B T RAINING D ETAILS OF B ASELINES B. T RACKING B ASELINES Joint Gated

Original: Gated Recurrent Unit The hidden state of the GRU is 100.
Revised: Recurrent Unit The hidden state of the GRU is 100.

Original: We use a dropout with a rate of 0.3 before any non-recurrent fully connected layers.
Revised: We use a dropout with a rate of 0.3 before any non-recurrent fully connected layers.

Original: We use the Adam optimizer with a learning rate of .001 and decay by a factor of 0.1 if we see no improvement on validation loss over a single epoch.
Revised: We use the Adam optimizer with a learning rate of .001 and decay by a factor of 0.1 if we see no improvement on validation loss over a single epoch.

Original: We stop training early if the development loss does not decrease for ﬁve epochs.
Revised: We stop training early if the development loss does not decrease for ﬁve epochs.

Original: The batch size is 64.
Revised: The batch size is 64.

Original: We use encoders, one for the entity selector, and one for the state change predictors.
Revised: We use encoders, one for the entity selector, and one for the state change predictors.

Original: Word embeddings are initialized with skipgram embeddings using a word2vec model trained on the training set.
Revised: Word embeddings are initialized with skipgram embeddings using a word2vec model trained on the training set.

Original: We use a vocabulary size of 7358 for words.
Revised: We use a vocabulary size of 7358 for words.

Original: Recurrent Entity Networks Memory cells are tied to the entities in the document.
Revised: Recurrent Entity Networks Memory cells are tied to the entities in the document.

Original: For a recipe with 12 ingredients, 12 entity cells are initialized.
Revised: For a recipe with 12 ingredients, 12 entity cells are initialized.

Original: All hyperparameters are the same as the in the bAbI task from Henaff et al.
Revised: All hyperparameters are the same as the in the bAbI task from Henaff et al.

Original: The learning rate start at 0.01 and is halved every 25 epochs.
Revised: The learning rate start at 0.01 and is halved every 25 epochs.

Original: Entity cells and word embeddings are 100 dimensional.
Revised: Entity cells and word embeddings are 100 dimensional.

Original: The encoder is a multiplicative mask initialized the same as in Henaff et al.
Revised: The encoder is a multiplicative mask initialized the same as in Henaff et al.

Original: Intermediate supervision from the weak labels is provided to help predict entities.
Revised: Intermediate supervision from the weak labels is provided to help predict entities.

Original: A separate encoder is used for computing the attention over memory cells and the content to write to the memory.
Revised: A separate encoder is used for computing the attention over memory cells and the content to write to the memory.

Original: Dropout of 0.3 is used in the encoders.
Revised: Dropout of 0.3 is used in the encoders.

Original: The batch size is 64.
Revised: The batch size is 64.

Original: We use a vocabulary size of 7358 for words, and 2996 for entities.
Revised: We use a vocabulary size of 7358 for words, and 2996 for entities.

Original: B. G ENERATION B ASELINES
Revised: B. G ENERATION B ASELINES

Original: Seq2seq Thehidden size of the GRU encoder and decoder is 200. A single layer is used for both the encoder and decoder.
Revised: Seq2seq The encoder and decoder are both single-layer GRUs with hidden size 200.

Original: Dropout of 0.3is used during training in the decoder.
Revised: We use dropout with probability 0.3 in the decoder.

Original: We use the Adam optimizer, a starting  learning rate 0.0003, which is halved every 5 epochs.and multiply the learning rate by 0.1 every 5 epochs.
Revised: We train with the Adam optimizer starting with a learning rate 0.0003 that is halved every 5 epochs.

Original: The encoder is bidirectional.
Revised: The encoder is bidirectional.

Original: The model is trained to minimize the negative loglikelihood of predicting the next word.
Revised: The model is trained to minimize the negative loglikelihood of predicting the next word.

Original: Attentive Seq2seq
Revised: Attentive Seq2seq

Original: The encoder is the same as in the seq2seq baseline.
Revised: The encoder is the same as in the seq2seq baseline.

Original: A multiplicative attention between the decoder hidden state and the context vectors is used to compute the attentionover the context at every decoder time step.
Revised: A multiplicative attention between the decoder hidden state and the context vectors is used to compute the attention

Original: The Neural Checklist We use a pretained model from Kiddon et al. The hidden state of the model is 256-dimensional. Parameters are uniformly initialized between -0.35 and 0.35.
Revised: EntNet Generator

Original: A learning rate of 0.1 is used for training and the model is trained with stochastic gradient descent. A batch size of 30 is used for training. The neural checklist is trainedto generate a full recipe, while our task involves just generating the next step ina recipe provided a set of preceding sentences. We input the full context into the neural checklist, allowing it to update its internals and then make it start generating when we reach the end of the context.
Revised: The model is trained in the same way as the NPN generator model in Appendix A.2 except that the state representations used as input are produced from by EntNet baseline described in Section 4.1 and Appendix B.1.

Original: C A NNOTATIONS C. A NNOTATING S TATE C HANGES
Revised: C A NNOTATIONS C. A NNOTATING S TATE C HANGES

Original: We provide workers with a verb, its deﬁnition, an illustrative image of the action, and a set of sentences where the verb is mentioned.
Revised: We provide workers with a verb, its deﬁnition, an illustrative image of the action, and a set of sentences where the verb is mentioned.

Original: Workers are provided a checklist of the six state change types and instructed to identify which of them the verb causes.
Revised: Workers are provided a checklist of the six state change types and instructed to identify which of them the verb causes.

Original: They are free to identify multiple changes.
Revised: They are free to identify multiple changes.

Original: Seven workers annotate each verb and we assign a state change based on majority vote.
Revised: Seven workers annotate each verb and we assign a state change based on majority vote.

Original: Of the set of 384 verbs extracted, only 342 have a state change type identiﬁed with them.
Revised: Of the set of 384 verbs extracted, only 342 have a state change type identiﬁed with them.

Original: Of those,entail multiple state change types.
Revised: Of those,entail multiple state change types.

Original: C. A NNOTATING E ND S TATES
Revised: C. A NNOTATING E ND S TATES

Original: We give workers a verb, a state change type, and an example with the verb and ask them to provide an end state for the ingredient the verb is applied to in the example.
Revised: We give workers a verb, a state change type, and an example with the verb and ask them to provide an end state for the ingredient the verb is applied to in the example.

Original: We then use the answers to manually aggregate a set of end states for each state change type.
Revised: We then use the answers to manually aggregate a set of end states for each state change type.

Original: These end states are used as labels when the model predicting state changes.
Revised: These end states are used as labels when the model predicting state changes.

Original: For example, a LOCATION change might lead to an end state of “pan,” “pot”, or “oven.”
Revised: For example, a LOCATION change might lead to an end state of “pan,” “pot”, or “oven.”

Original: End states for each state change type are provided in Table 7.
Revised: End states for each state change type are provided in Table 7.

Original: [Table]
Revised: [Table]

Original: type A 
Revised: C. A NNOTATING D EVELOPMENT AND T EST S ETS

Original: Annotators are instructed to note any entities that undergo one of the six state changes in each step, as well as to identify new combinations of ingredients that are created.
Revised: Annotators are instructed to note any entities that undergo one of the six state changes in each step, as well as to identify new combinations of ingredients that are created.

Original: For example, the sentence “Cut the tomatoes and add to the onions” would involve a SHAPE change for the tomatoes and a combination created from the “tomatoes” and “onions”.
Revised: For example, the sentence “Cut the tomatoes and add to the onions” would involve a SHAPE change for the tomatoes and a combination created from the “tomatoes” and “onions”.

Original: In a separate task, three workers are asked to identify the actions performed in every sentence of the development and test set recipes.
Revised: In a separate task, three workers are asked to identify the actions performed in every sentence of the development and test set recipes.

Original: If an action receives a majority vote that it is performed, it is included in the annotations.
Revised: If an action receives a majority vote that it is performed, it is included in the annotations.

Revised: D A DDITIONAL R ESULTS

Revised: R EMOVING

Revised: RAINING DATA

Revised: [Table]