Title: SIMULATING ACTION DYNAMICS WITH NEURAL PROCESS NETWORKS

Abstract: Understanding procedural language requires anticipating the causal effects of actions, even when they are not explicitly stated. In this work, we introduce Neural Process Networks to understand procedural text through (neural) simulation of action dynamics. Our model complements existing memory architectures with dynamic entity tracking by explicitly modeling actions as state transformers. The model updates the states of the entities by executing learned action operators. Empirical results demonstrate that our proposed model can reason about the unstated causal effects of actions, allowing it to provide more accurate contextual information for understanding and generating procedural text, all while offering more interpretable internal representations than existing alternatives.

1. INTRODUCTION:

Understanding procedural text such as instructions or stories requires anticipating the implicit causal effects of actions on entities. For example, given instructions such as "add blueberries to the muffin mix, then bake for one half hour," an intelligent agent must be able to anticipate a number of entailed facts (e.g., the blueberries are now in the oven; their "temperature" will increase). While this common sense reasoning is trivial for humans, most natural language understanding algorithms do not have the capacity to reason about causal effects not mentioned directly in the surface strings (Levy et al., 2015; Jia & Liang, 2017; Lucy & Gauthier, 2017) . The process is a narrative of entity state changes induced by actions. In each sentence, these state changes are induced by simulated actions and must be remembered.

In this paper, we introduce Neural Process Networks, a procedural language understanding system that tracks common sense attributes through neural simulation of action dynamics. Our network models interpretation of natural language instructions as a process of actions and their cumulative effects on entities. More concretely, reading one sentence at a time, our model attentively selects what actions to execute on which entities, and remembers the state changes induced with a recurrent memory structure. In Figure 1 , for example, our model indexes the "tomato" embedding, selects the "wash" and "cut" functions and performs a computation that changes the "tomato" embedding so that it can reason about attributes such as its "SHAPE" and "CLEANLINESS".

Our model contributes to a recent line of research that aims to model aspects of world state changes, such as language models and machine readers with explicit entity representations (Henaff et al., 2016; Yang et al., 2016; Ji et al., 2017) , as well as other more general purpose memory network variants (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016) . This worldcentric modeling of procedural language (i.e., understanding by simulation) abstracts away from the surface strings, complementing text-centric modeling of language, which focuses on syntactic and semantic labeling of surface words (i.e., understanding by labeling).

Unlike previous approaches, however, our model also learns explicit action representations as functional operators (See Figure 1 ). While representations of action semantics could be acquired through an embodied agent that can see and interact with the world (Oh et al., 2015) , we propose to learn these representations from text. In particular, we require the model to be able to explain the causal effects of actions by predicting natural language attributes about entities such as "LOCATION" and "TEMPERATURE". The model adjusts its representations of actions based on errors it makes in predicting the resultant state changes to attributes. This textual simulation allows us to model aspects of action causality that are not readily available in existing simulation environments. Indeed, most virtual environments offer limited aspects of the world -with a primary focus on spatial relations (Oh et al., 2015; Chiappa et al., 2017; Wahlstrom et al., 2015) . They leave out various other dimensions of the world states that are implied by diverse everyday actions such as "dissolve" (change of "COMPOSITION") and "wash" (change of "CLEANLINESS").

Empirical results demonstrate that parametrizing explicit action embeddings provides an inductive bias that allows the neural process network to learn more informative context representations for understanding and generating natural language procedural text. In addition, our model offers more interpretable internal representations and can reason about the unstated causal effects of actions explained through natural language descriptors. Finally, we include a new dataset with fine-grained annotations on state changes, to be shared publicly, to encourage future research in this direction.

2. NEURAL PROCESS NETWORK:

The neural process network is an interpreter that reads in natural language sentences, one at a time, and simulates the process of actions being applied to relevant entities through learned representations of actions and entities.

2.1. OVERVIEW AND NOTATION:

The main component of the neural process network is the simulation module ( §2.5), a recurrent unit whose internals simulate the effects of actions being applied to entities. A set of V actions is known a priori and an embedding is initialized for each one, F = {f 1 , ...f V }. Similarly, a set of I entities is known and an embedding is initialized for each one: E = {e 1 , ...e I }. Each e i can be considered to encode information about state attributes of that entity, which can be extracted by a set of state predictors ( §2.6). As the model reads text, it "applies" action embeddings to the entity vectors, thereby changing the state information encoded about the entities.

For any document d, an initial list of entities I d is known and

E d = {e i |i ∈ I d } ⊂ E entity state embeddings are initialized.

As the neural process network reads a sentence from the document, it selects a subset of both F ( §2.3) and E d ( §2.4) based on the actions performed and entities affected in the sentence. The entity state embeddings are changed by the action and the new embeddings are used to predict end states for a set of state changes ( §2.6). The prediction error for end states is backpropagated to the action embeddings, learning action representations that model the simulation of desired causal effects on entities. This process is broken down into five modules below. Unless explicitly defined, all W and b variables are parametrized linear projections and biases. We use the notation {e i } t when referring to the values of the entity embeddings before processing sentence s t .

2.2. SENTENCE ENCODER:

Given a sentence s t , a Gated Recurrent Unit (Cho et al., 2014) encodes each word and outputs its last hidden vector as a sentence encoding h t (Sutskever et al., 2014) .

2.3. ACTION SELECTOR:

Given h t from the sentence encoder, the action selector (bottom left in Fig. 2 ) contextually determines which action(s) from F to execute. For example, if the input sentence is "wash and cut beets", both f wash and f cut must be selected. To account for multiple actions, we make a soft selection over F, yielding a weighted sum of the selected action embeddings ft :

w p = MLP(h t ) wp = w p V j=1 w pj ft = w p F (1)

where MLP is a parametrized feed-forward network with a sigmoid activation and w p ∈ R V is the attention distribution over V possible actions ( §3.1). We compose the action embedding by taking the weighted average of the selected actions.

2.4. ENTITY SELECTOR:

Sentence Attention Given h t from the sentence encoder, the entity selector chooses relevant entities using a soft attention mechanism:

EQUATION

where W 2 is a bilinear mapping, e i0 is a unique key for each entity ( §2.5), and d i is the attention weight for entity embedding e i . For example, in "wash and cut beets and carrots", the model should select e beet and e carrot .

Recurrent Attention While sentence attention would suffice if entities were always explicitly mentioned, natural language often elides arguments or uses referent pronouns. As such, the module must be able to consider entities mentioned in previous sentences. Using ht , the model computes a soft choice over whether to choose affected entities from this step's attention d i or the previous step's attention distribution.

c = sof tmax(W 3 ht + b 3 ) a it = c 1 d i + c 2 a it-1 + c 3 0 (3)

where c ∈ R 3 is the choice distribution, a it-1 is the previous sentence's attention weight for each entity, a it is the final attention for each entity, and 0 is a vector of zeroes (providing the option to not change any entity). Prior entity attentions can propagate forward for multiple steps.

2.5. SIMULATION MODULE:

Entity Memory A unique state embedding e i is initialized for every entity i in the document. A unique key to index each embedding e i0 is set as the initial value of the embedding (Henaff et al., 2016; Miller et al., 2016) . After the model reads s t , it modifies {e i } t to reflect changes influenced by actions. At every time step, the entity memory receives the attention weights from the entity selector, normalizes them and computes a weighted average of the relevant entity state embeddings:

EQUATION

Applicator Given the action summary embedding ft and the entity summary embedding ēt , the applicator (middle right in Fig. 2 ) applies the selected actions to the selected entities, and outputs the new proposal entity embedding k t .

k t = ReLU( ft W 4 ēt + b 4 ) (6)

where W 4 is a third order tensor projection. The vector k t is the new representation of the entity ēt after the applicator simulates the action being applied to it.

Entity Updater:

The entity updater interpolates the new proposal entity embedding k t and the set of current entity embeddings {e i } t :

EQUATION

yielding an updated set of entity embeddings {e i } t+1 . Each embedding is updated proportional to its entity's unnormalized attention a i , allowing the model to completely overwrite the state embedding for any entity. For example, in the sentence "mix the flour and water," the embeddings for e f lour and e water must both be overwritten by k t because they no longer exist outside of this new composition.

2.6. STATE PREDICTORS:

Given the new proposal entity embedding k t , the state predictor (bottom right in Fig. 2 ) predicts changes to the resulting entity embedding k t along the following six dimensions: location, cookedness, temperature, composition, shape, and cleanliness. Discrete multi-class classifiers, one for each dimension, take in k t and predict a unique end state for their corresponding state change type:

EQUATION

For location changes, which require contextual information to predict the end state, k t is concatenated with the original sentence representation h t to predict the final state.

3. TRAINING:



3.1. STATE CHANGE KNOWLEDGE:

In this work we focus on physical action verbs in cooking recipes. We manually collect a set of 384 actions such as cut, bake, boil, arrange, and place, organizing their causal effects along the following predefined dimensions: LOCATION, COOKEDNESS, TEMPERATURE, SHAPE, CLEANLI-NESS and COMPOSITION. The textual simulation operated by the model induces state changes along these dimensions by applying actions functions from the above set of 384. For example, cut entails a change in SHAPE, while bake entails a change in TEMPERATURE, COOKEDNESS, and even LO-CATION. We annotate the state changes each action induces, as well as the end state of the action, using Amazon Mechanical Turk. The set of possible end states for a state change can range from 2 for binary state changes to more than 200 (See Appendix C for details). 

3.3. COMPONENT-WISE TRAINING:

The neural process network is trained by jointly optimizing multiple losses for the action selector, entity selector, and state change predictors. Importantly, our training scheme uses weak supervision because dense annotations are prohibitively expensive to acquire at a very large scale. Thus, we heuristically extract verb mentions from each recipe step and assign a state change label based on the state changes induced by that action ( §3.1). Entities are extracted similarly based on string matching between the instructions and the ingredient list. We use the following losses for training: Action Selection Loss Using noisy supervision, the action selector is trained to minimize the cross-entropy loss for each possible action, allowing multiple actions to be chosen at each step if multiple actions are mentioned in a sentence. The MLP in the action selector (Eq. 1) is pretrained.

Entity Selection Loss Similarly, to train the attentive entity selector, we minimize the binary cross-entropy loss of predicting whether each entity is affected in the sentence.

State Change Loss For each state change predictor, we minimize the negative loglikelihood of predicting the correct end state for each state change.

Coverage Loss An underlying assumption in many narratives is that all entities that are mentioned should be important to the narrative. We add a loss term that penalizes narratives whose combined attention weights for each entity does not sum to more than 1.

L cover = - 1 I d I d i=1 log S t=1 a it (9)

where a it is the attention weight for a particular entity at sentence t and I d is the number of entities in a document.

S t=1 a it is upper bounded by 1. This is similar to the coverage penalty used in neural machine translation (Tu et al., 2016) .

4. EXPERIMENTAL SETUP:

We evaluate our model on a set of intrinsic tasks centered around tracking entities and state changes in recipes to show that the model can simulate preliminary dynamics of the recipe task. Additionally, we provide a qualitative analysis of the internal components of our model. Finally, we evaluate the quality of the states encoded by our model on the extrinsic task of generating future steps in a recipe.

4.1. INTRINSIC EVALUATION -TRACKING:

In the tracking task, we evaluate the model's ability to identify which entities are selected and what changes have been made to them in every step. We break the tracking task into two separate evalua- 

Metrics:

In the entity selection test, we report the F1 score of choosing the correct entities in any step. A selected entity is defined as one whose attention weight a i is greater than 50% ( §2.4).

Because entities may be harder to predict when they have been combined with other entities (e.g., the mixture may have a new name), we also report the recall for selecting combined (CR) and uncombined (UR) entities. In the end state prediction test, we report how often the model correctly predicts the state change performed in a recipe step and the resultant end state. This score is then scaled by the accuracy of predicting which entities were changed in that same step. We report the average F1 and accuracy across the six state change types.

Baselines We compare our models against two baselines. First, we built a GRU model that is trained to predict entities and state changes independently. This can be viewed as a bare minimum network with no action representations or recurrent entity memory. The second baseline is a Recurrent Entity Network (Henaff et al., 2016) with changes to fit our task. First, the model can tie memory cells to a subset of the full list of entities so that it only considers entities that are present in a particular recipe. Second, the entity distribution for writing to the memory cells is re-used when we query the memory cells. The normalized weighted average of the entity cells is used as the input to the state predictors. The unnormalized attention when writing to each cell is used to predict selected entities. Both baselines are trained with entity selection and state change losses ( §3.3).

Ablations We report results on six ablations. First, we remove the recurrent attention (Eq. 3). The model only predicts entities using the current encoder hidden state. In the second ablation, the model is trained with no coverage penalty (Eq. 9). The third ablation prunes the connection from the action selector w p to the entity selector (Eq. 2). We also explore not pretraining the action selector. Finally, we look at two ablations where we intialize the action embeddings with vectors from a skipgram model. In the first, the model operates normally, and in the second, we do not allow gradients to backpropagate to the action embeddings, updating only the mapping tensor W 4 instead (Eq. 6).

4.2. EXTRINSIC EVALUATION -GENERATION:

The generation task tests whether our system can produce the next step in a recipe based on the previous steps that have been performed. The model is provided all of the previous steps as context.

We report the combined BLEU score and ROUGE score of the generated sequence relative to the reference sequence. Each candidate sequence has one reference sentence. Both metrics are computed at the corpus-level. Also reported are "VF1", the F1 score for the overlap of the actions performed in the reference sequence and the verbs mentioned in the generated sequence, and "SF1", the F1 score for the overlap of end states annotated in the reference sequence and predicted by the generated sequences. End states for the generated sequences are extracted using the lexicon from Section 3.1 based on the actions performed in the sentence.

Setup To apply our model to the task of recipe step generation, we input the context sentences through the neural process network and record the entity state vectors once the entire context has Good st-1 Add tomato paste, broth, garlic, chili powder, cumin, chile peppers, and water. st Bring to boil, then turn very low, cover and simmer until meat is tender. selected meat, garlic, chili powder, tomato paste, cumin, chiles, beef broth, water correct Same + [oil] Good st-4 Stir in oats, sugar, flour, corn syrup, milk, vanilla extract, and salt. st-3 Mix well. st-2 Drop by measuring teaspoonfuls onto cookie sheets. st-1 Bake 5 -7 minutes.

st Let cool. selected oats, sugar, flour, corn syrup, milk, vanilla extract, salt correct oats, sugar, flour, corn syrup, milk, vanilla extract, salt Good st-1 In a large saucepan over low heat, melt marshmallows.

st Add sprinkles, cereal, and raisins, stir until well coated. selected marshmallows, cereal, raisins correct marshmallows, cereal, raisins, sprinkles Bad st-3 Ladle the barbecue sauce around the crust and spread. st-2 Add mozzarella, yellow cheddar, and monterey jack cheese. st-1 Next, add onion mixture and sliced chicken breast .

st Top pizza with jalapeno peppers. selected jalapenos correct crust, sauce, mozzarella, cheddar, monterey jack, white onion, chicken, jalapenos Bad st-2 Combine 1 cup flour, salt, and 1 tbsp sugar. st-1 Cut in butter until mixture is crumbly, then sprinkle with vinegar .

st Gather dough into a ball and press into bottom of 9 inch springform pan. selected butter, vinegar correct flour, salt, sugar, butter, vinegar Table 3 : Examples of the model selecting entities for sentence s t . The previous sentences are provided as context in cases where they are relevant. been read ( §2.5). These vectors can be viewed as a snapshot of the current state of the entities once the preceding context has been simulated inside the neural process network. We encode these vectors using a bidirectional GRU (Cho et al., 2014) and take the final time step hidden state e I . A different GRU encodes the context words in the same way (yielding h T ) and the first hidden state input to the decoder is computed using the projection function:

EQUATION

)

where • is the Hadamard product between the two encoder outputs. All models are trained by minimizing the negative loglikelihood of predicting the next word for the full sequence. Implementation details can be found in Appendix A.

Baselines For the generation task, we use three baselines: a seq2seq model with no attention (Sutskever et al., 2014) , an attentive seq2seq model (Bahdanau et al., 2014) , and a similar variant as our NPN generator, except where the entity states have been computed by the Recurrent Entity Network (EntNet) baseline ( §4.1). Implementation details for baselines can be found in Appendix B.

5. EXPERIMENTAL RESULTS:



5.1. INTRINSIC EVALUATIONS:

Entity Selection As shown in Table 8 , our full model outperforms all baselines at selecting entities, with an F1 score of 55.39%. The ablation study shows that the recurrent attention, coverage loss, action connections and action selector pretraining improve performance. Our success at predicting entities extends to both uncomposed entities, which are still in their raw forms (e.g., melt the butter → butter), and composed entities, in which all of the entities that make up a composition must be selected. For example, in a Cooking lasagna recipe, if the final step involves baking the prepared lasagna, the model must select all the entities that make up the lasagna (e.g., lasagna sheets, beef, tomato sauce). In as compositional entities (Ex. 1, 3), and elided arguments over long time windows (Ex. 2). We also provide examples where the model fails to select the correct entities because it does not identify the mapping between a reference construct such as "pizza" (Ex. 4) or "dough" (Ex. 5) and the set of entities that composes it, showcasing the difficulty of selecting the full set for a composed entity.

State Change Tracking:

In Table 8 , we show that our full model outperforms competitive baselines such as Recurrent Entity Networks (Henaff et al., 2016) and jointly trained GRUs. While the ablation without the coverage loss shows higher accuracy, we attribute this to the fact that it predicts a smaller number of total state changes. Interestingly, initializing action embeddings with skipgram vectors and locking their values shows relatively high performance, indicating the potential gains in using powerful pretrained representations to represent actions.

Action Embeddings In our model, each action is assigned its own embedding, but many actions induce similar changes in the physical world (e.g.,"cut" and "slice"). After training, we compute the pairwise cosine similarity between each pair of action embeddings. In Table 4 , we see that actions that perform similar functions are neighbors in embedding space, indicating the model has captured certain semantic properties of these actions. Learning action representations through the state changes they induce has allowed the model to cluster actions by their transformation functions. Entity Compositions When individual entities are combined into new constructs, our model averages their state embeddings (Eq. 5), applies an action embedding to them (Eq. 6), and writes them to memory (Eq. 7). The state embeddings of entities that are combined should be overwritten by the same new embedding. In Figure 3 , we present the percentage increase in cosine similarity for state embeddings of entities that are combined in a sentence (blue) as opposed to the percentage increase for those that are not (red bars). While the soft attention mechanism for entity selection allows similarities to leak between entity embeddings, our system is generally able to model the compositionality patterns that result from entities being combined into new constructs.

5.2. EXTRINSIC EVALUATIONS:



Recipe:

Step Generation Our results in showing that the NPN generator can use information about ingredient states to reason about the most likely next step. The first and second examples are interesting as it shows that the NPN-aware model has learned to condition on entity state -knowing that raw butter will likely be melted or that a cooked flan must be refrigerated. The third example is also interesting because the model learns that cooked vegetables such as squash will sometimes be drained, even if it is not relevant to this recipe because the squash is steamed. The seq2seq and EntNet baselines, meanwhile, output reasonable sentences given the immediate context, but do not exhibit understanding of global patterns.

6. RELATED WORK:

Recent studies in machine comprehension have used a neural memory component to store a running representation of processed text (Weston et al., 2014; Sukhbaatar et al., 2015; Hill et al., 2015; Seo et al., 2016) . While these approaches map text to memory vectors using standard neural encoder approaches, our model, in contrast, directly interprets text in terms of the effects actions induce in entities, providing an inductive bias for learning how to represent stored memories. More recent work in machine comprehension also sought to couple the memory representation with tracking entity states (Henaff et al., 2016) . Our work seeks to provide a relatively more structured representation of domain-specific action knowledge to provide an inductive bias to the reasoning process.

Neural Programmers (Neelakantan et al., 2015; 2016) have also used functions to simulate reasoning, by building a model to select rows in a database and applying operation on those selected rows.

While their work explicitly defined the effect of a number of operations for those rows, we provide a framework for learning representations for a more expansive set of actions, allowing the model to learn representations for how actions change the state space.

Works on instructional language studied the task of building discrete graph representations of recipes using probabilistic models (Kiddon et al., 2015; Mori et al., 2014; 2012) . We propose a complementary new model by integrating action and entity relations into the neural network architecture and also address the additional challenge of tracking the state changes of the entities.

Additional work in tracking states with visual or multimodal context has focused on 1) building graph representations for how entities change in goal-oriented domains (Gao et al., 2016; Liu et al., 2016; Si et al., 2011) or 2) tracking visual state changes based on decisions taken by agents in environment simulators such as videos or games (Chiappa et al., 2017; Wahlstrom et al., 2015; Oh et al., 2015) . Our work, in contrast, models state changes in embedding space using only text-based signals to map real-world actions to algebraic transformations.

7. CONCLUSION:

We introduced the Neural Process Network for modeling a process of actions and their causal effects on entities by learning action transformations that change entity state representations. 

:



Figures and Tables:

Figure 1: The process is a narrative of entity state changes induced by actions. In each sentence, these state changes are induced by simulated actions and must be remembered.

Figure 2: Model Summary. The sentence encoder converts a sentence to a vector representation, ht. The action selector and entity selector use the vector representation to choose the actions that are applied and the entities that are acted upon in the sentence. The simulation module indexes the action and entity state embeddings, and applies the transformation to the entities. The state predictors predict the new state of the entities if a state change has occurred. Equation references are provided in parentheses.

Table 1 provides examples of annotations in this action lexicon. Example actions, the state changes they induce, and the possible end states 3.2 DATASET For learning and evaluation, we use a subset of the Now You're Cooking dataset(Kiddon et al., 2016). We chose 65816 recipes for training, 175 recipes for development, and 700 recipes for testing. For the development and test sets, crowdsourced workers densely annotate actions, entities and state changes that occur in each sentence so that we can tune hyperparameters and evaluate on gold evaluation sets. Annotation details are provided in Appendix C.3.

Table content: <table><tr><td>Action</td><td>State Change Types</td><td>End States</td></tr><tr><td>braise</td><td>COOKEDNESS; TEMPERATURE</td><td>COOKED; HOT</td></tr><tr><td>chill</td><td>TEMPERATURE</td><td>COLD</td></tr><tr><td>knead</td><td>SHAPE</td><td>MOLDED</td></tr><tr><td>wash</td><td>CLEANLINESS</td><td>CLEAN</td></tr><tr><td>dissolve</td><td>COMPOSITION</td><td>COMPOSED</td></tr><tr><td colspan="2">refrigerate TEMPERATURE; LOCATION</td><td>COLD; REFRIGERATOR</td></tr><tr><td>slice</td><td>SHAPE</td><td>SEPARATED</td></tr></table>

Results for entity selection and state change selection tions, entity selection and end state prediction, and also investigate whether the model learns internal representations that approximate recipe dynamics.

Table content: <table><tr><td>Model</td><td colspan="2">Entity Selection F1 UR CR</td><td colspan="2">State Change F1 ACC</td></tr><tr><td>2-layer LSTM Entity Recognizer</td><td colspan="2">50.98 74.03 13.33</td><td>-</td><td>-</td></tr><tr><td>Adapted Gated Recurrent Unit</td><td>45.94 67.69</td><td colspan="3">7.74 41.16 52.69</td></tr><tr><td colspan="2">Adapted Recurrent Entity Network 48.57 71.88</td><td colspan="3">9.87 42.32 53.47</td></tr><tr><td>-Recurrent Attention (Eq. 3)</td><td colspan="4">48.91 72.32 12.67 42.14 50.48</td></tr><tr><td>-Coverage Loss (Eq. 9)</td><td colspan="4">55.18 73.98 20.23 44.44 55.20</td></tr><tr><td>-Action Connections (Eq. 2)</td><td colspan="4">54.85 73.54 20.03 44.05 54.81</td></tr><tr><td>-Action Selector Pretraining</td><td colspan="4">54.91 73.87 20.30 44.28 55.00</td></tr><tr><td>+ Pretrained Action Embeddings</td><td colspan="4">55.16 74.02 20.32 44.02 55.03</td></tr><tr><td>-Action Embedding Updates</td><td colspan="4">53.79 70.77 18.60 44.27 55.02</td></tr><tr><td>Full Model</td><td colspan="4">55.39 74.88 20.45 44.65 55.07</td></tr></table>

Table 3, we provide examples of our model's ability to handle complex cases such

Table content: <table><tr><td>Action</td><td>Nearest Neighbor Actions</td></tr><tr><td>cut</td><td>slice, split, snap, slash, carve, slit, chop</td></tr><tr><td>boil</td><td>cook, microwave, fry, steam, simmer</td></tr><tr><td>add</td><td>sprinkle, mix, reduce, splash, stir, dust</td></tr><tr><td>wash</td><td>rinse, scrub, refresh, soak, wipe, scale</td></tr><tr><td>mash</td><td>spread, puree, squeeze, liquefy, blend</td></tr><tr><td>place</td><td>ease, put, lace, arrange, leave</td></tr><tr><td>rinse</td><td>wash, refresh, soak, wipe, scrub, clean</td></tr><tr><td>warm</td><td>reheat, ignite, heat, light, crisp, preheat</td></tr><tr><td>steam</td><td>microwave, crisp, boil, parboil, heat</td></tr><tr><td>sprinkle</td><td>top, pat, add, dip, salt, season</td></tr><tr><td>grease</td><td>coat, rub, dribble, spray, smear, line</td></tr></table>

Most similar actions based on cosine similarity of action embeddings

Table content: <table><tr><td/><td/><td colspan="3">Figure 3: Change in cosine similarity of entity</td></tr><tr><td/><td/><td colspan="2">state embeddings</td></tr><tr><td>Model</td><td colspan="2">BLEU ROUGE-L</td><td>VF1</td><td>SF1</td></tr><tr><td>Vanilla Seq2Seq</td><td>2.81</td><td colspan="3">33.00 16.17 40.21</td></tr><tr><td>Attentive Seq2Seq</td><td>2.83</td><td colspan="3">33.18 16.97 41.43</td></tr><tr><td>EntNet Generator</td><td>2.30</td><td colspan="3">32.71 17.53 42.43</td></tr><tr><td>NPN Generator</td><td>3.74</td><td colspan="3">35.64 20.12 43.40</td></tr></table>

Generation Results

Table content: <table/>

Table5indicate that sequences generated using the neural process network entity states as additional input yield higher scores than competitive baselines. The entity states allow the model to predict next steps conditioned on a representation of the world being simulated by the neural process network. Additionally, the higher VF1 and SF1 scores indicate that the model is indeed using the extra information to better predict the actions that should follow the context provided. Example generations for each baselines from the dev set are provided in Table6, Context Preheat oven to 425 degrees. Reference Melt butter in saucepan and mix in bourbon, thyme, pepper, and salt.NPN Melt butter in skillet. Seq2seq Lightly grease 4 x 8 baking pan with sunflower oil. Attentive Seq2seq Combine all ingredients and mix well.EntNet In a large bowl, combine flour, baking powder, baking soda, salt, and pepper.Context Pour egg mixture over caramelized sugar in cake pan. Place cake pan in large shallow baking dish. Bake for 55 minutes or until knife inserted into flan comes out clean. Reference Cover and chill at least 8 hours.NPN Refrigerate until ready to use. Seq2seq Serve at room temperature. Attentive Seq2seq Store in an airtight container.EntNet Store in an airtight container.Context Cut squash into large pieces and steam. Remove cooked squash from shells; Reference Measure 4 cups pulp and reserve remainder for another dish.

Table content: <table><tr><td>NPN Drain.</td></tr><tr><td>Seq2seq Mash pulp with a fork.</td></tr><tr><td>Attentive Seq2seq Set aside.</td></tr><tr><td>EntNet Set aside.</td></tr></table>

Examples of the model generating sentences compared to baselines. The context and reference are provided first, followed by our model's generation and then the baseline generations

Table content: <table/>

The model maintains a recurrent memory structure to track entity states and is trained to predict the state changes that entities undergo. Empirical results demonstrate that our model can learn the causal effects of action semantics in the cooking domain and track the dynamic state changes of entities, showing advantages over competitive baselines. Results for entity selection and state change selection on the development set when randomly dropping a percentage of the training labels

Table content: <table><tr><td>Model</td><td>Entity Selection F1 UR CR</td><td>State Change F1 ACC</td></tr><tr><td colspan="2">25% training data kept 54.34 75.71 21.17</td><td>2.52 50.23</td></tr><tr><td colspan="3">50% training data kept 55.12 76.04 19.05 36.34 54.66</td></tr><tr><td colspan="3">75% training data kept 56.64 76.03 21.00 48.86 57.62</td></tr><tr><td>100% training data</td><td colspan="2">56.84 74.98 21.14 50.56 57.87</td></tr></table>

Peer review (iclr2018):
 Summary This paper presents Neural Process Networks, an architecture for capturing procedural knowledge stated in texts that makes use of a differentiable memory, a sentence and word attention mechanism, as well as learning action representations and their effect on entity representations. The architecture is tested for tracking entities in recipes, as well as generating the natural language description for the next step in a recipe. It is compared against a suit of baselines, such as GRUs, Recurrent Entity Networks, Seq2Seq and the Neural Checklist Model. While I liked the overall paper, I am worried about the generality of the model, the qualitative analysis, as well as a fair comparison to Recurrent Entity Networks and non-neural baselines. Strengths I believe the authors made a good effort in comparing against existing neural baselines (Recurrent Entity Networks, Neural Checklist Model) *for their task*. That said, it is unclear to me how generally applicable the method is and whether the comparison against Recurrent Entity Networks is fair (see Weaknesses). I like the ablation study. Weaknesses While I find the Neural Process Networks architecture interesting and I acknowledge that it outperforms Recurrent Entity Networks for the presented tasks, after reading the paper it is not clear to me how generally applicable the architecture is. Some design choices seem rather tailored to the task at hand (manual collection of actions MTurk annotation in section 3.1) and I am wondering where else the authors see their method being applied given that the architecture relies on all entities and actions being known in advance. My understanding is that the architecture could be applied to bAbI and CBT (the two tasks used in the Recurrent Entity Networks paper). If that is the case, a fair comparison to Recurrent Entity Networks would have been to test against Recurrent Entity Networks on these tasks too. If they the architecture cannot be applied in these tasks, the authors should explain why. I am not convinced by the qualitative analysis. Table 2 tells me that even for the best model the entity selection performance is rather unreliable (only 55.39% F1), yet all examples shown in Table 3 look really good, missing only the two entities oil (1) and sprinkles (3). This suggests that these examples were cherry-picked and I would like to see examples that are sampled randomly from the dev set. I have a similar concern regarding the generation task. First, it is not mentioned where the examples in Table 6 are taken from – is it the train, dev or test set? Second, the overall BLEU score seems quite low even for the best model, yet the examples in Table 6 look really good. In my opinion, a good qualitative analysis should also discuss failure cases. Since the BLEU score is so low here, you might also want to compare perplexity of the models. The qualitative analysis in Table 5 is not convincing either. In Appendix A.1 it is mentioned that word embeddings are initialized from word2vec trained on the training set. My suspicion is that one would get the clustering in Table 4 already from those pretrained vectors, maybe even when pretrained on the Google news corpus. Hence, it is not clear what propagating gradients through the Neural Process Networks into the action embeddings adds, or put differently, why does it have to be a differentiable architecture when an NLP pipeline might be enough? This could easily be tested by another ablation where action embeddings are pretrained using word2vec and then fixed during training of the Neural Process Network. Moreover, in 3.3 it is mentioned that even the Action Selection is pretrained, which makes me wonder what is actually trained jointly in the architecture and what is not. I think the difficulty of the task at hand needs to be discussed at some point, ideally early in the paper. Until examples on page 7 are shown, I did not have a sense for why a neural architecture is chosen. For example, in 2.3 it is mentioned that for "wash and cut" the two functions fwash and fcut need to be selected. For this example, this seems trivial as the functions have the same name (and you could even have a function per name!). As far as I understand, the point of the action selector is to only have a fixed number of learned actions and multiple words (cut, slice etc.) should select the same action fcut. Otherwise (if there is little language ambiguity) I would not see the need for a complex neural architecture. Related to that, a non-neural baseline for the entity selection task that in my opinion definitely needs to be added is extracting entities using a pretrained NER system and returning all of them as the selection. p2 Footnote 1: So if I understand this correctly, this work builds upon a dataset of over 65k recipes from Kiddon et al. (2016), but only for 875 of those detailed annotations were created? Minor Comments p1: The statement "most natural language understanding algorithms do not have the capacity …" should be backed by reference. p2: "context representation ht" – I would directly mention that this is a sentence encoding. p3: 2.4: I have the impression what you are describing here is known in the literature as entity linking. p3 Eq.3: Isn*t c3*0 always a vector of zeros? p4 Eq.6: W4 is an order-3 tensor, correct? p4 Eq.8: What is YC and WC here and what are their dimensions? I am confused by the softmax, as my understanding (from reading the paragraph on the Action Selection Loss on p.5) was that the expression in the softmax here is a scalar (as it is done for every possible action), so this should be a sigmoid to allow for multiple actions to attain a probability of 1? p5: "See Appendix for details" -> "see Appendix C for details" p5 3.3: Could you elaborate on the heuristic for extracting verb mentions? Is only one verb mention per sentence extracted? p5: "trained to minimize cross-entropy loss" -> "trained to minimize the cross-entropy loss" p5 3.3: What is the global loss? p6: "been read (§2.5." -> "been read (§2.5)." p6: "We encode these vectors using a bidirectional GRU" – I think you composing a fixed-dimensional vector from the entity vectors? What*s eI? p7: For which statement is (Kim et al. 2016) the reference? Surely, they did not invent the Hadamard product. p8: "Our model, in contrast" use" -> "Our model, in contrast, uses". p8 Related Work: I think it is important to mention that existing architectures such as Memory Netwroks could, in principle, learn to track entities and devote part of their parameters to learn the effect of actions. What Neural Process Networks are providing is a strong inductive bias for tracking entities and learning the effect of actions that is useful for the task considered in this paper. As mentioned in the weaknesses, this might however come at the price of a less general model, which should be discussed. # Update after the rebuttal Thanks for the clarifications and updating the paper. I am increasing my score by two points and expect to see the ablations as well as the NER baseline mentioned in the rebuttal in the next revision of the paper. Furthermore, I encourage the authors to include the analysis of pretrained word2vec embeddings vs the embeddings learned by this architecture into the paper.