Understanding Procedural Text using Interactive Entity Networks (2020)

Abstract: The task of procedural text comprehension aims to understand the dynamic nature of entities/objects in a process. Here, the key is to track how the entities interact with each other and how their states are changing along the procedure. Recent efforts have made great progress to track multiple entities in a procedural text, but usually treat each entity separately and ignore the fact that there are often multiple entities interacting with each other during one process, some of which are even explicitly mentioned. In this paper, we propose a novel Interactive Entity Network (IEN), which is a recurrent network with memory equipped cells for state tracking. In each IEN cell, we maintain different attention matrices through specific memories to model different types of entity interactions. Importantly, we can update these memories in a sequential manner so as to explore the causal relationship between entity actions and subsequent state changes. We evaluate our model on a benchmark dataset, and the results show that IEN outperforms state-of-the-art models by precisely capturing the interactions of multiple entities and explicitly leverage the relationship between entity interactions and subsequent state changes.

## introduction procedural texts, e.g., scientific articles, instruction books, or recipes, are widely spread and useful. entity state tracking is the key task for procedural text comprehension. usually, an entity could have several targets (e.g., existence, location) to be tracked, and the system needs to predict the target changes of each entity involved in the process. state tracking is challenging because of the dynamic nature, the involvement of multiple entities, and the complexity of tracking targets. most recent approaches often use an rnn-based method to model the state changes across the process in an entity-by-entity manner, and use different classifiers for different targets @xcite @xcite . while these models can learn to leverage either local or global information and make state predictions to one entity with fair accuracy, they ignore the fact that there are often multiple entities interacting with each other during the procedure. here is an example that describes a chemical transformation step in the photosynthesis process: the water breaks into oxygen, hydrogen, and electrons. to perfectly capture this chemical conversion in the process, "entity-by-entity" systems need to separately find out that water is destroyed at this step, oxygen is created, hydrogen is created, and electrons are created at the same step, respectively. it is easy to see that if the system ignores the interactions among multiple entities, it will be prone to misunderstanding the role of different entities in the conversion, and consequently make similar state prediction for water as the other entities, although they are opposite in fact. another challenge is how to properly capture the relationship between entity interactions and their state changes. for example, consider the following steps in blood circulation: blood travels to the lungs. carbon dioxide is removed from the blood. oxygen is added to your blood. in order to figure out where the carbon dioxide is after the second sentence, humans may first want to see what role the carbon dioxide acts in that sentence, and quickly find that carbon dioxide is just taken away from the blood thus impossible to be in the blood. furthermore if we look back to the first sentence, we can finally get the right answer, lung. this tells us that an entity's location is closely related or even determined by the most recent event it involves in. however, current systems either model a general state of an entity regardless of specific tracking targets @xcite , or model different tracking targets (e.g., action, location) completely separately @xcite , without considering the relationship between different tracking targets. in this work, we focus on the scientific process understanding task @xcite , in which the tracking targets are action and location of entities. we propose a novel interactive entity network, ien, that explicitly models the interactions among multiple entities and explores the relationship between an entity's action and its location. ien is a two-layer rnn model, the bottom rnn encodes word-level information, and the upper rnn encodes the sentence-level information while keeping tracking entities' states. specifically, we design a new rnn cell, the ien cell, for the upper rnn, which contains two types of memory slots, entity slots, and location slots, for recording the states of all entities and location candidates, respectively. to track the state changes, all memory slots will be updated at each time step. we use two attention modules to model entity-entity interaction and location-entity interaction, respectively, thus each entity/location slot is able to obtain information from other entities before updating. to model the causal relationship between different tracking targets, we arrange the entity updating module to precede the location updating module in a sequential manner. in this way, we can use the action-aware entity information to update the location slots. we evaluate ien on propara, and the results show that our model can effectively model the interaction among multiple entities and leverage the causal relationship between entity actions and locations, thus outperforms existing systems. our contributions are in two-fold: 1) we propose a new model, ien, and design a new recurrent unit that explicitly models the interactions among multiple entities and leverages the causal relationship between entity actions and their subsequent state changes. 2) we conduct intensive experiments to show how our ien learns to encourage the synergy among different entities involved in one event, and explain how multiple tracking targets can be properly leveraged to improve context reasoning. ## related work recently, many procedural text comprehension datasets are constructed and relesed to prompt the research in this direction. babi @xcite is a qa dataset that the questions are about movements of entities, however it is synthetically generated and the language expression is relatively simple. recipes @xcite dataset introduces the task of predicting the locations of cooking ingredients. propara @xcite includes scientific procedural paragraphs, and the task is to predict the entities' actions and locations. in this paper, we continue this line of exploration using propara. the solutions are mainly rnn based or memory network based. most early models are designed for qa task, e.g., babi, and thus researchers pay more attention to question processing. entnet @xcite uses dynamic memories to maintain entity states, with a gated update at each step. these states are decoded to answer questions after each sentence is read. @xcite is an rnn-based model. given a question, qrn recurrently reduces the original query to a more informed query as it observes each context sentence through time. more recently, propara becomes the popular testbed and methods on propara focus more on state tracking. prolocal @xcite locally predicts the state changes described in each individual sentence. proglobal @xcite considers the entire paragraph while predicting states for an entity. as the whole context is incorporated, more state changes are captured, and result in a higher recall. however, this may lead to over-prediction. to address these problems, several models are proposed to incorporate different constraints. prostruct @xcite reformulate the procedural text comprehension task as a structured prediction task, and incorporates a set of commonsense constraints for globally consistent predictions. lace @xcite leverages label consistency among different paragraphs on the same topic during training. ncet @xcite uses a neural crf to explicitly capture the constrains. other interesting attempts include kg-mrc @xcite , which constructs dynamic bipartite graphs from the procedural text, and updates the graphs recurrently at each step. ## task formulation in this paper, we focus on the task of tracking multiple entities' states in scientific procedural text. formally, given: the state tracking task is to predict the states of each entity e i after reading each sentence s t , where an entity's state is a value of a property p j . for example, after reading co2 enters leaf, the system should predict that the existence of co2 is true, and the location of co2 is leaf. we use the propara dataset @xcite ) for experiments. propara contains 488 procedural texts, 391 for training, 43 for development and 54 for testing. each paragraph describes a particular scientific process. there are in total 3302 sentences, and 6.77 sentences per paragraph. on average, each paragraph contains 3.92 entities. the dataset tracks two types of entity property, action and location. for action, the model needs to determine which of the following actions the entity performed in the sentence: (1) create, the entity is created at this step. (2) destroy, the entity is destroyed at this step. (3) move, the entity is moved from one location to another. (4) none, none of above. for location, the model needs to determine where the entity locates before and after one sentence. the location could be any arbitrary span in the paragraph and needs to be extracted. there are two special locations: "-" denotes non-existence and "?" denotes unknown location. figure 1 gives an example of propara dataset. ## methods our complete approach consists of 3 stages: 1) preprocessing: we use off-the-shelf nlp tools to prepare the neural network inputs, including entity recognition and location candidates generation. 2) main model: we use interactive entity network to predict all entities' actions and locations sequentially. 3) postprocessing: we use human-written rules to revise some of the predictions to keep commonsense consistency. ## experiments we evaluate our model on the two comprehension tasks of propara dataset using the official evaluator 2 , and compare against competitive systems on the official leaderboard 3 . ## conclusion in this paper, we propose the interactive entity network, ien, for the multi-entity state tracking task, which learns to interpret complex processes by explicitly modeling the synergy among different entities involved in one event and leveraging the causal relationship between entity actions and their subsequent state changes. experiments on propara show that ien can better understand scientific procedural texts and outperforms state-of-the-art models.