Integrating External Event Knowledge for Script Learning (2020)

Abstract: Script learning aims to predict the subsequent event according to the existing event chain. Recent studies focus on event co-occurrence to solve this problem. However, few studies integrate external event knowledge to solve this problem. With our observations, external event knowledge can provide additional knowledge like temporal or causal knowledge for understanding event chain better and predicting the right subsequent event. In this work, we integrate event knowledge from ASER (Activities, States, Events and their Relations) knowledge base to help predict the next event. We propose a new approach consisting of knowledge retrieval stage and knowledge integration stage. In the knowledge retrieval stage, we select relevant external event knowledge from ASER. In the knowledge integration stage, we propose three methods to integrate external knowledge into our model and infer final answers. Experiments on the widely-used Multi- Choice Narrative Cloze (MCNC) task show our approach achieves state-of-the-art performance compared to other methods.

## introduction a script is a sequence of stereotypical events related to a protagonist. a restaurant script about "jack" can consist of "jack walked to a restaurant", "jack read the menu", "jack ordered food" and "jack ate food". script learning aims to predict the subsequent event given the event chain. according to the script above, we can predict the next event could be "jack payed for the food" or "jack left restaurant". script learning can be applied to a wide range of applications, such as storytelling, dialogue generation, discourse understanding and intention recognition, etc. there have been recent lines focusing on script learning. the first line models the event co-occurrence to infer the next event. event pair methods like pmi @xcite and event bigram @xcite aim to model the pair relation between predicted event and events within the chain. event chain methods like lstm @xcite model the event chain to predict the next event. pairlstm @xcite and sam-net @xcite both utilize event pair and event chain modeling to predict the subsequent event. sgnn @xcite constructs an event graph and utilizes the graph information. the second line goes beyond the event-occurrence and utilizes discourse relations or external commonsense knowledge. @xcite utilizes trans-* @xcite to learn the relation between events and @xcite aims to integrate external commonsense knowledge like sentiment and intention into event representations. however, the above studies ignore the effect of external event knowledge, which can provide abundant temporal and causal knowledge for events. with our observations, external event knowledge plays a significant role in understanding and predicting events. as shown in figure 1 , the solid circles in the line are events in the given chain and the dotted event is the event to predict. the triples in "(head event, relation, tail event)" format are the knowledge from external knowledge bases. when understanding the events in the chain, we can see that the triple jack was tired jack felt hungry he had lunch he received an urgent call he left home head event: he is hungry relation: result tail event: he has lunch head event: he does receive a call relation: precedence tail event: he departs away head event: he is tired relation: conjunction tail event: he is hungry "(he is hungry, result, he has lunch)" gives evidence to understand that the event "he had lunch" is the result of "jack felt hungry". when predicting the next event, we can see that the triple "(he do receive a call, precedence, he depart away)" can provide enough evidence to predict the next event will be "he left home". from the example, we can see that external event knowledge would benefit the script learning problem. in this work, we propose to utilize external event knowledge to help resolve the script learning problem. our approach consists of two stages: knowledge retrieval and knowledge integration. in the knowledge retrieval stage, we aim to locate the given individual event within the script in knowledge bases. we utilize elastic search to select relevant events and then we design rule-based methods to re-rank the relevant events to obtain the most relevant event. in the knowledge integration stage, we propose three methods including "tail only", "event template" and "relation embedding" to integrate external knowledge sources into our model to help infer the subsequent event. the contributions of this paper can be summarized as follows: ## related work script @xcite is an event sequence related to a protagonist. it is first defined by human-experts to help question answering systems, which is time-consuming and labor-intensive. in @xcite , the researchers propose to utilize statistical models to learn the cooccurrence between events and predict the subsequent event. many studies follow @xcite and propose many approaches. two major approaches are event pair modeling and event chain modeling. in event pair modeling, researchers adopt event n-gram @xcite or event pair cooccurrence @xcite to predict the subsequent event. @xcite first introduces word embeddings to represent events and achieves good results on the task. after that, embedding-based methods have been a dominant approach in this area. for event chain approaches, @xcite first utilizes long short term memory (lstm) to model the event chain and predict the subsequent event. many approaches not only consider event chain modeling, but also utilize event pair modeling. @xcite utilizes event pair modeling and event order into consideration and achieves better performance. @xcite proposes to use self attention mechanism to discover event segments as event chain modeling and combine event pair modeling to predict event. different from the above studies, @xcite constructs an event evolutionary graph to represent the co-occurrence between events and utilize the information from neighbors to predict events. the aforementioned methods only consider the evet co-occurrence. (lee and goldwasser, 2019) goes beyond the event co-occurrence and utilize discourse relations to model the relations between event. @xcite introduces external sentiment and intention information into events to improve event representations. in this work, we propose to integrate external event knowledge to help understand and predict the subsequent events, which can provide abundant temporal and causal information about events. ## problem definition as shown in figure 2 , script learning aims to predict the subsequent event e n+1 given the event @xmath0, e 2 , • • • , e n >. an event e i consists of four parts: subject e s , verb e v , object e o and prepositional object e p . we denote an event as "e v (e s , e o , e p )". for example, we can extract the event "gave(mary, the book, jack)" from the sentence " mary gave the book to jack". if an event do not have one part, it will be denoted as "none". following @xcite @xcite , we also adopt the multi-choice narrative cloze (mcnc) task to evaluate the effectiveness of different models. this task aims to select the right subsequent event from a set of events {e c 1 , e c 2 , • • • , e cm }, where m is the number of candidate event. accuracy is adopted as the metric. ## methodology our proposed approach consists of two stages: knowledge retrieval and knowledge integration. in the knowledge retrieval stage, we retrieve relevant knowledge from external event knowledge bases. in the knowledge integration stage, we integrate the retrieved knowledge into our model to infer final answers. we will first give a brief introduction to aser knowledge base and show the details in the following sections. elastic search 1.i make an urgent call 2. he does receive a call 3. first call have receive rerank search results lemmatization 1.he does receive a call 2.i make an urgent call 3. first call have receive aser figure 3 : knowledge retrieval process. during the lemmatization process, the word "received" will be converted to "receive".