S ubevent W riter: Iterative Sub-event Sequence Generation with Coherence Controller (2022)

Abstract: In this paper, we propose a new task of sub-event generation for an unseen process to evaluate the understanding of the coherence of sub-event actions and objects. To solve the problem, we design SubeventWriter, a sub-event sequence generation framework with a coherence controller. Given an unseen process, the framework can iteratively construct the sub-event sequence by generating one sub-event at each iteration. We also design a very effective coherence controller to decode more coherent sub-events. As our extensive experiments and analysis indicate, SubeventWriter can generate more reliable and meaningful sub-event sequences for unseen processes.

## introduction natural language understanding involves deep understanding of events. in the nlp community, there have been many event understanding tasks. most of them focus on parsing events into involved entities, time, and locations as semantic roles @xcite @xcite @xcite @xcite , or identifying their binary relations such as temporal or causal relations @xcite @xcite @xcite . however, our natural language can be used to describe relations more than binary ones. for example, processes @xcite , also known as scripts @xcite or activities @xcite , are complex events constituted by a sequence of sub-events. understanding processes can be more challenging than individual or pair of events. as shown in figure 1 , to complete the process of making a chocolate cake, we need to consider a sequence of actions, "mix," "add," "pour," and "bake," 1 code is available at https://github.com/ hkust-knowcomp/subeventwriter . process: make a chocolate cake 3. pour batter into cake pans. ## textual sub-event sequence generation we formally define the sub-event sequence generation task as follows. given a process s, we ask the model to generate sub-event sequences e, which are steps to solve the process. this task is essentially a conditional language modeling problem. specifically, given a process s consisting of n tokens: x 1 , x 2 , . . . , x n and a sequence e consists of m sub-events e 1 , e 2 , . . . , e m (each sub-event refers to a sentence containing t i tokens: y i,1 , y i,2 , . . . , y i,t i ), models aim to learn the conditional probability distribution by maximizing the following conditional probabilities in eq. ( 1 ): to step i+1 take candidate y with the largest sum step i seq2seq lm … p' coherence input: how to make a chocolate cake? step 1: mix the dry ingredients … step i: candidate x 40 cm x 19.05cm sub-events. the coherence controller is a discriminative model that can assign a coherence score to a sub-event sequence. it is fine-tuned independently on our synthetic data generated according to our manually designed coherence rules. finally, the framework appends the generated sub-event to the end of the input to serve as new context and start the next iteration. the detailed description of subeventwriter components is as follows: ## experiments we conduct extensive experiments and compare subeventwriter with a wide selection of baselines. ## main evaluation we show the results on the testing set of the wik-ihow dataset in besides, our framework improves more significantly on smaller-sized language models and is parameter efficient. with t5 going down from "3b" to "base", we observe improvements increase (e.g., from 6.76% to 10.41% for bleu-1). also, subeventwriter (t5-base) achieves comparable performance compared to all-at-once seq2seq (t5-3b) with only about 12% parameters foot_2 because generating one event each time is not hard to t5-base with the help of the coherence controller. another interesting observation is that when comparing between subeventwriter based on t5 and bart, t5 always performs slightly better than bart in both "base" and "large" sizes. such advances are consistent with intuition since t5 is about 1.5x -2x larger than bart. in the rest of this section, we conduct more analysis to demonstrate the reason behind the success of subeventwriter. ## related work understanding events has been a challenging task in nlp for a long time @xcite , to which the community has dedicated many works. @xcite first introduced the narrative cloze task, where models are asked to predict the next event from given ones. after them, a few works are devoted to better modeling the event representations @xcite @xcite @xcite . @xcite studied the story cloze test, where a system needs to choose the correct ending for a short story. nonetheless, those tasks emphasize the semantic similarity and relatedness among events, ignoring how events are organized coherently. a similar work to ours is process induction @xcite , where they proposed a statistical framework to generate a sub-event sequence of a given process. the framework aggregates existing events with conceptualization and instantiation. the difference between our work and theirs is that we consider the coherence of both actions and their objects in generation. tasks about processes in different forms are also studied, including sub-event sequence typing @xcite , sub-event selection @xcite , chronological ordering @xcite , script construction with specified length @xcite , and multi-relation prediction @xcite . compared to their settings, our work directly tackles the most challenging one, where models are asked to generate whole sub-event sequences. ## conclusion in this paper, we try to construct coherent subevent sequences by considering coherence in eventlevel decoding. our subeventwriter generates subevents iteratively. a coherence controller is introduced to re-rank candidates in each iteration. the extensive experiments demonstrate the effectiveness of subeventwriter.