What Are You Trying to Do? Semantic Typing of Event Processes (2020)

Abstract: This paper studies a new cognitively motivated semantic typing task,multi-axis event process typing, that, given anevent process, attempts to infer free-form typelabels describing (i) the type of action made bythe process and (ii) the type of object the pro-cess seeks to affect. This task is inspired bycomputational and cognitive studies of eventunderstanding, which suggest that understand-ing processes of events is often directed by rec-ognizing the goals, plans or intentions of theprotagonist(s). We develop a large dataset con-taining over 60k event processes, featuring ul-tra fine-grained typing on both the action andobject type axes with very large (10ˆ3∼10ˆ4)label vocabularies. We then propose a hybridlearning framework,P2GT, which addressesthe challenging typing problem with indirectsupervision from glosses1and a joint learning-to-rank framework. As our experiments indi-cate,P2GTsupports identifying the intent ofprocesses, as well as the fine semantic type ofthe affected object. It also demonstrates the ca-pability of handling few-shot cases, and stronggeneralizability on out-of-domain processes.

## introduction events are the fundamental building blocks of natural languages. to help machines understand events, extensive research effort has been devoted to inducing how events described in text are procedurally connected @xcite , and how they form event processes 3 (pichotta and mooney, 2014; @xcite . consequently, such prototypical schematic sequences of events have found important use cases including storyline construction @xcite , narrative cloze @xcite , biological process comprehension @xcite and diagnostic prediction @xcite . nonetheless, understanding an event process is not just about inducing temporal relations between events or inferring missing steps in an event sequence. as suggested by cognitive studies @xcite @xcite , a process of events is defined more by the goals, plans, intentions, or traits of its performer, rather than by physical characteristics. for example, a series of events digging a hole, putting in some seeds, filling with soil and watering the soil, occurs in a specific sequence since these steps are directed towards the central goal of planting a plant by the performer. similarly, we can tell that making a dough, adding toppings, preheating the oven and baking the dough is likely a chain of actions aimed at cooking pizza. indeed, aforementioned studies show that humans understand a plausible event process by hypothesizing the objectives those co-occurring events aim for, or the ultimate consequence the process seeks to accomplish. accordingly, we suggest that computational methods for event understanding would benefit from conceptualizing the intentions behind the processes. moreover, inducing intentions is crucial to rich understanding of text @xcite , and could potentially support other applications such as commonsense reasoning @xcite , summarization @xcite , reading comprehension @xcite and schema induction @xcite . to understand the intentions of event processes, the first contribution of this paper is to propose a new semantic typing task. the event process typing task seeks to retrieve ultra fine-grained type dig a hole fill a highball with ice put seeds in fill with soil water soil add vodka add pineapple juice stir with a bar set locations and dates compare airfares purchase a ticket action: make object: cocktail action: plant object: plant action: book object: flight information to summarize the goal and intention of the associated events. specifically, each event process is typed along two axes: the action type that describes the type of action the process takes, and the object type that semantically types the object(s) that the process seeks to affect. figure 1 shows several accordingly typed event processes. motivated by recent works on entity typing @xcite , our task employs large type vocabularies supporting diverse free-form semantic labels for both axes. to facilitate related research, we developed a large dataset extracted from wikihow foot_0 , as the second contribution of this paper. this dataset contains over 60,000 processes of primitive events, and features fine-grained action and object type labels for each process. while the dataset aims at creating rich examples of event process intentions, it is also a challenging dataset from two perspectives. first, vocabularies on both type axes are remarkably diverse, giving over 1,000 action type labels and over 10,000 object type labels. and, these fine-grained type vocabularies occur quite sparsely -around 68% of action types and 88% of object types occur fewer than 10 times. this leads to a few-shot learning scenario and, in nearly half of the cases, one-shot. second, the free-form type labels are generally external to the lexical content of the associated events appearing in a process. hence, this typing task could not be easily handled with an extractive method @xcite . while the task and dataset pose a non-trivial learning problem, the free-form type system allows for a practical form of indirect supervision based on gloss knowledge. as the third contribution, we propose a hybrid learning framework, p2gt (i.e., process-to-gloss based typing), to leverage such in-direct supervision for event process typing. instead of directly inferring the multi-axis type labels, we find it to be much easier to seize on the semantic relatedness between the process-gloss pair, as the gloss provides richer semantic information than the label itself. for few-shot cases, gloss definitions also represent useful side information to jump-start inducing labels that are rarely seen or completely unseen in training. the proposed framework fine-tunes a pre-trained language model to capture the relatedness of an event process and the gloss of types with a ranking task objective. to incorporate more precise gloss information, the training process deploys a word sense disambiguation (wsd) module for both verbs and nouns. joint learning for both action and object types is enforced to further complement scarce supervision signals. based on extensive experimental evaluation, the proposed framework exhibits promising performance of inferring the finegrained multi-axis type information. specifically, it outperforms a strong roberta-based baseline by 2.4-3.0 folds in recall@1. we also show that the incorporated gloss knowledge supports few-shot case prediction, and benefits our model's generalization to out-of-domain event processes. ## task and dataset we hereby formulate the task of multi-axis event process typing, and introduce the contributed dataset. ## process typing with gloss knowledge in this section, we present our method for the multiaxis event process typing task. the proposed p2gt framework conducts learning in three steps. a pretrained language model is first used to produce the representations of processes. then, the gloss information of type vocabularies is encoded as intermediate representations for type labels using the same language model, for which wsd is performed to refine the gloss information of polysemous labels during training. finally, the language model is finetuned with a ranking task objective to capture the association of process-gloss pairs. in the last step thereof, joint learning is performed for typing on both axes to complement the scarce supervision signals, where a process representation is separately projected and handled for action and object types in the latent space. figure 4 displays the overall model architecture. in the rest of this section, we introduce the technical details of each step for learning and inference. ## experiments to evaluate the proposed p2gt framework for event process typing, we conduct several experiments on the contributed dataset, and compare with a wide selection of baseline methods ( §4.1- §4.3). a case study is also provided on typing processes from an external dataset ( §4.4). ## conclusion we propose a new task of event process understanding, by semantically typing the intended action of an event process and the object(s) it seeks to affect. to facilitate research in this direction, we develop a new dataset, gathering over 60 thousand event processes with ultra fine-grained type vocabularies. we further propose a hybrid learning framework, which leverages indirect supervision from gloss knowledge. the proposed p2gt framework fine-tunes roberta to capture the association of process-gloss pairs. label gloss selection mechanisms and joint training are incorporated to further improve the performance. experiments show that p2gt offers promising performance on inferring the fine-grained type information, and exhibits satisfactory generalizability on out-of-domain event processes. for future work, we are interested in identifying salient events in processes, i.e., those that most significantly define the central goals. incorporating process typing into downstream tasks such as summarization and commonsense qa is also an important direction.