Corpus-based Open-Domain Event Type Induction (2021)

Abstract: Traditional event extraction methods require predefined event types and their corresponding annotations to learn event extractors. These prerequisites are often hard to be satisfied in real-world applications. This work presents a corpus-based open-domain event type induction method that automatically discovers a set of event types from a given corpus. As events of the same type could be expressed in multiple ways, we propose to represent each event type as a cluster of <predicate sense, object head> pairs. Specifically, our method (1) selects salient predicates and object heads, (2) disambiguates predicate senses using only a verb sense dictionary, and (3) obtains event types by jointly embedding and clustering <predicate sense, object head> pairs in a latent spherical space. Our experiments, on three datasets from different domains, show our method can discover salient and high-quality event types, according to both automatic and human evaluations.

## introduction one step towards converting massive unstructured text into structured, machine-readable representations is event extraction-the identification and typing of event triggers and arguments in text. most event extraction methods @xcite @xcite assume a set of predefined event types and their corresponding annotations are curated by human experts. this annotation process is expensive and time-consuming. besides, those manually-defined event types often fail to generalize to new domains. for example, the widely used ace 2005 event schemas 2 do not contain any event type 1 the programs, data and resources are publicly available for research purpose at https://github.com/ mickeystroller/etypeclus . ## problem formulation in this section, we first introduce some important concepts and then present our task definition. a corpus @xmath0, . . . , s n } is a set of sentences where each sentence s @xmath1,1 , . . . , w i,n ]. a predicate is a verb mention in a sentence and can optionally have an associated object in the same sentence. we follow previous studies @xcite and refer to the most important word in the object as the object head. for example, one predicate from the first sentence in figure 1 is "detain" and its corresponding object is "hundreds of people" with the word "people" being the object head. as predicates with the same lemma may have different senses, we disambiguate each predicate verb based on a verb sense dictionary v wherein each verb lemma has a list of candidate senses with example usage sentences. one illustrative example of our verb sense dictionary is shown in figure 3 . we refer to the sense of predicate verb lemma as the predicate sense. task definition. given a corpus s and a verb sense dictionary v, our task of event type induction is to identify a set of k event types where each type t j is represented by a cluster of predicate sense, object head pairs. ## the etypeclus framework the etypeclus framework (outlined in figure 2 ) induces event types in four major steps: (1) predicate and object head extraction, (2) salient predicate lemma and object head selection, (3) predicate sense disambiguation, and (4) latent space joint predicate sense and object head clustering. ## evaluation on ace/ere datasets we first evaluate etypeclus on two widely used event extraction datasets: ace (automatic content extraction) 2005 12 and ere (entity relation event) @xcite . for both datasets, we follow the same preprocessing steps from @xcite and use sentences in the training split as our input corpus. the ace dataset contains 17,172 sentences with 33 event types and the ere dataset has 14,695 sentences with 38 types. we test the performance of etypeclus on event type discovery and event mention clustering. ## evaluation on pandemic dataset to evaluate the portability of etypeclus to a new open domain, we collect a new dataset that includes 98,000 sentences about disease outbreak events 14 . we run the top-3 performing baselines and etypeclus to generate 30 candidate event types and evaluate their quality using intrusion test. specifically, we inject a negative sample from other clusters into each cluster's top-5 results and ask three annotators to identify the outlier. more details on how we construct the intrusions are in appendix. the intuition behind this test is that the annotators will be easier to identify the intruders if the clustering results are clean and tuples are semantically coherent. as shown in etypeclus. 15 interesting examples include tuples with the same predicate sense but object heads with different granularities (e.g., spread_2, virus and spread_2, coronavirus for spread-virus type), tuples with same object head but different predicate senses (e.g., prevent_1, spread , and mitigate_1, spread for prevent-spread type), and event types with predicate verb lemmas that are not directly linkable to ontonotes senses grouping (e.g., "immunize" and "vaccinate" for vaccinate type). weakly-supervised event extraction. some studies on event extraction @xcite @xcite propose to leverage annotations for a few seen event types to help extract mentions of new event types specified by just a few keywords. these methods reduce the annotation efforts but still require all target new types to be given. recently, some studies (huang 15 more example outputs are in appendix section h. @xcite @xcite ) use transfer learning techniques to extend traditional event extraction models to new types without explicitly deriving schemas of new event types. compared to our study, these methods still require many annotations for a set of seen types and their resulting vector-based event type representations are less human interpretable. another related work by @xcite uses gan to extract events from an open domain corpus. it clusters entity:location:keyword:date quadruples related to the same event rather than finds event types. ## conclusions and future work in this paper, we study the event type induction problem that aims to automatically generate salient event types for a given corpus. we define a novel event type representation as a predicate sense, object head cluster, and propose etypeclus that can extract and select salient predicates and object heads, disambiguate predicate senses, and jointly embed and cluster p-o pairs in a latent space. experiments on three datasets show that etypeclus can recover human curated types and identify new salient event types. in the future, we propose to explore the following directions: (1) improve predicate and object extraction quality with tools of higher semantic richness (e.g., a srl labeler or an amr parser);