Open Relation and Event Type Discovery with Type Abstraction (2022)

Abstract: Conventional “closed-world” information extraction (IE) approaches rely on human ontologies to define the scope for extraction. As a result, such approaches fall short when applied to new domains. This calls for systems that can automatically infer new types from given corpora, a task which we refer to as type discovery.To tackle this problem, we introduce the idea of type abstraction, where the model is prompted to generalize and name the type. Then we use the similarity between inferred names to induce clusters. Observing that this abstraction-based representation is often complementary to the entity/trigger token representation, we set up these two representations as two views and design our model as a co-training framework. Our experiments on multiple relation extraction and event extraction datasets consistently show the advantage of our type abstraction approach.

## introduction information extraction has enjoyed widespread success, however, the majority of information extraction methods are "reactive", relying on end-users to specify their information needs in prior and provide supervision accordingly. this leads to "closedworld" systems @xcite @xcite @xcite that are confined to a set of pre-defined types. it is desirable to make systems act more "proactively" like humans who are always on the lookout for interesting new information, generalize them into new types, and find more instances of such types, even if they are not seen previously. one related attempt is the open information extraction paradigm @xcite , which aims at extracting all @xcite triples from text that denote some kind of relation. while openie does not rely on pre-specified relations, its exhaustive and free-form nature often leads to noisy and redundant extractions. ## problem definition we first define the task of type discovery and then discuss the realization of this task to relations and events. given a set of unlabeled instances and an estimated number of unknown types |c u |, the goal of type discovery is to learn a model f that can map @xmath0. in the case of relation discovery, each instance x is an entity mention pair {h, t} embedded within a sentence context. as shown in figure 1 , the instance is "john earned a bachelor's degree from the university of wollongong" with the head entity mention "john" and the tail entity mention "university of wollongong". each entity mention is a span with start and end indexes (s, e) in the sentence. the associated label y in this case is relation type "school_attended". 1 in the case of event discovery, each instance x is a trigger word/phrase mention t with start and the acquisition of banko @xmath1. ## method our model is built on the observation that the token view and the mask view are often complementary and work well for different types. thus, the core of our model is the construction of two views and how they can be utilized for co-training. ## experiments in the following experiments, we refer to our model as tabs to stand for "type abstraction". ## analysis predicted type names. in ## related work relation type discovery early work in this direction represented relations as clusters of lexical patterns or syntactic paths @xcite @xcite @xcite . a wave of newer methods used learned relation representations @xcite @xcite @xcite @xcite , often defining the relation as a function of the involved entities. one key observation made by rsn @xcite and @xcite is the possibility of relational knowledge transfer from known relation types to new types. in this work, we also adopt this transfer setting and introduce a new idea of abstraction: a relation cluster is meaningful if it aligns well with a human concept. event type discovery our task of event discovery is similar to the verb clustering task in se-meval 2019 @xcite which requires mapping verbs in context to semantic matched type instances in cluster personnel: start-position condi rice has been chosen by president bush to become the new secretary of state ... if you were president, which national figures would you appoint to your cabinet and why? al-douri taught international law at baghdad university before becoming a diplomat ... chui sai on, who has been named [personnel: nominate] director of the sars task force ... conflict:demonstrate some 70 people were arrested saturday as demonstrators clashed with police at the end of a major peace rally ... between 2,500 and 3,000 people picketed the cnn studios in los angeles ... the crowd filled the street leading to the kazimiya mosque in the northeast of baghdad ... justice: arrest-jail some 70 people were arrested saturday as demonstrators clashed with police at the end of a major peace rally ... ferris disappeared from sight, and cnn has confirmed he was taken into custody. ## conclusions and future work in this paper we study the type discovery problem: automatically identifying and extracting new relation and event types from a given corpus. we propose to leverage type abstraction, where the model is prompted to name the type, as an alternative view of the data. we design a co-training framework, and demonstrate that our framework works favorably in both relation and event type discovery settings. currently we have assumed that the new types are disjoint to the old types and the model operates similarly to a transfer learning setting. while the model can be easily extended to handle both new types and old types, more analysis might be needed in this direction. one potential direction would be to explore a continual learning setting, where new types could emerge periodically.