A Context-Dependent Gated Module for Incorporating Symbolic Semantics into Event Coreference Resolution (2021)

Abstract: Event coreference resolution is an important research problem with many applications. Despite the recent remarkable success of pre-trained language models, we argue that it is still highly beneficial to utilize symbolic features for the task. However, as the input for coreference resolution typically comes from upstream components in the information extraction pipeline, the automatically extracted symbolic features can be noisy and contain errors. Also, depending on the specific context, some features can be more informative than others. Motivated by these observations, we propose a novel context-dependent gated module to adaptively control the information flows from the input symbolic features. Combined with a simple noisy training method, our best models achieve state-of-the-art results on two datasets: ACE 2005 and KBP 2016.

## introduction within-document event coreference resolution is the task of clustering event mentions in a text that refer to the same real-world events @xcite . it is an important research problem, with many applications @xcite @xcite . since the trigger of an event mention is typically the word or phrase that most clearly describes the event, virtually all previous approaches employ features related to event triggers in one form or another. to achieve better performance, many methods also need to use a variety of additional symbolic features such as event types, attributes, and arguments @xcite @xcite @xcite @xcite . previous neural methods @xcite @xcite also use noncontextual word embeddings such as word2vec 1 the code is publicly available at https://github.com/ laituan245/eventcoref . with the recent remarkable success of language models such as bert @xcite and spanbert @xcite , one natural question is whether we can simply use these models for coreference resolution without relying on any additional features. we argue that it is still highly beneficial to utilize symbolic features, especially when they are clean and have complementary information. there are several previous methods that use contextual embeddings together with type-based or argument-based information @xcite . for example, @xcite proposes a new mechanism to better exploit event type information for coreference resolution. despite their impressive performance, these methods are specific to one particular type of additional information. in this paper, we propose general and effective methods for incorporating a wide range of sym-bolic features into event coreference resolution. simply concatenating symbolic features with contextual embeddings is not optimal, since the features can be noisy and contain errors. also, depending on the context, some features can be more informative than others. therefore, we design a novel context-dependent gated module to extract information from the symbolic features selectively. combined with a simple regularization method that randomly adds noise into the features during training, our best models achieve state-of-the-art results on ace 2005 @xcite and kbp 2016 @xcite datasets. to the best of our knowledge, our work is the first to explicitly focus on dealing with various noisy symbolic features for event coreference resolution. ## experiments and results data and experiments setup we evaluate our methods on two english datasets: ace2005 @xcite and kbp2016 @xcite . we report results in terms of f1 scores obtained using the conll and avg metrics. by definition, these metrics are the summary of other standard coreference metrics, including b 3 , muc, ceaf e , and blanc @xcite . we use spanbert (spanbert-base-cased) as the transformer encoder @xcite . more details about the datasets and hyperparameters are in the appendix. we refer to models that use only trigger features as @xcite . in a baseline model, f ij is simply t ij (eq. 2). we refer to models that use only the simple concatenation strategy as [simple] (eq. 4), and models that use the simple concatenation strategy and the noisy training method as @xcite . ## conclusions and future work in this work, we propose a novel gated module to incorporate symbolic semantics into event coreference resolution. combined with a simple noisy training technique, our best models achieve competitive results on ace 2005 and kbp 2016. in the future, we aim to extend our work to address more general problems such as cross-lingual crossdocument coreference resolution.