Joint Detection and Coreference Resolution of Entities and Events with Document-level Context Aggregation (2021)

Abstract: Constructing knowledge graphs from unstructured text is an important task that is relevant to many domains. Most previous work focuses on extracting information from sentences or paragraphs, due to the difficulty of analyzing longer contexts. In this paper we propose a new jointly trained model that can be used for various information extraction tasks at the document level. The tasks performed by this system are entity and event identification, typing, and coreference resolution. In order to improve entity and event typing, we utilize context-aware representations aggregated from the detected mentions of the corresponding entities and events across the entire document. By extending our system to document-level, we can improve our results by incorporating cross-sentence dependencies and additional contextual information that might not be available at the sentence level, which allows for more globally optimized predictions. We evaluate our system on documents from the ACE05-E+ dataset and find significant improvement over the sentence-level SOTA on entity and event trigger identification and classification.

## introduction recently, large transformer models, such as bert @xcite , transformer-xl @xcite , and roberta @xcite , have attracted a lot of attention from the natural language processing (nlp) community. these models are typically pretrained on a large unlabeled corpus, and can be consequently fine-tuned for specific nlp tasks using a relatively small amount of supervised data. by adding shallow classifiers on top of the context-sensitive embeddings produced by these neural networks, state-of-the-art results have been achieved on various subtasks in information 1 code is available at https://github.com/ sam1373/long_ie extraction @xcite @xcite . despite the ability of transformer models to efficiently capture information across a long context, most ie work still focuses on extracting information from sentences @xcite , or, in some cases, short paragraphs @xcite . additionally, some work has been done where longer documents are represented by encoding sentences or paragraphs separately @xcite . while some datasets have been proposed which contain document-level annotations of entities and relations @xcite @xcite , very little work has been done in effectively utilizing the fully available document-level context in order to produce globally optimal predictions. the main contribution of this paper is the introduction and evaluation of our new neural ie model, which can be used to jointly perform various ie subtasks in the full document context. our model receives only the original document text as input. after identifying relevant entity and event trigger mentions in the text, we perform clustering to determine which entities or events each mention belongs to. in order to make full use of the contextual information related to an entity/event in a given document, we aggregate information from all of the corresponding mentions to create a documentlevel representation, which can then be used for type prediction of entities and events. we focus on constructing a model which can efficiently tackle the challenges that arise in this currently not well explored variant of the task. our approach achieves an improvement of about 2% absolute gain over the previous results on the ace05-e + dataset in terms of f-score for entity extraction and event detection. ## conclusions and future work aggregating information across an entire document can be highly effective for classifying entity and event mention types. this is particularly useful in cases where pronouns are used to refer to entities or events that are not explained within the same sentence. in the future, we plan to extend our approach to use document-level context for extraction of relations between entities and event arguments.