WEC : Deriving a Large-scale Cross-document Event Coreference dataset from W ikipedia (2021)

Abstract: Cross-document event coreference resolution is a foundational task for NLP applications involving multi-text processing. However, existing corpora for this task are scarce and relatively small, while annotating only modest-size clusters of documents belonging to the same topic. To complement these resources and enhance future research, we present Wikipedia Event Coreference (WEC), an efficient methodology for gathering a large-scale dataset for cross-document event coreference from Wikipedia, where coreference links are not restricted within predefined topics. We apply this methodology to the English Wikipedia and extract our large-scale WEC-Eng dataset. Notably, our dataset creation method is generic and can be applied with relatively little effort to other Wikipedia languages. To set baseline results, we develop an algorithm that adapts components of state-of-the-art models for within-document coreference resolution to the cross-document setting. Our model is suitably efficient and outperforms previously published state-of-the-art results for the task.

## introduction cross-document (cd) event coreference resolution is the task of identifying clusters of text mentions, across multiple texts, that refer to the same event. successful identification of such coreferring mentions is beneficial for a broad range of applications at the multi-text level, which are gaining increasing interest and need to match and integrate information across documents, such as multidocument summarization @xcite , multi-hop question answering @xcite and knowledge base population (kbp) @xcite . unfortunately, rather few datasets of reasonable scale exist for cd event coreference. notable datasets include ecb+ @xcite , meantime @xcite and the gun violence corpus (gvc) @xcite (described in section 2), where recent work has been evaluated solely on ecb+. when addressed in a direct manner, manual cd coreference annotation is very hard due to its worst-case quadratic complexity, where each mention may need to be compared to all other mentions in all documents. indeed, ecb+ contains less than 7000 event mentions in total (train, dev, and test sets). further, effective corpora for cd event coreference are available mostly for english, limiting research opportunities for other languages. partly as a result of this data scarcity, rather little effort was invested in this field in recent years, compared to dramatic recent progress in modeling within-document coreference. furthermore, most existing cross-document coreference datasets are restricted in their scope by two inter-related characteristics. first, these datasets annotate sets of documents, where the documents in each set all describe the same topic, mostly a news event (consider the malaysia airlines crash as an example). while such topicfocused document sets guarantee a high density of coreferring event mentions, facilitating annotation, in practical settings the same event might be mentioned across an entire corpus, being referred to in documents of varied topics. second, we interestingly observed that event mentions may be (softly) classified into two different types. one type, which we term a descriptive mention, pertains to a mention involved in presenting the event or describing new information about it. for example, news about the malaysian airline crash will include mostly descriptive mentions of the event and its sub-events, such as shot-down, crashed and investigated. naturally, news documents about a topic, as in prior event coreference datasets, include mostly descriptive event mentions. the other type, which we term a referential mention, pertains to mentions of the event in sentences that do not focus on presenting new information about the event but rather mention it as a point of reference. for example, mentions referring to the airplane crash, such as the malaysian plane crash, flight mh17 or disaster may appear in documents about the war in donbass or about flight safety. since referential event mentions are split across an entire corpus, they are less trivial to identify for coreference annotation, and are mostly missing in current newsbased datasets. as we demonstrate later, these two mention types exhibit different lexical distributions and seem to require corresponding training data to be properly modeled. in this paper, we present the wikipedia event coreference methodology (wec), an efficient method for automatically gathering a large-scale dataset for the cross-document event coreference task. our methodology effectively complements current datasets in the above-mentioned respects: data annotation is boosted by leveraging available information in wikipedia, practically applicable for any wikipedia language; mentions are gathered across the entire wikipedia corpus, yielding a dataset that is not partitioned by topics; and finally, our dataset consists mostly of referential event mentions. in its essence, our methodology leverages the coreference relation that often holds between anchor texts of hyperlinks pointing to the same wikipedia article (see figure 1 ), similar to the basic idea introduced in the wikilinks dataset @xcite . focusing on cd event coreference, we identify and target only wikipedia articles denoting events. anchor texts pointing to the same event article, along with some surrounding context, become candidate mentions for a corresponding event coreference cluster, undergoing extensive filtering. we apply our method to the english wikipedia and extract wec-eng, our english version of a wec dataset. the automaticallyextracted data that we collected provides a training set of a very large scale compared to prior work, while our development and test sets underwent relatively fast manual validation. due to the large scale of the wec-eng training data, current state-of-the-art cd coreference models cannot be easily trained and evaluated on it, for scalability reasons. we therefore developed a new, more scalable, baseline model for the task, while adapting components of recent competitive within-document coreference models @xcite @xcite . in addition to setting baseline results for wec-eng, we assess our model's competitiveness by presenting a new state-of-the-art on the commonly used ecb+ dataset. finally, we propose that our automatic extraction and manual validation methods may be applied to generate additional annotated datasets, particularly for other languages. overall, we suggest that future cross-document coreference models should be evaluated also on the wec-eng dataset, and address its complementary characteristics, while the wec methodology may be efficiently applied to create additional datasets. to that end, our dataset and code 12 are released for open access. ## related datasets this section describes the main characteristics of notable datasets for cd event coreference (ecb+, meantime, gvc). ## the wec methodology and dataset we now describe our methodology for gathering a cd event coreference dataset from wikipedia, and the wec-eng dataset created by applying it to the english wikipedia. we also denote how this methodology can be applied, with some languagespecific adjustments, to other wikipedia languages. ## conclusion and future work in this paper, we presented a generic low-cost methodology and supporting tools for extracting cross-document event coreference datasets from wikipedia. the methodology was applied to create the larger-scale wec-eng corpus, and may be easily applied to additional languages with relatively few adjustments. most importantly, our dataset complements existing resources for the task by addressing a different appealing realistic setup: the targeted data is collected across a full corpus rather than within topical document clusters, and, accordingly, mentions are mostly referential rather than descriptive. hence, we suggest that future research should be evaluated also on wec-eng, while future datasets, particularly for other languages, can be created using the wec methodology and tool suite, all made publicly available. our released model provides a suitable baseline for such future work.