FAM u S : Frames Across Multiple Sources (2024)

Abstract: Understanding event descriptions is a central aspect of language processing, but current approaches focus overwhelmingly on single sentences or documents. Aggregating information about an event across documents can offer a much richer understanding. To this end, we present FAMuS, a new corpus of Wikipedia passages that report on some event, paired with underlying, genre-diverse (non-Wikipedia) source articles for the same event. Events and (cross-sentence) arguments in both report and source are annotated against FrameNet, providing broad coverage of different event types. We present results on two key event understanding tasks enabled by FAMuS: source validation—determining whether a document is a valid source for a target report event—and cross-document argument extraction—full-document argument extraction for a target event from both its report and the correct source article.

## introduction recent years have witnessed a resurgence of interest in document-level event and argument extraction tasks, such as template filling @xcite @xcite , role-filler entity extraction @xcite , and event argument extraction @xcite @xcite . indeed, the earliest goals of information extraction (ie), as advanced by the message understanding conferences (mucs), were to develop systems capable of extracting document-level event structures (grishman and @xcite . while the renewed interest in these goals represents clear progress beyond the longstanding and dominant focus on sentence-level event extraction, recent work in this area suffers from two key shortcomings. for one, major benchmarks on these tasks, including @xcite , rams @xcite , wikievents @xcite , and docee for another, although whole-document extraction enables a richer understanding of an event than its sentence-level analogue, it is still constrained by the input document's description of that event, which may lack key details. the task of event linking partly remedies this by linking event mentions to a canonical entry in a knowledge base, but stops there, providing no actual extractions from those entries @xcite . this work introduces famus (frames across multiple sources), a dataset and benchmark aimed at addressing both of these shortcomings. famus provides event and cross-sentence argument annotations on over 1,255 wikipedia passages (or reports), each paired with cross-sentence argument annotations for the same event as described in the document cited as the passage's source. events and arguments are annotated against framenet @xcite , providing genuinely broad coverage with 253 diverse event types and five supporting documents per type. beyond the dataset itself, we on 16 july 2007 the inquiry was adjourned until 4 september with a final deadline for the submission of evidence of 14 august 2007 . on 11 september 2007 the inquiry was again adjourned until 19 september to allow the highways agency to review traffic evidence , with further adjournments until 18 december . ## task definitions to situate famus in the context of prior work, we first give a formal statement of the tasks it presents: ## background we are aware of no prior work that combines identification of a report event's source document (sv) with argument extraction from both the report and the source (cdae). however, both closely relate to a number of established tasks in the literature, which we survey briefly below. event linking (el) or event grounding is the task of associating an event description (typically, a single mention) with a canonical entry for that event in some knowledge base. it resembles sv in attempting to ground a target event mention in a text to a more comprehensive description of the same event in a source text. but whereas sv takes a candidate source text as input (along with the report), el aims to produce (a link to) one as output. introduced by @xcite as an eventcentric analogue to the more popular entity linking objective @xcite , el has received comparatively little attention. while nothman et al. used australian news articles for both report and source, more recent efforts have focused on wikipedia and wikidata. @xcite use wikipedia articles as source documents and present evaluations with both wikipedia and new york times report articles. @xcite , extending work by @xcite , propose an interesting hierarchical variant of the task, in which mentions must be linked to a set of hierarchically related events in wikidata. ## data collection the famus documents represent a subset of english documents from megawika @xcite , a dataset comprising millions of (report, source) pairs across 50 languages. below, we discuss data collection for our sv and cdae tasks. ## experiments we now describe the models and setup for experiments on sv and cdae. model hyperparameters, prompts and details can be found in appendix c. ## conclusion we have presented famus, a new dataset comprising reports (wikipedia passages) that describe an event, along with source documents for those events-featuring high-quality, fulldocument framenet frame and role annotations on both. we have also introduced two event understanding tasks enabled by famus: source validation-determining whether a candidate document is a valid source for a given report event-and cross-document argument extraction-extracting the arguments of an identified report event in both the report and its source. we have provided baselines for both tasks, along with detailed analysis of their performance, and release both these models and our data to facilitate future research.