QAEVENT : Event Extraction as Question-Answer Pairs Generation (2024)

Abstract: We propose a novel representation of document-level events as question and answer pairs (QAEVENT). Under this paradigm: (1) questions themselves can define argument roles without the need for predefined schemas, which will cover a comprehensive list of event arguments from the document; (2) it allows for more scalable and faster annotations from crowdworkers without linguistic expertise. Based on our new paradigm, we collect a novel and wide-coverage dataset. Our examinations show that annotations with the QA representations produce high-quality data for document-level event extraction, both in terms of human agreement level and high coverage of roles comparing to the pre-defined schema. We present and compare representative approaches for generating event question answer pairs on our benchmark.

## introduction event extraction (ee) is a challenging yet important task in information extraction research @xcite . the task aims at extracting event information from unstructured texts into a structured form, which mostly describes attributes such as "who", "when", "where", and "what" of realworld events that happened @xcite . the task involves extracting the trigger (predicate) for an event and identifying its arguments for a certain role from a sentence @xcite , or a document containing multiple sentences @xcite @xcite @xcite . however, highly skilled and trained annotators with linguistic expertise are required for labeling the event structures in the document @xcite , especially for domain-specific documents. 1 our dataset and code are available at https:// github.com/milind21/qag_ee we begin with a history of the dash family of sussex, england: ... mr. dash has three kids including barry ... [over 10 sentences] the head of the family, old mr. dash dies ... during an accident on july 5 ... [over 10 sentences] according to the plan, his estate were distributed among his surviving relatives: his nephew, henry, and his children. ... [over 10 sentences] barry found that estate inherited from his father, includes the money earned during given by his uncle. ## related work on semantic qa approaches using qa structures to represent semantic propositions has been proposed as a way to generate "soft" annotations, where the resulting representation is formulated using natural language, which is shown to be more intuitive for untrained annotators @xcite . this allows much faster and more large-scale annotation processes @xcite and when used in a more controlled crowdsourcing setup can produce high-coverage quality annotations for sentence-level tasks @xcite . both qasrl @xcite and qamr @xcite collect a set of qa pairs, each representing a single proposition, for a sentence. in qasrl, the main target is a predicate, which is emphasized by replacing all content words in the question besides the predicate with a placeholder, and the answer constitutes a span of the sentence. the annotation process itself for qasrl is very controlled, by suggesting questions created with a finite-state automaton. qamr, on the other hand, allows us to freely ask all kinds of questions about all types of content words in a sentence. the approach taken in qaevent differs significantly from the works of @xcite and @xcite . they propose a templatebased question generation for improving event extraction (under a prefined-schema paradigm) while our work is the first to propose a new paradigm in representing document-level events as qa pairs, which allows higher coverage and annotation efficiency. based on our experiments, we also observe that datasets annotated under qaevent paradigm improve the event extraction in general. ## dataset collection we describe our annotation process in detail and discuss the agreement between our qaevent annotations and the corresponding standard event extraction annotations in wikievents @xcite . ## question answer pair generation in this section, we present the various question answer pair generation (qag) methods. formally, given a document d, for every trigger t i in d, we aim to generate question answer pairs {(q 1 , a 1 ), ...(q j , a j )} to annotate arguments of triggers t i , where each qa pair represents one argument of the event. a j is supposed to be the answer corresponding to q j . ## event extraction performance this section discusses the benefits of qaevent dataset on improving the event extraction task performance. ## conclusion in this work, we show that document-level events can be represented using qa pairs. this representation results in scalable and fast annotations from crowd-sourcing. we presented a set of guidelines that can be used to collect event qa pairs and conducted crowd-sourcing for collecting a qaevent corpus. we found that: (1) annotation is more efficient under our paradigm, it takes a much shorter time as compared to the original wikievents annotation; (2) our annotations align well with wikievents event arguments, and in addition, cover more nuanced and fine-grained arguments/attributes. finally, we establish both rulebased and llm-based baselines on our benchmark.