Explicit, Implicit, and Scattered: Revisiting Event Extraction to Capture Complex Arguments (2024)

Abstract: Prior works formulate the extraction of event-specific arguments as a span extraction problem, where event arguments are explicit — i.e. assumed to be contiguous spans of text in a document. In this study, we revisit this definition of Event Extraction (EE) by introducing two key argument types that cannot be modeled by existing EE frameworks. First, implicit arguments are event arguments which are not explicitly mentioned in the text, but can be inferred through context. Second, scattered arguments are event arguments that are composed of information scattered throughout the text. These two argument types are crucial to elicit the full breadth of information required for proper event modeling.To support the extraction of explicit, implicit, and scattered arguments, we develop a novel dataset, DiscourseEE, which includes 7,464 argument annotations from online health discourse. Notably, 51.2% of the arguments are implicit, and 17.4% are scattered, making DiscourseEE a unique corpus for complex event extraction. Additionally, we formulate argument extraction as a text generation problem to facilitate the extraction of complex argument types. We provide a comprehensive evaluation of state-of-the-art models and highlight critical open challenges in generative event extraction. Our data and codebase are available at https://omar-sharif03.github.io/DiscourseEE.

## introduction event extraction (ee) is a challenging yet crucial nlp task required for event-centric information extraction. ee is the composition of two tasks: (i) event detection (ed), identifying if an event occurs in a text and (ii) event argument extraction (eae), extracting event-specific details or event arguments according to a pre-defined event ontology. existing works in ee have two key limitations. first, most prior works are focused on event extraction from formal texts (e.g., news or wikipedia figure 1: an example demonstrating complex event arguments that are prevalent in online discourse. this reddit post is narrated by a newly diagnosed prostate cancer patient who is seeking treatment information from online peers on the r/prostatecancer subreddit. in addition to explicit arguments, it contains implicit and scattered arguments that cannot be extracted using one contiguous span of text. second, existing works in eae extract an argument as a span in the input text @xcite . this is extremely limiting as many event arguments are implicit and only discernible from subtext. in figure 1 , we illustrate the need for complex argument types through the lens of a reddit post written by a newly diagnosed cancer patient. we find that only a small fraction of crucial event-specific details can be tied to contiguous spans of text in the post -demanding a more complex ee solution. for example, the patient's consideration of prostate removal surgery is only discernible through a mention of the "da vinci route", which is in reference to the da vinci surgical robot. such implicit information is crucial to providing in-depth event details and will improve the accuracy of large-scale event-centric information aggregation efforts. in this paper, we address these two limitations by reformulating ee annotation as text generation and introducing a novel dataset, discourseeewhich contains ee annotation for health-related discourse on reddit. by focusing on health discourse, we can provide a nuanced understanding of healthcare needs which has significant implications for public health research @xcite @xcite . discourseee introduces a novel ee annotation strategy that facilitates the extraction of the following three argument types: (i) explicit arguments: event details that are found directly in the document. (ii) implicit arguments: event details which are not directly mentioned in the document but can be inferred using context. (iii) scattered arguments: event details which are the composition of multiple pieces of information scattered throughout the document. while there is significant prior work on the extraction of explicit arguments, discourseee is the first to introduce implicit and scattered argument extraction. when compared to prior work, our ee formulation adds significant depth to the amount of event information that can be extracted, making discourseee well-suited to improve a range of downstream tasks such as questionanswering @xcite , or rumor @xcite , conflicting information @xcite , and misinformation @xcite detection from complex, online discourse. additionally, by changing the eae paradigm from span extraction to text generation, we better align ee with the abilities of large language models (llms), which have been shown to have limited capacity on extractive ee tasks in prior work @xcite . our major contributions are as follows. ## event ontology design event types: analyzing online discourse through an event-argument framework can inform data mining and knowledge discovery for several impactful domains including but not limited to healthcare, politics, public policy, finance, and law. as demonstrated in the motivating example in figure 1 , millions of patients across the world seek informational and emotional support in online peer communities (e.g., reddit, facebook) on different conditions, e.g., mental health, pregnancy, recovery from substance use disorder, cancer, and other chronic diseases. while our proposed method can be applied to any of these use cases, we found only one labeled, large-scale event dataset for online discourse / social media discussion, namely treat-ise @xcite . this dataset covers health discourse regarding medications for opioid use disorder (moud), a critical yet stigma11 . ## benchmarking ee models in this section, we detail our experimental setup for benchmarking a wide range of models on ed and eae for discourseee. ## related work event extraction with generative models: prior studies have approached ee as a token-level classi-fication or extractive task @xcite @xcite . recently, ee has been formulated as a text generation task using pre-trained language models, where the model is prompted to fill in natural language templates @xcite . with the advent of llms, generation-based ee gained more traction @xcite . however, most of these generative models are evaluated using span-based annotated datasets, which can underestimate their performance @xcite . this is because the models' predictions may differ from the exact ground-truth spans yet still be accurate. this work addresses this gap by proposing a relaxed-match evaluation metric, and presenting a new dataset and benchmarks to facilitate generative event extraction. event extraction from social media: existing research on social media primarily addresses event detection and often overlooks argument extraction, a gap discourseee addresses. for instance, parekh et al. (2024) detected epidemic-related events from tweets, while guzman-nateras et al. ( 2022 ) identified suicide-related events on reddit. arguments in social media data vary in span and are often ambiguous, implicit, and scattered. our approach to implicit and scattered argument formulation differs from existing works. we define implicit arguments as those not directly mentioned in the document but can be inferred from the context and scattered arguments composed of information throughout the text. in contrast, existing works define implicit arguments as those that are explicitly mentioned but outside the fixed sentence window of the event's trigger @xcite @xcite . they overlook arguments that are entirely implicit or scattered. the event argument aggregation task done by @xcite is aligned with our work, but they do not model implicit and scattered arguments. moreover, they only focus on six discrete argument roles (time, place, casualties, after effects, reason, and participant) without grounding them with specific events. in contrast, our ee formulation adds significant depth to the amount of event information that can be extracted. we annotate 41 arguments from three events and characterize four types of complex arguments: core, type-specific, subject-specific, and effect-specific. our formulation better captures the complexity and nuance of real-world ee applications. ## conclusion this paper presents discourseee, a discourse-level ee dataset with fine-grained annotations of complex event arguments and develops a novel pipeline for extracting these arguments. discourseee provides a new resource for studying implicit and scattered arguments within complex social discourse, which has not been previously explored. the best performing gpt-4 model achieved only a 41.98% overall f1 score, highlighting the challenges of extracting such intricate arguments. specifically, the models accurately extracted only 36.53% of implicit arguments and 54.12% of scattered arguments, underscoring that effective argument extraction remains an open challenge. we believe discourseee fills a critical gap in ee research and provides a valuable, timely dataset and benchmark for generative event extraction.