Generating Uncontextualized and Contextualized Questions for Document-Level Event Argument Extraction (2024)

Abstract: This paper presents multiple question generation strategies for document-level event argument extraction. These strategies do not require human involvement and result in uncontextualized questions as well as contextualized questions grounded on the event and document of interest. Experimental results show that combining uncontextualized and contextualized questions is beneficial,especially when event triggers and arguments appear in different sentences. Our approach does not have corpus-specific components, in particular, the question generation strategies transfer across corpora. We also present a qualitative analysis of the most common errors made by our best model.

## introduction event argument extraction @xcite is about identifying entities participating in events and specifying their role (e.g., the giver, recipient, and thing given in a giving event). transforming natural language into structured event knowledge benefits many downstream tasks such as machine reading comprehension @xcite , news summarization @xcite , coreference resolution @xcite , and dialogue systems @xcite . traditionally, corpora are limited to arguments within the same sentence an event belongs to. intersentential arguments are more challenging and have received less attention @xcite . figure 1 presents an example from rams @xcite , the largest corpus annotating multi-sentence event-argument structures. two out of four event-argument relations cross sentence boundaries. the current landscape of argument extraction is dominated by prompt-based methods @xcite and question-answering russia began airstrikes against the is infrastructures and destroyed more than 500 trucks with oil. figure 1 : example from rams (top, event trigger importing and its arguments). in this paper, we experiment with several strategies to generate questions for event argument extraction. questions for the artifact of importing are shown in the dashed box. q1 is generated following a role-specific template, q2 and q3 are generated prompting gpt-4, and q4 and q5 are generated by a weakly-supervised t5 model. as illustrated in figure 1 , for the artifact role of event trigger importing, we generate five questions and employ transformers to identify answers that pinpoint the argument (i.e., oil) or, alternatively, indicate that there is no answer. the first three questions are uncontextualized (i.e., disregard the document; they only take into account the event and argument role). on the other hand, the last two questions are contextualized (i.e., grounded on the event trigger and document). contextualized questions are generated with a t5 model trained using weak supervision. as we shall see, they yield substantial improvements even though they are noisy (e.g., the answer to q4 is not oil). the main contributions of this paper are: ## previous work event argument extraction @xcite has a long history in the field @xcite . initially, datasets focused on extracting arguments within the same sentence than the event @xcite . there are also corpora focused on intersentential arguments @xcite @xcite . early models were based on handcrafted features @xcite @xcite . like most nlp tasks, models for event argument extraction experienced a transformation building on word embeddings, rnns, and cnns @xcite . transformer-based approaches are currently the best performing. some efforts assume event triggers and argument spans are part of the input and present classifiers to identify the argument role @xcite . unlike them-and like the remaining previous works discussed below-we only assume event triggers. at a high-level, efforts to identify argument spans and their role can be categorized into sequence labeling, casting the problem as a question-answering task, and using generative models. sequence label classifiers approach the problem with the traditional bio encoding @xcite . framing the problem in terms of questions and answers is popular @xcite @xcite . doing so enables zero-shot @xcite and few-shot @xcite predictions. @xcite and @xcite ; @xcite leverage generative language models @xcite . language generation facilitates a more flexible extraction by generating the arguments rather than identifying spans in the input document. transfer learning has also been explored, including role overlapping knowledge @xcite , semantic roles @xcite , abstract meaning representations @xcite , and frameaware knowledge distillation @xcite . we approach the problem as a question answering task by querying a transformer model. unlike previous work, we explore both uncontextualized as well as contextualized questions grounded on the event and document of interest. the latter questions are generated with a weakly supervised model. this allows us to train the model with one corpus (rams) that transfer to other corpus (wikievents). ## generating questions for event argument extraction we transform rams event-argument structures into questions and answers. while gold eventargument annotations include gold answers (i.e., the argument), there are no gold questions asking for the arguments filling the roles of event triggers. hence, we propose five question generation strategies divided into two categories: uncontextualized and contextualized. unlike uncontextualized questions, contextualized questions are grounded on the event and document of interest. ## experiments we formulate the problem of event argument extraction as a question-answering task to empirically assess the proposed question generation strategies. our experiments foot_0 are designed to determine the optimal strategy as well as the optimal combination of uncontextualized and contextualized questions. additionally, we conduct zero-and few-shot inference on rams to benchmark the gpt-3 model w.r.t our simplest question generation approach. ## qualitative error analysis we close our analyses by examining the errors made by the best model (boldfaced in ## weakly generated question analysis our empirical assessment across two corpora for event argument extraction indicates that training with weakly supervised questions yields positive results. to further understand this, we conduct a manual analysis of 100 random questions generated document: this is why turkey became anxious when russia began delivering airstrikes against the is infrastructure and destroyed more than 500 trucks with oil already. this really got on erdogan and his company's nerves. they're importing not only oil, but wheat and historic artefacts as well." bilal erdogan denies russian allegations bilal erdogan this week denied continuous russian allegations that he and his family were profiting from the illegal smuggling of oil from isis -held territory in syria and iraq, after russia's defence ministry said it had proof that the erdogan family was benefiting directly from this trade. document: wikileaks says it has access to nsa source code and will be posting it for free, unlike an outfit known as the "shadow brokers" that had announced plans to auction off code they allegedly stole. univision won the bidding auction for gawker and will buy all seven of the company's media properties for @xmath0135 million (money) event: battle argument: no answer (instrument) question about the event and the argument role generated question: what is being imported from isis-held territory in syria and iraq? generated question: what was the key component in the auction that led to the purchase of all seven of gawker's media properties? generated question: what led to the capture of chataev in the lopota mountain valley in georgia? expected question: how much money did univision bid to win the auction for gawker, acquiring all seven of the company's media properties? expected question: what instrument was used in the battle between police and rebels take place? expected question: what is being imported from isis-held territory in syria and iraq? question about the event question about neither the event nor the argument role by the weakly supervised model. this analysis evaluates the generated questions for their relevance and accuracy in relation to the events and argument roles detailed within the document. overall, we found three different categories after analyzing 100 questions. the first category of questions are targeting both the event and the argument role, while the second category of questions focuses solely on the event. the third category of questions lacks relevance to both the event and the argument role. figure 5 exemplifies the 3 categories using the output from the weakly supervised t5 model. we observe that 73% questions generated are grounded in both the event and the argument role (first category). another 23% questions are grounded in the event of interest (second category) but aim to extract different information than the role of interest. only 4% questions are grounded in neither the event nor argument role of interest (third category). ## conclusions we have presented several question generation strategies for asking questions about the arguments of an event. our approach can be used seamlessly with any corpora; the only requirement is a list of events and the arguments they may take, information that existing corpora come with. combining uncontextualized and contextualized questions obtains the best results. contextualized questions incorporate cues about other arguments of the event of interest as well as other parts of the document, providing our bart-based models with information that is not otherwise available. crucially, our strategies to generate questions do not require manual intervention. additionally, the weakly supervised models to generate questions transfer across corpora (from rams to wikievents; different corpora, events, and arguments roles).