Thinking about how to extract: Energizing LLMs’ emergence capabilities for document-level event argument extraction (2024)

Abstract: There are two key challenges remaining for the document-level event argument extraction (DEAE) tasks: key feature forgetting and crossevent argument confusion. The emergence capability of large language models (LLMs) holds promise for solving the above two challenges. In this paper, we propose a document-level event argument extraction method based on guided summarization and reasoning (EAESR), which leverages the emergence capabilities of LLMs to highlight key event information and to clarify the explicit and implicit association between multiple events. Specifically, we generate document summarization information that shorten the length of the event context while preserving the key event features. In addition, we generate inter-event reasoning information, which helps EAESR make sense of the correlations between events and reduces their dependence on the event context, especially to better cope with the few-shot D-EAE task. Then, we obtain named entity information to enable EAESR to learn argument boundary features to improve the sensitivity of its argument boundary recognition. Eventually, we fused the above features and sentence features to make EAESR have summarizing and reasoning capabilities simultaneously. Extensive experiments on WIKIEVENTS and RAMS have shown that EAESR achieves a new state-of-the-art that outperforms the baseline models by 1.3% F1 and 1.6% F1, respectively, and averages 11% F1 in few-shot settings.

heading: 1 Introduction
text: Event argument extraction (EAE), which is a critical task in Event extraction (EE), can be divided into sentence-level EAE (S-EAE) and documentlevel EAE (D-EAE). As shown in Figure 1, S1 describes an Injury event, S2 describes an Identification event, etc., yet there are no events for S4 and
*Correponding author.
S5 in the document. Most existing D-EAE models (Zhang et al., 2020; Wei et al., 2021) encode the entire document, which results in event-independent information interfering with event feature modeling, making it difficult to accurately identify events. In order to increase the proportion of key event information in the encoded features, some D-EAE models (Ma et al., 2022; Zhang et al., 2023) set a fixed window to encode a portion of the document. Its lack of global features of the document leads to its missed extraction of arguments scattered outside the window. The above two problems can be summarized as key feature forgetting in the D-EAE task.
On the other hand, as in Figure 1, "18:30 on May 14, 2014" is the time of both the Injury event and Identification event. Yet the time of the Confess event requires reasoning to learn that it is the day after the Identification event. Existing D-EAE models (Xu et al., 2021; Wen et al., 2021; Huang et al., 2020) construct the document as a heterogeneous graph so that they can obtain associations
5520
between tokens in the whole document. They will easily identify the explicit association of the cooccurrence of the argument "18:30 on May 14, 2014", while it is difficult to reason about the implicit association of "The following day", which may cause the D-EAE model to incorrectly extract the time of the Confess event as "18:30 on May 14, 2014". This phenomenon can be referred to as cross-event argument confusion in the D-EAE task.
Recently, large language models (LLMs), like InstructGPT (Ouyang et al., 2022), ChatGPT1, and ChatGLM (Du et al., 2021), utilize the instructions to work well in various downstream tasks such as conversations, summarization generation, etc. Some works (Gao et al., 2023; Wei et al., 2023) have also tried to design reasonable prompts and utilize ChatGPT for event extraction with encouraging results. However, using LLMs directly for argument generation may cause serious precision problems due to the illusions of LLMs (Xu et al., 2024) that can result in generating arguments outside of context. Although it is not appropriate to apply LLMs directly for extracting arguments, we believe that the emergence capabilities of LLMs hold promise for D-EAE models to model complex implicit associations in events, especially if they are used for event association analysis rather than argument generation.
Inspired by this idea, we propose a documentlevel Event Argument Extraction method based on guided Summarization and Reasoning (EAESR), which utilizes LLMs to generate document summarization information and guidance descriptions for argument extraction as external supplementary features to address the key feature forgetting and cross-event argument confusion challenges of the D-EAE task. Specifically, for the key feature forgetting challenge, we first design the prompt for document summarization information generation to streamline the content of the document. As the structure between triggers and arguments in the event has a strong similarity with the structure between nodes and edges in the graph. We establish an abstract meaning representation (AMR) graph for the document summarization information connecting the tokens in the summarization information. Next, we use graph convolutional networks (GCN) to learn the weights of the edges in the AMR graph, not only to obtain the global features
1https://openai.com/blog/chatgpt
of the document but also to establish co-occurrence associations of the event elements.
For the cross-event argument confusion challenge, we design the prompt for reasoning information generation to analyze the explicit and implicit associations of inter-event arguments. We use LLMs to do a step-by-step analysis of the document’s event associations and obtain the event reasoning features. Through changing the learning objective from event features to event reasoning features, EAESR can accurately extract event arguments without relying on a large amount of training data. Since most arguments in an event are named entities, we design prompts to extract entities from sentences to build entity features in order to improve the sensitivity of EAESR for recognizing argument boundaries. Finally, we use the attention fusion layer combine the sentence features with the aforementioned features and obtain the event record. In summary, our proposed innovations and contributions are as follows: (1) We propose a novel D-EAE model, called EAESR. It utilizes the emergent summarization and reasoning capabilities of the LLMs to efficiently address key feature forgetting and cross-event argument confusion for the D-EAE task. (2) We change the learning objective of the D-EAE task from event features to event reasoning features that reduce EAESR’s dependence on large labeled data and improve its performance on the few-shot D-EAE task. (3) Extensive experiment results on WIKIEVENTS and RAMS show that EAESR achieves a new state-of-the-art that outperforms the baseline models by 1.3% F1 and 1.6% F1, respectively, and averages 11% F1 in few-shot settings.

heading: 2 The Proposed EAESR Method
text: As shown in Figure 2, EAESR contains three core components: Summarizing feature extraction extracts the event key information in the document through LLMs. This allows EAESR to obtain the global semantics of the document while shortening the encoding length. Reasoning feature extraction extracts the event argument extraction guidance description and entities through LLMs. This help EAESR to understand the explicit and implicit association of inter-event arguments and boundary features of the arguments. Event argument extrac-
tion fuses the sentence features with the above features through the attention fusion layer, so that EAESR can address both of key feature forgetting and cross-event argument confusion challenges.

heading: 2.1 Task formulation
text: We formulate the D-EAE task as a multiple-span boundary prediction task for the roles on dataset D. Given an instance (C, t, e, A(e)) ∈ D, where C = {si} denote the document, and si is the sentences in the document. t and e are the trigger and event type. A(e) = {(ri, spani), ...} denotes the set of event-specific role types, where ri denotes the role, and spani is the offset of the argument. The inputs to the event argument extraction module are sentence features fs, global features fg, reasoning features fr and entity features fe. The target output is an event record A
(e) Pred = {(ri, spani), ...}, which includes the roles and the model’s predicted arguments for those roles.

heading: 2.2 Summarizing feature extraction
text: Extracting summarization information from a document can filter out context that is not relevant to the key event information, such as adjectives, conjunc-
tions, and extra descriptive phrases. We first utilize the Summarizing information generation module to generate the document summarization information. Second, considering that AMR’s ability to build richer semantic associations from the summarization information, we utilize the AMR graph construction module to build an AMR graph for this summarization information. Finally, we use the global feature generation module to convert the AMR graph into a heterogeneous graph based on edge types. GCN is used to learn the weights of edges to update the degree of association between nodes in the AMR graph, and ultimately to obtain global features.
Summarizing information generation module: given the original document, we design the prompt (SP) for summarizing information extraction, which consists of three parts: event type, maximum length of the summary L, and document. For example, "summarize the text related to <e> from the following document, with a word limit of <L>: "<C>"". Based on the text summarization capability of LLMs, we input SP into ChatGPT to get the document summarization information. Appendix B shows the details.
AMR graph construction module: we use the
AMR parser2 to obtain the initial AMR graph of the summarizing information. The input of the AMR parser is the summarized content, and the output is a directed graph, where each node u denotes a semantic concept, e.g., city, name, etc., and each edge e describes the categorical semantic relationship between two concepts, e.g., location, time, etc. In the initial AMR graph, nodes with the same edge type have a higher probability of being relevant arguments. We follow previous works (Zhang and Ji, 2021; Xu et al., 2022) to further cluster the relevant edge types in the initial AMR graph into 8 major categories, including spatial, temporal, means, modifiers, operators, prepositions, core roles, and others, to get the final AMR graph Gs for extracting significant information from the document.
Global feature generation module: we define the AMR graph as a heterogeneous graph based on edge types, and the node embeddings in the heterogeneous graph are initialized using BERT. Then, we use L-layer GCN to learn the interaction weights between the nodes, calculated as shown in Eq (1):
h(l+1)u = σ( ∑
k∈K
∑
v∈N(u)k ∪{u}
1
cu,k W
(l) k h (l) u ), (1)
where σ is the ReLU function, K is the number of edges of the heterogeneous graph, in this paper K = 8, W (l)k is the trainable parameters, N (u) k denotes the neighbor of node u connected in the k-th edge, cu,k is a normalization constant, h (l) u is the embeddings of node u. We then use a linear layer ln to further transform H(L)u into the global feature fg, as shown in Eq (2) below:
fg = ln[H (L) u ], (2)
where H(L)u ∈ RN×H denotes the set of all nodes in the graph. N is the number of node, H is the hidden state. L is the number of GCN layers, and it is set to 3 in this study. fg ∈ RS×H , and S is the sequence length.

heading: 2.3 Reasoning feature extraction
text: Designing prompts based on downstream task requirements will purposefully improve the performance of downstream tasks, such as code prompts (Wang et al., 2022) for code generation tasks. One effective approach is that supplementing the model
2https://github.com/IBM/transition-amr-parser
with more intermediate processes, such as chain-ofthought (Wei et al., 2022), can improve the model’s performance on downstream tasks compared to a prompt with only inputs and outputs. Based on this consideration, we supplement the D-EAE task with more reasoning features and entity features. For reasoning feature construction, we design prompts (RP) such as,""<C>", in this document there are some <e> events, how do you find out the <ri> in the <ei> event? ... Please think step by step". Based on the reasoning capability of the LLMs, we input the RP into the ChatGPT to get the reasoning information Fr, which consists the event argument extraction steps, the analysis of associations between arguments, and preliminary argument extraction conclusions. Appendix B shows the details. Through the representation of explicit and implicit associations between arguments as a statement, it is able to guide the model to extract the arguments of the given event type step-by-step.
For entity feature construction, we design prompts (EP) such as, "You are an expert in the field of entity extraction, and now you are required to extract all the entities from the following sentence, "<si>"". LLMs have a large amount of general knowledge that can effectively recognize common entities in text, such as time, people, location, etc. We input EP into ChatGPT to get entity information Fe. Then, we use BERT to encode Fr and Fe to get reasoning features fr ∈ RS×H and entity features fe ∈ RS×H , respectively, as in Eq (3):
fr,e = BERT(Fr,e). (3)

heading: 2.4 Event argument extraction
text: We use BERT to encode si to get sentence features fs. Then, we use the attention fusion layer to fuse the sentence features with the above features to get fusion features H . Next, we define a simple but effective argument boundary prediction method, which given the roles and fusion features, EAESR recognizes the offset of the argument span and obtains the event record.
Attention fusion layer: sentence features contain direct features of events; global features, reasoning features, and entity features are used as external features to supplement sentence features for event argument extraction. We first fuse sentence features and global features to get F gcln, which enables EAESR to learn the global semantics of documents, and then we fuse sentence features and reasoning features to get F rcln, which enables EAESR
to learn associations between event arguments as shown in Eqs (4)-(6):
gamma = ln(fs) + g, (4)
beta = ln(fs) + b, (5)
F [r,g] cln = gamma× (f[r,g] −mean(f[r,g])) std(f[r,g]) +beta,
(6) where ln is a linear layer, g and b are the trainable parameters, mean(·) is the mean matrix of f[r,g] and std(·) is the standard deviation matrix of f[r,g].
In order to make EAESR learn both global and reasoning features and to improve the its sensitivity in recognizing the argument boundaries. We fuse global, reasoning, and entity features using a multihead attention mechanism as shown in Eqs (7)-(10) and Figure 3. Entity features have a greater impact on the boundary recognition of the argument span, so make it as V in the attention mechanism directly multiplied with the fusion result of Q and K. fs, F rcln, F g cln, and fe are fused to get the in-depth feature F rgsa . Inspired by the idea of residuals (He et al., 2016), we concatenate shallow features F gcln with in-depth features F rgsa and feed them into the LayerNorm(·) module and the GeLU function to obtain the final output H .
SA(Q,K, V ) = Softmax( QKT√
dk )V, (7)
F ssa = SA(F g cln, fs, fe), (8)
F rgsa = SA(F s sa, F r cln, fe), (9)
H = GeLU[LayerNorm(F rgsa + F g cln)]. (10)
Argument boundary prediction: given the feature representation H of the event and the set of roles ri, we use Eq (11) to compute the probability that each token in the sentence is selected as the start/end position of the argument span for each role. We then define the loss function as Eqs (12)-(13), and finally get the complete event record A
(e) Pred.
ps,ek = Sigmoid(H), (11)
Ls/e = − K∑
k=0
(1− p̂s/ek ) log(1− p s/e k )
+ p̂ s/e k log(p s/e k ),
(12)
L = Ls + Le, (13)
where psk and p e k are the probabilities of the token in the sentence as the start position and end position
of the argument, and p̂sk and p̂ e k are the labels of the start position and end position of the argument, respectively.

heading: 3 Experiment
text: Datasets We evaluate EAESR on two popular DEAE datasets: RAMS (Ebner et al., 2019) and WIKIEVENTS (Li et al., 2021). RAMS has 139 event types and 65 argument roles, and the average document length in the test set is 134 words. WIKIEVENTS has 50 event types and 59 argument roles, and the average document length in the test set is 789 words. Appendix A shows detailed statistics.
Baselines We compare our model with the following state-of-the-art baseline models: (1) D-EAE methods based on the span boundary prediction model: FEAE (Wei et al., 2021), BERT-CRF, TSAR (Xu et al., 2022). (2) D-EAE methods based on QA/MRC models: EEQA (Du and Cardie, 2020), EEQA-BART, DocMRC (Liu et al., 2021). (3) D-EAE methods based on generative models: BART-Gen (Li et al., 2021), PAIE (Ma et al., 2022), UnifiedEAE (Zhou et al., 2022), MemoryDocIE (Du et al., 2022), APE (Zhang et al., 2023), RA-DocEAE (Ren et al., 2023). Appendix C describes the above baseline model in detail.
Evaluation metric Following Ma et al. (2022), we adopt two evaluation metrics. (1) Argument Identification F1 score (Arg-I): an argument span is correctly identified when the predicted offset fits the ground truth span. (2) Argument Classification F1 score (Arg-C): both the span and the argument role type are matched with the ground truth.

heading: 4 Main results

heading: 4.1 Overall performance
text: Tabel 1 compares EAESR with the baseline models. We observe that EAESR performs best on RAMS and WIKIVENTS, which obtained +1.6% and +1.3% gains in F1 (Arg-C), respectively, that can prove EAESR is effective in both long-document D-EAE tasks (WIKIEVENTS) and short-document D-EAE tasks (RAMS). We also realized that TSAR achieved the second-best results on WIKIEVENTS yet not on RAMS, suggesting that AMR is more capable of modeling long content features. APE achieved the second-best results in RAMS and competitive results in WIKIEVENTS, indicating that overlapping knowledge between datasets plays an important role in improving the generalization performance of the D-EAE task.

heading: 4.2 Detailed analysis

heading: 4.2.1 Performance of LLMs on D-EAE tasks
text: In this section, we compare the performance of directly using LLMs (ChatGPT3.5 and ChatGLM2-
6B) and EAESR for event argument extraction in RAMS and WIKIEVENTS. For LLMs, we design the event argument extraction prompt as "You are an expert in the field of event extraction. Now you are required to extract the argument: <ri> from following sentence: <si>. Only output in the following format as <r1> is ’a word from the sentence’, ... without outputting other words or analysis". And as the parameters of ChatGLM2-6B are much less than those of ChatGPT3.5, we converted the EAE task to a dialog task and make instruction-tuning for ChatGLM2-6B. Table 2 shows the comparison results of LLMs and EAESR, in which it can be shown that EAESR is 7.4% F1 (Arg-C) and 9% F1 (Arg-C) higher than ChatGLM2-6B on RAMS and WIKIEVENTS, respectively, which suggests that there are serious precision issues caused by the direct use of LLMs to generate arguments that outside of context. ChatGLM2-6B outperforms ChatGPT3.5 by 5.4% F1 (Arg-C) and 18% F1 (Arg-C) on RAMS and WIKIEVENTS, respectively, suggesting that injecting task-specific knowledge into LLMs is more beneficial for the D-EAE task than the general knowledge owned by the LLMs them-
selves.

heading: 4.2.2 Impact of prompt on D-EAE tasks
text: In this section, we compare the impact of providing and not providing event-related information for prompt in the process of generating external supplementary information by using ChatGPT. For SP, the prompt (SP- with-events) that provides eventrelated information is the prompt used in this paper. The prompt that does not provide event-related information (SP-without-events) is as follows: "summarize the following document, with a word limit of <L>: "<C>".". For RP, the prompt (RP-withevents) that provides event-related information is the prompt used in this paper. The prompt that does not provide event-related information (RP-withoutevents) is as follows: "Analyze the main content of the document. Please think step by step.". Table 3 shows the results for different prompt settings, and we find that Prompt-with-events contributes more to EAESR than Prompt-without-events, suggesting that it is essential to provide prompts with appropriate event information for both generating document summarization information and event reasoning information.

heading: 4.2.3 Performance in few-shot settings
text: EAESR learns how to extract event arguments by learning event reasoning features. Its application of transfer learning to obtain event-overlapping knowledge further expands the knowledge accumulation. Thus, EAESR can be trained with only a few samples to achieve competitive results of some baseline models trained using all samples. In this section, we utilize RAMS to validate the performance of EAESR in a few-shot scenario. As shown in Figure 4, we find that when there are only 10, 50, 100, and 200 random event records in the training set, EAESR has an improvement of 17.8%F1, 11.8%F1, 8%F1, and 6.4%F1 compared to APE, respectively. Furthermore, EAESR requires only 10 random event records to exceed the training effect that APE requires 200 random event records to achieve, indicating that it can greatly reduce the labor cost associated with labeling training data.

heading: 4.3 Ablation study
text: In this section, we investigate the effectiveness of EAESR by removing each external supplementary feature in turn. (1) Global features: we replace F gcln with fs. (2) Reasoning features: we replace F scln with fs. (3) Entity features: we replace fe in
the SA(·) with fs. We summarize the results of ablation studies in Table 4 as follows: 1) After removing the global features, EAESR decreased by 0.3% F1 (Arg-C) and 1.5% F1 (Arg-C) on the RAMS and WIKIEVENTS, respectively. This suggests that providing EAESR with a global feature can be effective in complementing the current sentence semantic features with semantic information from other sentences, and that the effect is more pronounced as the document get longer. 2) After removing the reasoning features, EAESR decreased by 1.3% F1 (Arg-C) and 1.3% F1 (Arg-C) on the RAMS and WIKIEVENTS, respectively. This suggests that the reasoning information of events enhances the ability of EAESR to learn associations between events and that it is not sensitive to document length. 3) After removing the entity features, EAESR decreased by 0.9% F1 (Arg-C) and 0.5% F1 (Arg-C) on the RAMS and WIKIEVENTS, respectively. This suggests that entity features can serve to constrain EAESR in recognizing the boundaries of the arguments.

heading: 4.4 Case study
text: To visually demonstrate the benefits of using external supplementary information in our approach, we show two examples comparing the output of ChatGLM2-6B, APE and EAESR as shown in Figure 5 in Appendix. Example A describes the Conflict.Attack.DetonateExplode event, and we find
that ChatGLM2-6B tends to generate semantically rich arguments, which leads to over-extraction problems and generating arguments outside the original context. For instance, the word "school" is not included in the original context, but ChatGLM26B fails to adhere to the instructions and generates words outside the original context. We statistically analyze the extraction results of ChatGLM2-6B and EAESR in WIKIEVENTS, where the precision (Arg-C) of EAESR is 72.90%, while the precision (Arg-C) of ChatGLM2-6B is 62.87%. In ChatGLM2-6B’s extraction results, the loss of precision due to parsing errors was 1.70%, and the loss of precision due to hallucinations was 15.23%, which shows that the hallucinations of LLM have a huge impact on the EAE task. APE tends to extract the core words of an argument, leading to its missed extraction, e.g., Target’s argument is missing "Afghan". EAESR mitigates the problem of over- or under-extraction due to its use of entity features to constrain the boundaries of recognized arguments, and avoids extract arguments outside of the original context.
Example B describes the Life.Die.Unspecified event, in which the Reason’s argument is not in the same sentence as the other arguments. And it is necessary to use "woman" to relate the events in the two sentences in order to infer that "suicide" in the second sentence is the Reason of the Life.Die.Unspecified event. Because EAESR uses event reasoning features and document global features to provide implicit associations between events and global semantics of documents, it extracts "suicide", while the other two models do not.

heading: 5 Related works
text: Document-level Event Argument Extraction The goal of the D-EAE task is to extract event argu-
ments from the given triggers and roles. The methods of the D-EAE task can be divided into D-EAE based on span boundary prediction, QA-based models, and generative models. The D-EAE method based on span boundary prediction (Zhang et al., 2020; Dai et al., 2022; Yang et al., 2023; He et al., 2023) is to consider the D-EAE task as a classification task. In order to make the D-EAE model more focused on the associations between events in a document, some work (Liu et al., 2020; Chen et al., 2020; Liu et al., 2021) uses Question Answering (QA)/Machine Reading Comprehension (MRC) to understand the document semantics before extracting the event arguments. The D-EAE method based on generative models (Lu et al., 2021; Li et al., 2021; Hsu et al., 2022; Ren et al., 2023; Lin et al., 2023) to designing diverse prompts makes the DEAE task more relevant to the text generation task. This method allows the event’s arguments to be directly generated in the sequence-to-structure manner, however, some restriction of the generation (Lu et al., 2021) need to be taken to avoid the model generating words that are not within the event’s definition.
LLMs in Event Extraction LLMs have excellent emergence capabilities, and they can achieve impressive performance on a wide range of downstream tasks, such as ChatLaw (Cui et al., 2023), MuseChat (Dong et al., 2023), ChatReviewing (Berrezueta-Guzman et al., 2023), and so on. Some work has also attempted to apply LLMs to event extraction tasks. Wei et al. (2023) attempt to convert a zero-shot information extraction task into a multi-round QA task using ChatGPT. It achieves promising performance on the zero-shot information extraction task and even outperforms the fullshot model on some datasets (e.g., NYT11-HRL (Takanobu et al., 2019)). Gao et al. (2023) tests the
performance of ChatGPT on ACE2005 (Doddington et al., 2004) and show that due to ChatGPT’s lack of event specific knowledge, it is only 51.04% as effective as task-specific models (e.g., EEQA (Du and Cardie, 2020)) in long-tailed and complex scenarios. In summary, LLMs are far better at text comprehension than they are at event extraction, and how to exploit the potential of LLMs for event extraction tasks still needs to be thoroughly investigated.

heading: 6 Conclusion and future works
text: In this work, we propose a novel model of EAESR that can resolve key feature forgetting and crossevent argument confusion simultaneously. We utilize LLMs to generate external supplementary features related to events, including document global features, event reasoning features, and entity features. The document global features can provide EAESR with a global perspective and help it solve key feature forgetting. The event reasoning features and entity features can provide EAESR with explicit and implicit associations between events and help it solve cross-event argument confusion. Extensive experiments on RAMS and WIKIEVENTS demonstrate the effectiveness of our proposed model in the D-EAE task. In future work, we will explore the use of LLMs to generate supplemental features for other information extraction tasks, such as event reasoning features for event relation extraction tasks.
Limitations
Our goal is to utilize the emergence capabilities of LLMs to improve the performance of the D-EAE task, due to their large number of parameters, leading to the fact that inference on LLMs will be timeconsuming. As an example, generating document summarization information, event reasoning information, and entity information for WIKIEVENTS will take 30h on the NVIDIA A40 48GB GPU. This limitation is expected to be alleviated by the adoption of a lightweight generative model. In addition, EAESR is based on AMR graphs generated by a pre-trained AMR parser. The AMR graph of the generated document summarization inevitably has a certain possibility of imperfection, which leads to error propagation. In future work, applying LLMs to construct a joint extraction model for document global information will likely avoid error propagation.
Ethics Statement
Our work complies with the ACL Ethics Policy. The document level event argument extraction (DEAE) task is a well-defined classical task in the field of event extraction (EE). In this work, our use of existing datasets is licensed and consistent with their intended use. We see no other ethical issues.

heading: Acknowledgements
text: This work was supported by Beijing Natural Science Foundation (Grant No.4222032) and the Foundation for Innovative Research Groups of the National Natural Science Foundation of China (Grant No.61921003).

heading: A Datasets
text: We present detailed dataset statistics in Table 5.

heading: B Summarization and reasoning information
text: We present detailed summarization and reasoning information in Table 6.

heading: C Baselines
text: This section supplements the baseline models used in this paper.
D-EAE methods based on the span boundary prediction model: FEAE (Wei et al., 2021) trained a teacher model for implicit EAE by introducing a course knowledge extraction strategy. BERT-CRF defines the D-EAE task as a sequence labeling task. TSAR (Xu et al., 2022) is the first and sole work utilizing AMR for D-EAE.
D-EAE methods based on QA/MRC models: EEQA (Du and Cardie, 2020) defines the EAE task as an end-to-end question-answering (QA) task. EEQA-BART replaces BERT with BART for event extraction. DocMRC (Liu et al., 2021) regards the EAE task as a document reading comprehension (MRC) task.
D-EAE methods based on generative models: BART-Gen (Li et al., 2021) proposes a conditional generation approach to complete the D-EAE task. PAIE (Ma et al., 2022) utilizes multi-role prompts
under extractive settings to capture argument interactions. UnifiedEAE (Zhou et al., 2022) explores shared knowledge in different event extraction datasets using transfer learning. MemoryDocIE (Du et al., 2022) constructs a document memory store to record contextual event information. APE (Zhang et al., 2023) defines overlapping knowledge between EAE datasets and combines specific knowledge for event argument extraction. RA-DocEAE (Ren et al., 2023) validates the effectiveness of the data retrieval augmentation approach on the D-EAE task.
D Implementation details
We train the D-EAE task of RAMS by loading the pre-trained BERT parameters of WIKIEVENTS and train the D-EAE task of WIKIEVENTS by loading the pre-trained BERT parameters of RAMS. We use the base version of the pre-trained model for all models, like BERT-base, BART-base, and LLMs, which use ChatGPT3.5 and ChatGLM2-6B. We train models on NVIDIA-A40 by AdamW with a 0.1 warmup ratio and 0.01 weight decay. We set the initial learning rate to 1e-5, the batch size for training to 2, and the number of training epochs to 30.

heading: Future Work
text: In future work, we will explore the use of LLMs to generate supplemental features for other information extraction tasks, such as event reasoning features for event relation extraction tasks. Limitations
Our goal is to utilize the emergence capabilities of LLMs to improve the performance of the D-EAE task, due to their large number of parameters, leading to the fact that inference on LLMs will be timeconsuming. As an example, generating document summarization information, event reasoning information, and entity information for WIKIEVENTS will take 30h on the NVIDIA A40 48GB GPU. This limitation is expected to be alleviated by the adoption of a lightweight generative model. In addition, EAESR is based on AMR graphs generated by a pre-trained AMR parser. The AMR graph of the generated document summarization inevitably has a certain possibility of imperfection, which leads to error propagation. In future work, applying LLMs to construct a joint extraction model for document global information will likely avoid error propagation