Beyond Single-Event Extraction: Towards Efficient Document-Level Multi-Event Argument Extraction (2024)

Abstract: Recent mainstream event argument extraction methods process each event in isolation, resulting in inefficient inference and ignoring the correlations among multiple events. To address these limitations, here we propose a multiple-event argument extraction model DEEIA (Dependency-guided Encoding and Event-specific Information Aggregation), capable of extracting arguments from all events within a document simultaneously. The proposed DEEIA model employs a multi-event prompt mechanism, comprising DE and EIA modules. The DE module is designed to improve the correlation between prompts and their corresponding event contexts, whereas the EIA module provides event-specific information to improve contextual understanding. Extensive experiments show that our method achieves new state-of-the-art performance on four public datasets (RAMS, WikiEvents, MLEE, and ACE05), while significantly saving the inference time compared to the baselines. Further analyses demonstrate the effectiveness of the proposed modules. Our implementation is available at https://github.com/ LWL-cpu/DEEIA.

heading: 1 Introduction
text: Document-level event argument extraction (EAE) is a key process within Information Extraction (Hobbs, 2010; Grishman, 2015; Xia et al., 2022), focused on identifying event-related arguments and their respective roles in document-level texts. Recently, the leading-edge methods for this task delve into prompt-based techniques (Ma et al., 2022; Hsu et al., 2023), due to their great generalizability and competitive performance. Figure 1 (a) presents an example demonstrating EAE utilizing a prompt-based approach. Regarding the event e0 triggered by “bombarding”, the approach defines a corresponding event prompt and identifies arguments: “government” as the killer, “a number of
∗Corresponding author
areas” as the victim, and “shelling” as the instrument.
Mainstream EAE works (Zhou et al., 2024; Liu et al., 2023c; Ren et al., 2023; Liu et al., 2023a) can only process one event at a time. When encountering documents containing multiple events, the limitations of these single-event argument extraction (Single-EAE) methods become evident. (1) As shown in Figure 1 (b), for a document containing multiple events, Single-EAE methods have to perform numerous iterations to extract event arguments for all events, which process the same document text repeatedly, leading to inefficient extraction. (2) Single-EAE methods fail to capture the beneficial event correlations among multiple events (He et al., 2023; Zeng et al., 2022b). Figure 1 (b) illustrates the argument overlapping phenomenon which reflects the semantic correlations among events. The event Life.death and event
9470
Life.surrender share the argument “government” and there exists a strong event correlation between these two events. However, Single-EAE methods cannot utilize such correlations.
To tackle these limitations, this paper proposes a DEEIA (Dependency-guided Encoding and Event-specific Information Aggregation) model, a multiple-event argument extraction (Multi-EAE) method capable of simultaneously extracting arguments for all events within the document. We construct our DEEIA model based on the state-ofthe-art (SOTA) prompt-based Single-EAE model PAIE (Ma et al., 2022) and introduce a multi-event prompt mechanism to enable extracting arguments from multiple events simultaneously.
However, our Multi-EAE model faces the challenge of handling more complex information as it needs to simultaneously process different triggers, arguments roles, and prompts from multiple events. This requires the model with enhanced information extraction capabilities. Therefore, we design a Dependency-guided Encoding (DE) module to guide the model in correlating the various prompts with their respective event contexts. Furthermore, we propose an Event-specific Information Aggregation (EIA) module to provide event-specific contextual information for a better context understanding.
Figure 2 demonstrates that with the increase in the number of events within a document, the efficiency advantage of our DEEIA model becomes increasingly apparent. The performance surpassing Single-EAE baselines also demonstrates that our model effectively captures event correlations. The contributions of this paper are summarized as follows:
• We propose a multi-event argument extraction (Multi-EAE) method, capable of extracting the arguments of multiple events simultaneously, with the aim of improving the efficiency and performance of the EAE task.
• To tackle the challenges of Multi-EAE, we propose a Dependency-guided Encoding (DE) module and an Event-specific Information Aggregation (EIA) module, which provide dependency guidance and event-specific context information, respectively.
• Extensive experiments demonstrate that the proposed DEEIA model outperforms major benchmarks in terms of both performance and inference time. We provide comprehensive ablation studies and analyses.

heading: 2 Related Work
text: Recently, there has been an increasing interest in the task of document-level Event Argument Extraction (Wang et al., 2022; Liu et al., 2023b; Yang et al., 2023), a crucial component within the domain of Event Extraction (Ren et al., 2022; Yang et al., 2021). Current methods for documentlevel EAE can be classified into four main categories: (1) Span-based methods, which identify candidate spans and subsequently predict their roles (Zhang et al., 2020; Yang et al., 2023; Liu et al., 2017; Zhang et al., 2020; Liu et al., 2023c). (2) Generation-based methods (Li et al., 2021; Du et al., 2021; Wei et al., 2021; Huang et al., 2023) utilizing generative PLMs, such as BART (Lewis et al., 2019), to sequentially produce all arguments for the designated event. (3) Prompt-based methods (Ma et al., 2022; He et al., 2023; Nguyen, 2023; Wang et al., 2024), which use slotted prompts and leverage a generative slot-filling approach for argument extraction. (4) Large language model methods. Recently, some work (Zhang et al., 2024b; Zhou et al., 2023a) has attempted to explore to utilize large language models for EAE tasks, but the performance falls short of expectations. And the time and cost of inference are relatively high. Among them, prompt-based methods have been demonstrated superior generalizability and competitive performance (Hsu et al., 2023).
However, these EAE methods are Single-EAE methods, which can only process one event at a time. Recently, prompt-based EAE method, TabEAE (He et al., 2023) aims to capture event
co-occurrence (Zeng et al., 2022b) and trains the model on multi-event scheme. However, TabEAE requires separately processing prompts for different events, which is highly time-consuming. In this paper, we propose to extract event arguments concurrently, which significantly improves the efficiency of the EAE task in multi-event documents.

heading: 3 Methodology
text: In this section, we first provide a formal definition of multi-EAE task. Given an instance( X, {ei}Ki=1, {ti}Ki=1, {R(ei)}Ki=1 ) , where X = (w0, w2, . . . , wN−1) represents the document text with N words, K is the number of target events, and ei is the type of the i-th event. ti ⊆ X is the trigger word of the i-th event, and R(ei) indicates the set of roles associated with event ei. The task aims to extract a set of span Si for each event ei, which satisfies ∀a(r) ∈ Si, (a(r) ⊆ X) ∧ (r ∈ R(ei)). Most previous EAE methods are designed as Single-EAE methods, applicable when K = 1.
We then provide a detailed description of our DEEIA model, as shown in Figure 3. We first propose a multi-event prompt mechanism to enable extracting arguments from multiple events simultaneously (§ 3.1). Then we propose a Dependencyguided Encoding module (§ 3.2) and an Eventspecific Information Aggregation module (§ 3.3) to tackle the complexity and challenge of simultaneously extracting arguments of multiple events.

heading: 3.1 Multi-event Prompt Mechanism
text: Multi-event document instances involve the prompts of multiple events. Therefore, we propose
a multi-event prompt mechanism to enable extracting arguments from multiple events simultaneously. We first enhance the input text and event prompts and then concatenate them as the final input.
Preprocessed Text. Given an input text with a set of event triggers, each trigger is initially annotated with a unique pair of markers (⟨ti⟩ , ⟨/ti⟩), where i counts the order of occurrence. Then we tokenize the marked text into X , where ti is the i-th trigger:
X̂ = w0 ⟨t0⟩ t0 ⟨/t0⟩ ... ⟨ti⟩ ti ⟨/ti⟩ ...wN . (1)
Prompt Enhancement. We concatenate the prompt of each event and obtain the final prompt P .1 For each event prompt Pi, we utilize the eventschema prompts proposed by PAIE (Ma et al., 2022). We append the event type to the start of each corresponding prompt. This strategy helps the model distinguish different event prompts and integrates event type information, enriching the EAE process with more comprehensive information. Specifically, we wrap each event type with a unique pair of markers (⟨ei⟩ , ⟨/ei⟩) and obtain the final P as follows:

heading: P = ⟨e0⟩we00 ... ⟨/e0⟩P0... ⟨ei⟩wei0 ... ⟨/ei⟩Pi,
text: (2) where Pi is the prompt of the i-th event and w ei 0 is the first token of event type ei. Then we concatenate X̂ and P and put them into our dependency-guided encoder (in § 3.2.2). For
1If multiple events are of the same type, we retain only one prompt and use it to predict arguments of all such events.
input sequences that exceed the maximum length2, we employ a dynamic window (Zhou et al., 2021) technique to solve the problem, where the detailed algorithm is shown in .

heading: 3.2 Dependency-guided Encoding Module

heading: 3.2.1 Event Dependency Definition
text: Due to the fact that different events are associated with distinct trigger words, argument roles, and prompts, our model faces the challenge of information complexity (Bagga and Biermann, 1997; Li et al., 2023a) when processing multiple events simultaneously. Therefore, we propose to guide the model to associate the multi-event prompts with their corresponding event contexts with pre-defined dependencies. Considering that arguments can exhibit inter-event and intra-event relations within a context, we define the following two types of dependencies among triggers and prompts (including argument slots), which help to solve the information complexity problem of multiple events.
Intra-Event Dependency. (1) The connection between the trigger and prompt tokens within the same event. (2) The connection between prompt tokens within the same event prompt.
Inter-Event Dependency. (1) The connection between the trigger and prompt tokens of different events. (2) The connection between prompt tokens of different events.
These two dependencies not only reflect intra and inter-event relations within a multi-event context, but also establish interactions among triggers and argument slots, both within and across events.
Formally, for the input sequence S = (x1, x2, ..., xn) including triggers and prompts from multiple events, we introduce D = {dpij} to represent such two dependencies, where i, j ∈ {0, 1, ..., n} and dpij ∈ {Intra-event, Inter-event, NA} is a discrete variable denotes the dependency from xi to xj . NA indicates there is no dependency between xi and xj . Then the pre-defined event dependencies D will be integrated into the transformer to provide information guidance.

heading: 3.2.2 Dependency-guided Encoder
text: We improve vanilla self-attention mechanism (Vaswani et al., 2017) by adding a learnable
2The length of event prompts is much shorter than that of the text, and only a very small number of docs in Wikievents and MLEE datasets exceed the length limit.
attention bias, which integrates the event dependencies into the transformer.
For the input representation xi ∈ Rd, it is first projected into query/key/value vector: qi = xiWQ,ki = xiWK , vi = xiWV . Then a learnable bias is added to the vanilla self-attention mechanism, which helps the model perceive dependency relations among multiple events. The attention score aij is produced as follows:
aij = qik T j√
dk + γ · biasij , (3)
where biasij is the learnable attention bias for the attention between tokens xi and xj . γ is a hyperparameter for adjusting the influence of the bias. dk is the the hidden dimension of each attention head. Specifically, the attention bias depends on the dependency dpij and the context information, with specific parameters trained and then utilized in a compositional manner (Xu et al., 2021). We design the attention bias biasij as follows:
biasij =    0, if dpij is NA qiWdpijk T j +bdpij√
dk , otherwise
(4)
where Wdpij ∈ Rdk×1×dk and bdpij are the trainable parameters corresponding to dependency dpij . The remaining operations are the same as transformer mechanism. Then we apply this mechanism to each layer of the encoder. Note that we do not apply this mechanism to the decoder layers, and a detailed analysis is provided in Appendix C.3.

heading: 3.3 Event-specific Information Aggregation
text: In this section, we design an event-specific information aggregation (EIA) module which adaptively aggregates useful information for specific events. We hope the model can make use of the context information relevant to the specific event and the argument when extracting an argument. Therefore, we consider using triggers (representing the event) and slots (representing the argument) to measure the relevance between the target argument and context (prompt) information.
Specifically, we utilize the attention heads of argument slots and their associated triggers, derived from the pre-trained transformer encoder, to calculate the attention product for the input sequence tokens (including both context and prompt). The dot product of attention is designed to measure the
degree of association between the current event’s argument and every token of input context.
We adopt an encoder-decoder architecture. The encoder is employed to encode the input text, while the decoder is tasked with deriving the eventoriented context and context-oriented prompt representation Hde:
[A;Hen] = Encoders(S),
Hde = Decoder(Hen), (5)
where the Encoders is the dependency-guided encoder and the Decoder is a transformer-based decoder. S is the input of the concatenation of context X and prompt P , and A ∈ RH×ls×ls is the multihead attention matrix and Hen,Hde ∈ Rls×d. H is the attention head numbers and ls is the length of input sequence S.
For the k-th argument slot sk,i to be predicted in the i-th event, we first get the contextual attention vectors Ati ∈ Rls and Ask,i ∈ Rls from A, corresponding to the trigger ti and the slot in the prompt respectively. These vectors are obtained by averaging across all attention heads and associated subtokens3. Then for the argument slot sk,i, we obtain the context-enhanced vector ck,i ∈ Rd which adaptively aggregates useful context and prompt information for argument extraction.
pk = softmax(Ati · Ask,i ), ck,i = HenT pk,
(6)
where pk ∈ Rls is the computed attention weight vector for argument slot sk,i. Then ck,i is subsequently incorporated into the decoder output hsk,i ∈ Rd of slot sk,i to get h̃sk,i ∈ Rd:
h̃sk,i = tanh(W1[hsk,i ; ck,i]), (7)
where W1 ∈ R2d×d is learnable parameter.

heading: 3.4 Span Selection
text: After obtaining the final representation h̃sk,i for each slot within each event, we follow (Ma et al., 2022) and transform each of them into a set of span selector {Φstartsk,i ,Φ end sk,i }:
Φstartsk,i = h̃sk,i ◦wstart, Φendsk,i = h̃sk,i ◦wend,
(8)
where wstart,wend ∈ Rd are learnable parameters and ◦ represents element-wise multiplication.
3We only take the start token ⟨ti⟩ to represent the trigger.
Then Φstartsk,i and Φ end sk,i determine the start and end positions of slot sk,i in the original text:
logitstartk,i = softmax(HdeΦ start sk,i ) ∈ Rls , logitendk,i = softmax(HdeΦ end sk,i
) ∈ Rls , scorek,i(m,n) = logitstartk,i (m) + logit end k,i (n),
(ŝk,i, êk,i) = argmax (m,n)∈C scorek,i(m,n),
(9) where (ŝk,i, êk,i) is the predicted argument span. C = { (m,n) | (m,n) ∈ ls2, 0 < n−m ≤ l } ∪ {(0, 0)} contains all spans not exceeding the threshold l, along with the empty span (0, 0).
Following (Ma et al., 2022), we utilize Bipartite Matching Loss (Carion et al., 2020), which provides further consideration for the assignment of golden argument spans during training.
L = − K∑ i=1 ∑ (ŝk,i,êk,i)∈δ(Si) [log logitstartk,i (ŝk,i)
+ log logitendk,i (êk,i)], (10)
where K is the number of target events and i represents the i-th event. δ(Si) denotes the optimal assignment (Ma et al., 2022) calculated through the Hungarian algorithm (Kuhn, 1955).

heading: 4 Experiments

heading: 4.1 Experimental Setup
text: Datasets We evaluate our model on three document-level EAE datasets, including RAMS (Ebner et al., 2020),WikiEvents (Li et al., 2021) and MLEE (Pyysalo et al., 2012). Moreover, we also extend our evaluation of the model to the sentence-level ACE05 dataset (Doddington et al., 2004), as it includes a significant number of instances with multiple events. The detailed dataset description and statistics are shown in Appendix B.1. Evaluation Metrics Following previous works (Ma et al., 2022; He et al., 2023), we evaluate performance using two metrics: (1) strict argument identification F1 (Arg-I), where a predicted event argument is considered correct if its boundaries match those of any corresponding golden arguments. (2) Strict argument classification F1 (Arg-C), where a predicted event argument is considered correct only if both its boundaries and role type are accurate. We conduct experiments on 5 runs with different seeds and report the average results.
Baselines Our baselines include: (1) Two SOTA span-based methods, TSAR (Xu et al., 2022) and SCPRG (Liu et al., 2023c); (2) Five typical generation-based methods, DocMRC (Liu et al., 2021), EEQA (Du and Cardie, 2020), FEAE (Wei et al., 2021), BART-Gen (Li et al., 2021), and HRA (Ren et al., 2023); (3) Four prompt-based approaches: RKDE (Hu et al., 2023), PAIE, SPEAE (Nguyen, 2023) and TableEAE. We also compare with LLM approaches and conduct a detailed analysis in the Appendix C.5.
Most baselines are originally proposed as SingleEAE methods. For the Multi-EAE baselines, we extend PAIE and TabEAE to obtain PAIE-multi4 and TabEAE-multi5. Our experimental details are shown in Appendix B.2.

heading: 4.2 Main Results
text: Table 1 illustrates the performance comparison between our proposed DEEIA method and various baseline approaches across three datasets. (Experimental results on ACE05 dataset are shown in Appendix B.3.) Our approach consistently achieves optimal results across all datasets generally, irrespective of the evaluation metric employed. Further analyzing the experimental result data, we observe: (1) The performance of DEEIA outper-
4We extend the original PAIE (Ma et al., 2022) into the multi-EAE framework by annotating triggers within the context and concatenating prompts for multiple events. The rest of the approach remains consistent with the original PAIE.
5TabEAE (He et al., 2023) aims to capture event cooccurrence and trains the model on multi-event scheme but infers on single-event scheme. For a fair comparison, we train their method on multi-event instances and conduct inference on multi-event instances as well.
forms that of Single-EAE baselines. This indicates our DEEIA can effectively capture the beneficial event correlations, enhancing the performance of EAE task. (2) PAIE-multi exhibits significantly lower performance compared to PAIE, which illustrates that simultaneously processing multiple events significantly increases the difficulty of the task. While our DEEIA significantly outperforms the baseline PAIE-multi on three datasets, demonstrating that our DEEIA effectively addresses the challenge of multi-event information complexity. (3) Compared to PAIE and TabEAE, the improvement of our DEEIA model on RAMS is around 0.9- 1.2 F1, while on WikiEvents and MLEE, the improvement is around 1.2-1.7 F1 and 1.7-3.7 F1 respectively. Therefore, the improvement of DEEIA on WikiEvents and MLEE is more pronounced compared to RAMS. We hypothesize this may be because WikiEvents and MLEE contain a higher proportion of multi-event instances, in contrast to the RAMS dataset, which is primarily composed of single-event instances. (Distributions of event numbers on three datasets are shown in Appendix B.1.)

heading: 4.3 Ablation Study
text: To better illustrate the effectiveness of different components, we conduct ablation studies on RAMS and WikiEvents datasets in Table 2. We divide the dataset into two parts based on the number of events in each instance: those with # E > 1 and those with # E = 1, and report the Arg-C F1 scores to explore the impact of our modules on instances with single and multiple events.
Without Dependency-guided Encoding (DE).
We replace the Dependency-guided Encoding (DE) module with a vanilla transformer encoder. This results in performance reduction, primarily attributed to the decline in the performance of multi-event samples, which illustrates that DE module effectively provides dependency guidance for multievent extraction. We further explore the effectiveness of intra and inter dependencies. It is observed that both intra and inter dependencies contribute positively to the model. When remaining only one type of dependency, the intra dependency has a beneficial effect on the model, but the inter dependency has a negative effect.
Without Event-specific Information Aggregation (EIA). The performance of both multi-event and single-event samples has significantly declined on two datasets. This indicates that our EIA module can provide beneficial event-specific information. Moreover, when removing both DE and EIA modules, the performance decay exceeds that when removing a single module, which explains that our two modules can work together.
Without Prompt Enhancing (PE). When removing the event type information defined in § 3.1, the performance on two datasets decays slightly, which indicates that event type helps the model distinguish between different event prompts and integrates wider information.

heading: 5 Analysis

heading: 5.1 Efficiency Analysis
text: Table 3 reports the efficiency of different promptbased methods. First, compared to SingleEAE baseline PAIE, our method saves 8.76%, 32.96%, and 35.20% inference time on RAMS, WikiEvents, MLEE datasets respectively. This fully demonstrates the efficiency superiority of our Multi-EAE approach. Additionally, our method
significantly reduces 55.57%, 57.38% and 69.19% inference time compared to TabEAE-multi, with almost no increase in the number of parameters. This illustrates that our DEEIA model enhances the efficiency of the document-level EAE task.

heading: 5.2 Effect Analysis on Event Numbers
text: To further investigate the effectiveness of our method in addressing the multi-event information complexity problem, we divide the documents in the development sets of WikiEvents and MLEE into different groups based on the event numbers6. As illustrated in Figure 4, as the event number increases, we observe a decreasing trend in the performance of all models. We believe this is due to the fact that more events require the model to process more complex information and longer text, which is more difficult. Furthermore, we find that the baseline model PAIE-multi performs significantly worse on samples where the number of events exceeds two. In contrast, our model demonstrates a marked improvement in multi-event samples compared to PAIE-multi and TabEAE-multi, which shows the superiority of DEEIA in capturing the event correlations among multiple events. The results on WikiEvents dataset are in Appendix C.1.
6We do not use the RAMS dataset because the RAMS dataset has a low proportion of multi-event instances.

heading: 5.3 Analysis of Two Modules
text: Dependency Guidance To investigate how the attentive biases influence the self-attention mechanism, we visualize all attentive biases (calculated in Eq. 4) for the test sets of all three datasets. We conducted a detailed analysis in Appendix C.2. We find that both inter and intra dependencies can provide effective information guidance for Multi-EAE task. In datasets with a larger proportion of multievent documents, such as the MLEE dataset, both inter and intra dependencies exhibit significant effects, while in datasets with a smaller proportion of multi-event documents, such as RAMS, intra dependency plays a primary role.
Analysis of Information Aggregation To assess the effectiveness of our EIA module in capturing event-specific contextual information, we visualize the attentive weights pk in Eq. 6 of an argument “government” of Figure 1. As shown in Figure 5, our EIA module gives high weights to the context words, such as starving, shooting and surrender, prompt words such as die, killer and injurer, which benefits the argument extraction of “government”. Interestingly, some words such as surrender and shelling also act as triggers or arguments in other events, which reveals the EIA module’s capability to capture event correlations.

heading: 5.4 Error Analysis and Case Study
text: Error Analysis We further conduct error analysis to explore the effectiveness of our DEEIA model in Appendix C.4. We analyze all prediction errors on WikiEvents test set and categorize them into five classes. As shown in Figure 10, compared to the Single-EAE baseline PAIE, our DEEIA model reduces the number of errors from 312 to 292, indicating the effectiveness of DEEIA
in capturing event correlations. Compared to PAIEmulti, DEEIA reduces the number of errors from 358 to 292. Additionally, our DE and EIA modules also significantly reduce specific types of errors. The detailed analysis is shown in Appendix C.4
Case Study We conducted the case study to further explore the effect of our proposed modules in multi-EAE. As shown in Figure 6, this is a complex document containing four events and there exists the argument overlapping phenomenon. First, without the dependency-guide encoding (DE), our model fails to identify arguments such as “Sean Collier” and “gun”. However, with the DE module, our model correctly predicts the roles of these arguments. This demonstrates that the event dependencies provide beneficial guidance. Additionally, with the EIA module, our model is capable of extracting overlapping arguments like “Dzhokhar Tsarnaev” and “Silva”, which indicates that the EIA module provides event-specific contextual information for a better context understanding.

heading: 6 Conclusion
text: In this paper, we propose a Multi-EAE model DEEIA, which overcomes the inefficiency limitations of traditional EAE methods. The proposed Dependency-guide Encoding (DE) module and Event-specific Information Extraction (EIA) module effectively enhances the model’s ability to understand complex multi-event contexts. Our extensive experiments on three public benchmarks illustrate the superiority of our model in performance and efficiency.

heading: 7 Limitations
text: The primary limitation of our method is the issue of input length. Concatenated text and prompts are more easy to exceed the maximum input length. Currently, our solution to this is to use a sliding window approach (Zhou et al., 2021; Zhang et al., 2021) to encode the sequences of different windows and average the overlapping token embeddings of different windows to obtain the final representation. However, this method is not the optimal solution for processing long texts, which leads to the information loss and results in suboptimal performance. Therefore, in the future, we will explore to address the challenge of long input text with the aim of enhancing our DEEIA model.

heading: Acknowledgements
text: This work was supported by the National Science Foundation of China under Grant 62106038, and in part by the Sichuan Science and Technology Program under Grant 2023YFG0259.

heading: A Dynamic Window Algorithm
text: For sequences that surpass the maximum length of 512, we employ a sliding window approach to process longer sequences. For the general case of processing long input texts, we have designed the following Algorithm 1 based on sliding windows. In our implement, PAIE (Ma et al., 2022) has already utilized a sliding window to divide the long document into several instances. We only use the sliding window to process the prompts and ultimately obtain the final sequence representation through pooling. In our experiments, both d1 and d2 are set to 250.

heading: B Experimental Details
text: B.1 Dataset Statistics
We evaluate our proposed method on four event argument extraction datasets.
RAMS (Ebner et al., 2020) is a document-level EAE dataset with 9,124 annotated events from English online news, annotated event-wise. Following (He et al., 2023), we employ a sliding window approach to aggregate events in the same context into single instances with multiple events, following the original train/dev/test split.
WikiEvents (Zhang et al., 2020) is a documentlevel EAE dataset featuring events from English Wikipedia and associated news articles. It includes co-reference links for arguments, but we only use the exact argument annotations in our experiments.
MLEE (Pyysalo et al., 2012) is a documentlevel event extraction dataset, contains manually annotated abstracts from bio-medical publications in English. We follow the preprocessing steps outlined by (Trieu et al., 2020). Since there is only train/test data split for the preprocessed dataset, we employ the training set as the development set.
ACE05 (Doddington et al., 2004) is a labeled corpus used for information extraction, consisting of newswire, broadcast news, and telephone conversations. We employ its English event annotations for sentence-level Event Argument Extraction (EAE). The data preprocessing follows the method described by (Ma et al., 2022).
The detailed dataset statistics of three datasets are shown in Table 4. We also calculate the distributions of the number of events per instance on the three dataset, which are shown in Figure 7. As shown in Figure 7, three datasets exhibit different data distributions between singleevent samples and multi-event samples. For RAMS dataset, single events samples dominate the majority, while the proportion of multi-event samples is quite small. However, for WikiEvents and MLEE datasets, multi-event samples account for a significant proportion.
B.2 Experimental Details
According to TabEAE, using RoBERTa as the PLM outperforms BART across multiple approaches (such as PAIE and TabEAE). Therefore, we adopt RoBERTa as our PLM so as to compare to the SOTA method. Our implementation utilizes Pytorch and runs on a Tesla A100 GPU. We configure
the encoder using the initial 17 layers of RoBERTalarge (Liu et al., 2019). The decoder’s self-attention and feedforward layers inherit their weights from RoBERTa-large’s subsequent 7 layers. This division of a 17-layer encoder and a 7-layer decoder is empirically determined as the most effective configuration (He et al., 2023). It’s important to note that the decoder’s cross-attention component is initialized randomly, with its learning rate set at 1.5 times that of other parameters. More detailed hyperparameter setting is shown in Table 5. We utilize the prompts proposed in PAIE (Ma et al., 2022), which are shown in Table 9.
B.3 Experimental Results on ACE05
We evaluate our proposed model on the ACE05 dataset (Doddington et al., 2004), and the specific experimental results are shown in Table 6 (The reported results are averaged from 5 different random seeds). Experimental results demonstrate that our proposed DEEIA model also performs well on the sentence-level ACE05 dataset, significantly improving the extraction performance of multi-event instances.

heading: C Experimental Analysis
text: C.1 Effect Analysis on Event Numbers in WikiEvents
As illustrated in Figure 8, as the event number increases, we observe a decreasing trend in the performance of all models. We believe this is due to the fact that more events require the model to process
more complex information and longer text, which is more difficult. Furthermore, we find that the baseline model PAIE performs significantly worse on samples where the number of events exceeds two. In contrast, our model demonstrates a marked improvement in multi-event samples compared to PAIE-multi and TabEAE-multi, which shows the advantages of DEEIA in Multi-EAE.
C.2 Analysis of Dependency Guidance
To investigate the manner in which the learnable attentive biases influence the self-attention mechanism, we collect all attentive biases (calculated in Eq. 4) for the test sets of all three datasets. These biases are then categorized based on dependency types and averaged across all attention heads and instances. As shown in Figure 9, the self-attention scores are primarily determined by vanilla selfattention, with minimal influence from dependency information at bottom layers. However, as the number of layers increases, the impact of learnable attentive biases gradually becomes significant, especially between layers 12 to 16.
Additionally, we observe that for different datasets, the attentive bias distributions corresponding to inter-event and intra-event dependencies are different. For RAMS dataset, the attentive bias associated with intra-event dependency is relatively positive, but that corresponding to inter-event dependency is relatively negative. We believe that in the RAMS dataset, there are more single-event samples, and the model focuses more on learning intra-event information associations. However, for WikiEvents and MLEE datasets, the attentive biases for both dependencies are mostly positive, indicating that in these datasets, samples with multiple events are more prevalent, and both dependencies provide beneficial guidance for the model to solve the information complexity problem.
C.3 Architecture Variants
In this section, we explore the effect of different architectures and define two types of architecture variants. (1) To explore whether to integrate the dependency information into the decoder, we define DEEIAB , which integrates the dependency information to both the encoder and decoder. (2) Since the prompts for events are defined based on their event types, the same event types will have the same prompts. Therefore, in a document, if there are multiple events of the same type, whether to concatenate repeated prompts becomes an option. We concatenate repeated prompts, and call this DEEIAM .
As shown in Table 7, there is a minor performance drop across all three datasets when dependency information is integrated into the decoder. This implies that embedding dependency information during the encoding phase is adequate, and overloading the model with excessive informational guidance is not of benefit. Furthermore, concatenating repeated prompts results in a slight improvement in the performance for the RAMS and WikiEvents datasets, but a marginal decline for the MLEE dataset. Overall, the improvement is not substantial. We believe that while concatenating repeated prompts increases prompt diversity, it also extends the sequence length, thereby increasing the difficulty of long-distance reasoning.
C.4 Error Analysis
To compare different models in greater detail, we conduct error analysis on dev sets of RAMS, WikiEvents and MLEE datasets. We divide the errors into five categories, which is shown in Figure 10. Wrong span refers to the case where the predicted span and the true span have no intersection; Partial refers to the case where the predicted span and the golden span partially overlap, which means the predicted span is a proper subset of the golden span or vice versa; Overlap occurs when there is a non-partial case, indicating that there is overlap between the predicted span and the golden span; Over-extraction refers to the case where the golden span is empty while other span is predicted; Under-extraction refers to the case where the golden span is not empty while the predicted span is empty.
As shown in Figure 10, compared to the SingleEAE baseline PAIE, our DEEIA model reduces the number of errors from 312 to 292, indicating the effectiveness of DEEIA in capturing event correlations. Compared to the baseline PAIE-multi, our DEEIA model reduces the number of errors from 358 to 292, especially decreasing the number of Partial, Overlap, and Over-extract errors. The ablation study shows that our proposed DE module mainly reduces the Over-extract and Miss errors, demonstrating that structural guidance can effec-
tively help the model to deal with complex contexts. Meanwhile, the EIA module mainly reduces the Miss and Partial errors, indicating that this module offers event-specific contextual information for a better extraction of arguments.
C.5 Comparison with Large Language Models
Large language models (LLMs) have garnered substantial interest and attention from researchers, highlighting their extensive applicability across a wide array of tasks, such as text classification (Chen et al., 2021; Luo et al., 2024), dialogue (Zhang et al., 2023, 2024a), offensive language detection (Zhou et al., 2023c,b), graph tasks (Zeng et al., 2023, 2022a), and in particular, the formation Extraction (IE) tasks (Xu et al., 2023; Li et al., 2023b; Zhang et al., 2024b; Zhou et al.; Liu et al., 2022, 2024; Xia et al., 2024a,b). In this paper, we make a comparison with the recent state-of-the-art LLM-based approach presented in the work (Zhou et al., 2023a), which utilizes LLMs for the EAE task. We report their experimental results in Table 8. The experiments are conducted on three prominent large language models: text-davinci-003 (Ouyang et al., 2022), gpt-3.5turbo (Ouyang et al., 2022) and GPT-4 (Achiam et al., 2023). These models are accessed via the
public APIs from OpenAI’s services7. As shown in Table 8, compared to the supervised learning models, LLMs still show a significant performance gap in the EAE task. Additionally, the operational costs of large models are inherently high. Our approach outperforms LLMs in terms of efficiency, cost, and effectiveness in the document-level EAE task.
7https://openai.com/api/

heading: Future Work
text: Therefore, in the future, we will explore to address the challenge of long input text with the aim of enhancing our DEEIA model