Enhancing Document-level Event Argument Extraction with Contextual Clues and Role Relevance
Year: 2023

Abstract:
Document-level event argument extraction poses new challenges of long input and crosssentence inference compared to its sentencelevel counterpart. However, most prior works focus on capturing the relations between candidate arguments and the event trigger in each event, ignoring two crucial points: a) nonargument contextual clue information; b) the relevance among argument roles. In this paper, we propose a SCPRG (Span-triggerbased Contextual Pooling and latent Role Guidance) model, which contains two novel and effective modules for the above problem. The Span-Trigger-based Contextual Pooling (STCP) adaptively selects and aggregates the information of non-argument clue words based on the context attention weights of specific argument-trigger pairs from pre-trained model. The Role-based Latent Information Guidance (RLIG) module constructs latent role representations, makes them interact through roleinteractive encoding to capture semantic relevance, and merges them into candidate arguments. Both STCP and RLIG introduce no more than 1% new parameters compared with the base model and can be easily applied to other event extraction models, which are compact and transplantable. Experiments on two public datasets show that our SCPRG outperforms previous state-of-the-art methods, with 1.13 F1 and 2.64 F1 improvements on RAMS and WikiEvents respectively. Further analyses illustrate the interpretability of our model.

heading: 1 Introduction
text: Event argument extraction (EAE) aims to identify the arguments of events formed as entities in text and predict their roles in the related event. As the key step of event extraction (EE), EAE is an important NLP task with widespread applications, such as recommendation systems (Li et al., 2020)
∗Corresponding author: Hong Qu
and dialogue systems (Zhang et al., 2020a) for presenting unstructured text containing event information in structured form. Compared with previous works (Liu et al., 2018; Wadden et al., 2019; Tong et al., 2020) focusing on sentence-level EAE, more and more recent works tend to explore documentlevel EAE (Wang et al., 2022b; Yang et al., 2021; Xu et al., 2022), which needs to solve long-distance dependency (Ebner et al., 2020) and cross-sentence inference (Li et al., 2021) problems. Therefore, many works (Zhang et al., 2020b; Pouran Ben Veyseh et al., 2022) try to construct graphs based on heuristic rules (Xu et al., 2021; Liu et al., 2022a) or syntactic structures (Xu et al., 2022) and model logical reasoning with Graph Neural Networks (Kipf and Welling, 2016; Zeng et al., 2023b,a). However, all of state-of-the-art works ignore two crucial points: (a) the non-argument clue information; (b) the relevance among argument roles.
Non-argument clues are contextual text except target arguments that can provide important guiding information for the prediction of many complex argument roles. For example, in Figure 1, for the event Conflict and Attack, non-argument clues detonated, claim responsibility and terrorist attack can provide significant clue information for identifying arguments explosive belts and Islamic State. However, many previous works (Li et al., 2021; Xu et al., 2022) only utilize pre-trained transformer-
12908
based encoder to obtain global context information implicitly, ignoring that for different arguments appearing in events, they should pay attention to context information highly relevant to the entity (Zhou et al., 2021) and target event (Ebner et al., 2020). Therefore in this paper, we design a Span-Triggerbased Contextual Pooling (STCP) module, which merges the information of non-argument clues for each argument-trigger pair based on the their contextual attention product from pre-trained model, enhancing the candidate argument representation with additional relevant context information.
Some argument roles have close semantic relevance that is beneficial for argument extraction. For example, in Figure 1, there is close semantic relevance between roles injurer and victim, which can provide significant information guidance for the argument extraction of these two roles in the target event Conflict and Attack. Moreover, many roles co-occur in multiple events (Ebner et al., 2020; Li et al., 2021), which may have close semantic relevance. Specifically, we count and visualize the frequency of co-occurrence between 15 most frequent roles in RAMS dataset in Figure 2. For example, roles attacker, target and instrument co-occur frequently, demonstrating that they are more semantically relevant than other roles. In this paper, we propose a Role-based Latent Information Guidance (RLIG) module, consisting of role-interactive encoding and role information fusion. Specifically, we design a role-interactive encoder with roles added into the input sequence, where role embeddings can not only learn latent semantic informa-
tion of roles, but capture semantic relevance among roles. The latent role embeddings are then merged into candidate arguments through pooling and concatenating operations, providing information guidance for document-level EAE.
In this paper, we propose an effective documentlevel EAE model named SCPRG (Span-triggerbased Contextual Pooling and Role-based latent information Guidance) containing STCP module and RLIG module for the the aforementioned two problems respectively. Notably, these two modules leverage the well-learned attention weights from the pre-trained language model with no more than 1% new parameters introduced and are easily applied to other event extraction models, which are compact and transplantable. Moreover, we try to eliminate noise information by excluding argumentimpossible spans. Our contributions are summarized as follows:
• We propose a span-trigger-based contextual pooling module, which adaptively selects and aggregates the information of non-argument clues, enhancing the candidate argument representation with relevant context information.
• We propose a role-based latent information guidance module, which provides latent role information guidance containing semantic relevance among roles.
• Extensive experiments show that SCPRG outperforms previous start-of-the-art models, with 1.13 F1 and 2.64 F1 improvements on public RAMS and WikiEvents (Li et al., 2021) datasets. We further analyse the attention weights and latent role representations, which shows the interpretability of our model1.

heading: 2 Method
text: We formulate document-level event argument extraction as a multi-class classification problem. Given a document D consisting of N words, i.e. D = {w1, w2, ..., wN}, pre-defined event types set E , the corresponding role set Re and trigger t ∈ D for each event e ∈ E , this task aims at predicting all (r, s) pairs for each event in document D, where r ∈ Re is an argument role for event e ∈ E and s ⊆ D is a contiguous text span in D. Following (Ebner et al., 2020; Xu et al., 2022), we
1Our implementation is available at https://github. com/LWL-cpu/SCPRG-master
extract event arguments for each event in a document independently and Figure 3 shows the overall architecture of our SCPRG.

heading: 2.1 Role-interactive Encoder
text: Role Type Representation In order to capture semantic relevance among roles, we add role type information into the input sequence and make interaction among context and roles by multi-head attention, which obtains context and role representations in a shared knowledge space. Specifically, we construct latent embeddings of roles with different special tokens 2 in the pre-trained model, where each role type has a specific latent representation. On account that role names also contain valuable semantic information (Wang et al., 2022b), we wrap role names with special role type tokens and take the embedding of the start special toke as the role embedding. Taking the role Place as an example, we finally represent it as [R0] Place [R0], where [R0] is the special role type token of Place. Role-interactive Encoding For the input document D = {w1, w2, ..., wN}, the target event e and the corresponding role set Re = {r1, r2, r3, ...}, we concatenate them into a sequence as follows:
S = [CLS] [Ee] e [Ee] [SEP] w1 ... wN [SEP] [R1] r1 [R1] [R2] r2 [R2] ... [SEP], 2In our implement, we utilize [unused] tokens for BERT (Devlin et al., 2019) and add special tokens for RoBERTa (Liu et al., 2019).
where [Ee] is the special event token of event e. [R1] and [R2] are the special role type tokens of r1 and r2. We use the last [SEP] to represent none category. Next, we leverage the pre-trained language model as an encoder to obtain the embedding of each token as follows:
Hs = Encoder(S). (1)
Then we can obtain event representation He ∈ R1×d of the start [Ee], context representation Hw ∈ Rlw×d, and role representation Hr ∈ Rlr×d respectively from Hs, where lw is the length of word pieces list and lr is the length of role list. For input sequences longer than 512, we leverage a dynamic window to encode the whole sequence and average the overlapping token embeddings of different windows to obtain the final representation.
Significantly, through role-interactive encoding, the role embeddings can capture semantic relevance and adapt to the target event and context, which better guides the argument extraction.

heading: 2.2 Span-Trigger-based Contextual Pooling
text: Argument-impossible Spans Exclusion In order to eliminate noise information of useless spans, we reduce the number of candidate spans by excluding some argument-impossible spans, e.g. spans with comma in the middle. With such improvement, we reduce a quarter of candidate spans on
average and make our model pay attention to candidate spans with useful information. Span-Trigger-based Contextual Pooling For a candidate span ranging from wi to wj , most previous span-based methods(Zhang et al., 2020b; Xu et al., 2022) represent it through the average pooling of the hidden state of tokens within this span:
1 j−i+1 ∑j k=i h w k , where h w k is the k
th token embedding from Hw.
However, average pooling representation ignores the significant clue information of other nonargument words. Although self-attention mechanism of the pre-trained encoder can model tokenlevel interaction, such global interaction is specific to the event and candidate arguments. Therefore, we propose to select and fuse useful contextual information highly related to each tuple consisting of a candidate span and the event trigger word, i.e. (si:j , t). We directly utilize the attention heads of pre-trained transformer-based encoder for spantrigger-based contextual pooling, which transfers the well-learned dependencies from the pre-trained language model without learning new attention layers from scratch (Zhou et al., 2021).
Specifically, we use the token-level attention heads Aw ∈ RH×lw×lw of context from the last transformer layer in the pre-trained language model. Then we can obtain the context attention ACi:j ∈ Rlw of each candidate span ranging from wi to wj with average pooling:
ACi:j = 1 H(j − i+ 1) H∑
h=1
j∑
m=i
Awh,m. (2)
Then for span-trigger pair (si:j , t), we obtain the contextual clue information csi:j ∈ Rd that are important to candidate span by multiplying the attentions followed by normalization:
pci:j = softmax(A C i:j · ACt ),
csi:j = Hw pci:j , (3)
where ACt ∈ Rlw is the contextual attention of trigger t and pci:j ∈ Rlw is the computed attention weight vector for context.

heading: 2.3 Role-based Latent Information Guidance
text: RLIG module constructs latent role embeddings through role-interactive encoding in Sec. 2.1 and performs role information fusion through pooling operation, which provides valuable latent role information guidance.
Role Information Fusion In order to make each candidate argument get the useful information guidance of roles, we modify our span-trigger-based contextual pooling method to select role information adaptively. We get the latent role information rsi:j ∈ Rd for si:j through contextual pooling, by modifying the operation in Eq. 2 and Eq. 3:
ARi:j = 1H(j−i+1) ∑H h=1 ∑j m=iA r h,m,
pri:j = softmax(A R i:j · ARt ),
rsi:j = Hr pri:j ,
(4)
where Ar ∈ RH×lw×lr are attention heads of roles from the last transformer layer in the pre-trained language model. ARi:j ∈ Rlr is the role attention for each candidate span and ARt ∈ Rlr is the role attention of trigger t. pri:j ∈ Rlr is the computed attention weight vector for roles.
For a candidate span si:j , we fuse the average pooling representation, contextual clue information csi:j and latent role information rsi:j as follows:
si:j = tanh(W1[ 1
j − i+ 1
j∑
k=i
hwk ; c si:j ; rsi:j ]),
(5) where W1 ∈ R3d×d is learnable parameter.

heading: 2.4 Classification Module
text: Boundary Loss Since we extract arguments in span level, whose boundary may be ambiguous, we construct start and end representation with fully connected neural networks to enhance the representation of candidate spans: Hstart = WstartHs,Hend = WendHs, where Hs is the hidden representation of input sequence S. On this basis, we enhance the start and end representation by integrating context and role information with span-trigger-based contextual pooling as follows:
zstarti:j = Hstart pi:j ,
zendi:j = Hend pi:j ,
hstarti:j = tanh(W2[h start i ; zstarti:j ]),
hendi:j = tanh(W3[h end j ; zendi:j ]),
(6)
where hstarti and h end j are the i th and jth vector of Hstart and Hend. pi:j is the computed attention vector for both context and roles which is calculated similarly to Eq. 3 or Eq. 4 and W2,W3 ∈ R2d×d are learnable parameters. Then we obtain the final representation s̃i:j for a candidate span
as follows: s̃i:j = Ws[hstarti:j ; si:j ;hendi:j ], where Ws ∈ R3d×d is the learnable model parameter.
Finally, the boundary loss is defined to detect the start and end position following (Xu et al., 2022):
Lb = − ∑|D| i=1[y s i logP s i + (1− ysi ) log(1− P si )
+yei logP e i + (1− yei ) log(1− P ei )],
(7) where ysi and y e i denote golden labels and P s i = sigmoid(W4h start i ) and P e i = sigmoid(W5h end i ) are the probabilities of the word wi predicted to be the first or last word of a golden argument span. Classification Loss For a candidate span si:j in event e, we concatenate the span representation s̃i:j , trigger representation ht, their absolute difference |ht − s̃i:j |, element-wise multiplication ht ⊙ s̃i:j , event type embedding He and span length embedding Elen and get the prediction P (ri:j) of the candidate span si:j via a feed-forward network:
Ii:j = [̃si:j ;ht; |ht−s̃i:j |;ht⊙s̃i:j ;He;Elen], (8)
P (ri:j) = FFN(Ii:j). (9)
Considering most candidate arguments are negative samples and the imbalanced role distribution, we adopt focal loss (Lin et al., 2017) to make the training process focus more on useful positive samples, where α and γ are hyperparameters.
Lc = − ∑|D|
i=1 ∑|D| j=1 α[1− P (ri:j = yi:j))]γ
· logP (ri:j = yi:j). (10)
Finally, we have the train loss consisting of Lc and Lb with hyperparameter λ:
L = Lc + λLb. (11)

heading: 3 Experiments

heading: 3.1 Experimental Setup
text: Datasets and Metrics We evaluate the proposed model on two large-scale public document-level
EAE datasets, RAMSv1.0 (Ebner et al., 2020) and WikiEvents (Li et al., 2021) following the official train/dev/test split, whose detailed data statistic are shown in Table 1. Following (Xu et al., 2022), we report the Span F1 and Head F1 on dev and test sets for RAMS dataset. Span F1 requires the predicted argument spans to fully match the golden ones, while Head F1 evaluates solely on the head word3 of the argument span. Additionally, for WikiEvents dataset, we report the Head F1 and Coref F1 scores on test set for argument identification task (Arg IF) and argument classification (Arg CF) task respectively following (Li et al., 2021). The Coref F1 evaluates the coreference between extracted arguments and golden arguments as used by (Ji and Grishman, 2008) and the model achieves Coref F1 if extracted arguments are coreferential with golden arguments. Baselines We compare different categories of document-level EAE models which mainly consist of tagging-based methods such as BERT-CRF (Shi and Lin, 2019), BERT-CRFTCD (Ebner et al., 2020), span-based methods like Two-Step (Zhang et al., 2020b), Two-StepTCD (Ebner et al., 2020), TSAR (Xu et al., 2022), and other generation-based methods such as FEAE (Wei et al., 2021), BERTQA (Du and Cardie, 2020b), BART-Gen (Li et al., 2021), EA2E (Zeng et al., 2022). Moreover, we use BERTbase (Devlin et al., 2019) and RoBERTalarge (Liu et al., 2019) as the pre-trained transformer-based encoder. Hyperparameters Setting We set the dropout rate to 0.1, batch size to 8, and train our SCPRG using Adam (Kingma and Ba, 2014) as optimizer with 3e-5 learning rate. The hidden dimension d is 768 for SCPRGbase and 1024 for SCPRGlarge. In order to mitigate imbalanced role distribution problem, we set the weight ratio α of empty class and other classes to 10:1. We set hyperparameters γ
3The head word of a span is defined as the word that has the smallest arc distance to the root in the dependency tree.
to 2 and boundary loss weight λ to 0.1 for both two datasets. We train SCPRG for 50 epochs for RAMS dataset and 100 epochs for WikiEvents dataset.

heading: 3.2 Main Results
text: Table 2 shows the experimental results on both dev and test set in RAMS dataset. Compared with previous tagging-based and span-based methods like BERT-CRF and Two-Step, our SCPRG equipped with BERTbase yields an improvement of +8.46/+9.64 ∼ +6.36/+7.14 Span F1 and +7.68/+9.00 ∼ +5.38/+6.40 Head F1 on dev/test set, showing that our SCPRG framework has superiority in excluding impossible candidate spans and solving the imbalance of data distribution problem. Significantly, SCPRG with RoBERTalarge also outperforms previous state-of-the-art models BART-Genlarge4 (+3.68/+2.34 Span/Head F1 on test set) and TSARlarge (+1.14/+1.13 Span/Head F1 on test set). These results demonstrate the superior extraction ability of our model, benefiting from the effect of contextual clue information and latent role representation with semantic relevance.
Moreover, we further validate our SCPRG on WikiEvents and achieve new state-of-the-art performance in both tasks with base and large pretrained models, which can be viewed in Table 3. Our SCPRG outperforms previous competitive methods like TSAR and EA2E. Compared with TSARlarge, our SCPRG improves up to +0.64/+0.58 Head/Coref F1 for argument identification and +1.22/+1.29 Head/Coref F1 for argument classification on the test set. Besides, SCPRG also outperforms recent competitive generationbased method EA2Elarge in argument identification (+2.64/+0.33 Head/Coref F1) and argument classification (+2.31/+0.38 Head/Coref F1) tasks. These experimental improvements demonstrate the great advantage of our framework fused with argumentevent specific context information and the helpful guidance of latent role information.

heading: 3.3 Ablation Study
text: To better illustrate the capabilities of our components, we conduct ablation study on RAMS dataset as shown in Table 4. We also provide ablation study results on WikiEvents datasets in Appendix A.
First, when we remove span-trigger-based contextual pooling (STCP) module, both Span F1 and
4BART-Genlarge is based on BARTlarge (Lewis et al., 2019) which is pre-trained on the same corpus.
Head F1 score of SCPRGbase/ SCPRGlarge drop by 1.61/1.43 and 1.42/2.09 on test set, which indicates that our STCP plays a vital role in capturing the clue information of non-argument context that is crucial for document-level EAE.
Additionally, when removing role-based latent information guidance (RLIG) module5, the performance of SCPRGbase/ SCPRGlarge drops sharply by 1.03/1.04 Span F1 and 1.58/1.2 Head F1 on RAMS test set. It suggests that our RLIG module effectively guides argument extraction with meaningful latent role representations containing semantic relevance among roles. When removing both STCP and RLIG module, the performance decay exceeds that when removing a single module, which explains that our two modules can work together to improve the performance.
Moreover, when removing argument-impossible spans exclusion (ASE) operation, both SCPRGbase and SCPRGlarge have a performance decay, which
5We also remove the corresponding role tokens added in the input sequence.
indicates that excluding argument-impossible candidate spans eliminates noise information and contributes to argument extraction. Focal Loss helps to balance the representation of positive and negative samples, facilitating smooth convergence of the model during training. However, it does not contribute to improving the performance of the model.

heading: 3.4 Analysis of Context Attention Weights
text: To assess the effectiveness of STCP in capturing useful contextual information for candidate arguments, we visualize the contextual weights pci:j in Eq. 3 of an example of Figure 1. As shown in Figure 5, our STCP gives high weights to nonargument words such as attack, responsibility and terrorist attack, which are most relevant to the span-trigger pair (Islamic State, wounded). Interestingly, our STCP also gives relatively high attention weights to words in other arguments like explosive, Dozens and Kabul, which means that these argument words provide important information for the role prediction of Islamic State. The visualization demonstrates that our STCP can not only capture the non-argument clue information that is related to candidate spans, but model the information interaction among related arguments in an event.
Additionally, we also explore the attention weights based on different span-trigger pairs in an event. In Figure 4, we randomly select 30 candidate spans in an event and draw the heat map based on their attention weights to the context. The heat map shows that different candidate arguments focus on different context information, indicating that our STCP can adaptively select contextual information according to candidate argument spans.

heading: 3.5 Analysis of Role Information Guidance
text: To verify that our model can capture semantic relevance among roles, we visualize the cosine simi-
Method Params Dev Test
Span F1 Head F1 Span F1 Head F1
SCPRGbase 119.37M 46.56 53.38 48.94 56.10 -STCP 118.78M 46.18 52.87 47.33 54.68 -RLIG 118.78M 45.27 52.36 47.91 54.52 -STCP&RLIG118.19M 45.07 51.59 45.76 53.16 -ASE 119.37M 45.92 52.61 48.26 55.63 SCPRGlarge 372.90M 50.53 57.66 52.32 59.66 -STCP 371.85M 49.94 56.55 50.89 57.57 -RLIG 371.68M 49.96 57.32 51.28 58.46 -STCP&RLIG370.63M 48.33 54.04 47.52 55.61 -ASE 372.90M 49.80 56.31 51.73 58.48
Table 4: Ablation Study on RAMS for SCPRG.
larity between latent role representations from two events in RAMS dataset in Fig 6. As the figure shows, roles origin and destination, attacker and target have similar representations, which agrees with their semantics, demonstrating that our model can capture the semantic relevance among roles.
Moreover, in order to verify the beneficial guidance of role representations, we display the tSNE (van der Maaten and Hinton, 2008) visualization of arguments belonging to two different roles that co-occur in 5 different documents, along with corresponding latent role embeddings. As Figure 7a shows, arguments belonging to the same role in different documents are scattered over the whole embedding space due to their different target events and context. Notably, fused with latent role embeddings, in Figure 7b, the representation of arguments belonging to victim or place is more adjacent, which illustrates our RLIG provides beneficial latent role information guidance.

heading: 3.6 Analysis of Complexity and Compatibility
text: SCPRG is a simple but effective framework for document-level EAE, where both STCP and RLIG introduce few parameters. Specifically, STCP leverages the well-learned attention heads from the pretrained encoder and makes multiplication and normalization operation, which only introduces about 0.28% new parameters as shown in Table 4. Our RLIG only introduces about 0.3% new parameters in the role embedding layer6 and feature fusion layer. This makes the parameter quantity of our model approximate to the transformer-based encoder plus a MLP classifier.
6We add new special tokens for role types and therefore the RLIG module introduces more parameters in SCPRGlarge.
Additionally, the two proposed techniques STCP and RLIG have good transportability, which can be easily applied to other event extraction models, leveraging the attention heads of pre-trained transformer encoder such as BERT.

heading: 4 Related Works
text: Although deep learning has achieved significant success in many areas of computer vision (Li et al., 2022; Wang et al., 2023, 2022a; Pan et al., 2023; Wang and Chen, 2023) like 3D Scene Graph Generation (Liu et al., 2022c; Chen and Kou, 2023) and Image Semantic Segmentation (Zhang et al., 2022), its impact on Event Argument Extraction in natural language processing has been relatively limited. This is primarily due to the complexity and ambiguity inherent in natural language, which presents significant challenges for the accurate identification and extraction of event-related information.

heading: 4.1 Sentence-level Event Extraction
text: Previous approaches focus on extracting the event trigger and its arguments from a single sentence. (Chen et al., 2015) firstly propose a neural pipeline model for event extraction and (Nguyen et al., 2016; Nguyen and Grishman, 2015; Liu et al., 2017; Zhou et al., 2020) further extend the pipeline model to recurrent neural networks and convolutional neural networks. To model the dependency of words in a sentence, (Liu et al., 2018; Yan et al., 2019; Fernandez Astudillo et al., 2020) leverage dependency trees to model semantic and syntactic relations. (Wadden et al., 2019) enumerates all possible spans and construct span graphs with graph neural net-
works to propograte information. Some methods using transformer-based pre-trained model (Wadden et al., 2019; Wang et al., 2019; Tong et al., 2020; Lu et al., 2021; Liu et al., 2022b) also achieve remarkable performance.

heading: 4.2 Document-level Event Extraction
text: In real-world scenarios, a large number of event elements are expressed across sentences and therefore recent works begin to explore document-level event extraction (DEE). DEE focuses on extracting event arguments from an entire document and faces the challenge of the long distance dependency (Wang et al., 2022b; Xu, 2022).
For document-level EAE, the key step of DEE, most of previous works mainly fall into three categories: (1) tagging-based methods; (2) span-based methods; (3) generation-based methods. (Wang et al., 2021; Du and Cardie, 2020a) utilize the sequence labeling model BiLSTM-CRF (Zhang et al., 2015) for DEE. (Zheng et al., 2019) propose a transformer-based architecture and model
DEE as a serial prediction paradigm, where arguments are predicted in a predefined role order. Base on their architecture, (Xu et al., 2021) construct a heterogeneous graph and a tracker module to capture the interdependency among events. However, tagging-based methods are inefficient due to the restriction to the extraction of individual arguments, and the former extraction will not consider the latter extraction results. (Yang et al., 2021) propose an encoder-decoder framework that extracts structured events in a parallel manner. Besides, (Ren et al., 2022) integrate argument roles into document encoding to aware tokens of multiple role information for nested arguments problem. Other span based methods (Ebner et al., 2020; Zhang et al., 2020b) predict the argument roles for candidate text spans with a maximum length limitation. Moreover, (Xu et al., 2022) propose a twostream encoder with AMR-guided graph to solve long-distance dependency problem. On another aspect, (Li et al., 2021) formulate the problem as conditional generation and (Du et al., 2021) regards the problem as a sequence-to-sequence task. (Wei et al., 2021) reformulate the task as reading a comprehension task.

heading: 5 Conclusion
text: In this paper, we propose a novel SCPRG framework for document-level EAE that mainly consists of two compact, effective and transplantable modules. Specifically, our STCP adaptively aggregates the information of non-argument clue words and RLIG provides latent role information guidance containing semantic relevance among roles. Experimental results show that SCPRG outperforms
existing state-of-the-art EAE models and further analyses demonstrate that our method is both effective and explainable. For future works, we hope to apply SCPRG to more information extraction tasks such as relation extraction and multilingual extraction, where contextual information plays a significant role.

heading: 6 Limitations
text: Although our experiments prove the superiority of our SCPRG model, it is only applicable to document-level EAE tasks with known event triggers because both STCP and RLIG calculate the attention product of the trigger and candidate spans. However, in real-life scenarios, event triggers are not always available. In view of this problem, we have a preliminary solution and plan to improve our model in the next work. The core idea of our method is to select and integrate context and role information based on candidate arguments and target events. Based on this idea, we briefly provide two solutions for the above limitation. First, we can make the model predict the best candidate trigger words. Second, we can replace trigger words with special event tokens. In the next work, we plan to extend our model to document-level EAE tasks without trigger words and evaluate it through extensive experiments.

heading: Acknowledgements
text: This work was supported by the National Science Foundation of China under Grant 61976043, and in part by the Science and technology support program of Sichuan Province under Grant 2022YFG0313.

heading: A Ablation Study
text: In the main body of the paper, we conduct ablation study on RAMS dataset for SCPRGbase and SCPRGlarge. In order to fully evaluate the effect of different components on our model, we also provide the results of the ablation study on WikiEvents for for SCPRGbase and SCPRGlarge.
As shown in Table 5, when we remove STCP module, both Head F1 and Coref F1 score of SCPRGbase drop by 1.49/1.44 and 1.43/1.26 on test set for argument identification task (Arg IF) and argument classification (Arg CF) task, which demonstrates that our STCP captures the clue information of non-argument context that is significant for document-level EAE.
Additionally, when removing RLIG module, the performance of SCPRGlarge drops sharply by 0.96/2.02 Head F1 and 1.05/1.12 Coref F1 on Wikievent test set for both two tasks. Moreover, when we remove argument-impossible spans exclusion (ASE), both SCPRGbase and SCPRGlarge have a performance decay. These results indicate that both STCP and ASE are beneficial.

heading: B Co-occurrence Frequency Matrix
text: In this section, we show the complete cooccurrence frequency matrix which contains all roles in RAMS test set. We count the frequency of co-occurrence between every two roles and draw the heat map according to the frequency in Figure 8. It can be seen from the figure that the co-occurrence phenomenon exists between many roles, especially those occur in the same event, which indicates that there is semantic relevance among roles.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
6 Limitations
7 A2. Did you discuss any potential risks of your work? Our work does not involve potential risks
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1 Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments?

heading: 3.6 Analysis of Complexity and Compatibility
text: 3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 3.6 Analysis of Complexity and Compatibility
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 3.1 Experimental Setup
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 3.2 Main Results
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 3.1 Experimental Setup
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: For future works, we hope to apply SCPRG to more information extraction tasks such as relation extraction and multilingual extraction, where contextual information plays a significant role