Harvesting Events from Multiple Sources: Towards a Cross-Document Event Extraction Paradigm (2024)

Abstract: Document-level event extraction aims to extract structured event information from unstructured text. However, a single document often contains limited event information and the roles of different event arguments may be biased due to the influence of the information source. This paper addresses the limitations of traditional document-level event extraction by proposing the task of cross-document event extraction (CDEE) to integrate event information from multiple documents and provide a comprehensive perspective on events. We construct a novel cross-document event extraction dataset, namely CLES, which contains 20,059 documents and 37,688 mention-level events, where over 70% of them are cross-document. To build a benchmark, we propose a CDEE pipeline that includes 5 steps, namely event extraction, coreference resolution, entity normalization, role normalization and entity-role resolution. Our CDEE pipeline achieves about 72% F1 in end-to-end cross-document event extraction, suggesting the challenge of this task. Our work builds a new line of information extraction research and will attract new research attention. Our code and dataset will be available at https://github.com/cooper12121/CLES.

heading: 1 Introduction
text: In the realm of Natural Language Processing, document-level event extraction (DEE) has been a focal area of research, striving to distill structured information from unstructured text. This process typically involves identifying and categorizing events, along with their associated entities and relations, within a single document (Yang et al., 2018). This approach has demonstrated its effectiveness in numerous applications, such as information retrieval (Sankepally, 2019), content summarization (Zhang et al., 2021) and knowledge graph construction (Guan et al., 2023).
* Corresponding author.
Although significant advancements have been made (Xu et al., 2021; Yang et al., 2021a; Wang et al., 2023a), DEE often encounters limitations in terms of the scope and depth of information that it can provide. Different documents may present varying perspectives or emphasize different aspects of the same event, leading to a fragmented and sometimes biased understanding when viewed in isolation. Specifically, event information may be distributed across multiple documents. As shown in Figure 1, three documents contain different event mentions referring to the same event, where the bottom-left document includes the “Date” argument while the top-right document includes the “Location” argument.
Recognizing these limitations, we propose the task of cross-document event extraction (CDEE), which categorizes events into mention-level and concept-level. A mention-level event refers to the event defined within single document, while a concept-level event refers to a complete event
1913
obtained by integrating information from multiple documents. Compared with DEE, The key issue that CDEE aims to address is the problem of completeness, which means obtaining a complete representation of an event by aggregating event information from multiple documents. After integrating the extraction results of events from multiple documents, merging duplicate information and resolving conflicting information, the whole event can be built.
To foster research in this unexplored field, we have constructed a new cross-document event extraction dataset, called CLES (CrossLinkEventScope). Leveraging Wikipedia as the information source, we utilized the hyperlinks inside to identify the documents relevant to events and aggregate them into collections. These collections not only encompass multiple perspectives of a single event but also include detailed background information and various viewpoints related to the event. Afterwards, we employed a DEE tool (Zhang, 2023) to mine event mentions within documents and manually merge event mentions into a complete event. The results in both processes were manually checked to guarantee the annotation quality. Ultimately, the CLES dataset comprises 9 event types, over 37,688 mention-level events and 3,633 concept-level events, where over 70% are cross-document.
Besides the dataset, we also contribute a CDEE pipeline comprising 5 steps: (1) DEE: Event mentions as well as related arguments are extracted from individual document. (2) Event Coreference Resolution: Event mentions within an event collection are grouped by coreference relations. (3) Entity Normalization: A third-party entity linking library (Zhang, 2023) is utilized to align entities and then their attributes are standardized. (4) Role Normalization: The same type of roles across different documents are normalized by a role mapping table. (5) Entity-Role Resolution: Extracted results from different documents are aligned and integrated to eliminate repetitive and conflicted content.
Our experimental results show that the CDEE pipeline is able to achieve about 72% F1 for endto-end cross-document event extraction, revealing its effectiveness but also the challenge of this task. In the end, we highlight the contributions of this paper as:
1. We introduce a novel CDEE task, aiming to
extend the research scope of event extraction and provide a more comprehensive perspective.
2. We construct a new large-scale CDEE dataset, which provides abundant data and lays the foundation for future research.
3. We build a benchmark pipeline for CDEE, which can be used as a basic baseline for follow-up studies.

heading: 2 Related Work

heading: 2.1 Sentence-Level Event Extraction
text: Sentence-level event extraction has been extensively researched (Liu et al., 2018; Wadden et al., 2019; Hamborg et al., 2019; Wang et al., 2023b; Xu et al., 2023). Du and Cardie (2020) proposed a QA approach for event extraction to avoid the dependency of event extraction results on the previous entity recognition step. Lu et al. (2021) adopted a seq2seq model for event extraction, transforming it into a test2event task. Compared to traditional methods, this approach avoids dividing event extraction into multiple subtasks and can yield results in a single step. Wang et al. (2022a) introduced a novel structured pre-training framework that does not require fine-tuning on specific tasks. It transforms structured prediction into a sequence-based triple prediction task and achieved good results across multiple tasks.

heading: 2.2 Document-Level Event Extraction
text: In the past several years, there has been many methods and models for document-level event extraction, which can be categorized into several types: (1) Pipeline approach first identifies event triggers and event types, and then recognizes event arguments. (2) Sequence labeling approach treats the task as a multi-class classification problem and directly performs sequence labeling on text sequences to identify event triggers and arguments. (3) Graph-based methods and Generative-based methods.
Yang et al. (2018) transformed the DEE task into a Sentence-level Event Extraction (SEE) task, treating sentences containing event triggers and arguments as key-events. However, most of the dataset in this approach only detects information about a single event, without considering argument combinations. Zheng et al. (2019) addressed argument combinations by constructing a Directed
Acyclic Graph (DAG) without relying on trigger words, named Doc2DAG. However, generating the DAG graph in this manner is heavily influenced by false positives and false negatives, and the computational overhead for building the graph is significant. Yang et al. (2021b) employed a non-autoregressive decoder (NAD) and the Hungarian Algorithm for inference and gold matching. It achieves performance similar to Doc2EDAG with improved speed. Zhu et al. (2022) explored a complete graph for event extraction, where all entity pairs of the same event are fully connected. However, since this method suffers from missing argument roles, pruning is introduced to alleviate this issue.
Considering the rich relational information among event parameters in documents, which can establish long-distance relationship knowledge for events, Liang et al. (2022) proposed a relationenhanced document-level event extraction model. Although this model has achieved significant improvements, relation prediction requires the introduction of an additional transformer framework, making the model more complex and increasing computational overhead. Wan et al. (2023) introduced a Token-Token Bidirectional Event Completed Graph (TT-BECG) to addresse the inefficiency and error propagation problems associated with traditional pipeline methods.

heading: 2.3 Cross-Document Information Extraction
text: Although not many, there have been some studies on cross-document information extraction, such as event coreference resolution (Wu et al., 2020; Held et al., 2021; Eirew et al., 2022) and relation extraction (Yao et al., 2021; Lu et al., 2023).
In terms of coreference resolution, Yu et al. (2022) proposed a cross-document coreference resolution model that enhances event mention representation by extracting event arguments. Miculicich and Henderson (2022) addressed coreference resolution using a graph-based approach, while Chen et al. (2023) introduced discourse information to model documents, resulting in a significant performance improvement. Gao et al. (2024) proposed a cross-document coreference resolution model based on discourse information, modeling the structural and semantic information of documents through RST and lexical chains.
With respect to other directions, Caciularu et al. (2021) proposed a novel cross-document pre-training language model to learn rich contextual information across documents. Wang et al.
(2022b) proposed a cross-document relation extraction model based on bridge entities, which utilizes entity relation attention mechanisms across paths to facilitate interactions between entities. To our knowledge, there are still no studies on crossdocument event extraction.

heading: 3 CLES: A Cross-Document Event Extraction Dataset

heading: 3.1 Objective Definition
text: Our goal is to construct a large-scale, domainagnostic cross-document event extraction dataset, which covers a wide range of event types to reflect the rich content and diversity of Wikipedia. Additionally, we do not set restrictions on the time span of events, allowing for the inclusion of historical and contemporary events to enhance the temporal dimension and depth. Moreover, we select Chinese as the main language in building this dataset. To ensure the diversity and comprehensiveness of the dataset, we have defined a total of nine event categories, including ATTACK EVENT, SPORT EVENT, EVENT UNK, ELECTION EVENT, GENERAL EVENT, DISASTER EVENT, ACCIDENT EVENT, AWARD VENT, and OTHERS.
In constructing the dataset for cross-document event extraction, our main idea is to leverage Wikipedia as the information source and utilize hyperlinks added by authors when creating articles to identify and aggregate documents related to events. Each Wikipedia article typically pertains to a specific topic or event, and authors often add hyperlinks to key phrases that point to other related articles or detailed pages about events. These hyperlinks naturally form a network, clustering different documents together based on events.
Using this hyperlink network, we can cluster all documents pointing to the same event or topic, forming the collections of articles centered around specific events. These collections not only encompass multiple perspectives on a single event but also include detailed background information and various viewpoints related to the event. By analyzing and integrating these documents, we can capture comprehensive information about events from multiple sources and perspectives, providing a rich and multidimensional data foundation for cross-document event extraction. The process of dataset construction is illustrated in Figure 2 and the details in the process are explained in the fol-
lowing sections.

heading: 3.2 Data Collection and Cleaning
text: We borrowed the approach from Eirew et al. (2021) and optimized its data collection system to gather data from Wikipedia dump files. After obtaining the raw data, we first conducted data cleaning to ensure the quality and relevance to our goal, which involved several substeps. (1) Removing Non-Event Documents: We reviewed all crawled documents and excluded those that did not clearly describe specific events. For example, some documents might only briefly mention an event without providing detailed information or background about it. (2) Filtering Unrelated Documents: We filtered out documents that were unrelated to the current event document collection. Only the documents directly related to the events were retained in the dataset to ensure consistency and accuracy of the data.

heading: 3.3 Annotation and Validation
text: After completing data cleaning, we carried out a data selection process, that is, the maximum number of documents in each document collection is 10. For the document collections with more than 10 documents, we manually selected 10 documents with the richest event information.
Due to the large scale of documents in our data, the cost of manual annotation for all documents is prohibitively high. Therefore, we used an event extraction tool for annotating each document and then conducted manual verification. We adopted the method proposed by Peng et al. (2023) to label event trigger words, event arguments and their roles. To ensure the quality of the dataset, two annotators independently verified the results annotated by the tool and corrected labeling errors. We calculated the consistent rate between the tool and human annotators using the Fleiss’ Kappa algorithm (Fleiss, 1971). The kappa value is 0.72, indicating decent annotation quality of our dataset.
Based on the single-document event information annotated in the previous step, annotators de-duplicated the event-related argument information. Additionally, they eliminated irrelevant events based on the original document information. In the cases where an entity was assigned with multiple roles, the most accurate role was selected based on context. Ultimately, this process yielded the final event for each document collection. More specifically, in this process, based on our constructed role table and existing entity linking tools, we first perform coarse-grained filtering through a program we wrote, followed by verification and refinement of the merged results by annotators.

heading: 3.4 Dataset Statistical Analysis
text: The scale of the final dataset is shown in Table 1. In terms of scale, our dataset consists of over 20,000 documents and 37,000 events. This demonstrates that our dataset covers a vast amount of event information, spanning a wide range of time frames and diverse textual content. This necessitates event extraction models to possess strong generalization capabilities. Furthermore, the proportion of crossdocument events in our dataset exceeds 70%, indicating that the majority of events require synthesizing information from multiple documents, posing a challenge to the modeling capacity.
We also count the number of documents for each event type as shown in Table 2. It can be observed that our dataset has a power-law distribution across different event types, with ATTACK events being the most common. This also indicates that Wikipedia has the highest number of articles related to attack events.
To delve into the distribution of document collection sizes, we compiled the statistics on the distribution of the document number in each document collection, as shown in Table 3. It can be seen that our dataset has reasonable distributions of different document collection sizes. It contains both crossdocument events and a certain number of singledocument events. This indicates that even in the context of cross-document extraction, there are still some events that can be fully extracted from a single document. Therefore, our dataset can be used to evaluate the methods not only for document-level event extraction but also for cross-document event extraction.
Moreover, the statistical information related to trigger words, argument roles, and their lengths can be found in Table 4. The presence of long documents necessitates the model capability of handling long-distance dependency in text context and events. The average of 1.8 trigger words per document indicates that there may be multiple events within single document, posing a challenge for event extraction models. The total number of unique roles is 469, suggesting good uniformity in role definitions within our dataset. Other details of the dataset can be found in Appendix A.

heading: 4 Our Pipeline Framework for CDEE
text: Based on our dataset, we propose a new crossdocument event extraction framework, which mainly consists of the following components: event extraction, coreference resolution, entity normalization, role normalization and entity-role resolution.
The framework architecture is illustrated in Figure 3. First, the documents in a collection are input into the event extraction module and output independent event extraction results for each document. Then, the coreference resolution module eliminates irrelevant events and clusters the event mentions of the same event. Subsequently, event arguments are normalized by the entity and role normalization modules respectively. Finally, the entity-role resolution module performs deduplication and conflict resolution for the cross-document event extraction results, and yields complete representations of the events in the document collection.

heading: 4.1 Document-level Event Extraction
text: We follow the approach of Zheng et al. (2019) to construct an entity-based directed acyclic graph (EDAG) from the event records to perform event extraction. The model is mainly divided into the following parts:
(1) Entity Extraction and Embedding: Named Entity Recognition (NER) is performed using BERT (Devlin et al., 2019) and CRF to obtain the embeddings of entities and sentences.
(2) Document-level Encoding: Transformer encoder and RST (Rhetorical Structure Theory) (Mann and Thompson, 1987) tree are used to make entities aware of document-level context.
(3) Event Type Classification: Event classification task is performed based on the document-level sentence representations obtained from the previous step.
(4) Event Role Extraction: A directed graph based on entities is constructed to determine the event roles between entities.
The specific details of the event extraction model are provided in Appendix B.1.

heading: 4.2 Event Coreference Resolution
text: To ensure that all events in the document collection refer to the same event, without including any other irrelevant noise, following Gao et al. (2024) ,we introduce a coreference resolution module to group the event collections belonging to the same event. Here, building upon the event representations enhanced with RST obtained in Section 4.1, we train a multi-layer MLP to serve as the coreference resolution model, and perform binary classification to determine whether two event mentions are coreferential. For two event mentions ei and ej output by the event extraction module, the coreference probability is calculated by:
p = MLP(mi,mj ,mi ·mj) (1)
where mi and mj are the vector representations of events ei and ej respectively. In our method, we compute coreference scores for all event pairs within the same document collection. We retain all event mentions with the coreference scores greater than 0.5 for subsequent steps.

heading: 4.3 Entity Normalization
text: We normalize the extracted entities by linking them to a knowledge base to ensure consistent representation of the same entity across different documents. We use the method proposed by Zhang (2023) as our entity linking module. This method integrates existing Chinese dictionaries and utilizes contextual information for word sense disambiguation. Entity normalization mainly involves the following steps:
(1) Entity Linking: Linking entities in the document to the corresponding entities in the knowledge base to address the issues of homonyms and synonyms. For example, “United States” and “USA” should be represented as the same entity.
(2) Entity Standardization: Standardizing various attributes of entities, such as unifying date formats (e.g., “January 1, 2023” and “01/01/2023” to a standard format) and standardizing location names. This can be implemented using existing time standardization tools.

heading: 4.4 Role Normalization
text: This module ensures that the terms describing the same type of event argument roles are consistent across different documents. For example, if “winner” and “victors” refer to the same type of roles in different documents, they should be normalized. Based on the specific information of the roles that
we gathered in Section 3.4, we manually designed a mapping dictionary for roles to ensure that all roles appearing in our dataset can be mapped to unified representations. Our dataset contains a total of 469 unique roles. Based on this, we collected existing roles to build a role-mapping table. During the normalization process, if a role is not in the role mapping table, we add it as a new role to the mapping table. Table 10 and Table 11 provide some examples.

heading: 4.5 Entity-Role Resolution
text: The entity-role resolution module aligns and consolidates the results of multi-document event extraction. This module performs two main operations: deduplication and conflict resolution. Deduplication, which ensures that only one instance of duplicated event mentions extracted from multiple documents is retained. Conflict resolution (1) Conflict resolution in event time and location arguments: We select the time and location arguments that appear most frequently across the documents. (2) Conflict resolution in argument roles: When the same entity is assigned with different roles, we resolve such conflict by constructing a hierarchical role selection mechanism, where the high-level role is selected. For detailed information about the hierarchical role selection mechanism, refer to Appendix B.2.

heading: 5 Experiments
text: Given that our proposed model follows a pipeline architecture, we designed three experiments to test the performance of each module: document-level event extraction experiment, event coreference resolution experiment, and cross-document event extraction experiment. These three experiments precisely reflect the three most important aspects of our framework. Through the document-level event extraction experiment, we can validate the effectiveness of incorporating RST. Through the event coreference resolution experiment, we can evaluate the accuracy of our coreference resolution module in removing irrelevant events. Through the cross-document event extraction experiment, we are able to show the effectiveness of our pipeline framework.

heading: 5.1 Document-Level Event Extraction
text: Baselines We choose Doc2EDAG (Zheng et al., 2019) as the baseline. Since Doc2EDAG can
only be used for document-level event extraction, we compared our document-level event extraction module with this approach to demonstrate the impact of introducing discourse-level information (e.g., RST). We also compared our method with the approach RAAT (Liang et al., 2022), which incorporates additional entity relations to enhance the model performance. Metrics We use recall, precision, and F1 score as evaluation metrics. We separately calculate the metrics for the tasks of event type classification and event role extraction. Results The experimental results are shown in Table 5. Compared to Doc2EDAG, our module achieved an F1 score improvement of 0.4 for event type classification and 4.6 for event role extraction, respectively. This is because we introduced RST to better model document information, and GAT can learn rich structural information contained in the RST tree. For the event type classification task, our dataset is domain-independent and only includes 9 major event types, making it less challenging. However, the event role extraction task requires rich document information. Our RST tree can provide rhetorical relationships between different clauses in the document, helping the model filter out noise. Compared to RAAT, it does not have any particular features, hence the results of event type classification are similar. However, for event role extraction, RAAT introduces additional entity relationship information, leading to a noticeable improvement. RAAT results are close to ours, indicating that both entity relations and discourse information can enhance document understanding of the model.

heading: 5.2 Event Coreference Resolution
text: To remove event information irrelevant to the theme events of the document collection, we conducted cross-document event coreference resolution experiments. Baseline We chose Yu et al. (2022) as the baseline, which determines coreference by enhancing event mention representations with event argument information. Metrics We utilized the agglomerative clustering algorithm for clustering and reported R, P, and F1 scores on the MUC, B3, CEAF, and CoNLL metrics. Results The experimental results are shown in Table 6. Our model achieved certain performance improvements across all metrics. This is because there is often irrelevant event argument informa-
tion at the document level. Extracting arguments such as locations and times may result in incorrect results, and the event arguments in documents often exhibit long-distance dependency phenomena. Thus, it is necessary to explicitly introduce the structural information of the document, distinguishing the roles of different clauses. The RST tree used in our document encoding module is able to alleviate this issue.

heading: 5.3 Cross-document Event Extraction
text: Baseline Since there are no existing works in the field of cross-document event extraction, we designed a rule-based baseline for comparison. This baseline operates on our coreference resolution results, and performs entity/role normalization and integration. For entity/role normalization, we used dictionary matching to standardize entities and roles. For role integration, we employed the principle of maximum count, selecting the role with the highest frequency for each entity as the final entity role. The details of this baseline can be found in our code. Metrics We use recall, precision and F1 as evaluation metrics for Event Type Classification task and Event Role Extraction task. For Corefercence resolution, we use MUC, B3, CEAF and CoNLL as evaluation metrics.
Results The experimental results are shown in Table 7. It can be observed that our method outperforms the rule-based baseline. This is due to the fact that the rule-based baseline does not consider the context of entity occurrences during entity/role normalization, leading to more errors. Additionally, constructing role hierarchies helps resolve conflicts and is superior to the maximum count method.

heading: 5.4 Experiments Using LLM
text: To further show the challenge of our dataset and the complexity of our task, we conducted additional experiments using LLMs. The choice to employ LLM in our experiments stems from their advanced capabilities in handling various NLP tasks, which are essential for tackling the intricate challenges presented by our dataset:
1. Cross-document context demands a model with well understanding capability of long text and different topics.
2. Complex task procedure asks a model for the ability of assembling various information extraction skills such as trigger extraction, entity normalization and conflict resolution.
Settings We used Llama2-Chinese-7b-Chat1 to finetune on our dataset with 4 A100-80G GPUs.
1https://github.com/LlamaFamily/Llama-Chinese
Input Prompt
To accomplish the cross-document event extraction task, you will be provided with multiple documents. Your objective is to extract event information from these documents, integrate the extracted results for multiple events, and perform entity and role normalization during the integration process. This involves linking entities and roles to a unified representation, while filtering out irrelevant event extraction results. Subsequently, you will merge multiple results into a comprehensive structured representation of events. The output format should be as follows: { "type": event type, "trigger": event trigger, "arguments": ["role": role1, "entity": entity1,"role": role2, "entity": entity2], . . . . . . } document_input: document1: {....}, document2: {....}, ...
The learning rate is set to 2e-6, and the batch size is set to 8. The input prompt template is show in Textbox Input Prompt. Results The experimental results are shown in Table 7. Note that for LLM we have a more flexible approach for evaluation metrics calculation, that is, if the predicted outputs are contained within the gold standards, we consider the result to be correct.
The experimental results show that the use of LLM leads to significant performance improvements. This is because the LLM has been pretrained on a large amount of general data, possessing substantial knowledge capabilities. Furthermore, our dataset is domain-agnostic and covers a wide range, which contributes to the good performance of the model. Additionally, we have found that using fully parameterized finetuning tends to overfit our task. Although the training loss decreases, the error rate is relatively high in the test set, especially when we increase the number of documents per event.
Furthermore, the model’s outputs are greatly influenced by the prompts. We observed that when limiting the model to output results in JSON format, it does not always comply as expected. Also, our prompts are constructed entirely in a zero-shot
manner, where all sample labels are in JSON format, yet the model does not always adhere to our specifications. Moving forward, we plan to explore training the LLM using a few-shot approach to see if we can further improve performance.

heading: 6 Conclusion
text: In this paper, we introduced a novel task of crossdocument event extraction. A large-scale dataset, CLES, is proposed based on Wikipedia and a benchmark pipeline is built for the comparison of follow-up work. Experiments show the feasibility and challenges of our task and dataset. Our work paves the way for a more complex and comprehensive understanding of events, highlighting the importance of multi-document analysis in capturing real-world events. Our work extends the scope of information extraction and will lead a new line of NLP research.

heading: Acknowledgments
text: This work is supported by the National Key Research and Development Program of China (No. 2022YFB3103602), the National Natural Science Foundation of China (No. 62176187), the open project of Sichuan Provincial Key Laboratory of Philosophy and Social Science for Language Intelligence in Special Education (No. YYZN-2023-1).
Limitations
The limitation of this work lies in the fact that it is the first attempt at addressing cross-document event extraction. The implementations of various modules in our framework are not yet perfect. In the future, we will further refine the quality of our dataset and improve our framework to make this work more solid.
Ethics Statement
This research on cross-document event extraction utilizes data from publicly available sources, specifically Wikipedia, ensuring no personally identifiable information is involved. We acknowledge the potential biases in publicly sourced data and have taken steps to mitigate them through diverse representation and manual verification. The dataset and methodologies used are detailed in this paper for transparency and will be publicly accessible to foster further research. We have adhered to ethical standards in research, ensuring compliance with institutional and national guidelines.

heading: A CLES Dataset
text: To provide a more detailed overview of our dataset, we present some statistical information about the dataset here.
To further analyze the distribution of trigger words and roles in our dataset, we have compiled the statistics for the trigger words with the highest and lowest frequencies of occurrence, as shown in Tables 8 and 9, respectively. We also have compiled the statistics for the roles with the highest and lowest frequencies of occurrence, as shown in Tables 10 and 11.
It can be observed that “obtain’‘ and “defeat" appear most frequently as trigger words, which is related to the fact that ATTACK type events are most common in our dataset. Additionally, “Date” and ‘Location” appear most frequently as event arguments, indicating that Date and location information are often essential arguments for events.

heading: B Cross Document Event Extraction Architecture
text: B.1 The Details of Document-Level Event Extraction
B.1.1 Entity Extraction and embedding Firstly, we need to perform Named Entity Recognition (NER) on the input document. We employ a sota model proposed by Wang et al. (2022c), which utilizes BERT (Devlin et al., 2019) with a Conditional Random Field (CRF) layer for NER. Given a document d = {s1, s2, ..., sn}, where si represents a sentence, NER processing yields an entity set E = {e1, e2, e3, ..., ej}, where ei represents an entity. Since an entity mention may consist of multiple tokens, we employ the maximum pooling result
of these tokens as the embedding for the entity mention, i.e., ei = max-pooling([hi,j , ..., hi,k]), where hi,j represents the representation of the j-th token of mention i. For each sentence si, we also adopt the maximum pooling method to obtain the embedding of each sentence, ci = [hi,1, ..., hi,n], where hi,j represents the token of the j-th token of sentence si.
B.1.2 Document-level Encoding In the previous section, we obtained embeddings for each entity and sentence, encoding only the contextual information within the sentence scope. However, without interaction among the sentences of the document, this local encoding may not be sufficient for direct event parameter extraction, as the event parameter information may be distributed across different sentences. Therefore, it’s necessary to make entities and sentences aware of the document-level context. To encode document information more effectively, we introduce discourse information to model document information. We construct an RST tree to represent the rhetorical relations between different clauses in the document. The document is divided into Elementary Discourse Units (EDUs), and the constructed RST is used for subsequent processing. We use two modules to learn document-level context information:
(1) Transformer Encoder: The entity embeddings and sentence embeddings obtained from section 4.1 are added with positional encodings and then fed into the transformer encoder for interaction between different entities and sentences. Et = [e1t , ..., e Ne t ] = transformer(e1, ..., eNe , c1, ..., cNs) + position encoding, and Ct = [c1t , ..., c Ns t ] = transformer(e1, ..., eNe , c1, ..., cNs) + position encoding. where t represents the transformer encoder, Ne represents the number of entities, Ns represents the number of sentences.
(2) GAT Module: We construct a graph based on the built RST tree and use Graph Attention Network (GAT) (Velickovic et al., 2017) to learn the information between different nodes, representing rich structural information between EDUs. Eg = [e1g, e 2 g, ...e Ne g ] = GAT({n1, n2, ..., nN}). Similarly, Cg = [c1g, c 2 g, ..., c Ns g ] = GAT({n1, n2, ..., nN}). where g represents the GAT, ni represents each node in the RST tree. Finally, eig takes the node representation of the EDU where entity ei is located, and cig similarly takes the node representation of the EDU
where sentence ci is located. The final entity representation is E = [e1t ⊕ e1g, ..., e Ne t ⊕ eNeg ], C = [c1t ⊕ c1g, ..., cNst ⊕ cNsg ]. where ⊕ represents concatenation operation.
B.1.3 Event Type Classification We perform max-pooling on the document-level encoding C obtained from the previous step to get the document embedding d. Then, we use a 3- layer Multi-Layer Perceptron (MLP) for event type
classification.
Pt = softmax(MLP(d)) (2)
Pt represents the probability of each event type. We select the event type corresponding to the highest probability as the final event type.
B.1.4 Event Role Extraction We follow the approach of Zheng et al. (2019) to construct an entity-based directed acyclic graph
(EDAG) from the table event records. For each event type, we manually define the sequence of event roles. Then, we transform each event record into a parameter chain list according to this sequence, where each parameter node is either an entity or a special empty parameter NA. By sharing the same prefix path, we merge these lists into the EDAG. We perform path extension on each leaf node of the EDAG. For each entity to be extended, we create a new node for the entity based on the current role and connect the leaf node with the new node. We implement path extension as a classification task. B.2 Role Hierarchy To address potential conflicts in event roles during the information integration process, we constructed an event hierarchy for the nine event types to resolve conflicts. When the same entity holds multiple roles, we select the highest-level role as the final result. The specific information about the role hierarchy is shown in Table 12, with levels ranging from Level 1 to Level 5 in decreasing order of priority. For more details, please refer to our code repository.

heading: Future Work
text: In the future, we will further refine the quality of our dataset and improve our framework to make this work more solid