# Chinese Document-Level Emergency Event Extraction Dataset and Corresponding Methods
## 3 Chinese Document-Level Emergency Event Extraction Dataset
### Dataset Construction  
The construction of the event extraction dataset can be divided into two parts: the development of the event extraction structure and the event extraction data annotation. The event extraction structure, as the basis of the event extraction task, specifies the structured information to be annotated and extracted. Therefore, the characteristics of the information to be annotated and the characteristics of the event extraction task should be considered in the process of developing the event extraction structure.  
On the one hand, we analyzed and summarized the 5000 collected news reports and found that there were numerous types of emergency events, making it impractical to enumerate them all. Finally, we classified the emergencies based on whether the emergencies were caused by natural factors, and the original type of emergencies became a role attribute of the \"HazardType\" emergencies. At the same time, four types of events were defined based on the emergency event itself, the casualties resulting from the emergency event, and the rescue efforts by the organization. These events were categorized as \"InjureHead\", \"Rescure\", \"Accident\", and \"NaturalHazard\", and each event is described in Appendix A.  
On the other hand, in the process of developing event extraction structure, the verb that causes a change in things or states is generally used as the trigger word, and the time and place of the event as well as the participants are used as the key factors of the event. However, in the research related to DEE, it can be found that there are often events in a chapter that do not have obvious trigger words or do not contain trigger words, so the trigger-word-based event extraction method does not work well in DEE. Therefore, [PERSON] et al. [9] redesigned the DEE task and constructed ChFinAnn, a DEE dataset without trigger words, and validated its performance. The experimental results show that this trigger-word-free construction method met the requirements of the DEE task and greatly improved the efficiency of the data annotation work. Since our study also focused on document-level tasks, we adopted this trigger-free approach to formulate the event extraction structure.  
Considering the above two aspects together, we defined 19 event role types to describe event information. In Appendix A, we introduce each role and provide an explanation of the event type to which it belongs.  
Figure 2: Statistics on the number of various emergency events.  
After the development of the event extraction structure, we used manual annotation to annotate the 5000 emergency event documents we collected according to the developed event extraction structure. We annotated more than 10,000 event information in total, and divided the annotated data into a training set, validation set, and test set according to a ratio of 8:1:1. Figure 3 shows the statistical information about the number of events in each category after dividing the dataset.  
In the process of data annotation, we made the following arrangements in the annotation process to ensure the reliability and accuracy of the annotated data: the annotated data of annotator A will be given to annotator B for validation, the annotated data of annotator B will be given to annotator C for validation, the data of annotator C will be validated by annotator A, and so on. Additionally, in order to further enhance the verification process, we employed the voting method to resolve disagreements during the verification process. After manually verifying the dataset, we proceeded to further assess its quality by utilizing the existing DEE model for final verification, as described in the experimental section.