# Chinese Document-Level Emergency Event Extraction Dataset and Corresponding Methods
## 5 Experiments
### Comparison Experiment and Result Analysis  
Given that our dataset follows the concept of triggerless word annotation, we utilized the following model as a baseline model for comparison experiments as well as a quality validation model for the dataset:  
* Doc2 EDAG: An end-to-end model that converts DEE into a table-population task, directly populating event tables with entity-based paths for extensions. * GreedyDec: This model is a baseline model in the Doc2 EDAG model that uses a greedy strategy to populate against an event table. * DE-PPN: This model uses multi-granularity decoders for parallel extraction of events to improve the speed of event extraction while effectively addressing the challenges of multiple events and argument scattering of document-level events. In addition, to verify the quality of the dataset, we conducted an experiment involving manual extraction of document-level event information. For human evaluation, we randomly selected 1000 events and invited three individuals to manually extract the event information. The final result is an average of the three individuals' findings. Based on the experimental setup in Section 5.1, we used a manual approach to extract event information and analyze it in comparison to the results of the baseline model on the CDEEE dataset. Simultaneously, we trained our proposed model RODEE and analyzed the results of RODEE against the baseline model experiments under the same experimental conditions. It should be noted that in the experimental results below, we highlight in bold the data from the human evaluation (Human), the RODEE model, and other baseline models that outperform the RODEE model. Tables 2 and 3 show the results obtained by each model on the CDEEE dataset for each event type, as well as the overall experimental results. We can observe that the scores achieved by humans on the CDEEE dataset were much higher than those of the existing DEE models. On the one hand, this indicates the high quality of our labeled dataset, and on the other hand, it also indicates that there is still more room for improvement in the DEE task. Considering that the existing DEE model methods use sequence annotation to complete the candidate entity prediction task in the candidate entity extraction stage and embed the role types of candidate entities to assist in the DEE task, our CDEEE dataset is annotated for the candidate entity role overlapping problem. Therefore, we modified the baseline model by removing the role type embedding module from the baseline model and naming the modified baseline model as Doc2 EDAG*, GreedyDec*, and DE-PPN*. By observing Tables 2 and 3, it can be seen that in the CDEEE dataset annotated with overlapping role issues, the model approach that employs embedding of individual role types to assist in completing the DEE task performed lower overall compared to the DEE model approach that does not involve role type embedding. Therefore, we believe that incomplete embedding of role type information not only fails to enhance the overall performance of the DEE task but may also have negative effects, potentially reducing the performance of the DEE task to some extent. We can also observe from Tables 2 and 3 that the overall performance of our proposed RODEE model was better than that of the existing DEE task model, with an improvement of 7 percentage points in the precision P of our model compared to the best-performing DE-PPN model among the baseline models, and compared to the best-performing Doc2 EDAG* model among the baseline models in terms of F1 values, our model's F1 value improved by 3.9 points. In addition, we also observed that the recall R and F1 values for the \"InjureDead\" event were lower than those of the Doc2 EDAG* model, which we attribute to the fact that