# Chinese Document-Level Emergency Event Extraction Dataset and Corresponding Methods
## 5 Experiments
### Ablation Experiment  
To validate the effectiveness of our improvements in the model, we conducted ablation experiments on certain modules. Firstly, we performed ablation experiments on the feature fusion of candidate entity role information. Previously, we had also removed the role information features of candidate entities in various baseline models and obtained better experimental results compared to the source model. Therefore, we also need to verify whether the fusion of role information features positively or negatively affects our overall task. Secondly, due to the use of the pre-trained language model RoBERTa-wwm, we need to verify whether the overall performance improvement in the task is solely attributed to the pre-trained language model. It is well known that pre-trained language models significantly improve performance in various natural language processing tasks. In our case, to address the overlapping problem of argument entity role information, we required more granular information and thus adopted a pre-trained language model, RoBERTa-wwm, which can capture more text information. We made this pre-trained language model an important component of our model. For various reasons, we also conducted ablation experiments on this part. However, in order to support our need for obtaining more granular information, we did not remove the pre-trained language model, but rather added it to the  
Figure 9: Comparison of candidate entity extraction results.  
baseline model, DE-PPN. In general, we compared the results of the DE-PPN model, which does not incorporate role type embedding and pre-trained language model modules, with the RODEE module we built and the following two ablation modules:  
* \(-\)RoleType: removes the role type embedding module from the RODEE model.
* \(+\)BERT: adds the RoBERTa-wwm pre-trained language model to the DE-PPN baseline model.  
In Table 4, we present the results of our model -RoleEmb after removing the candidate entity role information features, as well as the baseline model DE-PPN with the addition of the same pre-trained language model +BERT that we used. It is evident that our model still maintains its advantageous position compared to the aforementioned two models. Our model achieved a 1.1-point improvement in the F1 score compared to the -RoleEmb model, which removes the candidate entity role information feature. This demonstrates that incorporating correct candidate entity role information plays a beneficial role in our DEE task. Moreover, as shown in Section 5.1, incomplete or incorrect candidate entity role features can negatively impact the overall performance of document-level event extraction. Furthermore, we can observe that DE-PPN achieved a 6.9-point increase in the F1 score after incorporating a pre-trained language model. However, when compared to our RODEE model, there was still a 3.2-point gap in the F1 score. We can conclude that although the powerful performance of the pre-trained language model is leveraged in our model, our proposed model improvements for the candidate entity role overlapping problem still contribute significantly to the performance of the DEE task.