# Chinese Document-Level Emergency Event Extraction Dataset and Corresponding Methods
## 5 Experiments
### Comparison Experiment and Result Analysis
\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**InjureDead**} & \multicolumn{3}{c}{**Resecure**} & \multicolumn{3}{c}{**Accident**} & \multicolumn{3}{c}{**NaturalHazard**} \\  & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline GreedyDec & 79.5 & 66.0 & 72.2 & 62.2 & 26.2 & 36.9 & 79.5 & 70.4 & 74.7 & 73.7 & 71.2 & 72.4 \\ GreedyDec* & 86.8 & 69.1 & 77.0 & 65.5 & 24.3 & 35.4 & 80.6 & 73.6 & 76.9 & 76.4 & 72.6 & 74.4 \\ Doc2 EDAG & 85.8 & 65.3 & 74.2 & 75.5 & 47.0 & 58.0 & 82.8 & 69.4 & 75.5 & 78.0 & 74.6 & 76.2 \\ Doc2 EDAG* & 85.8 & **72.0** & **78.3** & 78.5 & **50.8** & **61.7** & 78.1 & **74.7** & 76.4 & 75.5 & 76.3 & 75.9 \\ DE-PPN & 87.4 & 46.5 & 60.7 & 72.0 & 31.9 & 44.2 & 89.8 & 65.9 & 76.0 & 84.4 & 77.6 & 80.8 \\ DE-PPN* & 85.8 & 46.2 & 60.0 & 69.0 & 32.1 & 43.8 & 89.6 & 66.5 & 76.4 & 85.1 & 78.6 & 81.7 \\ \hline
**RODEE** & **95.8** & **54.2** & **69.2** & **79.4** & **44.5** & **57.0** & **96.4** & **74.5** & **84.1** & **94.7** & **89.6** & **92.1** \\
**Human** & **99.9** & **95.2** & **97.4** & **95.4** & **94.5** & **94.9** & **98.5** & **94.8** & **96.6** & **98.7** & **96.6** & **97.6** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Experimental results of DEE models for various events in the CDFEE dataset. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Precision** & **Recall** & **F1** \\ \hline GreedyDec & 76.7 & 60.3 & 67.5 \\ GreedyDec* & 79.4 & 62.1 & 69.7 \\ Doc2 EDAG & 81.4 & 64.5 & 71.9 \\ Doc2 EDAG* & 78.9 & 69.3 & 73.8 \\ DE-PPN* & 86.2 & 56.9 & 68.6 \\ DE-PPN* & 85.4 & 57.3 & 68.6 \\ \hline
**RODEE** & **93.2** & **66.6** & **77.7** \\
**Human** & **98.5** & **95.2** & **96.8** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Overall experimental results of DEE in CDFEE dataset. event \"InjureDead\" has only four role types and a low role overlapping rate, and thus the performance of our proposed model for the candidate entity role overlapping problem is slightly lower than that of the Doc2 EDAG* model for this class of events. Additionally, the recall rate R and F1 score for the \"Rescure\" event in the Doc2 EDAG* model were slightly higher than those of our model. We believe that although the \"Rescure\" event has more role types compared to the \"InjureDead\" event, the overall role overlap in this event is relatively low, which leads to the aforementioned phenomenon. To further analyze the performance of RODEE, we also conducted experiments on the candidate entity extraction subtask, and the results are shown in Figure 9. As can be seen in Figure 9, RODEE not only outperformed the other models on the DEE task, but also on the candidate entity extraction subtask. Compared with other models, RODEE improved at least 11 percentage points on the F1 score for the candidate entity extraction subtask. We believe this is due to the fact that more deep information about the text is obtained to improve the performance of the model for candidate entity extraction. This demonstrates that the performance of candidate entity extraction, as the first subtask of the DEE task, has a significant impact on the overall DEE task, and the improvement in the candidate entity extraction subtask has a positive effect on the DEE task.