# DEEDP: Document-Level Event Extraction Model Incorporating Dependency Paths
## 5 Experiments and Analysis
### Results on Single-Event and Multi-Event  
Following previous works [8; 11; 12], and we performed experiments on a single-record set (S.), in which only one event is described in a document, and a multi-record set (M.), in which multi event records are embedded in a document. In other words, single-record set refers to documents in this set containing only one event record, and multi-record set refers to documents that contain multiple event records. The comparison results for different scenarios are listed in Table 4. We can observe that (1) it is a more challenging task for multiple event record extraction compared with single-record event extraction, and as a result, the extraction performance of all models drops significantly; (2) however, our model, DEEDP, still achieves a 23.2, 2.2, 2.6, 3.6 average Micro-F1 score over DCFEE-M, DE-PPN, MMR, and Doc2 EDAG, respectively; (3) DEEDP-O outperforms one event record baselines on the single-record set, demonstrating the effectiveness of the strategies proposed in this work; (4) but DEEDP-O only produces one event record and, as a result, it works poorly in the scenario of a multi-record set, and the performance decreases significantly compared with testing on the single-event set.  
In conclusion, we find that a multi-granularity encoder can model more contextualized information for multi-record event extraction than models that apply only sentence-level and document-level encoders, such as Doc2 EDAG and DE-PPN. In addition, explicitly modeling the syntactic features of the input sentences can compensate or enhance some features discarded in the mean pooling or max pooling operations, benefiting information aggregation in the sentence-document feature fusion stage. Thus DEEDP achieves a higher F1 score than MMR.  
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{4}{c}{**EF**} & \multicolumn{4}{c}{**ER**} & \multicolumn{4}{c}{**EU**} & \multicolumn{4}{c}{**EO**} & \multicolumn{4}{c}{**EP**} \\ \cline{2-13}  & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** \\ \hline DCFEE-O & 66.0 & 41.6 & 51.1 & 84.5 & 81.8 & 83.1 & 62.7 & 35.4 & 45.3 & 51.4 & 42.6 & 46.6 & 64.3 & 63.6 & 63.9 \\ DCFEE-M & 51.8 & 40.7 & 45.6 & 83.7 & 78.0 & 80.8 & 49.5 & 39.9 & 44.2 & 42.5 & 47.5 & 44.9 & 59.8 & 66.4 & 62.9 \\ GreedyDec & 79.5 & 46.8 & 58.9 & 83.3 & 74.9 & 78.9 & 68.7 & 40.8 & 51.2 & 69.7 & 40.6 & 51.3 & **85.7** & 48.7 & 62.1 \\ Doc2 EDAG & 77.1 & 64.5 & 70.2 & 91.3 & 83.6 & 87.3 & 80.2 & 65.0 & 71.8 & 82.1 & 69.0 & 75.0 & 80.0 & 74.8 & 77.3 \\ DE-PPN & 77.8 & 55.8 & 64.9 & 75.6 & 76.4 & 76.0 & 76.4 & 63.7 & 69.4 & 77.1 & 54.3 & 63.7 & 85.5 & 43.0 & 57.2 \\ DE-PPN & 78.2 & **69.4** & 73.5 & 89.3 & 85.6 & 87.4 & 69.7 & **79.9** & 74.4 & 81.0 & 71.3 & 75.8 & 83.8 & 73.7 & 78.4 \\ MMR-1 & 81.2 & 48.7 & 60.9 & 82.9 & 73.2 & 77.8 & 81.2 & 45.1 & 58.1 & 75.5 & 45.8 & 57.0 & 84.3 & 50.8 & 63.4 \\ MMR & 78.4 & 65.5 & 71.3 & 89.3 & 88.1 & 88.7 & 79.5 & 66.4 & 72.4 & 83.5 & 71.4 & 76.9 & 82.3 & 74.1 & 78.0 \\ \hline DEEDP-O & **83.5** & 50.1 & 62.9 & 77.3 & 65.2 & 70.7 & 80.5 & 58.7 & 67.9 & **84.1** & 62.6 & 71.8 & 84.9 & 54.3 & 66.2 \\ DEEDP & 79.6 & 68.9 & **73.9** & **92.5** & **89.7** & **91.1** & **82.1** & 71.8 & **76.6** & 81.3 & **73.5** & 77.2 & 82.6 & **75.5** & **78.9** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Overall event-level precision (P), recall (R), and F1 scores evaluated on the ChFinAnn test set.