# Type Hierarchy Enhanced Event Detection without Triggers
## 4 Experiments and Evaluation
### Overall Evaluation  
In this section, we analyze the results of the proposed framework (see Table 4). They show that the overall performance of this model is better than other baselines. Specifically, we can have the following points from the results.  
* Our approach is significantly improved compared with the traditional neural network models DMCNN and JRNN for event extraction methods. It indicates that bi-LSTM can enrich semantic information. Furthermore, we add part-of-speech features that also help the event detection task.
* THEED attains a much better performance improvement (about 4.5% and 2.5% in FI scores) compared with TBNNAM and TEXT2 EVENT, respectively, which do not have access to the trigger annotations or the trigger offsets. The results show that our proposed type hierarchy concept modular component is useful. Compared with the other methods of querying an event type/subtype in the \"embedding table\" to obtain a single semantic piece of information, the type hierarchy concept can effectively enrich the representation of event types. However, TEXT2 EVENT exploits the event schema to guide event generation and uniformly predicts all event types, The nonbalanced distribution of event types will have less impact on the model, so its recall is higher than THEED. However, the model is much more complex than THEED, and the F1 value is lower than THEED  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **P (\%)** & **R (\%)** & **F1 (\%)** \\ \hline DMCNN & 75.7 & 66.0 & 70.5 \\ JRNN & 66.0 & 73.0 & 69.3 \\ TBNNAM & 76.2 & 64.5 & 69.9 \\ TEXT2 EVENT & 69.6 & 74.4 & 71.9 \\ DEEB-RNN3 & 72.3 & 75.8 & 74.0 \\ OntoED & 76.6 & 75.0 & 76.2 \\ Query-And-Extract & - & - & 73.5 \\
**THEED (ours)** & 78.1 & 71.6 & 74.7 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Experimental results of event detection on ACE2005 corpus.  
Figure 5: Experimental results on test dataset with different settings of \(\lambda\).  
* Although our proposed approach does not use annotated triggers, it achieves improvements over other models, namely DEEB-RNN3 and Query-And-Extract. This is because our method adopts both word-level and context-level representations, which is helpful for the event detection task. On the other hand, DEEB-RNN3 exploits document embedding, which may learn the relationship between event types in the same document, leading to a higher recall than our method. However, the method requires annotated event information.
* THEED can outperform all compared methods except OntoED, which formulates event detection as a process of event ontology population; it enables the utilization and propagation of event knowledge, especially from high-resource to low-resource event types. Despite the lack of propagation and inference of knowledge, our model has a close F1 score to OntoED. However, our method requires fewer manual annotations.
* From the experimental results, we can observe a phenomenon that the recall of the model is relatively lower. This is because if a sentence contains two identical event types, such as \"Attack, Attack\", we treat them as two different events, while other models treat them as only one event. On the other hand, the nonbalanced distribution of the number of training events also affects the recall of our method.