# Extracting Domain-Specific Chinese Named Entities for Aviation Safety Reports: A Case Study
## 4 Experiments
### Experimental Results and Discussions
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Embedding** & **F1 Score** & **Time Per Iteration** &
\begin{tabular}{c} **Optimal Training** \\ **Duration** \\ \end{tabular} \\ \hline ERNIE & 0.9773 & 21.7 s & 617 s \\ ERNIE + general dictionary & 0.9770 & 22.6 s & 489 s \\ ERNIE + domain dictionary & 0.9785 & 21.9 s & 548 s \\ ERNIE + double dictionary & 0.9786 & 23.3 s & 608 s \\ ERNIE + double dictionary + Pinyin + Glyph & 0.9793 & 21.6 s & 730 s \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of experimental results of different embeddings for IDCNN-BiGRU-CRF. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Embedding** & **F1 Score** & **Time Per Iteration** &
\begin{tabular}{c} **Optimal Training** \\ **Duration** \\ \end{tabular} \\ \hline ERNIE & 0.9752 & 20.4 s & 381 s \\ ERNIE + general dictionary & 0.9755 & 20.0 s & 172 s \\ ERNIE + domain dictionary & 0.9767 & 21.9 s & 336 s \\ ERNIE + double dictionary & 0.9777 & 20.5 s & 889 s \\ ERNIE + double dictionary + Pinyin + Glyph & 0.9780 & 22.1 s & 271 s \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of experimental results of different embeddings for BiGRU-CRF. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Embedding** & **F1 Score** & **Time Per Iteration** &
\begin{tabular}{c} **Optimal Training** \\ **Duration** \\ \end{tabular} \\ \hline ERNIE & 0.9751 & 21.4 s & 501 s \\ ERNIE + general dictionary & 0.9744 & 20.6 s & 301 s \\ ERNIE + domain dictionary & 0.9752 & 21.8 s & 383 s \\ ERNIE + double dictionary & 0.9757 & 21.0 s & 473 s \\ ERNIE + double dictionary + Pinyin + Glyph & 0.9779 & 21.7 s & 489 s \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of experimental results of different embeddings for BiLSTM-CRF. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Embedding** & **F1 Score** & **Time Per Iteration** &
\begin{tabular}{c} **Optimal Training** \\ **Duration** \\ \end{tabular} \\ \hline ERNIE & 0.9752 & 19.6 s & 1013 s \\ ERNIE + general dictionary & 0.9766 & 19.6 s & 621 s \\ ERNIE + domain dictionary & 0.9769 & 21.1 s & 687 s \\ ERNIE + double dictionary & 0.9774 & 19.9 s & 714 s \\ ERNIE + double dictionary + Pinyin + Glyph & 0.9779 & 21.1 s & 816 s \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of experimental results of different embeddings for IDCNN-CRF. 'F1 score' is the F1 value score calculated via the model during testing. 'Memory' is the memory consumed via training the model. The results of different pre-training models are shown in Table 7. Comparing Word2 Vec with the current classic pre-training model, the F1 score has increased by approximately 3-4%. BERT is the basic pre-training model, and other pre-training models are improvements based on BERT. TinyBert is a distilled version of the BERT model, with fewer parameters and a faster training speed, but the pre-training model in the specific entity extraction model in this paper is average. Compared with BERT, RoBERTa has more parameters in the model, and proposes a dynamic mask. The F1 score of RoBERTa shows certain improvements in the specific entity extraction model compared with that of BERT. ERNIE integrates external knowledge sources, performs task-specific and domain-specific pre-training, and performs best in this experiment. In this section, we compare the performance of our approach with that of the existing feature fusion algorithm. In order to verify that the method of feature fusion used in the text is better, this paper designs two sets of traditional methods to directly add and splice vectors. The method used in this paper is compared with them. The results of different feature fusion methods are shown in Table 8. Compared with the traditional feature fusion method, the method in this paper has achieved better results. The Concat method directly fuses features, which lead to more redundant information, and the improvement of classification prediction is relatively limited.