# Extracting Domain-Specific Chinese Named Entities for Aviation Safety Reports: A Case Study
## 4 Experiments
### Experimental Results and Discussions  
In this section, we compare the performance of our approach with that of existing state-of-the-art algorithms as well as that of baseline methods. In order to verify the effectiveness of the model, the impact of the model structure and the domain dictionary embedding method on the specific entity extraction task of textual datasets is discussed. BiLSTM-CRF, BiGRU-CRF and IDCNN-CRF are designed and compared with the IDCNN-BiLSTM-CRF structure in the paper. On this basis, a comparison of ERNIE embedding, ERNIE embedding with a general dictionary, ERNIE embedding with a domain dictionary, ERNIE embedding with a double dictionary and the embedding of our model is proposed. 'Embedding' is the method used by the model in the embedding layer. 'F1 score' is the F1 value score calculated by the model during testing. 'Time per iteration' is the time it takes for one epoch of the model during training. 'Optimal training duration' is the time it takes to train to the optimal model. All of the above models perform relatively well in Tables 3-6. Overall, the effect of the ERNIE embedding model integrated with double word embeddings, pinvin embedding and glyph embedding is better than that of other embeddings, and is improved by 0.03-0.35%. The optimal training time is generally reduced. There is some increase in the single iteration time due to there being more features. BiLSTM-CRF, BiGRU-CRF, and IDCNN-CRF are classic methods in named entity recognition that can achieve good results. Compared to the BiLSTM model, BiGRU requires less training time and consumes less resources while \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Batch\_size & 64 \\ Epoch & 200 \\ learning\_rate & 0.0001 \\ BiGRU\_hidden\_size & 300 \\ kernel\_size & 3 \\ num\_filters & 128 \\ Blocks & 4 \\ dilation\_l & [1,1,2] \\ embedding\_num & 768 \\ BiGRU\_num\_layer & 2 \\ IDCNN\_input\_dropout & 0.5 \\ IDCNN\_hidden\_dropout & 0.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Model parameters. its performance can reach the same level as that of BiLSTM. Compared to BiGRU, the IDCNN consumes more time during training, indicating that BiGRU is the optimal choice for single-channel models when extracting aviation safety reports. In addition, the F1 score of the IDCNN-BiGRU hybrid model used in this paper is better than that of other single models, indicating that the hybrid model can extract features from different angles and has better research significance. When keeping the feature extraction structure unchanged, the ERNIE word embedding vector in the embedding layer is compared with other character and word fusion vectors. If a general dictionary is used to train word vectors, it is difficult to accurately partition the words in aviation safety texts, resulting in a slight decrease in the performance of the model. Replacing a regular dictionary with a domain dictionary can improve the performance of the model by accurately partitioning the text, resulting in better results than those of the ERNIE word embedding vector. If a general dictionary and domain dictionary are used together, not only can specialized terms related to a specific field be identified, but also, the common dictionary has richer feature representations, further improving the results. Adding pinyin feature vectors and glyph feature vectors to the character embedding vector can also improve the F1 score to some extent. In this section, we compare the performance of our approach with existing advanced pre-training models. To verify the effectiveness of our model, the impact of different pre-trained models on specific entity extraction tasks in textual datasets is discussed. The traditional Word2 Vec and the pre-training models ERNIE-1.0, ERNIE-3.0, Bert-base-Chinese, TinyBert and RoBERTa are used for comparison. 'Model' is the pre-trained model used.