## 4 Experiments Results and Analysis
### Analysis of Experimental Results  
Using the trained parameters as model parameters for the validation and test sets, two experiments were done in order to verify the effectiveness of the BERT-MF-BiGRU-CRF model on the shipping enterprise credit dataset. This paper focuses on the BiGRU-CRF model, BiLSTM-CRF model, Word2 Vec-BiGRU-CRF model, BERT-BiLSTM-CRF model, BERT-BiGRU-CRF model, and BERT-MF-BiGRU-CRF model. The same training framework and parameters are used for each model, and each model is defined as follows:  
1. Model in this paper (BERT-MF-BiGRU-CRF): use BERT to obtain character features; sort that may be composed according to word matching into the sets B, M, E, and S; obtain word vector features, word nature features, and word length features; and stitch the obtained features as the input of the BiGRU-CRF model. 2. HMM model: borrowing from the cascading HMM model proposed by [PERSON] et al. (2017), the model consists of two parts, the underlying HMM and the adapted HMM. The former is used to recognize non-nested entities, and the latter is used to recognize nested entities. 3. BiLSTM-CRF model: a classical model in the field of named entity recognition. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline GRU dimension per layer & 150 \\ Dropout & 0.5 \\ Optimizer & Adam \\ Learning Rate & 0.0005 \\ Batch size & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Model parameters. * SVM-BiLSTM-CRF model: borrowing from the SVM-BiLSTM-CRF model proposed by [PERSON] et al. [33]. It has an additional step compared with the BiLSTM-CRF model, i.e., using SVM to filter out sentences containing named entities. * BiGRU-CRF model: similar to the BiLSTM-CRF model, BiGRU as a variant of BiLSTM with a simpler model structure. * BERT-BiLSTM-CRF model: Based on the BiLSTM-CRF model, BERT is added, and word vectors are obtained by BERT pre-training. * Word2 Vec-BiGRU-CRF model: Word vectors are obtained by Word2 Vec, and the word vectors are input to the BiGRU layer for contextual feature extraction, and finally, sequence annotation is performed by CRF. * BERT-BiGRU-CRF model: character encoding is obtained by BERT, the word vector is input to the BiGRU layer for contextual feature extraction, and finally, the sequence is annotated by CRF. The results of NER experiments for different models are shown in Table 5 below. The precision, recall, and F1 values of the BiLSTM-CRF model only reached 69.84%, 68.16%, and 69.01% on the shipping firm credit dataset. The BiGRU-CRF model has a simpler structure compared to the BiLSTM-CRF model, and the recognition on the shipping enterprise credit dataset is a little better, with an accuracy improvement of 1.19%; although the recognition effect is improved, there is great room for improvement. In order to verify the effectiveness of Word2 Vec and BERT for character-level feature extraction in shipping enterprise credit text, a comparison experiment between Word2 Vec-BiGRU-CRF and BERT-BiGRU-CRF was conducted. The experimental results show that the precision, recall, and F1 values of the Word2 Vec-BiGRU-CRF model for entity recognition are 73.35%, 72.74%, and 72.99%, respectively, and the precision, recall and F1 values of the BERT-BiGRU-CRF model for entity recognition are 84.54%, 82.14%, and 83.38%, respectively. It can be clearly seen that the BERT-BiGRU-CRF model entity recognition is better with significantly higher precision and recall, and the F1 value is improved by 10.39%. With the same base model, it can be concluded from this experiment that BERT is more suitable for shipping enterprise credit text character-level feature extraction. By adding the BERT pre-training model to the BiLSTM-CRF model to form the BERT-BiLSTM-CRF model, compared to the BiGRU-CRF model and BiLSTM-CRF model, the accuracy, recall, and F1 values were improved by 14.66% and 9.68%, 13.58% and 10.27%, and 14.03% and 9.79%, respectively, which are all three significant improvements.