# Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition
## 4 Experiments
### Experimental Results and Analysis
#### 4.4.1 Comparison with Baseline Models  
To evaluate the performance of the proposed method, we tested and analyzed the existing algorithms. In this study, we chose six open-source algorithms as baseline models: BiLSTM-CRF, Lattice LSTM [18], BERT-BiLSTM-CRF [26], BERT-FLAT [28], BERT-MECT [17], and LE-BERT [29]. Among them, BiLSTM-CRF is a widely used sequence labeling algorithm. Lattice LSTM fuses lexical information and character information on the basis of BiLSTM-CRF. BERT-BiLSTM-CRF uses the pre-trained model BERT to enhance the character representation of the input sentence. BERT-FLAT utilizes lexical information on the basis of BERT-BiLSTM-CRF. Additionally, it uses model-level fusion to integrate the lexical information and pre-trained model. BERT-MECT uses radical information based on FLAT  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Parameter** & **Parameter Value** \\ \hline Character embedding dimension & 768 \\ Lexicon embedding dimension & 200 \\ Radical embedding dimension & 200 \\ Initial learning rate & BERT of MMIEE-BERT: \(5\times 10^{-5}\) \\  & Others of MMIEE-BERT: \(1\times 10^{-4}\) \\  & LSTM layer: \(1\times 10^{-4}\) \\  & CRF layer: \(1\times 10^{-3}\) \\ Minimum learning rate & \(1\times 10^{-5}\) \\ LSTM layer number & 2 \\ LSTM hidden size & 300 \\ Training epochs & 20 \\ Training batch size & 4 \\ Maximum sequence length & 200 \\ Dropout ratio & 0.5 \\ Optimizer & Adam [40] \\ \hline \hline \end{tabular}
\end{table}
Table 6: Parameter setting of the proposed model.  
to further enhance the character representation. It also uses a model-level fusion approach. LE-BERT has BERT learn the lexical information to accomplish CNER. In contrast to the above methods, LE-BERT is a sub-model-level fusion method. The proposed method integrated external information with the pre-trained model using sub-model-level fusion. Finally, the character embedding of all models without BERT was generated by random initialization. For each model, we ran the experiment five times to acquire stable experimental results. The averages and standard deviations of the evaluation metrics are reported in Table 7.  
Several observations can be drawn from Table 7. Firstly, the BiLSTM-CRF method performed poorly in recognition. The reason was that it did not use external information or a pre-trained language model to enhance the character representation. Secondly, we can see that adding lexical information to character-based models can improve the performance in recognizing mechanics entities. Compared with BiLSTM-CRF, the micro-averaged F1 score of Lattice LSTM was improved by 1.34 percentage points. Thirdly, the F1 score of BERT-BiLSTM-CRF was significantly improved after using the pre-trained language model BERT on the basis of Bi-LSTM-CRF. This demonstrates that BERT is very helpful in recognizing named entities in the field of mechanics. Fourth, comparing the recognition of Lattice LSTM and BERT-BiLSTM-CRF, we found that the pre-trained language model BERT could improve the recognition ability of the model more than lexical information. Fifth, the comparison results of BERT-BiLSTM-CRF, BERT-FLAT, and BERT-MECT demonstrate that the integration of BERT and lexical information through model-level fusion achieved a higher F1 value than using BERT alone. Sixth, the comparison results of BERT-FLAT, BERT-MECT, and LE-BERT demonstrate that sub-model-level fusion has more advantages than model-level fusion in extracting mechanics entities. Finally, the recognition performance of the proposed MMIEEE-BERT model was the best, achieving a 91.57% F1 score. This was 0.93 percentage points higher than the F1 score of LE-BERT and 1.24 percentage points higher than the F1 score of BERT-MECT. This demonstrates that our model made a significant improvement in recognizing mechanics entities by integrating multi-embedding information into the underlying transformer of BERT through the information fusion layer.