# Multi-Meta Information Embedding Enhanced BERT for Chinese Mechanics Entity Recognition
## 4 Experiments
### Experimental Results and Analysis
#### 4.4.4 Ablation Experiment  
In this section, we present the two ablation experiments conducted using MMIEE-BERT to evaluate the influence of different structures and components on the model's capability, i.e., the influence of input features (IF experiment) and the influence of different embedding layers and encoding layers (EE experiment). To evaluate the influence of input features on the proposed model, an IF experiment was conducted. Specifically, the four groups of experiments presented in Figure 9 were constructed. For Experiment 1 (Base), BERT, lexical embedding, and radical embedding were deleted on the basis of MMIEEEE-BERT. For Experiment 2 (w/o LR), both lexical features and radical features were deleted on the basis of MMIEEEE-BERT. For Experiment 3 (w/o R), radical features were deleted on the basis of MMIEEEE-BERT. For Experiment 4 (w/o L), lexical features were deleted on the basis of MMIEEEE-BERT. The main results of the ablation experiments are reported in Table 11.  
From the experimental results, we found that the combination of BERT, lexical information, and radical information produced the most significant improvement in recognition performance. Among them, the use of BERT alone caused the most obvious improvement in model performance, greater than the improvement brought by the use of lexical embedding or radical embedding alone. Additionally, compared with radical information, lexical information brought a greater improvement in the model's performance. In summary, each feature proposed in this article could improve the recognition performance. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Resume**} & \multicolumn{2}{c}{**Yidu-54K**} & \multicolumn{2}{c}{**MSRA**} & \\ \cline{2-9}  & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** \\ \hline BiLSTM-CRF & 92.85 & 93.19 & 93.02 & 68.99 & 71.96 & 70.45 & 91.54 & 90.52 & 91.03 \\ Lattice LSTM & 93.22 & 93.68 & 93.45 & 72.50 & 70.94 & 71.71 & 93.99 & 91.94 & 92.96 \\ BERT-BiLSTM & 96.52 & 95.40 & 95.96 & 74.22 & 76.92 & 75.55 & 95.62 & 94.53 & 95.07 \\ BERT-FLAT & 96.72 & 95.77 & 96.24 & 74.73 & 78.11 & 76.38 & 96.29 & 95.28 & 95.78 \\ BERT-MECT & 96.91 & 96.07 & 96.49 & 74.58 & 78.32 & 76.41 & 96.66 & 95.58 & 96.12 \\ Le-BERT & 96.97 & 96.32 & 96.65 & **74.74** & 78.87 & 76.75 & **96.73** & 95.54 & 96.13 \\ Proposed & **96.98** & **96.50** & **96.74** & 74.57 & **79.54** & **76.98** & 96.71 & **95.99** & **96.35** \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of recognition effect of each model on different datasets. In the table, **P** denotes precision, **R** denotes recall, and **F** denotes **F1** score. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Type** & **Training** & **Dev** & **Test** \\ \hline \multirow{2}{*}{Resume} & Sentences & 3.80 k & 0.46 k & 0.48 k \\  & Entities & 1.34 k & 0.16 k & 0.15 k \\ \hline \multirow{2}{*}{Yidu-54 k} & Sentences & 5.67 k & 3.12 k & 2.82 k \\  & Entities & 11.69 k & 5.97 k & 6.00 k \\ \hline \multirow{2}{*}{MSRA} & Sentences & 46.4 k & - & 4.4 k \\  & Entities & 74.8 k & - & 6.3 k \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Statistics of the three datasets. The EE experiment mainly evaluated the influence of different encoding layers and embedding layers on recognition performance. For the embedding layer, we chose BERT-Base, BERT-Lexicon-ML (BERT-L-M), BERT-Radical-ML (BERT-R-M), BERT-Lexical-Radical-ML (BERT-LR-M), and MMIIEE-BERT. Among them, BERT-Base is the basic BERT model, BERT-L-M has an embedding layer that fuses the BERT model with lexical information in the model-level manner, BERT-R-M has an embedding layer that fuses the BERT model with radical information in the model-level manner, and BERT-Lexical-Radical-ML fuses the BERT model with lexical information and radical information in the model-level manner.