# Chinese Named Entity Recognition Based on Boundary Enhancement with Multi-Class Information
## 4 Experiment
### Ablation Study  
In order to determine the influence of the four modules of multi-class information embedding, syntax analysis layer, CTAT, and gate mechanism on the model performance, an ablation study with one of the four modules was conducted and is described below. The experiments were conducted on the MSRA and AISHELL datasets. The results are shown in Figure 8. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{4}{c}{**CLUENER (\%)**} & \multicolumn{4}{c}{**BOSON (\%)**} \\ \cline{2-7}  & **P** & **R** & **F1** & **P** & **R** & **F1** \\ \hline Char-Based & 83.05 & 49.49 & 62.21 & 67.45 & 57.57 & 62.12 \\ Word-Based & 75.52 & 54.55 & 63.34 & 72.93 & 60.01 & 65.84 \\ BiLSTM+CRF & 70.26 & 66.60 & 68.38 & 69.98 & 67.74 & 72.36 \\ IDCNN+CRF & 68.09 & 64.03 & 66.00 & 69.04 & 66.22 & 67.60 \\ Lattice LSTM+CRF & 74.08 & 73.19 & 73.63 & 78.73 & 70.52 & 74.40 \\ SoftLexicon & 76.56 & 75.84 & 76.20 & 80.38 & 78.04 & 79.19 \\ MECT & 84.53 & 80.67 & 82.55 & 86.63 & 82.32 & 84.42 \\ KVMN & 87.18 & 86.30 & 86.74 & 89.03 & 87.30 & 88.15 \\ BEMCI & 87.62 & 88.09 & **87.85** & 91.14 & 89.46 & **90.29** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The result of [PERSON] and BOSON. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Models**} & \multicolumn{4}{c}{**WEIBO (\%)**} & \multicolumn{4}{c}{**AISHELL (\%)**} \\ \cline{2-7}  & **F1(NE)** & **F1(NM)** & **F1(ALL)** & **P** & **R** & **F1** \\ \hline Char-Based & 46.03 & 55.26 & 52.67 & 85.79 & 85.12 & 85.45 \\ Word-Based & 50.39 & 60.97 & 56.70 & 86.12 & 85.23 & 85.67 \\ BiLSTM+CRF & 57.48 & 63.05 & 58.83 & 89.03 & 87.27 & 88.44 \\ IDCNN+CRF & 57.36 & 61.98 & 57.72 & 90.06 & 85.69 & 87.50 \\ Lattice LSTM+CRF & 53.04 & 62.25 & 58.79 & 91.48 & 86.29 & 88.81 \\ SoftLexicon & 56.46 & 62.13 & 61.35 & 91.40 & 89.89 & 90.30 \\ MECT & 60.07 & 62.29 & 63.30 & 91.47 & 89.96 & 90.71 \\ KVMN & **63.34** & **67.91** & **68.43** & 92.57 & 90.20 & 91.37 \\ BEMCI & 60.10 & 67.53 & 66.11 & 92.11 & 91.66 & **91.88** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The result of WEIBO and AISHELL. Figure 8 shows that without the multi-class information embedding, the F1-scores decrease by 2.50% and 0.97% on the MSRA and AISHELL datasets, respectively. Without the syntax analysis layer, the F1-scores decrease by 1.07% and 0.65%. Without CTAT, the F1-scores decrease by 1.31% and 0.51%. Without the gate mechanism, the F1-scores decrease by 0.22% and 0.16%. Based on the differences in F1-score reductions, it can be concluded that all four modules contribute to enhancing the entity recognition performance. The multi-class information embedding has the strongest contribution, as it adds additional information for capturing deeper semantic meaning; CTAT is the second most influential, and the syntax analysis layer ranks third. Although the gate mechanism has the smallest contribution, the decrease indicates the importance of balancing the weights for contextual and syntax information. To determine the impact of the three additional types of embeddings on the model, we can conduct experiments using four different settings: using only character embedding, using character and word embeddings, using character and radical embeddings, and using character and pinvin embeddings. By comparing the performance of these four settings, we can assess the contribution of each type of embedding in improving the model's recognition effectiveness. The character embedding serves as the baseline embedding, while the other three embeddings provide additional semantic information to enhance the model's recognition performance. The results of the BEMCI embedding ablation experiments are shown in Figure 9.  
Figure 8: Ablation result. Figure 9: Embedding ablation result. From Figure 9, it can be observed that when only character embeddings are used, the F1-scores on the two datasets are 92.53% and 90.91%, respectively.