# PNER: Applying the Pipeline Method to Resolve Nested Issues in Named Entity Recognition
## 5 Results and Discussion
### Discussion of Results on the CMEEE Dataset  
Firstly, we referred to the performance of three pre-trained models on the CMEEE dataset provided by [PERSON] et al. in CBLUE. These three pre-trained models are as follows:  
* BERT-wvm-ext-base [35]. This pre-trained model adopts the whole-word masking approach, masking the entire word as the smallest unit, and is trained on a Chinese language corpus. * BERT-base [13]. This is the basic BERT model, comprising 110 million parameters, and is trained on a Chinese language corpus. * MacBERT-large [36]. This is an enhanced BERT model that employs the masked language model (MLM) as a pre-training task. In our PNER model, specifically for the entity boundary recognition task, we utilized three different pre-trained models. We then benchmarked these against the standards set  
Figure 4: Data annotation method in text classification model. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Train** & **Dev** & **Test** & **Labeling Type** \\ \hline CMEEE & 59,577 & 19,534 & - & 9 \\ GermEval 2014 & 30,337 & 2784 & 6437 & 12 \\ GENIA & 45,929 & 4337 & 5474 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Dataset size statistics after inserting identifiers. by [PERSON] et al. [22]. The outcomes of this comparison are detailed in Table 10 below, where the same pre-trained models and parameters were used for the comparison, and the baseline model data were obtained from CBLUE [22]. Our experiments revealed that, across the board, all the enhanced versions of our model outperformed their respective baseline counterparts. Notably, when using the MacBERT pre-trained model, the improvement in performance was even more significant compared to the other two enhanced versions. In fact, it achieved the highest score. This finding indicates that while there is a boost in performance for entity boundary recognition tasks, the enhancement is even more pronounced in tasks related to recognizing entity categories. This leads to an overall more substantial improvement in the performance of our model. During the classification task of our study, we introduced identifiers in the pre-processing stage of the dataset to indicate whether the entity being classified was a main entity or a sub-entity. Additionally, we annotated the dataset with position index identifiers for both the main and sub-entities. For training, we employed the BERT-base pre-trained model on the pre-processed CMeEE training set, and we carried out testing on the validation set. Alongside this, we monitored the classification performance across various entity categories. The model's performance in different categories is presented in Table 11. It is important to note that we are not certain whether the inclusion of identifiers will affect the performance of the classification model. Therefore, we need to conduct an experiment to determine this. Our primary objective in the classification task was to achieve higher accuracy. To this end, we conducted experiments to compare scenarios with and without the inclusion of identifiers. We evaluated the accuracy of both approaches, and the comparative findings are depicted in Table 12. The results clearly demonstrate that including identifiers significantly enhances the model's classification precision. This underscores the vital role these identifiers play in effectively classifying different entity categories. \begin{table}
\begin{tabular}{c c c} \hline \hline
**CMeEE** & **Model** & **F1 Score** \\ \hline \multirow{3}{*}{Baseline} & BERT-wwm-ext-base [35] & 61.7 \\  & BERT-base [13] & 62.1 \\  & MacBERT-large [36] & 62.4 \\ \hline \multirow{3}{*}{Our model} & PNER(BERT-wwm-ext-base) & 62.2 \\  & PNER(BERT-base) & 62.9 \\ \cline{1-1}  & PNER(MacBERT) & 63.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison results on the CMeEE dataset.