# Medical Named Entity Recognition Fusing Part-of-Speech and Stroke Features
## 4 Analysis and Results from Experiments
### Results  
In this experiment, the evaluation criteria are F1 score (F1), precision (P), and recall (R). On the CCKS2019 dataset, the following models were used to compare:  
* IDCNN-CRF, the encoding layer uses the expansion convolution network. Unlike the traditional convolution network, the expansion convolution expands the receptive field of the model by adding holes in the convolution kernel and obtains more context by using fewer convolution layers. * BiLSTM-CRF, a popular model for handling NER tasks, takes the word vector representing the input text from the embedding layer, feeds it into BiLSTM to obtain characteristics, and then outputs the expected tag results following CRF. * BERT-CRF, the encoding layer using the BERT model. * BERT + BC, acquiring word vectors by pre-training the BERT model, followed by BiLSTM-CRF for the NER task. * BERT-WWM + BC obtains word vectors by pre-training the BERT-WWM model. Unlike BERT, in the initial pre-training phase, BERT-WWM modifies the training sample generation approach and increases the whole word mask. * RoBERTa-WWM-ext + BC, using the RoBERTa-WWM-ext model in the embedding layer, RoBERTa has enhanced the training tasks and data generation methods over BERT. Table 3 displays the outcomes of the experiment. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline BiLSTM hidden size & 128 \\ Embedding size & 768 \\ Learning rate & \(3\times 10^{-5}\) \\ Max sequence length & 150 \\ Dropout & 0.5 \\ Batch size & 16 \\ Epoch & 30 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters of the model. It can be seen from the experimental outcomes that the model suggested in this study has F1 score, precision, recall that are 78.65%, 77.49%, and 79.84%, respectively, on the CCKS2019 dataset, among which the F1 score and precision reach the maximum in all comparison models. The results of the first three experiments reveal that the BERT model outperforms IDCNN and BiLSTM in terms of recognition effect when CRF is utilized as the decoder. By first masking part of the sentence's terms and then letting the model predict the unmasked words, BERT improved its capacity for semantic processing and did well on the NER challenge. The recognition effect is improved by adding a BiLSTM layer between BERT and CRF, as can be seen from a comparison of the third model's and fourth model's results. This is because the forward and backward LSTM layers reinforce the relationship before and after the text sequence, which enhances the recognition effect. The model described in this study builds on the BERT pre-training model by adding category weighted loss function, part-of-speech information, and stroke information. All three assessment indexes have advanced in comparison to the BERT + BC model, with the F1 score rising by 1.39%. Although the recall rate of our model is slightly lower than that of the RoBERTa-WWM-ext + BC model and the BERT-WWM + BC model, its precision is higher. On the whole, the F1 score of the comprehensive index is increased by 0.51% and 1.20%, respectively, which shows that embedding word part-of-speech information and stroke information as multiple features is helpful for the model to achieve a better recognition effect. To further support the reliability of each element of the model outlined in this study, the following comparative tests were conducted: (1) by adding the class-weighted loss function; (2) by adding part-of-speech features; (3) by adding stroke features extracted by BiLSTM; (4) by adding stroke features extracted by CNN. In Table 4, the experimental findings are displayed. When the loss function improves, the F1 score increases by 0.59% in comparison to the Baseline model. And the F1 score rises by 0.67% when the part-of-speech element is included. When adding stroke features, BiLSTM and CNN are used to extract stroke features, the F1 score is enhanced by 0.15% using BiLSTM and by 0.50% using CNN. Each component of the model described in this research is found to be effective.