## 3 Results
### Model Performance Comparison Analysis  
The evaluation of the model uses commonly used evaluation metrics using precision (Precision, P), recall (Recall, R), and F1 value. The expressions for precision (P), recall (R), and F1 values are as follows:  
\[\text{P}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{4}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Entity Type** & **Train** & **Validation** & **Test** \\ \hline Body & 1128 & 152 & 158 \\ Disease & 2784 & 338 & 407 \\ Drug & 781 & 97 & 126 \\ Method & 288 & 33 & 40 \\ Operation & 323 & 54 & 58 \\ pastHistory & 950 & 132 & 105 \\ Result & 82 & 4 & 8 \\ Symptom & 2343 & 307 & 300 \\ Test & 2359 & 327 & 295 \\ Time & 137 & 28 & 23 \\ Treatment & 89 & 21 & 8 \\ Value & 2275 & 323 & 287 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics on the number of dataset entities. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Relation Type** & **Train** & **Validation** & **Test** \\ \hline body–symptom & 1207 & 162 & 168 \\ method–drug & 423 & 62 & 76 \\ test–result & 84 & 4 & 8 \\ test–value & 2275 & 323 & 287 \\ time–disease & 15 & 2 & 1 \\ time–operation & 171 & 32 & 26 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Statistics on the number of relations in the dataset. \[\text{R}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{5}\]  
\[\text{F1}=\frac{2\text{PR}}{\text{P}+\text{R}} \tag{6}\]  
Among them: (True Positive) means to predict the positive class as a positive class; FP (False Positive) means to predict the negative class as a positive class; FN (False Negative) means to predict a negative class as a negative class; (TP + FP) means to predict, in the results of the positive class, the proportion of the correct number; (TP + FN) indicates the proportion of the samples that are actually positive classes, correctly judged as positive classes. In order to avoid the influence of dividing the dataset, we divided the data into ten equal fractions, and selected one of them as the test set and the rest as the training set, one at a time. The ten-cross validation results are shown in Table 4:  
As can be seen in Table 4, the average F1 of ten-cross validation entity recognition is 0.9179, and that of relation, recognition is 0.9265. The overall results were relatively stable. In order to further verify the effect of the model proposed in this paper, the following baseline models are selected for comparison:  
1. Multi-Head [11]: Use BiLSTM-CRF for named entity recognition and incorporate the recognized results into relation extraction as soft labels. 2. SpERT [12]: The core is the pre-train transformer network BERT. Entity recognition and relation classification for all spans in the input sentence via strong negative sampling, span filtering, and local context representation. 3. Table-Sequence Encoders [13]: It is proposed to use table encoding and sequence encoding to extract features related to named entity and relation classification, respectively, and propose a mechanism to make the two encodings interact. 4. UniRE [14]: Use a unified label space to represent entities and relations, classify them with a classifier, and eliminate the gap between the two subtasks. From the data in Table 5, it can be seen that the improved model in this paper has achieved the best results in F1 score compared with other models. Compared with the Multi-Head model, the F1 score of entity recognition and relation extraction has been greatly improved, respectively, by 2.4% and 4.9%. The reason is that the relation extraction model of the Multi-Head model uses real labels in the training phase, but uses the results of entity recognition in the previous phase in the inference phase, resulting in inconsistency between training and inference. In the Multi-Head model, entity recognition and relation extraction are decoded separately, and the interaction information between the two is not enough. Compared with the SpERT model, the F1 score is increased by 1.3% and 3.0%, respectively.