## 4 Experiments and Results Analysis
### Results and Analysis  
The performance of the model was evaluated using the F1 value, F1-overlap, accuracy, and recall on the entity relationship dataset in the grid field constructed in Chapter 1 for training and evaluation. A comparison experiment was conducted to verify the effectiveness of the proposed model for the grid data relationship extraction. The grid field data entity relationship extraction model was compared to the BiLSTM-CRF, BERT-CE, BERT-CRF, and BERT-BiLSM-CRF models. The obtained results are shown in Table 6. It can be seen that the proposed model had the optimal recognition effect in grid business data relationship extraction, compared with the other models. On the gird dataset, it had precision, recall, and F1 values of 89.55, 85.91, and 87.92%, respectively. 1. The BiLSTM-CRF model used word2 vec as the embedding layer. However, its word vectors were static and cannot be adjusted according to the input context words. Therefore, it had a low performance on the power grid dataset, with an F1 value of only 63.30%. 2. The BERT-CE model used the BERT pre-trained language model as the embedding layer to adequately capture the contextual representation of the characters and thus had better access to the deep semantic information. On the grid dataset, the F1 value of the model was 85.16%. 3. The BERT-CRF model added conditional random field (CRF) to the BERT pre-trained language model, which improved its F1 value by 0.13% compared with the cross entropy loss module via sequentially annotating the output of BERT. 4. The BERT-BiLSTM-CRF model also used the BERT pre-trained language model to capture the contextual semantics of the grid business data, while utilizing recurrent neural networks to capture richer meanings. It also used the CRF for classification. It had an F1 value of 86.19%, which presented an improvement of 1.03% compared with Model 2. 5. For the RoBERTa-CE model with embedded semantic dependencies and lexicality, the RoBERTa pre-trained language model with dynamic MASK was used to capture the contextual semantics of the grid business data, embed semantic dependencies and \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Index** & **Model** & **Precision/\%** & **Recall/\%** & **F1-Score/\%** \\ \hline
1 & BiLSTM-CRF & 62.08\% & 64.58\% & 63.30\% \\
2 & BERT-CE & 87.07\% & 83.33\% & 85.16\% \\
3 & BERT-CRF & 87.15\% & 83.51\% & 85.29\% \\
4 & BERT-BiLSTM & 87.95\% & 84.51\% & 86.19\% \\  & CRF & **89.55\%** & **85.91\%** & **87.92\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison between the performance of different relational extraction models. The * symbol denotes the RoBERTa with incorporated semantic dependency and lexical embedding strategies. lexicality, and efficiently combine the information of the subject and object in order to improve their associativity and strengthen the linkage of the relational entities. Therefore, compared with the above model, the recognition performance was significantly improved, and the F1 value was 87.92% on the grid dataset. Compared with Model 2, the F1 value improved by 2.76%, which presented optimal recognition results. Table 7 presents the ablation experimental results of the model. The removal of the lexical embedding and semantic dependency embedding under the benchmark of the RoBERTa model reduced the performance of the model by 1.08%, which demonstrated that the correlation between related entities can be enhanced through effective embedding of lexical and semantic dependencies. Model 4, using the original RoBERTa, only differed by 0.78% compared to model 5 using the embedded lexical and semantic dependency BERT, while model 8, without any embedding enhancement, reduced the performance by 1.68% compared to model 4. Since RoBERTa used larger training data and was more powerful than BERT for deep characterization of sentences, the addition of lexical and semantic dependency embedding effectively narrowed the gap between the two and enhanced the deep characterization ability of the model.