# Entity Relationship Extraction Based on a Multi-Neural Network Cooperation Model
## 4 Experiments and Results
### Main Results and Analysis  
In terms of experimental comparison, on the small sample corpus, this method was compared with two types of current commonly used entity classification methods. In the feature extraction layer, experimental comparisons were conducted on the traditional Word2 vec model and Bert model, and the Word2 vec model used version 0.2.0 \"Tencent AI Lab embedding Corps\" published by Tencent AI laboratory. The Bert model used the Roberta model. The parameter settings and results obtained are illustrated in Table 4.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline
**No.** & **Model** & **Training Set** & **Precision** & **Recall** & **F1** & **Best Epoch** \\ \hline
1 & Word2 vec–GRU–FC & Epoch = 145, & 44.4\% & 41.5\% & 42.6\% & 138 \\ \hline
2 & Word2 vec–BiGRU–FC & learning\_rate = 0.002, & 65.4\% & 66.6\% & 66.0\% & 46 \\  & ([PERSON] et al., 2018) & hidden\_dim = 64, & 23.6\% & 32.9\% & 27.5\% & 139 \\ \hline
3 & Word2 vec–LSTM–FC & dropout = 0.4 & 58.1\% & 59.2\% & 58.6\% & 131 \\ \hline
5 & [PERSON]–BiGRU–FC & & 75.6\% & 74.4\% & 76.0\% & 134 \\  & ([PERSON] et al., 2022) & & 74.6\% & 72.0\% & 73.3\% & 138 \\ \hline
6 & [PERSON]–BiLSTM–FC & & 73.5\% & 72.4\% & 72.9\% & 89 \\ \hline
7 & ([PERSON] and [PERSON], 2021) & & & & & \\ \hline
8 & RBF before optimization &
\begin{tabular}{c} Epoch = 145, \\ learning\_rate = 0.004 \\ dim = 100, hidden\_dim = 64, \\ BATCH = 4, \\ dropout = 0.4 \\ \end{tabular} & 89.8\% & 89.4\% & 89.6\% & 34 \\ \hline
9 & RBF using Exponential Decay &
\begin{tabular}{c} Epoch = 145, \\ learning\_rate using the \\ learning\_rate using the \\ Exponential Decay algorithm, \\ embedding\_dim = 100, \\ hidden\_dim = 64, \\ BATCH = 4, \\ dropout = 0.4 \\ \end{tabular} & 90.8\% & 90.2\% & 90.5\% & 47 \\ \hline
10 & Our RBF model &
\begin{tabular}{c} Epoch = 145, \\ learning\_rate using cosine \\ annealing, \\ embedding\_dim = 100, \\ hidden\_dim = 64, \\ BATCH = 4, \\ dropout = 0.4 \\ \end{tabular} & **91.0\%** & **91.6\%** & **91.9\%** & **47** \\ \hline \end{tabular}
\end{table}
Table 4: Comparison of experimental results.  
Figure 5: Learning rate optimization based on cosine annealing.  
Due to the small amount of data, the times required for completing 145 epochs in columns 8, 9, and 10 with better training results in Table 4 were 1:31:02, 1:36:27, and 1:27:40. There was not much difference between them. The F1 and the loss value of other models on multi-neural network cooperation under different iteration times are illustrated in Figures 6 and 7. They mainly included two categories: using Word2 vec as the encoder, and using [PERSON] as the encoder.  
The relationship extraction effect of our RBF model before and after optimization is illustrated in Figure 8.  
Further experiments were conducted to optimize the model. For the RBF model, the cosine annealing algorithm was compared with the exponential decay algorithm, and the results are shown in Figure 9.  
Figure 6: Relationship extraction effect of models with Word2 vec as the encoder. (**a**) The variation in F1 value at different epochs; (**b**) the variation in LOSS value at different epochs.  
Figure 7: Relationship extraction effect of models with Bert as the encoder. (**a**) The variation in F1 value at different epochs; (**b**) the variation in LOSS value at different epochs.