## 4 Experiment
### Results  
In this section, we measure the performance of CSEA through experiments. We compare CSEA with several recent excellent baselines: TransE is a representative distance-based model; Distmult and ComplEx are excellent semantic matching models. ; HAKE possesses good hierarchical modeling abilities; and both Rot-Pro and HousE are excellent models based on projection and rotation. **Evaluation metrics. ** In the cyber security link prediction task, we use MRR, MR, and Hits@k as the evaluation metrics; MRR is the mean reciprocal rank of the correct entities, MR is the mean rank of the correct entities, andHit@K is the proportion of correct entities ranking in the top K. For Hits@K, we use Hits@1/3/10 to evaluate the performance of the models. A higher MRR, lower MR, and higher Hits@1/3/10 indicate better performances. **Hyperparameters. ** The hyperparameter settings have an important impact on the applied methods. We control the hyperparameters of each method to ensure the fairness of the experiment. Table 3 shows common experimental hyperparameters. **Cybersecurity knowledge graph evaluation. ** Although our current cybersecurity knowledge graph is relatively complete, it is intended to simulate the phenomenon of having a lack of facts in the field of cybersecurity. We remove head entities or tail entities of triples in the test set to form prediction tasks (\(<7,relation,tail>\) and \(<head,relation,?>\)). The removed head entities or tail entities can be used as standard answers to test the completion performance of the models for the cybersecurity knowledge graph. Table 4 summarizes the experimental results of our CSEA model compared to the other models. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **MRR** & **MR** & **Hits@1** & **Hits@3** & **Hits@10** \\ \hline TransE [22] & 0.713 & 308 & 0.673 & 0.734 & 0.777 \\ DistMult [33] & 0.690 & 622 & 0.639 & 0.735 & 0.760 \\ ComplEx [34] & 0.735 & 673 & 0.720 & 0.745 & 0.762 \\ HAKE [30] & 0.741 & 411 & 0.727 & 0.749 & 0.769 \\ Rot-Pro [53] & 0.745 & 377 & 0.730 & 0.753 & 0.770 \\ HousE [31] & 0.747 & 314 & 0.732 & 0.753 & 0.775 \\ CSEA & **0.756** & **148** & **0.735** & **0.764** & **0.793** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Average evaluation results on CybersecurityKG; results in bold are the best. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **\#E** & **\#R** & **\#TR** & **\#VA** & **\#TE** \\ \hline CybersecurityKG & 4095 & 9 & 7686 & 1100 & 2200 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The quantity statistics of CybersecurityKG; #E and #R represent the number of entities and relations, respectively, while #TR,#VA, and #TE represent the size of the training set, validation set, and test set, respectively. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Max Steps & 20,000 \\ Valid Steps & 1000 \\ Learning Rate & 0.00005 \\ Batch Size & 128 \\ \(\gamma\) & 6.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: List of hyperparameter values of the models. As shown in the Table 4, our model achieves good results in all indicators. The above experimental results show that CSEA can complete missing information better than a single model. Missing entities can be found more quickly among individuals with the highest scores. CSEA produces more accurate predictions in practical applications. In addition, we performed the predict head and predict tail tasks for our model and the other models. Table 5 records the average prediction results of the predict head and predict tail tasks. Compared with the other models, our method achieves similar improvements in the predict head and predict tail tasks. Different models have different sensitivities to each relationship. The experimental performance of the model may be too good for one relationship in the knowledge graph completion task, while the performance of the model may be poor for other relationships, greatly affecting the average prediction results.