# A Knowledge Graph Embedding Model Based on Cyclic Consistency--Cyclic CKGE
## 5 Experiment
### Evaluation Metrics
The final top ten/total number is Hit@10. \[\text{hit@k}=\frac{1}{|\text{T}_{\text{test}}|}\sum{}_{(\text{h},\text{r}, \text{t})\in\text{T}_{\text{test}}}\frac{1}{2}\bigl{(}|\{(\text{h},\text{r}, \text{t})\mid\text{rank}_{\text{r},\text{t}}(\text{h})\leqslant\text{k}\}|+| \{(\text{h},\text{r},\text{t})\mid\text{rank}_{\text{h},\text{r}}(\text{t}) \leqslant\text{k}\}|\bigr{)} \tag{8}\]  
For the Cyclic_CKGE (our) model parameter efficiency, as shown in Table 5, Cyclic_CKGE (parameter of 0.23 M) in FB15k-237 outperformed DistMult (parameter of 1.89 M) on three of the five indicators. However, the Cyclic_CKGE model with a parameter of only 0.23 M obtains the best result on FB15k-237, reaching 0.425 Hits@10. Compared with the best model, R-GCN, its parameter exceeds 8 M and reaches 0.417 Hits@10. Overall, Cyclic_CKGE's parametric efficiency is more than 17 times that of R-GCNs and more than 8 times that of DistMult. For Freebase as a whole, these models will be over 82 GB for R-GCNs, over 21 GB for DistMult and only 5.2 GB for Cyclic_CKGE. On the basis of grid search, the hyperparameters of the Cyclic_CKGE model are selected according to the mean exchange rank (MRR). Grid search has the following hyperparameter ranges: embed cull [0.0, 0.1, 0.2], feature map cull [0.0, 0.1, 0.2, 0.3], projection layer cull [0.0, 0.1, 0.3, 0.5], embed size [100, 200], batch size [64, 128, 256], learning rate [001, 003] and label smoothness [0.0, 0.1, 0.2, 0.3]. On WN18, YAGO3-10 and FB15k, the combined effect of the following parameters is better: embedding cull of 0.2, feature map cull of 0.2, projection layer cull of 0.3, embedding size of 200, batch size of 128, learning rate of 0.001 and label smoothness of 0.1. For the \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Parametric** & **Embeddedness** & **hit@3** & **hit@10** \\ \hline \multirow{3}{*}{DistMult} & 1.89 M & 128 & 0.25 & 0.41 \\  & 0.95 M & 64 & 0.25 & 0.39 \\  & 0.23 M & 16 & 0.17 & 0.31 \\ \hline \multirow{3}{*}{Cyclic\_CKGE} & 5.05 M & 200 & 0.35 & 0.49 \\  & 1.89 M & 96 & 0.35 & 0.49 \\ \cline{1-1}  & 0.95 M & 54 & 0.33 & 0.46 \\ \cline{1-1}  & 0.46 M & 28 & 0.30 & 0.43 \\ \cline{1-1}  & 0.23 M & 14 & 0.28 & 0.40 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: DistMult model and Cyclic_CKGE model parameter comparison. Countries data set, we increased embedded cull to 0.3, implicit cull to 0.5 and set label smoothness to 0. Validation sets are evaluated every three cycles using early stops, based on common levels on validation sets (WN18, FB15k, YAGO3-10) and average statistics for OC-PR (Countries). For Countries, unlike other data sets, the variance of the results is high, so the average execution is 10 times to produce a 95% confidence interval. For distances and complex results after training 1-1, AdaGrad is optimized using the embedding size of 100 ([PERSON], [PERSON], and [PERSON], 2011) [(43)] by standardizing the model by forcing the L2 criterion for entity embedding to be 1. By comparing the parameter scale with the experimental results, it is verified that the Cyclic_CKGE model can obtain better experimental results with fewer parameters, as shown in Table 6.  
The specific link prediction results of the four types of models on the data sets WN18 RR and FB15k-237 are shown in Table 7. The Cyclic_CKGE model has achieved state-of-the-art performance on all metrics, as well as on some metrics on the FB15k and WN18. On Countries, it solved the challenges of S1 and S2 and performed well on S3, scoring higher than other models such as DistMult, as shown in Table 7.