# A Knowledge Graph Embedding Model Based on Cyclic Consistency--Cyclic CKGE
## 5 Experiment
### Evaluation Metrics  
We carried out horizontal comparison experiments on different models in multiple knowledge graph data sets (WN18, FB15k, YAGO3-10, Countries). The evaluation index formula (Mean Reciprocal Ranking (MRR), hit@k) was used for the analysis. Experiments show that the proposed model not only has scalability, but can also infer and model several relational patterns well such as symmetry and antisymmetry. Therefore, complex medical relational reasoning is very suitable for the model proposed in this paper. This paper mainly selects the following commonly used knowledge graph data sets. The data set details are as follows. WN18 is a subset of WordNet, consisting of 18 relationships and 40,943 entities. Most of the 151,442 triples consist of conjunctions and superword relations, and for this reason, WN18 tends to follow a strict hierarchy. FB15k is a subset of Freebase that contains about 14,951 entities with 1345 different relationships. Yago3-10 is a subset of YAGO3, consisting of at least 10 relationships per entity. It has 123,182 entities and 37 relationships. Most triples involve descriptive attributes of people, such as citizenship, gender and occupation. Countries is a benchmark data set that is useful for evaluating a model's ability to learn long-term dependencies between entities and relationships. It consists of three subtasks \begin{table}
\begin{tabular}{c c} \hline
**Hyperparameter** & **Disposition** \\ \hline \#negative samples & 2048 \\ loss function & BCE\_adv \\ gamma & 3.10 \\ adv:weight & 1.93 \\ regularizer & FRO \\ reg.weight & \(6.51\times 10^{-6}\) \\ dropout rate & 0.00 \\ optimizer & Adam \\ learning rate & \(6.43\times 10^{-4}\) \\ batch size & 512 \\ dimension size & 1000 \\ inverse relation & False \\ \hline \end{tabular}
\end{table} 
Table 4: Hyperparameter configuration of the model. \begin{table}
\begin{tabular}{c c} \hline
**Required Plug-In Environment** & **Versions** \\ \hline python & 3.6 \\ elasticsearch & 7.10.1 \\ numpy & 1.20.3 \\ jieba & 0.39 \\ pandas & 1.1.5 \\ rank-bm & 0.2.1 \\ scikit-learn & 0.24.1 \\ scipy & 1.5.4 \\ \hline \end{tabular}
\end{table} 
Table 3: The operating environment required using the system. that increase in difficulty in a step-by-step fashion, where the minimum path length to find a solution increases from 2 to 4. In order to evaluate the performance of various models on triplet extraction of a knowledge graph, this paper uses the link prediction method to give the cross-sectional comparison results of the TransE model, DistMult model, ConvE model and CCO model on various performance indicators. The specific step is to predict the missing head or tail entity of the triplet in the data set. Specifically, MRR (Mean Reciprocal Ranking) and hit@10 were used to measure the model performance level. Specific definitions are as follows:  
Mean Reciprocal Ranking: Mean Reciprocal Ranking (MRR), which is defined as the arithmetic mean of the reciprocal of all triples of the test set, is shown in Formula (7). MRR is an international mechanism used to evaluate search algorithms where the first result corresponds to a score of 1, the second matches to a score of 0.5 and the NTH matches to a score of 1/n. If there is no match, the sentence score is 0. The final score is the sum of all the scores. In the formula, rank represents the link prediction rank of the triplet. \[\text{MRR}=\frac{1}{|\text{T}_{\text{test}}|_{(\text{h},\text{r},\text{t}) \in\text{T}_{\text{test}}}}\sum_{2}\biggl{(}\frac{1}{\text{rank}_{\text{r}, \text{t}}(\text{h})}+\frac{1}{\text{rank}_{\text{h},\text{r}}(\text{t})} \biggr{)} \tag{7}\]  
hit@k: hit@k represents the ratio of triples where a single sort is in the first k during the test, as shown in Formula (8). Regarding hits@k, the larger the value, the better the link prediction of the model. k is usually 1, 3 and 10. Again, arrange the f function values as described above, and then see if the correct answer for each test triplet is in the top ten of the sequence, and if so, count +1.