## 4 Experiment
### Evaluation Metrics  
This paper makes use of the AUC score, ACC score, precision rate, and recall rate as evaluation metrics to measure the CTR prediction performance and top-K recommendation of users for items. As shown in Table 2, all samples are divided into four combinations according to their actual classification and model prediction results. Namely, the true positive example (TP), the true negative example (TN), the false positive example (FP), and the false negative example (FN). Where TP represents the positive samples that users are interested in; items with which they have not interacted can be correctly predicted by the model. TN represents the negative samples that users are not interested in; items with which they have not interacted can be correctly predicted by the model. FP means that the model mistakenly predicts the negative samples that users are not interested in as the positive samples that users are interested in. FN means that the model mistakenly predicts the positive samples that users are actually interested in as the negative samples that users are not interested in. The confusion matrix is shown in Table 2.  
AUC is defined as the area under the ROC curve. The abscissa of the ROC curve is the false-positive rate (FPR) and the ordinate is the true-positive rate (TPR). The larger the AUC, the better the recommendation effect of the model. The AUC index is shown in Equation (23).  
\[\text{FPR}=\frac{\text{FP}}{\text{FP}+\text{TN}},\ \text{TPR}=\frac{\text{TP}}{ \text{TP}+\text{TN}} \tag{23}\]  
Accuracy is the percentage of all predictions that are correctly predicted. The ACC index is shown in Equation (24).  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{TN}+ \text{FN}} \tag{24}\]  
The precision rate is used to indicate how many of the items that the model predicts users may be interested in are items that the user will actually be interested in when the users have not interacted with items. The formula of precision is as follows in Equation (25).  
\[\text{Precision}\ =\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{25}\]  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **MovieLens-1M** & **MovieLens-20M** & **BookCrossing** & **Last.FM** \\ \hline Users & 6036 & 138,159 & 17,860 & 1872 \\ Items & 2445 & 16,954 & 14,910 & 3846 \\ Interactions & 753,772 & 13,501,622 & 139,746 & 42,364 \\ Enities & 182,011 & 102,569 & 24,039 & 9366 \\ Relations & 12 & 32 & 18 & 60 \\ KG triples & 20,782 & 499,474 & 19,793 & 15,518 \\ Sparsity & 0.9489 & 0.9947 & 0.9994 & 0.9941 \\ level & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Details of the four public datasets.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Clicked** & **Not Clicked** \\ \hline Recommended & True-Positive(TP) & False-Positive(FP) \\ Not Recommended & False-Negative(FN) & True-Negative(TN) \\ \hline \hline \end{tabular}
\end{table}
Table 2: The confusion matrix.  
The recall rate is used to indicate how many of the items that the user is actually interested in are correctly predicted by the model. The formula of recall is as follows in Equation (26).  
\[\text{Recall}\ =\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{26}\]