# CKGAT: Collaborative Knowledge-Aware Graph Attention Network for Top-N Recommendation
## 5 Experiments
### Experimental Results
#### 5.4.4 Hyperparameter Sensitivity (RQ4)  
The CKGAT method generates user/item embeddings by aggregating the ripple set embeddings. Therefore, the maximum hop number \(L\) of the ripple sets directly affects the generation of user/item embeddings, and then affects the recommendation performance. In order to verify the influence of the hyperparameter \(L\) on the recommendation performance, we analyzed the sensitivity of CKGAT to \(L\) through experiments. In the experiments, \(L\) was changed from one to four, while other hyperparameters remained fixed. The experimental results are shown in Table 3, with the best results among different parameter values on a dataset highlighted in bold. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Datasets** & **Metrics** & **CKAN** & \(\text{CKGAT}_{\text{w/o Att}}\) & \(\text{CKGAT}_{\text{w/o GAT}}\) & **CKGAT** \\ \hline \multirow{3}{*}{Last.FM} & precision@10 & 0.0310 & 0.0353 & 0.0338 & **0.0390** \\  & recall@10 & 0.1215 & 0.1383 & 0.1345 & **0.1481** \\  & F1-measure@10 & 0.0469 & 0.0562 & 0.0530 & **0.0587** \\  & NDCG@10 & 0.0893 & 0.0935 & 0.0928 & **0.0984** \\ \hline \multirow{3}{*}{Book-Crossing} & precision@10 & 0.0149 & 0.0152 & 0.0154 & **0.0157** \\  & recall@10 & 0.0537 & 0.0562 & 0.0553 & **0.0603** \\  & F1-measure@10 & 0.0183 & 0.0185 & 0.0186 & **0.0193** \\  & NDCG@10 & 0.0623 & 0.0628 & **0.0640** & 0.0635 \\ \hline \multirow{3}{*}{MovieLens 20M} & precision@10 & 0.0610 & 0.0622 & 0.0628 & **0.0640** \\  & recall@10 & 0.1273 & 0.1282 & 0.1280 & **0.1319** \\  & F1-measure@10 & 0.0687 & 0.0692 & 0.0694 & **0.0719** \\  & NDCG@10 & **0.0962** & 0.0923 & 0.0935 & 0.0935 \\ \hline \multirow{3}{*}{Dianping-} & precision@10 & 0.0360 & 0.0430 & 0.0406 & **0.0450** \\  & recall@10 & 0.1254 & 0.1545 & 0.1432 & **0.1855** \\  & F1-measure@10 & 0.0500 & 0.0566 & 0.0542 & **0.0632** \\  & NDCG@10 & 0.0883 & 0.0993 & 0.0965 & **0.1168** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Recommendation accuracy of CKAN, CKGAT, and the two variants on the four datasets. We have the following observations from the experimental results:  
1. CKGAT achieves the best recommendation performance when \(L=3\) and \(L=2\) on the Last.FM and Book-Crossing datasets, respectively, and achieves the best recommendation performance when \(L=1\) on both the MovieLens 20M and Dianping-Food datasets. 2. When \(L\) increases to four, the recommendation performance of CKGAT on the four datasets is the worst. This may be because when the maximum hop number is large, CKGAT introduces entities with lower relevance to users/items and generates inaccurate user/item embeddings, thus limiting the recommendation performance. In addition, CKGAT learns the high-order entity representations with the help of their multiple neighboring entities in multi-hop ripple sets. Therefore, the number \(K\) of neighboring entities directly affects the learning of the entity's high-order representation and then affects the recommendation performance. In order to verify the influence of hyperparameter \(K\) on the recommendation performance, we further analyzed the sensitivity of CKGAT to \(K\) through experiments. In the experiments, \(K\) varied from two to five while other hyperparameters remained fixed. The experimental results are shown in Table 4, with the best results among different parameter values on a dataset highlighted in bold. We have the following observations from the experimental results:  
1. CKGAT achieves the best recommendation performance when \(K=3\) and \(K=4\) on the Last.FM and Book-Crossing datasets, respectively, and achieves the best recommendation performance when \(K=5\) on both the MovieLens 20M and Dianping-Food datasets. 2. When \(K\) decreases to two, the recommendation performance of CKGAT on the four datasets is the worst.