# Recommendation Algorithm Based on Heterogeneous Information Network and Attention Mechanism
## 4 Experiment and Analysis
### Model Evaluation and Analysis  
The user-item interaction records in the datasets were randomly divided into training and test sets, with 80% as the training set and 20% as the test set. AMNRec was compared with six benchmark algorithms, and the experimental results are shown in Table 3. The following can be seen from the tables:  
1. It is observed that the complete AMNRec model in this paper consistently outperforms the other comparable models on the three datasets, verifying the effectiveness of AMNRec on the ranking recommendation task and indicating the effectiveness of AMNRec in proposing the use of neighbor information to supplement the missing information of meta-paths.
2. Considering the three variants of AMNRec, it can be found that the overall performance is as follows: AMNRec-total > AMNRec 1-hop > AMNRec 2-hop > AMNRec-atten. The results show that although AMNRec 1-hop is competitive with other methods, it is still worse than the complete AMNRec-total, and it can be seen that the one-hop neighbor information supplementation plays a crucial role. AMNRec-atten proves the effectiveness of introducing an attention mechanism to extract multi-hop neighbor information.
3. Among these compared methods, the HIN-based methods (FMG, MCRec) are superior to the collaborative filtering methods (ItemKNN, BPR, MF, and NeuMF) in most cases, indicating the usefulness of heterogeneous information embedding. In addition, NeuMF also achieves competitive performance due to its use of neural networks, but its performance is still inferior to MCRec due to the lack of heterogeneous information. Among the benchmark methods for multi-source information networks, MCRec performs the best, indicating that collaborative attention can make good use of meta-path-based contexts for a recommendation, thus improving the recommendation efficiency.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**MovieLens**} & \multicolumn{2}{c}{**LastFM**} & \multicolumn{2}{c}{**Yelp**} \\ \cline{2-7}  & **Recall@10** & **NDCG@10** & **Recall@10** & **NDCG@10** & **Recall@10** & **NDCG@10** \\ \hline AMNRec 2-hop & 0.3533 & 0.6889 & 0.6067 & 0.8476 & 0.7862 & 0.6446 \\ AMNRec 1-hop & 0.3687 & 0.7056 & 0.6123 & 0.8525 & 0.7972 & 0.6523 \\ AMNRec-atten & 0.3606 & 0.7094 & 0.5956 & 0.7978 & 0.7938 & 0.6465 \\ AMNRec-total & 0.3762 & 0.7125 & 0.6234 & 0.8639 & 0.8127 & 0.6675 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results of ablation experiments performed on MovieLens, LastFM, and Yelp.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**MovieLens**} & \multicolumn{2}{c}{**LastFM**} & \multicolumn{2}{c}{**Yelp**} \\ \cline{2-7}  & **Recall@10** & **NDCG@10** & **Recall@10** & **NDCG@10** & **Recall@10** & **NDCG@10** \\ \hline ItemKNN & 0.1536 & 0.5161 & 0.4513 & 0.7981 & 0.5421 & 0.5378 \\ BRP & 0.1946 & 0.6459 & 0.4492 & 0.8099 & 0.5504 & 0.5549 \\ MF & 0.2053 & 0.6511 & 0.4634 & 0.7210 & 0.5350 & 0.5322 \\ NeuMF & 0.2090 & 0.6587 & 0.4678 & 0.8104 & 0.5857 & 0.5713 \\ FMG & 0.2165 & 0.6682 & 0.4916 & 0.8263 & 0.5951 & 0.5861 \\ MCRec & 0.2256 & 0.6900 & 0.5068 & 0.8526 & 0.6326 & 0.6301 \\ AMNRec-total & **0.4062** & **0.7275** & **0.6234** & **0.8639** & **0.8127** & **0.6675** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results of effectiveness experiments on MovieLens, LastFM, and Yelp.