## 4 Experiments
### Evaluation Results  
We have evaluated the performance of our proposed approach, FANA-CF, which uses two groups of users' ratings (friends and natives) in order to generate recommendations, by comparing it with existing item-based collaborative filtering, user-based collaborative filtering, and the personalized mean approach, which uses every user's rating to generate recommendations. The Foursquare dataset has been used to test the recommendation quality. We randomly selected 20% of ratings from the evaluation dataset as the test set and used the remaining 80% as the training set. We also conducted the cross validation in terms of k-fold cross validation (5-fold) on the training set, by randomly selecting the different test and training sets each time, and taking the average of the results. The evaluation metrics that we used to test the recommendations' quality were mean absolute error (MAE), root mean squared error (RMSE), and normalized discounted cumulative gain (nDCG). Table 3 shows the comparison of MAE performance among IBCF, UBCF, PersMean, and our proposed approach, FANA-CF, using 5-fold cross validation. We also calculated the average of MAE values and the standard deviation for each approach. The MAE evaluation metric is used to evaluate the average magnitude of errors in a set of prediction ratings. For this evaluation, a lower value indicates that the approach scores better in terms of accuracy. From the evaluation, it can be seen that our proposed approach (FANA-UBCF), which uses two groups of users' ratings, friends and natives, to generate recommendations, scored an average of 14.182 \(\pm\) 0.679, slightly outperforming the existing IBCF, UBCF, and PersMean approaches that use every user's rating to generate recommendations. This shows that by using two groups of users' ratings to generate recommendations instead of just using every user's ratings to generate recommendations will have a lower error rate (i.e., higher accuracy). as friends have more commonly rated items (i.e., the same preferences). For further analysis of the results, we show a box and whisker plot in Figure 4 with Friedman test results [23]. Note that a \(p\) value < 0.05 means the experimental result in Table 3 is statistically significant. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Experiment \#** & **IBCF** & **UBCF** & **PersMean** & **FANA-IBCF** & **FANA-UBCF** \\ \hline
1 & 16.780 & 14.723 & 20.304 & 16.630 & 14.720 \\
2 & 15.274 & 13.741 & 20.896 & 15.233 & 13.721 \\
3 & 17.419 & 15.097 & 21.816 & 16.259 & 15.097 \\
4 & 13.787 & 13.201 & 20.189 & 13.774 & 13.201 \\
5 & 16.566 & 14.194 & 20.778 & 16.568 & 14.173 \\ Average & 15.965 \(\pm\) 1.293 & 14.191 \(\pm\) 0.676 & 20.797 \(\pm\) 0.576 & 15.693 \(\pm\) 1.082 & **14.182 \(\pm\) 0.679** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of mean absolute error (MAE) performance among item-based collaborative filtering (IBCF), user-based collaborative filtering (UBCF), personalized mean (PersMean), and our proposed approach, FANA-CF, using 5-fold cross validation. We have calculated the average of MAE value and the standard deviation for each approach. We performed pairwise Wilcoxon signed-rank tests [24] to identify which pairs were significant. Table 4 shows the pairwise Wilcoxon signed-rank test of MAE. It can be seen that our proposed algorithms mostly outperformed other algorithms. Table 5 shows the comparison of RMSE performance among our proposed approach (FANA-CF), IBCF, UBCF, and PersMean using 5-fold cross validation. We have calculated the average RMSE and the standard deviation for each approach. RMSE evaluation metric is a quadratic scoring rule which measures the average magnitude of the error. For this evaluation, our proposed approach, FANA-UBCF, with the average of 16.892 \(\pm\) 0.513, slightly outperformed IBCF, UBCF, and PersMean approaches.