# A Top-N Movie Recommendation Framework Based on Deep Neural Network with Heterogeneous Modeling
## 4 Experiments
### Performance and Comparison
According to the HR, MFDNN had the best performance, MF and DNN were similar, and their trends were consistent. According to the NDCG value, MFDNN performed best, followed by MF and, finally, DNN. The results of the comparison experiments verify that MFDNN provided the best recommendation performance. In order to evaluate the recommendation performance of MF and DNN, which learn embedded features independently, separate training processes were employed to facilitate the sharing of the best MF and DNN embedding layers; these are, respectively, denoted as MFDNN: (share MF) and MFDNN: (share DNN). The HR and NDCG values for MFDNN: (share MF), MFDNN: (share DNN), and MFDNN are shown in Figure 12. Figure 12 shows that, although there are small batches of data indicating that the performance was the best when embedding layers were shared, the overall trend shows that MFDNN performed the best. According to the HR, MFDNN provided the best overall performance, while MFDNN: (share MF) performed better than MFDNN: (share DNN) at the early epochs; subsequently, the performance of MFDNN: (share MF) decreased. According to the NDCG, the overall trend is consistent with HR. Optimal performance was achieved when MF and DNN learned embedding independently. Figure 10: HR and NDCG values for MFDNN- - and MFDNN. In order to evaluate the performance of MFDNN over other recommendation models, we trained two state-of-the-art recommendation models (DMF and NCF) and three HIN-based methods (HeteCF, HeteCF, and CMF) separately. The HR and NDCG values for MFDNN and the two baseline methods are shown in Figure 13 (MovieLens 1m) and Figure 14 (Netflix). The HR and NDCG values for MFDNN and the three HIN-based baseline methods are shown in Figure 15 (MovieLens 1m) and Figure 16 (Netflix). The best performance results of each baseline method are shown in Figure 17.  
Figures 13-16 show that MFDNN achieved the highest performance. According to the HR, MFDNN performed the best, followed by the two baseline methods NCF, DMF, and the HIN-based baseline methods, HeteCF, HeteMF, and CMF. According to the NDCG value, MFDNN performed the best, followed by the two baseline methods, NCF, DMF, and the HIN-based baseline methods, HeteCF, HeteMF, and CMF. Further analysis indicates that the performance of NCF was better than that of DMF. As NCF uses implicit feedback data and DMF uses explicit feedback data, the results are consistent with the experimental results shown in Figure 9. Table 4 shows the best HR and NDCG values for the MFDNN and baseline methods. Figure 11: HR and NDCG values for MF, DNN, and MFDNN. Figure 8:0 The HR and NDCG values for MFDNN: (share MF), MFDNN: (share DNN), and MFDNN. Figure 8:1 The HR and NDCG values for the MFDNN and baseline methods. Figure 8:2 The HR and NDCG values for the MFDNN and HIN-based baseline methods. Figure 8:3 HR and NDCG values for MFDNN and baseline methods. Figure 8:4 The results of the MFDNN and baseline methods. Figure 8:5 The HR and NDCG values for the MFDNN and HIN-based baseline methods. Table 4 clearly shows that MFDNN outperformed the two baseline methods. In terms of HR, MFDNN achieved an average 5.4% improvement over DMF, an average 2.3% improvement over NCF on the MovieLens 1m datasets. In terms of HR, the two compared methods underperformed MFDNN by an average of 2.75% in terms of HR on the Netflix datasets. The five methods underperformed MFDNN by an average of 3.75% in terms of NDCG. The performance improvements provided by MFDNN are statistically significant according to these results. (4) Selection of N of Top-N  
In the above experiment, the value of N was 10; however, with different N values, the HR and NDCG values will also be different. We selected N values of 5, 10, and 15 to train the model and chose the best performance in the training epochs. The HR and NDCG values for different top-N values are shown in Figure 8:6  
Table 5 clearly shows that MFDNN performed the best when N was 15; however, we cannot conclude that the larger the value of N, the better the performance.