## 3 Materials and Methods
### Evaluation
#### 3.3.3 Performance Comparison of Matrix Factorization and Popularity-Based Models  
The popularity-based recommender system simply recommends the most popular items to all users, without taking into account the specific preferences or past behavior of individual users. We chose to evaluate the popularity-based recommender system using the MAE metric, which measures the average absolute difference between the actual and predicted ratings. This allows us to compare the accuracy of the popularity-based recommender system to that of the collaborative filtering methods, which do make individualized predictions.  
Table 3 includes the average RMSE and MAE values for each method, based on the values provided. It also includes a popularity-based recommender system for comparison.  
We can see that SVD has the lowest average RMSE of 0.6439, followed closely by SVD++ with an average RMSE of 0.6462. NMF has the highest average RMSE of 0.6478. Additionally, we can see that the differences in RMSE values between the methods are relatively small.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{6}{c}{**RMSE**} \\ \hline
**Method** & **Fold 1** & **Fold 2** & **Fold 3** & **Fold 4** & **Fold 5** & **Average** \\ \hline
**NMF** & 0.6485 & 0.6433 & 0.6487 & 0.6545 & 0.6439 & 0.6478 \\
**SVD** & 0.6426 & 0.6467 & 0.6389 & 0.6489 & 0.6422 & 0.6439 \\
**SVD++** & 0.6486 & 0.6456 & 0.6433 & 0.6465 & 0.6471 & 0.6462 \\ \hline  & \multicolumn{6}{c}{**MAE**} \\ \hline
**Method** & **Fold 1** & **Fold 2** & **Fold 3** & **Fold 4** & **Fold 5** & **Average** \\ \hline
**NMF** & 0.4582 & 0.4523 & 0.4566 & 0.4543 & 0.4578 & 0.4558 \\
**SVD** & 0.4631 & 0.4591 & 0.4621 & 0.4587 & 0.4613 & 0.4609 \\
**SVD++** & 0.4606 & 0.4632 & 0.4614 & 0.4633 & 0.4591 & 0.4615 \\ \hline \hline \end{tabular}
\end{table}
Table 2: RMSE and MAE evaluation of NMF, SVD, and SVD++.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Average RMSE** & **Average MAE** \\ \hline
**NMF** & 0.6478 & 0.455 \\ \hline
**SVD** & 0.6439 & 0.460 \\ \hline
**SVD++** & 0.6462 & 0.462 \\ \hline
**Popularity-based** & - & 0.475 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Performance comparison of NMF, SVD, SVD++, and popularity-based recommender system using average RMSE and MAE.  
For the MAE metric, we can see that NMF has the lowest average MAE of 0.455, followed closely by SVD with an average MAE of 0.460. SVD++ has the highest average MAE of 0.462. The popularity-based recommender system has an average MAE of 0.475, which is higher than all three collaborative filtering methods.  
The reason why there is no value for popularity-based RMSE in the table is that the popularity-based recommender system does not predict ratings for individual users or items, and therefore cannot be evaluated using RMSE.  
These results suggest that both SVD and NMF are effective methods for generating accurate recommendations, with SVD performing slightly better in terms of RMSE and NMF performing slightly better in terms of MAE. SVD++ is also a viable option, but has slightly higher RMSE and MAE values than SVD and NMF. The popularity-based recommender system, on the other hand, is not as effective in generating accurate recommendations, as reflected by its higher MAE value.  
It is important to note that these results are based on a specific dataset, and the performance of the methods may vary with different datasets. Therefore, it is important to conduct further experiments and evaluations of different datasets to confirm these results and to choose the most appropriate method for a given recommendation task.