# Evolving Matrix-Factorization-Based Collaborative Filtering Using Genetic Programming
## 3 Results
### Results of the Evolutionary Method  
In order to analyze the final aggregation functions \(h\) generated by the GP algorithm, we have built graphical tree representations of the best individuals for each dataset. In Figure 2, we plot the best individual for the MovieLens dataset (EMF-4). On the other hand, in Figure 3, we plot the best individual for the FilmTrust dataset (EMF-10). Furthermore, for the sake of completeness, the rest of the individuals have been included in Appendix A. Recall that the correspondence between node symbols and operators are defined in Table 1.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **MAE** & **MSE** \\ \hline PMF & 0.7225 & 0.8492 \\ BiasedMF & **0.7160** & 0.8406 \\ NMF & 0.7672 & 0.9867 \\ BNMF & 0.7500 & 0.8860 \\ EMF-1 & 0.7256 & 0.8728 \\ EMF-2 & 0.7195 & 0.8332 \\ EMF-3 & 0.7210 & 0.8427 \\ EMF-4 & 0.7197 & **0.8282** \\ EMF-5 & 0.7195 & 0.8592 \\ EMF-6 & 0.7220 & 0.8377 \\ EMF-7 & 0.7255 & 0.8721 \\ EMF-8 & 0.7193 & 0.8442 \\ EMF-9 & 0.7161 & 0.8441 \\ EMF-10 & 0.7163 & 0.8381 \\ EMF (best) & 0.7161 & 0.8282 \\ EMF (worst) & 0.7256 & 0.8728 \\ EMF (avg) & 0.7205 & 0.8472 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Mean Absolute Error (MAE) and Mean Squared Error (MSE) in predictions for MovieLens dataset.  
Regarding the performance of the evolutionary process, Figure 4 shows the evolution of the 25 th percentile of fitness values for each run of the experiments. As it can be seen, the fitness values of the population decrease as the evolutionary process goes on, pointing out the quality of the solutions within the population improves. This phenomenon can be observed for both MovieLens and FilmTrust datasets.  
Concerning the execution time of our evolutionary approach, the average EMF execution time is 8.069 h, with a standard deviation of 5.029 when using the MovieLens dataset, while the median is 6.483. The average execution time for the Filmtrust dataset is 8.335 h, with a standard deviation of 11.231 and with a median of 4.099 h. Experiments have been run in a computer with an Intel i7 CPU, 16 Gb RAM, and Ubuntu 18.10 as the operating system.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **MAE** & **MSE** \\ \hline PMF & 0.7514 & 1.1321 \\ BiasedMF & 0.6277 & 0.7050 \\ NMF & 0.7950 & 1.3710 \\ BNMF & 0.6598 & 0.6987 \\ EMF-1 & 0.6046 & 0.6705 \\ EMF-2 & 0.6114 & 0.6653 \\ EMF-3 & 0.6013 & 0.6778 \\ EMF-4 & 0.6303 & 0.6780 \\ EMF-5 & 0.6109 & 0.6846 \\ EMF-6 & 0.6087 & 0.6808 \\ EMF-7 & 0.6108 & 0.6652 \\ EMF-8 & 0.6075 & 0.6672 \\ EMF-9 & 0.6209 & 0.7050 \\ EMF-10 & **0.5993** & **0.6581** \\ EMF (best) & 0.5993 & 0.6581 \\ EMF (worst) & 0.6303 & 0.7050 \\ EMF (avg) & 0.6105 & 0.6752 \\ \hline \hline \end{tabular}
\end{table}
Table 4: MAE and MSE in predictions for FilmTrust dataset.  
Figure 2: Tree representation of the best individual (EMF-4) achieved for the MovieLens dataset. The correspondence between node symbols and operators are defined in Table 1.  
Figure 4: Evolution of the 25 th percentile (first quartile) of the population fitness for each run on both (a) MovieLens and (b) FilmTrust datasets.  
Figure 3: Tree representation of the best individual achieved for the FilmTrust dataset (EMF-10). The correspondence between node symbols and operators are defined in Table 1.