# Evolving Matrix-Factorization-Based Collaborative Filtering Using Genetic Programming
## 2 Materials and Methods
### Experimental Setup  
In order to evaluate the performance of the proposed method, we have designed the experiments described in this subsection. All these experiments have been carried out using two of the most popular datasets in the CF field: MovieLens [23] and FilmTrust [24]. The main properties of these datasets can be found in Table 2. We have chosen these datasets due their reduced number of ratings, since the proposed method's computational cost is too expensive to be evaluated with larger datasets. The main objective of our experimental evaluation is to compare the performance of EMF with respect to the standard MF techniques used for CF. We have selected the following MF techniques as baselines: PMF [25], the most popular factorization model for CF-based RS; BiasedMF [11], a factorization model that incorporate biases to represent the variations in the rating behavior of users and items; NMF [26], a model that factorizes the rating matrix into two new non-negative factors matrices; and BNMF [27], a model that factorizes the rating matrix into two non-negative matrices with an understandable probabilistic meaning. Selected baselines contain hyperparameters that must be tuned for each dataset. In order to maximize the accuracy of the predictions computed for each baseline, we have tuned these hyperparameters by performing a grid search approach. As a result of this search, the hyperparameters has been fixed to the following: for PMF, \(k=6\), \(\lambda=0.085\), and \(\gamma=0.01\) for both MovieLens and FilmTrust; for BiasedMF, \(k=6\), \(\lambda=0.06\), and \(\gamma=0.01\) for MovieLens and \(k=12\), \(\lambda=0.095\), and \(\gamma=0.03\) for FilmTrust; for NMF, \(k=6\) for MovieLens and \(k=6\) for FilmTrust; and for BNMF, \(k=6\), \(\alpha=0.9\), and \(\beta=5\) for MovieLens and \(k=4\), \(\alpha=0.8\), and \(\beta=5\) for FilmTrust. The proposed method, EMF, also contains two hyperparameters to control the SGD process. These hyperparameters are set to \(\gamma=0.001\) and \(\lambda=0.095\) for MovieLens and \(\gamma=0.035\) and \(\lambda=0.095\) for FilmTrust. These values have been chosen empirically. Furthermore, MF requires to set the number of latent factors (\(k\)). In the proposed approach, this hyperparameter must be interpreted as the maximum number of latent factors to be used during the GP optimization, since the GP algorithm may discard some of these latent factors to optimize the fitness function. We have set \(k=6\) for MovieLens and \(k=10\) for FilmTrust. All experiments has been carried out by splitting datasets into training and test sets. In both datasets, we have randomly selected 80% of the users and 80% of the items as training users and items, respectively. Remaining users and items has been labeled as test. The main goal of the proposed method is to improve the accuracy of the predictions performed with MF-based CF by evolving the aggregation function \(h\) that combines the user and item latent factors using GP. To quantify this improvement, we measured the accuracy of the predictions computed by EMF and selected baselines using MAE and Mean Squared Error (MSE) quality measures. MAE has been defined previously in Section 2.2. The MSE (Equation (7)) is defined as the averaged squared difference between the real rating (\(r_{u,i}\)) and the one computed by the aggregation function \(h\):  
\[MSE=\frac{1}{\#R^{test}}\sum_{<u,i>\in R^{test}}(r_{u,i}-h(p_{u,1},\ldots,p_{u,k},q_{i,1},\ldots,q_{i,k}))^{2}, \tag{7}\]  
where \(R^{test}\) represents the set of \((u,i)\) pairs for which the rating of the user \(u\) to the item \(i\) (\(r_{u,i}\)) has been selected as test. In order to ensure the convergence of the evolutionary method towards a definite solution, we have screened the time evolution of the sequence of best aggregate functions. Recall, as a result