# New Vector-Space Embeddings for Recommender Systems
## 4 Evaluation and Results  
### Prediction  
The proposed models are tested on some of the most popular datasets in the Recommender Systems field: \"MovieLens\" [42]: \"MovieLens 100 K\" and \"MovieLens 1 M\". Moreover, we have also tested the \"ModCloth\" [43] dataset, which is a sparser dataset. Sparsity here refers to the fact that each user rated only a small number of items. Furthermore, we evaluated the performance on \"Amazon: Magazine_Subscriptions\" (Amazon MS) [44]. This dataset has a relatively small number of reviews/ratings, thus the results give us insight into how the approaches perform in such cases. Table 1 outlines the different aspects for each dataset used. For all these datasets, the ratings provided are on a scale from 1 to 5. For each dataset, we adopted the following experimental procedure:  
* Applied 5-fold cross-validation. * Five experiments were performed such that four folds were used as the training set and the remaining fifth fold was used as the test set, where in each experiment the folds were rotated as in the standard K-fold validation procedure. * The training set consisted of the transactions fed to the vectors' learning algorithm where users' and items' vectors of dimension 50 were learned for all the used datasets except the \"Amazon MS\" dataset where the vector's dimension is only 10. The ratings in the given transactions were first normalized by dividing by five before being fed to the algorithm (so the maximum rating value became 1 and the minimum rating value became 0.2). * The test set was the set used for performance evaluation by predicting the rating value for each transaction in the set. The prediction was performed by calculating the dot product between the user's vector and the item's vector involved in the transaction. Then, the obtained predicted rating was restored to the original ratings' scale (for example by multiplying by 5). \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Dataset**} \\ \hline
**Aspect** & **MovieLens 100 K** & **MovieLens 1 M** & **ModCloth** & **Amazon MS** \\ \hline Total number of ratings & 100,000 & 1,000,209 & 99,893 & 2275 \\ Number of users & 943 & 6040 & 44,784 & 348 \\ Number of items & 1682 & 3706 & 1020 & 157 \\ Maximum number of ratings by one user & 737 & 2314 & 250 & 30 \\ Minimum number of ratings by one user & 20 & 20 & 1 & 3 \\ Mean number of ratings by one user & 106 & 165.6 & 2.2 & 6.8 \\ Median number of ratings by one user & 65 & 96 & 1 & 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Datasets Aspects. Figure 1: Recommender Systems Average Pooling Method. * For each test set, two evaluation metrics were computed: Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) [45]. \(RMSE=\sqrt{(\frac{1}{|Q|})\sum_{(u,i)\in Q}(r_{ui}-\hat{r}_{ui})^{2}}\) \(MAE=\frac{1}{|Q|}\sum_{(u,i)\in Q}|r_{ui}-\hat{r}_{ui}|\), where \(Q\) is the test set, \(r_{ui}\) is user \(u\)'s true rating given to item \(i\) and \(\hat{r}_{ui}\) is the predicted rating of the Recommender System. * The mean value for each metric RMSE and MAE was computed for all the five folds. This is the value recorded in the results. Furthermore, we compared the obtained results with the benchmark methods introduced in [46; 47]. These methods are described as follows:  
* Singular Value Decomposition (SVD) [48]: SVD is the matrix factorization technique where the \(m\times n\) ratings matrix \(R\) (that includes users' ratings for the items) is factored into three matrices as follows: \(R=U.S.V^{T}\), where \(U\) and \(V\) are orthogonal matrices of sizes \(m\times r\) and \(n\times r\) respectively. \(r\) is the rank of the rating matrix \(R\). \(S\) is an \(r\times r\) diagonal matrix that includes all singular values of matrix \(R\) stored in descending order. The SVD approach is a kind of dimensionality reduction method and it provides the best lower rank approximations for the original matrix \(R\). The matrix \(S\) can be reduced to \(S_{k}\) by including only the largest \(k\) diagonal values.