# Deep Matrix Factorization Approach for Collaborative Filtering Recommender Systems
## 3 Results  
In this section we describe the empirical experiments designed to evaluate the performance of the proposed method. According to the standard CF's evaluation framework [32], the evaluation of the quality of the predictions is carried out by measuring the Mean Absolute Error (mae) of the model output. The MAE is defined as average absolute difference between the actual ratings \(R\) and the predicted ratings \(\hat{R}\) of the test split \(R^{\text{test}}\):  
\[\text{MAE}=\frac{1}{\#R^{\text{test}}}\sum_{\langle u,i\rangle\in R^{\text{ test}}}|R_{u,i}-\hat{R}_{u,i}|. \tag{10}\]  
Analogously, the quality of the top \(l\) recommendations is evaluated by the precision and recall quality measures on the recommendation lists. Remark that, given an user \(u\), his recommendation list of \(l\) items, \(L^{l}_{u}\), is the collection of the \(l\) items \(i_{1},\ldots,i_{l}\) with \(\hat{R}_{u,i_{1}},\ldots,\hat{R}_{u,i_{l}}\) the highest \(l\) predicted ratings. Now, we fix a parameter \(\theta\geq 0\), which plays the role of a minimum threshold to discern whether or not an item is of interest to the users. The precision is defined as the average proportion of successful recommendations of the recommendation list:\[\text{precision}=\frac{1}{\#U^{\text{test}}}\sum_{u\in U^{\text{test}}}\frac{ \left\{i\in L_{u}^{l}|R_{u,i}\geq\theta\right\}}{l}, \tag{11}\]  
where \(U^{\text{test}}\) is the collection of users in the test split. In the same vein, the recall is defined as the averaged proportion of successful recommendations included in the recommendation list with respect to the total number of test items the user \(u\) likes:  
\[\text{recall}=\frac{1}{\#U^{\text{test}}}\sum_{u\in U^{\text{test}}}\frac{ \left\{i\in L_{u}^{l}|R_{u,i}\geq\theta\right\}}{\left\{i\in R_{u}^{\text{ test}}|R_{u,i}\geq\theta\right\}}, \tag{12}\]  
where \(R_{u}^{\text{test}}\) is the collection of items in the test split at the \(u\)-th row of \(R\). The experimental evaluation has been carried out using MovieLens [33], FilmTrust [34] and MyAnimeList [35] datasets. Table 1 contains their main parameters. To ensure the reproducibility of these experiments, all of them have been run using the benchmark version of these datasets included in Collaborative Filtering for Java (CF4) [36]. The selection of the baselines has been done trying to pick a representative sample of the existing mf models. A first approach, based on the deep nature of the proposed method, is to focus on existing deep factorization models such as FAuST [25] and DNMF [27]. However recall that, as stated in Section 1, these methods were designed for image processing, not for solving cf problems, so a performance similar to other standard mf-based cf algorithms is not guaranteed. Trying to discern whether these methods would be suitable for cf, we proposed a preliminary experiment for evaluating the performance of FAuST against the MovieLens 100K dataset. To tune the hyper-parameters, a grid search was conducted whose limits are as follows. It was searched for factorizations of the rating matrix \(R\) into up to 4 smaller matrices, with 6 or 9 hidden factors for each level and a gradient descent step size (learning rate) of \(\gamma=0.0001\) or 0.01. Notice that FAuST also allows you to impose a sparsity constraint on each factor matrix. For this experiment, it was allowed a 0% of sparsity (i.e., fully dense matrix with no restrictions) or a 20% of sparsity (that is, up to 80% of non-vanishing entries are allowed). The best found model attained a mae of 3.49262 with a factorization into 4 matrices, all with 9 hidden factors, no sparsity constraints and a learning rate of \(\gamma=0.01\). This result is very far from the current baselines reported in the literature, which are typically around 0.75 as we will see below, showing that the existing deep methods are unsuitable as ss. In is worthy to mention that, although FAuST with no sparsity constraints is very similar to the usual PMF [29] algorithm, it looks for a factorization into normalized matrices with Frobenius norm 1.