# Deep Matrix Factorization Approach for Collaborative Filtering Recommender Systems
## 2 Materials and Methods
In the general case, if we computed \(s-1\) steps of the deep learning procedure, we compute the \(s\)-th matrix of errors as  
\[E^{s}=E^{s-1}-\hat{E}^{s-1}=E^{s-1}-P^{s-1}\cdot Q^{s-1}, \tag{4}\]  
Figure 1: Illustrative explanation of proposed DeepMF method. and we look for a factorization into matrices \(P^{s}\) of order \(n\times k^{s}\) and \(Q^{s}\) of order \(k^{s}\times m\) such that, as much as possible in the euclidean norm,  
\[E^{s}\approx\hat{E}^{s}=P^{s}\cdot Q^{s}. \tag{5}\]  
Once the deep factorization process ends after \(N\) steps, the original rating matrix \(R\) can be reconstructed by adding the estimates of the errors as  
\[R\approx\hat{R} =\hat{E}^{0}+E^{1}=\hat{E}^{0}+\hat{E}^{1}+E^{2}=\ldots\approx \hat{E}^{0}+\hat{E}^{1}+\hat{E}^{2}+\ldots+\hat{E}^{N} \tag{6}\] \[=P^{0}\cdot Q^{0}+P^{1}\cdot Q^{1}+\cdots+P^{N}\cdot Q^{N}=\sum_{s =0}^{N}P^{s}\cdot Q^{s}.\]  
For any step \(s=0,\ldots,N\), the factorization \(E^{s}\approx P^{s}\cdot Q^{s}\) is sought by the standard method of minimizing the euclidean distance between \(E^{s}\) and \(P^{s}Q^{s}\) by gradient descent with regularization, as in PMF [29]. In this way, if we write \(E^{s}=(e^{s}_{u,i})\), the rows of \(P^{s}\) are denoted by \(p^{s}_{1},p^{s}_{2},\ldots,p^{s}_{n}\) and the columns of \(Q^{s}\) are denoted by \(q^{s}_{1},q^{s}_{2},\ldots,q^{s}_{m}\), the loss function is given by  
\[\mathcal{F}^{s}(p^{s}_{u},q^{s}_{i}) =||E^{s}-P^{s}\cdot Q^{s}||^{2}+\lambda^{s}\left(\sum_{u=1}^{n}||p ^{s}_{u}||^{2}+\sum_{l=1}^{m}||q^{s}_{i}||^{2}\right) \tag{7}\] \[=\sum_{e^{s}_{u,i}\
eq\bullet}\left(e^{s}_{u,i}-p^{s}_{u}\cdot q^ {s}_{i}\right)^{2}+\lambda^{s}\left(\sum_{u=1}^{n}||p^{s}_{u}||^{2}+\sum_{i=1 }^{m}||q^{s}_{i}||^{2}\right),\]  
where \(\lambda^{s}\) is the regularization hyper-parameter of the \(s\)-th step to avoid overfitting. The previous loss function \(\mathcal{F}\) can be derived with respect to \(p^{s}_{u}\) and \(q^{s}_{i}\) resulting the following update rules for learning the model parameters  
\[E^{1}=R-\hat{E}^{0}=E^{0}-P^{0}\cdot Q^{0}. \tag{2}\]0 \[E^{1}=R-\hat{E}^{0}=E^{0}-P^{0}\cdot Q^{0}. \tag{2}\]1  
where \(\gamma^{s}\) is the learning rate hyper-parameter of the \(s\)-th step to control the learning speed. In this way, after finishing the nested factorization, the predicted ratings are collected in the matrix \(\hat{R}=(\hat{R}_{u,i})\), where the predicted rating of the user \(u\) to the item \(i\) is given by  
\[E^{1}=R-\hat{E}^{0}=E^{0}-P^{0}\cdot Q^{0}. \tag{2}\]2  
Note that the proposed method consists of successive repetitions of a mf process using the results of the previous mf as input. Therefore, it is possible to make an algorithmic implementation of the proposed method using a recursive approach. Algorithm 1 contains the pseudo-code of the recursive implementation of DeepMF. The algorithm receives as input the \(E\) matrix, which contains either the users' ratings to the items (\(R=E^{0}\)) for the first call or the errors of the previous factorization \(E^{s}\) for the successive ones. It also receives the model hyper-parameters: the number of latent factors on each step (\(K\)), the number of iterations of the gradient descent optimization on each step (\(T\)), the learning rates (\(\Gamma\)) and the regularizations (\(\Lambda\)). Note that these hyper-parameters were stacked so that each of the factorizations performed uses different hyper-parameters. The hyper-parameters of the first factorization will be placed at the top of the stack, those of the second factorization in the next one and so on until the parameters of the deeper factorization, which will be placed at the bottom of the stack. This allows us to define the stopping criteria of the algorithm: as soon as a stack is empty the learning process will be finished. Similarly, the output of the algorithm will be a stack containing the pairs \(\langle P,Q\rangle\) with the hidden factors of each of the factorizations carried out.