# DPMF: Decentralized Probabilistic Matrix Factorization for Privacy-Preserving Recommendation
## 3 The Proposed Model
### Matrix Co-Factorization  
The key idea of matrix co-factorization is to associate implicit and explicit feedback together with the PMF model. Specifically, the model of decomposing explicit feedback matrix \(R\) and implicit feedback matrix \(M\) are linked by the user latent factor matrix \(U\). Based on Bayesian law [19], we define the posterior distribution over user latent factor matrix \(U\), item selecting latent factor matrix \(Z\) and item rating latent factor matrix \(V\) as:  
\[\ln p(U,V,Z,Q|R,M;\sigma_{U}^{2},\sigma_{V}^{2},\sigma_{Z}^{2}, \sigma_{Q}^{2},W) \tag{13}\] \[\propto p(R|U,V,Q;\sigma_{R}^{2})p(M|U,Z;W)p(U|\sigma_{U}^{2})p( V|\sigma_{V}^{2})p(Z|\sigma_{Z}^{2})p(Q|\sigma_{Q}^{2})\] \[=\prod_{i=1}^{m}\prod_{j=1}^{n}[\mathcal{N}(R_{ij}|g([U_{i}+ \frac{\sum\limits_{k=1}^{n}I_{ik}^{R}Q_{k}}{\sum\limits_{k=1}^{n}I_{ik}^{R}} |V_{j}),\sigma_{R}^{2})]^{\frac{R}{\sigma}}_{i}\prod_{(i,k)\in P}g(U_{i}^{T}Z _{k})\prod_{(i,k)\in N}(1-W_{i,k}g(U_{i}^{T}Z_{k}))\] \[\times\prod_{i=1}^{m}\mathcal{N}(U_{i}|0,\sigma_{U}^{2})\prod_{j =1}^{n}\mathcal{N}(V_{j}|0,\sigma_{V}^{2})\prod_{j=1}^{n}\mathcal{N}(Z_{j}|0, \sigma_{Z}^{2})\prod_{k=1}^{n}\mathcal{N}(Q_{k}|0,\sigma_{Q}^{2})\]  
Similar as we obtain the loss function in Equation (6), by maximizing the probability of the posterior distribution in Equation (13), we have the following loss:  
\[\min_{U,V,Z,Q}\mathcal{L}=\frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{n} I_{ij}^{R}(g([U_{i}+\frac{\sum\limits_{k=1}^{n}I_{ik}^{R}Q_{k}}{\sum\limits_{k=1 }^{n}I_{ik}^{R}}|V_{j})-R_{i,j})^{2} \tag{14}\] \[-\lambda(\sum\limits_{(i,k)\in P}\ln g(U_{i}^{T}Z_{k})+\sum \limits_{(i,k)\in N}\ln(1-W_{i,k}g(U_{i}^{T}Z_{k})))\] \[+\frac{\lambda_{U}}{2}||U||_{F}^{2}+\frac{\lambda_{V}}{2}||V||_{ F}^{2}+\frac{\lambda_{Z}}{2}||Z||_{F}^{2}+\frac{\lambda_{Q}}{2}||Q||_{F}^{2}\]  
where \(\lambda=\sigma_{R}^{2}\), with \(\lambda_{U}\), \(\lambda_{V}\), \(\lambda_{Z}\), \(\lambda_{Q}\) defines similar regularization parameters. The main idea for our method is training over explicit and implicit feedback using stochastic gradient descent (SGD) [20], so first we need to obtain the gradients of \(\mathcal{L}\) with respect to \(U_{i}\), \(V_{j}\), \(Z_{j}\) and \(Q_{k}\):  
\[\frac{\partial\mathcal{L}}{\partial U_{i}}=\left\{ \begin{aligned} &-\lambda\frac{g^{\prime}(U_{i}^{T}Z_{j})}{g(U_{i}^{T}Z_{j})}Z_{j}+ \lambda_{U}U_{i},\text{ if }(i,j)\in P,\\ &\lambda\frac{W_{i,j}g^{\prime}(U_{i}^{T}Z_{j})}{1-W_{i,j}g(U_{i} ^{T}Z_{j})}+\lambda_{U}U_{i},\text{ if }(i,j)\in N,\\ & g^{\prime}(\mathcal{U}_{i}^{T}V_{j})(g(\mathcal{U}_{i}^{T}V_{ j})-R_{i,j})V_{j}+\lambda_{U}U_{i},\text{ if }I_{ij}^{R}=1,\end{aligned} \right. \tag{15}\]  
\[\frac{\partial\mathcal{L}}{\partial V_{j}}=g^{\prime}(\mathcal{U}_{i}^{T}V_{ j})(g(\mathcal{U}_{i}  
If \(I_{ij}^{R}=1\), then for all \(k\) that satisfies \(I_{ik}^{R}=1\) we have:  
\[\frac{\partial\mathcal{L}}{\partial Q_{k}}=g^{\prime}(\hat{U}_{i}^{T}V_{j})(g( \hat{U}_{i}^{T}V_{j})-R_{i,j})V_{j}\frac{I_{ik}^{R}}{\sum\limits_{k=1}^{n}I_{ ik}^{R}}+\lambda_{Q}Q_{k}, \tag{18}\]  
where \(\hat{U}_{i}\) denotes \(U_{i}+(\sum\limits_{k=1}^{n}I_{ik}^{R}Q_{k})/\sum\limits_{k=1}^{n}I_{ik}^{R}\) to simplify the equations. After \(T\) rounds of training, the model reaches its convergence, as in Algorithm 1. As shown in (19), by mapping the product of \(U_{i}\) and \(V_{j}\) in the range of \([0,1]\), we can predict user \(i\)'s rating for the unknown items, which corresponds to the rating prediction task. Similarly, the logistic function value of the product \(U_{i}^{T}Z_{j}\) represents the predicted probability of user \(i\) selecting item \(j\), which corresponds to the item recommendation task.