# A Top-N Movie Recommendation Framework Based on Deep Neural Network with Heterogeneous Modeling
## 3 Our Approach: MFDNN
### Implementation of MEDNN  
A linear combination of potential features of a user and an item can be learned by matrix factorization, and \(p_{u}\) and \(q_{i}\) are used to represent the potential vectors of \(u\) and \(i\), respectively. The matrix factorization estimates the inner product of \(p_{u}\) and \(q_{i}\) as the prediction function value, as shown in Equation (8):  
\[\hat{y}_{ui}=p_{u}^{T}q_{i}=\sum_{k=1}^{K}p_{uk}q_{ik}. \tag{8}\]  
where \(K\) denotes the dimensions of latent space. However, using a simple inner product to estimate complex user-item interactions in the low-dimensional latent space limits the expression of MF and affects the generalization ability of the model. Thus, we define the mapping function of the first layer of MF as Equation (9):  
\[\phi(p_{u}^{F},q_{i}^{F})=p_{u}^{F}\odot q_{i}^{F}. \tag{9}\]  
where \(\odot\) denotes the element-wise product of vectors; the output of MF is given by Equation (10):  
\[\hat{y}_{ui,MF}=\alpha_{out}(h^{T}\phi(p_{u}^{F},q_{i}^{F})). \tag{10}\]  
where \(\alpha_{out}\) is an activation function. In consideration of convergence speed, we used the ReLU (rectified linear unit) function as the activation function, which is simply defined as \(\max(0,\,\,\text{x})\); \(h^{T}\) is the weight vector. For DNN, we first obtain the first layer results by processing the embedding layer using Equation (11):  
\[f_{1}=\phi_{1}(p_{u}^{I},q_{i}^{I}). \tag{11}\]  
In the same manner, the second layer results are obtained using Equation (12):  
\[f_{2}=\alpha_{2}(W_{2}^{T}f_{1}+b_{2}). \tag{12}\]  
Figure 3: Framework of **MEDNN**. where \(W_{2}^{T}\) and \(b_{2}\) are the weight matrix and biased vector, respectively, and \(\alpha_{2}\) is the activation function. The results of the N-th layer are obtained using Equation (13):  
\[f_{N}=\alpha_{N}(W_{N}^{T}f_{N-1}+b_{N}). \tag{13}\]  
According to Equations (11)-(13), we obtain the final DNN prediction result using Equation (14):  
\[\hat{y}_{ui,DNN}=\alpha(W^{|H|+1}\cdot\left[p_{u}^{I},q_{i}^{I}\right]^{T}+b^{ |H|+1}). \tag{14}\]  
where \(\alpha\) is the activation function, \(H\) is the number of hidden layers, and \(W\) and \(b\) are the weight matrix and biased vectors, respectively. Here, we chose the ReLU function as the activation function. The sigmoid activation function maps the output of each neuron to the (0,1) interval. This may hamper the performance of the model, and it is likely to cause an over-fitting problem; that is, when the output approaches 0 or 1, the neuron stops learning. Although [PERSON] mitigates the problem of the sigmoid to a certain extent, the result is a scaled version of the sigmoid function. Therefore, the ReLU activation function was selected for the model. The ReLU activation function avoids over-fitting and supports sparse data so that the model does not overfit. The explicit and implicit prediction results are generated by MFDNN. After obtaining the \(\hat{Y}^{+}\) and \(\hat{Y}^{-}\) by executing MFDNN, we can obtain the final user-item prediction results according to Equation (15):  
\[\hat{Y}=\omega_{1}\hat{Y}^{-}+\omega_{2}\hat{Y}^{+}(0\leqslant\omega_{1} \leqslant 1,0\leqslant\omega_{2}\leqslant 1). \tag{15}\]  
where \(\omega_{1}+\omega_{2}=1\), \(\omega_{1}\) is the weight of implicit feedback and \(\omega_{2}\) is the weight of explicit feedback. As for the best recommendation list for target users, we find the optimal weights by minimizing the objective function using Equation (16):  
\[ \begin{split} L&=\sum_{(u,i)\in Y}log\hat{y}_{ui} -\sum_{(u,i)\in Y^{-}}log(1-\hat{y}_{ui})+\frac{\lambda}{2}\left\|\omega_{1}^ {2}+\omega_{2}^{2}\right\|\\ &=-\sum_{(u,i)\in Y\cup Y^{-}}y_{ui}log\hat{y}_{ui}+(1-y_{ui}) log(1-\hat{y}_{ui})+\\ &\frac{\lambda}{2}\left\|\omega_{1}^{2}+\omega_{2}^{2}\right\|.\end{split}  \tag{16}\]  
In the network structure, each layer employs fewer neurons in succession. By using a small number of hidden units at the upper layer, more abstract features can be learned from the data.