# Personalized Scholar Recommendation Based on Multi-Dimensional Features
## 4 The Mul-RSR Approach
### Model Training and Optimization
#### 4.3.2 Mul-RSR Training and Optimization  
We first employ Greedy Layer-Wise Pre-Training [34] to initialize model parameters, and unsupervised learning is used in each layer of the model to preserve the input information. Since pre-training optimizes each hidden layer in the network structure, these parameters are the local optimum of this layer, and they are used as initial parameters for subsequent training optimization.  
The following is the training process of the model. We define the network symbols, as shown in Table 1.  
The first step of the model training is forward propagation. This process starts from the input layer and then pushes forward layer by layer throughout the neural network. It calculates the state and activation value of each layer. This propagation process is mathematically expressed by the following formulas:  
\begin{table}
\begin{tabular}{c l} \hline \hline
**Symbol** & **Meaning** \\ \hline \(L_{l}\) & All neuron nodes in layer \(l\) \\ \(y_{l}^{(l)}\) & The output of the \(j\)-th neuron node in the \(l\) layer \\ \(u_{l}^{(l)}\) & The input of the \(j\)-th neuron node in the \(l\) layer \\ \(W_{l}\) & The weight matrix from layer \(l\)-1 to layer \(l\) \\ \(f(\cdot)\) & The activation function \\ \(b_{l}^{(l)}\) & The bias of the \(j\)-th node in the \(l\)-th layer \\ \hline \hline \end{tabular}
\end{table}
Table 1: Symbols in the network model.  
\[y_{l}^{(j)}=f(u_{l}^{(j)}) \tag{7}\]  
where  
\[u_{l}^{(j)}=\sum_{i\in L_{l-1}}W_{l}^{(ij)}y_{l-1}^{(i)}+b_{l}^{(j)} \tag{8}\]  
The above two formulas can be combined into the following formula:  
\[y_{l}=f(u_{l})=f(W_{l}y_{l-1}+b_{l}) \tag{9}\]  
In addition, the activation function uses the _Sigmoid_ regression function.  
The second training step is backpropagation. The model parameters are adjusted and optimized by using stochastic gradient descent (SGD). The propagation direction is from the output layer to the input layer. The partial derivatives and errors of each layer are calculated. The parameters of each layer are updated to optimize the model. The purpose of backpropagation is to minimize the loss function. Assuming that the label corresponding to the neuron in the output layer of the \(k\)-th layer in the network is \(t\), we use a quadratic loss function, expressed as follows:  
\[E=\frac{1}{2}\sum_{j\in L_{k}}(t^{(j)}-y_{k}^{(j)})^{2} \tag{10}\]  
Given the learning rate \(\alpha\), the parameter update formula is shown below [35; 36].  
\[W_{l}=W_{l}-\alpha\frac{\partial E}{\partial W_{l}} \tag{11}\]  
\[b_{l}^{(j)}=b_{l}^{(j)}-\alpha\frac{\partial E}{\partial b_{l}^{(j)}} \tag{12}\]  
The third training step is to use the attention mechanism to adjust the input feature weights again. First, we use the scoring function to calculate the correlation between the query vector and the input eigenvalue vector; it is expressed as:  
\[s(x_{i},q)=v^{T}tanh(Wx_{i}+Uq) \tag{13}\]  
\[a_{i}=\frac{exp(s(x_{i},q))}{\sum_{j=1}^{N}exp(s(x_{i},q))} \tag{14}\]  
where \(v\), \(W\), \(U\) are the weight values of the attention network, \(x_{i}\) is the input vector of eigenvalues, \(q\) is the query vector and \(a_{i}\) is attention distribution. Then, the calculation of weighted average of the attention distribution is as follows:  
\[att(X,q)=\sum_{n=1}^{N}a_{i}x_{i} \tag{15}\]  
where the set \(X\) represents the eigenvector matrix of \(N\) groups of inputs.