# An Improved Deep Belief Network Prediction Model Based on Knowledge Transfer
## 3 Methods
### PLSR-Based Fine-Tuning Algorithm  
In the fine-tuning stage, the PLSR method is adopted. PLSR is a many-to-many modeling method. When multiple dependent variables and independent variables have multiple correlations, and the sample of observation data is small, the regression model established by PLSR has advantages that traditional classical regression analysis does not have. Although PLSR is a modeling method, it is also a supervised learning algorithm. This paper uses the PLSR method to fine-tune the supervised parameters of the created DBN structure. Using the PLSR method to replace the BP algorithm can overcome the problems of the gradient-based supervised learning algorithm, such as local optimization and gradient dispersion. Since the PLSR method has no gradient calculation, this has also become the main advantage of PLSR. The fine-tuning steps based on PLSR are as follows:  
S1: Extract the hidden layer feature vector matrix. After pre-training, the feature vectors of all hidden layers are extracted by using the input data X of the sample and the learned weights:  
\[(H_{l},H_{l-1},\ldots,H_{1})\ =\ (\hat{v}_{l},\hat{v}_{l-1},\ldots,\hat{v}_{1})X \tag{19}\]  
S2: Use PLSR to fine-tune network parameters. The PLSR modeling process is top-down, and PLSR equations are established every two layers, so as to obtain the output weight matrix optimized by the PLSR model. Figure 6: Perform a one-step process of expanding the network depth. \[Y\ =\ u_{1}r_{1}+F \tag{30}\]  
E and F in Equations (29) and (30) are the residual matrices of the two regression equations, respectively, and the regression coefficient vectors \(c_{1}\) and \(r_{1}\) are calculated using the least square method as:  
\[c_{1}\ =\ \frac{H_{l}^{T}u_{1}}{\|u_{1}\|^{2}} \tag{31}\]  
\[r_{1}\ =\ \frac{Y^{T}u_{1}}{\|u_{1}\|^{2}} \tag{32}\]  
(2) Extract the second component  
Substituting the residual matrix \(E\) and \(F\) for \(H_{l}\) and \(Y\), and using the same method to calculate \(p_{2}\) and \(q_{2}\), we get:  
\[u_{2}\ =\ Ep_{2} \tag{33}\]  
\[v_{2}\ =\ Fq_{2} \tag{34}\]  
Similarly, \(p_{2}\) and \(q_{2}\) are the unit eigenvectors corresponding to the largest eigenvalues of \(E^{T}FF^{T}E\) and \(F^{T}EE^{T}F\), respectively. Therefore, the regression equation of E and F to \(u_{2}\) is:  
\[E\ =\ u_{2}c_{2}^{T}+E_{1} \tag{35}\]  
\[F\ =\ u_{2}r_{2}+F_{1} \tag{36}\]  
Then, calculate the second set of regression coefficients:  
\[c_{2}\ =\ \frac{E^{T}u_{2}}{\|u_{2}\|^{2}} \tag{37}\]  
Figure 6:0  
(3) By analogy, starting from step (3), the optimal number of principal components can be determined by minimizing the absolute error (AE), and the iteration can be stopped. Figure 6:1  
where \(Y\) is the expected output, and \(\hat{Y}\) is the predicted output of extracting m principal components. (4) Establish the PLSR equation and get the optimized output weight vector  
After the value of m is determined, the PLSR equation of \(Y\) versus \(u_{1},u_{2},\ldots,u_{m}\) is finally obtained:  
Figure 6:2  
Equation (40) can be written in matrix form as follows:  
Figure 6:3  
Therefore, the PLSR model of \(y\) and \(h_{l}\) can be described as:  
Figure 6:4In Formula (42), \(\hat{y}\) is the prediction vector of \(h_{l}\), \(m(l)\) is the optimal number of components in the \(l\)th layer, \(R_{m(l)}\) is the regression coefficient matrix of \(h_{l}\), \(P_{m(l)}\) is the transformation matrix. Therefore, the output weight matrix optimized by the PLSR model is:  
Figure 6:5  
Next, the PLSR modeling process is repeated every two layers from \(h_{l-1}\) and \(h_{l-2}\) to \(h_{1}\) and \(v_{in}\). When the input layer and the first hidden layer are executed, the parameter fine-tuning phase ends, and all optimized weight matrices are obtained:  
\[\left\{ \begin{array}{l}w_{l}^{*}\ =\ R_{m(l-1)}^{T}P_{m(l-1)}^{T}\\ w_{l-1}^{*}\ =\ R_{m(l-2)}^{T}P_{m(l-2)}^{T}\\ \ldots\\ w_{1}^{*}\ =\ R_{m(in)}^{T}P_{m(in)}^{T}\end{array} \right. \tag{44}\]  
At this point, the independent construction process of DBN is complete.