# An Improved Deep Belief Network Prediction Model Based on Knowledge Transfer
## 3 Methods
### DBN Structure Construction Method Based on Knowledge Transfer  
This section provides the DBN self-construction pre-training stage based on knowledge transfer to improve the convergence speed and prediction performance of the model. The specific implementation steps are as follows:  
S1: Standardize the data set. Including the analysis of the data set, classification, removal of noise, false data, feature standardization, etc. Standardize the data range to the [0,1] interval. Formula (12) is used for standardization processing. \[x_{ij}^{*}\ =\ \frac{x_{ij}-\overline{x}_{j}}{s_{j}} \tag{12}\]  
\[s_{j}\ =\ \sqrt{\frac{1}{N}\sum\limits_{j\ =\ 1}^{N}\left(x_{j}-\overline{x} \right)^{2}} \tag{13}\]  
where \(x_{ij}\) represents the sample point, \(\overline{x}_{j}\) represents the mean value of the \(j\)-th variable, and \(s_{j}\) represents the standard deviation of \(x_{j}\). The formula for standard deviation is shown in (13). After preprocessing, the \(N\) training samples \(X\) are divided into three categories: training set \(X_{tr}\), validation set \(X_{\text{ru}}\) and test set \(X_{te}\). S2: Initialize model parameters. Initialize a basic DBN structure with a network depth of 2, including the visible layer (input layer), hidden layer, and output layer. In addition, there are strict requirements on the batch classification of experimental data, the setting of the number of initial hidden layer units, the setting of initial weights and biases, the learning rate and the number of iterations. Faced with a large number of data samples, it is necessary to classify samples for model training. Small batches of sample data can alleviate the computational burden. We set the batch of experimental data to 30 records per training. The number of initial hidden layer units is set to 40 for a single hidden layer unit. Initialize the connection weight, the visible layer and the hidden layer unit bias to N (0,0.1) random numbers conforming to the normal distribution. Learning rate is an important parameter, which not only affects the training efficiency of pre-training, but also affects the parameter update in the fine-tuning stage. We set the learning rate in the pre-training phase to 0.01, and the learning rate in the fine-tuning phase to 0.1. A reasonable number of iterations can effectively improve model performance. In order for each RBM to be fully trained, the number of iterations is set to 300. S3: Pre-training (feature learning). In pre-training, each RBM uses the CD algorithm from bottom to top to complete the training and save the network parameters \(W^{(i)}\in R^{m\times n}\) and \(W^{(i+1)}\in R^{n\times p}\). The purpose of model training is that DBN can reflect real data to the greatest extent after feature learning. This process can predict and classify data by extracting features and reducing dimensionality of high-dimensional data. S4: Calculate the reconstruction error. In each layer of RBM, the reconstruction value of the input sample of the visible layer is obtained through the CD algorithm, and the reconstruction error is calculated by the difference between the reconstruction value and the initial input sample. If the reconstruction error reaches the minimum preset value, stop building and go to S7, otherwise, go to S5. The definition of reconstruction error is as follows:  
\[\text{R}error\ =\ \frac{\sum\limits_{i}^{n}\sum\limits_{j}^{m}\left|v_{ij}- \vartheta_{ij}\right|}{n\cdot m} \tag{14}\]  
where \(n\) and \(m\) are the number of training samples and the number of features, respectively, and \(v_{ij}\) and \(\vartheta_{ij}\) are the original input and reconstruction input, respectively. In order to prevent over-fitting of training data or large deviation of reconstruction data, while balancing the network training cost, when the difference between the two reconstruction errors is less than the preset value \(\varepsilon\), the network stops expanding.