## Materials and Methods
### Deep Belief Network (DBN)
#### 2.2.1 Forward RBM Pre-Training  
The Boltzmann machine (BM) is a probabilistic modeling method based on an energy function. It has a strong unsupervised learning ability. In theory, it can learn arbitrarily complex rules and apply the rules to the data. Inter-unit connections of the inner- and inter-layer are complex, so there are also disadvantages such as a long training time, a large number of calculations, and difficulty in obtaining the probability distribution [33].  
RBM is an improvement over BM. It is a two-layer recursive neural network. Random binary input and random binary output are connected via symmetrical weights, as shown in Figure 2. The two-layer system consists of n dominant neurons (corresponding to the visible variable) and m recessive neurons (corresponding to the hidden variable h), where the v and h elements are binary variables whose state is 0 or 1. There is a weight value connection between the cell layer and the hidden cell layer, but there is no connection between the cells in the layer.  
RBM is an energy-based model. For a given set of states \((v,h)\), the joint configuration energy function [34] of the visible and hidden units is  
\[E(v,h|\theta)=-\sum_{i=1}^{n}\sum_{j=1}^{m}w_{ij}h_{j}v_{i}-\sum_{j=1}^{m}c_{j }h_{j}-\sum_{i=1}^{n}b_{i}v_{i}. \tag{10}\]  
Figure 1: Deep belief network structure. RBM: Restricted Boltzmann Machine; BP: Back-Projection.  
where \(\theta=(\omega,b,c)\) is the parameter of the RBM model; \(v_{i},b_{i}\) are the respective states and offsets of the i-th visible unit; \(h_{j},c_{j}\) are the respective states and offsets of the \(j\)-th hidden unit; \(\omega_{ij}\) is the \(i\)-th visible unit and the connection weight between the \(j\)-th hidden units. The structure of the RBM model is very special. There are no connections within the layers and all the layers are fully connected. When the activation state of each visible layer unit is given, the activation states of the neurons in the hidden layer are independent of each other, and \(\sigma(x)=\frac{1}{1+\exp(-x)}\) is a sigmoid activation function. Therefore, the activation probability of the \(j\)-th neuron in the hidden layer is  
\[p(\,h_{j}=1|v,\theta)=\sigma(b_{j}+\sum_{i}v_{i}w_{ij}) \tag{11}\]  
Similarly, when the state h of the hidden unit is given, the active states \(v_{i}\) of the visible units are also mutually independent, and the activation probability of the i-th visible unit is  
\[p(\,v_{i}=1|h,\theta)=\sigma(c_{i}+\sum_{j}h_{j}w_{ij}) \tag{12}\]  
RBM is a stable network structure. By maximizing the log likelihood function \(L(\theta)\) of the RBM on the input training set to obtain the model parameter \(\theta\), the training data set can be fitted. The hidden layer can subsequently be used as the characteristics of the visible layer data.  
\[\hat{\theta}=\underset{\theta}{\text{argmax}}L(\theta)=\underset{\theta}{ \text{argmax}}\sum_{t=1}^{N}\text{lg}\,P(v^{(t)}\Big{|}\text{h},\theta) \tag{13}\]  
To quickly train a log likelihood gradient of an RBM, the data distribution of Gibbs sampling [35] can be used as the RBM definition expectation, and then the weight and offset parameter update criteria can be obtained:  
\[\left\{\begin{array}{l}w_{ij}^{k+1}=w_{ij}^{k}+\epsilon(<v_{i}h_{j}>_{data} -<v_{i}h_{j}>_{recon})\\ a_{i}^{k+1}=a_{i}^{k}+\epsilon(<v_{i}>_{data}-<v_{i}>_{recon})\\ b_{j}^{k+1}=b_{j}^{k}+\epsilon(<h_{j}>_{data}-<h_{j}>_{recon})\end{array}\right. \tag{14}\]  
where \(\epsilon\) is the learning rate, taking any value in the interval [0,1]; the data are the expectation of the distribution of the original observation data; recon is the desired distribution defined by the RBM model.  
Figure 2: RBM structure diagram (V stands for the display element and h stands for the hidden element).