## 2 Methodologies
### Deep Belief Network (DBN)
The flow chart of the greedy layer-by-layer learning algorithm is shown in Figure 4, whose detail process is as follows:  
First, for the first RBM, which is trained by the contrastive divergence (CD) algorithm, its outputs (the hidden layer, reconstructed visible layer and reconstructed hidden layer) can be obtained using the following equations:  
\[\left(h_{j}\right)_{data}^{(t_{1})}=p(h_{j}^{(t_{1})}=1\big{|}\mathbf{v}^{(t_{1})} \,)=\sigma(b_{j}+\sum_{i}{(v_{i})_{data}^{(t_{1})}w_{ij}}), \tag{1}\]  
\[\left(v_{i}\right)_{recon}^{(t_{1})}=p(v_{i}^{(t_{1})}=1\big{|}\mathbf{h}^{(t_{1}) }\,)=\sigma(a_{i}+\sum_{j}{(h_{j})_{data}^{(t_{1})}w_{ij}}), \tag{2}\]  
\[\left(h_{j}\right)_{recon}^{(t_{1})}=p(h_{j}^{(t_{1})}=1\big{|}\mathbf{v}^{(t_{1}) }\,)=\sigma(b_{j}+\sum_{i}{(v_{i})_{recon}^{(t_{1})}w_{ij}}), \tag{3}\]  
Figure 4: Flow chart of the greedy layer-by-layer learning algorithm. where \(\big{(}h_{j}\big{)}_{data}^{(t_{1})}\) represents the calculated output of the hidden layer neuron \(\big{(}h_{j}\big{)}\) at the \(t1\)th update, \((v_{l})_{data}^{(t_{1})}\) represents the input of the visible layer neuron \((v_{i})\) at the \(t1\)th update. \((v_{l})_{recon}^{(t_{1})}\) and \(\big{(}h_{j}\big{)}_{recon}^{(t_{1})}\) represents the reconstructed output of the visible layer neuron \((v_{i})\) and hidden layer neuron \(\big{(}h_{j}\big{)}\) at the \(t1\)th update, respectively. \(p(h_{j}^{(t_{1})}=1\big{|}v^{(t_{1})})\) represents the probability that the hidden layer is activated when it is updated. \(\sigma(x)\) is the logistic sigmoid function. The probability that the hidden layer, the reconstructed visible layer and the reconstructed hidden layer are activated when they are updated is the corresponding output value. After the visible layer and hidden layer are reconstructed, the model parameters of this RBM are updated as  
\[\Delta w_{ij}=\eta\big{(}\big{\langle}(v_{i})_{data}^{(t_{1})}(h_{j})_{data} ^{(t_{1})}\big{\rangle}-\big{\langle}(v_{i})_{recon}^{(t_{1})}(h_{j})_{recon} ^{(t_{1})}\big{\rangle}\big{)}, \tag{4}\]  
Figure 3.0  
Figure 3.1  
where \(\Delta w_{ij}\) is the update values of the weight, \(\Delta a_{j}\) is the bias of the visible layer, \(\Delta b_{j}\) is the bias of the hidden layer and \(\eta\) is the learning rate. Through the above formula, the initial weights of the model are learned from the structure of the input data in the pre-training stage, which can greatly improve the performance of the DBN. Then, it is judged whether the number of pre-trainings (\(t_{1}\)) reaches the maximum number of iterations (\(T_{1}\)). If the number of pre-trainings (\(t_{1}\)) does not reach the maximum number of iterations (\(T_{1}\)), the updated model parameters are used as the initial value to continue to update the first RBM. Until the iteration number of pre-training (\(t_{1}\)) reaches the maximum number of iterations (\(T_{1}\)), the pre-training of the first RBM ends. The model parameters of this RBM are used for the visible layer of the next RBM. Finally, the CD algorithm is used to train the high-level RBMs sequentially. Until the model parameters of all RBMs are updated, the pre-training of the DBN is over. The greedy layer-by-layer learning algorithm optimizes the weight of a DBN that is linear to network size and depth in time complexity. Moreover, since the approximation of the likelihood function only requires one step in this algorithm, the training time is significantly reduced. It can be found that, using the greedy layer-by-layer learning algorithm, the training of the DBN can be simplified to the training of multiple RBMs. Therefore, for the DBN, the computing process can be simplified, and the training speed can be improved. Moreover, the data in this stage are not labeled, and thus, the data mining ability of the DBN has been improved too. (3) Reverse fine-tuning  
In this stage, the network is trained by the supervised learning method. Therefore, the data have labels.