# Improved Hierarchical Adaptive Deep Belief Network for Bearing Fault Diagnosis
## 2 Deep Belief Network and Its Improvement
### Restricted Boltzmann Machines (RBMs)
 \tag{9}\]  
where \(\langle\cdot\rangle_{p}\) represents mathematical expectations about distribution p, and \(p(h|v;\theta)\) indicates the probability distribution of the hidden layer, while the training data are known. The parameters \(w,a\), and \(b\) in \(\theta\) are expressed as follows:  
\[ \begin{array}{l}\frac{\partial\ln p(v;\theta)}{\partial w_{ij}}=p(h_{j}=1|v; \theta)v_{i}-\sum\limits_{v}p(v;\theta)p(h_{j}=1|v;\theta)v_{i}\end{array}  \tag{10}\]  
\[ \begin{array}{l}\frac{\partial\ln p(v;\theta)}{\partial a_{i}}=v_{i}-\sum \limits_{v}p(v;\theta)v_{i}\end{array}  \tag{11}\]  
\[\begin{method. The \(k\)-step Gibbs sampling method aims to initialize the state \(v^{(0)}\) of the visible neuron with a training sample (or a random state of the visible layer) and alternately perform the following sampling steps:  
\[\left\{ \begin{array}{l}h^{(i)}=G\Big{(}p\Big{(}h\big{|}v^{(i-1)}\Big{)}\\ v^{(i)}=G\Big{(}p\Big{(}v\big{|}h^{(i-1)}\Big{)}\Big{)}\end{array} \right.,i=1,2, \cdots,k \tag{13}\]  
where \(G(\cdot)\) represents the operation of Gibbs sampling. \(h^{(i)}\) and \(v^{(i)}\) represent the intermediate results of \(h\) and \(v\), respectively. When the number of sampling steps is large enough, a sample of the Gibbs distribution obeying the RBM representation can be obtained, and the gradient can be estimated by the following formula:  
\[ \begin{array}{rl}\frac{\partial\ln p(v;\theta)}{\partial w_{ij}}&=p(h_{j}=1|v ;\theta)v_{i}-\sum\limits_{v}p(v;\theta)p(h_{j}=1|v;\theta)v_{i}\\ &=p(h_{j}=1|v;\theta)v_{i}-p(h_{j}=1|v^{(k)};\theta)v_{i}^{(k)}\\ \\ \frac{\partial\ln p(v;\theta)}{\partial h_{i}}&=v_{i}-\sum\limits_{v}p(v; \theta)v_{i}\\ &=v_{i}-v_{i}^{(k)}\\ \\ \frac{\partial\ln p(v;\theta)}{\partial h_{j}}&=p(h_{j}=1|v;\theta)-\sum \limits_{v}p(v;\theta)p(h_{j}=1|v;\theta)\\ &=p(h_{j}=1|v;\theta)-p(h_{j}=1|v^{(k)};\theta)\end{array}  \tag{14}\]  
Typically, the large number of sampling steps leads to low training efficiency of RBM, especially when the characteristics of the training samples are large. [PERSON] (1992) proposed the contrastive divergence fast training algorithm. In this algorithm, the state of the visual neuron is first set to the training sample, and then the binary state of all hidden layer neurons are calculated via Gibbs sampling. Equation (7) and Gibbs sampling are applied to determine the state of the visible layer neurons to obtain \(v^{(1)}\). Generally, \(v^{(1)}\) is sufficient to estimate the gradient. The update criterion for parameter \(\theta=\{W,a,b\}\) is the following:  
\[ \begin{array}{l}W_{ij}^{[t]}\gets W_{ij}^{[t-1]}+\eta_{w}(\left\langle v _{i}h_{j}\right\rangle_{data}^{0}-\left\langle v_{i}h_{j}\right\rangle_{recon }^{1})\\ a_{i}^{[t]}\gets a_{i}^{[t-1]}+\eta_{a}(\left\langle v_{i}\right\rangle_{ data}^{0}-\left\langle v_{i}\right\rangle_{recon}^{1})\\ b_{j}^{[t]}\gets b_{j}^{[t-1]}+\eta_{b}(\left\langle h_{j}\right\rangle_{ data}^{0}-\left\langle h_{j}\right\rangle_{recon}^{1})\end{array}  \tag{15}\]  
where \(\eta_{w}\) is the learning rate of connection weight \(w\), \(\eta_{a}\) is the learning rate of visual layer deviation value \(a\), and \(\eta_{b}\) is the learning rate of hidden layer deviation value \(b\), their values range from 0 to 1. \(\langle\cdot\rangle_{data}\) stands for the original distribution of the data, \(\langle\cdot\rangle_{recon}\) represents the distribution of model definitions after a one-step Gibbs sampling.