# Improved Hierarchical Adaptive Deep Belief Network for Bearing Fault Diagnosis
## 2 Deep Belief Network and Its Improvement  
### Restricted Boltzmann Machines (RBMs)  
The DBN consists of stacked multilevel RBMs, which are based on the energy function definition and are the key preprocessing units in the field of deep learning. The RBM model is designed to fit the input data with maximum probability and train its hidden layer to capture high-order correlations in the input data. As shown in Figure 1, RBM consists of a hidden layer and a visible layer. A connection exists between each neuron node in the adjacent layers, and no connection exists within the same layer. Let \(v\)= \(\{v_{1},v_{2},\cdots,v_{i},\cdots,v_{n}\}\) denote the visible neuron nodes' states in the visible layer, and \(h\)= \(\{h_{1},h_{2},\cdots,h_{j},\cdots,h_{m}\}\) show the hidden neurons' states in the hidden layer. The neuron of an RBM is Boolean, i.e., there are only two states, 0 and 1. State 1 means that the neuron is activated, and state 0 means that the neuron is inhibited. The energy function [26] for a certain number of neuronal states is expressed as follows:  
\[E(v,h;\theta)=-\sum_{i=1}^{n}\sum_{j=1}^{m}w_{ij}v_{j}h_{j}-\sum_{i=1}^{n}a_{i }v_{i}-\sum_{j=1}^{m}b_{j}h_{j} \tag{1}\]  
where the RBM model parameters \(\theta=\{W,a,b\}\), \(a_{i}\) is the offset gradient of the visible layer, \(b_{j}\) the offset gradient of the hidden layer. \(v_{i}\) is the status of the \(i\)th neuron in the visual layer. \(h_{j}\) is the status of the \(j\)th hidden neuron. \(w_{ij}\) is the connection weight between the \(i\)th visible neuron and the \(j\)th hidden neuron. According to the Equation (1), the joint probability distribution of visual neurons and hidden neurons is defined as follows:  
\[P(v,h;\theta)=\frac{1}{Z(\theta)}\exp(-E(v,h;\theta)) \tag{2}\]  
Let \(Z(\theta)\) denote the normalization factor, which is expressed as:  
\[Z(\theta)=\sum_{v}\sum_{h}\exp(-E(v,h;\theta)) \tag{3}\]  
Figure 1: Architecture of a Restricted Boltzmann Machine (RBM). Thus, the edge probability and conditional probability of visual and hidden neurons can be acquired, as follows:  
\[ \begin{array}{l}P(v;\theta)=\frac{\sum\limits_{h}\exp(-E(v,h;\theta))}{\sum \limits_{v}\frac{E}{h}\exp(-E(v,h;\theta))}\\ P(v|h;\theta)=\frac{\exp(-E(v,h;\theta))}{\sum\limits_{v}\exp(-E(v,h;\theta))} \\ P(h;\theta)=\frac{\sum\limits_{v}\exp(-E(v,h;\theta))}{\sum\limits_{v}\exp(-E( v,h;\theta))}\\ P(h|v;\theta)=\frac{\exp(-E(v,h;\theta))}{\sum\limits_{h}\exp(-E(v,h;\theta))} \end{array}  \tag{4}\]  
The \(j\)th hidden neuron is activated with the probability calculated by:  
\[ \begin{array}{l}p(h_{j}=1|v;\theta)\quad=\frac{\sum\limits_{h_{j}}p(h_{j}=1,h_{j};v;\theta)}{\sum\limits_{h}p(h,v;\theta)}\\ =\sigma(\sum_{i=1}^{n}w_{ij}v_{i}+b_{j})\end{array}  \tag{6}\]  
where \(\sigma=1/(1+e^{-x})\) named sigmoid function is used here. Analogously, the \(i\)th visual neuron is activated with the probability calculated by:  
\[ \begin{array}{l}p(v_{i}=1|h;\theta)\quad=\frac{\sum\limits_{h_{j}}p(v_{i}=1,v_{h_{j}};v;\theta)}{\sum\limits_{v}p(v,h;\theta)}\\ =\sigma(\sum_{j=1}^{m}w_{ij}h_{j}+a_{i})\end{array}  \tag{7}\]  
In the training RBM process, the visible neurons' edge probability in the Gibbs distribution should be as close as possible to the input data distribution. Thus, the parameters \(\theta\) are optimized to maximize the edge probability \(P(v;\theta)\), the related logarithmic likelihood function is shown as:  
\[ \begin{array}{l}\theta^{*}=\underset{\theta}{\text{argmax}}L(\theta)= \underset{\theta}{\text{argmax}}\sum\
olimits_{v}\ln P(v;\theta)\end{array}  \tag{8}\]  
To acquire the optimal parameters \(\theta^{*}\), the log-likelihood function of RBM is maximized by the stochastic gradient descent technique. The partial derivative of the log-likelihood function with respect to the parameter is:  
\[ \begin{array}{l}\frac{\partial\ln p(v;\theta)}{\partial\theta}=\langle- \frac{\partial E(v,h;\theta)}{\partial\theta}\rangle_{p(h_{j};\theta)}- \langle-\frac{\partial E(v,h;\theta)}{\partial\theta}\rangle_{p(v,h;\theta)} \end{array}