# Bearing Fault Diagnosis Method Based on Improved VMD and Parallel Hybrid Neural Network
## 2 Model Construction Principle
### Recurrent Neural Network  
The recurrent neural network (RNN) can model and process sequence data [22]. Compared with the traditional feedforward neural network, there is a feedback connection between the neurons in the RNN, so that the RNN can remember and process the past input, to process the data with time sequence, sequence, or language nature [23].  
As an improved model of the cyclic network, the long short-term memory (LSTM) network realizes sequential signal processing through three gating units: the input gate filters the effective input, the forgetting gate controls the inheritance ratio of historical information, and the output gate adjusts the external response strength [24]. This design effectively alleviates the problem of historical information attenuation in the traditional circular network. However, in the face of long-term continuous monitoring data, the model still faces the problems of gradient disappearance and a significant increase in computational load. In contrast, the gated recurrent unit (GRU) adopts the simplified design concept and integrates the two gate control units of the forgetting gate and the input gate into an update gate [25] on the premise of maintaining the core functions to reduce the number of model parameters. This optimization improves the operation speed of processing data sets of the same size and achieves the effect of saving training and testing time. Its structure is shown in Figure 1.  
Figure 1: GRU network structure diagram.  
In the figure, \(z_{t}\) represents the update gate, \(r_{t}\) is the reset gate, \(h_{t-1}\) represents the value of the hidden unit at the previous time, \(x_{t}\) represents the input data at the current time, \(h_{t}\) represents the output data of the hidden layer, and \(\overset{\sim}{h}_{t}\) is the candidate state obtained after the compound operation of the state of the previously hidden layer \(h_{t-1}\) and the current input \(x_{t}\). The calculation process of the GRU network is as follows, where \(\sigma\) is the sigmoid activation function, and \(W_{z}\), \(W_{r}\) and \(W_{h}\) are the parameters of each gate neuron, respectively.  
\[z_{t}=\sigma(W_{z}\cdot[h_{t-1},x_{t}]) \tag{6}\]  
\[r_{t}=\sigma(W_{r}\cdot[h_{t-1},x_{t}]) \tag{7}\]  
\[\overset{\sim}{h}_{t}=tanh(W_{h}\cdot[r_{t}*h_{t-1},x_{t}]) \tag{8}\]  
\[h_{t}=(1-z_{t})*h_{t-1}+z_{t}*\overset{\sim}{h}_{t} \tag{9}\]  
Bidirectional long short-term memory (BiLSTM) and a bidirectional gated recurrent unit (BiGRU) introduce a bidirectional cyclic structure based on LSTM and GRU, which is composed of two unidirectional LSTMs or GRUs with opposite directions. Their advantage is that they can consider the impact of historical information and future information [26]. The bidirectional loop structure can effectively solve the limitations of RNN when processing long sequence data to improve the performance and stability of the model. Specifically, taking the BiGRU model as an example, for the forward GRU, its hidden state sequence \(\overset{\sim}{h_{1}},\overset{\sim}{h_{2}}\ldots\overset{\sim}{h_{t}}\) according to the conventional GRU calculation method is calculated from the beginning of the sequence to the end of the sequence. For the reverse GRU, its hidden state sequence \(\overset{\sim}{h_{t}},\overset{\sim}{h_{t-1}}\ldots\overset{\sim}{h_{1}}\) starts from the ending position of the sequence and reversely calculates to the starting position of the sequence. After obtaining the sequence of forward and reverse hidden states, BiGRU splices the forward and reverse hidden states of each time step to obtain the final hidden state representation. That is, for time step T, the final hidden status \(h_{t}\)BGRU is Formula (10), while for BiGRU, the GRU structure is replaced by the LSTM structure, and its calculation method remains the same, so it will not be repeated in the text.  
\[h_{t}{}^{BIGRU}=\left[\overset{\sim}{h_{t}},\overset{\sim}{h_{t}}\right] \tag{10}\]