## 3 The Proposed Methodology
### DNN Construction-Based Deep Learning
#### 3.1.2 Design DNN Construction  
Recent research shows that it is impossible to transfer deep features and classifiers from the source domain to the target domain using a pure deep high-level model [42; 43]. Therefore, we develop the integrated method of DNN to learn the classifiers and transferable invariant features of source and target domains by combining the LSTM and CNN models, as shown in Figure 3. The proposed model includes three convolutional modules, one LSTM module, one shallow layer, three dense layers and one softmax classifier. One-dimensional convolution was chosen because the vibration signal contains only one-dimensional data. In certain processes, initially adding a batch back to the convolution layer and pooling layer may cause the input to be pulled back to the convolution layer to obey the standard normal distribution. This can prevent the gradient from disappearing while further speeding up convergence and training speed. Then, adding the LSTM network after the pooling layer, can solve the problem of long-term dependence or gradient explosion to better refine the properties. Finally, to prevent overfitting and improve the generalisation ability, the dropout layer is added to the full connection layer. The structure of the CNN consists of Convolution, BatchStandardisation and Maximum Pooling. The input of Convolution is (None, 2048, 1), the output is (None, 128, 16), and the input of BatchStandardisation is (None, 128, 16), the output is (None, 128, 16), the input of MaxPooling1D is (None, 128, 16), the output is (None, 64, 16). After the CNN structure has been traversed three times, the LSTM network structure is added, whose input is (None, 16, 32) and output is (None, 16, 4). Table 1 shows the selection of architectural parameters of the DNN model by grid search and k-fold cross-validation [44]. Figure 3: The architecture of DNN. Figure 2: The principle of SMOTE. A deep neural network fault diagnosis model for significant faults is created and trained, which can be described as follows:  
\[DNN_{s}\ =Feed\,forward(f_{s1},f_{s2},\ldots,f_{sN}) \tag{3}\]  
\[[DNN_{s}\,,\theta_{s}\,]=train(DNN_{s},X_{s},\theta_{s}) \tag{4}\]  
\(DNN_{s}\) consists of CNN and LSTM. \(f_{sj}\) represents the number of neurons in the jth buried layer of \(DNN_{s}\), \(j=1,2,\ldots,N\). First, as shown in Equation (4), the model \(DNN_{s}\) from significant fault is trained. \(X_{s}\) is a training dataset from significant faults. \(\theta_{s}=\{\theta_{s1},\theta_{s2},\ldots,\theta_{sN}\}\) is an initial set of parameters for the network \(DNN_{s}.\theta_{si}=\{W_{si},\ b_{si}\}\) represents the set of parameters of bias and weight matrix of the input layer and hidden layer in \(DNN_{s}\), which are initialized randomly. Second, Layer-by-layer training updates \(DNN_{s}\) parameters \(\theta_{s^{\prime}}\). Third, abstract features \(F_{sN}=\sigma(W_{sN}\ldots(\sigma(W_{s2}(\sigma(W_{s1}X_{s}+b_{s1})+b_{s2}))+ \ldots+b_{sN})\) are captured. Fourth, using \(F_{sN}\) as input data, the softmax classifier is trained to update and obtain the softmax parameters \(\theta_{ss^{\prime}}\). Fifth, \(DNN_{s}\) parameters are fine-tuned and optimised using a supervised back-propagation algorithm. \(DNN_{s}\) is optimised by minimizing loss function with labelled dataset, which can be described as follows:  
\[loss=-\sum_{i=1}^{outputsize}y_{i}.logp_{i} \tag{5}\]  
where \(p_{i}\) is actual output probability, and \(y_{i}\) is expected output. As the mode includes CNN and LSTM, the loss function is enhanced as follows:  
\[loss=-\sum_{i=1}^{outputsize}y_{i}.logp_{i}+\lambda \tag{6}\]  
with  
\[\lambda=\frac{F\text{max}}{F_{cnn}+F_{lstm}},F\text{max}=\text{max}(F_{cnn},F _{lstm}),\lambda\subset(0.01,1) \tag{7}\]  
\[R=\frac{TP}{TP+FN} \tag{8}\]