# Domain Adaptation Network with Double Adversarial Mechanism for Intelligent Fault Diagnosis
## 2 Theoretical Background  
### Convolutional Neural Network  
The convolutional neural network (CNN) is simply the inner product operation of the image and the filter matrix. On the one hand, the CNN reduces the complexity of the model through local connection, thus reducing the risk of network overfitting; on the other hand, the CNN adopts the method of weight sharing and the number of weights is reduced, which makes it easy to optimize the network [39]. The basic structure of CNN includes a convolution layer, activation layer, pooling layer and full connection layer. The main functions of each part in CNN are as follows.  
The first is the convolution layer. The main function of convolution is to extract features, so as to obtain a new set of features. The mathematical formula can be expressed as  
\[z^{[l]}=W^{[l]}*a^{[l-1]}+b^{[l]} \tag{1}\]  
where \(a^{[l-1]}\) represents the input of layer \(l\), \(z^{[l]}\) represents the convolution output of layer \(l\) and \(W^{[l]}\) and \(b^{[l]}\) represent the corresponding weight and bias, respectively.  
The second is the activation layer. The purpose of the activation layer is to add nonlinear factors to solve problems that the linear model cannot solve. The activation function adopts scaled exponential linear units (\(SELU\)) with its own regularization function, which can prevent network overfitting to a certain extent. The mathematical expression is as follows:  
\[Z^{[l]}=SELU(z^{[l]}) \tag{2}\]  
where \(Z^{[l]}\) represent the output of layer \(l\) after activation.  
When the net input distribution is consistent (such as normalization to the standard normal distribution), the optimization efficiency will be improved. Therefore, batch normalization (\(BN\)) is added after the activation and the mathematical formula is simplified as follows:  
\[B^{[l]}=BN(Z^{[l]}) \tag{3}\]  
Then, there is the pooling layer. The pooling layer is mainly used to select features and reduce the number of neurons in the feature mapping group, so as to reduce the dimension of features, thus avoiding network overfitting. The proposed method mainly uses two pooling functions, max pooling (max) and global average pooling (average). Max poolingis mainly used in the first few layers of CNN and global average pooling is mainly used in the last layer. The mathematical expressions are as follows:  
\[h^{[l]}=\max(B^{[l]}) \tag{4}\]  
\[H^{[l]}=average(B^{[l]}) \tag{5}\]  
where \(h^{[l]}\) and \(H^{[l]}\), respectively, represent the outputs of the CNN after max pooling and global average pooling.  
Finally, there is the full connection layer. It recombines the highly abstracted features after multi-layer convolution. Then, it carries out normalization and outputs a probability for all kinds of classification cases. Finally, the classifier can obtain classification results according to the probability obtained by full connection. Using \(f^{[l+1]}\) to represent the output of the full connection layer \(l+1\) and \(p(x)\) to represent the probability output of Softmax, the formulas are as follows, respectively.  
\[f^{[l+1]}=W^{[l+1]}H^{[l]}+b^{[l+1]} \tag{6}\]  
\[p(x)=[p(C=n|x)]=\frac{\exp\Bigl{(}f^{[l+1]}_{n}\Bigr{)}}{\sum\limits_{n=1}^{N} \exp\Bigl{(}f^{[l+1]}_{n}\Bigr{)}} \tag{7}\]  
where \(n\) represents the label to which the current sample belongs and \(N\) represents all categories of the sample.