## 2 Relevant Theoretical Models
### CNN + Transformer Network Architecture  
For signals with non-stationary characteristics, time-frequency methods can simultaneously provide both time and frequency information [36]. For stable signals such as vibration signals of faulty bearings, the FFT can accurately extract the amplitudes and phases of each frequency component. However, the time-frequency method may instead introduce redundant information. There is a lot of environment noise in the time-domain signal, which will make the effective features in the signal blurred. By using the FFT to convert the observed signal into a frequency-domain signal, it is possible to observe more directly which frequency components dominate the signal, which is also conducive to improving the model's efficiency. The computational complexity of the Discrete Fourier Transform is reduced from \(O(N^{2})\) to \(O(N\ logN)\). The specific process is as shown in Equation (9) as follows:  
\[X[k]=\sum_{m=0}^{N-1}x[n]e^{-j2\pi\hbar n/N} \tag{9}\]  
where \(e^{-j2\pi\hbar n/N}\) is a complex exponential function, and \(k\) is the index in the frequency domain. A CNN + Transformer transfer learning network is proposed in this paper. The frequency-domain signals of one-dimensional fault signals after FFT transformation and the time-domain features of the signals themselves are extracted for global features through the CNN convolutional pooling layer. Subsequently, the Transformer encoder captures the dependencies in the sequence data. Finally, decisions on classification are made using the fully connected layer. The classification is output through the softmax activation function. The network's main architecture is shown in Figure 1.  
The feature extractor aims to extract the shallow features of the vibration signal, and to extract the shallow features, a convolutional neural network is built. The convolution part is one of the most common CNN models. The convolution layer is the CNN's core part, and its primary purpose is to extract characteristics from signals. When all the time-frequency signals from the source domain enter the convolution layer, through several convolution kernels of the same size, the convolution operation results of the signal after several convolution kernels can be output. The process of the time-frequency signal passing through the convolutional layer is shown in Equation (10). \[y[n]=\sum_{m=0}^{M-1}x\bigg{[}n+m-\frac{M-1}{2}\bigg{]}h[m] \tag{10}\]  
where \(y[n]\) represents the output signal value at position \(n\), \(x[n]\) represents the input signal value at position \(n\), \(h[n]\) is the convolution kernel value at position \(n\), and \(M\) is the length of the convolution kernel. The network can learn complex functional relationships and output accurate prediction results, mostly relying on the nonlinear characteristics of the activation function. When the traditional activation functions Sigmoid (S-shaped function) and tanh (hyperbolic tangent function) input the extreme-value region, the gradient is close to 0, which is prone to causing the vanishing gradient problem and further affecting the training effect of the network. The _ReLU_ (Modified Linear Unit) function can alleviate these situations. The ReLU function is shown in Equation (11). \[ReLU(x)=\max(0,x) \tag{11}\]  
where \(x\) is the feature output by the previous convolution layer. The number of samples that need to be trained is relatively large. After the samples undergo convolution and the _ReLU_ function, the maximum feature value of a certain region is selected as the representative through the max pooling layer, effectively compressing the features and reducing the number of training parameters. The max pooling operation is shown in Equation (12). \[O[i]=max_{m}\{I[i+m]\} \tag{12}\]  
where \(O[i]\) represents the value of the output feature, \(I[i+m]\) represents the input feature value \(i+m\), \(m\) represents the offset within the window, and \(max_{m}\) represents the maximum value of the given window.