## 3 Methodology
### Frequency Domain Feature Extraction Module
#### 3.3.2 Frequency Domain Feature Extraction Based on Transformer  
In the frequency domain feature extraction module, the input samples first undergo a time-frequency transformation. The data is transferred to the frequency domain space, so as to obtain deeper timing features in the frequency domain space. From the conversion process of FFT, the output result \(f^{j}_{l}\) is a complex number and \(f^{j}\) is a set of complex sequences, i.e., the converted data is not in the range of real numbers and cannot be directly input into the model for training. In the frequency domain, amplitude and phase can fully reflect the characteristics of data in the frequency domain. The amplitude in the frequency domain refers to the amplitude information of the signal at different frequencies, reflecting the energy magnitude of the signal at different frequency components; the phase refers to the phase information of the signal at different frequencies, describing the phase shift of the signal waveform at different frequencies. After saving the complex data \(f^{j}_{l}\) as magnitude and phase values, the problem that complex numbers cannot be input into the model training can be solved. Specifically, the above is written in the following form: \(f^{j}_{l}=m+in\), where \(m\), \(n\) are real numbers, and the calculation of magnitude value \(a\) and phase value \(b\) is:  
\[a=\sqrt{m^{2}+n^{2}} \tag{10}\]  
\[b=arctan\left(\frac{m}{n}\right). \tag{11}\]  
The complex sequence \(f^{j}\) can be saved as two sets of timing sequences: the amplitude sequence \(a^{j}\) and the phase sequence \(b^{j}\), after the amplitude and phase are computed. Taking each input sample \(X=\left\{x^{1},x^{2},x^{3},\ldots x^{t}\ldots,x^{M}\right\}\), which is transformed as described above, we can say that \(X\rightarrow\left\{A,P\right\},\) where \(A\) is the amplitude sequence, \(P\) is the phase sequence, \(A,P\in R^{T\times M}\). The Transformer differs from traditional CNN or RNN frameworks in that it consists of two parts: an Encoder and a Decoder. The Transformer decoder is similar in structure to the encoder and consists of multiple coding layers stacked in three parts: a multi-head self-attention mechanism, residual linking with layer normalization, and a feedforward network. The magnitude and phase sequences are input-embedded and position-encoded before being fed into the Transformer encoder. The input \(X\) embedding process is:  
\[Q,K,V=Embedding(X) \tag{12}\]  
where: \(Q\) and \(K\) contain the positional information of the input data and \(V\) represents the numerical information of the input data. They will be used for the computation of the attention in the Transformer, and the computation process is:  
\[Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d_{K}}})V \tag{13}\]  
where: \(d_{K}\) is the dimension of the input vector. The multi-attention mechanism is to input \(Q\), \(K\), \(V\) into multiple self-attention modules after mapping them through a fully connected network. Then, the outputs of multiple self-attention are spliced and finally the outputsare integrated through the fully connected layer. The arithmetic process of the multi-head self-attention mechanism is as follows:  
\[MultiHead(Q,K,V)=Concat(head_{1},head_{2},\ldots head_{n})W^{0} \tag{14}\]  
\[Head_{i}=Attention(Q_{i},K_{i},V_{i}) \tag{15}\]  
where: \(W^{0}\) is the parameter matrix. Multiple outputs are connected after calculating the results of each attention head separately to obtain the output \(X^{T}\) of the multi-head attention layer. In the Transformer architecture, the Feed-Forward Network (FFN), Residual Connection, and Layer Normalization (LN) are core components. The FFN is used to perform independent nonlinear transformations on each feature, with the calculation process as follows  
\[FFN(X)=ReLU(XW_{1}+b_{1})W_{2}+b_{2} \tag{16}\]  
where: \(X\) is the input vector, \(W_{1}\) and \(b_{1}\) are the weight and bias of the first layer, \(W_{2}\) and \(b_{2}\) are the weight and bias of the second layer.