## 2 Methods
### Network Structure  
ViT is a deep learning model based on the Transformer architecture, first introduced by Google Research in 2020 [43]. Unlike traditional convolutional neural networks, it uses a self-attention mechanism to capture global features, improving feature extraction. The model structure is shown in Figure 5, where the ViT follows the classic design without modifications. Given an input Mel-spectrogram \(\mathbf{X}\in\mathbb{R}^{H\times W\times C}\), where \(H\) and \(W\) denote the height and width of the image, and \(C\) represents the number of channels, the image is first divided into \(N=\frac{HW}{P^{2}}\) non-overlapping patches of size \(P\times P\). Each patch is then flattened into a one-dimensional vector and passed through a linear projection layer that maps the original dimension \(P^{2}\cdot C\) to a fixed dimension \(D\), forming the initial patch embedding sequence \(\{\mathbf{z}_{1},\mathbf{z}_{2},\dots,\mathbf{z}_{N}\}\). Since the Transformer architecture itself lacks an inherent mechanism for modeling spatial positions, ViT introduces learnable positional embeddings \(\{\mathbf{p}_{1},\mathbf{p}_{2},\dots,\mathbf{p}_{N}\}\), which are added to the corresponding patch vectors to retain the spatial structure of the input. The resulting patch embedding sequence is then fed into the Transformer encoder, as shown in the red box of Figure 5. The encoder consists of alternating layers of Multi-Head Self-Attention (MSA) and Multilayer Perceptron (MLP) blocks. Each sub-layer is preceded by Layer Normalization, and followed by a Residual Connection, ensuring training stability and effective convergence. This completes one Transformer encoder block, which can be stacked multiple times to build a deeper ViT architecture. Figure 4: Mel-spectrograms at different sampling intervals: (**a**) 1 ms, (**b**) 2 ms, and (**c**) 4 ms.  
Figure 5: ViT structure. The MSA module serves as one of the core components, responsible for capturing long-range dependencies between different image patches from a global perspective. Unlike traditional convolutional operations, which rely on local receptive fields, MSA explicitly models the interactions between any pair of positions within the input sequence, enabling the network to learn global contextual relationships. As prepared in the previous stage, the input to the MSA module consists of N vectors, each of dimension \(D\), which together form an input matrix \(\mathbf{X}\in\mathbb{R}^{N\times D}\). The MSA module first applies three independent linear projections to the input, generating the Query (Q), Key (K), and Value (V) matrices:  
\[\mathbf{Q}=\mathbf{X}\mathbf{W}^{Q},\mathbf{K}=\mathbf{X}\mathbf{W}^{K}, \mathbf{V}=\mathbf{X}\mathbf{W}^{V} \tag{3}\]  
where \(\mathbf{W}^{Q},\mathbf{W}^{K},\mathbf{W}^{V}\in\mathbb{R}^{D\times d}\) are learnable weight matrices, and \(d\) denotes the dimensionality of each attention head. The computation performed by each attention head is as follows:  
\[\text{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\text{Softmax}\bigg{(} \frac{\mathbf{Q}\mathbf{K}^{T}}{\sqrt{d}}\bigg{)}\mathbf{V} \tag{4}\]  
This mechanism computes attention weights by measuring similarity between Queries and Keys, followed by a weighted sum of the Value vectors. In ViT, a multi-head attention mechanism is employed to capture diverse relationships among tokens. Specifically, \(h\) independent attention heads are applied in parallel:  
\[\text{head}_{i}=\text{Attention}(\mathbf{X}\mathbf{W}_{i}^{Q},\mathbf{X} \mathbf{W}_{i}^{K},\mathbf{X}\mathbf{W}_{i}^{V}),i=1,2,\dots,h \tag{5}\]  
Each head uses its own set of projection weights. The outputs from all heads are then concatenated and projected back to the original embedding dimension:  
\[\text{MSA}(\mathbf{X})=\text{Concat}(\text{head}_{1},\text{head}_{2},\dots, \text{head}_{h})\mathbf{W}^{O} \tag{6}\]  
where \(\mathbf{W}^{o}\in\mathbb{R}^{hd\times D}\) is the output projection matrix that restores the sequence to dimension \(D\).