## 2 Proposed Methods
### CL-Transformer Classification Model Based on Contrastive Learning  
In most supervised learning methods for audio classification, training is guided by cross-entropy loss. The cross-entropy loss measures the difference between the output probability distribution and one-hot encoded label and the model tries to reduce this loss by mapping the samples of the same class to their respective label vector. The core idea of contrastive loss is to optimize by defining positive and negative pairs, drawing closer samples belonging to the same class (positive pairs) and repelling those from different classes (negative pairs) [12]. In supervised contrastive loss, actual labels are used to generate positive pairs and negative pairs. Introducing supervised contrastive loss as a supplement to cross-entropy loss can further distinguish samples from different classes. By contrasting positive and negative sample pairs, the model is encouraged to learn the subtle differences between different classes, rather than simply memorizing specific labels. Furthermore, this mechanism implicitly mitigates the influence of noise in audio samples, enhancing the overall robustness of the classification system. Audio Spectrogram Transformer (AST) is a fully attention-based model capable of handling variable-length inputs [32]. Pre-trained AST models demonstrate superior performance in downstream tasks. However, when confronted with a limited dataset for such downstream tasks, the pre-trained model can easily overfit the characteristics of the training data, making it less adaptable to new tasks. As a flexible and strategic regu  
Figure 3: Combined Data Augmentation Process Diagram. larization technique, the incorporation of contrastive learning loss acts as a formidable safeguard, effectively mitigating the overfitting phenomenon and enhancing the model's generalizability. Based on the above, this paper proposes an improved CL-Transformer classification model for sound classification in noisy environments. This model framework modifies the Patch Embedding layer of the AST model and introduces the Patch-Mix method for feature fusion. Unlike traditional feature fusion methods that increase the feature space dimensions, Patch-Mix ingeniously offers a more holistic feature representation without augmenting the feature dimensions, simultaneously bolstering resistance capability against noise disturbances. Additionally, supervised contrastive loss is introduced as a supplement to cross-entropy loss. The model is shown in Figure 4.  
First, the model executes Patch Embedding on the input spectrogram, dividing it into a series of patches and adding learnable positional encodings to each patch. Then, Patch-Mix operation is carried out, mixing patches from the original spectrogram and the noisy spectrogram. These mixed patches are used as inputs to the Transformer encoder. Drawing upon the encoder's refined outputs, the model concurrently calculates both cross-entropy loss and contrastive loss, each uniquely encapsulating disparate information of the learning objective. Judging by a strategic redistribution and harmonious fusion of these two loss functions, we derive the definitive final loss, which subsequently steers the training trajectory of our method with superb accuracy. The baseline model AST is denoted as \(f(\cdot)\). The classifier model Linear Head consists of a normalization layer and a fully connected layer. The normalization layer processes the output of the Transformer Encoder, and the fully connected layer connects to the input features to the output label dimensions. The cross-entropy loss function is used to calculate the loss. The loss calculation is as follows:  
\[L_{i}^{cross-entropy}=-\sum_{i}y_{i}^{one-hot}\log\left(y_{i}^{logits}\right) \tag{2}\]  
where \(y_{i}^{one-hot}\) is the one-hot encoded true label of sample \(i\), and \(y_{i}^{logits}\) is the classifier output for that sample. Figure 4: CL-Transformer Network Model Diagram.