## 4 Results
### Baseline Experiments  
We set up three sets of baseline experiments. The goals of the baseline experiments include determining the best hyperparameters, evaluating the performance of the baseline models, and assessing the effect of the datasets on the baseline models. In the first set of experiments, we validated the accuracy and stability of the selected evaluation models DAE [27], SE-MSFN [29], and LSTM2 [28] for various batch sizes, since the setting of hyperparameters has a significant impact on the training of deep learning models [48; 49]. During our experiments, we found that the hyperparameter batch size has a significant impact on model training. The test results on the 2016A dataset are shown in Figure 3, where the vertical error bars represent the standard deviation of the three experiments, and the shorter the error bars, the better the model stability. On the balanced dataset, we can see that (1) the recognition accuracy of the three models decreases as the batch size increases; (2) the SE-MSFN recognition accuracy outperforms the LSTM2 and DAE; and (3) the SE-MSFN and LSTM2 stability outperform the lightweight model. We set the batch size of all subsequent experiments to 128 to ensure a fair comparison and to account for performance, training efficiency, and stability. In the second set of experiments, we analyze and compare the performance of the baseline model on the datasets 2016A, 2016A-1, 2016A-10, 2016A-20, and 2016A-50. As can be seen from Figure 4, the performance of the baseline models tends to decrease as the degree of data imbalance increases and the amount of data decreases. The best-performing model on each dataset is SE-MSFN, which not only has consistently optimal recognition accuracy, but also has good model stability. SE-MSFN and LSTM2 perform similarly on both the well-balanced and the slightly imbalanced datasets. At imbalance factors of 20 and 50, the accuracy of LSTM2 decreased sharply, and the average recognition accuracy of SE-MSFN exceeded that of LSTM2 by 15.51% and 14.79% for the three experiments, and LSTM2 was 11.70% and 7.15% lower than the DAE of the lightweight model, respectively. Figure 5 shows the training process of LSTM2. The model was trained with high accuracy on the 2016A-1 and 2016A-50 datasets, and the validation loss started to increase at around 40 epochs, while the validation accuracy leveled off. We used an early stop strategy to avoid overfitting by monitoring the validation loss. However, the validation accuracy of the model on 2016A-1 was significantly higher than the results on 2016A-50. As the training process progressed, the gap between the model's training and validation accuracy grew significantly larger on 2016A-50 than on 2016A-1, indicating that LSTM2 overfitted more severely on the small-scale imbalanced dataset. Figure 3: Performance of the baseline model for recognition on the 2016A dataset at various batch sizes. In the third set of experiments, we investigate the two models with the best average recognition performance (SE-MSFN and LSTM2) on a class balance dataset for each category. Figure 6 shows the results. We can easily see that the average recognition accuracy of both models for WBFM signals, which are hard samples, is less than 40%. This is due to the fact that the modulated signal is a real audio stream with silent periods generated from a real audio stream with silent periods [28]. All existing methods make effective identification difficult. In addition, as can be seen from Figure A1, the models easily confuse QAM16 and QAM64 due to the fact that QAM16 and QAM64 are members of the same modulation family, while QAM16 is a subset of QAM64. When the SNR is below 0 dB, the 8 BPSK and QPSK signals have a lower recognition accuracy, resulting in an average recognition accuracy of less than 80%. According to our rules for constructing our class-imbalanced data, the signals of the four modulation types QAM16, QAM64, QPSK, and WBFM belong to the minority class, the tail-end class of the long-tailed distribution data.