# Enhancing Automatic Modulation Recognition for IoT Applications Using Transformers
## 4 Experimental Results and Discussion
### Comparison with Other Baseline Methods
This highlights the suitability of our proposed models for IoT applications. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **F1 Score** & **Number of Parameters** \\ \hline DenseNet [21] & 56.93 & 3.3 M \\ CLDNN [21] & 61.14 & 1.3 M \\ VTCNN2 [23] & 61.53 & 5.5 M \\ CNN2 [22] & 60.94 & 1 M \\ ResNet [7] & 64.62 & 107 K \\ Mcformer [11] & **65.03** & - \\ \hline TransIQ (large variant) & **65.75** & 229 K \\ TransIQ (small variant) & **65.61** & 179 K \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparative analysis of F1 scores and parameter Counts across various models on the RadioML2016 [16] dataset. The highest F1 scores are in bold among the baselines and proposed models. Figure 5: Modulation-recognition accuracy comparison between the two variants of TransIQ and other baseline models on the CSPB.ML_2018+ dataset with a change in the SNR, where the SNR ranged from \(-\)19 dB to +20 dB. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **F1 Score** & **Number of Parameters** \\ \hline DenseNet [21] & 57.87 & 21.6 M \\ CLDNN [21] & 61.26 & 7.1 M \\ VTCNN2 [23] & 47.29 & 42.2 M \\ CNN2 [22] & 52.57 & 3.8 M \\ ResNet [7] & **65.48** & 164 K \\ \hline TransIQ (large variant) & **65.80** & 229 K \\ TransIQ (small variant) & 65.39 & 179 K \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparative analysis of F1 scores and parameter counts across various models on the CSPB.ML_2018+ dataset. The highest F1 scores are in bold among the baselines and proposed models. The confusion matrix, illustrated in Figure 6, shows the performance of the top-performing baseline model (ResNet), as well as TransIQ variants on the CSPB.ML.2018+ dataset. This matrix is a critical tool for evaluating model accuracy, detailing the relationship between actual and predicted class instances through its structured design. It is an \(N*N\) matrix, where \(N\) represents the number of classes for classification. Specifically, each row in the matrix corresponds to actual class instances, while each column reflects the model's predictions. The numbers along the diagonal of the matrix represent the true positives for each class, meaning the number of times the model correctly predicted each class. On the other hand, off-diagonal elements represent misclassifications. According to Figure 6, here, the confusion matrix is an \(8*8\) matrix, aligning with the eight different modulation types in the CSPB.ML.2018+ dataset. For instance, in the confusion matrix of the ResNet model, the value located at the intersection of the first row and the first column is 0.97. This indicates that 97% of the instances truly belonging to BPSK modulation were correctly classified as BPSK by the model. To illustrate, the element in the second row and first column of ResNet's confusion matrix has a value of 0.02, indicating that 2% of the instances that are actually QPSK modulation were incorrectly predicted as BPSK by the model. Further analysis of it reveals that ResNet still has significant confusion between QAM modulations. Some higher order PSK modulations are also classified as QAM. TransIQ is better able to correctly classify PSK modulations and shows a small amount of confusion among higher-order PSK. TransIQ is also much better able to discern between 16 QAM and other higher order QAM modulations. Both variants of TransIQ showed similar confusion between 64- and 256-QAM.