# CNN-BiLSTM-DNN-Based Modulation Recognition Algorithm at Low SNR
## 4 Experiment and Conclusion Analysis
### Analysis of Experimental Results  
#### 4.2.1 Recognition Accuracy  
The fundamental challenge of modulation signal classification recognition is one of classification, where classification accuracy is defined as the proportion of correctly predicted samples among all input samples, as stated in the Formula (16) [21]:  
\[acc=\frac{tp+tn}{tp+tn+fp+fn} \tag{16}\]  
where \(tp\) represents a sample labeled as correct and a model prediction as also correct. \(tn\) represents a sample that is labeled wrong and a model prediction that is also wrong, and \(fp\) represents a sample that is labeled wrong and a model prediction that is correct. \(fn\) represents a sample marked as correct but mispredicted by the model.  
To validate the proposed algorithmic structure, a comparison was made with algorithms such as a modulation classification convolutional neural network (MCNet) [22], convolutional gated recurrent fully connected deep neural network (CGDNet) [23], residual network (ResNet) [24], multi-channel convolutional long short-term deep neural network (MCLDNN) [25], and improved convolutional neural network-based automatic modulation classification network (IC-AMCNet) [26]. Figure 4 shows the accuracy of each SNR of the six models under the RadioML2016.10b dataset. Ours in the figure represents the model proposed in this paper. The classification accuracy of different network structures can be clearly seen in the figure, and when SNR rises, so does its accuracy. The maximum recognition accuracy at 12 dB reaches 93.79%, the greatest of all comparison models. The CNN-BiLSTM-DNN network structure performs noticeably better in terms of recognition accuracy than other networks when the signal-to-noise ratio is low, ranging from \(-\)10 dB to 4 dB. This further validates the efficacy of the SP module denoising method and enhances modulated signal classification and recognition.  
To demonstrate our innovative model, we conducted simulations on the RadioML2016.10a dataset. According to Figure 5, the rate of recognition of our CNN-BiLSTM-DNN network structure surpasses that of competing algorithms, achieving an impressive 93.18% accuracy  
Figure 4: Comparison of recognition accuracies of different models in the RadioML2016.10b dataset.  
at an SNR of 4 dB. The results captured in Tables 1 and 2 showcase the peak recognition accuracy and average performance of our experiment.  
Table 3 compares the modulation recognition results of different models at \(-10\) dB, \(-8\) dB, \(-6\) dB, \(-4\) dB, \(-2\) dB, \(0\) dB, \(2\) dB, and \(4\) dB. The table's results indicate that the proposed CNN-BiLSTM-DNN and other models except MCLDNN have maximum increases of 8.16%, 8.15%, 6.87%, 8.19%, 8.94%, 6.75%, 5.43%, and 4.82%, respectively, under low signal-to-noise ratios.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **Maximum Accuracy (\%)** & **Average Accuracy (\%)** \\ \hline CNN-BiLSTM-DNN & **93.79** & **64.76** \\ MCLDNN & 93.67 & 64.09 \\ MCNet & 89.51 & 60.95 \\ CGDNet & 91.21 & 62.13 \\ ResNet & 90.99 & 60.78 \\ IC-AMCNet & 92.59 & 62.21 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental results of different models in the RadioML2016.10b dataset. (**bold**: best)  
Figure 5: Comparison of recognition accuracies of different models in the RadioML2016.10a dataset.