# CNN-BiLSTM-DNN-Based Modulation Recognition Algorithm at Low SNR
## 4 Experiment and Conclusion Analysis
### Analysis of Experimental Results
#### 4.2.2 Confusion Matrix  
Matrices are used to characterize the degree of confusion between different features [27]. In the confusion matrix, different shades of color indicate the effect of classification, and the darker the diagonal color, the better the classification effect.  
The confusion matrices for several network types with an SNR of 0 dB are displayed in Figures 6 and 7. Both AM-DSB and WBFM have a quiescent period, during which the signal is almost zero, and the network can only randomly classify invalid signals; as can be seen from the graph, the misclassification of the modulation format is mainly between AM-DSB and WBFM. The prediction accuracy of CNN-BiLSTM-DNN for these two types of modulation formats is significantly better than those of other models. CNN-BiLSTM-DNN improves the classification performance of most modulation formats compared with other models at low SNRs. Furthermore, 16 QAM is also easily confused with 64 QAM because their modulation types are very similar, but they use different orders, which makes the network unable to identify the two modulation methods accurately. After denoising the I/Q samples by the SP module, the influence of the modulation type on noise is reduced, and the CNN-BiLSTM-DNN model accurately identifies 97% of 16 QAM and 98% of 64 QAM under the RadioML2016.10b dataset. Under the RadioML2016.10a dataset, the CNN-BiLSTM-DNN model accurately identifies 90% of 16 QAM and 94% of 64 QAM, and the confusion between the two is significantly reduced, which exceeds the ability of other models to identify these two modulation types.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{8}{c}{**Average Accuracy (\%)**} \\ \cline{2-9}  & \(\mathbf{-10}\) dB & \(\mathbf{-8}\) dB & \(\mathbf{-6}\) dB & \(\mathbf{-4}\) dB & \(\mathbf{-2}\) dB & \(\mathbf{0}\) dB & \(\mathbf{2}\) dB & \(\mathbf{4}\) dB \\ \hline CNN-BiLSTM-DNN & \(\mathbf{27.73}\) & \(\mathbf{39.55}\) & \(56.18\) & \(\mathbf{69.23}\) & \(80.05\) & \(\mathbf{88.68}\) & \(\mathbf{91.32}\) & \(\mathbf{93.18}\) \\ MCLDNN & \(23.50\) & \(38.09\) & \(\mathbf{56.32}\) & \(67.27\) & \(\mathbf{81.05}\) & \(88.36\) & \(89.59\) & \(90.86\) \\ MCNet & \(22.45\) & \(38.00\) & \(56.00\) & \(65.27\) & \(72.45\) & \(78.23\) & \(79.09\) & \(82.27\) \\ GCDNet & \(18.64\) & \(33.36\) & \(51.95\) & \(64.95\) & \(74.95\) & \(79.27\) & \(81.59\) & \(84.00\) \\ ResNet & \(20.45\) & \(29.95\) & \(48.32\) & \(58.45\) & \(68.91\) & \(77.95\) & \(79.14\) & \(82.23\) \\ IC-AMCNet & \(19.59\) & \(33.18\) & \(53.00\) & \(63.59\) & \(73.00\) & \(80.23\) & \(81.55\) & \(84.77\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Classification and recognition results of different frameworks with low SNRs in the RadioML2016.10a dataset. (**bold**: best)  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{8}{c}{**Average Accuracy (\%)**} \\ \cline{2-9}  & \(\mathbf{-10}\) dB & \(\mathbf{-8}\) dB & \(\mathbf{-6}\) dB & \(\mathbf{-4}\) dB & \(\mathbf{-2}\) dB & \(\mathbf{0}\) dB & \(\mathbf{2}\) dB & \(\mathbf{4}\) dB \\ \hline CNN-BiLSTM-DNN & \(\mathbf{32.37}\) & \(\mathbf{45.58}\) & \(\mathbf{57.85}\) & \(\mathbf{72.65}\) & \(\mathbf{86.13}\) & \(\mathbf{91.48}\) & \(\mathbf{92.81}\) & \(\mathbf{93.30}\) \\ MCLDNN & \(30.14\) & \(40.02\) & \(55.30\) & \(72.10\) & \(85.47\) & \(91.14\) & \(92.52\) & \(93.26\) \\ MCNet & \(27.82\) & \(38.86\) & \(53.03\) & \(67.85\) & \(78.71\) & \(84.74\) & \(87.38\) & \(88.48\) \\ GCDNet & \(31.76\) & \(40.83\) & \(51.79\) & \(67.53\) & \(80.12\) & \(87.01\) & \(89.96\) & \(90.07\) \\ ResNet & \(25.36\) & \(37.43\) & \(50.98\) & \(64.46\) & \(77.19\) & \(84.93\) & \(88.47\) & \(90.33\) \\ IC-AMCNet & \(24.21\) & \(37.73\) & \(54.13\) & \(68.55\) & \(81.71\) & \(88.38\) & \(90.53\) & \(91.77\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: Classification and recognition results of different frameworks with low SNRs in the RadioML2016.10b dataset. (**bold**: best)