# Signals Intelligence System with Software-Defined Radio
## 3 Results  
After completing the training process, a number of 46 models resulted, each of which has a certain accuracy value. Table 3 lists the first 16 models that achieved the best accuracy among the 46 generated.  
For integration, the model with the highest accuracy is chosen. For this model, the results obtained from the validation of the model after its training are also generated.  
The time required to obtain this model is 9 h, 21 min, and 19 s. During this time, both the actual training and the validation of the results were carried out. All the metrics obtained from the validation of this model are:  
* Accuracy = 0.64831;
* Average macro precision = 0.68130;
* Average micro accuracy = 0.79536.  
The most important characteristic of the trained model is the confusion matrix. It can be seen in Table 4. As can be seen in the confusion matrix, the best results are recorded for the modulations: FM, BPSK, GMSK, OOK, and 4 ASK. The worst results are recorded for the modulations: 8 PSK, 16 APSK, 32 APSK, QPSK, and 8 ASK.  
The resulting accuracy differs for each modulation. Some modulations have a better accuracy than others. The accuracy for each modulation is listed in Table 5.  
\begin{table}
\begin{tabular}{l l l l l l l l l l} \hline \hline  & **16 PSK** & **32 PSK** & **4 ASK** & **BASK** & **8 PSK** & **BPSK** & **FM** & **GMSK** & **OOK** & **QPSK** \\ \hline
16 PSK & 634 & 599 & 0 & 0 & 611 & 1 & 1 & 10 & 0 & 448 \\ \hline
32 PSK & 618 & 639 & 0 & 0 & 579 & 0 & 2 & 12 & 1 & 453 \\ \hline
4 ASK & 0 & 0 & 1715 & 424 & 0 & 0 & 0 & 0 & 165 & 0 \\ \hline BASK & 0 & 0 & 664 & 1616 & 0 & 0 & 0 & 0 & 24 & 0 \\ \hline
8 PSK & 607 & 664 & 0 & 0 & 613 & 1 & 0 & 6 & 0 & 413 \\ \hline BPSK & 7 & 4 & 0 & 0 & 5 & 2281 & 0 & 2 & 0 & 5 \\ \hline FM & 0 & 1 & 0 & 0 & 0 & 0 & 2303 & 0 & 0 & 0 \\ \hline GMSK & 58 & 58 & 0 & 0 & 68 & 1 & 2084 & 0 & 35 \\ \hline OOK & 0 & 0 & 58 & 0 & 0 & 0 & 0 & 0 & 2246 & 0 \\ \hline QPSK & 440 & 399 & 0 & 0 & 442 & 1 & 1 & 215 & 0 & 806 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Confusion matrix.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Algorithm Name** & **Accuracy** & **Sampling** \\ \hline SparseNormalizer, GBoostClassifier & 0.64831 & 100.00\% \\ \hline MaxAbsScaler, LightGBM & 0.60833 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.60117 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.58980 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.58559 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.57396 & 100.00\% \\ \hline RobustScaler, LightGBM & 0.56163 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.55794 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.55551 & 100.00\% \\ \hline StandardScalerWrapper, XGBoostClassifier & 0.54870 & 100.00\% \\ \hline SparseNormalizer, LightGBM & 0.54132 & 100.00\% \\ \hline SparseNormalizer, GBoostClassifier & 0.53568 & 100.00\% \\ \hline MaxAbsScaler, GradientBoosting & 0.53229 & 100.00\% \\ \hline MaxAbsScaler, XGBoostClassifier & 0.52804 & 100.00\% \\ \hline RobustScaler, LightGBM & 0.52682 & 100.00\% \\ \hline RobustScaler, LightGBM & 0.52248 & 100.00\% \\ \hline \hline \end{tabular}
\end{table}
Table 3: Accuracy of models.  
\begin{table}
\begin{tabular}{l l l l l l l l l l l} \hline \hline
**Modulation16 PSK** & **32 PSK** & **4 ASK** & **8 ASK** & **8 PSK** & **BPSK** & **FM** & **GMSK** & **OOK** & **QPSK** \\ \hline Accuracy & 0.26\% & 0.27\% & 0.72\% & 0.68\% & 0.25\% & 0.96\% & 0.97\% & 0.88\% & 0.95\% & 0.34\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: Accuracy for each modulation.