# A Deep Learning Framework for Signal Detection and Modulation Classification
## 4 Results
### Performance on Signal Detection  
For multi-signal detection, we need to know each signal carrier frequency, start-stop time and modulation format. Figure 10 shows some detection results from our model. From Figure 10a,b, it is indicated that our model is beneficial for multi-signals detection, and it can accurately estimate the relevant information about each signal. Moreover, our model has very a promising application prospect in engineering because it has a good visualization effect. Figure 10: _Cont._To some extent, our model is not perfect yet, and there are still some aspects that need to be improved. From Figure 10c,d, we can learn that once the signal length is large, the estimation of the signal start-stop time is not precise, while the estimation of the carrier frequency is precise. The cause of this phenomenon may be that the time-frequency spectrum has large deformation and extreme length-with radio, while the natural image is not. Therefore, we need to further optimize the default box in the SSD networks. Figure 10e,f show the network performance when there is no signal exists. It can be observed that the model does not produce a false alarm, which is useful in engineering. Figure 11 shows our model detection precision versus different SNRs. We choose the mean Average Precision (mAP) as the performance index of the model. To calculate the mAP, we need to calculate precision and recall. For calculating precision and recall, we need to identify True Positive (TP), False Positive (FP), True Negative (TN), and False Negative (FN). Recall is defined as the proportion of all positive examples ranked above a given rank. Precision is the proportion of all examples above that rank which are from positive. The Average Precision (AP) summarizes the shape of the precision/recall curve. Hence, the mAP is the mean of all the AP values across all classes as measured above. They can be calculated as follows  
\[\text{Precision}= \frac{TP}{TP+FP},\text{Recall}=\frac{TP}{TP+FN} \tag{29}\] \[AP=\int_{0}^{1}\text{P}(r)dr\] (30) \[mAP=\frac{\sum\limits_{\text{num\_classes}}AP_{i}}{\text{num\_ classes}} \tag{31}\]  
It can be deduced that with the increase of the SNR, the mAP value of the SSD network is increasing. When the SNR is 5 dB, the mAP value can reach 90% in _IoU_ is 0.5. Different _IoU_ thresholds can lead to different results. Although the increase of the threshold can obtain more reliable signal carrier frequency and start-stop time, it sacrifices the precision of signal detection. Besides, we can adopt some traditional methods to further estimate these signal parameters. Finally, we choose 0.5 as the threshold of _IoU_. Figure 10: The SSD networks detection results. (**a**) ideal result 1; (**b**) ideal result 2; (**c**) not perfect result 1; (**d**) not perfect result 2; (**e**) no signal result 1; (**f**) no signal result 2. Once we detected the signals, we need to evaluate the precision of the estimated parameters. We use the normalized offset of the estimated and the actual parameters as the criterion of measurement. They can be presented as follows:  
\[\Delta f=\frac{\left|f_{pre}-f_{real}\right|}{R} \tag{32}\]  
\[\Delta t_{start}=\frac{\left|t_{pre\_start}-t_{true\_start}\right|}{T},\Delta t_ {stop}=\frac{\left|t_{pre\_stop}-t_{true\_stop}\right|}{T} \tag{33}\]  
where \(f_{pre}\) is the predicted value of the carrier frequency, \(f_{real}\) is the actual value of the carrier frequency, \(R\) is the symbol rate, \(t_{pre\_start}\) and \(t_{pre\_stop}\) are the predict values of the start and the stop time, \(t_{true\_start}\) and \(t_{true\_stop}\) are the actual values of the start and the stop time, and \(T\) is the signal duration. Table 2 shows the carrier frequency and the start-stop time precision when the signal is detected. It can be seen that the precision of the carrier frequency is higher than start and stop time. These phenomena are consistent with Figure 10c,d. And in future research, we need to combine the prior information of the signal to design the default boxes and the networks.