# Deep Learning-Based End-to-End Carrier Signal Detection in Broadband Power Spectrum
## 4 Experiments
### Performance Comparison to Other Methods  
To demonstrate the superiority of the SCN, the performance comparison is carried out with four other methods, including the double-thresholds method [7], the slope tracing method [9], the FCN-based models [21], and SigdetNet [22]. Both of these methods are evaluated on the same test datasets. The two deep learning-based methods (FCN-based models and SigdetNet) are trained with 20,000 simulation samples, as mentioned in Section 4.2.  
Table 2 shows the performance comparison between the SCN method and other methods. Compared with the double-thresholds method and slope tracing method, both of the deep learning-based methods are significantly superior. From our analysis, setting the appropriate thresholds for these threshold-based methods is difficult, which is the main reason for the vast performance gaps from the deep learning-based methods.  
Compared with SigdetNet, the deep learning method provides different results [22], while we found that FocalLoss performs better than DiceLoss in our experiments. However, the proposed SCN achieves better performance when the SCN scale is greater than 10.  
As an improvement of the FCN-based method, the SCN method improves the \(AP\) from 98.32% to 99.88%, \(AR\) from 98.13% to 99.12%, and \(F\)_-score_ from 98.22% to 99.48%, respectively. It is evident that the proposed method achieves higher \(AP\) scores than the FCN-based method at all different scales of networks. However, the higher \(AP\) scores scarify the \(AR\) scores when the scale of the network is small in the SCN method.  
Figure 9: The performance comparison of SCN-\(11\)x trained with different numbers of simulation samples. (**a**) \(F\)_-scores_ change with training epochs. (**b**) PR curve.  
Table 3 shows the complexity comparison between SCN and the other deep learning-based methods. Although the floating-point operations (FLOPs) and network parameters of the SCN models are dozens of times more than the SigdetNet and FCN-based methods, the inference time cost only increases by 22.0% and 52.2%. Due to the fact that the proposed SCN method is an end-to-end deep learning-based network without post-processing, while the FCN-based and SigdetNet methods only predict the classification probability of each point of the inputs, they cannot directly achieve the detection results. Therefore, without the post-processing time cost, the proposed method achieves higher performance using a considerably more complex CNN-based model.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & _AP_ & _AR_ & _F-Score_ \\ \hline Double-Thresholds & 77.64\% & 68.21\% & 72.62\% \\ \hline Slope Tracing & 89.18\% & 88.63\% & 88.90\% \\ \hline SigdetNet with DiceLoss & 95.64\% & 98.82\% & 97.20\% \\ \hline
**SigdetNet with FocalLoss** & **98.01\%** & **98.79\%** & 98.40\% \\ \hline FCN-Based 1 & 90.29\% & 89.47\% & 89.88\% \\ \hline FCN-Based 2  
Furthermore, the SCN method prevents the typical problem and false detection of the FCN-based method, which successfully improves the detection performance. For example, Figure A4 shows that the SCN method locates the precise subcarrier, whose bandwidth is very wide and the power distribution is relatively uneven, while it is a typical example of false detection in FCN-based method.