# Detecting Jamming in Smart Grid Communications via Deep Learning
## 6 Performance Evaluation
### Distance Between the Jammer and the Receiver  
For this experiment, we consider \(\mathcal{E}\) features the same transmission power as \(\mathcal{A}\), i.e., \(RJP=1\), and we vary the distance between \(\mathcal{E}\) and \(\mathcal{B}\) while setting the distance \(\mathcal{A}\)-\(\mathcal{B}\) to 30 m. We consider the two reference use cases introduced in Section 5.1.  
_Scenario 1._ We trained our model by limiting the number of epochs to 15. Figure 13(a) shows the accuracy of three reference **CNNs! (CNNs!)**, i.e., _resnet-18, resnet-50, and inception3_, as a function of the distance between \(\mathcal{E}\) and \(\mathcal{B}\). We observe that all three networks are characterized by the same behavior, and the accuracy decreases when the distance \(\mathcal{E}\)-\(\mathcal{B}\) reaches 76 m: CNN cannot detect the presence of a jammer (more than a random guess) when the distance \(\mathcal{E}\)-\(\mathcal{B}\) is greater than 90 m (assuming the distance \(\mathcal{A}\)-\(\mathcal{B}\) equals 30 m).  
Figure 12: BER as a function of the distance between the receiver and the jammer, with different RJP levels. The distance between the transmitter and the receiver (legitimate link) has been set to 30 m.  
Recalling Figure 12, we highlight the importance of this finding. In fact, the jammer can be identified when its position is far enough not to affect the quality of the link. In fact, BER \(\approx\) 0 when the distance \(\mathcal{B}\)-\(\mathcal{E}\) is between 50 and 76 m.  
We also report the average training time for each network in Figure 14b. We recall that the networks have been pre-trained on the ImageNet dataset and designed for the classification of 1000 labels in such a dataset. As previously discussed, we re-trained the network to expose them to images representing \(I-Q\) samples. Moreover, we adapted the output layers to the number of classes of our problem, i.e., _jammed_ and _non-jammed_.  
_Scenario 2._ We evaluated the effectiveness of our solution by selecting the optimal parameters, as discussed in Section 5.2. Specifically, we selected the logsig and satlin functions as encoder and decoder transfer functions, respectively, with a sparsity regularization of 10, a sparsity proportion of 0.5, and a weight regularization of \(L_{2}\)[66]. We also set the number of epochs to 100, the number of hidden units to 32, and used the K-fold cross-validation with \(K=5\).  
Figure 14c shows the performance of the classifier (autoencoders) when the jammer (\(\mathcal{E}\)) is moved away from the receiver. We found that the maximum distance where a jammer can be detected is about 68 m. It should be noted that this distance is much less than the one experienced for _Scenario 1_, i.e., 76 m. In fact, we recall that _Scenario 1_ assumes the a priori knowledge of jammed signals; i.e., the CNN models have been trained with jammed signals, thus making the model more sensitive to the jamming phenomenon and being able to detect the presence of the jammer further away from the jammer itself.  
We also consider the analysis associated with the **TP! (TP!)** and the TNR, as shown in Figure 14d. We observe that the decrease in accuracy at 70 m (Figure 14c) is justified by the drop in TPR in Figure 14d. In fact, as expected, the TNR is not affected by the distance since the actual challenge is to detect the presence of the jammer at large distances between \(\mathcal{E}\) and \(\mathcal{B}\) (TPR). We also evaluated the effectiveness of different combinations of hyperparameters, as discussed in Appendix A.