# Risk Assessment Method of Solar Smart Grid Network Security Based on TimesNet Model
## 4 Analysis of the Results of NSR Evaluation for Smart Grid
### Performance Analysis of SGID Model  
To verify the Bi-LSTM-based SGID model's performance, two publicly available network security datasets, NSL-KDD and CICIDS2017, are utilized to assess the model, which are segmented into training and testing sets in a 7:3 ratio. The NSL-KDD and CICIDS2017 datasets contain a large amount of detailed information on network traffic characteristics and attack types, which is of great significance for training and evaluating intrusion detection models, and can provide valuable references and inspirations for intrusion detection research in smart grids. The NSL-KDD dataset is a binary anomaly detection dataset that uses 41 features to describe each flow. In addition, each traffic has a label, namely normal and abnormal, indicating whether the corresponding traffic is normal or abnormal. The CICIDS2017 dataset collected data for 5 days, each consisting of 79 feature attributes and 1 category label, with normal traffic accounting for 80% and attack traffic accounting for 20%. The open source code for the NSL-KDD dataset can be found on the website: [[https://github.com/[PERSON]/NSL_KDD](https://github.com/[PERSON]/NSL_KDD)]([https://github.com/[PERSON]/NSL_KDD](https://github.com/[PERSON]/NSL_KDD)) (accessed on 11 December 2024). The open source code for the CICIDS2017 dataset can be found on the website: [[https://github.com/beyzann/CICIDS2017_Dataset_Exploration_A1](https://github.com/beyzann/CICIDS2017_Dataset_Exploration_A1)]([https://github.com/beyzann/CICIDS2017_Dataset_Exploration_A1](https://github.com/beyzann/CICIDS2017_Dataset_Exploration_A1)) (accessed on 11 December 2024). This study followed the research of [PERSON] et al. for parameter settings, which have been experimentally validated for their effectiveness and can therefore be used as a reference [26]. The iteration count was set to 200, epochs to 50, learning rate to 0.001, number of hidden layer neurons to 64, LSTM layers to 2, and Dropout to 0.2, and the Sigmoid activation function and Adam optimizer were used. The number of iterations is 200. Figure 9 shows the loss value variation curves of the proposed model in two datasets. In Figure 9a, in the NSL-KDD dataset, the model reaches convergence after about 50 iterations, and the final loss value remains stable at around 0.012. In Figure 9b, \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Index**} & \multicolumn{4}{c}{**Models**} \\ \cline{2-5}  & **GA-BPNN** & **ARN** & **CNN-LSTM** & **Improved TimesNet** \\ \hline DR/\% & 88.36 & 90.04 & 93.74 & 97.04 \\ FDR/\% & 2.01 & 1.82 & 1.73 & 1.21 \\ F1 value/\% & 86.53 & 88.13 & 90.24 & 92.69 \\ Accuracy/\% & 91.06 & 93.39 & 94.57 & 97.15 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of electricity theft detection performance among four models. Figure 8: Detection accuracy and ROC curve of four models. in the CICIDS2017 dataset, the curve rapidly decreases before 20 iterations of the model and then converges at approximately 50 iterations, with the final loss value stabilizing at around 0.02. This indicates that the SGID model based on Bi-LSTM has shown good convergence and fast convergence in different datasets. To verify the improvement effect of the proposed improvement strategy, the CICIDS2017 dataset is used, and the research model is compared with traditional DBN, LSTM, and Bi-LSTM using accuracy and False Negative Rate (FNR) as evaluation metrics, as listed in Figure 10. In Figure 10a, the detection accuracy of the research model is the highest (97.54%), followed by Bi-LSTM (91.43%), and DBN has the lowest detection accuracy (90.07%). In Figure 10b, the FNR of the research model is the lowest (1.21%), and the DBN is the highest (2.48%). The intrusion detection performance of the research model is superior to DBN and Bi-LSTM, indicating that this improvement strategy has certain effectiveness. To further validate the superiority, it is compared with CNN-CBAM, CNN-GRU, and PSO-SVM. Figure 11 compares the intrusion detection performance of four models.