# Smart Attacks Learning Machine Advisor System for Protecting Smart Cities from Smart Threats
## 7 Comparative Analysis and Discussion  
The NSL-KDD dataset is used in Table 7 to compare the proposed system to existing state-of-the-art approaches. We compared our proposed system to other previously mentioned ML techniques [5; 13; 14; 15; 16; 17; 18; 19; 20; 22; 23]. Related to Table 7, our system is highly accurate and has a low percentage of false alarms. The model in [15] is a straightforward DNN with just three hidden layers. The parameters of this system need to be tuned in order to achieve better results. The model in [16] was also a DNN, and it achieved almost the same results as the model in [15]. The authors [17] used a semi-supervised approach when the model was trained exclusively on a tiny training set by self-evolution. It was trained on fewer data, which led to other approaches performing better than it. In [18], the model used multiple feature selectors to achieve the best result, using a gain ratio that selected nine features. Training the model with more features would lead to better results. The model in [19] earned the highest accuracy and F1-score; nevertheless, they used PCA as a feature selection approach to optimize the system's overall performance. Additionally, the study in [19] demonstrates that PCA may outperform techniques such as information gain, which is widely utilized for feature selection. The model in [20] is GRU-RNN, with just three hidden layers. The model parameters in [20] need to be tuned in order to achieve better results. In [21], the authors used the multi-level classification approach used by the proposed approach, and achieved a high level of precision. They applied KNN, ELM, and HELM in their models. The model in [6] used A-PCA-I-ELM, but it used all 41 features and thus achieved good results. The model in [22] used ILECA, where the dimensionality was reduced with ELM. Using more features may lead to better results. In [25], the authors used conventional LSTM and BiDLSTM. BiDLSTM achieved better results. The model achieved the best f-score and false alarm rate, but it used all of the features. In [24], the best detection rate was achieved by solving the class imbalance problem using oversampling techniques, and then DNN was used for classification. Figure 11 shows a comparison in terms of accuracy between SALMA and previous state-of-the-art techniques. Table 8 compares the **SALMA** and other state-of-the-art methods, using the KDD-CUP99 dataset [55]. Our proposed system was compared to different Machine Learning approaches [24; 25; 26; 27; 28; 29; 30]. Our approach achieved the best accuracy. In [28], the authors used Simplified Swarm Optimization (SSO) guided via Weighted Local Search (WLS). Other optimization functions may achieve better results. In [29], mutual information is used as a feature selector fed to the ML model. Other feature selectors may achieve better \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Accuracy (\%)** &
\begin{tabular}{c} **False Alarm** \\ **Rate (\%)** \\ \end{tabular} & **Precision (\%)** & **Recall (\%)** & **F1-Score (\%)** \\ \hline Simple DNN [15] & 75.75 & 3.21 & 92.50 & 59.95 & 74.13 \\ \hline Semi-Supervised Approach [17] & 77.26 & N.A & N.A & N.A & N.A \\ \hline Random Forest [18] & 81.95 & N.A & N.A & N.A & N.A \\ \hline Decision Tree [19] & 88.74 & 3.99 & 83.24 & **96.5** & **89.38** \\ \hline GRU-RNN [20] & 89 & N.A & 89 & 89.5 & 89.2 \\ \hline KNN, ELM, and HELM [21] & 84.29 & 6.3 & **94.18** & 77.18 & 84.83 \\ \hline DNN [16] & 75.9 & N.A & N.A & N.A & N.A \\ \hline Classifier-level DNN [24] & 85.56 & N.A & **97.09** & 76.94 & 85.85 \\ \hline LSTM [25] & 87.26 & 4.03 & 90.34 & 87.26 & 88.03 \\ \hline BiDLSTM [25] & 91.36 & **0.88** & 92.81 & 91.36 & **91.67** \\ \hline ILECA [22] & 92.35 & N.A & N.A & N.A & N.A \\ \hline A-PCA-I-ELM [6] & 81.22 & N.A & **96.1** & N.A & N.A \\ \hline
**Proposed System (SALMA)** & **95.04** & **2.48** & 92.05 & 79.59 & 85.37 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison between the proposed system and previous approaches to the NSL-KDD dataset.