# Hybrid Deep Learning-Based Intrusion Detection System for RPL IoT Networks
## 5 Experiment Setup and Evaluation
### Evaluation Results
#### 5.3.3 Supervised Multi-Class Classification  
Since the DANN model's main purpose is to conduct multi-classification, the model is trained using the combined attack datasets to distinguish among four classes: normal, DIS, rank, and wormhole. Different depths of hidden layers (1, 2, 3, and 4 layers) were implemented to find the model's optimal one. Figure 12 shows the loss and AUC using a single DANN hidden layer model. It took almost the full training trials (1000 epochs) to achieve a higher performance. Table 14 presents the performance metrics with a single-layer multi-class classification. The model achieves an overall accuracy of 96%, with a higher precision, recall, and F1-score in detecting the normal class, and the lowest one with the wormhole attack class. Figure 11: DANN Model Loss and AUC for Wormhole Attack Binary Classification. Figure 10: DANN Model Loss and AUC for Rank Attack Binary Classification. Figure 9: DANN Model Loss and AUC for DIS Attack Binary Classification. Using two hidden layers, the model achieves the best loss and AUC performance at 500 training epochs, as shown in Figure 13. Table 15 shows that the overall class detection accuracy is 96%. Similarly, as with using the one-layer model, the highest precision, recall, and f1-score are in normal class detection, and the lower performance metrics are in detecting the wormhole attack. Figure 14 shows that the loss and AUC reach the optimal performance at 300 epochs only by using three hidden layers. The model also gives an overall 98% classification accuracy, as presented in Table 16. The higher precision, recall, and f1-score performance \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline Normal & 0.99 & 0.99 & 0.99 & \\ DIS Attack & 0.98 & 0.97 & 0.98 & \\ Rank Attack & 0.86 & 0.77 & 0.82 & \\ Wormhole Attack & 0.65 & 0.67 & 0.66 & \\ \hline Average & 0.87 & 0.85 & 0.86 & \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Performance Metrics with Single-Layer Multi-Class Classification. Figure 12: Loss and AUC With One DANN Hidden Layer Used in Multi-Class Classification. Figure 13: Loss and AUC With Two DANN Hidden Layers Used in Multi-Class Classification. percentage are achieved in classifying the normal class (99%). The overall average performance metrics are very good (92%) in classifying the three attacks in addition to the normal traffic. Adding more hidden layers does not always mean improving model performance. Figure 15 shows that, with four hidden layers, the model requires more than 600 training epochs (which is more than in the three-layers model) to reach the best loss and AUC performance. Table 17 also presents that the overall accuracy detection rate is 97% less than when using three layers. Moreover, it is noticed that the average precision, recall, and f1-score of the four classes are less than those in the three-layers model. By considering these results, it was decided to use three hidden layers in the DNN-IDS multi-class model. As mentioned earlier, the DL-IDS using the IoTR-DS dataset is also evaluated against other classical and deep learning models to see how the model performs compared to those models. Since it is difficult to evaluate the hybrid model, we have considered comparing the f1-score and accuracy performance of the supervised part (DANN) against the J48, KNN, SVM, and LSTM models, and the results are presented in Table 18. As shown, the DANN outperforms the other models in terms of both f1-score and accuracy metrics. The \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline Normal & 0.99 & 0.99 & 0.99 & \\ DIS Attack & 0.98 & 0.99 & 0.98 & \\ Rank Attack & 0.89 & 0.88 & 0.89 & \\ Wormhole Attack & 0.80 & 0.81 & 0.80 & \\ \hline Average & 0.92 & 0.92 & 0.92 & \\ \hline \hline \end{tabular}
\end{table} 
Table 16: Performance Metrics with Three-Layer Multi-Class Classification.