# Intrusion Detection in IoT Using Deep Residual Networks with Attention Mechanisms
## 6 Model Evaluation and Discussions
### Hyperparameter Settings  
Hyperparameters are parameters that are artificially adjusted before or during training. To ensure the best performance of the proposed deep learning model, optimal hyperparameters are determined through extensive experimentation. The hyperparameters involved in the experimental process include learning rate, epoch, and batch size. The details are shown in Table 3.  
* Learning rate (LR): A critical hyperparameter in deep learning that regulates the network model's learning progress. In this paper, we set three learning rate values to obtain the best performance. * Batch size: This parameter indicates the number of samples selected for one training. The batch size affects the memory usage, as well as the optimization and speed of the model. The batch size cannot be set too large or too small, so we set the range of batch size to 16, 32, 64, 128, 256, 512, 1024. * Epoch: The epoch represents the process of training all the training samples once. Too many epochs can lead to overfitting, while too few epochs may result in suboptimal training parameters. In this study, the number of epochs is set to 100. We divided the ToN_IoT dataset into a training set and a test set by the ratio of 80% and 20%, respectively. Initially, we set the learning rate to 0.01, epochs fixed to 100 times, and batch size range set to 16, 32, 64, 128, 256, 512, and 1024. We conducted experiments at different batch sizes and recorded the results for the above learning rates in Table 3. A detailed performance comparison of these parameters is shown in Figure 6a. The best \begin{table}
\begin{tabular}{c c} \hline
**Record Type** & **Number of Records** \\ \hline Analysis & 2000 \\ Backdoor & 1746 \\ DoS & 12,264 \\ Exploits & 33,393 \\ Fuzzers & 18,184 \\ Generic & 40,000 \\ Reconnaissance & 10,491 \\ Shellcode & 1133 \\ Worms & 130 \\ Normal & 56,000 \\ \hline \end{tabular}
\end{table} 
Table 2: UNSW-NB15 dataset record description. tested accuracy of the model is 96.82% for a learning rate of 0.01 and a batch size of 64. The lowest value of accuracy for this learning rate is 90.74% for a batch size of 1024. For other batch sizes, the accuracy and other performance scores are above 90%, demonstrating the model's robustness to changes in batch size. In the second stage, we fixed the learning rate at 0.001 and the epoch remained fixed at 100 times, while the batch sizes ranged from 16, 32, 64, 128, 256, 512, and 1024. By keeping the learning rate and epoch fixed, we conducted experiments at different batch sizes and recorded the results in Table 3. A detailed performance comparison of these parameters is shown in Figure 5(b). The best test accuracy is 99.55% when the batch size is 512. At batch sizes of 64, 128, 256, and 1024, the accuracy of the model was greater than 99%. At lower batch sizes of 16 and 32, the accuracy and other performance scores decreased but were still close to 99%. This indicates that a learning rate of 0.001 with a batch size of 512 provides the optimal balance between training stability and performance. Finally, we set the learning rate to 0.0001, and the other parameters were kept the same as those in the first two stages. The detailed performance comparison of these parameters is shown in Figure 5(c). The best test accuracy is 99.24% when the batch size is 32. For batch sizes of 16, 64, and 256, the accuracy of the model is above 98%. For other batch sizes, the accuracy and other performance scores decreased but the results were above 90%. The comparative experimental results of the above performance parameters show that the proposed model achieves the best results when the learning rate is 0.001, epochs are 100, and the batch size is 512. This combination of parameters consistently provides high accuracy and stable performance across different configurations, demonstrating the robustness and generalizability of the proposed model.