# HH-NIDS: Heterogeneous Hardware-Based Network Intrusion Detection Framework for IoT Security
## 5 Results
### Accuracy
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Hardware Resources Usage**} \\ \hline
**Approach** & **Resources** & **Utilisation** & **Available** & **Utilisation (\%)** \\ \hline \multirow{4}{*}{HLS} & LUT & 11,783 & 53,200 & 22.15 \\ \cline{2-5}  & LUTRAM & 242 & 17,400 & 1.39 \\ \cline{2-5}  & FF & 16,236 & 106,400 & 15.26 \\ \cline{2-5}  & BRAM & 44 & 140 & 31.43 \\ \cline{2-5}  & DSP & 152 & 220 & 69.09 \\ \hline \multirow{4}{*}{Verilog} & LUT & 28,004 & 53,200 & 52.64 \\ \cline{2-5}  & LUTRAM & 1412 & 17,400 & 8.11 \\ \cline{2-5}  & FF & 33,974 & 106,400 & 31.93 \\ \cline{2-5}  & BRAM & 20 & 140 & 14.29 \\ \cline{2-5}  & DSP & 219 & 220 & 99.55 \\ \hline \multicolumn{5}{c}{**Timing and Power**} \\ \hline  & **F_max** & **Static** & **Dynamic** & **Total** \\ \hline HLS & 102.5 MHZ & 0.15 W & 1.64 W & 1.79 W \\ \hline Verilog & 102.4 MHZ & 0.14 W & 1.42 W & 1.56 W \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The UNSW-NB15 dataset implemented results on PYNQ-Z2 SoC FPGA. Figure 7: The trained results in fifty epochs with three different learning rates: 0.0005, 0.001, and 0.01 for the MAX78000 microcontroller. training patterns to the first line chart. In detail, the higher LR (0.001 and 0.01) reported the highest validation accuracy at around 99.71%, while the 0.0005 LR training process achieved the best accuracy of 96.37% at epoch 47. The four best models, in terms of validation accuracy, were collected from the trained results above. Evaluation metrics were calculated, including accuracy, precision, recall and F1-score on test data. Table 8 shows inference results on the MAX78000 EVKIT on two models, which are denoted as (1) and (2). Model (1) on the UNSW-NB15 dataset achieves 98.57% accuracy and a 93.47% F1-score. The precision is 6.98% lower than the recall value (90.11% compared to 97.09%), which means that the model is more sensitive to attack records than normal records. In addition, model (2) on the IoT-23 dataset reports 92.69% accuracy and, 95.13% F1-score. Accordingly, Table 9 represents inference results on the PYNQ-Z2 board from two models, which are denoted as (3) and (4). Model (3) reproduces similar results to model (1) on the UNSW-NB15 dataset, with 98.43% accuracy and 92.71% F1-score. The highest accuracy and F1-score (99.66% and 99.81%) are achieved by the model (4) on the IoT-23 dataset. We have compared both our trained NN model on the IoT-23 dataset to related works in Table 10. The authors in [32] have achieved 99.70% accuracy by using DNN. They have improved this result to 99.99% by applying LSTM in [49]; however, F1-score is not mentioned in this research. Authors in [22; 34] have reported RF as the best algorithm for detecting attacks with 100% and 99.50% accuracy, respectively. Authors in [37] have achieved accuracy with only 0.14% greater than our NN model. However, their experiments have not applied to the full IoT-23 dataset [34; 37]. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline UNSW-NB15 (1) & 98.57 & 90.11 & 97.09 & 93.47 \\ \hline IoT-23 (2) & 92.69 & 95.04 & 95.22 & 95.13 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The inference results (in %) for the two trained models on the MAX78000 EVKIT. Figure 8: The trained results in fifty epochs with three different learning rates: 0.0005, 0.001, and 0.01 for the PYNQ-Z2 SoC FPGA. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline UNSW-NB15 (3) & 98.43 & 87.95 & 98.01 & 92.71 \\ \hline IoT-23 (4) & 99.66 & 99.97 & 99.65 & 99.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: The inference results (in %) for the two trained models on the PYNQ-Z2 board.