# HH-NIDS: Heterogeneous Hardware-Based Network Intrusion Detection Framework for IoT Security
## 5 Results
### Accuracy  
The training, validation and inference accuracy will be discussed in this section. There are two training schemes corresponding to the targeted hardware: MAX78000 EVKIT and PYNQ-Z2. The training processes were run in each scenario with three different learning rates (LR): 0.0005, 0.001, and 0.01 on the UNSW-NB15 and IoT-23 datasets. Figure 7 depicts the training and validation accuracy in the first scheme, which was aimed at the MAX78000 EVKIT. In detail, the line chart in Figure 7a shows trained results on the UNSW-NB15 dataset. The highest LR (0.01) returned the best score at 97.96% accuracy (epoch 8). However, this number dropped significantly, to approximately 96.82%, in the rest of the training process, after the quantisation-aware training was activated. The training processes with lower LR reported better results, which drew increasing trends in validation accuracy, from 97.12% to the peak at 97.84% from epoch 9 to epoch 40 (with LR at 0.001). \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{5}{c}{**Hardware Resources Usage**} \\ \hline
**Approach** & **Resources** & **Utilisation** & **Available** & **Utilisation (\%)** \\ \hline \multirow{5}{*}{HLS} & LUT & 13,089 & 53,200 & 24.60 \\ \cline{2-5}  & LUTRAM & 242 & 17,400 & 1.39 \\ \cline{2-5}  & FF & 16,224 & 106,400 & 15.25 \\ \cline{2-5}  & BRAM & 42.5 & 140 & 30.36 \\ \cline{2-5}  & DSP & 152 & 220 & 69.09 \\ \hline \multirow{5}{*}{Verilog} & LUT & 22,329 & 53,200 & 41.97 \\ \cline{2-5}  & LUTRAM & 1420 & 17,400 & 8.16 \\ \cline{2-5}  & FF & 28,304 & 106,400 & 26.60 \\ \cline{2-5}  & BRAM & 20 & 140 & 14.29 \\ \cline{2-5}  & DSP & 220 & 220 & 100 \\ \hline \multicolumn{5}{c}{**Timing and Power**} \\ \hline  & **F\_max** & **Static** & **Dynamic** & **Total** \\ \hline HLS & 102.1 MHZ & 0.15 W & 1.68 W & 1.83 W \\ \hline Verilog & 101.2 MHZ & 0.14 W & 1.39 W & 1.53 W \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The IoT-23 dataset implemented results on PYNQ-Z2 SoC FPGA. Figure 7b illustrates the training and validation accuracy on the IoT-23 dataset. The accuracy can reach up to 99.98% before dropping dramatically to 89.38% (0.001 LR) and 74.94% (0.01 LR) due to the activation of quantisation-aware training. The 0.0005 and 0.001 LR training processes showed a slight improvement over time, with training accuracy around 92.85%. The lower LR values (0.0005 and 0.001) produced better models in the quantisation-aware training processes. The accuracy fluctuated if LR was 0.01 on both datasets. In the second scheme, the training processes were aimed at the PYNQ-Z2 board. Figure 8 represents two charts showing accuracy results on the two datasets. The line chart in Figure 7(a) shows trained results on the UNSW-NB15 dataset. The 0.01 LR reported 96.84% accuracy in training and validation at epoch one, before increasing to 97.71% in the subsequent seven epochs; this number fluctuated slightly in the rest of the training process. The 0.001 LR also reported the same results, with the best validation accuracy being 97.70% from epoch 40, while the smallest LR (0.0005) can only reach 97.15% accuracy at epoch 50. The second line chart, which are trained results on the IoT-23 dataset, shows similar