# HH-NIDS: Heterogeneous Hardware-Based Network Intrusion Detection Framework for IoT Security
## 5 Results
### Performance
#### 5.3.2 FPGA Performance  
The PL configuration time is 340 ms on average, including the time for loading NN parameters into BlockRAM. The line graph in Figure 10 shows the differences in inference times between CPU, GPU and FPGA. The horizontal and vertical axes present the number of records per test (input buffer size) and the processing time, respectively. By doubling the number of records sent each time, the CPU and GPU inference times increase gradually to approximately 0.92 ms, for both platforms, at the buffer size of \(2^{12}\) (4096 records). The CPU inference time rises exponentially to 3.01 ms, while the GPU starts to perform better,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**System** & **Model** & **Accuracy (\%)** & **F1-Score (\%)** \\ \hline
[32] & DNN & 99.70 & 87.00 \\ \hline
[49] & LSTM & 99.99 & N/A \\ \hline
[34] & RF & 100.00 & 100.00 \\ \hline
[37] & NN & 99.80 & 99.85 \\ \hline
[22] & RF & 99.50 & 99.00 \\ \hline HH-NIDS & NN & 99.66 & 99.81 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Result comparison to related works in the IoT-23 dataset.  
Figure 9: The pre-process, inference, and post-process time from different implementations.  
with only 1.44 ms, at the end of the test. HLS and Verilog implementation approaches on the PYNQ-Z2 SoC FPGA return a similar inference time, which starts at approximately 0.44 ms in the beginning and then increases slightly to 0.48 ms at the buffer size of 1024. This time is approximately 1.72 ms when the buffer size is 16,384 (\(2^{14}\)), which is equal to 9,525,581 records per second.  
Next, a timer is placed in the inference code after DMA has sent data from the PS to the PL to measure the data transfer time. The timer reported that it takes 0.067 ms, on average, for an input on the PYNQ-Z2 SoC FPGA to be inferred and transferred back to the PS. In addition, the data transfer time from the PS to the PL is close to 0.42 ms. Therefore, the NN block on PYNQ-Z2 SoC, with a Xilinx xc7z020-1 clg400c device, is 2.5 and 4.8 times (0.067 ms compared to 0.17 ms and 0.32 ms) faster than Intel(r) CoreTM i7-9750H 2.6 GHz CPU and NVIDIA GeForce GTX 1650 GPU in inference time, respectively. The processing speed of the pipelined design on FPGA can be calculated through the waveform simulation result in Figure 11.  
The waveform displays the primary input and output AXI4-STREAM interfaces in the NN block. The testbench sends a new input at each clock cycle to demonstrate a network input flow. Each record takes 28 clock cycles to be processed (between the s00_axis_tvalid and m00_axis_tvalid signals). In this pipeline design, the input flow can go through the system continuously without waiting for other processing inputs in the system to be finished; thus, the system overhead for processing one flow is 28 clock cycles, which is equal to 0.28 us on the PYNQ-Z2 (operating at 100 MHz). This means that the pipelined NN block on the FPGA accelerator at 100 MHz is 53.5 times faster than the max78000 EVKIT in simulation. Some factors affected the inference time, in practice, and will be discussed in Section 6.  
Finally, the average inference times after ten trials are shown in Table 11. To ensure a fair comparison with the work in [47], the number of records for sending in DMA buffer size was made equal to the number in these works (22,544 records). The authors in [47; 50] used the same NSL-KDD dataset for experimenting with their NN models. Our FPGA  
Figure 11: The waveform simulation results from the NN on FPGA using a Verilog approach.  
Figure 10: The CPU, GPU, and FPGA inference times for different input buffer sizes.  
implementation is 4.15 times faster than the same platform in [47] and 110.1 times faster than the CPU implementation in [50].