# HPC Platform for Railway Safety-Critical Functionalities Based on Artificial Intelligence
## 6 Test Results
### AI Model Inference Performance Results  
The results for the evaluation can be seen in Table 1, together with the evaluation results printed on the input image in Figure 4. In the figure, we can see several columns of execution times of the inference of one image using the Tiny-YOLOv4 model at different platforms.  
The VCU118 (SELENE) corresponds to the SELENE platform, using the HLSinf accelerator as the AI hardware accelerator of the platform, the EDDL library as an inference library and the SAF as the interface between the HLSinf accelerator (100 MHz) and the EDDL library. The Alveo corresponds to the inference time of the inference outside the  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Tiny YOLOv4** & **VCU118 (100** & **Alveo (250 MHz)** & **AGX Xavier** & **GGVU** & **AGX Xavier** \\  & **Layers** & **MHz) (SELENE)** & **SELENE)** & **(1.377 MHz)** & **GGVU (250 MHz)** \\ \hline \hline \multicolumn{4}{c}{All executed on CPU} & 45,685,922 & - & - & - \\ \hline FP32 & HLSinf & 1799 & 364 & & \\  & Transform & 479 & 7 & 17 & 94 \\  & Others & 4 & 0 & & \\ \hline Total & 2282 & 371 & & & \\ \hline FP16 & HLSinf & 1548 & 119 & & \\  & Transform & 726 & 6 & & \\  & Others & 4 & 2 & & \\ \hline Total & 2278 & 127 & & & \\ \hline INT8 & HLSinf & 796 & 66 & & \\  & Transform & 1135 & 15 & N/A & N/A \\  & Others & 4 & 1 & & \\ \hline Total & 1935 & 82 & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Tiny-YOLOv4 inference times on SELENE’s VCU118 board and the comparison with (a) the inference when SELENE’s HW accelerator is executed at Xilinx Alveo (also using EDDL) (b) the inference when the use case is executed on the GPU of AGX Xavier and (c) the inference at CPU frequency downscaled AGX Xavier (in order to be able to set a same level comparison). “Transform” and “Others” layers are executed in CPUs. All measurements are given in milliseconds (ms).  
Figure 4: Platform landmark (**a**) and Passenger (**b**) detectionsfor automatic accurate stopping and safe passenger use case. Note: (**b**) image corresponds to a platform shot. It is an example that emulates the images captured from the rear-view mirrors of the train, as the images captured for these tests are not publishable due to GDPR issues.  
SELENE platform, using an Intel i7-7800-X (3.45 GHz) for the non-supported HLSinf layers and the HLSinf accelerator (200-250 MHz) deployed on the Alveo U200 board for the supported HLSinf layers. The AGX Xavier GPU corresponds to inference on the GPU of the [PERSON] family AGX board. Finally, the last column corresponds to the GPU downscaled because the results needed to be adjusted for the frequency of the GPU (1377 MHz) to equal the frequency of the Xilinx Alveo (200-250 MHz) in order to compare the performance of the accelerator isolated from the underlying physical technology, which limits the operation frequency.  
The SELENE VCU118 platform results in Table 1 include different layer execution configuration and bit precision levels. Note that the time for one forward operation in the CPU is 4,568,592 ms, while in FP32 using the HLSinf accelerator the time lowers to 2282 ms. This result means an acceleration factor of \(\times\)2002 that rises to \(\times\)2361 when running on INT8 precision.  
Comparing the GPU downscaled and the Alveo EDDL columns, the GPU behaves better while using FP32 precision. On the FP16, the HLSinf accelerator achieves 119 ms inference time per image. Taking into account that not all the layers are embedded in the accelerator, it produces slightly more inference time than downscaled. When the precision falls back to INT8, the inference time for HLSinf is 66 ms per image, together with the CPU preprocessing time required, the time to execute a forward pass on one image is 82 ms. Unfortunately, the comparison at INT8 precision is not possible as the GPU available drivers do not handle fewer than 16 bits per parameter.