## 4 Experimental Results
### Orin Board  
In this section, we evaluate the performance of TensorRT optimization on a Jetson Orin board, focusing on its effect on real-time autonomous driving tasks on edge devices [37]. Without optimization, the inference times are significant, hindering the efficient execution of real-time tasks, particularly in off-road environments with complex terrain and obstacles. Table 4 lists the inference times of multi-class classification models. After applying TensorRT to the Jetson Orin board, we observed substantial improvements in performance, primarily because of switching from FP32 to FP16 or INT8 computations and efficient memory management. FP16, which utilizes tensor cores, was ideal for faster inference in complex models, whereas INT8 excelled in simpler models. FP32, which was more computationally demanding, resulted in slower inference times, making FP16 more suitable for real-time tasks on edge devices. ViT-based models (ViT-S/16 and ViT-B/16) could not be executed on the Jetson Orin because of their high complexity, even after conversion to ONNX. However, applying TRT to ViT-based models allows successful onboarding and rapid inference. Table 5 lists the inference times for the regression models. Similar to the multi-class classification results, these results show that INT8 performed exceptionally well in simpler models. However, it can be observed that FP32 sometimes exhibits better performance in ViT-based models. These findings highlight TensorRT's ability to enhance inference performance for real-time applications on edge devices. During testing, the average GPT inference time from inputting an image and a prompt pair to receiving the final output was approximately **7.7 s**. However, this time is highly sensitive to environmental factors, such as network speed and server load, which can cause delays. This variability presents a major challenge in autonomous driving, where real-time processing is essential. By contrast, our proposed method significantly reduces the inference time by eliminating the dependency on external servers and network conditions. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Inference Time (ms)** & **w/o TRT** & **w FP32** & **w FP16** & **w INT8** \\ \hline ResNet-18 & 176.8 & 8.6 & 5.1 & 2.3 \\ ResNet-50 & 350.1 & 14.3 & 8.2 & 5.1 \\ ResNet-101 & 350.1 & 20.1 & 16.4 & 8.7 \\ \hline ViT-S/16 & - & 19.4 & 13.8 & 18.0 \\ ViT-B/16 & - & 25.8 & 20.0 & 23.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Multi-class classification inference time on an Orin board. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Inference Time (ms)** & **w/o TRT** & **w FP32** & **w FP16** & **w INT8** \\ \hline ResNet-18 & 178.2 & 2.7 & 7.8 & 2.8 \\ ResNet-50 & 432.8 & 7.3 & 17.2 & 6.1 \\ ResNet-101 & 990.5 & 13.3 & 18.9 & 8.7 \\ \hline ViT-S/16 & - & 13.1 & 18.8 & 19.2 \\ ViT-B/16 & - & 19.0 & 27.1 & 20.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Regression inference time on an Orin board. The lightweight model operates entirely on edge devices, ensuring consistent and reliable real-time predictions of drivable areas regardless of environmental factors. However, inference time on edge devices is inherently non-deterministic due to variations in GPU workload and system conditions. To provide a more comprehensive analysis, Table 6 presents the average inference time, standard deviation, and percentile-based inference times, offering insights into the variability and distribution of processing delays. This robust low-latency performance makes the approach more suitable for real-time autonomous driving applications, especially in dynamic off-road environments, where rapid decision-making is crucial for vehicle safety and operational success. Additionally, in this study, GPT-4o serves both as a teacher and as the ground truth (GT). GPT-4o provides the traversability predictions for the given off-road scenes, which are used as the ground truth for the student model to learn from.