## 4 Real-World Use Cases with Sample Results
### Image Segmentation  
FPGA Platform
Field programmable gate arrays (FPGAs) are a common computational platform choice for space robotics because they offer several key advantages over traditional microprocessors, including high performance and relatively low power consumption. They have already been used in space missions and, therefore, radiation-resistant variants are available [52]. The possibilities for using FPGAs in artificial intelligence have broadened with the availability of tools that allow for the quantisation and compilation of deep learning models.  
\"Identifying and avoiding terrain hazards is crucial for the safety of planetary rovers. [...] The rover must avoid driving over any potential hazards to its wheels, particularly angular embedded rocks\", as described in [53]. Therefore, the key task is terrain analysis, which can be defined as pixel-wise image segmentation. One of the obstacles that occur in the lunar environment is rocks. To demonstrate that our synthetic data are helpful and that the simulator provides meaningful, measurable aid in application development, we cross-validate datasets. We use synthetic data from LunarSum and ALLR to train the neural networks and real ALLD lunar data to validate them. For this purpose, we use the UNet [54] segmentation network, which was created with a resnet10t backend [55] and trained with recent improvements, such as the cosine learning rate schedule and data augmentation. The UNet architecture is a well-known neural network framework, designed to assign class membership to individual pixels within an image. Specifically, we opted for the UNet implementation, which adopts an encoder-decoder structure and leverages skip connections to facilitate learning of spatial correlations across corresponding encoding stages, as detailed in [56].  
Using the synthetic data generated using the described simulator resulted in a better F1 score than in the case when the synthetic data was taken from the ALLD dataset. This indicates that the generated synthetic data more closely resembled real-life conditions. This was even more evident when both parts of the real life data set were used for testing, as the difference in the F1 score was even greater.  
In the same way, as in [57], we used the Xilinx VCK190 developer kit to perform deep learning inference. We quantised and compiled the models trained using our synthetic data. After quantisation, the models reached the F1 score values of 0.459 and 0.541 for ALLD (real)and ALLD (real)+ our (real). Comparison between data set combinations (Table 2) shows improvement in performance on real images with our synthetic data being used for training. Example predictions of the network in real images are presented in Figure 6a (from the ALLD dataset) and Figure 6b (from the analogue lunar research station). Next, we built an ROS 2 node running on VCK190 that performed inference on the input images (/lunarsim/left_camera/compressed topic) and published a blended image with a heat map overlay for the presence of rocks (Figure 6c).  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Precision** & **[EMAIL_ADDRESS]** & **Speed [img/s]** \\ \hline FP-32 & 0.424 & 40.27 \\ \hline FP-16 & 0.432 & 44.62 \\ \hline INT-8 & 0.411 & 48.96 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance and speed comparison with different weights representation precision. YOLOv7 was used as an example object detection model for our synthetic data. Jetson NX was chosen as the reference platform.  
Figure 6: A result of rocks segmentation performed by VCK190 developer kit: (**a**) image from the ALLD (real image part) dataset; (**b**) image from the analogue lunar research station; (**c**) image from our synthetic dataset.