# Towards an embedded architecture based back-end processing for AGV SLAM applications

Mohammed Chghaf, Sergio Rodríguez Flórez, Abdelhafid El Ouardi

► To cite this version:

Mohammed Chghaf, Sergio Rodríguez Flórez, Abdelhafid El Ouardi. Towards an embedded architecture based back-end processing for AGV SLAM applications. Microprocessors and Microsystems: Embedded Hardware Design, 2025, 118, pp.105206. 10.1016/j.micpro.2025.105206. hal-05301654

HAL Id: hal-05301654

https://hal.science/hal-05301654v1

Submitted on 8 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

# Towards an embedded architecture based back-end processing for AGV SLAM applications

Mohammed Chghaf* , Sergio Rodríguez Flórez , Abdelhafid El Ouardi

SATIE Laboratory, Université Paris-Saclay, ENS Paris-Saclay, CNRS, Av. des Sciences Bâtiment 660, Gif-sur-Yvette, 91190, France

## ARTICLE INFO

**Keywords:**

Embedded systems

Hardware architecture

Image processing

Robotics

SLAM

## ABSTRACT

Place recognition plays a crucial role in the Simultaneous Localization and Mapping (SLAM) process of self-driving cars. Over time, motion estimation is prone to accumulating errors, leading to drift. The ability to accurately recognize previously visited areas through the place recognition system allows for the correction of these drift errors in real-time. Recognizing places based on the structural aspects of the environment tends to be more resilient against variations in lighting, which can cause incorrect identifications when using feature-based descriptors. Nevertheless, research has predominantly focused on using depth sensors for this purpose. Inspired by a LiDAR-based approach, we introduce an inter-modal geometric descriptor that leverages the structural information obtained through a stereo camera.

Using this descriptor, we can achieve real-time place recognition by focusing on the structural appearance of the scene derived from a 3D vision system. Our experiments on the KITTI dataset and our self-collected dataset show that the proposed approach is comparable to state-of-the-art methods, all while being low-cost. We studied the algorithm's complexity to propose an optimized parallelization on GPU and FPGA architectures. Performance evaluation on different hardware (Jetson AGX Xavier and Arria 10 SoC) shows that the real-time requirements of an embedded system are met. Compared to a CPU implementation, processing times showed a speed-up between 4x and 10x, depending on the architecture.

## 1. Introduction

The development and implementation of real-time and embedded front-end blocks of the SLAM system has seen significant advancements in recent years [1–3]. However, designing cost-effective and energy-efficient architectures for back-end functionalities, such as loop closure, remains a challenging feat. Traditionally, loop closure detection using LiDAR-based place recognition descriptors can be more accurate [4,5]. While these systems have proven to be highly effective, their high cost, substantial computational needs, and high power consumption make them less suitable for low-cost, embedded SLAM systems that require energy efficiency.

One way to address this challenge is developing methodologies that can extend expensive LiDAR-based systems to lower-cost alternatives, such as stereo vision systems. These systems, through the use of low-cost stereo cameras, offer the capability to create 3D environmental maps from 2D images, thereby providing an inexpensive and energy-efficient alternative to LiDAR-based systems [6].

However, these extensions require more time in the processing flow. The design of an accelerated architecture for this functional block is mandatory to achieve on-the-fly processing on a system-on-chip.

Our challenge is to boost the algorithm's performance on low-power architectures to ensure on-the-fly processing. FPGAs are considered the best choice for stream processing. Contrary to GPUs, which only provide parallelism for data processing and acceleration, FPGAs can provide data, task, and pipeline parallelism, which makes them more suitable for stream processing [7], especially for embedded systems.

Following these considerations, this paper proposes an extension the LiDAR-based place recognition descriptor Scan Context [8] to camera-based systems. Then we propose an algorithm-architecture mapping using CPU–GPU and CPU-FPGA architectures. The performance of these architectures are then evaluated and compared.

In particular, we make the following contributions:

1. We propose an adaptation of the LiDAR Scan Context to stereo camera point cloud context, making it an inter-modal global scene descriptor.

2. We present a parametric optimization protocol of sensor-algorithm coupling to achieve better performance in terms of loop detection accuracy.

<footnote>* Correspondence to: SATIE, Av. des Sciences Bâtiment 660, Gif-sur-Yvette, 91190, France.
E-mail address: mohammed.chghaf@gmail.com (M. Chghaf).</footnote>
3. We propose an algorithm-architecture mapping using CPU-GPU and CPU-FPGA architectures

4. We present the implementation and acceleration of the proposed global inter-modal descriptor on GPU and on FPGA.

The remaining of the article is organized as follows: Section 2 introduces related works on loop closure detection. Section 3 presents the proposed methodology to design the inter-modal global descriptor. Section 4 describes the global algorithm-architecture mapping followed. Section 5 presents the experimental results. Section 6 concludes this article and highlights future orientations.

## 2. Related works

Several works have recently focused on implementing the place recognition module of SLAM on embedded architectures. Yu et al. [9] proposed a Decentralized SLAM (DSLAM) where robots can share locations and environmental information. Specifically, they presented a Decentralized Place Recognition (DPR) by building a CNN-based real-time DSLAM system on embedded FPGA platforms. The authors used NetVLAD [10] for place recognition. Although the employed FPGA (Xilinx MPSoC) has significantly improved the energy efficiency of NetVLAD's CNN inference, its processing time over KITTI sequences was around 422 ms. Allowing the processing of only one frame every 12 frames in order to keep the processing in real-time.

Another approach was proposed by Wu et al. [11] based on 3D scene reconstruction leverages fast point feature histograms (FPFH) and the iterative closest point (ICP) method, enhanced by a revised weight calculation formula and the use of best-bin-first (BBF) for data dimension reduction. This technique facilitates rapid and efficient generation of complex color point cloud models, showcasing significant improvements in processing speed and accuracy for massive point cloud data.

In order to achieve real-time performance, the work presented in [12], proposed an architecture that uses semantically meaningful local features and their 3D geometrical verification for efficient place recognition. The proposed end-to-end architecture is fueled by a deep neural network, a bag-of-words database, and 3D geometrical verification. For the feasibility of using the system in a real-time application, it has been tested on an NVIDIA Jetson AGX Xavier. On average, the sum of the whole loop closure detection processing time (including inference, feature extraction and geometrical verification), is around 23.63 ms. Therefore, the system can be run on a real-time system with approximately 42 FPS. However, the network needs intensive computations. Consequently, the method is not suitable for low powered systems.

More recently, Elouaret et al. [13], proposed a low-footprint and high-performance accelerator for feature and image recognition in the context of autonomous vehicle navigation. First, the design workflow included an Image Processing (IP) that provides pixel information for each visual landmark detected in each captured image. Second, an FPGA board implementation of N-LOC, which is a bio-inspired neural architecture for navigation. Third, a distributed version of N-LOC in order to deal with larger neural networks. When compared to a pure software solution, the proposed approach demonstrates that the hardware-based IP implementation achieves up to ×9 lower latency and ×7 higher throughput (frames/second). All while being energy efficient. Indeed, this system has a power footprint as low as 2.74 W, which is up to ×5.5–6 less than what NVIDIA Jetson TX2 consumes on average.

Other studies focused on the front-end block of SLAM by proposing a CPU-FPGA architecture. The system designed by El Bouazzaoui et al. [14] using a high-level synthesis (HLS) approach based on the OpenCL paradigm incorporates both feature extraction and matching processing blocks. This heterogeneous approach enables on-the-fly processing of 1280 × 720 pixels resolution images.

Inspired by these works, we propose to adapt a LiDAR-based geometric descriptor, specifically the Scan Context [8], for camera-based place recognition. The primary motivation behind this adaptation is to leverage the robustness and geometric information inherent in LiDAR-based methods while utilizing the cost-effectiveness and ubiquity of cameras. To achieve this, we focus on the context of stereo cameras, which can be employed to compute depth maps similar to those obtained from RGB-D cameras such as the Intel RealSense D455. The stereo camera setup involves capturing images from two slightly offset cameras, which allows us to estimate the depth information by computing the disparity between corresponding points in the left and right images. Using the computed depth map, we can generate a point cloud representation of the scene akin to that produced by LiDAR sensors. With the point cloud representation in place, we proceed to calculate the Scan Context descriptor for the camera-based setup. Our ultimate goal is to ensure processing on-the-fly sensor data processing for AGVs applications at a high-definition resolution.

Although prior works such as [9–12] have achieved strong recognition performance, they present key limitations when applied to real-time, embedded SLAM systems. These methods often rely on computationally intensive inference pipelines, demand high-end GPU hardware, and lack the deterministic performance guarantees required for AGVs operating under real-time constraints. Additionally, the works presented in [13,14] focus mainly on the SLAM front-end, without providing a solution to the place recognition and loop closure challenge. In contrast, our proposed approach has been designed specifically for embedded deployment. We adapt the Scan Context descriptor, originally developed for LiDAR, to work with stereo camera data (which is a more cost-effective and widely available sensor). Additionally, we apply a sensor-algorithm co-design methodology to tailor the descriptor to the stereo modality's characteristics, such as its narrower field-of-view and denser point cloud. Our descriptor is fully geometric and handcrafted, making it inherently more amenable to parallelization and efficient hardware acceleration on both GPU and FPGA platforms. The result is a solution that preserves competitive accuracy while ensuring real-time processing and reduced power consumption.

## 3. Proposed approach

In what follows, we introduce the algorithm used to compute the disparity map from which the point cloud is computed. Then we present the parametric optimization carried out.

### 3.1. Semi-global matching algorithm

Stereo processing is a crucial technique in computer vision and robotics, as it enables the estimation of depth information from a pair of images captured by slightly offset cameras. Semi-Global Matching (SGM), introduced by Hirschmuller [15], has emerged as one of the most widely used and effective methods for disparity estimation in stereo processing. SGM relies on a pixel-wise matching cost computation followed by an aggregation process that enforces smoothness constraints while preserving sharp object boundaries in the resulting disparity map. The method is computationally efficient and provides high-quality depth estimation, making it suitable for various real-world applications, such as 3d point cloud generation from stereo image for place recognition in our context.

In Fig. 1, we illustrate the various steps involved in the disparity computation using SGM. First, the algorithm computes the Census Transform using the acquired input images. Second, these features are used for similarity evaluation, resulting in the generation of a local matching cost for each pixel and possible disparity (maximum disparity used in our case is 128). Third, to minimize errors, a smoothing cost is incorporated. Finally, the disparity map is then determined based on the smoothed costs. Usually a median filter is applied to remove outliers.
<figure><img src="image_4.png" /><figcaption>Fig. 1. The pipeline involves the census transform of input left and right images. The cost is computed for each pixel for each disparity level *d* (128 in our case). Cost aggregations computed following 8 paths for each pixel and for each disparity level. For each path, a smoothing aggregated cost is calculated. Finally, disparity is computed as output of the pipeline.</figcaption></figure>

## 3.2. Scan context algorithm

The Scan Context is a global descriptor used for LiDAR point-clouds. First, the 3D scan is divided into azimuthal and radial bins in the sensor coordinate and in an equally spaced fashion. Each scan acts as a global keypoint and thus is referred to a scan context as an egocentric place descriptor. Its detailed description is given in [8]. In Algorithm 1, we outline the entire workflow to compute the Scan Context based on the camera point cloud derived from the disparity map.

<caption>**Algorithm 1:** Pseudocode of the Scan Context Calculation</caption>

Input: Disp = Disparity map; ($f_x, f_y$): Focal length; $B$ = Baseline; ($c_x, c_y$): Optical center; NumRings: Number of rings; NumSectors: Number of Sectors; MaxDepth: Maximum depth

Output: SC = Scan Context Descriptor

/* Maximum depth for the camera */
1 MaxDepth = 20;

/* Loop over disparity map rows and columns */
2 for i ← 0 to Disp.rows do
3 for j ← 0 to Disp.colts do
4 Depth = $f_x * B/Disp(i, j)$;
5 if (Depth < MaxDepth) then
6 Height = ($i - c_y$) * Depth/$f_y$;
7 Lateral = ($j - c_x$) * Depth/$f_x$;
8 Azimuth = arctan2(Lateral, Depth);
9 /* Compute the ring index based on Azimuth and maximum depth */
10 Ring_index = MAX(MIN(NumRings, INT(ceil((Azimuth / MaxDepth) * NumRings))), 1);
11 /* Compute the sector index based on azimuth angle */
12 Sector_index = MAX(MIN(NumSectors, INT(ceil((Azimuth / 360.0) * NumSectors))), 1);
13 /* Attributing the maximum Height to each bin */
14 if (SC(Ring_index, Sector_index) < Height) then
15 | SC(Ring_index, Sector_index) = Height;
16 end
17 end
18 end

Due to the limited FoV and the limited range of Cameras, the fixed parameters for LiDAR are not adapted to the camera context. Indeed, the FoV and the range must be further discretized to obtain a reliable representation of the current scene. Fig. 2 shows the limitations of the Field-of-View of Stereo Camera generated point cloud and its maximum

<figure><img src="image_5.png" /><figcaption>Fig. 2. Illustration of the partition applied to 3D scan from Stereo Camera and its corresponding left image. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)</figcaption></figure>

range along with the corresponding captured image. The geometrical information of points detected on the left buildings are well represented in Sectors 1 to 3 (colored in yellow). On the other hand, the electrical pole and the fence on the right are represented by the bins in Sectors 55 and 56 (colored in pink).

By contrast, increasing the number of sectors and rings used in the process will significantly increase the computing time of the descriptor. Therefore, reducing the overall performance of the system. Hence, an algorithm architecture co-design approach is necessary to harness the capabilities of this descriptor without losing in timing performances.

## 3.3. Parametric optimization

Our approach to the parametric optimization of sensor-algorithm coupling involves assessing the algorithm's performance on a variety of sequences with different predetermined numbers of sectors and rings. To analyze the accuracy of each configuration, we employ the precision-recall curve, which demonstrates the trade-off between precision and recall across various thresholds. In our specific case, we adjust the acceptance threshold for revisited places within a range of 0 to 1. A substantial area under the curve signifies both high recall and high precision. High precision is associated with a low false positive rate,
<figure><img src="image_6.png" /><figcaption>**Fig. 3.** Precision-Recall curves comparison. LiDAR-20 × 60 is the original implementation of Scan Context using LiDAR data, 20 rings and 60 sectors. CamPCL is the implementation using stereo camera-based point cloud under different parametrization of rings and sectors.</figcaption></figure>

<caption>**Table 1**</caption>

Place recognition metrics comparison over KITTI dataset. LiDAR-20 × 60 is the original implementation of Scan Context using LiDAR data, 20 rings and 60 sectors. CamPCL is the implementation using stereo camera-based point cloud under different parametrization of rings and sectors. We highlight in green the best results using stereo camera data.

<table><thead><tr><th rowspan="2">Configuration</th><th colspan="4">00</th><th colspan="4">02</th><th colspan="4">05</th><th colspan="4">06</th><th colspan="4">07</th></tr><tr><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th></tr></thead><tbody><tr><td>CamPCL-20 × 60</td><td>0.70</td><td>0.64</td><td>0.53</td><td>0.80</td><td>0.73</td><td>0.52</td><td>0.69</td><td>0.71</td><td>0.52</td><td>0.69</td><td>0.63</td><td>0.60</td><td>0.56</td><td>0.57</td><td>0.68</td><td>0.63</td><td>0.56</td><td>0.57</td><td>0.68</td></tr><tr><td>CamPCL-20 × 260</td><td>0.85</td><td>0.79</td><td>0.68</td><td>0.86</td><td>0.78</td><td>0.56</td><td>0.69</td><td>0.80</td><td>0.79</td><td>0.98</td><td>0.93</td><td>0.89</td><td>0.58</td><td>0.59</td><td>0.64</td><td>0.58</td><td>0.58</td><td>0.59</td><td>0.64</td></tr><tr><td>CamPCL-60 × 60</td><td>0.83</td><td>0.76</td><td>0.63</td><td>0.88</td><td>0.80</td><td>0.73</td><td>0.62</td><td>0.71</td><td>0.69</td><td>0.80</td><td>0.74</td><td>0.62</td><td>0.47</td><td>0.51</td><td>0.56</td><td>0.47</td><td>0.51</td><td>0.56</td><td>0.56</td></tr><tr><td>CamPCL-60 × 260</td><td><strong>0.91</strong></td><td><strong>0.85</strong></td><td>0.75</td><td>0.90</td><td>0.83</td><td><strong>0.84</strong></td><td>0.78</td><td><strong>0.90</strong></td><td>0.90</td><td>0.90</td><td>0.85</td><td>0.80</td><td><strong>0.62</strong></td><td>0.60</td><td>0.66</td><td>0.62</td><td>0.60</td><td>0.66</td><td>0.66</td></tr><tr><td>CamPCL-100 × 60</td><td>0.85</td><td>0.78</td><td>0.68</td><td>0.88</td><td>0.81</td><td>0.65</td><td>0.61</td><td>0.69</td><td>0.71</td><td>0.83</td><td>0.78</td><td>0.75</td><td>0.42</td><td>0.46</td><td>0.62</td><td>0.42</td><td>0.42</td><td>0.46</td><td>0.62</td></tr><tr><td>CamPCL-100 × 260</td><td>0.90</td><td>0.84</td><td>0.75</td><td>0.90</td><td>0.82</td><td>0.82</td><td>0.79</td><td><strong>0.90</strong></td><td><strong>0.92</strong></td><td>0.90</td><td>0.86</td><td>0.80</td><td>0.53</td><td>0.51</td><td>0.64</td><td>0.53</td><td>0.53</td><td>0.51</td><td>0.64</td></tr><tr><td>CamPCL-140 × 60</td><td>0.86</td><td>0.79</td><td>0.68</td><td>0.89</td><td>0.81</td><td>0.73</td><td>0.76</td><td>0.86</td><td>0.86</td><td>0.85</td><td>0.78</td><td>0.73</td><td>0.61</td><td><strong>0.65</strong></td><td><strong>0.74</strong></td><td>0.61</td><td><strong>0.65</strong></td><td>0.74</td><td>0.61</td></tr><tr><td>CamPCL-140 × 260</td><td><strong>0.91</strong></td><td><strong>0.85</strong></td><td><strong>0.78</strong></td><td><strong>0.91</strong></td><td>0.83</td><td><strong>0.84</strong></td><td>0.80</td><td><strong>0.90</strong></td><td>0.91</td><td><strong>0.99</strong></td><td><strong>0.95</strong></td><td><strong>0.90</strong></td><td><strong>0.62</strong></td><td><strong>0.65</strong></td><td>0.70</td><td>0.62</td><td>0.62</td><td>0.70</td><td>0.62</td></tr><tr><td>CamPCL-180 × 60</td><td>0.86</td><td>0.79</td><td>0.69</td><td>0.89</td><td>0.81</td><td>0.73</td><td>0.65</td><td>0.74</td><td>0.74</td><td>0.89</td><td>0.82</td><td>0.66</td><td>0.46</td><td>0.52</td><td>0.62</td><td>0.46</td><td>0.52</td><td>0.62</td><td>0.52</td></tr><tr><td>CamPCL-180 × 260</td><td>0.90</td><td>0.83</td><td>0.78</td><td>0.84</td><td><strong>0.84</strong></td><td>0.78</td><td>0.71</td><td>0.78</td><td>0.80</td><td>0.97</td><td>0.92</td><td>0.80</td><td>0.40</td><td>0.44</td><td>0.62</td><td>0.40</td><td>0.44</td><td>0.44</td><td>0.62</td></tr><tr><td>LIDAR-20 × 60</td><td>0.92</td><td>0.88</td><td>0.87</td><td>0.83</td><td>0.88</td><td>0.77</td><td>0.82</td><td>0.80</td><td>0.80</td><td>0.99</td><td>0.99</td><td>0.99</td><td>0.83</td><td>0.87</td><td>0.87</td><td>0.83</td><td>0.87</td><td>0.87</td><td>0.87</td></tr></tbody></table>

while high recall is linked to a low false negative rate. Elevated scores for both metrics indicate that the classifier produces accurate results (high precision) and identifies the majority of positive outcomes (high recall) (see Table 1).

Fig. 3 displays the outcome of this analysis when applied to sequences from the KITTI dataset [16]. The term “CamPCL” denotes a point cloud obtained from a camera, with the first subsequent number representing the number of rings and the second indicating the number of sectors. Generally, LiDAR-based Scan Context continues to surpass the stereo camera-based descriptor in terms of performance. Nonetheless, by increasing the number of sectors and rings utilized to characterize the scene, we can achieve accuracy comparable to that of LiDAR-based Scan Context. Even though, the stereo camera-based

approach is constrained by its Field-of-View (90°) and range (20 m) (see Fig. 4).

### 3.4. Parameters validation

In order to validate the fixed parameters ($N_s = 260$ and $N_r = 140$), we evaluate the CamPCL SC using these parameters over the self-collected dataset. We carried out data acquisition experiments using our laboratory instrumented vehicle to collect an outdoor dataset and assess the performance of CamPCL SC in dynamic outdoor environments. The vehicle is equipped with a Realsense Intel D455 depth camera, a VLP-16 LiDAR, and an RTK-GNSS receiver to provide ground truth positioning data. The D455 camera features provides long sensing range of up to
<figure><img src="image_7.png" /><figcaption>Fig. 4. Illustration of the Scan Context extended to point cloud generated from stereo camera.</figcaption></figure>

Table 2

Place recognition metrics comparison over self-collected dataset. We compare the original Scan Context approach using LiDAR data with the extended CamPCL SC using camera data.

<table><thead><tr><th rowspan="2">Source</th><th colspan="3">01</th><th colspan="3">02</th><th colspan="3">03</th></tr><tr><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th></tr></thead><tbody><tr><td>LiDAR</td><td>0.69</td><td>0.80</td><td>0.83</td><td>0.80</td><td>0.87</td><td>0.71</td><td>0.51</td><td>0.60</td><td>0.54</td></tr><tr><td>Camera</td><td>0.92</td><td>0.82</td><td>0.80</td><td>0.85</td><td>0.79</td><td>0.81</td><td>0.91</td><td>0.79</td><td>0.81</td></tr></tbody></table>

20 m and includes two infrared imagers with a power consumption of around 2 W. The VLP-16 LiDAR offers a range of up to 100 m and operates at approximately 8 W of power consumption. Several frames were dominated by the sky or the shade and the sequences are recorded in a dynamic environment with cars and buses. Fig. 5 illustrates a selection of the trajectories collected during the process.

In Table 2 we compare the obtained results with those achieved using LiDAR data to generate the Scan Context. It can be observed clearly, that the CamPCL SC using camera data outperforms the Scan Context using LiDAR data. This is mainly due to the fact that the LiDAR used in the self-collected dataset is a VLP-16 (16 channels). As a result, the generated point cloud is sparse (compared to the VLP-64 used in KITTI dataset). On the other hand, the camera used in the self-collected dataset is the Realsense Intel D455 which can provide an accurate depth map with a resolution of 1280 × 720. Therefore, the generated point cloud using the D455 is denser and richer.

4. Algorithm-architecture mapping

Semi-Global Matching (SGM), has garnered significant attention in the research community due to its ability to provide accurate depth information and computational efficiency. As a result, there has been extensive investigation into the acceleration of SGM for various hardware architectures to enhance its performance and expand its range of applications. Among these architectures, GPUs and Intel FPGAs have emerged as popular platforms for implementing and accelerating SGM. The acceleration of SGM on Graphics Processing Units (GPUs) using the CUDA programming framework has been a prominent area of investigation. GPUs, characterized by their massive parallel processing capabilities, have emerged as a powerful platform for accelerating compute-intensive tasks, such as those encountered in the SGM algorithm. Numerous studies have focused on optimizing SGM using CUDA to harness the algorithm's inherent parallelism, leading to significant speed improvements and real-time performance for high-resolution stereo image pairs [17,18]. Also, Ruf et al. [19] presented ReS2tAC, a real-time stereo vision system based on the SGM algorithm, optimized for embedded ARM CPUs (using NEON SIMD) and CUDA-enabled GPUs, achieving up to 46 FPS and low error rates on public

benchmarks. The system is validated through deployment on a UAV, demonstrating its suitability for onboard real-time depth estimation.

In addition to GPU-based acceleration, the potential of Field Programmable Gate Arrays (FPGAs) has also been explored for implementing and accelerating SGM algorithms. FPGAs offer a high degree of flexibility in terms of hardware customization, enabling energy-efficient performance through hardware-software co-design. Recent works have leveraged the OpenCL programming model to develop FPGA-based SGM implementations, illustrating the feasibility and advantages of using FPGAs for stereo vision tasks [20,21]. These studies highlight the potential of FPGAs in providing energy-efficient and adaptable solutions for SGM-based disparity estimation, further broadening the applicability of SGM across various scenarios and hardware platforms. More recently, Kalomiros et al. [22] presented a hardware-optimized implementation of the SGM stereo algorithm for FPGA devices, specifically targeting the Intel Stratix V and Xilinx Zynq UltraScale+ platforms.

Following these considerations, we propose in the next subsections a hardware acceleration using GPU architecture and FPGA architecture of the Scan Context based on point cloud generated from stereo camera. We denote it for the rest of the paper as CamPCL Scan Context (SC).

4.1. GPU acceleration

The Scan Context creation based on point cloud generated from stereo camera can be efficiently parallelized on the GPU since all the calculations can be accomplished out independently.

Based on the generated point cloud from the disparity map computed using the algorithm presented in [17], we take advantage of the SIMT (Single Instruction Multiple Threads) programming model offered by CUDA [23] for NVIDIA GPUs. Computing the CamPCL SC can be carried out by launching many *kernels* executed in parallel by all the threads on the CUDA-Cores. The same *kernel* is executed in parallel by all the threads. With the CUDA architecture, threads are organized hierarchically. They are indeed grouped into *thread blocks*, themselves grouped into *grid of blocks*.

We propose a GPU implementation of the CamPCL SC. First, the disparity map is loaded to the GPU. Then, it is divided into a grid of blocks (of size (39, 47)). Each block is further partitioned into 256 threads (of size (32, 8)). These dimensions are computed on the base of the KITTI dataset images' resolution of 1241 × 376 pixels and enables the processing of all the disparity map by the parallel threads.

In order to ensure optimal memory access patterns and reduce latency, global memory is used for storing the input disparity map and the resulting CamPCL SC. Meanwhile, local memory serves as a buffer to store intermediate results and facilitate warp communication. To achieve data consistency and ensuring accurate calculations, threads within a warp are synchronized to ensure that data is available when determining the value corresponding to each bin in the CamPCL SC. All computations, are using 32-bit floating-point. This choice ensures numerical stability without significantly impacting performance.

Once all the threads have completed their tasks and the CamPCL SC has been computed, the results are written back to the global memory and copied back to CPU. This organization of threads, memory usage, and synchronization allows for an efficient and parallelized computation of the Scan Context from the given disparity map, harnessing the power of the GPU architecture for improved performance and real-time processing. In Fig. 6 we illustrate the parallelization scheme of the CamPCL SC using GPU.

4.2. FPGA acceleration

We present the implementation process of the CamPCL SC descriptor using FPGA technology with the aim of pushing processing as close to the sensor as possible. This method significantly improves real-time image processing performance by leveraging the parallel processing
<figure><img src="image_8.png" /><figcaption>Fig. 5. GNSS trajectories from our self-collected dataset recorded using the SATIE laboratory, around the cities of Saclay and Saint-Aubin.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 6. Global parallelization scheme of the CamPCL SC using GPU.</figcaption></figure>

capabilities of FPGA architectures. This category of hardware architectures must be considered since it possess a distinct advantage in the form of any-to-any I/O connectivity, which facilitates seamless connections with various devices, networks, and storage systems without necessitating a host CPU [24]. As a result, FPGAs excel in general-purpose processing due to their direct high-speed sensor interfacing capabilities and exceptional bandwidth, making them advantageous in hardware architecture compared to GPUs.

Fig. 7 presents the CPU-FPGA mapping of the CamPCL SC. Initially, the non-optimized version of our implementation executes the algorithm sequentially. In this approach, the CamPCL SC descriptor processes input data (disparity map) sequentially, which results in sub-optimal performance due to inherent inefficiencies in the processing flow. This version, though functional, lacks the ability to take full advantage of the FPGA’s parallel processing capabilities, thereby limiting the potential for real-time image processing.

To address these limitations, we introduce an optimized version of the implementation. This optimized version is designed to prefetch specific regions of the disparity map and store them in local memory, which enables efficient memory access and reduces latency. By offloading the computationally intensive task of accessing global memory to local memory, this implementation substantially accelerates the overall processing time.

Furthermore, we enhance the performance of the optimized version by duplicating the CamPCL SC kernel four times, as shown in Fig. 8. This modification allows us to compute the descriptor for four separate groups of sectors concurrently, increasing the throughput of our implementation. By exploiting the parallelism inherent in FPGA architectures, we can process a higher number of sectors simultaneously, thereby significantly reducing the time required to generate a complete CamPCL SC descriptor.

Another optimization we have incorporated is a factor of four unrolling of the loop. Loop unrolling is a technique that replicates the loop body multiple times to reduce the overhead associated with loop control structures, such as branching and iteration. In our implementation, we unroll the loop by a factor of four, which results in a more efficient processing flow and further contributes to the overall performance gains achieved by our optimized CamPCL SC descriptor implementation on FPGA hardware. Finally, we adopted a fixed-point representation with 16-bit width to reduce resource usage. This trade-off maintains acceptable accuracy while minimizing logic and DSP usage.

## 5. Experimental results

In this Section we evaluate the CamPCL SC implementation on a Jetson AGX Xavier NVIDIA GPU and on a DE5a-Net DDR4 Arria 10 FPGA. We use for comparison a PC workstation with an 8-cores i7 running at 2.3 GHz. The frames used for the evaluation are the KITTI dataset sequences recorder with a PointGrey Flea2 with a 10 Hz frame rate and a resolution of 1241 × 376.

### 5.1. Resource usage

The resource usage of the CamPCL SC implementation on a DE5a-Net DDR4 Arria 10 FPGA is shown in Table 3. The development Kit used operates at 50 MHz and has 1150K logic elements (LEs). In order to save resources, we followed the performance-improvement rules proposed by Intel® FPGA SDK for OpenCL™ Pro Edition Best Practices Guide. Indeed, unrolling the outer loops of a nested-loop structure increases resource utilization significantly. Therefore, we unrolled only the inner loops.
<figure><img src="image_10.png" /><figcaption>Fig. 7. CPU-FPGA mapping of the CamPCL SC descriptor. The kernel is duplicated four times to improve the design efficiency.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 8. Scan Context kernels duplication diagram. Disparity map is divided into four chunks and distributed over the four kernels. Each kernel uses loop unrolling to compute the ring index, the sector index and the corresponding height of each pixel; then, stores its corresponding results (sub-Scan Context).</figcaption></figure>

<table><caption>**Table 3**
FPGA resource utilization. ALUTs Adaptive Look-Up Tables, FFs Flip Flops, RAMs Random Access Memory blocks, DSPs Digital Signal Processing blocks.</caption><thead><tr><th>Kernel</th><th>ALUTs</th><th>FFs</th><th>RAMs</th><th>DSPs</th></tr></thead><tbody><tr><td>CamPCL SC Kernel 1</td><td>58 543</td><td>57 674</td><td>194</td><td>98</td></tr><tr><td>CamPCL SC Kernel 2</td><td>58 543</td><td>57 674</td><td>194</td><td>98</td></tr><tr><td>CamPCL SC Kernel 3</td><td>58 543</td><td>57 674</td><td>194</td><td>98</td></tr><tr><td>CamPCL SC Kernel 4</td><td>58 543</td><td>57 674</td><td>194</td><td>98</td></tr><tr><td>Kernel subtotal</td><td>234 172</td><td>230 696</td><td>776</td><td>392</td></tr><tr><td>Global interconnect</td><td>8253</td><td>12 498</td><td>61</td><td>0</td></tr><tr><td>Board interface</td><td>68 306</td><td>136 480</td><td>343</td><td>0</td></tr><tr><td>System description ROM</td><td>0</td><td>67</td><td>2</td><td>0</td></tr><tr><td>Total</td><td>310 731 (36%)</td><td>379 741 (22%)</td><td>1182 (44%)</td><td>392 (26%)</td></tr><tr><td>Available</td><td>854 400</td><td>1 708 800</td><td>2713</td><td>1518</td></tr></tbody></table>

## 5.2. Functional validation

In order to ensure that the accuracy of place recognition is maintained, we re-evaluate the AUC, the F1-maximum score and the Extended Precision over KITTI sequences. The performance of the CPU, GPU and FPGA implementation of the CamPCL SC descriptor is summarized in Table 4. The overall performance of the GPU and FPGA implementations are similar to the CPU implementation. Slight differences can be noticed and they are due to rounding and truncations used when performing arithmetic operations in the computation of the corresponding bin height. Another reason is the partitioning strategies employed in GPU and FPGA. On one hand, GPU uses fine-grained parallelism, enabling the processing of all pixels of input disparity map simultaneously. On the other hand, FPGA is designed for coarse-grained parallelism and is customized to maximize throughput for a specific task. Moreover, being limited by the available computational resources

in FPGA results in different data access patterns from those of GPU. Which ultimately leads to slightly different results.

## 5.3. Overall performance analysis

Table 5 summarizes the timing performances achieved over each sequence of the KITTI dataset. The presented times are the median processing time per iteration over all the frames of KITTI dataset. Since we propose to evaluate the proposed approach as a system, these times take into account data transfers as well as processing times. Overall, we successfully reduce the computation time by 9 to 10 orders of magnitude using the Jetson AGX. Compared to the Arria 10 SoC-FPGA, we reduce the computation time by 4 orders of magnitude. On the laptop GPU (GeForce RTX 3080), the computation time is reduced by 15 to 16 orders of magnitude (allowing a computation time of 1.35 ms on average). These performances show that we are able to
<caption>**Table 4**</caption>

Functional validation of CPU, GPU and FPGA implementation of CamPCL SC. Functional validation of CPU, GPU and FPGA implementation of CamPCL SC. For each implementation we compare the performance over the KITTI dataset using place recognition metrics (AUC, F1 and EP).

<table><caption>**Table 5**</caption><thead><tr><th rowspan="2">Sequences<br>Implementation</th><th colspan="3">00</th><th colspan="3">02</th><th colspan="3">05</th><th colspan="3">06</th><th colspan="3">07</th></tr><tr><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th><th>AUC</th><th>F1</th><th>EP</th></tr></thead><tbody><tr><td>CPU</td><td>0.91</td><td>0.85</td><td>0.78</td><td>0.91</td><td>0.83</td><td>0.84</td><td>0.80</td><td>0.90</td><td>0.91</td><td>0.99</td><td>0.95</td><td>0.90</td><td>0.62</td><td>0.65</td><td>0.70</td></tr><tr><td>GPU</td><td>0.90</td><td>0.85</td><td>0.77</td><td>0.90</td><td>0.83</td><td>0.83</td><td>0.80</td><td>0.90</td><td>0.91</td><td>0.99</td><td>0.96</td><td>0.96</td><td>0.61</td><td>0.61</td><td>0.70</td></tr><tr><td>FPGA</td><td>0.91</td><td>0.85</td><td>0.78</td><td>0.90</td><td>0.83</td><td>0.83</td><td>0.80</td><td>0.90</td><td>0.89</td><td>0.99</td><td>0.96</td><td>0.95</td><td>0.62</td><td>0.65</td><td>0.71</td></tr></tbody></table>

Comparison of the per-frame processing time of CamPCL SC. Processing times are evaluated on CPU (baseline), on a GPU for embedded applications (Jetson AGX Xavier), and on an optimized version of the algorithm implemented on an Arria 10 SoC-FPGA.

<table><thead><tr><th>Seq.</th><th>CPU baseline (in ms)</th><th>GPU (in ms)</th><th>Speedup</th><th>FPS</th><th>SoC-FPGA (in ms)</th><th>Speedup</th><th>FPS</th></tr></thead><tbody><tr><td>00</td><td>22.04</td><td>2.30</td><td><strong>9.57</strong></td><td><strong>434.78</strong></td><td>5.02</td><td><strong>4.39</strong></td><td><strong>199.20</strong></td></tr><tr><td>02</td><td>21.87</td><td>2.30</td><td><strong>9.50</strong></td><td><strong>434.78</strong></td><td>5.43</td><td><strong>4.03</strong></td><td><strong>184.16</strong></td></tr><tr><td>05</td><td>22.09</td><td>2.28</td><td><strong>9.67</strong></td><td><strong>438.60</strong></td><td>5.35</td><td><strong>4.13</strong></td><td><strong>186.91</strong></td></tr><tr><td>06</td><td>21.44</td><td>2.14</td><td><strong>10.00</strong></td><td><strong>467.29</strong></td><td>5.33</td><td><strong>4.02</strong></td><td><strong>187.62</strong></td></tr><tr><td>07</td><td>21.25</td><td>2.26</td><td><strong>9.42</strong></td><td><strong>442.48</strong></td><td>5.32</td><td><strong>3.99</strong></td><td><strong>187.97</strong></td></tr></tbody></table>

compute the CamPCL SC using an embedded architecture that can process data on the fly. In the work presented in [20], a disparity map is generated for an image resolution of 1920 × 1080 pixels and a disparity level of 128, at a frequency of 50 FPS. Considering KITTI dataset [16], where data is acquired at 10 FPS and the resolution of images is 1241 × 376 pixels, real-time performance is achieved by our proposed FPGA implementation running at 180 FPS.

It is worth noting that while the GPU implementation currently uses 32-bit floating-point precision (FP32), a reduced precision configuration (FP16) could potentially further improve its performance. However, our focus was on maintaining numerical consistency across platforms (CPU and GPU). Exploring FP16 on GPU remains an interesting future optimization direction, especially given that the FPGA implementation already operates successfully with 16-bit fixed-point arithmetic and no observed loss in accuracy.

An FP16 implementation can offer better performance, particularly in terms of memory bandwidth usage, as FP16 data occupies half the space of FP32 data. This reduces memory transfers and thus speeds up processing. While on some GPUs, FP16 data may be converted to FP32 before computation, modern GPUs (notably NVIDIA GPUs starting from Volta, Ampere, and Hopper architectures) include dedicated hardware units (Tensor Cores, CUDA cores) optimized to process more FP16 operations per cycle than FP32.

Although the GPU implementation shows outstanding performance, FPGAs offer notable advantages over CPUs and GPUs, particularly in the context of energy consumption per frame. They are inherently designed to optimize power efficiency due to their reconfigurable nature, which allows for the precise tailoring of hardware resources to cater to specific processing tasks. This customization enables FPGAs to execute computations with a higher degree of parallelism, reducing the number of clock cycles required to process each frame and subsequently lowering the overall energy consumption [25].

Such an FPGA implementation, enables the use of *Scan Context* as a place recognition descriptor for low-cost and energy-efficient stream-based FPGA implementation of SLAM algorithm. The work presented in [26], shows that such implementations result in higher energy efficiency than GPU implementations.

## 6. Conclusion

In this work we presented an extension of the LiDAR Scan Context to Camera Point Cloud Scan Context (CamPCL SC). First, we carried out a parametric optimization and study to determine the sensor-algorithm coupling and to find the best combination of parameters necessary to achieve better performance in terms of loop detection accuracy. Second, we proposed a GPU and an FPGA implementation that takes into account the parameters specified by the previous sensor-algorithm

optimization protocol. We have shown that the algorithm's complexity has been preserved to guarantee the same quality of results in software as in hardware. The design was implemented on a Jetson AGX Xavier and on a DE5a-Net DDR4 Arria 10 FPGA architecture. The FPGA implementation shows that the OpenCL design is ×4 faster than the C++ implementation running on a high-performance CPU. The achieved throughput is around 180 FPS at 1280 × 720 pixels resolution, which meets the requirements to perform on-the-fly processing.

In perspective, an in-depth study will focus on the application of the same approach by combining both Camera and LiDAR data. In particular, we will propose a SLAM system dedicated for autonomous ground vehicle using both LiDAR and Camera. Future work will also focus on the extension of this geometric descriptor to odometry rather than only place recognition. In order to meet real-time requirements of an embedded system, an algorithm-architecture mapping approach will be necessary.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Data availability

The data used in this work is available online and is open source.

## References

[1] D.D. Nguyen, A. El Ouardi, S. Rodriguez, S. Bouaziz, FPGA implementation of HOOFR bucketing extractor-based real-time embedded SLAM applications, J. Real-Time Image Process. 18 (3) (2021) 525–538.

[2] N.M. Belmessaoud, Y. Bentoutou, M.C. El-Mezouar, FPGA implementation of feature detection and matching using ORB, Microprocess. Microsyst. 94 (2022) 104666.

[3] M. Chghaf, S. Rodriguez, A.E. Ouardi, Camera, LiDAR and multi-modal SLAM systems for autonomous ground vehicles: A survey, J. Intell. Robot. Syst. 105 (1) (2022) 2.

[4] M. Chghaf, S.R. Flórez, A. El Ouardi, A multimodal loop closure fusion for autonomous vehicles SLAM, Robot. Auton. Syst. 165 (2023) 104446.

[5] M. Chghaf, S. Rodríguez Flórez, A. El Ouardi, Extended study of a multi-modal loop closure detection framework for SLAM applications, Electronics 14 (3) (2025) 421.

[6] M. Eisoldt, M. Flottmann, J. Gaal, S. Hinderink, J. Vana, M. Tassemeier, M. Rothmann, T. Wiemann, M. Porrmann, ReconfROS: An approach for accelerating ROS nodes on reconfigurable SoCs, Microprocess. Microsyst. 94 (2022) 104655.

[7] S. Wu, D. Hu, S. Ibrahim, H. Jin, J. Xiao, F. Chen, H. Liu, When FPGA-accelerator meets stream data processing in the edge, in: 2019 IEEE 39th International Conference on Distributed Computing Systems, ICDCS, IEEE, 2019, pp. 1818–1829.
[8] G. Kim, A. Kim, Scan context: Egocentric spatial descriptor for place recognition within 3D point cloud map, in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2018, pp. 4802–4809.

[9] J. Yu, F. Gao, J. Cao, C. Yu, Z. Zhang, Z. Huang, Y. Wang, H. Yang, CNN-based monocular decentralized SLAM on embedded FPGA, in: 2020 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW, IEEE, 2020, pp. 66–73.

[10] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, J. Sivic, NetVLAD: CNN architecture for weakly supervised place recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5297–5307.

[11] P. Wu, W. Li, M. Yan, 3D scene reconstruction based on improved ICP algorithm, Microprocess. Microsyst. 75 (2020) 103064.

[12] R. Islam, H. Habibullah, Place recognition with memorable and stable cues for loop closure of visual SLAM systems, Robotics 11 (6) (2022) 142.

[13] T. Elouaret, S. Colomer, F. De Melo, N. Cuperlier, O. Romain, L. Kessal, S. Zuckerman, Implementation of a bio-inspired neural architecture for autonomous vehicles on a multi-FPGA platform, Sensors 23 (10) (2023) 4631.

[14] I. El Bouazzaoui, S. Rodríguez Flórez, A. El Ouardi, FPGA architecture-based front-end processing for SLAM applications, J. Real-Time Image Process. 22 (2) (2025) 73.

[15] H. Hirschmuller, Accurate and efficient stereo processing by semi-global matching and mutual information, in: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 807–814.

[16] A. Geiger, P. Lenz, R. Urtasun, Are we ready for autonomous driving? The KITTI vision benchmark suite, in: Conference on Computer Vision and Pattern Recognition, 2012.

[17] D. Hernandez-Juarez, A. Chacón, A. Espinosa, D. Vázquez, J.C. Moure, A.M. López, Embedded real-time stereo estimation via semi-global matching on the GPU, Procedia Comput. Sci. 80 (2016) 143–153.

[18] P. Sawant, Y. Temburu, M. Datar, I. Ahmed, V. Shriniwas, S. Patkar, Single storage semi-global matching for real time depth processing, in: Computer Vision, Pattern Recognition, Image Processing, and Graphics, Springer, 2020, pp. 14–31.

[19] B. Ruf, J. Mohrs, M. Weinmann, S. Hinz, J. Beyerer, ReS2tAC—UAV-borne real-time SGM stereo optimized for embedded ARM and CUDA devices, Sensors 21 (11) (2021) 3938.

[20] M. Dehnavi, M. Eshghi, FPGA based real-time on-road stereo vision system, J. Syst. Archit. 81 (2017) 32–43.

[21] C. Banz, S. Hesselbarth, H. Flatt, H. Blume, P. Pirsch, Real-time stereo vision system using semi-global matching disparity estimation: Architecture and FPGA-implementation, in: 2010 IEEE International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation, pp. 93–101.

[22] J. Kalomiros, J. Vourvoulakis, S. Vologiannidis, A hardware accelerator for the semi-global matching stereo algorithm: An efficient implementation for the stratix V and Zynq UltraScale+ FPGA technology, ACM Trans. Reconfigurable Technol. Syst. 17 (1) (2024) 1–25.

[23] I. Buck, GPU computing with NVIDIA CUDA, in: ACM SIGGRAPH 2007 Courses, 2007, pp. 6–es.

[24] A. Vision, Kria K26 SOM: The ideal platform for vision AI at the edge, 2021.

[25] M. Qasaimeh, K. Denolf, J. Lo, K. Vissers, J. Zambreno, P.H. Jones, Comparing energy efficiency of CPU, GPU and FPGA implementations for vision kernels, in: 2019 IEEE International Conference on Embedded Software and Systems, ICESS, IEEE, 2019, pp. 1–8.

[26] H. Sun, Q. Deng, X. Liu, Y. Shu, Y. Ha, An energy-efficient stream-based FPGA implementation of feature extraction algorithm for LiDAR point clouds with effective local-search, IEEE Trans. Circuits Syst. I. Regul. Pap. (2022).

<figure><img src="image_12.png" /></figure>

Mohammed Chghaf received the M.S. degree in electrical engineering in 2015 (specializing in automation and industrial computing) from Mohammadia School of Engineers, Rabat, Morocco. He received the M.S. degree in embedded systems and data processing in 2019 from Paris-Saclay University, Orsay, France. He is currently pursuing the Ph.D. degree with SATIE laboratory, Paris-Saclay University. His research activities are focused on multimodal embedded SLAM systems for autonomous vehicles.

<figure><img src="image_13.png" /></figure>

Sergio Rodríguez Flórez received his M.S. and Ph.D. degrees from University of Technology of Compiègne, France, in 2007 and 2009, respectively. Since 2011, he is an Associate Professor of the Paris-Sud University. His research activities are focused on dynamic scene analysis through multimodal perception intended to enhance Intelligent Transportation Systems applications.

<figure><img src="image_14.png" /></figure>

Abdelhafid El Ouardi received the M.S. degree from Pierre and Marie Curie University in 2001 and the Ph.D degree in Electronics from Paris-Sud University in 2005. He worked at Henri Poincaré University, Nancy, as a researcher in 2005–2006. He is currently an Associate Professor at Paris-Sud University, France. In Embedded Systems team of SATIE lab, his research interests include Hardware–Software co-design, evaluation and instrumentation of embedded systems, design of smart architectures for image and signal processing, SLAM applications.