## 3 Robot Scene Understanding for Navigation Planning
These hierarchical mapping techniques were implemented in indoor environments where semantic information extraction is less complex, however, and have not yet been applied to unstructured outdoor environments. Kimera-multi [106] is a state-of-the-art metric semantic SLAM system for multi-robot systems. The method is an extension of Kimera [105] and was tested in computer-simulated environments based on SLAM datasets and outdoor datasets collected by ground robots. This concept creates a 3D mesh that labels outdoor environments using semantic labels to gain high-level scene understanding. Kimera-multi outperforms the accuracy of Kimera [105] visual-inertial odometry SLAM. Overall, Robot semantic understanding requires the classification of both objects and places. To extract these predefined object models, conditional random field models, CNN-based scene and place classification algorithms, and scene graphs (e.g., robot scene graphs [107], dynamic scene graphs [105]) are utilised. SLAM techniques can produce sparse or dense geometric or semantic maps of outdoor environments. However, these maps may not have adequate detail for safe local robot navigation where terrains consist of abrupt deviations and vegetation. Therefore, deep learning and image-based scene understanding are necessary directions to identify terrain properties and suitable environment regions for robot navigation in challenging outdoor unstructured environments. Recent robot navigation trends have emerged to segment camera egocentric images using computer vision approaches such as visual attention mechanisms [108]. Ref. [109] presents a camera-only approach for segmenting egocentric images to assist robot navigation. The segmentation procedure consists of soft labelling of the images according to three levels of driveability. For every pixel, soft labels are mapped by assessing original semantic classes. The SegNet [110] based deep convolutional encoder-decoder architecture was used for the pixel classification. Another paper [111] introduces an RGB-based method to classify terrains into navigability levels by extracting multiscale features. This work achieved state-of-the-art performance on the RUGD [112] and RELLIS-3D [113] datasets at the time of publication. A multi-head self-attention network architecture was used to classify the terrain into smooth, rough, bumpy, forbidden, obstacle, and background regions. This network requires further improvements to identify drastic terrain geometry changes (i.e., slopes) to gain higher accuracy in navigability segmentation. In general, 2D scene understanding can lose essential details from the mapping process of 2D data to the 3D real world. Therefore, 3D scene understanding-based methods are used with the most recent robot navigation applications. Ref. [114] presented a 3D semantic segmentation method using a CNN encoder-decoder architecture. This approach inputs RGB images and depth images for the fusion of feature maps, with the FuseNet [115] architecture taken as the base for the CNN. Visual dynamic object-aware SLAM [116] combines SLAM with dynamic modelling of scenes, enabling robots to operate effectively in dynamic environments. This work introduces instance segmentation of dynamic objects with the camera trajectory. The model can identify and track stationary structures and dynamic objects and integrate them into the traditional SLAM framework. This framework does not need prior knowledge of objects and their shapes or models. The method uses monocular depth estimation to achieve instance-level segmentation and optical flow. Then, static and dynamic object features are tracked to trace the trajectories. The local map is updated in each consecutive frame. Environmental scene understanding using only monocular cameras can be challenging in mapping applications. As a result, semantic segmentation using LiDAR, radar, and stereo cameras has been investigated. The development of deep learning techniques has increased the usage of point cloud data in robot 3D scene understanding applications.