## 2 Robot Environment Perception for Navigation
### LiDAR and Camera Data Fusion
The network does not require dense ground truth maps, and training is done in an end-to-end manner that shows a broad generalisation capability in various real-world scenarios. The sparsity of LiDAR depth measurements can vary in real-world applications. One advantage of the network proposed in this work is that it handles a high range of sparsity up to the point where the LiDAR sensor has no depth measurements. A feedback loop has been incorporated into the network to connect outputs with inputs to compensate for noisy LiDAR depth measurements and misalignments between the LiDAR and stereo sensors. This network is currently regarded as one of the state-of-the-art methods for LiDAR-stereo camera fusion. * Multimodal Object Detection Reliable object detection is a vital part of the autonomous navigation of robots/vehicles. Object detection in autonomous navigation is described as identifying and locating various objects in an environment scene in the form of bounding boxes, including dynamic objects and static objects. Object detection may become difficult due to sensor accuracy, lighting conditions, occlusions, shadows, reflections, etc. One major challenge in object detection is occlusion, which consists of different types. The main occlusion types are self-occlusion, inter-object occlusion, and object-to-background occlusion [78]. Early image-based object detection algorithms commonly included two steps. The first stage was dividing the image into multiple smaller sections. Then, these sections were conveyed into an image classifier to identify whether the image section contained an object or not. If any object was detected in an image section, the respective portion of the original image was marked with the relevant object label. The sliding window approach is one way of achieving the above-mentioned first step [79]. A different set of algorithms uses a technique in contrast to the sliding window approach by grouping similar pixels of an image to form a region. These regions are then fed to a classifier to identify semantic classes (with grouping done using image segmentation methods). Further improved image segmentation can be achieved by using the selective search algorithm [80]. The selective search algorithm emphasises a hierarchical grouping-based segmentation algorithm. In this method, initially detected image regions are merged in a stepwise manner by selecting the most similar segments until the whole image represents a single region. These regions resulting from each step are added to the region proposals and fed to a classifier. The classifier performance depends on the used region proposal method. This object detection approach does not produce real-time performance suitable for autonomous navigation applications. However, advances such as SSPnet [81], Fast Regional-based Convolutional Neural Networks (R-CNNs) [82], and Faster R-CNN [83] were introduced to address this issue. The Faster R-CNN network generates a feature map by utilising the CNN layer output, and the region proposal generation is achieved by sliding a window (comprising three different aspect ratios and sizes) over the feature map. Each sliding window is mapped to a vector and fed to two parallel classification and regression networks. The classification network calculates the probability of region proposals containing objects, and the regression network indicates the coordinates of each of the proposals. Object detection research has been mainly employed in the autonomous vehicle industry (for vehicle and pedestrian detection [84]) and mobile robotics. In contrast to camera-only object detection, sensor fusion has been implemented in different real-world applications to obtain more accurate and robust detection results. As previously discussed, LiDAR and camera sensor fusion are some of the most used and highest-performing sensor fusion methods. LiDAR and camera sensor fusion object detection approaches consist of two main techniques. These are the sequential and one-step models [54].