# Automatic Understanding and Mapping of Regions in Cities Using Google Street View Images
## 2 Related Work
Among these works, the next level is represented by those that, in addition to the 2D information, use 3D sensors for gathering information on the surroundings in order to generate a metric 3D map with semantic annotations [19; 21; 24]. This approach applies different matching procedures between 2D and 3D data to assign semantic labels produced by an image labeling model, to the points of a 3D cloud or capture. In this case, the result is pixel-level classification or a semantic segmentation process where every point of a 3D cloud will have a semantic label related to the RGB image labeling. The main objective of semantic mapping procedures is the generation of a map that extensively represents the zone or area where the robot or vehicle will be navigating. In order to build maps, current works notably make use of [19], Kinect Fusion [21] and ORBSLAM2 [24]. The methodology mentioned above covers the more common working strategies for generating semantic maps for navigation tasks. It is worth mentioning that other research works apply a similar processing pipeline, but use distinct algorithms in each phase of the pipeline. In the semantic classification phase, we can find a greater diversity of techniques, although the most successful and popular are the CNN. The CNN architectures vary for each work. Some authors apply a pretrained classification model [17; 18], while others use a modified version of some well-known convolutional architectures [13; 16; 19; 20]. Specifically, we might mention YOLO [18], SSD [24], CNN [13; 16; 19; 20], and a merging of RNN-CNN [21]. The use of these CNN models allows the generation of a semantic descriptor for labeling pixels, as presented in [17; 20]. As another part of the pipeline, some works apply a data association phase [21; 24]. This procedure focuses on assigning a semantic category, obtained by a classification model, to a specific cluster of points that have been detected in a 3\(D\) point cloud. In this way, it is possible to have an identified object in an RGB image, associated with the location of set of points existing in a 3D cloud. The main goal of this approach is to generate a semantic map based on the objects present and identified in an RGB image. Using a similar processing pipeline, the work of [PERSON] et al. [13] focuses on outdoor scenarios. These authors applied segmentation algorithms and 2.5\(D\) grids maps to create semantic maps for autonomous navigation [13]. This work generates a map for semantically annotated elements in an image. The images consist of a view of an off-road path: the system assigns labels relevant to the navigation procedure, such as trail or grass. Images are captured with an all-terrain vehicle and 3D information is obtained from a LiDAR sensor, producing a map where each 3D point has a semantic label. Additionally, the work presented in [16] focuses on using micro-aerial vehicles for applying a semantic segmentation to an image capture taken from the vehicle. This approach follows a procedure but was designed for understanding images from an aerial viewpoint. This was designed for scouting purposes and to identify certain objects, such as cars, from the air. In the same way, the research presented in [20] develops a topologic-semantic map for use in autonomous navigation in cities. This approach mainly seeks to assign a category to every pixel on an image using semantic segmentation with CNN models. Images are captured from the vehicle itself during the navigation and are labeled with a semantic descriptor of 4 dimensions/categories. The objective is to provide the vehicle with the knowledge of what is in its field of view to ease the navigation through the location. This approach does not use metric information. As mentioned, map generation allows for multiple applications, such as the case of [22], which uses map generation to create maps for tourism use. To perform this task, the authors' system determines the salience of map elements, using bottom-up vision-based image analysis and top-down web-based information extraction techniques.