# Collaborative Mobile Robotics for Semantic Mapping: A Survey
## 2 Single-Robot Semantic Mapping
### Data Acquisition and Processing
#### 2.1.1 Perception-Based Data Acquisition  
Sensor observations are used to represent the geometry of the environment and extract its semantics autonomously. The geometric representation is usually obtained using state-of-the-art SLAM methods, which consist of simultaneously estimating the state of a robot and building a model of the environment perceived by its sensors [10]. In contrast, semantic information is mainly extracted using object-detection and semantic-segmentation techniques. Regarding systems integrating object detection, [PERSON] et al. [4] proposed an approach to create a semantic occupancy map of a home environment. The sonar-based SLAM method proposed in [22] is used to create an occupancy grid map. Then, using odometry and a stereo camera, a method based on object detection and triangulation was used to add the object's topological area with its category to the occupancy map. In addition, [PERSON] et al. [23] proposed a system to create a multi-layer map of an indoor environment. The laser-based EKF-SLAM algorithm [24] was used to create the low-level layers, and then receptive field coocurrence histograms (RFCH) [25], an RGB camera-based object-recognition algorithm, was used to collect the physical feature categories and establish links to the high-level layer. A recent trend in object detection is semantic segmentation, a paradigm that assigns a class label to pixels in an input image [26]. The semantic-mapping approach proposed in [27] incorporates semantic segmentation using a video stream from a monocular camera for 3D reconstruction of indoor and outdoor environments. The monocular semi-dense SLAM algorithm [28], which is effective in indoor and outdoor environments, is used to represent the geometric structure of the environment. In parallel, a pre-trained convolutional neural network (CNN) is used for 2D semantic segmentation. In addition, [PERSON] et al. [29] proposed an approach that uses an RGB-D camera to map a 3D environment and reconstruct models of the detected objects on the fly. ORB-SLAM2 [30] is used to perform mapping and camera localization on each RGB-D image. In parallel, objects are detected on  
Figure 1: The single-robot semantic-mapping process. the RGB images using a CNN and the associated point cloud is segmented in 3D. In [27; 29], image-based deep-learning techniques were used for semantic information acquisition due to their great progress in object detection and segmentation based on 2D images. In addition to these systems that perform object recognition and classification from visual data, there are other works that use geometric data from laser sensors to extract high-level information. [PERSON] and [PERSON] [31] proposed a system that combines visual cues and laser range data using SVM classifiers to distinguish indoor areas (hallway, kitchen, meeting room, etc.). Geometric feature extraction was used to determine the shape and size properties of the environment, and visual feature extraction was used to obtain the appearance and objects present in the environment. Although exploring object-detection and segmentation techniques is not the main focus of this work, they are necessary to understand how semantic mapping works. For interested readers, there are recent reviews of deep-learning-based semantic segmentation that provide a comprehensive survey on the topic. These reviews cover almost all popular image datasets and semantic segmentation methods, and for all modalities, such as RGB [32; 33], RGB-D [34] and 3D data [35; 36]. Since perceptions are used in semantic mapping to extract both geometric and semantic information, the choice of appropriate sensors, therefore, depends mainly on the type of geometric representation to be obtained (a 2D/3D map, a point cloud, etc.) and the level of the semantic description of the environment. The most popular sensors for geometric mapping are 2D/3D lasers, and, recently, RGB-D cameras are increasingly used [12]. These cameras are relatively inexpensive and can provide an RGB image and a depth map simultaneously.