# Collaborative Mobile Robotics for Semantic Mapping: A Survey
## 3 Collaborative Semantic Mapping
### Multi-Robot Data Association and Semantic Maps Fusion
#### 3.3.1 Data Association
This algorithm is divided into two steps: the E-step and M-step. The E-step establishes the correspondences between the robot map and the received map by computing the minimum relative distances using geometric and semantic data, and the M-step uses these correspondences to update the occupancy and the semantic-class probabilities of the robot global map voxels. In this work, the main advantage is that the EM algorithm can assign a probabilistic data association and update it iteratively instead of assigning hard decisions. In [47], voxel-to-voxel correspondences were established to associate local 3D semantic maps; in [52], point-to-point correspondences were established to associate local 2D semantic maps. This work is different from other works because it proposes a method to establish the association of data coming from heterogeneous robots, namely, an aerial vehicle map and a ground vehicle map. This method is specifically designed for the agricultural context, where the generated maps are typically composed of similar and repetitive patterns. Specifically, the aerial vehicle can quickly provide a coarse reconstruction of a large area,  
Figure 8: An example of collaborative semantic mapping in mixed indoor–outdoor environment. Red trajectory is robot 1, and orange trajectory is robot 2. Top: local semantic maps generated by two robots. Bottom: collaboratively generated global semantic map [47]. which can be updated with portions of more detailed, higher resolution maps generated by the ground vehicle visiting selected areas. First, a 2D grid map is created for each robot using its colored point cloud generated from the collected GPS/IMU-tagged RGB images. This map is composed of a multi-modal semantic representation of the environment, where each cell stores the vegetation index and height information of the local surface. Then, the idea to match these two maps is to find a dense flow of point-to-point correspondences between them using the large diplacement dense optical flow (LDOF) system [58]. The key intuition for using this method is that points belonging to one cloud locally share similar displacement vectors that associate them with points in the other cloud. The LDOF system is modified to fit the environment specificity by including the vegetation index and the geometric properties of the surface in the calculation of the cost function. **Topological-semantic maps matching:** In this method, the data-association problem is defined in a different way. Indeed, the type of local maps is a graph augmented with semantic information. The problem is, therefore, to match the nodes of these graphs using semantic and geometric data. This issue is addressed in [53], where the authors propose a different technique for matching topological-semantic maps. They did not use the state-of-the-art sub-graph fusion techniques based on combining geometric and topological information [59, 60], but instead proposed a new technique based solely on semantic and topological information. It does not require the global or relative pose of the robots to merge the maps. Each robot builds its partial oriented semantic topological map (OSTM) using the exploration algorithm [54]. This map is a semantically structured topological graph of the environment, where vertices represent locations in a building (room, hallway) and edges connect locations with a direction to traverse them. Then, to merge the sub-graphs, a bio-inspired algorithm [61], called inverse Warrington's object recognition model (IWORM), is used. It makes the correspondences between the two subgraphs using two comparison layers. The first layer, called semantic categorization, compares the two sub-graphs in terms of node orientation and semantic label. The second layer, called perceptual categorization, compares the two sub-graphs in terms of local structure, such as the number of neighbors of a vertex and the connections between those neighbors, in order to address labeling and orientation errors that may occur during the single-robot semantic-mapping process.