# Object-Level Semantic Map Construction for Dynamic Scenes
## 3 Methodology
### Object Instance Segmentation and Data Association with Object Database  
In this step, the current frame RGB-D data can be accurately segmented into object instances using the algorithm provided in [38], combined with semantic and geometric cues, which provide the foundation for recognition of the dynamic part in the current frame, object model reconstruction, and fusion. Meanwhile, using \(T_{rou}\), the segmentation result is associated with the object database to identify the dynamic object part in the current frame or to create a new object. Figure 3 display the flowchart of the segmentation algorithm.  
In the segmentation stage, the deep neural network Mask-RCNN [6] is used to provide object masks with semantic labels in semantic cues. Although this algorithm is impressive and provides good object masks, it tends to provide imperfect object boundaries and cannot be executed at the frame rate. Therefore, geometric segmentation, which produces higher-quality instance boundaries rapidly but tends to result in over-segmentation by assuming that the object convexity is taken out, is performed. Each stage of results processing of geometric segmentation on current frame depth data is shown in Figure 4: (a) represents the depth data processed by bilateral filtering; (b) represents the original boundary result generated after geometric segmentation; (c) represents the boundary result after dilation and erode operations that connected boundary inner points and eliminated isolated noise points; and (d) represents the corresponding color display for each component surrounded by the segmentation boundary after statistical analysis. Through statistical overlay analysis shown in Figure 3, we fuse two segmentation results to obtain high-quality segmentation and semantic recognition results in real time, compensating for the shortcomings of both the semantic and geometric cues. The boundary of the semantic segmentation mask becomes the boundary of one or more geometric segmentation components where the overlap rate reaches 65%. The final segmentation result is represented as instance collection \(\{C_{i}\,,L_{i}|\ \mathrm{i}\to\ 0\dots N\}\), where \(C_{i}\) is the segmentation instance, \(L_{i}\) is the semantic category, and \(N\) is the instance number.  
Next, the fusion segmentation results are associated with the object database through \(T_{rou}\), as shown in Figure 5. The OpenGL pipeline is used to render the model ID image from the dynamic and static object databases updated by the current frame. The numbers of pixels in the overlapping part between instances \(C_{i}\) in the segmentation results and each ID \(I_{m}\) in the model ID rendering image are counted. If the pixel number is more than 65% \(\cdot|C_{i}|\) and the semantic category information is consistent between \(C_{i}\) and \(m\), then an association is created. If \(m\) is dynamic, then \(C_{i}\) is treated as the dynamic part \(C_{i}^{dynamic}\) in the  
Figure 3: Flowchart of object instance segmentation based on geometric and semantic cues.  
current frame. If \(C_{i}\) is not associated with any object model in the database, a new object model is created.