# Semantic Visual SLAM Algorithm Based on Improved DeepLabV3+ Model and LK Optical Flow
## 2 Relevant Theories
### Octree Map Construction
The semantic segmentation results are used to construct the point cloud and octree map with semantic information, and the overall structural framework is shown in Figure 4.  
The RGB image is simultaneously passed into the tracking thread and semantic segmentation thread. In the tracking thread, the system extracts ORB feature points from the image and tracks them. In the semantic segmentation thread, firstly, the lightweight network MobileNetV2 [29] is used as the backbone feature extraction network for the semantic segmentation network DeepLabv3+ [30], and a lightweight semantic segmentation model is utilized to segment the mask of dynamic targets by segmenting all prior dynamic targets on a pixel-by-pixel basis. The network structure is shown in Figure 5. Next, the images after fusion of feature point extraction and semantic segmentation are fused to detect the ORB feature points covering the prior dynamic targets and remove them. Then, the dynamic target edge feature points and non-prior dynamic target points are further detected and rejected using the LK optical flow method. Finally, the static background occluded by the dynamic target is repaired using a time-weighted multi-frame fusion technique on the keyframes. Only the filtered relatively stable static feature points are used for feature matching and position calculation, thus ensuring the accuracy and robustness of this visual SLAM system in indoor dynamic environments. Figure 3: Sketch of an octree structure. The MobileNetV2 also adopts the depth separable convolution to reduce the number of model parameters while using the inverse residual structure to increase the dimensionality of the convolution to improve the ability of the model to extract features, and then it adopts the linear bottleneck structure to avoid the destruction of the low-dimensional spatial information by the nonlinear activation function to improve the performance of the network. As shown in Table 1, MobileNetV2, as the backbone feature extraction network, is compared with Xception, which is also trained using the widely used dataset PASCAL VOC2012. The input image size is 512 \(\times\) 512; the number of parameters is reduced by 89%, calculated from (54,723,837 \(-\) 5,828,429)/54,723,837 \(\times\) 100%; and the number of operations is reduced by 68%, calculated from (8.350 \(-\) 2.649)/8.350 \(\times\) 100%. In comparison, the accuracy decreases by no more than 5%, the time to process an image is 158 ms, and the inference speed is improved by 42.3%. The use of the lightweight network MobileNetV2 to replace the backbone network solves the problem of many network parameters and excessive dependence on hardware resources. It has a smaller network model and a faster processing speed to meet the real-time demand while ensuring accuracy. Figure 4: The overall framework of the semantic visual SLAM system. Figure 5: Lightweight semantic segmentation network model. To semantically segment the indoor environment, this study chooses the PASCAL VOC2012 dataset to pre-train the model. The dataset contains 20 different classes of objects as the training targets, possessing the dynamic targets of human, cat, and dog, and the static targets of cups, tables, chairs, and monitors. Considering the configuration of the experimental platform, the model training adopts the model-based migration learning method, loads the pre-training weights, and freezes the batch normalization layer of the backbone network MobileNetV2, with batch_size = 2 when freezing the training and batch_size = 4 when unfreezing the training, which greatly saves the model training time and effectively relieves the small amount of local data, small batch size, and insufficient computing power. The LK optical flow is utilized to further detect and reject non-prior dynamic target points, and the threshold d in Equation (7) takes the value of 5.