# Detail Guided Multilateral Segmentation Network for Real-Time Semantic Segmentation
###### Abstract  
With the development of unmanned vehicles and other technologies, the technical demand for scene semantic segmentation is more and more intense. Semantic segmentation requires not only rich high-level semantic information, but also rich detail information to ensure the accuracy of the segmentation task. Using a multipath structure to process underlying and semantic information can improve efficiency while ensuring segmentation accuracy. In order to improve the segmentation accuracy and efficiency of some small and thin objects, a detail guided multilateral segmentation network is proposed. Firstly, in order to improve the segmentation accuracy and model efficiency, a trilateral parallel network structure is designed, including the context fusion path (CF-path), the detail information guidance path (DIG-path), and the semantic information supplement path (SIS-path). Secondly, in order to effectively fuse semantic information and detail information, a feature fusion module based on an attention mechanism is designed. Finally, experimental results on CamVid and Cityscapes datasets show that the proposed algorithm can effectively balance segmentation accuracy and inference speed.  
semantic segmentation; feature fusion; trilateral parallel; graph convolution +
Footnote †: journal: Medical Image Computing  
202  
especially for small, segmented objects, the loss of details will be more serious. To solve this problem, inspired by BiSeNet v2 and Gated-SCNN [9], this paper proposes a detail guided multilateral segmentation network (DGMNet) for real-time semantic segmentation. The network is composed of three paths to maximize the use of spatial detail information, high-level semantic information and context information. Figure 1 shows the superiority of the proposed algorithm. In general, the contributions of this paper are as follows:  
* A context fusion strategy guided by spatial details is proposed, which effectively reconstructs the spatial information lost in the global environment, so that the detailed information can still be retained in the deeper features, and improves the segmentation performance of small and thin objects.
* A semantic information supplement path is designed to further supplement the semantic information of the lightweight network, increase the connection between local information and global information and further improve the accuracy of image segmentation.
* A three-path feature fusion module based on full attention is designed, so that the features with different information output from the three paths can be effectively fused, and the features are refined to guide the network to extract more valuable features, to improve the image segmentation ability.
* DGMNet achieved good segmentation results on Cityscapes [10] and CamVid [11] datasets. Specifically, for the Cityscapes dataset with a resolution of 512 \(\times\) 1024 and the CamVid dataset with a resolution of 960 \(\times\) 720 on the NVIDIA RTX 2080 Ti, the mIoU value can reach 76.9% and 74.8% at 141.5 FPS and 121.4 FPS, respectively.  
The rest of this paper is organized as follows. In Section 2, the related work of semantic segmentation is introduced. In Section 3, the specific structure of the proposed DGMNet is described in detail. In Section 4, the effectiveness of the DGMNet is verified via comparative experiments. In Section 5, the paper is summarized.  
Figure 1: Comparison of the reasoning speed (FPS) and mIoU (%) of different networks on the test dataset of Cityscapes.