# Detail Guided Multilateral Segmentation Network for Real-Time Semantic Segmentation
## 3 Detail Guided Multilateral Segmentation Network
### Network Structure
#### 3.2.3 Semantic Information Supplement Path  
The stacking of convolutions has limited connection between local information on the graph, which limits the effective receptive field of the network. In order to strengthen the connection between local information and global information and effectively expand the range of the receptive field, we propose a scheme using a graph convolution neural network to further supplement the semantic relationship (capture long-distance dependency) between regions of a relatively long distance on the graph and further promote the network's learning in the current context. Inspired by GAS [21], to improve the global reasoning ability, we add a graph convolution path to obtain supplementary semantic information. Figure 4 shows the detailed structure of SIS-path with graph convolution.  
Figure 3: Detail feature supplement module.  
We use the features of the output of the last layer of the backbone network of MobileNet v3 [22] in the context path to build the graph structure of the SIS-path. First, the feature map is mapped into the graph, and pixels with similar features are assigned to the same vertex to determine the vertices of the graph structure. The edges of the graph structure are used to measure the similarity between the vertices. Secondly, the convolution on the graph structure is used to update the vertex features based on the feature information of the edges. Finally, the updated features are interpolated into the feature map via reprojection to recover the pixel-to-vertex assignment. Thus, the connection between local and global features is constructed and the dependencies between regions are encoded. On this basis, we construct associations between different layers of the feature map to associate channels with different features and further improve the network's ability to perform global learning. First, we assign channels with similar features to the same vertex to construct another graph structure. The edges of the graph structure measure the similarity between vertices. The encoding process is the same as GAS. The formula in the semantic information supplement path is expressed as follows:  
\[\begin{cases}Y_{GCNS}=GCNs(graph1(X_{C})),\\ Y_{GCNC}=GCNs(graph2(X_{C}))\end{cases} \tag{6}\]  
\[\begin{cases}Y_{ST}=Graph2 Tensor(Y_{GCNS}),\\ Y_{CT}=Graph2 Tensor(Y_{GCNC})\end{cases} \tag{7}\]  
\[Y_{SIS}=X_{CP}\oplus(conv(concat(Y_{ST},Y_{CT}))) \tag{8}\]  
where \(X_{C}\in\mathbb{R}^{C\times H\times W}\) represents the output of the last layer of the CF-path, which is the input of the SIS-path, \(graph1(\cdot)\) the transformation of the input tensor into the spatial graph structure feature \(G_{\mathbb{S}}\in\mathbb{R}^{N\times D}\), \(graph2(\cdot)\) is the transformation of the input tensor into the channel graph structure feature \(G_{\mathbb{C}}\in\mathbb{R}^{N\times D}\), \(GCNs\) represents the graph convolutional layer, \(Graph2 tensor(\cdot)\) represents the transformation of graph structure features into tensor, \(conv\) represents 1 \(\times\) 1 convolution, \(\oplus\) represents the element-wise sum, and \(concat\) represents the channel-wise concatenation.  
Figure 4: Semantic information supplement module (SIS Module).